Toolio platform for Private AI/Large Language Model Agents

Created a platform, Toolio, to run private Large Language Model (LLM) agents and tool-calling workflows on Apple Silicon, and also to constrain LLM output to a JSON schema, enhancing usability & inference speed.

Problem

Generative AI can be difficult to steer for controlled workflows, such as custom API Integration. It can also be hard to integrate such API usage (tool-calling) in a single, private platform, which is required e.g. for health-care and CRM needs.

My Contributions

Used my extensive AI/Machine Learning & Python programming experience to take a steering library for the output, and form it to API/tool requests based on the user query. This involved designing am async REST API, implemented via FastAPI, incorporating OpenAI/GPT-style tool-calling flow.

Outcomes

Toolio immediately garnered much interest (30 GitHub stars in 24 hours) from people facing the same output control and tool-calling needs, an area poised to unleash the keenest innovations.