Toolio platform for Private AI/Large Language Model Agents
Uche Ogbuji
Created a platform, Toolio, to run private Large Language Model (LLM) agents and tool-calling workflows on Apple Silicon, and also to constrain LLM output to a JSON schema, enhancing usability & inference speed.
Problem
Generative AI can be difficult to steer for controlled workflows, such as custom API Integration. It can also be hard to integrate such API usage (tool-calling) in a single, private platform, which is required e.g. for health-care and CRM needs.
My Contributions
Used my extensive AI/Machine Learning & Python programming experience to take a steering library for the output, and form it to API/tool requests based on the user query. This involved designing am async REST API, implemented via FastAPI, incorporating OpenAI/GPT-style tool-calling flow.
Outcomes
Toolio immediately garnered much interest (30 GitHub stars in 24 hours) from people facing the same output control and tool-calling needs, an area poised to unleash the keenest innovations.
Like this project
Posted Jul 3, 2024
Created an open-source platform, Toolio, to run private AI/LLM agents & tool-calling workflows on Apple Silicon; also to constrain LLM output to JSON schema.
Likes
0
Views
7
