Voice-Enabled AI Assistant for Fastn.ai
Umar Farooq
Fastn Voice Force
A voice-enabled AI assistant powered by the OpenAI Realtime API and OpenAI Agents SDK, featuring specialized agents for Fastn.ai tool integration and documentation assistance.
About the OpenAI Agents SDK
This project uses the OpenAI Agents SDK, a toolkit for building, managing, and deploying advanced AI agents. The SDK provides:
A unified interface for defining agent behaviors and tool integrations.
Built-in support for agent orchestration, state management, and event handling.
Easy integration with the OpenAI Realtime API for low-latency, streaming interactions.
Extensible patterns for multi-agent collaboration, handoffs, tool use, and guardrails.
For full documentation, guides, and API references, see the official OpenAI Agents SDK Documentation.
Features
This project includes two specialized AI agents:
Fastn Tools Agent: Dynamically fetches and executes Fastn.ai tools for various integrations including Google Workspace, Slack, and other productivity platforms.
Fastn Docs Agent: Provides comprehensive assistance with Fastn documentation, best practices, and getting started guidance.
Setup
Prerequisites
Node.js (v16 or higher)
OpenAI API key
Fastn.ai API credentials
Installation
Install Dependencies
Environment Configuration
Copy the sample environment file and configure your API keys:
Update
.env with your credentials:Note: The Fastn Docs Agent uses pre-configured credentials and doesn't require additional setup.
Getting API Credentials
OpenAI API Key: Get your API key from OpenAI Platform
Fastn Tools API Credentials: Visit Fastn UCL Platform to:
Create your account and workspace
Generate API keys for your tools integration
Get your Space ID from your workspace settings
The Fastn Docs Agent comes pre-configured with documentation access and doesn't require separate API credentials.
Running the Application
Start Development Server
Open Application
Navigate to http://localhost:3000
The app defaults to the
Fastn Tools AgentSwitch between agents using the "Agent Set" dropdown in the bottom toolbar
Agent Architecture
Fastn Tools Agent
The Fastn Tools Agent dynamically fetches and executes tools from the Fastn.ai platform. It can:
Access Google Workspace (Docs, Sheets, Calendar)
Integrate with Slack for messaging and notifications
Handle various productivity and workflow automation tasks
Provide intelligent tool selection based on user requests
Fastn Docs Agent
The Fastn Docs Agent specializes in providing assistance with Fastn documentation and knowledge base. It can:
Answer questions about Fastn features and capabilities
Provide getting started guidance and tutorials
Explain best practices and usage patterns
Help troubleshoot common issues
Offer examples and code snippets
Benefits
Voice-First Interface: Natural conversation experience with low-latency responses
Dynamic Tool Access: Automatically discovers and uses the latest Fastn.ai tools
Specialized Knowledge: Dedicated documentation agent for comprehensive Fastn guidance
Seamless Integration: Direct connection to Fastn.ai platform and third-party services
Modular Design: Easy to extend with additional agents for specific use cases
Intelligent Routing: Automatically selects the right tools based on user intent
Configuration
The application is configured with two main agent sets:
Fastn Tools Agent (
src/app/agentConfigs/fastnAgent.ts)Dynamically fetches available tools from Fastn.ai platform
Handles complex tool execution workflows
Provides intelligent parameter mapping and validation
Fastn Docs Agent (
src/app/agentConfigs/fastnDocsAgent.ts)Connects to Fastn serviceAgent API for documentation queries
Provides contextual help and examples
Assists with onboarding and troubleshooting
Adding Custom Agents
To add your own agent:
Create a new agent configuration in
src/app/agentConfigs/Add the agent to the
allAgentSets mapping in src/app/agentConfigs/index.tsUpdate the UI selector options in
src/app/components/BottomToolbar.tsxAPI Integration
The application integrates with Fastn.ai through two main endpoints:
Tools API:
https://live.fastn.ai/api/ucl/getTools - Retrieves available toolsExecution API:
https://live.fastn.ai/api/ucl/executeTool - Executes tools with parametersServiceAgent API:
https://live.fastn.ai/api/v1/serviceAgent - Queries documentationUsage
Interacting with Fastn Tools Agent
"Create a new Google Doc called 'Meeting Notes'"
"Send a Slack message to the team channel"
"Schedule a meeting for tomorrow at 2pm"
"Add a new row to my project tracking sheet"
Interacting with Fastn Docs Agent
"How do I get started with Fastn?"
"What integrations does Fastn support?"
"Show me examples of workflow automation"
"Help me troubleshoot my API connection"
Output Guardrails
Assistant messages are checked for safety and compliance before they are shown in the UI. The guardrail call now lives directly inside
src/app/App.tsx: when a response.text.delta stream starts we mark the message as IN_PROGRESS, and once the server emits guardrail_tripped or response.done we mark the message as FAIL or PASS respectively. If you want to change how moderation is triggered or displayed, search for guardrail_tripped inside App.tsx and tweak the logic there.Navigating the UI
You can switch between agent sets using the "Agent Set" dropdown in the bottom toolbar.
The conversation transcript is on the left, including tool calls, tool call responses, and agent actions. Click to expand non-message elements.
The event log is on the right, showing both client and server events. Click to see the full payload.
On the bottom toolbar, you can disconnect, toggle between automated voice-activity detection or push-to-talk, turn off audio playback, toggle logs, and select different agents.
Resources
Fastn Platform
Fastn UCL Platform - Get your API keys and manage your Fastn workspace
Fastn Documentation - Complete guide to Fastn features and integrations
Developer
Umar Farooq LinkedIn - Connect with the developer
Umar Farooq Portfolio - View portfolio and projects
OpenAI
OpenAI Platform - Manage your OpenAI API keys
Realtime API Documentation - Learn about the Realtime API
Contributing
This project demonstrates voice-enabled AI agents for Fastn.ai integration. Feel free to extend it with additional agents, tools, or integrations that enhance the Fastn workflow automation experience.
For questions, suggestions, or collaboration opportunities, reach out via LinkedIn or check out the developer's portfolio.
Like this project
Posted Oct 6, 2025
Developed voice-enabled AI assistant for Fastn.ai using OpenAI SDK.
