Custom chatGPT voice assistant chatbot



Table of Contents

Client's Objective

I am looking for a chat bot solution for residents of my old age home, where a virtual assistant personalised with each user's data helps them answer their personal questions which they may ask regarding their family, friends and help them with their tasks and common doubts. For example, a user may ask: Query : Where is John? Reply : Your Friend John has gone to the market, He'll be back around 10 AM

Project Specifications

• integrate voice input that lets user to record and send input
• implement spoken responses
• option to send text input
• get response based on custom data
• a character for visualisation that:
  • stays Idle during input
  • 'thinks' when the response is being loaded
  • 'speaks' when the response is spoken out

Proposed Ideas

• have an authentication mechanism for each user that loads the document with data of the particular user based on the user authenticated
• use of costly characterAPI responsible for assistant speaking out response was replaced by three GIF that was updated based on the user action:
  • Idle animation during input
  • 'thinking' animation when the response is being loaded
  • speaking animation when the response is spoken out

Technologies & Services Used

Language Model: ChatGPT Scripting Language: Python Speech Recognition: WhisperAI Text to Speech: AWS Polly Custom Doc Consummation: langchain Server & Hosting: Digital Ocean

Solution Provided

Delivered a cost efficient & responsive, virtual assistant as a web app that included a 'chat' styled conversation box to add to the feeling of texting and an interactive virtual character that speaks out responses. The features included:

• choice of input query (text/voice command)
• response generation based on custom data
• profiles for several user with each unique data mapped with each unique user
• response from the assistant are spoken out as well as produced as a text reply
• custom response length for context preserving & lowering API costs
• preserving logs of conversation to be reviewed later
• dark mode option

Chatbot Previews

To preview & analyse the chatbot please visit: https://asiffarhankhan-custom-gpt-voice-assistant.hf.space

Note : The above URL for the chatbot repository may have been made private 
       at the time of viewing this. Please contact me for requesting a preview 
       asiffarhankhan[at]outlook.com







Mobile Dark/Light Mode

Desktop View Dark Mode