Summary

FAQs

• How can you ensure consistency?

  We, and OpenAI are aware that GPT models can easily drift from their intended purpose, or even straight-up ignore instructions from the first message. We stay on track and up-to-date with current model progressions as active members of the community for both communication and developer channels. The key to consistency with these models is to accept that they are a black-box, which means that empirical data is all that we can use, and attempt to derive from. So, it may come as a hard truth when there are 100 instructions and we say that these must be reduced to 20, or that they must be changed. It's not because you are wrong, it's because we know the current model limitations!

• What is RAG?

  RAG, or Retrieval Augmented Generation, simplifies interactions by using a database of documents formatted for the Language Model to understand. When a user's query necessitates specific information, the model retrieves it from this database. For instance, if a user inquires about a particular product, the model consults the database to obtain the relevant product details. This information is then used to augment the model's response, ensuring it bases its answers on accurate, up-to-date data. We use Weaviate as a Knowledge Graph Database to accomplish this.

• Is it safe to use GPT for public-facing services?

  Yes, and no. GPT's extensive knowledge and power are significantly enhanced with a well-integrated RAG system, providing accurate, non-hallucinated information and the ability to acknowledge gaps in its data for certain queries. However, challenges arise with unusual or improperly formulated questions. These could be due to a user's unfamiliarity with how to ask effectively, or deliberate 'prompt injections' aimed at confusing the system. To address this, we strongly advise incorporating a Human-In-The-Loop system for any public-facing chatbot. This setup ensures that if the AI encounters any anomalies or complex queries, it promptly defers to a human for accurate responses. While this may seem like a limitation, it's actually a strategic advantage. The vast majority of inquiries are efficiently handled by the AI, reserving human intervention for only those rare, intricate cases. This approach not only cuts costs and saves time but also allows for the collection of valuable insights from these edge cases. Over time, this data helps us refine the AI's capabilities, gradually increasing its proficiency in handling a wider range of queries autonomously.

• Are hallucinations an issue?

  Hallucinations in Large Language Models, rather than being solely a drawback, can be seen as a wellspring of creativity and an expression of the model's 'free-thinking' capabilities. These imaginative responses add a unique dimension to the AI's interaction. Certainly, in scenarios where accuracy is paramount, such as answering factual questions, hallucinations can pose a challenge. This is where a robust, thoroughly tested RAG system plays a crucial role. It ensures the AI has access to verified information, thereby reducing the likelihood of inaccurate responses. Additionally, empowering GPT to recognize and admit when it doesn't have sufficient data to answer a question is a key feature in maintaining reliability. By balancing this creative aspect of language models with reliable data retrieval and the ability to decline answering when uncertain, we strike a harmonious balance between innovation and accuracy.