Overview

Powerbroker AI, a startup that supports insurance companies, partnered with Sermonis AI to develop an innovative solution for analyzing extensive insurance policies. The project aimed to create an agent that could act as an insurance policy analyst, fielding questions pertaining to the inclusion of specific inclusions and carvebacks in various types of insurance policies, including business auto coverage (car dealerships), building and personal property coverage, commercial property business income coverage, terrorism risk coverage, war and state cyber security coverage, commercial general liability coverage, and business auto coverage. By leveraging cutting-edge technology and expertise, Powerbroker AI sought to revolutionize the way insurance policies are analyzed, making it easier for insurance companies to understand their policies and make informed decisions.

The partnership between Powerbroker AI and Sermonis AI was a strategic one, as both companies shared a common goal of transforming the insurance industry through innovation. By combining their expertise in RAG technology and natural language processing, they were able to develop a solution that exceeded expectations and delivered significant value to Powerbroker AI's clients.

Background

Powerbroker AI faced challenges with poor reliability in identifying specific terms in extensive insurance policies, including inclusions and carvebacks. This was due to the complexity of these policies, which often contain ambiguous language and nuanced terminology. As a result, manual analysis of these policies was time-consuming and prone to errors, leading to delays and inefficiencies in the underwriting process.

The types of insurance policies analyzed as part of this project included business auto coverage (car dealerships), building and personal property coverage, commercial property business income coverage, terrorism risk coverage, war and state cyber security coverage, commercial general liability coverage, and business auto coverage. These policies often contained complex language and nuanced terminology, making it difficult for human analysts to accurately identify specific inclusions and carvebacks.

To address this challenge, Powerbroker AI turned to Sermonis AI for help. With their expertise in RAG technology and natural language processing, Sermonis AI was able to develop a customized solution that could accurately identify specific terms in insurance policies.

Approach

The approach taken by Sermonis AI was to develop a RAG system that could accurately analyze extensive insurance policies and identify specific inclusions and carvebacks. This involved several key steps:

- First, the team at Sermonis AI worked with Powerbroker AI to understand their requirements and goals for the project.

- Next, they developed a customized RAG system using LangChain, leveraging Anthropic's Claude 2.1 and GPT-4.

- The system was designed to analyze input documents, extract relevant information, and produce accurate results.

- To ensure the accuracy of the system, the team at Sermonis AI conducted extensive testing and validation.

The result was a RAG system that could accurately identify specific inclusions and carvebacks in insurance policies with high precision and recall. This system was able to handle complex language and nuanced terminology, making it an ideal solution for Powerbroker AI's clients.

Results

The results of the project were impressive. The RAG system developed by Sermonis AI achieved remarkable accuracy and speed in identifying specific terms in insurance policies. A collection of benchmark queries that previously failed worked after the prompt optimizations provided by Sermonis AI.

To measure the success of the project, Powerbroker AI used a set of key performance indicators (KPIs) that included:

- Accuracy: The system's ability to accurately identify specific inclusions and carvebacks in insurance policies.

- Speed: The time it took for the system to analyze input documents and produce results.

- Recall: The system's ability to retrieve all relevant information from input documents.

The results showed significant improvements in accuracy, speed, and recall, demonstrating the effectiveness of the RAG system developed by Sermonis AI.

Impact

The successful implementation of the RAG system had a significant impact on Powerbroker AI's operations. The improved accuracy and speed enabled the company to present their system to alpha customers, including NYC-based insurance companies.

The feedback from these customers was overwhelmingly positive, with many expressing surprise at the accuracy and speed of the system. This feedback validated the effectiveness of the RAG system developed by Sermonis AI and demonstrated its potential for transforming the insurance industry.

Conclusion

The partnership between Powerbroker AI and Sermonis AI demonstrates the power of RAG technology in transforming insurance policy analysis. By leveraging cutting-edge technology and expertise, Powerbroker AI achieved significant improvements in accuracy and speed, enabling them to deliver exceptional value to their clients.

This case study highlights the potential for RAG technology to revolutionize the insurance industry by making it easier for insurance companies to understand their policies and make informed decisions. As the insurance industry continues to evolve, we can expect to see more innovative solutions like this one that leverage RAG technology to drive business success.