AI Project to Review and Compare Policies

Problem Statement

The client had multiple large confidential documents (800+ pages each) on policies in 20+ countries which were inconsistent. They needed someone to review and find out the inconsistencies. They did not have the human resource available to carry out this review within the tight deadline imposed by the regulator.

Solution

Discussed with the client to better understand their problem and what are their requirements

Agreed with the client what success and end deliverable look like

Engaged with few developers (AI and Non-AI) to explore potential solutions

Presented various solutions to the client each with their pros and cons together with recommended solution

Client agreed to the solution where ChatGPT like experience will be implemented for them on one of their private machines - leveraging localized version of Large Language Model (LLM). This solution was preferred over a cloud based one due to the confidentiality of the documents involved and reduced costs

Drafted a letter of proposal to officially engage with the client clearly articulating the roles & responsibilities of stakeholders involved.

Proposed processes and controls surrounding the new workflow, leveraging my prior consulting experience in audit, process improvement and problem-solving

Agreed timelines, and created a project plan using ClickUp to make sure open and transparent communications takes place at all levels between the business, IT and my team

Whilst discussing client's existing processes and workflows, helped them identify more opportunities where they can leverage Generative AI

Prepared training materials to help the client with topics like prompt engineering

Worked with existing Change and Tech teams to integrate the new implementation into their Business-As-Usual (BAU) processes and workflows e.g. updating an existing Zapier based Policy Compliance Workflow to include a GenAI prompt

Solution delivered included retrieval augmented generation (RAG) of LLM (Llama-2) to improve the quality of the chat response

Held regular communication with the client over Slack and Zoom calls

Ran daily stands-ups (Scrums) to progress the implementation with the AI developers whilst holding preview calls with the client at set-upon intervals

Technology Stack: LM Studio, Llama-2 LLM, Zapier, Slack, Zoom, ClickUp, Calendarly.

End Result: Client was able to complete their analysis and share findings with the regulator a week before the deadline, project was a success.