AI Project to Review and Compare Policies

Problem Statement

The client had multiple large confidential documents (800+ pages each) on policies in 20+ countries which were inconsistent. They needed someone to review and find out the inconsistencies. They did not have the human resource available to carry out this review within the tight deadline imposed by the regulator.

Solution

1. Discussed with the client to better understand their problem and what are their requirements
2. Agreed with the client what success and end deliverable look like
3. Engaged with few developers (AI and Non-AI) to explore potential solutions
4. Presented various solutions to the client each with their pros and cons together with recommended solution
5. Client agreed to the solution where ChatGPT like experience will be implemented for them on one of their private machines - leveraging localized version of Large Language Model (LLM). This solution was preferred over a cloud based one due to the confidentiality of the documents involved and reduced costs
6. Drafted a letter of proposal to officially engage with the client clearly articulating the roles & responsibilities of stakeholders involved.
7. Proposed processes and controls surrounding the new workflow, leveraging my prior consulting experience in audit, process improvement and problem-solving
8. Agreed timelines, and created a project plan using ClickUp to make sure open and transparent communications takes place at all levels between the business, IT and my team
9. Whilst discussing client's existing processes and workflows, helped them identify more opportunities where they can leverage Generative AI
10. Prepared training materials to help the client with topics like prompt engineering
11. Worked with existing Change and Tech teams to integrate the new implementation into their Business-As-Usual (BAU) processes and workflows e.g. updating an existing Zapier based Policy Compliance Workflow to include a GenAI prompt
12. Solution delivered included retrieval augmented generation (RAG) of LLM (Llama-2) to improve the quality of the chat response
13. Held regular communication with the client over Slack and Zoom calls
14. Ran daily stands-ups (Scrums) to progress the implementation with the AI developers whilst holding preview calls with the client at set-upon intervals
15. Technology Stack: LM Studio, Llama-2 LLM, Zapier, Slack, Zoom, ClickUp, Calendarly.
16. End Result: Client was able to complete their analysis and share findings with the regulator a week before the deadline, project was a success.