Interactive learning through AI-powered Project Simulation
Ajay Pawriya
User Researcher
UX Designer
Product Designer
ChatGPT
Figma
Miro
Bluelearn
Product Prototype showcase
Project Overview
Imagine a learning experience that feels as engaging as a real-world project. Our platform uses AI to create a real-life learning experience that keeps students motivated and develops their skills.
Team Members
Ajay Pawriya (Me) - Product Designer
Rohan - Product Manager
Problem Statement
Initial Problem statement
The engagement and completion rates in traditional online learning platforms are notoriously low. Students often feel disconnected from the material, citing a lack of interaction and practical application.
Refined Problem Statement
How can we create a more engaging learning experience that simulates real-life projects and maintains user interest throughout the learning process?
Research
Context
In the landscape of major Ed-tech platforms like Coursera and Duolingo, there has been little innovation in the last decade. The average completion rate for courses like those on Udemy is around 10%, indicating a significant engagement problem.
Reference Article: Online courses have a completion rate of 12% to 15%
Target Audience
Age: 17-25 yrs old
Employment: Tech savvy, unemployed actively looking for job/internships or looking to upskill
People living in Tier 1 and 2 areas, since they know the real benefit of getting a job and learning on site
Competitor Analysis
Existing platforms rely heavily on passive learning methods. Innovations like job simulations provided by platforms like The FORAGE are interesting but lack interactivity and personalization, which are crucial for engagement.
Solutioning
Design Considerations
The primary considerations were:
Scalability - focusing on minimal human oversight and maximal automation.
Personalization - It should feel as if the product was made for that particular user
Ideation
Let's look inside and think of ourselves, how did we, employed people, learn a new skill?
Mostly by learning on the job. Where you are given tasks, timelines and constraints to work within.
This led us to the Idea of "Project Simulation" powered by AI, using an LLM API for dynamic and personalized project generation and chat support.
Research article that validated our approach to this solution. 7 out of 10 employees prefer to learn at work.
Tech feasibility check
No point building anything if later down the line you realize that the solution is not feasible. Hence, I tend to keep communicating with my Dev team to do feasibility checks alonside.
💡 TAKE MY WORD FOR IT: Often times you'll get easier and crazier solutions from your developers than sitting alone and brainstorming.
Me and Rohan(PM) sharing the idea with our tech team to get their ideas
Usability Testing
It was a big step that we were taking towards reimagining online learning. It would take a lot of Bandwidth from multiple teams to work on this.
To convince the stakeholders for this project and the user feedback as well, we ran a MVP testing batch for a week.
MVP Testing Process
MVP Testing structure
Experiment Setup
Multiple users were requested to volunteer for the Beta testing programme for this. To ensure no bias or skewed results, we took small chunks of users from various cohorts.
People who had never heard of bluelearn
Bluelearn users who never used Projects feature in the app
Bluelearn users who had completed atleast 1 project from bluelearn library
A WhatsApp group was created for each participant, including the user, the product manager, and an AI client powered by ChatGPT. Participants were unaware that ChatGPT facilitated the interactions.
MVP Beta testing WhatsApp groups
Process flow
I, as client, used ChatGPT to interpret and respond to users' messages, simulating dynamic, real-time conversations. This setup aimed to mimic genuine client-project communications without disclosing the AI's involvement.
But the user wasn't made aware that the responses are coming from AI model.
Experiment Flow and Environment
AI Training and Prompting
We trained our own custom AI ChatGPT to take on this task. It required us to iterate over our prompt over 20 times.
Prompt was rigorously tested along with all the edge cases and possible responses.
A sneak-peak into earlier version of our prompt 👀. (Don't try to steal it. Its very early variant and is very inefficient)
Process Flow
Daily Interactions: Users engaged daily with the AI client, discussing project progress and receiving guidance.
Role of ChatGPT: ChatGPT generated both reactive and proactive responses to keep the conversation engaging and responsive.
Submission and Feedback: Users submitted project deliverables for evaluation based on interaction quality and project completeness.
Observations and Feedback
Engagement: The responsive AI-driven interaction significantly enhanced user engagement with average <1hr response time
Completion Rates: About 80% of participants completed and submitted their projects, demonstrating improved engagement compared to traditional methods.
User Experience Feedback: All participants noted an improvement with the AI simulation; 67% found it better than traditional projects, and 33% considered it much superior.
Final Designs
This was an additional feature that we had launched on bluelearn, hence I used existing brand guidelines, components and style guide.
💡By the way, I took up the lead to build a robust and scalable Design system for bluelearn from scratch. Check out the case study here.
Possible ideas for next versions
Project Feedback & Evaluation: Since the entire chat between the client and the user happened on-the-platform, we can use it to give users feedback on multiple aspects like:
Automatic Case Study Builder: The entire conversation contains the steps, the challenges, the scope and much more data that can be used to automatically prepare draft 1 of case study for the projects. (If you're a designer you'd know how tiresome it is for some to write case studies to update their portfolio)
Community Feedback on project submissions: No matter how advance an AI model et, the real and qualitative feedback is best given by real humans. Hence, in future we also plan to revamp our existing submissions feed to be more suitable for feedback collection
Checkout the Figma Prototype for some ideas of new submissions feed that I had.
Conclusion
Growth happens by doing things you are unqualified to do
This was the first time I ever worked on AI powered ideas. This project not only enhanced the learning experience but also pushed the boundaries of educational technology by integrating AI into the core of the learning process. The results from the MVP testing underscored the potential of this innovative approach, setting the stage for further development and refinement.
