The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Back to feedPost
Before I share research findings, I ask AI to attack them first.
Not summarize. Attack.
It's easy to walk away from a few interviews feeling confident about a pattern. AI is good at making that confidence look even more solid — clean summary, clear themes, done. That's exactly the problem. Confidence isn't the same as evidence.
So before I write up any findings, I run them through this: "Here are my research findings: [paste]. Play devil's advocate. What would a skeptical reader say is wrong with this conclusion? What's the weakest part of my evidence?"
It's caught things like a "pattern" that was really just one participant's opinion repeated across my notes.
Takes 2 minutes. Cheaper than being wrong in public.
What's one thing AI has pushed back on when you've tried something like this?
#UXResearch #UserResearch #AIinResearch
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Challenges
View all
Related posts
Option A
At CRITER.AI, we’re building AI that helps migrants avoid expensive legal mistakes and visa bans by auditing Australian visa applications against immigration law. We’ve been getting a healthy amount of investor interest, but there was one recurring piece of feedback: our pitch deck was lackluster.
Fair enough. As a lawyer, my creativity usually goes into finding the legal issue everyone else missed, not deciding which shade of blue says “invest now.”
So we set out to change that. We chose Contra over its competitors because the caliber of people looked considerably stronger.
When I posted our request for a pitch-deck redesign, I expected a few people to reach out with quotes. As of today, we have had 303 applicants.
For a bootstrapped business, working through that volume was simply impossible. So we created a challenge:
“Here is my pitch deck. Pick the slide you think needs the most work, and show me what you can do.”
What followed was fascinating.
101 applicants (around 1/3) never replied. Of those who did, around 10% simply reiterated what they had already said. One person replied that, with 250 applicants, they were not going to bother trying. lol.
Several asked whether the test would be paid. 12 sent a quote for the work to be paid first, after which I could decide whether to go further.
Less than 25% of the applications were obviously AI-generated. We are by no means AI-phobic, but the Claude watermark was perhaps a little too much. You know who you are dude!
2 people had trouble opening the Google file we sent. I’m still not entirely sure why.
7 people were so impressed by the work they did that they watermarked it, labelled it “draft,” or otherwise made it impossible to download for further review.
12 people reached out through LinkedIn rather than the app; a further 18 emailed; and 1 person even called me.
18 people not only sent an example slide; they also signed up for our free immigration-advice beta test.
From 303 applicants, we narrowed the field to 15 people who were simply too good to ignore.
Every one of those candidates took the time to understand what we were trying to do, what CRITER.AI is about, and spoke to us as people rather than as another job listing. Two spotted mistakes in the pitch deck that we would not have noticed without them. Another created a 3-minute Loom presentation explaining the changes they made and why they mattered. Most went beyond the requested single slide. One applicant redesigned the entire pitch deck and sent it watermark-free.
It was a difficult decision, but the person we chose had just that little bit more talent than an exceptionally strong group. They not only showed that they understood what we wanted and impressed us with the slide they redesigned; they also suggested another slide we had never considered relevant, then created it using information we had not even provided.
They took our AI immigration-law business and made our pitch deck speak to investors on a human level.
Can't wait to come back to resign our website, social media and other design work.
Lovely and impressive!