Summary

FAQs

• Who's behind this service?

  I'm Josh Pitzalis, working alongside my development partner Sasha Koss. Together we built Chirr App, one of the first Twitter thread schedulers back in 2017, and have over 15 years of combined development experience. Sasha is the author of date-fns, the second most popular JavaScript utility library in the world according to the 2024 State of JS report. We've been exploring AI infrastructure and evaluation systems through projects like Mind Control (a CMS for production AI prompts) and DaisyChain (a no-code prompt chaining tool). Our backgrounds combine deep software engineering expertise with an understanding of the LLM evaluation process and practical lessons form building two AI-infrastructure projects.

• What's required from my team?

  - 2 hours of your time. - You can share access to your AI systems and current performance data over the call if you want it to be reviewed, but these is no expectation to.

• Do you work with early-stage startups?

  For the development and evaluation services we offer, we tend to focus on growth-stage companies (Series A-C) with established AI products and meaningful API spending ($5K+ monthly). However, the consultation service is designed for any stage company that needs strategic clarity on their AI approach - whether you're pre-Series A exploring AI possibilities or a larger company deciding between different implementation paths.

• What if my AI system is unique or complex?

  Perfect - that's exactly what I specialize in. Generic monitoring tools and cookie-cutter solutions fail because every AI application has unique failure modes and requirements. I generally work with you to create custom evaluation frameworks and development approaches that understand your specific domain, whether that's legal document analysis, medical diagnosis, financial risk assessment, or something completely different.

• How do you handle different tech stacks?

  All development work is done in TypeScript, which integrates well with most modern web applications. For evaluation services, I work with your existing infrastructure regardless of language - the monitoring and evaluation systems are designed to work alongside your current setup.

• What about data privacy and security?

  I work within your existing security protocols and can sign NDAs as needed. For sensitive data, we can work with synthetic data or implement evaluation systems that don't require exposing actual customer data. All code and documentation developed during our engagement belongs entirely to you.

• Can you work with regulated industries?

  Yes, I have experience working with companies in legal, healthcare, and financial services. I understand the additional compliance requirements and can help design evaluation systems that meet regulatory standards while still providing actionable insights.

• What happens if the project needs to be extended?

  All services include built-in buffer time, but if scope changes significantly, we'll discuss timeline adjustments upfront. The goal is always to deliver complete, working solutions within the stated timeframe.

• Do you provide ongoing support after the engagement?

  All services include a 90-day support period for questions and adjustments. For longer-term support or additional features, we can discuss a follow-up engagement.

• What about guarantees or refunds?

  For the consultation service, if you don't find the session valuable you'll get a full refund. I'll only charge you if you genuinely found the consultation useful to your situation. Since AI development is still such an early-stage field with everyone doing things differently, there's a strong chance our experience can help based on the problems we've solved before, but if we can't add value to your specific situation, then we don't charge you - it's that simple. For the development and evaluation services, all engagements include a satisfaction guarantee. If you don't see measurable improvement or deliverables that meet the agreed specifications, we'll work together to make it right or provide appropriate compensation.