Job Description
Bolt combines advanced automation with expert human support to resolve customer issues. Both deserve the same honest measurement. You own the independent assessment of whether a customer's problem was actually solved, by a machine or by a person, and you make that measurement continuous, efficient and trusted.
As Quality and Evals Lead you replace manual quality audits with automated quality assessment, build the evaluation framework for AI resolution, and own the truth on agent-level and routine-level outcomes: reopen rates, repeat contacts, satisfaction on every contact, policy adherence where it matters. You sit in CX Performance, deliberately apart from the teams that build automation and run the vendors.
- Build and run the evaluation framework for automated resolution: golden sets, rubric-based grading, regression tests, release gates. No routine scales to a new market or intent without passing it.
- Replace manual quality audits with automated quality assessment across partner teams, at agent level, with calibration against human review on a sample.
- Own the quality truth: one definition of solved, reopened and repeated per vertical, reported weekly beside automation rate and cost, baselined honestly.
- Publish the quality floor every automation step must meet (at least as good as human on satisfaction and reopens) and hold the line when the schedule is tight.
- Lead a team of two to three quality specialists and work daily with Analytics, Resolution Engine and the automation team without being managed by them.
Requirements
- You've built evaluation or quality measurement for LLM-based or automated customer resolution at scale, or you have run quality for a large support operation and taught yourself the evals toolkit since.
- You're fluent in SQL, comfortable in Python, and you know how to design a sample, a rubric and a control.
- You have replaced a manual audit process with an automated one and can describe what you lost and what you gained.
- You are comfortable being the person who says a number is not true, and you say it with evidence.
- You build credibility with business leaders and engineers, communicate directly and challenge decisions with evidence. You're rigorous about data quality and honest about what the results show.
- You work effectively with few management layers, take ownership of outcomes, move quickly and use resources thoughtfully.
Company offers
- Accelerate your professional growth with unique career opportunities.
- Enjoy a rewarding salary and stock options, knowing that as Bolt succeeds, so do you.
- Take care of your physical and mental health with our wellness perks.
- Celebrate 5 years at Bolt with a 1-month paid sabbatical to recharge.
- Connect with colleagues at annual company events and smaller team gatherings.
- Balance flexibility and in-person collaboration with our hybrid model, including at least 12 monthly in-office days.