Today, we’re launching Candor, a synthetic user research platform.
This isn’t an ad for the product. It’s about what I learned getting it ready to launch.
We started in March. I’ve written all the code (thanks AI!) My colleague, Elizabeth van Monsjou, who leads research at Highline Beta, has provided the expertise, research rigour and a ton of testing. Tim Sun, one of our product designers stepped in last minute to help as well.
Candor hasn’t been our full-time job, which means it’s taken longer than I would have liked. I get twitchy if it takes more than 3 months to build an MVP.
But here we are.
Candor is now live, and I spent most of last week and the weekend grinding through final details. It reminded me of something that’s increasingly easy to forget in the age of AI coding: The gap between “the product works” and “the product is launchable” is significant. Most of it isn’t code.
AI made building Candor dramatically easier. It didn’t solve onboarding, positioning, design, customer segmentation, support, feedback, metrics, sales motion or distribution.
Those things still require judgment.
Here’s what I learned.
The happy path is a lie
We piloted Candor with real users for a month before launch. We learned a lot.
Of course they found bugs. More importantly, they showed us how people actually tried to use the product.
It’s easy when you’re building something to focus on the happy path. You know how everything is supposed to work. You know what to type into every field. You move quickly through testing because, subconciously, you’re trying to confirm that what you built works.
Real users and customers don’t work that way. They go in with fresh eyes and do what people do: muck things up. 😂
It only took two people using Candor for us to see where we’d gone wrong.
The first step in Candor is creating a project and generating an audience of synthetic users to research. To do that well, Candor needs information from the user.
Our first test users weren’t sure what to provide.
They didn’t know what “good” looked like. They didn’t know how much detail to include or how their inputs would affect the synthetic participants being generated.
Some gave us almost nothing. Others wrote a lot without much intent behind it. Both could produce poor results. And if Candor generates weak synthetic users, the customer blames Candor.
They’d be right to.
We had a classic garbage-in, garbage-out problem, with an AI-twist: poor inputs don’t necessarily generate an obvious error. They can generate something that looks plausible but isn’t very good.
We made a bunch of changes:
We simplified the workflow. Originally you created a project, then an audience, then participants. Each felt like a separate object and it wasn’t obvious what decisions users were being asked to make. Today it’s a single three-step flow. You finish with what you actually want: synthetic users you can research.
We added better examples. Candor supports B2B and B2C research, so examples for text inputs change depending on which you choose. Context matters.
We clearly separated required and optional fields and explained why we’re asking for each one.
We added quality checks on inputs that help people improve weak inputs before Candor generates participants.
None of this surfaces if you only walk the happy path. You built the thing, so you use it the way it was designed to be used. Your users don’t know the design. They just start doing things.
Get real people into your product early. Watch where they struggle. Look specifically for places where user behaviour can create bad outcomes.
And deliberately test edge cases. Some edge cases are fine to ignore. Others are product holes masquerading as unusual behaviour.
AI has dramatically accelerated development. That makes it even easier to rush past these details. Don’t.
Sweat the details that affect whether someone gets value.
There are always bugs. There are always improvements. There is always another thing you could refactor, redesign, optimize or clean up.
AI coding makes this worse.
AI is good at finding problems. It’s also good at over-complicating solutions.
Every time Claude Code or Codex (or any AI coding tool) does something for you, it’s going to find more things to do. It’s so easy to agree with the AI tool and get it to keep doing things. Suddenly you’re four hours deep on something no user will ever notice.
I’ve written before that vibe coding without system design is a trap. But there’s another trap: AI has made the marginal cost of changing software so low that it becomes harder to stop changing it.
Before AI, the cost of another feature or refactor forced prioritization. Now another change feels cheap. Twenty minutes here. Thirty minutes there. Forever.
At some point I had to draw a line with Candor and say, “F*ck it, ship it”.
That’s a judgement call, not a surrender. Polishing Candor forever would delay the thing that matters much more now: putting it into customers’ hands and learning whether they actually get enough value to keep using it and pay for it.
The purpose of an MVP isn’t to eliminate uncertainty. It’s to move uncertainty from inside your head into the market.
Eventually you have to ship.
I want to interrupt this post for a moment to introduce you to the Highline Beta Shop.
It’s a merch store for startup folks and corporate innovators, with clothing, accessories, prints and more.
Every Focused Chaos subscriber gets a 20% discount. 🎉
Please go check it out and buy some merch! 😍
OK, back to our regularly scheduled post…
An MVP isn’t an excuse for ugly
I’m a fan of the MVP. I’ve defended it more than once.
Minimum never meant crappy.
People are inundated with software, particularly new AI products. They make very fast judgments about whether something deserves their attention.
A rough product from a company that looks credible may get another chance. A rough product that also looks sloppy probably won’t.
Candor’s first website was completely AI-generated. I focused most of my energy on the content and relatively little on the design. It was OK. But OK is invisible.
At the very last minute I asked Tim Sun, one of our product designers at Highline Beta, for help. I gave him a couple days. What he produced is dramatically better.
We’re still working through messaging. For example, one line we liked, “Building got easier. Building the right thing didn’t.” resonated with a particular audience but not everyone. So we removed it.
There’s definitely more to do, especially inside the product. Candor needs real UI/UX love. Small things compound: the words on a button, a single line of instruction above a field, an empty state. Those are product decisions, not decoration.
An MVP doesn’t need to be beautiful, but it can’t look like it’s completely AI-generated with no attention to detail. Otherwise it won’t clear the credibility threshold required for someone to give it a real chance.
Your ICP can be fuzzy but your data shouldn’t be
I don’t know Candor’s ICP (ideal client profile) with certainty. I think that’s true for a lot of startups at launch.
My current hypothesis is that Candor fits larger companies with multiple product and/or marketing teams, plus consultants who run a lot of market and customer research. But we’ve also seen smaller and mid-sized startups get value from it. Different buyers, different research cadence, different willingness to pay. Potentially different reasons for using the product at all.
The mistake would be averaging them together.
You don’t have to be right about your ICP on day one. But it’s a mistake to blend different customer segments together.
Right from the get go, I’m trying to define segments:
If someone books a demo, I ask a few questions to understand more about them.
When a user signs up, the onboarding process asks for a bit of context as well, without creating too much friction.
We can also enrich customer data to learn more about the companies signing up.
At the top of the funnel, I want to understand who shows up.
Then I want to understand what happens by segment. Do senior product leaders at large companies use Candor more than senior marketing leaders? Do research and insights teams use Candor more than product teams? Do consultants run more studies? Which segments activate faster? Which retain? Which pay more?
The goal isn’t to create ten segments and drown in analysis.
It’s to avoid concluding, for example, that “retention is mediocre” when one customer segment loves the product and another doesn’t care.
I’m fairly certain I’ll be surprised by which segments emerge.
Before launch, define a handful of plausible segments. Then track them separately. Your customers will tell you which segmentation actually matters.
Your value proposition is a hypothesis too
Before launch, you’ve probably landed on a value proposition you believe in. Maybe two. That’s the right starting point. It isn’t the same thing as knowing.
Customers constantly surprise you about why they care about what you built.
With Candor, we currently see at least three potential value propositions:
Research is dramatically faster.
People who aren’t professional researchers can still run useful research.
Companies can do research more frequently because the cost and effort drops.
Which one matters most?
I don’t know yet.
And I suspect the answer changes by customer segment. A consultant may buy speed because speed is margin. A large product organization might buy frequency because PMs can run research without waiting for access to a research team. A smaller startup may care most about access: they want customer insight but don’t have an internal research or a big research budget.
Same software. Different product in the customer’s head.
That made the website surprisingly difficult. We have two jobs on one page:
Explain enough about synthetic research that someone understands and trusts the concept.
Communicate why they should care.
Those goals fight each other. Explain too much and you’re teaching a category instead of selling a benefit. Explain too little and people may not trust what the product is doing.
I used Jason Cohen’s new book, Hidden Multipliers to help work through this. His framing around need states and the hierarchy of needs forces you to separate the functional need from the deeper one underneath it.
For now, Candor’s website leans fairly heavily on functional benefits because we believe establishing credibility around the quality of synthetic users and research is essential.
I’m not convinced we have it right. That’s OK (for now).
Write down your competing value propositions before launch. Then listen for which ones customers repeat back to you without being prompted.
That’s far more valuable than arguing internally about which headline sounds best.
Usage is usually the first signal. But usage tells you what someone did, not why.
Why did they come back? Why did they stop? Did the output actually help? Was something confusing? You have to ask.
We built two short feedback mechanisms directly into Candor at moments when the user is best positioned to judge quality:
immediately after Candor generates synthetic participants
immediately after Candor generates a research report
My goal is to determine if what Candor generates is good and creates value for users.
We’ve also added two other feedback loops:
First, we’ll try to speak with as many early users as possible after they’ve actually used the product. Those conversations will almost certainly produce richer insight than any survey.
Second, we installed Fullstory, with all inputs excluded from capture, so we can understand how people actually navigate the product.
The key: Have a feedback plan in place before launching that covers what people are doing and why, through quantitative and qualitative analysis.
Decide what “good usage” means before you see the data
You should define what good usage looks like before launch. Otherwise every number becomes easy to rationalize.
Every product has some natural frequency of use. I wrote about this in 4 Steps to Building Super Sticky Products, including the HighScore House story where we set an expected benchmark and adjusted it once real behaviour gave us better information.
For Candor, I’m not sure what the right frequency is. That bugs me. 😏
Candor isn’t a daily use product. It may not even be weekly. How often does a company actually run user research? Monthly? Quarterly? I’m targeting weekly use, but I think that’s ambitious.
Infrequent-use products are harder to evaluate. If someone hasn’t logged into Candor for three weeks, have they churned? Or are they simply between research projects? We may not know for months. That makes early retention data especially difficult to interpret.
Still, pick a number.
An expected usage cadence gives you something to falsify. Without one, inactivity can always be explained away.
For Candor, that number is a hypothesis, not a truth. And like everything else, I expect it to vary by customer segment.
Support is a signal, not just a service
Decide what support you’re willing to provide before the first confused customer asks for it.
Candor has detailed help documentation and Candor Bot, an in-product chat interface with access to those docs. Candor Bot also understands some context about where you are in the product, which means it can answer questions such as, “What am I supposed to put here?”. That kind of AI support is increasingly common. I don’t know how effective ours will be.
We’re also offering training sessions to new customers. I’m wary of adding onboarding friction, but for such a new category it might be the difference between someone getting value and someone bouncing.
There’s an irony when it comes to support. You want it to be excellent, but you also want almost nobody to need it.
Having said that, I’m convinced it can be a differentiator. With everyone automating customer support, having access to a real human could be amazing. I plan to be very proactive.
But, if everyone needs a training session to use Candor, we’ve failed at the product.
Track what people ask. Repeated support questions aren’t just support issues. They’re product data.
The sales motion you want versus the one you get
Candor is a B2B product. The motion I want is simple:
People discover Candor
They sign up
They try it with our free trial
Some percentage convert into paying customers
What percentage? No idea yet. A high percentage would be…nice. 😆
We also have a “Book a demo” option on the site, because synthetic user research is still unfamiliar to a lot of people. We want to give people an option to engage with us before jumping in. Hopefully this reduces the number of people that sign up, poke around a bit, get confused and leave. At the same time we can’t do a demo for everyone.
The problem is economics. If everyone needs a demo, the motion probably doesn’t work at our price point.
If Candor requires a traditional enterprise sales motion with multiple meetings, procurement, pilots and long sales cycles, we may be in trouble. We don’t necessarily have the bandwidth or business model to support that.
That’s why we’ve kept self-serve signup prominent. “Try for Free” is a strong CTA.
What I want to learn isn’t simply, “Will people buy Candor?” I also need to learn if Candor can be sold in a way that makes sense for the business.
You can have a product customers want and still have a broken business if the cost of acquiring, selling and supporting those customers is too high. Set a threshold for how much human involvement your pricing model can tolerate. Then watch what the market actually demands.
Launching is legitimately just the beginning
A couple years ago I wrote a post called “What Comes Before Zero to One”. I’ve been thinking about this a lot while building Candor.
People often talk about “zero to one” as the process of building and launching something. I don’t think launching gets you to one.
Getting to one means you’ve demonstrated real traction and shown that you’re consistently creating value. Note: That’s still not the same as product-market fit. But it’s much further than, “We have a website,” or “We have 3 users trying things out.”
Candor already has a few customers, a few pilots underway and a waitlist of ~200 people we’re working through.
That’s a good starting position. It isn’t a business.
Now comes the hard part. Marketing. Distribution. Conversion. Retention.
We still have to figure out which customers care most and why. We have to learn why people come back, whether they’ll pay and what breaks when usage increases.
We have a content plan that should help with SEO and AEO. We’ll do social. We’ll work our network. That won’t be enough on its own.
We’re not planning to spend heavily right now because Candor is one of several initiatives at Highline Beta. But we do want customers. And wanting isn’t a channel.
A launch is you standing up and saying: “Here I am. I’m ready.” Usually you get a little burst of activity. And then…😴
That’s when the real work begins. The launch isn’t the milestone. Month two is. And month three. And four. And five. Ask me again then what I’ve learned.
The gap between "works" and "launchable" is so real. On the garbage-in problem in the first step: besides text examples, a short clip showing someone filling the audience inputs well can set the "what good looks like" bar faster than copy. Full disclosure, I work on Nereo (https://www.nereo.io/), an AI video tool, so I'm biased toward video, but onboarding inputs seem like a place where showing beats telling. Congrats on the Candor launch.
Wow Ben, this is a rich post. Like a walk through my career memory lane. Congrats on the launch! The "happy path illusion" is the one I keep relearning building agentic apps. The gap between "the demo works" and "a stranger can actually get through it unassisted" is brutal. AI makes it dangerously easy to keep polishing the demo instead of closing that gap.
The gap between "works" and "launchable" is so real. On the garbage-in problem in the first step: besides text examples, a short clip showing someone filling the audience inputs well can set the "what good looks like" bar faster than copy. Full disclosure, I work on Nereo (https://www.nereo.io/), an AI video tool, so I'm biased toward video, but onboarding inputs seem like a place where showing beats telling. Congrats on the Candor launch.
Wow Ben, this is a rich post. Like a walk through my career memory lane. Congrats on the launch! The "happy path illusion" is the one I keep relearning building agentic apps. The gap between "the demo works" and "a stranger can actually get through it unassisted" is brutal. AI makes it dangerously easy to keep polishing the demo instead of closing that gap.