The gap between "works" and "launchable" is so real. On the garbage-in problem in the first step: besides text examples, a short clip showing someone filling the audience inputs well can set the "what good looks like" bar faster than copy. Full disclosure, I work on Nereo (https://www.nereo.io/), an AI video tool, so I'm biased toward video, but onboarding inputs seem like a place where showing beats telling. Congrats on the Candor launch.
Wow Ben, this is a rich post. Like a walk through my career memory lane. Congrats on the launch! The "happy path illusion" is the one I keep relearning building agentic apps. The gap between "the demo works" and "a stranger can actually get through it unassisted" is brutal. AI makes it dangerously easy to keep polishing the demo instead of closing that gap.
Thanks Brent! And agreed...the gap between the demo and production-grade software is real. People have to quickly learn the importance of product management and best practices with respect to development. You need tests built-in. You need security checks. Etc.
Getting real user feedback seems way more useful than polishing the product again and again based on your own thinking. I also liked how you kept collecting information from different parts of the user experience instead of looking at just one thing. The part about weak inputs still giving believable but bad results was interesting too, because with AI it’s not enough to see if something works, you also have to check if the result is actually good. I am curious, did this change how you think about testing AI products?
Thanks Vikram. Assessing the results is tough. It's what evals are for. We're spot checking today, and also engaging users/customers to get their feedback, since they know their users/customers and their research better than we do.
The real test is if people take action on any of the research synthesis that Candor provides them.
The gap between "works" and "launchable" is so real. On the garbage-in problem in the first step: besides text examples, a short clip showing someone filling the audience inputs well can set the "what good looks like" bar faster than copy. Full disclosure, I work on Nereo (https://www.nereo.io/), an AI video tool, so I'm biased toward video, but onboarding inputs seem like a place where showing beats telling. Congrats on the Candor launch.
Wow Ben, this is a rich post. Like a walk through my career memory lane. Congrats on the launch! The "happy path illusion" is the one I keep relearning building agentic apps. The gap between "the demo works" and "a stranger can actually get through it unassisted" is brutal. AI makes it dangerously easy to keep polishing the demo instead of closing that gap.
Thanks Brent! And agreed...the gap between the demo and production-grade software is real. People have to quickly learn the importance of product management and best practices with respect to development. You need tests built-in. You need security checks. Etc.
Indeed!
Getting real user feedback seems way more useful than polishing the product again and again based on your own thinking. I also liked how you kept collecting information from different parts of the user experience instead of looking at just one thing. The part about weak inputs still giving believable but bad results was interesting too, because with AI it’s not enough to see if something works, you also have to check if the result is actually good. I am curious, did this change how you think about testing AI products?
Thanks Vikram. Assessing the results is tough. It's what evals are for. We're spot checking today, and also engaging users/customers to get their feedback, since they know their users/customers and their research better than we do.
The real test is if people take action on any of the research synthesis that Candor provides them.