I've been an APM at Dashverse for about a year and a half. We build AI-powered analytics for short-form video creators. In that time I've shipped features that worked, features that flopped, and a few that taught me something I couldn't have read in a blog post. Here's what stuck.
The model is not the product
Every PM at an AI company learns this eventually. The model is a capability — what you build around it is the product. The UX, the defaults, the way you surface uncertainty, when you show results and when you hold them back. That's all product work, and it's where most AI features succeed or fail.
We spent weeks tuning prompts on a feature that users found confusing not because the output was wrong, but because we showed it in the wrong place at the wrong moment. The model wasn't the problem. The product was.
Latency is a feature
Not just a metric. When something takes 8 seconds to load, users don't sit patiently — they assume it's broken. We added a skeleton loader with a progress message and our perceived quality score went up even though nothing about the model changed. What you show while the AI is thinking matters as much as what it eventually says.
Prompt engineering is product design
The prompt is the spec. When you write one, you're making decisions about tone, format, scope, edge cases — the same decisions you'd make when writing a PRD. The difference is that the prompt is also the implementation. There's no handoff to an engineer. That compresses the feedback loop dramatically and rewards people who can think at both levels simultaneously.
Users don't know what AI can't do
They've seen the demos. They've used ChatGPT. They assume your AI feature can do everything theirs can. When it can't — when it hallucinates, misreads context, or just refuses — the trust drop is sharp.
The fix is explicit constraint-setting before the interaction, not apology after. Tell users what this feature does and doesn't do, upfront, in plain language. Set the box clearly and they'll work within it. Leave it open and they'll walk into walls.
Evaluation is the hardest unsolved problem
How do you know if your AI feature got better? Not in the lab — in production, for real users, on real inputs you've never seen. We're still figuring this out. The teams I've seen do this well treat eval like a product in itself: they build it deliberately, instrument it early, and revisit it whenever something breaks in unexpected ways.
What I'm still figuring out
How to communicate AI product changes to users without eroding trust. How to write roadmaps when the capabilities of your core dependency are changing every few months. How to decide when a feature is "good enough" when the ceiling keeps moving. These are genuinely hard, and I don't think the playbook exists yet. That's what makes this moment interesting.
← Back to Writing