50+ Versions of a System Prompt: What It Actually Takes to Build an AI That Personalizes Well
Most cold emails get ignored because they sound like every other cold email. Here are the three things every reply-worthy email does differently.
50+ Versions of a System Prompt: What It Actually Takes to Build an AI That Personalizes Well
When we started building Coldrply, I assumed the hard part would be the engineering — wiring up Supabase, getting Nylas to talk to Gmail and Outlook, handling OAuth without losing my mind.
I was wrong. The hard part was getting the AI to write emails that didn't sound like AI wrote them.
The problem with version 1
Our first system prompt did what you'd expect: take a LinkedIn bio, pull a detail, drop it into a template. Technically personalized. Practically ignorable.
It read like every "I saw you're the VP of Sales at [Company]" email that's trained everyone to stop reading past the first line. We'd built a slightly smarter mail-merge tool, not a personalization engine.
Why "good enough" wasn't good enough
Coldrply's entire premise rests on Sam McKenna's SMYKM framework — Show Me You Know Me. The idea is simple: reference something specific enough that a prospect can tell a human actually looked at their profile, even when a human didn't.
That's a high bar. It means the AI has to distinguish between a detail that's interesting (a podcast appearance, a recent post, an award) and a detail that's just available (job title, company name, years of experience). Version 1 couldn't tell the difference. Neither could dozens of versions after it.
What actually changed along the way
A few shifts made the difference:
- We stopped grading on vibes. Early on, "does this sound good?" was a gut check. We built a real scoring rubric — specificity, relevance, natural phrasing, absence of generic openers — and started scoring every output against it.
- We tested against real bios, not clean examples. Sample bios are too easy. Real LinkedIn profiles are messy, sparse, or stuffed with buzzwords. The prompt had to hold up against the bios that gave it nothing to work with, not just the ones that gave it plenty.
- We built in a bail-out. If a bio genuinely didn't have enough signal, the earlier versions would still force a personalization — badly. Later versions learned to flag when a bio was too thin, rather than fabricating relevance. That became its own feature (a bio quality indicator).
- We optimized for restraint, not cleverness. Some of the worst outputs weren't generic — they were trying too hard. Forced humor, over-familiar tone, name-dropping for the sake of it. Good personalization reads like something a busy person wrote in 30 seconds after actually reading a profile, not like a machine showing off.
By the 50-somethingth version, we had a consistent 9.5/10 across our test set. That's the version that shipped — 50-plus rewrites later.
The part nobody talks about
Founders love to talk about launch day, growth hacks, and revenue milestones. Almost nobody talks about the 40-plus versions that didn't work, or the afternoons spent staring at an email output thinking "this is technically personalized and somehow still bad."
If you're building anything AI-powered, here's the honest breakdown: the model is maybe 20% of the product. The other 80% is prompt engineering, edge-case hunting, and the willingness to throw away a version that's "fine" because fine isn't the bar.
Coldrply exists because we didn't stop at version 1 — or version 10, or version 30. If your cold emails still sound like version 1, that's exactly the problem we built this to solve.