98.9%
field accuracy
0
invented values, in 0 of 50 conversations
78%
conversations finished
10.8
questions on average, vs a 25-field form
$0.0040
agent cost per conversation (+$0.0013 for the simulated customer)
What the numbers mean
- Correct: matches the owner's true fact, or left empty because the owner didn't know it.
- Invented: a field filled that the owner never could have stated: a fact they didn't know, or a question that doesn't apply to their business. In insurance a made-up answer can void a policy, so this matters more than accuracy.
- Wrong / missed: the owner knew it, and the agent recorded something else / nothing.
- Accuracy counts every field that applies to the business, except the free-text description. Simulated customers are easier than real people and occasionally state facts they were told they don't know, so the transcripts are published below to be read.
Sample conversations
- #8 · roofing, SC, rambling 100% · 4 questions
- #1 · daycare, ID, rambling 100% · 3 questions
- #2 · restaurant, IL, typos 100% · 4 questions
Accuracy by field
| Field | Correct | Accuracy |
|---|---|---|
| Deep fryers | 12/13 | 92% |
| Offers delivery | 12/13 | 92% |
| Seats | 12/13 | 92% |
| Serves alcohol | 12/13 | 92% |
| Annual revenue | 49/50 | 98% |
| Industry | 49/50 | 98% |
| Alcohol share of sales (%) | 6/6 | 100% |
| Business name | 50/50 | 100% |
| Pool on premises | 13/13 | 100% |
| Licensed capacity (children) | 13/13 | 100% |
| Employees | 50/50 | 100% |
| Business vehicles | 50/50 | 100% |
| Claims in last 3 years | 50/50 | 100% |
| Tallest roof worked on (ft) | 12/12 | 100% |
| Residential or commercial | 12/12 | 100% |
| State | 50/50 | 100% |
| State licensed | 13/13 | 100% |
| Work subcontracted (%) | 6/6 | 100% |
| Transports children | 13/13 | 100% |
| Uses subcontractors | 12/12 | 100% |
| Torch-down / hot work | 12/12 | 100% |
| Years in business | 50/50 | 100% |
By industry
| daycare | 100.0% |
| other | 100.0% |
| restaurant | 96.9% |
| roofing | 99.3% |
By speaking style
| changes answer | 100.0% |
| many facts | 93.5% |
| rambling | 100.0% |
| state names | 100.0% |
| terse | 100.0% |
| typos | 100.0% |
| vague numbers | 98.7% |
Every conversation in run 2
| # | Industry | Style | Accuracy | Questions | Finished | Problems |
|---|---|---|---|---|---|---|
| 1 | daycare | rambling | 100% | 3 | yes | |
| 2 | restaurant | typos | 100% | 4 | yes | |
| 3 | other | many facts | 100% | 25 | no | |
| 4 | roofing | vague numbers | 92% | 5 | yes | wrong: Annual revenue |
| 5 | daycare | state names | 100% | 10 | yes | |
| 6 | restaurant | changes answer | 100% | 9 | yes | |
| 7 | other | terse | 100% | 23 | yes | |
| 8 | roofing | rambling | 100% | 4 | yes | |
| 9 | daycare | typos | 100% | 8 | yes | |
| 10 | restaurant | many facts | 100% | 3 | yes | |
| 11 | other | vague numbers | 100% | 25 | no | |
| 12 | roofing | state names | 100% | 9 | yes | |
| 13 | daycare | changes answer | 100% | 10 | yes | |
| 14 | restaurant | terse | 100% | 8 | yes | |
| 15 | other | rambling | 100% | 4 | yes | |
| 16 | roofing | typos | 100% | 3 | yes | |
| 17 | daycare | many facts | 100% | 3 | yes | |
| 18 | restaurant | vague numbers | 100% | 11 | yes | |
| 19 | other | state names | 100% | 25 | no | |
| 20 | roofing | changes answer | 100% | 9 | yes | |
| 21 | daycare | terse | 100% | 6 | yes | |
| 22 | restaurant | rambling | 100% | 3 | yes | |
| 23 | other | typos | 100% | 2 | yes | |
| 24 | roofing | many facts | 100% | 3 | yes | |
| 25 | daycare | vague numbers | 100% | 8 | yes | |
| 26 | restaurant | state names | 100% | 25 | no | |
| 27 | other | changes answer | 100% | 25 | no | |
| 28 | roofing | terse | 100% | 9 | yes | |
| 29 | daycare | rambling | 100% | 10 | yes | |
| 30 | restaurant | typos | 100% | 5 | yes | |
| 31 | other | many facts | 100% | 25 | no | |
| 32 | roofing | vague numbers | 100% | 9 | yes | |
| 33 | daycare | state names | 100% | 25 | no | |
| 34 | restaurant | changes answer | 100% | 4 | yes | |
| 35 | other | terse | 100% | 3 | yes | |
| 36 | roofing | rambling | 100% | 8 | yes | |
| 37 | daycare | typos | 100% | 7 | yes | |
| 38 | restaurant | many facts | 58% | 25 | no | wrong: Industry missed: Seats, Serves alcohol, Deep fryers, Offers delivery |
| 39 | other | vague numbers | 100% | 5 | yes | |
| 40 | roofing | state names | 100% | 25 | no | |
| 41 | daycare | changes answer | 100% | 7 | yes | |
| 42 | restaurant | terse | 100% | 9 | yes | |
| 43 | other | rambling | 100% | 25 | no | |
| 44 | roofing | typos | 100% | 4 | yes | |
| 45 | daycare | many facts | 100% | 7 | yes | |
| 46 | restaurant | vague numbers | 100% | 7 | yes | |
| 47 | other | state names | 100% | 25 | no | |
| 48 | roofing | changes answer | 100% | 7 | yes | |
| 49 | daycare | terse | 100% | 10 | yes | |
| 50 | restaurant | rambling | 100% | 4 | yes |
Run history
| Run | Date | Note | Personas | Accuracy | Invented | Finished | Avg questions |
|---|---|---|---|---|---|---|---|
| 4 | 2026-11-08 | repeat of run 3, same code, to measure run-to-run variance | 50 | 99.3% | 0 | 90% | 7.5 |
| 3 | 2026-11-08 | after summary fixes (state names, hide industry other, classification hint) | 50 | 99.8% | 1 | 98% | 6.8 |
| 2 | 2026-11-08 | first full run | 50 | 98.9% | 0 | 78% | 10.8 |
| 1 | 2026-11-08 | smoke test | 3 | 100.0% | 0 | 100% | 5.0 |