Case Study
Orça Comigo — A WhatsApp AI Agent for Household Finance
A solo, year-long build: turning a voice note or a photo of a receipt into a shared household budget, without ever leaving WhatsApp.
Completed — product discontinued. The engineering is in production use today as the base of our WhatsApp agent stack.
Technologies: WhatsApp Business Cloud API · Speech-to-text · OCR · LLM function calling
Context
Personal finance apps have a retention problem that has nothing to do with design: every transaction has to be typed in by the user. Orça Comigo was a one-year R&D bet on a simple question — if the interface disappears into WhatsApp, and the input is a three-second voice note, does the habit survive? We built the product solo, end to end, from architecture to landing page.
What We Built
Conversational capture. A voice note ("gastei 50 no mercado"), a text message, or a photo of a receipt — no app, no login, no form.
Structured extraction. Transcription, then an LLM with function calling returning a typed record: amount, category, date, payer, shared vs. personal.
Shared state. Both partners write to the same budget and see the same balance, with the other partner notified automatically.
Natural-language queries. "How much is left this month?" answered against the couple's live data, not a generic model answer.
A web dashboard for the month view and budget setup, plus subscription billing.
Architecture
At a high level, the system moved through six stages:
An inbound webhook from the WhatsApp Business Cloud API, decoupled from processing through an async queue.
A media pipeline: audio download to speech-to-text; image to OCR to text normalization.
An extraction layer: an LLM call against a strict function/tool schema, with validation and fallback prompts for ambiguous values.
A domain layer with multi-tenant isolation per couple, and idempotency on message IDs to survive WhatsApp retries.
A notification layer sending template messages back to the partner, respecting WhatsApp's 24-hour service window.
A web app for the dashboard, authentication, and subscription billing.
Cloud infrastructure with containerized deployment and CI/CD.
Engineering Decisions Worth Stealing
Idempotency first. The WhatsApp webhook retries — without deduplication by message ID, a couple sees the same expense twice and stops trusting the numbers on day one.
Typed extraction over free-form parsing. A tool schema with validation turns an LLM into a predictable component; free-text parsing does not survive real users.
The 24-hour window shapes the product. Anything proactive is a template message with a cost attached, so "send a reminder" is a business decision, not a feature toggle.
Latency is the product. Past a few seconds in a chat interface, users assume it failed and send again — so the whole pipeline, from voice note to confirmed record, was designed around staying under that line.
Cost per conversation is a design constraint, not an afterthought. Transcription plus extraction plus reply had to fit a tight per-interaction budget for the unit economics to make sense.
Results — And the Honest Part
The engineering worked. The business did not, and the reason is the interesting part for anyone building an AI assistant today:
Low friction per action is not the same as low friction overall. A voice note costs five seconds; remembering to send it, dozens of times a month, from two different people, is unpaid work.
Automatic data is the moat. Competitors that pull transactions through open banking turn the user from a typist into a reviewer — that's a product difference, not a technical detail.
Effort frequency has to match pain frequency. The pain arrives with the monthly statement; the product asked for daily work.
We shut the product down and kept the stack. What was built for couples is, with the domain layer swapped, the same thing a business needs to capture receipts from field teams, collect documents from clients, or run any structured task inside WhatsApp — the category Meta's platform rules explicitly still allow, unlike general-purpose assistants.
What This Means If You're Hiring Us
We have shipped a production WhatsApp AI agent, with the boring parts solved: Business API onboarding and templates, retries and idempotency, media pipelines, typed LLM extraction, multi-tenant data isolation, cost and latency control. And we will tell you when the idea itself is the weak part — we did it to our own product first.
The product lives on at orcacomigo.com