Case Study:
Secretary Agent.

Just Two Tools

Basic Workflow

1. Talk with Chat Claude → get a spec for "what to build"
2. Chat Claude turns that spec into an instruction sheet (TASK.md)
3. Hand the TASK.md to Claude Code in VS Code → it actually builds it
4. Report the result back to Chat Claude → get the next instruction sheet
5. Repeat steps 1–4 until it's complete

Taking Stock of Secretarial Work (a Starting Point for the Interview)

"I want a secretary" is easy to say, but putting into words what you actually want handed over is surprisingly hard. Based on what the role covers in practice, here's a starting point for the interview.

Scheduling

• Adding, changing, and cancelling events (say it out loud and it's on the calendar)
• Detecting conflicts and keeping realistic gaps that account for travel time
• Reading out the day's schedule at a set time — first thing in the morning, the night before
• Recurring events, and accounting for public holidays and days closed

Expense Reports

• Reading the date, amount, payee, and line items from photos of receipts and invoices
• Sorting into account and tax categories (on the assumption a human does the final check)
• Monthly tallies and exporting to the format you submit
• A reminder when unprocessed receipts are piling up

Email

• Classifying incoming mail (needs a reply / FYI only / no action) and setting priority
• Drafting replies (a human decides on sending, as a rule)
• Applying templates to routine inquiries
• Catching forgotten replies (flagging threads with no response after a set time)

Meeting Minutes from Audio

• Hand it a recording and it runs transcript, key points, and report as one continuous job — translated into the language you want if needed
• It can be built without adding a single API. Transcription reuses a local Whisper model, and summarizing and translation are both done by the agent's own LLM
• This pays off with meetings in another language: a discussion held in one language comes back as a report in another, with no "listen again, then translate" round trip

Cleaning up meeting notes, drafting application forms, and shortlisting flights and hotels can be covered too. Travel arrangements are not a required feature — if travel is rare in your line of work, leave it out from the start.

Why Split It Across Several Agents Instead of One

When people hear "secretary agent," they tend to picture a single do-everything agent. This case study recommends splitting by task instead.

Because the Judgment Each Task Needs Is Different

• Scheduling calls for the caution not to over-pack the calendar; email triage calls for the breadth not to miss anything important; expense work puts strict accuracy about numbers above everything else. Put all of that into one personality and the standard shifts from one instruction to the next

Because You Can Separate the Scope of Authority

• The email agent needs read access to your inbox; the expense agent doesn't. Split them and each gets only the minimum it needs. If something does misfire, the blast radius stays inside that agent's own territory

Because It's Easier to Isolate When Something Breaks

• When the expense tally looks wrong and the roles are split, you know immediately where to look. With everything in one agent, tracking down which instruction interfered takes time

Send Instructions by Voice, Get Reports by Voice

The heart of this case study is that your secretary keeps working during the hours you're away from your computer. To do that, you turn Discord into a doorway that carries audio in both directions.

Inbound (you → the agent)

• Audio you record with Discord's voice message feature gets transcribed by the agent and taken as an instruction
• Transcription runs on a local Whisper model, so the audio is never sent anywhere and there's no API usage fee. When the audio covers schedules, client names, and amounts, this is the setup to build on

Outbound (the agent → you)

• Reports come back as both audio and text — let the audio wash over you on the move, and follow the text when you want to check something later
• Which speech synthesis engine to use depends on the language, and on whether anything may leave your machine. The approach is not to commit to one engine but to listen to several and choose
• The criterion is that it must not sound mechanical. Amounts and dates in particular go straight to your work, so always test the formats you'll actually use

Overall Schedule (Six Phases)

What people want handed over differs completely from person to person, so there's no ready-made TASK.md that lets you skip the interview. Always start from the Phase 0 requirements interview.

Start Now: the Phase 0 Kickoff Prompt

Just paste this prompt as-is into a new conversation with "Chat Claude" (claude.ai) to kick off the Phase 0 requirements interview.

Disclaimer and Important Notes