Kylie, WhatsApp Agent

The problem

Most agent demos live in a web page nobody returns to. The interesting constraint was the opposite: put the agent where the conversation already happens, on WhatsApp, and make it behave like a contact rather than a chatbot. That means no app to install and no new interface, but it also means accepting whatever arrives, which on WhatsApp is text, voice notes and photographs rather than a clean text field. It also means the agent has to remember you between conversations, because a contact that forgets last week is not a contact.

The approach

Kylie is a FastAPI service behind the WhatsApp Cloud API webhook, with a LangGraph workflow as the brain. Incoming audio is transcribed and incoming images are described before they reach the graph, so every modality becomes text by the time routing happens. On the way out the reply can become a synthesised voice note or a generated image. Conversations are stored as embeddings in Qdrant, which gives both session memory and recall across sessions.

Technical decisions

1. 01

   Decision 1: Where the agent lives

   Chose

   Inside WhatsApp through the Meta Cloud API

   Considered

   • A web chat interface
   • A Telegram bot

   Because

   Distribution was the point. Someone can try it from a contact card without installing anything, and the constraints of a real messaging channel are what make the build interesting.

2. 02

   Decision 2: Orchestration

   Chose

   LangGraph with explicit state and routing

   Considered

   • A prompt chain
   • A single model call with tools

   Because

   Five different reply types (conversation, image, audio, tools, search) need different work. A graph makes the routing decision explicit and inspectable rather than hidden in a prompt.

3. 03

   Decision 3: Long-term memory

   Chose

   Qdrant with all-MiniLM-L6-v2 embeddings

   Considered

   • Stuffing the full history into the prompt
   • A relational store with keyword lookup

   Because

   Recall needs to work by meaning, not by keyword, and the prompt budget is finite. Semantic retrieval returns the few facts that matter for this message instead of everything ever said.

4. 04

   Decision 4: Multimodality

   Chose

   Normalise everything to text at the edges

   Considered

   • A multimodal model handling raw audio and images end to end

   Because

   One brain, one state shape. Transcribing and describing at the boundary means the graph never has to branch on modality, and each converter can be swapped independently.

Implementation

The pieces that carried over to the MCP edition, and the ones that did not:

• Carried over

  The LangGraph brain, the memory split, the multimodal edges, the Groq model choices and the character prompt all survived the rewrite more or less intact.

• Replaced

  The calendar tools, the web search client and the schedule lookup were hand-written and imported directly into the graph. In the MCP edition they live behind a protocol boundary instead.

What did not work

Results

Kylie reached 110 or more GitHub stars and 34 forks, and Qdrant found the project through a LinkedIn post and featured it on X and LinkedIn as a practical, end-to-end example of an agent with vector memory. Those are the numbers I have; message volume and latency were never instrumented in this version, which is itself part of why the next one starts with tracing.