pacod-ap

Why it exists

pacod-ap is built for agents whose job is to give correct, sourced answers in one domain — where a wrong price, rate or due date is worse than no answer. Three rules shape it:

• Every number comes from a structured source of truth, never the model's memory.
• Every claim cites a source — a knowledge-base passage, a fact's source_url, or an allow-listed web page.
• Admins correct an agent without a redeploy — models, prompts, directives, web search and caching apply on the next turn.

One deployment, many agents

An agent is a profile: one row in the profiles table plus everything scoped to it. Two ship with the platform:

• starter — a generic template: a small fictional help desk ("Nimbus Notes") that exercises the knowledge base and the reference-facts tools. Copy it to start your own agent.
• taxes — an advanced example for Bulgarian tax and accounting, with its own prompts, facts, law corpus and web-search allow-list (make seed-example PROFILE=taxes). Its corpus is there to show retrieval on a real body of law; it is not kept up to date, and nothing the agent says is legal or tax advice.

Isolation between agents is a security boundary: every per-agent row carries a profile_id, and each agent has its own Qdrant collections.

Users see only the agents assigned to them; admins work globally or scoped to one agent. Quotas count per user across all agents. Agents speak Bulgarian and English; a third language is a code change, not configuration.

The pieces

Every container has one name on the shared Docker network pacod-net: ap-* for pacod-ap's own services, pacod-* for the components it shares with the other pacod projects. The app's own chat is the only chat interface.

How a chat turn flows

The guards run before anything streams, so a rejection is a proper HTTP status, not a mid-stream error:

1. Auth — the pacod-id token is verified; an ended session or disabled account is refused.
2. Agent — the requested agent must be assigned to the user; a user who names none gets their only assignment, never a guess.
3. Language — detected per turn and checked against the agent's languages.
4. Quota — the gateway's per-user token limit, plus a request quota and throttle in Redis.
5. Topic — an illegal-phrase filter, then a cheap classifier: off-topic is turned away, an unclear question gets clarifying questions back.

Then the server-sent-event stream opens. A first question may be answered from the agent's semantic response cache; otherwise a small LangGraph graph runs:

• plan — the planner returns JSON: which tools to call, and whether the question is complex.
• execute tools — all planned calls run concurrently: kb_search, get_fact / list_facts, web_search.
• synthesize — the primary model streams the answer from that context, with extended thinking on complex turns, followed by suggested follow-ups.

Older turns are folded into a rolling summary, so a long chat doesn't resend its transcript. The OpenAI-compatible /v1 runs the same guards and agent (without the response cache), where each agent appears as a model. It is not routed through the edge; clients reach it on the internal network.

Knowledge, facts and the web

The planner picks a source by the kind of claim:

• Numbers, prices, rates, limits and dates come only from reference facts — typed rows in PostgreSQL, served over MCP. The code, not the model, sets the agent and reference year on each call, so the planner cannot reach another agent's facts.
• Explanations come from the knowledge base: dense search in Qdrant, a cross-encoder rerank, then small chunks expanded to their parent passage. A separate, down-weighted curated collection holds notes that admins wrote or approved.
• Recent information comes from web search, limited to a per-agent domain allow-list.

A fact has one generic shape, so a help desk's prices and a tax agent's due dates share a table — a new domain is data, not schema:

{"fact_key": "plan_price_monthly", "variant": "plus", "category": "pricing",
 "legal_year": 2026, "value_type": "money", "value_num": 6, "unit": "EUR",
 "label_en": "Plus plan — price per user per month",
 "source_url": "https://acme.example/pricing"}

An optional verifier, off by default, checks the answer's numbers against the tool results. Retrieval fine-tuning from user feedback is run by hand; nothing schedules it.

The improvement loop

Each agent improves from its own traffic. A worker reviews the agent's recent, redacted conversations daily or on demand. It judges each answer against what the assistant actually read — the verbatim context and the directives in force — so a verdict is not a guess.

Similar findings are grouped into clusters, and a cluster is the ticket a reviewer acts on. Nothing reaches the curated collection until an admin approves it, unless an auto-insert flag (off by default) is enabled and strict checks pass; a claim the sources contradict is never auto-inserted.

Security and isolation

• Identity is delegated. Accounts, passwords, sessions and roles live in pacod-id. The API follows pacod-id's feed, so an ended session or disabled account is refused within seconds. Account and role changes are made at pacod-id as the calling admin: only a superuser makes a superuser, and the last admin cannot be removed.
• Agents are fenced in code. The agent is authorized before any work, tool arguments that touch agent data are set by code, /v1/models lists only the caller's agents, and every per-agent cache key carries the agent's id.
• Secrets stay put. LLM provider keys live only in the gateway. Web-search keys are encrypted in the database. Secret files are generated, 0600 and gitignored, and internal shared tokens fail closed.
• Prompt safety. A refusal of illegal requests tops the planner and synthesizer prompts and outranks admin directives; admin and user text is always inserted as a value, never as a template.

Running and operating it

Three Docker Compose stacks start in order — make up-all does it:

1. The app — two compose files. The app file holds pacod-ap's own services, Redis, PgBouncer and the embeddings service; the standalone file holds its copies of the shared components: Caddy, PostgreSQL, Qdrant and pacod-id (and mailpit locally).
2. The gateway — pacod-gw's app file, from a checkout next to this one. It signs in through the same pacod-id, keeps its database in the same PostgreSQL and is served by the same Caddy.
3. Monitoring — Prometheus, Alertmanager and Grafana; optional, and started by make up-all when monitoring/.env exists.

Because the shared components sit in their own file, pacod-demo can load pacod-ap's app file next to the other pacod projects and run one of each shared component; the result runs at demo.pacod.io. Only one copy of each shared component can run on a host, so standalone pacod-ap does not run beside another pacod stack.

Locally the gateway answers with its mock providers: scripted replies, no cost.

Configuration reads a Redis cache, then an app_settings row keyed by agent, then the environment default; an outage falls back to the defaults instead of blocking a turn.

LLM calls are attributed to user, session, turn, agent and purpose, so the console's usage and cost pages read the gateway's reports rather than keeping a ledger. A timed-out call is never retried — the provider bills it anyway — so the user asks again.

Observability — an audit row per request, an opt-in per-step Debug Console, structured logs, and Prometheus metrics on every service, with Grafana dashboards for the app, infrastructure, web traffic, crawlers and usage, plus the gateway's own.

Backup — make backup writes one archive with the database dumps, Qdrant snapshots, config and secrets; make restore reverses it.

Build your own agent

There are two paths:

• In the console — a superuser creates an agent on the Profiles page or in the visual agent editor. A new agent starts in dev; you test it there and publish when it's ready. Versions are insert-only, so publishing and rolling back just move a pointer. An AI assistant drafts prompts and other text, but never edits the graph itself.
• As code — a ProfileSpec in pacod-api/scripts/profiles/<id>.py (topic, languages, prompt and setting overrides) and data in pacod-api/data/profiles/<id>/: reference facts, a corpus and a golden set of questions. make seed-example PROFILE=<id> installs it.

Then chat with it, check the Debug Console and the gateway's trace of a turn, and run make eval against the golden set. The improvement loop takes it from there.

Read more

• Runbook — stacks, secrets, real providers, backup, troubleshooting.
• The LLM gateway — attribution, retries, quotas, usage.
• Build your own agent — the full walkthrough.
• API architecture and UI architecture — the design in depth.