Build the whole product in six layers.How anyone builds a scalable, secure, and governed product.
Every serious product is built in the same six layers. This is the full operating stack — what belongs in each layer, and what makes it scale, stay secure, and stay governed. Tap a layer to see what runs there; the reference tools, how I've built each in production, the operating rules, and how to measure it are underneath.
The six layers
- Start with the six layers. Tap any number to expand it and see what runs in that layer.
- Open the detail only if you want it. The full comparison, the operating rules, and the client harness are collapsed below.
- End at the bottom. "How to measure AI" is the thinking that ties it together.
1
2
3
4
5
6
The full breakdown
Reference stack vs. what I've shipped 35 layers
| Layer | Recommended | Codex / Claude Code | What I used | Evidence |
|---|---|---|---|---|
| Web interface | Next.js, React, Tailwind, shadcn/ui | Yes | React, Next.js, TS, Tailwind | Fundingo & SLP package.json |
| Admin interface | Retool → custom Next.js | Yes | Custom internal views | second-brain VantacaControlsView.jsx |
| Backend / API | Next.js API routes → NestJS | Yes | Vercel fns · CF Workers · edge fns | second-brain/api/*.js |
| Database | PostgreSQL / Supabase / RDS | Yes | Supabase Postgres; self-host on AWS | multiple projects; SLP self-host |
| Data access | Prisma ORM | Yes | Prisma (SQLite) + Supabase client | Fundingo package.json |
| Tenant isolation | Postgres RLS + policies | Yes | RLS + mandatory isolation test | SLP CLAUDE.md §4; insert-only RLS |
| Permissions | WorkOS RBAC → OpenFGA | Yes | RLS owner_id policies | second-brain schema.sql; SLP auth.uid() |
| Authentication | WorkOS AuthKit | Yes | Supabase Auth · middleware gate | trip _middleware.js; SLP in-mem JWT |
| AI models | OpenAI → Anthropic & Bedrock | Yes | Anthropic API + AWS Bedrock | api/claude.js; SLP voice server.js |
| Model gateway | LiteLLM → custom | Yes | Custom server-side proxy | api/claude.js (server-side key) |
| AI guardrails | OpenAI guardrails + Zod | Yes | Zod + HIPAA policy checks | Fundingo zod; SLP CLAUDE.md §0 |
| Agent framework | OpenAI Agents / Vercel AI SDK | Yes | Retell · orchestrator · MCP | api/orchestrator-run.js; ~/vantaca-mcp |
| AI evaluations | Langfuse + OpenAI Evals | Partly | Retell + n8n eval flows | attested — Retell evals; n8n scoring |
| Prompt / version mgmt | Langfuse | Yes | Salesforce Prompt Builder | attested — Agentforce; Agentblazer Legend |
| Business workflows | Temporal Cloud | Yes | n8n (not Temporal) | attested — Vantaca WF (200); SMS spec |
| Background jobs | Inngest / Trigger.dev | Yes | pg_cron · edge fns · sweep jobs | api/sweep.js; rc-scheduler-tick + pg_cron |
| Search | Postgres FTS → OpenSearch | Yes | Second Brain · Google Drive | connector: search_all; Drive search_files |
| Files / documents | AWS S3 | Yes | Google Drive · Second Brain | connector: Drive; second-brain upload_file |
| Payments | Stripe | Yes | Stripe Buy Button | attested — SOFA JCC buy_btn · pk_live (Webflow) |
| External integrations | Custom adapters; Nango | Yes | Vantaca · Salesforce · Apollo · RC | ~/vantaca-mcp (73 endpoints); 3pl-salesforce |
| Logs & infra monitoring | Datadog / CloudWatch | Yes | Datadog (agent + proxy) | api/datadog.js; agent v7.80.2 on droplet |
| Application monitoring | Sentry | Yes | Datadog (Sentry excluded, PHI) | DD monitors on app logs; SLP §1 |
| Product analytics | PostHog | Yes | Retell · n8n analytics | attested — Retell + n8n run analytics |
| Audit logs | WorkOS Audit / append-only PG | Yes | agentlogs (app-level) | api/sweep.js + orchestrator-run.js |
| Testing | Vitest, Playwright, Pact, k6 | Yes | Vitest + Playwright | Fundingo & SLP package.json |
| Deployment | GitHub Actions + Vercel | Yes | GitHub Actions · Vercel · Cloudflare | SLP CI green; git-based deploy |
| Feature flags | PostHog / LaunchDarkly | Yes | Manual master-switch | RCControlsView.jsx; rc_scheduler switch |
| Secrets | AWS Secrets Manager | Yes | Vercel env / .env (server-side) | process.env in api/claude.js, datadog.js |
| Infrastructure | AWS ECS/Fargate, RDS, S3, CF, WAF | Yes | DigitalOcean · Vercel · CF · AWS Supabase | droplet 134.209.126.217; no ECS |
| Infrastructure-as-code | Terraform | Yes | docker-compose · Caddyfile | SLP infra/ compose + Caddyfile + setup.sh |
| Edge protection | AWS WAF / Cloudflare | Yes | Cloudflare (tunnel · Pages · WAF) | vantaca.aventary.com tunnel; CF Pages |
| Compliance management | Vanta / Drata | Partly | Manual HIPAA hardening | SLP CLAUDE.md §0/§10 + RUNBOOK |
| Security scanning | GH Advanced Security, Semgrep | Yes | Custom hardening checks | SLP scripts/check-hardening.sh |
| Vector / RAG storage | pgvector | Yes | Not confirmed | ai_memories exists; embeddings unverified |
| Events and queues | AWS EventBridge / SQS | Yes | Not used | — |
My operating rules 8
How the stack is actually run. personal-ai-hype-cycle ↗
- 1No single AI existsRoute each task to the model best at it. Many by design.
- 2Wrap legacy systemsExtend what already runs the business; don't rip it out.
- 3Build verification loopsAI hallucinates. Use AI to audit AI, and gate on the check.
- 4Interfaces are temporaryOptimize the workflow underneath, not the dashboard on top.
- 5Don't build everythingBuy the 80%; spend the build budget on the last 20%.
- 6Create external memoryKnowledge the AI reaches across sessions — the OS underneath.
- 7Close feedback loopsAutomate the check so it runs before output reaches a user.
- 8Make learning intentionalCompound skill across tools so each project speeds the next.
The client harness — 8 questions 8
The tool is commodity; the harness is the advantage. Ask these before buying and every quarter after. the-harness-audit ↗
- 1AccountabilityOne named owner answerable for its performance.
- 2RiskSafeguards before AI touches a customer or a dollar.
- 3ProofTransparency on what it did and how well.
- 4PaybackQuantify the work actually removed from a human.
- 5CostKill redundant tools; consolidate the stack.
- 6AdvantageFeed proprietary data so output can't be copied.
- 7Quality at scaleVerification keeps pace with output velocity.
- 8CapabilityDelegate whole tasks, not endless prompt tweaks.
Measuring in a world of AI
Why it matters: a good answer and a finished job are two different things. An agent can write a perfect summary and still route to the wrong team, drop the handoff, or mark a call "resolved" while the caller is already dialing back. Without measurement you can't tell resolved from worsened — and you can't improve what you can't see. The next teams won't win on prompts; they'll win on loops.
- AI describes — raw labels: intent, sentiment, status.
- Workflow judges — scores those labels against your rules.
- Systems prove — tool calls, records, timestamps, not claims.
- Humans calibrate — a gold set defines "good" before judges scale.
One score hides the failures that matter. Keep the dimensions visible — a clean 92 can bury an agent that understood the intent and still did the wrong thing.
What you measure — different at every altitude:
Did the machine do what it said?
Deterministic system facts.Cheap, boring checks you run on everything: did the tool call fire, was the route confirmed or only guessed, is it up, is it fast. Facts the system should just know.
Is the AI itself any good?
AI has its own tests.Evals against a human-set rubric, retrieval grounding (does the context match), judge-vs-human agreement by failure type — not average score. Grade it before you scale it.
Did the work actually get done?
Test the outcome, not the text.The only bar that pays: did the AI resolve, shorten, or worsen the work? First-contact vs. repeat-contact, tied to the P&L. A dispatcher answers it in five seconds.
A scoreboard tells you that you lost; a learning system tells you why, which version caused it, whether it repeated, and when to roll it back. Read the friction and each point becomes the next item on the backlog. the-metrics-became-the-product-backlog ↗