Building a RAG pipeline that actually holds up in production

Modern software teams are rethinking how they ship. The old playbook — ticket, review, deploy on Friday, pray — is being replaced by continuous delivery pipelines that treat production as the default environment.

Why this matters now

The last twelve months have compressed a decade of tooling change into a handful of quarters. LLM-assisted refactors, remote-first workflows, and platform teams that ship paved roads have made small teams meaningfully more productive.

"The best teams I work with are shipping ten times more code and worrying about it half as much." — a staff engineer at a Series C startup

What we tested

We spent three weeks with early access to the new tooling, deploying real workloads and measuring:

• Cold-start latency under bursty traffic
• Cost per million requests at steady state
• Developer time-to-first-deploy on a fresh laptop

ts
export async function deploy(app: App) {
  const build = await bundle(app);
  await push(build);
  return waitForHealthy(app.id);
}

The numbers were striking. Cold starts dropped 62% versus the previous generation, and the median engineer had their first deploy live in under nine minutes.

The catch

Nothing is free. The new model assumes stateless workloads, opinionated observability, and a willingness to trust the platform. Teams still running long-lived VMs will need to plan a migration — this is a different mental model, not a drop-in replacement.

For now, the trajectory is clear: infrastructure keeps getting less interesting, and that is exactly the point.

“The best teams treat their platform like a product, not a project.”

Maya Okafor

AI Editor

Former ML research engineer. Writes about frontier models, evals, and the messy reality of shipping AI.