Aakash Sehrawat
Go back

I Ship LLM Systems for a Living. I Can't Derive Backprop.

I’ve shipped retrieval systems that answer questions over nearly 300,000 documents.

I also couldn’t compute a gradient if you asked me to. Not “I’d need an hour to refresh” — I mean I don’t know how backpropagation works.

Both of those things are true, and the second one has started to bother me.

So why does it bother me?

Because I’m proud of what I’ve built, not embarrassed by it. Those systems run reliably in production, used daily by thousands of people at a large enterprise client — real work, with real consequences when it breaks.

But building them is exactly what made me want to understand them even more. Two years of shipping AI has left me more curious about this field than when I started, not less. And curiosity that never converts into understanding just becomes a ceiling.

The gap

There are two kinds of people building with AI right now.

The first kind composes. We pick an embedding model because a benchmark said so, wire up a vector store, add a reranker when recall is bad, swap in a bigger model when quality is bad, and ship. It works. I’ve been doing it for two years and the systems I’ve built are used every day.

The second kind understands what they’re composing. When retrieval plateaus, they reason about why this embedding model collapses distinctions that matter in this corpus. When someone asks “should we fine-tune?”, they don’t have an opinion — they have an experiment. When inference costs too much, they know which knob moves what, because they’ve measured it.

Here’s the contrast, in code. This is roughly what I write on a normal Tuesday:

# Four lines. Every one is a decision I made by reading someone else's benchmark.
retriever = vectorstore.as_retriever(search_kwargs={"k": 20})
docs = retriever.invoke(query)
reranked = reranker.rerank(query, docs, top_n=10)
answer = llm.invoke(prompt.format(context=reranked, question=query))

And this is what I want to be able to write from memory, and defend line by line:

# Also four lines. I didn't write this one — I copied it in to make the point.
def backward(self, dout):
    dx = dout @ self.W.T
    self.dW = self.x.T @ dout
    self.db = dout.sum(axis=0)
    return dx

I want to be the second kind of engineer — not for a title, but because the first kind has a ceiling and I’ve hit it. Concretely, hitting it looks like this:

Every one of those is the gap between using a thing and knowing it. Under production pressure, that gap is where you get found out.

Why do this publicly

Three reasons, in order of how much I believe them.

1. A private plan costs nothing to abandon. I’ve written versions of this plan before. They lived in a notes app and died there, and nobody noticed — including me. This one has witnesses.

2. Writing is the compression test. You cannot write 2,000 clear words about something you only half-understand: the holes surface as vagueness, and you feel them while typing. Publishing closes the loop, because someone who knows more than me gets to say I’m wrong. That correction is cheap here. It’s expensive in production.

3. Invisible work doesn’t compound. Four years of shipping, two of them on AI systems, and almost none of it is public. That’s a mistake I’d like to stop making.

There’s a fourth, less noble reason: I want to be findable by people doing this work. Writing is the cheapest way I know to do that.

Where I actually am

The honest inventory. What I can do today, with production systems behind it:

What I can’t do, stated plainly:

That last one is the reason for this post.

The plan

Twenty-two months. Fifteen to twenty-five hours a week, alongside a full-time job — which is the real constraint, and the reason the timeline is long rather than heroic. Three phases:

Phase 1 — FOUNDATIONS          Phase 2 — BUILD & DEPTH        Phase 3 — SPECIALIZE
months 1-5                     months 6-12                    months 13-22

math -> classical ML           3 flagship projects            preference tuning
DL internals                   fine-tuning + evals            inference economics
transformer from scratch       MCP + multimodal               OSS + paper reproduction

gate: can I defend             gate: is any of this           gate: am I known for
every line of it?              actually good?                 one specific thing?

Phase 1 — Foundations (months 1–5). Linear algebra, calculus, probability — enough to read papers and reason about training, not a math degree. Classical ML implemented from scratch in NumPy. Then Karpathy’s sequence: micrograd, makemore, nanoGPT, and if I get there, nanochat. The bar isn’t “I watched the videos.” The bar is a transformer I wrote whose every line I can defend.

Phase 2 — Build and depth (months 6–12). Three flagship projects, each public, deployed, and measured: a RAG system at a scale that breaks the naive design, an agent platform with trajectory evals and real users, and one differentiator I haven’t chosen yet. Alongside them: a real fine-tune with measured lift, MCP servers, and an eval harness running in CI.

Phase 3 — Specialize (months 13–22). Preference tuning hands-on, inference economics I’ve benchmarked myself, open-source contributions, and a paper reproduction where I document honestly what didn’t reproduce.

Running underneath all of it are six disciplines I think a senior engineer owes their team, whether or not anyone ever interviews them on it: algorithms, low-level design, system design (including ML and LLM system design, where my day job and my study meet), CS fundamentals, AI depth, and the ability to explain any of it to someone who doesn’t share my context. Not all six at once — one or two in active focus at a time, the rest ticking over.

What I’ll ship

Roughly monthly, for twenty-two months:

The rule I’m holding myself to: every month ships something public. Courses are inputs; artifacts are output. A month that ends with no artifact failed, regardless of how much I felt I learned.

Honest outcome distribution

I’ve read enough posts like this to know how they usually end: a confident timeline, then silence around month four. So here’s my real estimate.

Most likely (~50%): I finish the substance, late. Phases slip, the flagship projects take longer than planned, and a month or two goes to life instead of this. I end up with the depth and most of the artifacts, on a longer clock. This is the median outcome, and it’s a good one.

Likely (~30%): it runs roughly as designed. Not through discipline, but because the plan is built to survive bad months — one active focus at a time, extend rather than abandon, and a hard rule that sleep and training are not raidable for study hours.

Possible (~20%): it partly stalls. If it does, I know exactly how: the tutorial trap. Eight finished courses, one shipped project. I know because that’s how I’ve approached learning for most of my career — plenty consumed, almost nothing built from it.

What I won’t do is quietly stop and let the archive imply I finished.

Monthly check-ins

At the end of each month: a short post covering what shipped, what I got wrong, and what’s next — including the months where the answer is “less than I planned.” Those are more useful to read than the wins, and considerably more useful to write.

If you’re doing this work — especially if you’ve gone deep on evals, retrieval at scale, or inference economics — I’d like to hear where I’m wrong. That’s the whole reason to do this here rather than in a private notebook.

Month one starts now. The first post on the math is next.

Share this post:
Share this post on WhatsappShare this post on FacebookShare this post on XShare this post on TelegramShare this post on PinterestShare this post via email
Previous Post
Text-to-SQL: Easy to Build, Hard to Trust