---
title: "Thinking Fast and Slow"
url: "https://www.morphllm.com/blog/thinking-fast-and-slow"
description: "There are two kinds of inference: someone is waiting, or nobody is watching. The alpha lives at the ends. The open stack serves the middle."
date: "2026-07-21"
author: "Tejas Bhakta"
---
# Thinking Fast and Slow

Every inference request in the world is one of two kinds, and the model has nothing to do with which. The question is whether a human is waiting for the answer.

Plot productivity against how long you let an agent run and you get a U:

![The agent world is diverging: experienced productivity against agent autonomy, high at 100ms and at 10 hours, with a trough around one minute](/images/blog/agent-world-diverging.svg)

The left end is deep work: instant tools that keep you in flow on the hardest problems. The right end is parallel background agents you check on tomorrow. The trough around one minute is the worst spot in the product space. Not enough to delegate, not fun to wait.

The two ends want opposite things from the serving stack.

## Fast: someone is waiting

Speed is the product here, and the numbers are less forgiving than most people assume:

- The probability a developer breaks flow rises about 10% per second of waiting.
- In customer cohorts that never hit an error, conversion doubles when speed doubles.

People don't leave because the code was wrong. They leave because the cursor blinked too long. The levers are latency and cache. Mostly cache:

- Programming traffic shares 97% of its prefix tokens: same system prompt, same repo map, same files, with a few new lines at the bottom.
- Serve that naively and you pay to re-read a codebase that hasn't changed since the last keystroke.
- A cache-aware router takes prefix hit rate from 20% to 75%.
- We carry the hot cache across machines, so a request can land anywhere in the fleet and still find its prefix. A cache that lives and dies with one GPU is a cache that mostly misses.

## Slow: nobody is watching

Agents crossed the line where you can leave them alone: nightly crons, batch refactors, software that rewrites itself while the team sleeps. Nobody sees these run, which flips every priority:

- Latency is worth nothing. A million tokens at 3am has one metric, dollars.
- The open kernels leave 2 to 3x of margin and capacity on the table in this regime, because they were tuned for neither end. They were tuned for the middle.

Side by side, the two regimes barely share a spec sheet:

|  | Fast | Slow |
|---|---|---|
| Who's waiting | A human, in flow | Nobody |
| Lives at | 100ms to 10s | 1h to 10h |
| What wins | Latency and cache | Price per token |
| The mechanism | 97% shared prefixes, 75% hit rate | Batch density, 2 to 3x reclaimed |

## The middle is not a business

The trough of the U is a person chatting with a model at 30 to 60 tokens per second. vLLM and SGLang serve it well. It's a solved regime, which is exactly why there's no alpha in it, and it's where nearly everyone chooses to compete.

Kahneman split thinking into a fast system that reacts and a slow one that deliberates. Inference is splitting the same way. The fast system keeps a human in flow, and sub-second latency on a hot cache is what flow costs. The slow system grinds through the night, and price per token is all it answers to. [Morph](/products/models) builds the serving stack for both. We write the kernels for the two ends. The open stack writes them for the middle.

The ends are where the workload is going. The middle is just where the benchmarks are.