The Art of Leaving the Other Party in the Dark

or: You Can’t Build Efficient Systems This Way

The Bare Request

Every API call is a small act of withholding.

You send GET /orders?status=open. The service on the other end does its job: it runs the query, serializes the rows, sends them back. Transaction complete. And it never learns the one thing that would have let it help you: why you asked.

It doesn’t know that for every order it just handed you, you’re about to come back and ask for the customer, then the line items – the same round trip, forty more times. It doesn’t know a copy from thirty seconds ago would have been fine. It doesn’t know whether you need this in the next 50 milliseconds or whether you’re a nightly batch job that won’t care if the answer shows up in an hour. It doesn’t know if this is the load-bearing call behind a user staring at a spinner, or a speculative prefetch you’ll throw away. It knows one thing: someone wants the open orders, and they want them now.

Everything is now. For no reason.

That’s the default shape of how we integrate systems, and we question it so rarely that “now” stopped looking like a choice. Request goes out, response comes back – present tense, full stop. But “now” is a demand we make by default, not because the work demands it. Most calls aren’t urgent. They just have no way to say so.

What the Request Leaves Out

Look at what a normal call leaves unsaid:

• When I need it. “Within two minutes” is wildly different from “right now.”
• How fresh it has to be. A slightly stale answer is often good enough.
• What comes next. If I’m rendering a list, I probably know I’ll need every item.
• How much it matters. Load-bearing and nice-to-have calls look identical on the wire.

None of that travels with the request. We strip the intent and send the bare imperative: give me this thing. Then we act surprised when the system underneath can’t be efficient.

A server kept in the dark has no room to be clever in the ways that actually help. It can’t batch what it didn’t know was coming, prefetch toward a future it can’t see, serve a cheap stale copy to someone who never said staleness was acceptable, or deprioritize work that never admitted it was optional. Each request is individually reasonable. Collectively, the system has almost nothing to optimize.

This is the part worth saying plainly: the silence isn’t simplicity. We call it simplicity. “It’s just a GET.” But a lot of what we praise as clean, minimal interface design is really information being withheld – sometimes from laziness, sometimes from a genuine belief that less is more, occasionally as a kind of craft. We’ve gotten good at saying as little as possible to the systems we depend on, and we treat that fluency as a skill. It’s the art of leaving the other party in the dark. And it has the texture of skill right up until you try to make the whole thing fast.

The proof that it’s a choice is that the efficient corners of our infrastructure are exactly the ones where the caller tells the truth.

Where Systems Already Get Smarter

SQL is the obvious case. You don’t tell the database how to retrieve your rows; you describe what you want and let the planner figure out the rest. Because you said what instead of how, the engine has room to be smart: indexes, join order, caching. Declarative beats imperative precisely because it hands the other side more intent, not less.

But the cleverness is sealed inside the query. SQL knows what you want from the data; it does not know what you want from the answer. The moment the results cross back over the boundary, the system goes dark again.

A few corners of our infrastructure do try to speak across that boundary. HTTP caching is one: Cache-Control, stale-while-revalidate – a vocabulary for exactly the freshness tolerance SQL never gets to hear. When callers use it, the whole system gets to relax. Most don’t. The vocabulary exists; we just don’t speak it.

Deadlines are another. gRPC deadlines, a context.Context that carries a timeout down the whole call chain – that’s how soon finally riding along with the request. Every hop can see the remaining budget and give up early instead of doing doomed work.

GraphQL deserves real credit here, because it took direct aim at the fan-out we opened with. The client can describe the shape of what it needs in one shot instead of discovering it one blind request at a time. But it still leaves the richer context outside the model: urgency, freshness, priority, and what the caller will do once the response arrives.

The pattern is consistent. Systems get efficient when callers say more – what, how soon, how fresh, what’s coming next – and stay slow when callers say nothing and demand everything immediately.

Intent Taped to the Envelope

And here’s the tell that ties it together: look at where we put the intent on the rare occasions we express it at all. Freshness goes in a cache header. Timing goes in a deadline. Priority, when it exists at all, rides in some metadata field on the envelope. We keep smuggling our needs into the transport – bolting them to the outside of the message, because the message itself has no room for them. The payload says here is the data; everything about what the data is for, how current it has to be, and what we’ll do with it next gets wedged into the protocol layer as an afterthought. We’ve been grading headers and deadlines as wins, but they’re really evidence of the defect: intent with nowhere to live, taped to the envelope.

A Standing Conversation

Which means the bare request was never the unit we wanted. The honest version of integration isn’t a sharper imperative, or even a single richer envelope – it’s a bidirectional stream. Two parties holding an open channel, each one continuously publishing what it has, what it wants, how soon, how stale a copy it will tolerate, what it is delivering and on what terms and for how long that holds – and each free to revise any of it as the situation moves. Not a request answered and forgotten, but a standing conversation, where either side can say I’ll need this shortly, or the copy you’re holding is still good, or never mind – statements about state and intent flowing both ways, in the content, where they can be reasoned about, instead of frozen into the shape of a URL and the headers wrapped around it.

The server isn’t just answering, either. It can stream assertions about its own process of fulfillment: accepted, planned, partial result available, this copy is 18 seconds old, waiting on an upstream, retrying, this branch failed, this answer is valid until conditions change. The caller can revise its needs while that is happening: loosen freshness, cancel speculative work, raise priority, accept a partial answer. Failure becomes an asserted state in the conversation instead of a terminal exception thrown across a boundary.

RDF, Reopened

Say that out loud and you’ve described something the field already built and mostly walked away from: a common model for making statements about resources – what they are, how they relate, what’s being asserted about them – carried in the content itself. RDF. Linked data, the semantic web, the whole earnest stack we decided wasn’t worth the trouble.

Depending on who you ask, RDF lost on plenty: ergonomics, tooling, aesthetics, timing. But the core idea wasn’t wrong. The deeper failure was unamplified need and the cost of ontological abundance. The need was real but never loud: one more bespoke integration always cost less at the margin than adopting a whole shared-meaning stack, so the pain stayed diffuse and the coordination never paid for itself. And the vocabularies it ran on were scarce – hand-built, brittle, expensive to align, with no cheap way to map one party’s ontology onto another’s. You can’t run a marketplace of statements when everyone has to agree on the dictionary first and the dictionaries barely exist.

Both of those are finally negotiable. The need is being amplified for us, as more of the traffic on every wire becomes systems and agents talking to systems and agents instead of people clicking through screens – and producing, extending, and translating between vocabularies, the part that used to take a standards committee and a decade, is getting cheap.

We may not need everyone to settle on one ontology anymore; we need something that can fluently bridge many, and for the first time that’s within reach. The problem RDF was aimed at never left. What changed is that the two reasons we couldn’t solve it are starting to give.

That is the part worth recovering. RDF’s promise was never the ceremony; it was the ability to make assertions ordinary enough that different parties could publish, consume, and revise them without agreeing on one private API shape.

In intentionally simplified RDF-ish form, that flow is not a stream of responses. It is a stream of claims both parties can subscribe to, revise, and reason over:

# caller: publishes need and tradeoffs
:caller-need-001
  a need:OpenOrders ;
  need:deadline "PT2M" ;
  need:acceptsStaleness "PT30S" .

# service: accepts and starts a plan
:service-plan-001
  a state:FulfillmentPlan ;
  state:respondsTo :caller-need-001 ;
  state:status state:Accepted .

# service: publishes conditional answer
:partial-answer-001
  a state:PartialAnswer ;
  state:answers :caller-need-001 ;
  state:includes :order-1847, :order-1852 ;
  state:freshness "PT18S" ;
  state:validUntil state:ConditionsChange .

:order-1847
  a order:OpenOrder ;
  order:customer :customer-22 ;
  order:total "142.00" .

:order-1852
  a order:OpenOrder ;
  order:customer :customer-31 ;
  order:total "89.50" .

# service: reports retrying branch
:upstream-branch-001
  a state:FulfillmentBranch ;
  state:status state:Retrying ;
  state:blockedBy state:RateLimit .

# caller: cancels branch; accepts answer
:caller-cancel-001
  a need:Cancellation ;
  need:cancels :upstream-branch-001 .

:caller-revision-001
  a need:Revision ;
  need:includes :caller-cancel-001 ;
  need:accepts :partial-answer-001 .

# service: finalizes; omits canceled branch
:final-answer-001
  a state:FinalAnswer ;
  state:answers :caller-need-001 ;
  state:supersedes :partial-answer-001 ;
  state:includes :order-1847, :order-1852 ;
  state:omits :upstream-branch-001 ;
  state:reason state:CanceledByCaller .

The Bill Arrives

Bare requests can be elegant. They win at local simplicity: one caller, one operation, one answer now. That is a useful shape when the work really is isolated, urgent, and complete at the boundary.

But most integration work is not like that. The caller knows deadlines, freshness tolerance, priority, dependency shape, and what might be canceled later. The service knows availability, partial progress, cache state, upstream delays, retries, and failure conditions. When those facts stay private, both sides make worse choices. The caller over-asks, the service over-serves, and the system pays for certainty it did not need.

The richer stream wins because it gives both sides room to optimize the whole process instead of each call in isolation. It lets work be batched when it is coming, deferred when it can wait, served stale when stale is acceptable, retried when useful, canceled when obsolete, and explained when partial or failed. The other party can only be as smart as you let it be – and right now we mostly ask it to fetch one thing at a time, in the dark, in a perpetual present tense, for no reason at all.