The Same Problem in Three Places

Encoding is only interesting because data moves. Where it moves changes who has to agree with whom, and for how long.

Through a database

The writer is your application. The reader is your application, later. Possibly years later, possibly a version nobody remembers writing.

There is a specific trap here that people meet exactly once and never forget.

An old version of the service reads a record containing a field it does not know about, modifies one attribute, and writes the whole record back. The unknown field is gone — not because anything failed, but because the old code round-tripped through an object model that had no place to put it.

stored     { id, name, preferences }        written by v2
v1 reads   { id, name }                     preferences is unknown, dropped
v1 writes  { id, name }                     preferences is now permanently lost

No error is raised. Nobody notices until a v2 instance reads that record and finds the field missing. The fix is to preserve unknown fields through a read-modify-write cycle, which some libraries do automatically and many do not — worth checking rather than assuming, because the failure is silent and permanent.

Through a service

Two programs, upgraded independently, communicating in real time.

REST and HTTP APIs. Compatibility is by convention: add optional fields, ignore unknown ones, never change the meaning of an existing field. When something must break, version the endpoint — /v2/orders — and run both until the clients migrate. Public APIs live with old clients essentially forever; a mobile app from three years ago is still out there and still making requests.

RPC frameworks — gRPC, Thrift — use the schema formats from the last lesson, so compatibility is enforced by the encoding rather than by convention. That is a real advantage, and it does not remove the problem: adding a required field still breaks old clients, whatever the framework promises.

Through a message queue

The producer writes a message. Sometime later — seconds, or hours — a consumer reads it. Possibly several consumers, possibly ones the producer has never heard of.

This decouples deploys almost entirely, and that is the main reason to do it:

• The producer does not need to know who consumes, or how many.
• A consumer can be down for an hour without the producer noticing.
• New consumers can be added without touching the producer.
• Messages can be replayed to reprocess history.

The compatibility consequence is that messages must be forward and backward compatible for as long as they might sit in the queue or be replayed — and if you replay from the beginning of a retained log, that is the entire history of the system.

The pattern across all three

The database column is the one people underestimate. A service contract breaks loudly and immediately. A message-format mistake surfaces when a consumer next runs. A database compatibility mistake surfaces in eighteen months, in a report someone runs for the first time, and by then the data is gone.

What to take away

Compatibility is the same question everywhere — can both sides handle each other’s data — but the horizon differs wildly. Requests are transient, messages last as long as retention, and database records outlive every version of the code. Preserve unknown fields, register your message schemas, and treat the database as the case where mistakes are permanent.