Unknown Protobuf Fields Need a Boundary Before They Reach Internal Services
Protocol Buffers are designed for evolution. A parser that does not recognize a field can retain it and write it back when the message is reserialized. The language guide documents this retention behavior for proto3 and proto2. It is a useful compatibility property: an old service can modify a message without destroying fields that a newer service understands.
That same property changes the meaning of validation in a multi-service path. A public gateway may parse a request into a current message type, inspect the fields it knows, and forward the reserialized object. Unknown fields carried through the gateway can reach an internal service with a newer schema that gives them meaning. The gateway has effectively forwarded a command it never authorized.
Validation is schema- and boundary-dependent
The risk is not that unknown fields exist. The risk is that a component claims to validate a message while preserving fields outside its validation model. If the design requires a closed command, the boundary should reject unknown fields or strip them before forwarding. If forward compatibility is required, the receiving service must apply authorization to every field it interprets, including fields that the gateway did not see.
The issue becomes sharper when the gateway maps protobuf into a second representation. A proxy may convert a message to JSON, drop unknowns, and reconstruct it later. That can be safe, but only if the transformation and the receiving schema agree about field presence and defaults. Proto3 serialization is not canonical, so a byte comparison of messages is not a safe substitute for semantic validation and authorization.
Do not use a protobuf hash as a signature over a partially validated request. Field ordering, unknown fields, and encoding choices can produce different bytes for the same logical message. Authenticate the transport and the signed payload under a documented canonicalization rule, then validate the decoded command against a schema and an authorization policy.
Design options
The most conservative public boundary uses a dedicated request message that contains only the fields the public API supports. The gateway validates it, authorizes the caller, and constructs a new internal command from named fields. Unknown fields are discarded by construction rather than by a fragile recursive strip.
An internal compatibility boundary can preserve unknown fields, but it should be an explicit contract. The internal service owns authorization for the full message version it accepts. It should not trust a gateway assertion that a message was already validated if the gateway's schema is older than the receiving schema.
Record the schema version and descriptor hash at each tier. Add tests that send a field understood by the downstream service but absent from the gateway schema, then verify the resulting authorization decision. Test an unknown field that changes a oneof or enum value, and test a message that round-trips through each language runtime used by the system.
Operate the schema as code
Schema changes should go through the same review as code. Mark removed fields as reserved, document compatibility expectations, and review new security-sensitive fields with the services that will interpret them. A field named is_admin, tenant_id, or actor deserves a dedicated threat review even if the public gateway cannot currently set it.
Unknown-field retention is not a vulnerability by itself. It is a warning that a message may mean more than one component thinks it means. The security boundary should be explicit about where unknowns stop, which service interprets them, and how the authorization layer learns about them.
Top comments (0)