The interesting part here is that the same input data goes through two completely different encoding paths depending on whether a file is attached. That's a pretty nasty boundary to leave untested.
I've seen a related issue in FastAPI's GitHub discussions (#6151), where combining an uploaded file with a dictionary declared as Body(...) resulted in a 422 because the request switched from JSON to multipart/form-data. The fix was to move the structured fields into Form().
What connects both cases is that adding a file isn't just adding another field. It changes the serialization contract of the entire request.
I'd also test this at the HTTP boundary, not just against the internal encoder: inspect the prepared Content-Type, generated multipart field names, and actual payload, then run the same parameterized cases with and without files. That would catch regressions that unit tests around the dictionary itself could easily miss.
Really like this series. These are exactly the kinds of bugs that survive for months because the test data never crosses the boundary that triggers them.
Solo developer building practical CLI tools for Linux auditing and B2B data extraction. Creator of Vaultcheck and NPILift. I write about silent failures, false green lights, and infrastructure reliabi
Yes to the boundary, with the caveat this bug forces: with files attached the raise happens inside prepare(), before a socket exists, so the test never reaches the point where you can read Content-Type or the payload. The assertion has to sit on the prepare() call itself.
@pytest.mark.parametrize("key",[1,3.5,b"k1","k1",None])@pytest.mark.parametrize("files",[None,{"f":("a.txt",b"x")}])deftest_prepare_same_dict(key,files):req=requests.Request("POST","https://example.invalid/",data={key:"a"},files=files)p=req.prepare()# AttributeError for int/float keys only when files is set
ct=p.headers["Content-Type"].split(";")[0]assertct==("multipart/form-data"iffileselse"application/x-www-form-urlencoded")
I ran that matrix at 5 key types x 2 value types x 2 file states - 20 cells, requests 2.34.2 / urllib3 2.8.0 / CPython 3.11.18: 4 cells fail, all non-string keys, all with files.
The part that surprised me is that a body-level check can't see any of it. b'k1' and 'k1' emit identical multipart bytes, because urllib3 decodes bytes keys before translate() - the legality differs, the wire output doesn't. So byte comparison passes on a dict that is one edit away from raising, and your parameterized run has to assert on the key type, not on the payload.
Your FastAPI case is the same shape one layer up: attaching a file doesn't add a field, it swaps the serialization contract, and the failure lands in a branch no file-free test ever executes. Only the FastAPI one told you which parameter was wrong; this one names a Python type you never passed.
That's a good catch. You're right, I was placing the assertion too late. In this case, prepare() is the actual boundary where the failure needs to be tested.
One thing your bytes vs str observation makes me think about is normalization. If both keys can produce the same multipart field name, what happens when a dictionary contains both b"k1" and "k1"? They can coexist as Python dictionary keys but end up representing the same field on the wire.
I'd be inclined to validate field names before choosing the encoding path, rather than letting each encoder enforce a different set of rules. If bytes keys are supported, collision detection after normalization would also be worth considering. Otherwise, rejecting non-string keys consistently with a clear TypeError seems safer than silently coercing them.
The interesting part here is that the bug isn't just about multipart encoding failing. It's about two serialization paths accepting different input contracts for what looks like the same API.
Solo developer building practical CLI tools for Linux auditing and B2B data extraction. Creator of Vaultcheck and NPILift. I write about silent failures, false green lights, and infrastructure reliabi
Ran both paths on requests 2.34.2 / urllib3 2.8.0: neither rejects bytes keys â {b"k1": "a"} sends k1=a without a file, and Content-Disposition: form-data; name="k1" with one, so a TypeError guard inside either encoder is already too late.
The collision is the worse half. {b"k1": "a", "k1": "b"} plus a file emits two parts both named k1 and the server keeps one, while the urlencoded path sends k1=a&k1=b and a MultiDict keeps both â same dict, two different success semantics, no exception on either.
So validate before the branch and assert on the parsed result (exactly one field named k1, otherwise raise), not on body bytes: byte-level assertions go green on both paths while the field is already gone.
That's an interesting distinction. I'd take the collision test one step further and separate what the encoder emits from what the server actually exposes to the application.
There's a detail in RFC 7578 (section 5.2) worth considering here: multipart processors shouldn't coalesce parts with identical field names. So the two k1 parts aren't necessarily lost during parsing. They can survive in a MultiDict while ordinary dictionary-style access silently returns just one value. Werkzeug is a good example of this behavior with getlist() versus regular key access.
I'd test this with a field that is expected to be scalar, not just compare the serialized bodies. For example, if tenant_id arrives twice after normalization, the application shouldn't silently accept whichever value its parser happens to return.
Something like this:
Normalize field names before selecting the encoder.
Detect collisions after normalization.
Preserve repeated values at the parsing boundary.
Reject duplicate values when the API schema expects exactly one.
That last part matters because the same request can look perfectly valid at the transport level while violating the application's data contract.
I think there's a useful distinction here: the encoder should guarantee a consistent representation, but the application boundary should guarantee unambiguous meaning. Fixing the first without testing the second could leave the silent failure intact.
Solo developer building practical CLI tools for Linux auditing and B2B data extraction. Creator of Vaultcheck and NPILift. I write about silent failures, false green lights, and infrastructure reliabi
You're right and my wording was wrong on one point: the server doesn't keep one. Reran it just now â the body ships two parts both named k1, and werkzeug's MultiPartParser returns getlist("k1") == ["a", "b"] with nothing coalesced, which is your point.
The value is lost one line later, at dict-style access: form["k1"] == "a". Order decides the winner, not the parser â swap the two dict entries and form["k1"] becomes "b" instead. So the assertion belongs on the parsed mapping, per key, with a count:
p=requests.Request("POST","https://example.invalid/",data={b"tenant_id":"1","tenant_id":"2"},files={"f":("a.txt",b"x")}).prepare()form,_=MultiPartParser().parse(BytesIO(p.body),p.headers["Content-Type"].split("boundary=")[1].encode(),len(p.body))vals=form.getlist("tenant_id")assertlen(vals)==1,f"duplicate tenant_id after normalization: {vals}"
Measured on requests 2.34.2 / urllib3 2.8.0 / werkzeug 3.2.0.dev0, both dict orders, file attached and not attached.
Unmeasured: whether starlette or DRF coalesce or reorder before your handler sees the object â I only ran werkzeug's parser, so I wouldn't ship that half as a test I hadn't run.
We're a place where coders share, stay up-to-date and grow their careers.
The interesting part here is that the same input data goes through two completely different encoding paths depending on whether a file is attached. That's a pretty nasty boundary to leave untested.
I've seen a related issue in FastAPI's GitHub discussions (#6151), where combining an uploaded file with a dictionary declared as
Body(...)resulted in a 422 because the request switched from JSON to multipart/form-data. The fix was to move the structured fields intoForm().What connects both cases is that adding a file isn't just adding another field. It changes the serialization contract of the entire request.
I'd also test this at the HTTP boundary, not just against the internal encoder: inspect the prepared Content-Type, generated multipart field names, and actual payload, then run the same parameterized cases with and without files. That would catch regressions that unit tests around the dictionary itself could easily miss.
Really like this series. These are exactly the kinds of bugs that survive for months because the test data never crosses the boundary that triggers them.
Yes to the boundary, with the caveat this bug forces: with files attached the raise happens inside
prepare(), before a socket exists, so the test never reaches the point where you can read Content-Type or the payload. The assertion has to sit on theprepare()call itself.I ran that matrix at 5 key types x 2 value types x 2 file states - 20 cells, requests 2.34.2 / urllib3 2.8.0 / CPython 3.11.18: 4 cells fail, all non-string keys, all with files.
The part that surprised me is that a body-level check can't see any of it.
b'k1'and'k1'emit identical multipart bytes, because urllib3 decodes bytes keys beforetranslate()- the legality differs, the wire output doesn't. So byte comparison passes on a dict that is one edit away from raising, and your parameterized run has to assert on the key type, not on the payload.Your FastAPI case is the same shape one layer up: attaching a file doesn't add a field, it swaps the serialization contract, and the failure lands in a branch no file-free test ever executes. Only the FastAPI one told you which parameter was wrong; this one names a Python type you never passed.
That's a good catch. You're right, I was placing the assertion too late. In this case,
prepare()is the actual boundary where the failure needs to be tested.One thing your
bytesvsstrobservation makes me think about is normalization. If both keys can produce the same multipart field name, what happens when a dictionary contains bothb"k1"and"k1"? They can coexist as Python dictionary keys but end up representing the same field on the wire.I'd be inclined to validate field names before choosing the encoding path, rather than letting each encoder enforce a different set of rules. If bytes keys are supported, collision detection after normalization would also be worth considering. Otherwise, rejecting non-string keys consistently with a clear
TypeErrorseems safer than silently coercing them.The interesting part here is that the bug isn't just about multipart encoding failing. It's about two serialization paths accepting different input contracts for what looks like the same API.
Ran both paths on requests 2.34.2 / urllib3 2.8.0: neither rejects bytes keys â
{b"k1": "a"}sendsk1=awithout a file, andContent-Disposition: form-data; name="k1"with one, so a TypeError guard inside either encoder is already too late.The collision is the worse half.
{b"k1": "a", "k1": "b"}plus a file emits two parts both namedk1and the server keeps one, while the urlencoded path sendsk1=a&k1=band a MultiDict keeps both â same dict, two different success semantics, no exception on either.So validate before the branch and assert on the parsed result (exactly one field named
k1, otherwise raise), not on body bytes: byte-level assertions go green on both paths while the field is already gone.That's an interesting distinction. I'd take the collision test one step further and separate what the encoder emits from what the server actually exposes to the application.
There's a detail in RFC 7578 (section 5.2) worth considering here: multipart processors shouldn't coalesce parts with identical field names. So the two
k1parts aren't necessarily lost during parsing. They can survive in a MultiDict while ordinary dictionary-style access silently returns just one value. Werkzeug is a good example of this behavior withgetlist()versus regular key access.I'd test this with a field that is expected to be scalar, not just compare the serialized bodies. For example, if
tenant_idarrives twice after normalization, the application shouldn't silently accept whichever value its parser happens to return.Something like this:
That last part matters because the same request can look perfectly valid at the transport level while violating the application's data contract.
I think there's a useful distinction here: the encoder should guarantee a consistent representation, but the application boundary should guarantee unambiguous meaning. Fixing the first without testing the second could leave the silent failure intact.
You're right and my wording was wrong on one point: the server doesn't keep one. Reran it just now â the body ships two parts both named k1, and werkzeug's MultiPartParser returns getlist("k1") == ["a", "b"] with nothing coalesced, which is your point.
The value is lost one line later, at dict-style access: form["k1"] == "a". Order decides the winner, not the parser â swap the two dict entries and form["k1"] becomes "b" instead. So the assertion belongs on the parsed mapping, per key, with a count:
Measured on requests 2.34.2 / urllib3 2.8.0 / werkzeug 3.2.0.dev0, both dict orders, file attached and not attached.
Unmeasured: whether starlette or DRF coalesce or reorder before your handler sees the object â I only ran werkzeug's parser, so I wouldn't ship that half as a test I hadn't run.