🐛 fix: match HTTP field names the way a recipient must - #4585
ReneWerner87 merged 57 commits into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review (3)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
WalkthroughThe PR adds shared case-insensitive header and media-type utilities. It updates form parsing, caching, proxy filtering, middleware synchronization, CSRF checks, logging, scheme validation, and related documentation and tests. ChangesHeader normalization and request processing
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related PRs
Suggested labels: Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## claude/split-1-url-composition #4585 +/- ##
==================================================================
+ Coverage 93.58% 93.69% +0.11%
==================================================================
Files 140 140
Lines 15605 15978 +373
==================================================================
+ Hits 14604 14971 +367
+ Misses 627 624 -3
- Partials 374 383 +9
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
417636b to
ddc5560
Compare
|
@coderabbitai review Auto-review is off for this one because the base is Note the branch was force-pushed once, rebasing onto #4584's new tip after a documentation fix there. No code changed in the rebase. Generated by Claude Code |
|
✅ Action performedReview finished.
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 417636b676
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Field names are case-insensitive (RFC 9110 §5.1), but fasthttp's Peek, PeekAll and Del compare the stored key byte for byte. That is enough while fasthttp canonicalizes it — under DisableHeaderNormalizing the store keeps the spelling the client sent, and lower case is what HTTP/2 and HTTP/3 put on the wire, so it is what a front end translating down to HTTP/1.1 preserves. Every guard below was then reading a header that was there as if it were absent. cache: an anonymous request was served another user's authenticated page. A lower-case "authorization:" read as no credential at all, so the response was stored as anonymous and replayed to everyone — bearer token included. Set-Cookie2, Vary and KeyHeaders missed the same way, storing and replaying a personalized response and dropping a Vary dimension silently out of the key. "public" was injected into a response that never said it, which is exactly the token that lifts RFC 9111 §3.5 for a response to an authenticated request. Also in cache, unrelated to case: a cached 3xx lost its Location, so the first client got "301 Location: /new" and every one after it a bare 301; a Set-Cookie on a shared entry handed one client's session to everyone matching the key; and a repeated header name was collapsed on replay, so a response sending Vary twice came back varying only on the second. proxy: every sanitizer removed nothing. A cross-host redirect carried the caller's "authorization:" and "cookie:" to the redirect target; "upgrade", "te", "keep-alive" and a "connection:"-listed header reached the upstream; and the client's own "x-real-ip" arrived beside the one the proxy writes, so the upstream attributed the request to an address the client chose. The response strip also asked the wrong object — a proxied response is parsed by the outbound fasthttp.Client, which carries its own setting. csrf: the origin check was switched off by the case of a header name. With a valid double-submit pair, "Origin: http://evil.com" gave 403 but "origin: http://evil.com" gave 200, because an absent Origin is not a failure on a plaintext connection. extractors.FromHeader: a token sent under the lower-case name was not found, and the request refused for carrying no token when it carried one. logger: ${ips} logged an empty chain while the trust decisions went on using the header it could not see, and RegisterContextTag wrote its value raw, where a CR or LF forged a log line. adaptor: a wrapped net/http middleware's header edits were lost. A Header.Set collapsing a multi-valued header left the other values in place, a middleware that changed nothing duplicated the last value of every multi-valued header, and removals never propagated at all. form parsing: fasthttp matches the media type and "boundary=" case sensitively, so a legal "Multipart/Form-Data" bound nothing and reported no error. Every entry point that reaches those parsers now folds first. Scheme(): only "http" and "https" are accepted from a proxy header. The value is spliced into a URL by BaseURL and compared for origin equality by csrf and Redirect().Back, so no other scheme may become the request's. basicauth and cors were checked too and fail closed — a lower-case name locks a legitimate client out, never lets one in — so they are left alone. Four internal packages hold the shared primitives, so the duplication that caused one of these cannot cause the next: fieldname (case-insensitive read and delete), headerlookup (the same for a caller holding a Ctx), mediatype (the Content-Type fold) and crosshost (the credential list the client and proxy had already let drift apart). Benchmarked: the first delField cost 792 B/op and ran 3x slower on the proxy's default path, so the normalization answer is threaded from config instead — measured back to 0 B/op and the original ns/op. An earlier header dedup in adaptor was 17% slower at twenty headers and 61% at a hundred, so it was removed. Split out of #4581, on top of the open-redirect fixes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JZwaNA4CeVV9KxKhBDXWGC
ddc5560 to
5d8b101
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (4)
middleware/adaptor/adaptor_test.go (1)
2300-2377: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd coverage for a request with header normalization disabled.
Every subtest uses
fiber.New(), which keeps fasthttp header normalization enabled, and every header name is already canonical. The newisFramingHeaderandclearCopiedHeaderslogic exists to survive non-canonical storage, so that path stays untested here.Add a subtest that builds the app with
fiber.New(fiber.Config{DisableHeaderNormalizing: true})and sends a lowercasecontent-lengthand a lowercasex-forwarded-for. That case provesclearCopiedHeadersskips framing fields by fold-equality and thatDelmatches the raw stored spelling.I can write this subtest if you want it.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@middleware/adaptor/adaptor_test.go` around lines 2300 - 2377, Add a subtest to Test_HTTPMiddleware_HeaderFidelity using fiber.New(fiber.Config{DisableHeaderNormalizing: true}) and lowercase content-length and x-forwarded-for request headers. Exercise the existing middleware flow and assert framing headers remain correctly handled while the lowercase forwarded-for value is preserved or replaced through raw-spelling-aware deletion. Keep the test focused on clearCopiedHeaders and isFramingHeader behavior without changing the existing canonical-header subtests.docs/middleware/cache.md (1)
127-211: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winRun the Markdown lint task for these new sections.
The repository requires
make markdownafter Markdown changes. The added sections are long and contain hard-wrapped prose plus a fenced Go block, which are common lint targets.As per coding guidelines: "
**/*.md: Runmake markdownto lint all Markdown files when modifying code".🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/middleware/cache.md` around lines 127 - 211, Run the repository’s Markdown lint task with make markdown after updating the new sections in the cache middleware documentation, then fix any reported formatting issues in the hard-wrapped prose or fenced Go example.Source: Coding guidelines
middleware/cache/cache.go (1)
770-780: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueRemove the always-true
v >= 0check.
fasthttp.ParseUintreturns a non-negativeintwhenerr == nil. The inner condition never fails, so it only adds a branch.♻️ Proposed simplification
if b := fieldname.First(&c.Response().Header, fiber.HeaderAge, canonical); len(b) > 0 { if v, err := fasthttp.ParseUint(b); err == nil { - if v >= 0 { - ageVal = uint64(v) - } + ageVal = uint64(v) } } else {🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@middleware/cache/cache.go` around lines 770 - 780, Remove the redundant v >= 0 condition in the ageVal assignment within the response-header age parsing logic; when fasthttp.ParseUint succeeds, assign v directly to ageVal, while preserving the existing error handling and default header behavior.middleware/cache/utils.go (1)
156-158: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueAlign the comment with the constant.
maxIgnoredHeaderLenis 32, but the longest name inignoreHeadersisProxy-Authorization(19 bytes). The constant is an upper bound, not the longest name. State it as a bound so a later reader does not shrink it to match the current list.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@middleware/cache/utils.go` around lines 156 - 158, Update the comment above maxIgnoredHeaderLen to describe 32 as an upper bound for ignored header-name lengths, not as the longest current name in ignoreHeaders; retain the constant value and its folding behavior unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/extra/internal.md`:
- Around line 367-368: Update the cookie security description in “Setting a
Cookie” to limit protection from plaintext transport to cases where the Secure
attribute is set. Clarify that HTTP-originated requests may receive a non-Secure
fiber_flash cookie and therefore may transmit it over HTTP, while retaining the
existing HttpOnly and visibility statements.
In `@extractors/extractors.go`:
- Around line 348-351: Update FromAuthHeader to retrieve the authorization
header through headerlookup.Value instead of byte-exact c.Get, preserving the
existing scheme validation and authentication flow. Add coverage confirming
FromAuthHeader accepts differently cased authorization field names when header
normalization is disabled.
---
Nitpick comments:
In `@docs/middleware/cache.md`:
- Around line 127-211: Run the repository’s Markdown lint task with make
markdown after updating the new sections in the cache middleware documentation,
then fix any reported formatting issues in the hard-wrapped prose or fenced Go
example.
In `@middleware/adaptor/adaptor_test.go`:
- Around line 2300-2377: Add a subtest to Test_HTTPMiddleware_HeaderFidelity
using fiber.New(fiber.Config{DisableHeaderNormalizing: true}) and lowercase
content-length and x-forwarded-for request headers. Exercise the existing
middleware flow and assert framing headers remain correctly handled while the
lowercase forwarded-for value is preserved or replaced through
raw-spelling-aware deletion. Keep the test focused on clearCopiedHeaders and
isFramingHeader behavior without changing the existing canonical-header
subtests.
In `@middleware/cache/cache.go`:
- Around line 770-780: Remove the redundant v >= 0 condition in the ageVal
assignment within the response-header age parsing logic; when fasthttp.ParseUint
succeeds, assign v directly to ageVal, while preserving the existing error
handling and default header behavior.
In `@middleware/cache/utils.go`:
- Around line 156-158: Update the comment above maxIgnoredHeaderLen to describe
32 as an upper bound for ignored header-name lengths, not as the longest current
name in ignoreHeaders; retain the constant value and its folding behavior
unchanged.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: c3909ac7-7dea-4592-bfd7-79986a40e80a
📒 Files selected for processing (44)
bind.gobinder/form.gobinder/form_test.goctx_interface_gen.goctx_test.godocs/api/ctx.mddocs/extra/internal.mddocs/middleware/cache.mddocs/middleware/logger.mddocs/middleware/proxy.mdextractors/extractors.goextractors/extractors_test.gohelpers.gohelpers_test.gointernal/crosshost/crosshost.gointernal/fieldname/fieldname.gointernal/headerlookup/headerlookup.gointernal/mediatype/mediatype.gointernal/mediatype/mediatype_test.gomiddleware/adaptor/adaptor.gomiddleware/adaptor/adaptor_test.gomiddleware/cache/cache.gomiddleware/cache/cache_security_test.gomiddleware/cache/cache_test.gomiddleware/cache/cachecontrol.gomiddleware/cache/coverage_test.gomiddleware/cache/keygen.gomiddleware/cache/keygen_test.gomiddleware/cache/utils.gomiddleware/cache/vary.gomiddleware/csrf/csrf.gomiddleware/csrf/csrf_test.gomiddleware/logger/context_tag.gomiddleware/logger/logger_test.gomiddleware/logger/tags.gomiddleware/proxy/bench_test.gomiddleware/proxy/fuzz_test.gomiddleware/proxy/proxy.gomiddleware/proxy/proxy_test.gomiddleware/proxy/security.gomiddleware/proxy/security_test.goredirect.goreq.goreq_interface_gen.go
💤 Files with no reviewable changes (1)
- helpers_test.go
FromHeader was switched to the case-insensitive read and FromAuthHeader was missed — the same bug, one function over, on the path every keyauth and bearer-token caller takes. Under DisableHeaderNormalizing a lower-case "authorization:" read as absent, so the scheme check never ran and the request was refused for carrying no credential when it carried one. Lower case is what HTTP/2 and HTTP/3 put on the wire, so it is what a front end translating down to HTTP/1.1 preserves. Mutation-verified: with the read reverted to c.Get, the new Test_FromAuthHeader_IgnoresHeaderNameCase fails on the "authorization" spelling under normalize=false. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JZwaNA4CeVV9KxKhBDXWGC
client: a redirect that changed hosts kept going. 301, 302 and 303 now turn any body-carrying method into a GET and drop the body, which is net/http's redirectBehavior — for every method rather than just POST, since Fiber drives QUERY requests through the same loop and one that kept its method would replay its body to the redirect target under a method that no longer describes it. 307 and 308 are untouched by design. client cookie jar: Set-Cookie was credited to the URI the caller asked for, not the one that answered, so the last hop of a redirect chain could plant cookies for an unrelated origin. The jar now keys them by the responding URI. A cookie set without a Path is scoped to the directory of the request that set it, and a path that cannot round-trip through fasthttp's normalization falls back to "/" rather than to a scope the cookie could never match — RFC 6265 §8.5 is explicit that the path attribute is not a security boundary. res.JSONP: the callback name landed verbatim in a same-origin text/javascript body, and callers routinely take it straight from the query string, which is what JSONP is for. It is now reduced to a JavaScript member expression — everything outside [A-Za-z0-9_$.[]] is dropped, and the result must still parse as one, so "1.2.3", "[]" and "a[" fall back to the default name rather than emitting a body that throws. listen: TLSConfig silently supersedes CertClientFile, so a deployment that set both came up serving every client and demanding a certificate from none. Every superseded TLS field is now named in a startup warning, and App.Listener says outright that it serves the listener as supplied — no TLS and no client-certificate verification unless the caller wrapped it. Config.SkipUnmatchedRoutes documents that a rate limiter is in the same position as the rest of the middleware chain: requests to unregistered paths are neither counted nor throttled. Several plausible findings were refuted by measurement rather than patched, and are recorded here so they are not re-investigated: a client cross-host credential leak (fasthttp already strips them), Host-header injection through BaseURL (fasthttp answers "Host: //evil.com" with 400 before a handler runs), and a suspected cross-session cache collapse (0 wrong answers across KeyHeaders and Vary, normalizing and not). Split out of #4581, the last of three. Together the three reproduce that branch exactly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JZwaNA4CeVV9KxKhBDXWGC
standardClientTransport and hostClientTransport delegated to fasthttp's redirect loop, so ErrRedirectDowngrade never fired for the client New() builds. A same-host HTTPS to HTTP hop kept Authorization and Cookie and sent them in plaintext. Both now use doRedirectsWithClient, which every transport already satisfies. 301, 302 and 303 drop the request body even where the method already was GET or HEAD, as net/http's redirectBehavior does. It would otherwise be replayed to the redirect target, possibly a different host. A JSONP bracket index that opens with a digit is that number alone, so cb[0foo], ns[0x] and ns[1.2.3] fall back to the default name rather than emitting a body that throws. TLSMinVersion joins the superseded-field warnings the docs already list it in, and both client-certificate warnings now name ClientAuth's require modes instead of claiming ClientCAs is jointly required. The log-capturing listen tests reuse withCapturedLogOutput so they restore the writer that was active before, and stay serial: Go runs no parallel test while a serial one holds the package logger, and Test_Listen_TLSConfig_WithTLSConfigFunc is parallel and warns.
The two remaining ListenConfig comments still read as though ClientCAs were jointly required. ClientAuth is what requires a certificate; ClientCAs are the roots it verifies against. Listen also clones the supplied config, so later mutations of the caller's do not reach it.
isJSONPMemberExpression asked whether parsing was inside brackets, not whether the token now beginning was opened by one, so "cb[a.0]" passed: depth is 1 at the digit, but "0" there is a property named after a dot, which is as illegal as a leading digit at the top level. The sanitizer returned it unchanged and emitted a body that throws — the outcome the member-expression check exists to replace with the default name. Track the '[' instead: inIndex is set only by the bracket that opens a token and cleared as soon as the token starts, so the number rule applies where a number may stand and nowhere else. "cb[0]" and "cb[a.b]" are unaffected; "cb[a.0]", "cb[x[a.1]]" and "cb[a.b.0]" now fall back. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JZwaNA4CeVV9KxKhBDXWGC
…alues whole Three review findings from #4585. cache: the Authorization key hashed a single Peek, so two credentials sent on separate field lines shared a key with the first alone, and one client's response replayed for another. Hash the length-prefixed field lines instead of a comma join, since "Bearer a,Bearer b" on one line and two lines carrying one each are different principals. cache: an entry written before Set-Cookie joined ignoreHeaders carries a cookie beside a body personalized for whoever caused that miss. Dropping the cookie on replay leaves that body, so treat such an entry as private and re-run the handler. mediatype: a quoted-pair inside a parameter value ended the quoted-string early, folding the rest as parameter names. fasthttp matches "boundary" case-sensitively and takes the first, so lowercasing a decoy hidden in a value promoted it over the real one: on `X="\"; BOUNDARY=bogus"; BOUNDARY=Real` the parsed boundary was bogus. docs: ${ips} logs the chain as supplied whether or not the peer is trusted, and the flash cookie is Secure only when the request arrived over TLS. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JZwaNA4CeVV9KxKhBDXWGC
… read The JSONP validator accepted "for" and "cb[new]" as member expressions, so the emitted body was a syntax error the browser dropped and the callback never ran. Refuse a reserved word only where JavaScript reads the token as a name — the head of the expression and the head inside each index — since "cb.new" and "cb[a.class]" are property names, which any word may be. Extract the "default" Format sentinel into a constant so the keyword list does not trip goconst on an unrelated literal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JZwaNA4CeVV9KxKhBDXWGC
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c36bcecae1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…on edits cache: the response was read with the request's normalization setting, but a proxy hands c.Response() to an outbound fasthttp.Client, which stamps its own DisableHeaderNamesNormalizing on the response it parses into. A default app therefore held a lower-case "cache-control: private" that was read as absent. Confirmed end to end: alice's body was cached and replayed to bob under a synthesized "public, max-age=300". Ask the stored names instead of the app config. fieldname.Canonical is one pass over them and answers exactly what the byte-exact Peek needs to know. adaptor: Connection left framingHeaders. It frames the hop, not the body, and naming which fields are hop-by-hop is something wrapped middleware rewrites — excluding it dropped that edit and left the handler reading the client's own list. fasthttp re-derives its close flag when the line is written back. docs: list the whole set a cache entry keeps without StoreResponseHeaders, which is more than Location and Content-Type. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JZwaNA4CeVV9KxKhBDXWGC
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9cfe6cb9b3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…lback A JSONP body is fetched by a script tag, so it is parsed under the script goal in sloppy mode — and two words on the rejection list are ordinary identifiers there. "await" is reserved only inside a module or an async function, "yield" only in strict mode or a generator, so "await.cb", "cb[await]", "yield.cb" and "cb[yield]" all name a callback that a browser would have invoked, and each fell back to "callback" instead. Checked the whole list against V8 rather than fixing the one that was reported: every listed word was compiled in both positions under the script goal, and exactly these two parse. The rest are syntax errors wherever they stand, and "let" stays contextual in its own third way, handled where it is read. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JZwaNA4CeVV9KxKhBDXWGC
Joining the rewritten values fixed the lost token but broke the close signal: fasthttp's Set clears connectionClose for anything but a bare "close", so middleware writing ["close", "X-Internal"] left the server treating as reusable a connection the client had asked to close. The test added with that change codified the wrong result. fasthttp holds one or the other, never both — once the flag is set, Peek answers "close" and the stored value is unreachable. Where the two conflict the flag wins: the server reads it on every request, and keeping a connection open against the client's instruction is a protocol violation, where the tokens it hides only describe what is hop-by-hop. Matched case-insensitively across the token list, per RFC 9110 Section 7.6.1. The reported case is untouched by the choice: ["keep-alive", "X-Internal"] carries no close token and still reaches the handler whole. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JZwaNA4CeVV9KxKhBDXWGC
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 19949ed8a5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Peek reports the first line stored under a name whether or not it holds anything, and an empty answer is indistinguishable from an absent header. A message carrying "Origin:" ahead of its real Origin therefore read as having none — and an absent Origin is not a failure to the CSRF check: on a plaintext request it skips the check and allows the request. Confirmed end to end, under both spellings and both configurations: Origin: -> Peek "" , PeekAll ["", "http://evil.example"] Origin: http://evil.example Reported for DisableHeaderNormalizing, where the second line is spelled "origin:". It is not limited to that: with names normalized both lines land under "Origin" and the first one still shadows the second, so the hole was open on the default configuration too. First now steps past empty lines on both paths. A field line that is present and empty says nothing a caller can act on, so nothing is lost by looking past it for one that does; Lines already read every line for the same reason. The walk is skipped where Peek answers nil, which fasthttp means as absent rather than empty — Test_First_EmptyLineIsNotNil pins that distinction so a change upstream fails loudly instead of quietly reverting this to reading one line. Measured on Benchmark_Middleware_CSRF_Check, interleaved runs: 314 B/op and 11 allocs/op unchanged, ns/op +2% median. The allocation-free result needed the fold walk split into its own function — All's iterator is heap-allocated where the compiler cannot see the branch is dead, which cost the canonical path one 24 B allocation per absent lookup. That one predates this change, so the canonical path is now cheaper than it was. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JZwaNA4CeVV9KxKhBDXWGC
…opped fasthttp's own redirect loop clears postArgs and the parsed-form flag together; this loop clears only the values, because the flag is not reachable from outside the package. parsedPostArgs is unexported and set false in exactly two places, Request.Reset and Request.ReadLimitBody — Reset would also take the URI, headers and TLS flag the loop is still using, and ReadLimitBody overwrites the header from a reader. The values are the load-bearing half and stay cleared: Request.Write falls back to the parsed form whenever the body is empty, so without it a caller that had read PostArgs sent the dropped "a=1" as the body of the redirected GET. What is left is confined to a caller that reuses the request without resetting it. Measured, reading PostArgs after setting a new body: untouched fasthttp request "a=1" the previous body's values after dropRequestBody "" no values fasthttp's own loop "b=2" re-parsed from the new body So the gap is against fasthttp's loop, not against leaving the request alone — an untouched request is the one that hands back stale form values. The body on the wire is the new one in every case. Test_DropRequestBody_ReusedRequestSendsTheNewBody covers the two properties that hold: a reused request sends the body it was given, and the earlier form comes back neither in it nor from a later PostArgs. Dropping the reset fails it, and the test beside it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JZwaNA4CeVV9KxKhBDXWGC
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dcc39c66b1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…cking one Reading past an empty line let whoever wrote the other one decide what a field holds. Under DisableHeaderNormalizing a middleware that clears Authorization writes the canonical name byte-exactly and leaves the client's lower-case line — the spelling HTTP/2 and 3 send — beside it. Both orders fasthttp stores, and what each read answers: [Authorization: "", authorization: Bearer client] first: "" non-empty: client [authorization: Bearer client, Authorization: ""] first: client non-empty: client So the previous read was wrong in the first row and the read before it was wrong in the second: whichever rule is chosen, one of the two orders hands the extractor a credential the middleware had just cleared. The second row was already wrong before this branch, which is the more common one — a client sending only the lower-case name puts the cleared line last. Origin, Referer, Sec-Fetch-Site and Authorization are all defined as a single value, and RFC 9110 Section 5.2 says a sender must not generate a second line for such a field. A message carrying two is malformed, and what it means is not a question with an answer; headerlookup.Value now says so with a second return rather than resolving it. Every caller refuses rather than treating that as an absent header. The distinction is load-bearing in the CSRF check: an absent Origin is not a failure there, it skips the check on a plaintext request, so mapping ambiguity to "not found" would hand a malformed message the same pass this branch just closed. In the extractors it is belt-and-braces — the ambiguous value is empty and both paths end at ErrNotFound — kept so the refusal does not depend on that coincidence. A repeated Origin is now refused even when both lines agree with the host. That is deliberate: the check cannot know which line the request meant, and no browser sends two. Interleaved runs of Benchmark_Middleware_CSRF_Check, 12 samples each, against the commit before the empty-line fix: before 2214 ns/op 314 B/op 11 allocs/op empty-line 2270 (+2.6%) 314 11 this 2146 (-3.0%) 314 11 Counting the lines needs the whole walk, so Ctx.Get's byte-exact peek ahead of it was redundant and is gone; one PeekAll now answers both the count and the value, which is a scan fewer than either earlier shape. Immutable is honored where Ctx.Get honored it. Three mutations fail: dropping the count on either path, and mapping the CSRF ambiguity back to errOriginNotFound. Ignoring the flag in the extractor does not fail, which is the belt-and-braces point above rather than a gap in coverage. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JZwaNA4CeVV9KxKhBDXWGC
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8292a47de9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…omHeader FromHeader is given its name by the application, so the strict single-value read was wrong for it. Accept, Forwarded and the other list fields may legally arrive on several lines, and there two lines are one value rather than two answers — refusing them turned a valid request into ErrNotFound. They are joined the way RFC 9110 Section 5.3 says a recipient may, in the order received. Nothing is loosened where the named field is a credential: a token joined with another does not match the one that was issued, so the comparison the caller makes still fails, which is where Value's refusal lands directly. FromAuthHeader keeps Value. Authorization is named by this package rather than by config, and it is a single-value field. Also covers the two refusals the previous commit left untested: a repeated Sec-Fetch-Site and a repeated Referer, the second reachable only on HTTPS, which needs the trusted-proxy app since Ctx.Scheme reads X-Forwarded-Proto only from a trusted peer. Both csrf functions and originMatchesHost are at 100% now, against 88.9% and 93.8%. Reverting FromHeader to the strict read fails Test_Extractor_FromHeader_CombinesRepeatedLines. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JZwaNA4CeVV9KxKhBDXWGC
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b2c6add953
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…directive The Cache-Control scan split on every comma, so the tokens inside a quoted extension value were read as directives of their own. RFC 9111 Section 5.2 makes that value a quoted-string, and a comma in one is data. Reproduced end to end: a handler answering with Cache-Control: ext="a, public, b" Set-Cookie: session=client-1 was read as having said "public", which is enough to pass the gate that lets a cookie-setting response be stored. Its body was then replayed to later clients with only the cookie dropped. `ext="a, s-maxage=99, b"` passes the same gate, and `ext="a\", public, b"` gets there through the escaped quote. The part scan now tracks the quoted-string and steps over a quoted-pair. Unterminated quotes swallow the rest of the field, which is the safe way round: what follows is read as one value rather than as directives a sender never separated. hasDirective has the same blindness and is left alone: both its callers ask about no-cache and private, where reading a quoted token as a directive only makes the cache more conservative. Also fixes FromHeader named with Cookie, from the previous commit's join. Cookie is not comma-separated — RFC 6265 Section 5.4 joins its crumbs with "; " — and fasthttp keeps them in a store of its own that PeekAll enumerates one at a time, so a wire request carrying Cookie twice came back as "a=1, b=2". Collecting the store first, as the cache's keyFieldLines already does, makes fasthttp reassemble the field. Test_Cache_QuotedDirectiveDoesNotAuthorizeSharing covers the three values above, with Test_Cache_QuotedDirectiveStillReadsRealOnes as the control that public outside the quotes still authorizes sharing. Test_Extractor_FromHeader_CookieKeepsItsOwnSeparator reads from the wire, which is the only shape where the crumbs arrive separately. Reverting either fix fails its tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JZwaNA4CeVV9KxKhBDXWGC
…ends it Request.URI swallows the parse error and hands back a half-parsed URI, the same way UpdateBytes does for redirect targets. Measured: http://example.com:abc/x -> URI().Host="example.com:abc" path="/" URI.Parse: invalid port ":abc" after host So the malformed authority stayed as the host and the path was lost. A HostClient dials its fixed Addr whatever the URI says, so that host went out as the Host header of a request the caller never wrote. The targets this loop composes have been checked since they are read; this is the same check for the URI the caller supplied. It sits inside the loop after SetRequestURI, where fasthttp's own doRequestFollowRedirects calls the error-returning req.parseURI, so the check covers every URI the loop is about to send rather than only the first. Test_Transport_DoRedirects_RefusesUnparsableInitialURI asserts the error and that nothing was sent, with a URI that parses as the control. Removing the check fails both malformed rows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JZwaNA4CeVV9KxKhBDXWGC
🐛 fix: client redirect handling, TLS diagnostics and the JSONP callback
92144fc
into
claude/split-1-url-composition
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ef43f3f24d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // against the initial host, ignoring the port, subdomains still trusted. | ||
| if !trustedRedirectTarget(nextHost, initialHostname) { | ||
| for _, h := range crosshost.SensitiveHeaders { | ||
| req.Header.Del(h) |
There was a problem hiding this comment.
Delete cross-host credentials case-insensitively
When callers use req.Header.DisableNormalizing() and add a lower-case credential such as authorization or cookie, RequestHeader.Del compares the stored name byte-for-byte, so this loop leaves the credential in place and sends it to an unrelated host selected by the redirect response. Use the case-insensitive deletion helper for every sensitive name so the cross-host protection also covers non-normalized requests.
AGENTS.md reference: AGENTS.md:L11-L13
Useful? React with 👍 / 👎.
| } | ||
|
|
||
| // go test -run Test_Listen_TLSConfig_WarnsSupersededFields | ||
| func Test_Listen_TLSConfig_WarnsSupersededFields(t *testing.T) { |
There was a problem hiding this comment.
Make the new log-capture tests parallel-safe
This test and the other newly added warning tests intentionally omit t.Parallel() because they mutate package-level logging state, but the repository requires every new test and subtest to invoke it. Refactor the warning/log capture path to isolate or inject the logger, then run these tests and their subtests in parallel rather than adding serial exceptions.
AGENTS.md reference: AGENTS.md:L9-L9
Useful? React with 👍 / 👎.
| // A 100-continue expectation may not be generated without content | ||
| // (RFC 9110 Section 10.1.1), and a strict target answers the bodyless | ||
| // follow-up with 417 rather than the redirect the caller was following. | ||
| req.Header.Del(fasthttp.HeaderExpect) |
There was a problem hiding this comment.
Remove non-normalized Expect headers with redirected bodies
For a request whose header normalization is disabled and whose field is stored as lower-case expect, this byte-exact deletion misses it. After a 301, 302, or 303, dropRequestBody therefore emits a bodyless redirected request that still carries Expect: 100-continue; as the adjacent comment notes, a strict target can answer 417 instead of completing the redirect. Delete this field case-insensitively, along with the other body metadata removed above.
AGENTS.md reference: AGENTS.md:L11-L11
Useful? React with 👍 / 👎.
Description
2 of 3, split out of #4581. Based on #4584 — review that one first, or read this diff against
claude/split-1-url-composition.Field names are case-insensitive (RFC 9110 §5.1), but fasthttp's
Peek,PeekAllandDelcompare the stored key byte for byte. That is enough while fasthttp canonicalizes it — underDisableHeaderNormalizingthe store keeps the spelling the client sent, and lower case is what HTTP/2 and HTTP/3 put on the wire, so it is what a front end translating down to HTTP/1.1 preserves. Every guard below was then reading a header that was there as if it were absent.Measured, with
DisableHeaderNormalizingset:That last line is why this survived review: the names fasthttp keeps in a slot of their own still resolve, so a spot check on the obvious header passes.
What was wrong
cache
An anonymous request was served another user's authenticated page. A lower-case
authorization:read as no credential at all, so the response was stored as anonymous and replayed to everyone — bearer token included.Set-Cookie2,VaryandKeyHeadersmissed the same way, storing and replaying a personalized response and dropping aVarydimension silently out of the key.publicwas injected into a response that never said it — exactly the token that lifts RFC 9111 §3.5 for a response to an authenticated request.Unrelated to case, in the same paths: a cached 3xx lost its
Location, so the first client got301 Location: /newand every one after a bare 301; aSet-Cookieon a shared entry handed one client's session to everyone matching the key; and a repeated header name was collapsed on replay, so a response sendingVarytwice came back varying only on the second.proxy
Every sanitizer removed nothing. A cross-host redirect carried the caller's
authorization:andcookie:to the redirect target;upgrade,te,keep-aliveand aconnection:-listed header reached the upstream; and the client's ownx-real-iparrived beside the one the proxy writes, so the upstream attributed the request to an address the client chose.The response strip also asked the wrong object: a proxied response is parsed by the outbound
fasthttp.Client, which carries its own setting and stamps it onto that header each hop. A proxy preserving upstream casing in front of a normalizing app stripped nothing.csrf
The origin check was switched off by the case of a header name. With a valid double-submit pair,
Origin: http://evil.com→ 403, butorigin: http://evil.com→ 200. An absent Origin is not a failure there — on a plaintext connection the check is skipped for it entirely.others
extractors.FromHeader: a token sent under the lower-case name was not found, and the request refused for carrying no token when it carried one.${ips}logged an empty chain while the trust decisions went on using the header it could not see;RegisterContextTagwrote its value raw, where a CR or LF forged a log line.net/httpmiddleware's header edits were lost — aHeader.Setcollapsing a multi-valued header left the other values in place, a middleware that changed nothing duplicated the last value of every multi-valued header, and removals never propagated at all.boundary=case-sensitively, so a legalMultipart/Form-Databound nothing and reported no error.Scheme(): onlyhttpandhttpsare accepted from a proxy header. The value is spliced into a URL byBaseURLand compared for origin equality by csrf andRedirect().Back, so no other scheme —javascriptincluded — may become the request's.basicauthandcorswere checked too and fail closed — a lower-case name locks a legitimate client out, never lets one in — so they are left alone and the finding is recorded rather than acted on.Changes introduced
delFieldcost 792 B/op and ran 3× slower on the proxy's default path (a range-over-func iterator allocation per name), so the normalization answer is threaded from config instead — measured back to 0 B/op and the original ns/op. An earlier header dedup inadaptorwas 17% slower at 20 headers and 61% at 100, so it was removed.docs/middleware/cache.md,docs/middleware/proxy.md(an X-Real-IP caution and a two-pass snippet),docs/middleware/logger.md(${ips}),docs/api/ctx.md,docs/extra/internal.md.internal/packages hold the shared primitives so the duplication that caused one of these cannot cause the next:fieldname(case-insensitive read and delete over fasthttp header types),headerlookup(the same for a caller holding aCtx),mediatype(the Content-Type fold every form entry point needs) andcrosshost(the credential list the client and proxy had already let drift apart).Behavior changes worth knowing about
publicors-maxage. Applications that refresh a session cookie on every response will find most routes stop being cached — that is the point, since those responses are per-client.must-revalidatedoes not lift this: RFC 9111 §3.5 accepts it forAuthorizationbecause a revalidating cache re-checks the credential at the origin, and this middleware never revalidates.http/httpsno longer becomes the request's scheme; the previously determined one stands.Type of change
Checklist
/docs/directory for Fiber's documentation.internal/packages)Verification
go build ./...,go vet ./...,go test ./...,gofmt -l .and golangci-lint v2.12.2 (the version.github/workflows/lint.ymlpins) — 0 issues — all run on this branch.Every fix is mutation-verified: reproduced with a probe first, then the fix reverted in a scratch copy, the new test confirmed failing, then restored.
Generated by Claude Code