Repository navigation
Fix JSON formatter embedding non-JSON body as raw value - #2326
Conversation
| } | ||
|
|
||
| // Slow path: attempt full JSON validation | ||
| try { |
| if (body == null || body.isEmpty()) { | ||
| return false; | ||
| } | ||
| final char firstChar = body.trim().charAt(0); |
There was a problem hiding this comment.
Might as well throw in the last char too?
|
|
||
| private static class DefaultJsonFieldWriter implements JsonFieldWriter { | ||
|
|
||
| private static final JsonMapper MAPPER = new JsonMapper(); |
There was a problem hiding this comment.
Use input JsonMapper instead?
When a request body has a JSON content-type header but contains
non-JSON content (e.g. Base64-encoded ciphertext, binary data, or
truncated/malformed JSON), the formatters were embedding the raw
body directly using @JsonRawValue / writeRawValue(), producing
malformed JSON log output that breaks Logstash and Elasticsearch
ingestion.
Fix applied to all four formatters:
- JsonHttpLogFormatter (Jackson 3)
- FastJsonHttpLogFormatter (Jackson 3)
- JsonHttpLogFormatterJackson2 (Jackson 2)
- FastJsonHttpLogFormatterJackson2 (Jackson 2)
Each formatter now uses a two-stage validation approach:
1. Fast path: check first non-whitespace character against valid
JSON start characters ({ [ " digit - t f n). Obvious non-JSON
content like ciphertext is rejected immediately at zero parse cost.
2. Slow path: only if the fast check passes, attempt full JSON
validation via readTree(). If parsing fails, fall back to logging
the body as a safely quoted plain string.
This ensures the overall JSON log structure is always valid while
preserving existing behaviour for valid JSON bodies and keeping
the performance impact minimal for the common case.
Also resolves the long-standing TODO comment in
JsonHttpLogFormatterJackson2.prepareBody() regarding missing
JSON validation.
Test assertions in WriteBodyMaxSizeTest updated to reflect the
corrected behaviour: truncated bodies (produced by BodyMaxSizeFilter)
are now correctly logged as quoted strings rather than raw invalid JSON.
Fixes #2318
|
Thanks for the feedback @skjolber! Streaming validation replaced readTree() with a JsonParser-based streaming approach across all four formatters, consuming tokens without building a tree object in memory Also updated two test method names (shouldEmbedInvalid* → shouldNotEmbedInvalid*) and their assertions to reflect that {"name":"Bob"}; is correctly now logged as a quoted string rather than embedded raw. |
|
Fix looks good overall. The two-stage validation approach (fast char check + streaming JsonParser) is efficient — avoids building a full tree in memory. Covers all four formatters consistently. |
|
@kasmarian could you please add the bugfix label? I don't have permission to add labels as an external contributor, and it's currently blocking the COMPR/pr-labels check. |
|
@Vinu2111 there is a lot of duplicated code. Also see comment on issue. |
|
@skjolber thanks for the feedback. I've addressed your three inline comments switched to streaming parser, added last char check, and injected the mapper via constructor. |
|
Zero code duplication should be the goal. Maybe the maintainers could weigh in here. Validating JSON payload is pure overhead when
The JSON validation step should at least be possible to toggle based on payload, i.e. as a feature and possibly per request/response too. |
|
I second the performance concern from @skjolber here. Adding an additional json parsing to all payloads by default will have a performance penalty, where most of the clients may not be OK with. Also the challenge of invalid json payloads is something I'd expect to be exceptional, rather normal. @AllArgsConstructor shouldn't be a problem in terms of backwards compatibility as long as you preserve current constructors and overload them with the feature toggle(s). The latter one will be used by Spring module that'll pass the state of the toggle(s). |
Extract shared JsonUtil validation helper per module. Add opt-in validateJsonBody flag (default: false) to all four formatters and wire it through LogbookProperties for Spring Boot users. Fixes #2318
|
@skjolber @kasmarian refactored based on your feedback. Extracted shared JsonUtil helper per module (zero duplication). Added opt-in validateJsonBody flag defaulting to false existing behavior unchanged. Wired through LogbookProperties for Spring Boot users. All 213 tests pass. |
| private final ObjectMapper mapper = new ObjectMapper(); | ||
|
|
||
| @Test | ||
| void shouldAcceptValidJsonStartAndEnd() { |
There was a problem hiding this comment.
Could you convert this to a parameterized test instead? This way it'll give a more readable assertion error when a concrete test variant fails instead of an assertion line.
There was a problem hiding this comment.
same apply for the similar tests here
…alidateJsonBody property
|
Converted the tests to @ParameterizedTest with @MethodSource across both modules, with a per-case .as(...) message so failures now point to the exact input that broke. Also added validate-json-body to the README config table and to spring-configuration-metadata.json so it's auto-discoverable in IDEs. All checks are green now, including update-toc. |
| by [JoJan](https://commons.wikimedia.org/wiki/User:JoJan) is licensed under a | ||
| | `logbook.write.status-code-based` | Enables status-code-aware log levels. Responses with 2xx/3xx are logged at `TRACE`, 4xx at `WARN`, 5xx at `ERROR`. Requests are always logged at `TRACE`. | `false` | | ||
| | `logbook.write.max-body-size` | Truncates the body up to `max-body-size` characters and appends `...`. <br/> :warning: Logbook will still buffer the full body, if the request is eligible for logging, regardless of the `logbook.write.max-body-size` value | `-1` (disabled) | | ||
| | `logbook.write.validate-json-body` | Validates JSON bodies before embedding as raw JSON values. Adds a small parsing overhead per request/response. | `false` | |
There was a problem hiding this comment.
Can we add here (and in the spring-configuration-metadata) half of a sentence on the behavior if the validation fails? (i.e. logged as quoted string)
There was a problem hiding this comment.
Done - added a clause on the failure behavior to the validate-json-body row in the README table. It now reads: "...If validation fails, the body is logged as a quoted string instead. Adds a small parsing overhead per request/response." Also applied the same clarification to spring-configuration-metadata.json for consistency.
| Logbook is ready to use out of the box for most common setups. Even for uncommon applications and technologies, it should be simple to implement the necessary interfaces to connect a | ||
| library/framework/etc. to it. | ||
| # Logbook: HTTP request and response logging |
There was a problem hiding this comment.
Can we revert the pure whitespace changes?
There was a problem hiding this comment.
Confirmed - re-checked the full README diff and there are no pure whitespace-only changes remaining. The diff is now scoped to just the intentional content edit above.
There was a problem hiding this comment.
@Vinu2111 that's not true, have a look at the README.md file in the PR, it has
Lines changed: 1188 additions & 1187 deletions label.
There was a problem hiding this comment.
You're right, sorry about that, my local copy had gotten resaved with LF line endings, causing every line to show as changed. I've reset README.md to the upstream version and re-applied only the two intentional content edits. The README diff is now 3 +/- lines as expected. Thanks for flagging it!
…e LF line endings for TOC
|
👍 |
Fixes #2318
Problem
When a request body has a JSON content-type header but contains non-JSON content (e.g. Base64-encoded ciphertext, binary data, or truncated/malformed JSON), all four JSON formatters were embedding the raw body directly using @JsonRawValue
/writeRawValue(), producing malformed JSON log output that breaks downstream log ingestion (Logstash, Elasticsearch). This also resolves the long-standingTODOcomment inJsonHttpLogFormatterJackson2.prepareBody()` regarding missing JSON validation.Fix
Applied to all four formatters:
JsonHttpLogFormatter(Jackson 3)FastJsonHttpLogFormatter(Jackson 3)JsonHttpLogFormatterJackson2(Jackson 2)FastJsonHttpLogFormatterJackson2(Jackson 2)Each formatter now uses a two-stage validation approach:
{["digit-tfn). Obvious non-JSON content like ciphertext is rejected immediately at zero parse cost.readTree(). If parsing fails, fall back to logging the body as a safely quoted plain string.Behaviour
Test changes
WriteBodyMaxSizeTestassertions updated to reflect corrected behaviour - truncated bodies produced byBodyMaxSizeFilterare now correctly logged as quoted strings rather than raw invalid JSON fragments.