Repository navigation
Painless: script_aware String search + regex limit-factor for replaceAll/replaceFirst - #151086
Conversation
…All/replaceFirst
Adds three pieces:
Step 7 — Cancellation-aware String search augmentations
Reimplements String.indexOf(String), String.indexOf(String, int),
String.lastIndexOf(String), String.lastIndexOf(String, int), and
String.contains(CharSequence) as @script_aware Augmentation methods
with an explicit O(n*m) outer-position scan that polls
PainlessScript._pollCancellation() between candidate starting
positions. An attacker-controlled haystack/needle pair (lots of
false-positive prefix matches) can no longer escape the search
timeout in a single JDK indexOf call. Fast path delegates straight
to the JDK when no cancellation check is installed.
Fix A — Regex limit-factor protection for CharSequence.replaceAll/replaceFirst
These Augmentation entries previously called pattern.matcher(receiver)
with the raw receiver, bypassing the LimitedCharSequence character-
read cap enforced for other regex augmentations. Both methods now
take an injected regex_limit_factor and wrap the receiver in
LimitedCharSequence, closing a preexisting protection gap where a
catastrophic-backtracking pattern in a script could run unbounded.
Fix B — Per-match cancellation polling in replaceAll
Adds @script_aware to CharSequence.replaceAll(Pattern, Function) so
the augmentation polls between matches. Many-match sweeps now
honour the search timeout independent of the per-match regex work.
replaceFirst is not @script_aware (a single match doesn't benefit
from per-match polling) but still gets Fix A.
Infrastructure fix needed for the @script_aware + @inject_constant
combination, which had not been used together before:
- DefaultUserTreeToIRTreePhase.visitCall: when the resolved method
is both @script_aware and has @inject_constant, the script-aware
receiver must be added to the invoke arg list BEFORE the
injected constants so the runtime stack ([PainlessScript, receiver,
injections..., user args]) matches the Java method signature.
Also widens augmentedOffset to skip the PainlessScript slot when
indexing parameterTypes for the injection type check.
- Def.lookupMethod: when a @script_aware method has injections,
insertArguments must target position 2 (after PainlessScript and
receiver) rather than position 1.
Tests cover all three pieces: per-method fires-cancel-runnable tests
for the String search methods, a circuit-breaker test for the regex
limit factor, a per-match polling test for replaceAll, no-runnable
fast-path tests, and an end-to-end YAML test for indexOf under a
100 ms search timeout.
|
Hi @jdconrad, I've created a changelog YAML for you. |
🔍 Preview links for changed docs⏳ Building and deploying preview... View progress This comment will be updated with preview links when the build is complete. |
ℹ️ Important: Docs version tagging👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version. We use applies_to tags to mark version-specific features and changes. Expand for a quick overviewWhen to use applies_to tags:✅ At the page level to indicate which products/deployments the content applies to (mandatory) What NOT to do:❌ Don't remove or replace information that applies to an older version 🤔 Need help?
|
prdoyle
left a comment
There was a problem hiding this comment.
My comments below are on code I presume we just lifted out of the JDK implementation, so probably moot.
| return Math.min(start, n); | ||
| } | ||
| int limit = n - m; | ||
| for (int i = start; i <= limit; i++) { |
There was a problem hiding this comment.
This has no special handling for UTF-16 surrogate pairs. I suspect this is ok given how indexOf works but might be worth a double-check, and perhaps a few emojis in the test cases.
There was a problem hiding this comment.
This matches the behavior of the JDK afaik.
| int n = receiver.length(); | ||
| int m = search.length(); |
There was a problem hiding this comment.
These are some confusing variable names. I guess they came from the indexOf implementation?
|
Pinging @elastic/es-core-infra (Team:Core/Infra) |
|
@prdoyle We match the JDK behavior, but the code is our own, not lifted from the JDK. |
|
Yeah, given the contracts of these methods, there are only so many reasonable implementations. |
Guard negative fromIndex in lastIndexOfWithPolling to match the JDK (returns -1 regardless of needle), rename the scan locals to be descriptive, and add a parity test covering surrogate pairs, empty needles, and out-of-bounds fromIndex against the JDK.
Cover the polling-branch output of replaceAll, def-receiver dispatch for the string-search and regex augmentations (capturing lambdas, method references), the unlimited regex-limit-factor branch, scan edge cases (needle longer than haystack, non-String CharSequence contains), and a replaceAll search-timeout yaml test.
…All/replaceFirst (elastic#151086) Adds cancellation-aware String search augmentations
Cancellation-aware String search augmentations
Reimplements
String.indexOf(String),String.indexOf(String, int),String.lastIndexOf(String),String.lastIndexOf(String, int), andString.contains(CharSequence)as@script_awareAugmentationmethods with an explicit O(n*m) outer-position scan that pollsPainlessScript._pollCancellation()between candidate starting positions.An attacker-controlled haystack/needle pair (lots of false-positive prefix matches) can no longer escape the search timeout in a single JDK
indexOfcall. Fast path delegates straight to the JDK when no cancellation check is installed.Fix 1 — Regex char-limit protection for
CharSequence.replaceAll/replaceFirstThese Augmentation entries previously called
pattern.matcher(receiver)with the raw receiver, bypassing theLimitedCharSequencecharacter-read cap enforced for the other regex augmentations (Pattern.matcher,Pattern.split,Pattern.splitAsStream). Both methods now take an injectedregex_limit_factorand wrap the receiver inLimitedCharSequence, closing a preexisting protection gap where a catastrophic-backtracking pattern in a script could run unbounded.Fix 2 — Per-match cancellation polling in
replaceAllAdds
@script_awaretoCharSequence.replaceAll(Pattern, Function)so the augmentation polls between matches. Many-match sweeps now honour the search timeout independent of the per-match regex work.replaceFirstis not@script_aware(a single match doesn't benefit from per-match polling) but still gets Fix A'sLimitedCharSequencewrapping.Infrastructure fixes (needed for
@script_aware+@inject_constant)The combination was not used together before this PR. Two places needed updates so the runtime stack matches the resolved Java method signature
[PainlessScript, receiver, injections..., user args]:DefaultUserTreeToIRTreePhase.visitCall: when the resolved method is both@script_awareand has@inject_constant, the script-aware receiver is added to the invoke arg list before the injected constants. Also widensaugmentedOffsetto skip the PainlessScript slot when indexingparameterTypesfor the injection type check.Def.lookupMethod: when a@script_awaremethod has injections,insertArgumentstargets position 2 (after PainlessScript and receiver) rather than position 1.