Skip to content

Painless: script_aware String search + regex limit-factor for replaceAll/replaceFirst - #151086

Merged
jdconrad merged 10 commits into
elastic:mainfrom
jdconrad:painless-cancellation-aware-string-ops
Jun 15, 2026
Merged

jdconrad merged 10 commits into
elastic:mainfrom
jdconrad:painless-cancellation-aware-string-ops

Conversation

@jdconrad

@jdconrad jdconrad commented Jun 10, 2026 •

Copy link
Copy Markdown
Contributor

Cancellation-aware String search augmentations

Reimplements String.indexOf(String), String.indexOf(String, int), String.lastIndexOf(String), String.lastIndexOf(String, int), and String.contains(CharSequence) as @script_aware Augmentation methods with an explicit O(n*m) outer-position scan that polls PainlessScript._pollCancellation() between candidate starting positions.

An attacker-controlled haystack/needle pair (lots of false-positive prefix matches) can no longer escape the search timeout in a single JDK indexOf call. Fast path delegates straight to the JDK when no cancellation check is installed.

Fix 1 — Regex char-limit protection for CharSequence.replaceAll/replaceFirst

These Augmentation entries previously called pattern.matcher(receiver) with the raw receiver, bypassing the LimitedCharSequence character-read cap enforced for the other regex augmentations (Pattern.matcher, Pattern.split, Pattern.splitAsStream). Both methods now take an injected regex_limit_factor and wrap the receiver in LimitedCharSequence, closing a preexisting protection gap where a catastrophic-backtracking pattern in a script could run unbounded.

Fix 2 — Per-match cancellation polling in replaceAll

Adds @script_aware to CharSequence.replaceAll(Pattern, Function) so the augmentation polls between matches. Many-match sweeps now honour the search timeout independent of the per-match regex work.

replaceFirst is not @script_aware (a single match doesn't benefit from per-match polling) but still gets Fix A's LimitedCharSequence wrapping.

Infrastructure fixes (needed for @script_aware + @inject_constant)

The combination was not used together before this PR. Two places needed updates so the runtime stack matches the resolved Java method signature [PainlessScript, receiver, injections..., user args]:

  • DefaultUserTreeToIRTreePhase.visitCall: when the resolved method is both @script_aware and has @inject_constant, the script-aware receiver is added to the invoke arg list before the injected constants. Also widens augmentedOffset to skip the PainlessScript slot when indexing parameterTypes for the injection type check.
  • Def.lookupMethod: when a @script_aware method has injections, insertArguments targets position 2 (after PainlessScript and receiver) rather than position 1.

…All/replaceFirst

Adds three pieces:

Step 7 — Cancellation-aware String search augmentations
  Reimplements String.indexOf(String), String.indexOf(String, int),
  String.lastIndexOf(String), String.lastIndexOf(String, int), and
  String.contains(CharSequence) as @script_aware Augmentation methods
  with an explicit O(n*m) outer-position scan that polls
  PainlessScript._pollCancellation() between candidate starting
  positions.  An attacker-controlled haystack/needle pair (lots of
  false-positive prefix matches) can no longer escape the search
  timeout in a single JDK indexOf call.  Fast path delegates straight
  to the JDK when no cancellation check is installed.

Fix A — Regex limit-factor protection for CharSequence.replaceAll/replaceFirst
  These Augmentation entries previously called pattern.matcher(receiver)
  with the raw receiver, bypassing the LimitedCharSequence character-
  read cap enforced for other regex augmentations.  Both methods now
  take an injected regex_limit_factor and wrap the receiver in
  LimitedCharSequence, closing a preexisting protection gap where a
  catastrophic-backtracking pattern in a script could run unbounded.

Fix B — Per-match cancellation polling in replaceAll
  Adds @script_aware to CharSequence.replaceAll(Pattern, Function) so
  the augmentation polls between matches.  Many-match sweeps now
  honour the search timeout independent of the per-match regex work.
  replaceFirst is not @script_aware (a single match doesn't benefit
  from per-match polling) but still gets Fix A.

Infrastructure fix needed for the @script_aware + @inject_constant
combination, which had not been used together before:
  - DefaultUserTreeToIRTreePhase.visitCall: when the resolved method
    is both @script_aware and has @inject_constant, the script-aware
    receiver must be added to the invoke arg list BEFORE the
    injected constants so the runtime stack ([PainlessScript, receiver,
    injections..., user args]) matches the Java method signature.
    Also widens augmentedOffset to skip the PainlessScript slot when
    indexing parameterTypes for the injection type check.
  - Def.lookupMethod: when a @script_aware method has injections,
    insertArguments must target position 2 (after PainlessScript and
    receiver) rather than position 1.

Tests cover all three pieces: per-method fires-cancel-runnable tests
for the String search methods, a circuit-breaker test for the regex
limit factor, a per-match polling test for replaceAll, no-runnable
fast-path tests, and an end-to-end YAML test for indexOf under a
100 ms search timeout.
@jdconrad jdconrad added >enhancement :Core/Infra/Scripting Scripting abstractions, Painless, and Mustache labels Jun 10, 2026
@elasticsearchmachine

Copy link
Copy Markdown
Collaborator

Hi @jdconrad, I've created a changelog YAML for you.

@github-actions

github-actions Bot commented Jun 10, 2026 •

Copy link
Copy Markdown
Contributor

🔍 Preview links for changed docs

⏳ Building and deploying preview... View progress

This comment will be updated with preview links when the build is complete.

@github-actions

Copy link
Copy Markdown
Contributor

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

@prdoyle prdoyle left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My comments below are on code I presume we just lifted out of the JDK implementation, so probably moot.

return Math.min(start, n);
}
int limit = n - m;
for (int i = start; i <= limit; i++) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has no special handling for UTF-16 surrogate pairs. I suspect this is ok given how indexOf works but might be worth a double-check, and perhaps a few emojis in the test cases.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This matches the behavior of the JDK afaik.

Comment on lines +1682 to +1683
int n = receiver.length();
int m = search.length();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are some confusing variable names. I guess they came from the indexOf implementation?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll fix this.

@jdconrad
jdconrad marked this pull request as ready for review June 12, 2026 16:51
@jdconrad
jdconrad requested a review from a team as a code owner June 12, 2026 16:51
@elasticsearchmachine elasticsearchmachine added the Team:Core/Infra Meta label for core/infra team label Jun 12, 2026
@elasticsearchmachine

Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@jdconrad

Copy link
Copy Markdown
Contributor Author

@prdoyle We match the JDK behavior, but the code is our own, not lifted from the JDK.

@prdoyle

prdoyle commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Yeah, given the contracts of these methods, there are only so many reasonable implementations.

jdconrad added 7 commits June 12, 2026 10:23
Guard negative fromIndex in lastIndexOfWithPolling to match the JDK
(returns -1 regardless of needle), rename the scan locals to be
descriptive, and add a parity test covering surrogate pairs, empty
needles, and out-of-bounds fromIndex against the JDK.
Cover the polling-branch output of replaceAll, def-receiver dispatch for
the string-search and regex augmentations (capturing lambdas, method
references), the unlimited regex-limit-factor branch, scan edge cases
(needle longer than haystack, non-String CharSequence contains), and a
replaceAll search-timeout yaml test.
@jdconrad
jdconrad merged commit 3eb278f into elastic:main Jun 15, 2026
36 checks passed
valeriy42 pushed a commit to valeriy42/elasticsearch that referenced this pull request Jun 18, 2026
…All/replaceFirst (elastic#151086)

Adds cancellation-aware String search augmentations
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Core/Infra/Scripting Scripting abstractions, Painless, and Mustache >enhancement Team:Core/Infra Meta label for core/infra team v9.5.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants