Repository navigation
Description
Elasticsearch Version
9.5.5/9.3.0
Installed Plugins
No response
Java Version
OpenJDK 27 (27+35-2325, Oracle Corporation)
OS Version
Chainguard / Linux 5.15.0-1080-gcp (amd64)
Problem Description
We are seeing a reproducible java.lang.StackOverflowError when multiple script_score clauses wrap match queries on semantic_text fields inside the same compound bool query.
The issue appears to be caused by recursive query processing during automatic kNN pre-filtering. A semantic_text match is internally rewritten into a kNN query. While building that query, Elasticsearch inspects the surrounding bool query for conditions that can be applied as automatic pre-filters. When other script_score-wrapped semantic queries are present in the same bool structure, each rewritten kNN query may inspect the surrounding clauses for automatic pre-filtering, including clauses that themselves contain semantic queries.
This creates a recursive cycle roughly like:
kNN → auto pre-filtering → bool/script_score → semantic query → kNN → auto pre-filtering → ...
until the Java call stack is exhausted.
In our reproducer, two semantic script_score clauses inside should work, while adding another semantic script_score in either must or must_not triggers the failure.
The issue is reproducible in both Elasticsearch 9.3.0 and 9.5.5.
This is not limited to a failed search request: the StackOverflowError is treated as a fatal error and terminates the affected Elasticsearch node. In our two-data-node cluster, the shard request was retried on the replica and triggered the same failure there, making the cluster unavailable.
Steps to Reproduce
-
Use an index containing at least two
semantic_textfields, for example:semantic_field_Asemantic_field_B
-
Run the following query with two
script_scoreclauses insideshould:
{
"query": {
"bool": {
"should": [
{
"script_score": {
"query": {
"match": {
"semantic_field_A": {
"query": "semantic query A"
}
}
},
"script": {
"source": "_score >= params.min_score ? 5 * _score : 0",
"params": {
"min_score": 0.7
}
}
}
},
{
"script_score": {
"query": {
"match": {
"semantic_field_B": {
"query": "semantic query A"
}
}
},
"script": {
"source": "_score >= params.min_score ? 5 * _score : 0",
"params": {
"min_score": 0.7
}
}
}
}
],
"minimum_should_match": 1
}
}
}This query works as expected.
- Add another
script_score-wrapped semantic query in amustclause in the same compoundboolquery:
{
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"script_score": {
"query": {
"match": {
"semantic_field_A": {
"query": "semantic query A"
}
}
},
"script": {
"source": "_score >= params.min_score ? 5 * _score : 0",
"params": {
"min_score": 0.7
}
}
}
},
{
"script_score": {
"query": {
"match": {
"semantic_field_B": {
"query": "semantic query A"
}
}
},
"script": {
"source": "_score >= params.min_score ? 5 * _score : 0",
"params": {
"min_score": 0.7
}
}
}
}
],
"minimum_should_match": 1
}
},
{
"script_score": {
"query": {
"match": {
"semantic_field_B": {
"query": "semantic query B"
}
}
},
"script": {
"source": "_score >= params.min_score ? 5 * _score : 0",
"params": {
"min_score": 0.7
}
}
}
}
]
}
}
}-
The query triggers a
java.lang.StackOverflowErrorwhile Elasticsearch is building the query. -
The same behavior can also be reproduced by placing the additional semantic
script_scoreclause inmust_notinstead ofmust. -
Reproduced on Elasticsearch 9.3.0 and 9.5.5.
Logs (if relevant)
java.lang.StackOverflowError
at org.elasticsearch.index.query.support.AutoPrefilteringUtils.pruneQuery(...)
at org.elasticsearch.index.query.support.AutoPrefilteringUtils.pruneBoolQuery(...)
at org.elasticsearch.search.vectors.KnnVectorQueryBuilder.applyAutoPrefilteringIfEnabled(...)
at org.elasticsearch.search.vectors.KnnVectorQueryBuilder.doToQuery(...)
at org.elasticsearch.index.query.NestedQueryBuilder.doToQuery(...)
at org.elasticsearch.index.query.functionscore.ScriptScoreQueryBuilder.doToQuery(...)
at org.elasticsearch.index.query.BoolQueryBuilder.doToQuery(...)
at org.elasticsearch.search.vectors.KnnVectorQueryBuilder.applyAutoPrefilteringIfEnabled(...)
...
The same sequence repeats recursively until the Java call stack is exhausted.
Elasticsearch then terminates the affected node with:
fatal error in thread [elasticsearch[instance-0000000025][search][T#1]], exiting
Activity
elasticsearchmachine commentedon Oct 16, 2026
Bug analysis
Verdict: High-confidence hypothesis, based on reading the code on
main(e6ef806037d). I wrote a focused unit test but could not run it here because the local JVM was unavailable.Root cause:
BoolQueryBuilder.addBooleanClausespushes the siblingmust/filter/invertedmust_notclauses intoAutoPrefilteringScopebefore converting each clause.KnnVectorQueryBuilder.applyAutoPrefilteringIfEnabledthen callstoQuery()on each pruned prefilter while that scope is still active. The recursion check only skips a prefilter when it is the knn query itself (this == scopedPrefilter.query()).AutoPrefilteringUtils.pruneQuerydoes not look insidescript_score, so thenestedquery thatsemantic_textproduces stays inside it unpruned. So when sibling knn A buildsscript_score(nested(knn B)), knn B sees that samescript_scoreas its own prefilter and builds it again, which repeats until the stack overflows.shouldclauses are never collected as prefilters, which is why theshould-only query works.Reproduced: Not executed (local
javagave permission denied). These test cases are inreproduction.patch, inAbstractKnnVectorQueryBuilderTestCase, and run viaKnnFloatVectorQueryBuilderTests:testAutoPrefilteringWithScriptScoreWrappedKnnInMustDoesNotRecursetestAutoPrefilteringWithScriptScoreWrappedKnnInMustNotDoesNotRecursetestAutoPrefilteringWithConstantScoreWrappedKnnInMustDoesNotRecurse. Reading the code,constant_score/function_scorewrappers should trigger the same loop.Recommended fix:
AutoPrefilteringScope.suspend()). A prefilter is only used for filtering and does not need prefiltering itself. This is sketched incandidate-fix.patch, which has not been tested.pruneQuerylook insideScriptScoreQueryBuildersonested(semantic) prefilters are pruned there as well.Risk: Blocker. A valid user query raises an uncaught
StackOverflowErrorthat kills the node (fatal error ... exiting). Any user who can search can trigger it, and it affects 9.3.0 and later.Additional notes
holiholiholiholi: theelasticorg membership lookup returned 404, so I classified them as an external contributor. This is inferred, not verified.AbstractQueryTestCase, building the bool query with aSearchExecutionContext), because the loop is purely in query-builder-to-Lucene conversion. An internal cluster test withsemantic_text(in the inference plugin) would be more faithful but much more expensive../gradlew :server:test --tests "org.elasticsearch.search.vectors.KnnFloatVectorQueryBuilderTests.testAutoPrefilteringWith*"script_scoreclauses inshould, or use separate queries or a retriever, until this is fixed.elasticsearchmachine commentedon Oct 16, 2026
This issue has been labeled
blocker, which will prevent Elasticsearch Serverless from being promoted until the blocking condition is resolved.To resolve this blocker, all of the following must be true:
Fixes #NNN) in the PR descriptionNote
If no linked PRs are found, the blocker cannot be resolved even if the issue is closed.
Optional — narrow the scope with
Introduced in:If the bug was introduced in a specific commit, add the following line to this issue's description:
This tells the promotion check that builds predating this commit are unaffected and should not be blocked. For the commit SHA, both the short (7-character) and full (40-character) SHA are accepted.
For more information, see the Promotion Blocker documentation.
elasticsearchmachine commentedon Oct 16, 2026
Pinging @elastic/es-search-relevance (Team:Search Relevance)
Prevent SO in automatic KNN prefilter (#162315)
Prevent SO in automatic KNN prefilter (#162315) (#162574)
Prevent SO in automatic KNN prefilter (#162315) (#162576)