Skip to content

Avoid caching multiple times in doc-partitioning - #142913

Merged
dnhatn merged 8 commits into
elastic:mainfrom
dnhatn:esql-doc-caching
Feb 24, 2026
Merged

dnhatn merged 8 commits into
elastic:mainfrom
dnhatn:esql-doc-caching

Conversation

@dnhatn

@dnhatn dnhatn commented Feb 23, 2026 •

Copy link
Copy Markdown
Member

I have been looking into enabling doc-partitioning by default for ES|QL. There are two major overheads:

  1. Multiple threads executing against the same segment can populate the cache for that segment multiple times.
  2. The current work-stealing queue is not locality friendly.

This PR addresses the first issue by introducing a query cache wrapper for doc-partitioning. Only one thread will populate the cache for a specific leaf; other threads will be blocked and yield execution until the cache is populated, then use the cached iterators to execute the query.

I tested this PR with:

FROM hits 
| WHERE URL LIKE \"google\" AND SearchPhrase != \"\" 
| STATS min_URL = MIN(URL), c = COUNT(*) BY SearchPhrase 
| SORT c DESC
| LIMIT 10
Run Segment partitioning (main) Doc partitioning (main) Doc partitioning (this PR)
1 8080 ms 3028 ms 2974 ms
2 7051 ms 1755 ms 1693 ms
3 7215 ms 1728 ms 1708 ms
4 7140 ms 16548 ms 7051 ms
5 112 ms 102 ms 102 ms

The key difference is run 4, where caching is triggered. Doc partitioning on main took twice as long as segment partitioning due to redundant cache population. With this change, doc partitioning should be better than the default segment partitioning in most cases.

However, we will still need to address the second issue before enabling doc-partitioning for all queries.

@elasticsearchmachine

Copy link
Copy Markdown
Collaborator

Hi @dnhatn, I've created a changelog YAML for you.

@dnhatn
dnhatn marked this pull request as ready for review February 24, 2026 00:49
@dnhatn
dnhatn requested review from martijnvg and nik9000 February 24, 2026 00:49
@elasticsearchmachine

Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Feb 24, 2026
@dnhatn
dnhatn requested a review from parkertimmins February 24, 2026 00:49

@nik9000 nik9000 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lovely! I scanned it tonight but haven't finished. Will look tomorrow.

public long cost() {
throw new UnsupportedOperationException();
}
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@dnhatn
dnhatn requested a review from nik9000 February 24, 2026 07:01

@martijnvg martijnvg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks Nhat!

);
return new WeightAndCache(searcher.createWeight(query, scoreMode, 1), queryCache::blockedOnCaching);
} else {
return new WeightAndCache(ctx.searcher().createWeight(query, scoreMode, 1), unused -> null);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, could you use LuceneSlice.NEVER_BLOCKED here

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

++ I pushed ab6621b

@parkertimmins
parkertimmins self-requested a review February 24, 2026 16:22

@parkertimmins parkertimmins left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Though I can't say I understand it in it's entirety. Definitely worth getting some more eyes.

@dnhatn

dnhatn commented Feb 24, 2026

Copy link
Copy Markdown
Member Author

Thanks friends!

@dnhatn
dnhatn merged commit c01f219 into elastic:main Feb 24, 2026
35 checks passed
@dnhatn
dnhatn deleted the esql-doc-caching branch February 24, 2026 20:08
dnhatn added a commit that referenced this pull request Feb 26, 2026
I have been looking into replacing the head-stealing slice queue with a 
bisection work-splitting queue; however, the improvement is small, which
does not justify the complexity added with the new queue. With slices of
250K documents, the locality and page cache thrashing or even postings
iterating is not too expensive; hence I've decided not to move forward
with that change. I believe the caching fix in #142913 addresses the
remaining concern there.

This change enables doc-partitioning for all queries except for queries 
that are expensive to build the scorer. Specifically, we disable
doc-partitioning for MultiTermQuery and PointRangeQuery.

That means many queries from Kibana with time-range queries will still 
run with segment partitioning. I will look into how to enable
doc-partitioning for such queries later.
PeteGillinElastic pushed a commit to PeteGillinElastic/elasticsearch that referenced this pull request Feb 27, 2026
I have been looking into replacing the head-stealing slice queue with a 
bisection work-splitting queue; however, the improvement is small, which
does not justify the complexity added with the new queue. With slices of
250K documents, the locality and page cache thrashing or even postings
iterating is not too expensive; hence I've decided not to move forward
with that change. I believe the caching fix in elastic#142913 addresses the
remaining concern there.

This change enables doc-partitioning for all queries except for queries 
that are expensive to build the scorer. Specifically, we disable
doc-partitioning for MultiTermQuery and PointRangeQuery.

That means many queries from Kibana with time-range queries will still 
run with segment partitioning. I will look into how to enable
doc-partitioning for such queries later.
dnhatn added a commit that referenced this pull request Mar 9, 2026
A bulk scorer of a Weight can be non-null before caching but become null 
after caching. This happens when the query matches no docs (but not
rewritten to MatchNoDocsQuery), and RoaringDocIdSet is used for caching.
RoaringDocIdSet returns null for no matches, and CachingWeight returns
null for scorerSupplier in this case.

In ES|QL, we assume that once a Weight returns a non-null 
ScorerSupplier, it will never return null on subsequent calls, but this
is not true after caching.

This happened before once but became more frequent with #142913.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/ES|QL AKA ESQL >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v9.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants