Repository navigation
Add panama simd implementation of contains function for BinaryDocValuesContainsTermQuery - #143922
Merged
Merged
Conversation
Add a vectorized substring search using the Panama Vector API. The SIMD implementation broadcasts both the first and last bytes of the search term, compares them against vector-sized chunks in parallel, and only performs full verification at candidate positions where both bytes match. Made-with: Cursor
Made-with: Cursor
Replace Arrays.mismatch with a direct byte loop for candidate verification. Fall back to scalar for values shorter than 24 bytes where SIMD overhead exceeds its benefit. Remove redundant benchmark method that always used the scalar fallback. Made-with: Cursor
Collaborator
|
Hi @parkertimmins, I've created a changelog YAML for you. |
Contributor
Author
|
Buildkite benchmark this with clickbench-columnar-mode please |
Use Panama ByteVector to compare middle bytes in chunks
instead of a scalar byte-by-byte loop. Also add termLen=50
to the benchmark to exercise the SIMD path.
Results show no measurable difference because middleBytesMatch
is called too infrequently (first+last byte filter eliminates
~99.99% of candidates) and on AVX2 (32-byte vectors) the SIMD
loop only engages for termLength >= 34.
Benchmark results (ops/us, higher is better):
termLen valueLen panamaSimd scalar
5 20 76.6 89.8
5 35 82.6 52.3
5 100 55.7 22.8
5 500 35.5 5.2
5 1000 20.9 2.8
10 100 55.2 18.9
10 500 27.7 4.8
10 1000 18.0 2.6
20 100 53.0 20.2
20 500 23.4 4.5
20 1000 15.2 2.6
50 500 23.9 4.2
50 1000 14.0 2.3
Made-with: Cursor
This reverts commit a16f623.
parkertimmins
force-pushed
the
parker/simd-contains
branch
from
March 10, 2026 02:59
c4038db to
4afb88d
Compare
Contributor
Author
|
Results of a micro benchmark: |
Contributor
Author
|
Buildkite benchmark this with clickbench-columnar-mode please |
1 similar comment
Contributor
Author
|
Buildkite benchmark this with clickbench-columnar-mode please |
Contributor
Author
|
Buildkite benchmark this with clickbench-columnar-mode please |
Contributor
Author
|
Buildkite benchmark this with clickbench-columnar-mode please |
Collaborator
💚 Build Succeeded
This build ran two clickbench-columnar-mode benchmarks to evaluate performance impact of this PR. History
|
Contributor
Author
parkertimmins
marked this pull request as ready for review
March 30, 2026 14:12
Collaborator
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
Member
|
🚀 nice results! |
ChrisHegarty
approved these changes
Mar 30, 2026
ChrisHegarty
left a comment
Contributor
There was a problem hiding this comment.
Very nice! The code looks clean and well tested. LGTM
mouhc1ne
pushed a commit
to shmuelhanoch/elasticsearch
that referenced
this pull request
Mar 31, 2026
…luesContainsTermQuery (elastic#143922) Add a vectorized substring search using the Panama Vector API. The SIMD implementation broadcasts both the first and last bytes of the search term, compares them against vector-sized chunks in parallel, and only performs full verification at candidate positions where both bytes match.
ncordon
pushed a commit
to ncordon/elasticsearch
that referenced
this pull request
Apr 1, 2026
…luesContainsTermQuery (elastic#143922) Add a vectorized substring search using the Panama Vector API. The SIMD implementation broadcasts both the first and last bytes of the search term, compares them against vector-sized chunks in parallel, and only performs full verification at candidate positions where both bytes match.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

The
contains()function matches terms that contain a given substring in BinaryDocValuesContainsTermQuery. The existing scalar implementation is based on theStringUTF16.indexOfLatin1Unsafefunction. This change adds a SIMD implementation.