Skip to content

Catch DateTimeException in EsqlDataTypeConverter - #137744

Merged
thecoop merged 12 commits into
elastic:mainfrom
thecoop:esql-date-time-exceptions
Jan 2, 2026
Merged

thecoop merged 12 commits into
elastic:mainfrom
thecoop:esql-date-time-exceptions

Conversation

@thecoop

@thecoop thecoop commented Nov 7, 2025

Copy link
Copy Markdown
Member

DateTimeException is not a fatal exception, it is an input error, so convert it to IllegalArgumentException

Fixes #137741

@thecoop

thecoop commented Nov 7, 2025

Copy link
Copy Markdown
Member Author

I'm not familiar with ESQL to know how this should be tested

@elasticsearchmachine elasticsearchmachine added v9.3.0 Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) labels Nov 7, 2025
@elasticsearchmachine

Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@elasticsearchmachine

Copy link
Copy Markdown
Collaborator

Hi @thecoop, I've created a changelog YAML for you.

@ivancea

ivancea commented Nov 11, 2025

Copy link
Copy Markdown
Contributor

Thanks for the fix!

About testing, 2 ways (we usually do both:

  • Each function has unit tests with such cases. For example, DateParseTests. You can add a new test case for the specific failing input (e.g. "2026-02-29"), and expect that it throws a warning and returns a null.
    For example, similar to this one:

cases.add(
new TestCaseSupplier(
List.of(DataType.KEYWORD, DataType.KEYWORD),
() -> new TestCaseSupplier.TestCase(
List.of(
new TestCaseSupplier.TypedData(new BytesRef("yyyy-MM-dd"), DataType.KEYWORD, "first"),
new TestCaseSupplier.TypedData(new BytesRef("not a date"), DataType.KEYWORD, "second")
),
"DateParseEvaluator[val=Attribute[channel=1], formatter=Attribute[channel=0]]",
DataType.DATETIME,
is(nullValue())
).withWarning("Line 1:1: evaluation of [source] failed, treating result as null. Only first 20 failures recorded.")
.withWarning(
"Line 1:1: java.lang.IllegalArgumentException: "
+ "failed to parse date field [not a date] with format [yyyy-MM-dd]"
)
)
);

It should automatically test both folding and runtime evaluating cases.

Then, we would add a CSV test with the failing query (Like ROW a=DATE_PARSE("2026-02-29"), but also one with a FROM, as constants use a different code path), where you can also expect warnings and nulls.
Example:

evalDateParseWrongDate
row a = "2023-02-01 foo" | eval b = date_parse("yyyy-MM-dd", a) | keep b;
warning:Line 1:37: evaluation of [date_parse(\"yyyy-MM-dd\", a)] failed, treating result as null. Only first 20 failures recorded.
warning:Line 1:37: java.lang.IllegalArgumentException: failed to parse date field [2023-02-01 foo] with format [yyyy-MM-dd]
b:datetime
null
;

Also, these affect multiple code paths, which are used by multiple functions too:

  • ToDateTime: ROW a="2026-02-29"::date
  • ToDateNanos: ROW a="2026-02-29"::date_nanos (I'm not sure if this one actually uses your fixed code, so worth checking)
  • Bucket: ROW a="2026-02-01"::date | STATS BY BUCKET(a, 5, "2026-02-29", "2026-02-29") (We should check combinations of one or the other failing individually)
  • TRange: This one requires a @timestamp attribute in the index (You can make the csv tests with the k8s dataset, that should have it), and use something like TRANGE("2026-02-29", "2026-02-29") (Same as Bucket, test combinations)

Those are the ones I found at least, maybe you can track some other important code. It would be nice to have both unit tests and CSV tests for them.

About the fix per se, should we have the check inside the DateFormatter class/sub-class? I think that would fix the ToDateNanos case too, but not sure.

@ivancea
ivancea self-requested a review November 11, 2025 13:47
@thecoop

thecoop commented Nov 13, 2025

Copy link
Copy Markdown
Member Author

DateFormatter should still throw DateTimeException, as that is a general-purpose class that is used in lots of different situations. A DateTimeException might be a server error in one use case, and a user error in another. It's up to the caller to translate the DateTimeException to the right externally-visible error code

@thecoop

thecoop commented Nov 13, 2025

Copy link
Copy Markdown
Member Author

I've added a few tests - not sure how thorough this needs to be though

@thecoop
thecoop requested a review from carlosdelest December 23, 2025 09:12

@carlosdelest carlosdelest left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the fix!

@thecoop
thecoop merged commit 7f8715b into elastic:main Jan 2, 2026
35 checks passed
@thecoop
thecoop deleted the esql-date-time-exceptions branch January 2, 2026 11:10
samxbr pushed a commit to samxbr/elasticsearch that referenced this pull request Jan 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/ES|QL AKA ESQL >bug Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v9.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Weird date parsing resulting in 500

4 participants