Repository navigation
feat: show confidence level on AI field marker - #9901
Conversation
Data model for AI source tracking, shared by all controllers: the snippets the model read, their page and rectangle when the source has them, and a confidence level. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Opt-in via FormAIController.setSourceTrackingEnabled. When on, the fill_form tool description asks the LLM to wrap each value it fills in an envelope carrying the snippets it read, their locations, and a confidence level. The reported source is available from FieldValueChangeEvent.getFieldSource and FormAIController.getFieldSource, can be restored after a reload with restoreFieldSource, and lasts as long as the field holds the value it was reported with. Bad source data is dropped and logged, never blocking the value. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Line-removal and operator mutation over the feature's changed lines found untested behavior: rect and page boundary values, the page-region location without a rect, object-shaped extracts, drop-path logging, null-guard messages, stale-source cleanup on value revert, and ValueSource extracts normalization. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A fill_form write carrying no source kept the field's earlier entry, and when the written value matched the one that source was recorded with, getFieldSource described the new write with a citation it never had. Every successful write now replaces the source, clearing it when none was reported. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A page number beyond the int range truncated to its low 32 bits in asInt() and could pass the range check as a wrong page. Whole-number floats (e.g. 2.0) are now accepted, consistent with how integer fields convert values. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Stale entries were dropped only when getFieldSource read them, so an application consuming sources through the change event alone — or a field edited and then removed from the form — accumulated entries for the life of the controller. Behavior of the getters is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The enum said "source" where the wording the LLM sees says "document"; a reader comparing the two saw different meanings for the same level. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When source tracking is on and the model reported a confidence level for a filled value, the automatic field marker now shows it via the marker's confidence indicator. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A fill that writes the value the field already had fires no change event and is never re-marked, but it does replace the source — a marker kept from an earlier turn kept showing a level the new source did not include. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> ^ Conflicts: ^ vaadin-ai-components-flow-parent/vaadin-ai-extensions-flow/src/main/java/com/vaadin/flow/component/ai/form/FormAIController.java
e49c5ed to
a8a5348
Compare
…onfidence # Conflicts: # vaadin-ai-components-flow-parent/vaadin-ai-core-flow/src/main/java/com/vaadin/flow/component/ai/common/PageRegion.java # vaadin-ai-components-flow-parent/vaadin-ai-core-flow/src/main/java/com/vaadin/flow/component/ai/common/Rect.java # vaadin-ai-components-flow-parent/vaadin-ai-core-flow/src/main/java/com/vaadin/flow/component/ai/common/SourceExtract.java # vaadin-ai-components-flow-parent/vaadin-ai-extensions-flow/src/main/java/com/vaadin/flow/component/ai/form/FormAIController.java # vaadin-ai-components-flow-parent/vaadin-ai-extensions-flow/src/test/java/com/vaadin/flow/component/ai/form/SourceTrackingTest.java
|
/code-review |
vaadin-review-bot
left a comment
There was a problem hiding this comment.
Reviewed the changes — left 1 comment and collected 1 finding below that could not be attached to the diff.
| Finding | |
|---|---|
| 📝 | The confidence indicator text cannot be localized from Flow |
| 🧪 | No test covers the new applyValue path setting a level on a reused marker |
📝 The confidence indicator text cannot be localized from Flow
The web component renders the level as visible text taken from i18n.confidence.{low,medium,high}, and puts it into the field's aria-describedby. FieldMarkerI18n maps only message, revert, badgeLabel and badgeTooltip, so the deep merge in I18nMixin always falls back to the English defaults for the new text — a fully localized app still shows "High confidence".
This also makes two javadocs over-promise:
FieldMarkerI18n: "Use to localize the marker; any text leftnullfalls back to the built-in English default."setFieldMarkerEnabled… rathersetFieldMarkerI18n: "Sets the texts shown by the AI field marker — the "AI" badge, its tooltip, and the popover with the revert control — replacing the built-in English defaults."
FieldMarkerI18n.java:26 ·documentation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Added a nested FieldMarkerI18n.Confidence class with low/medium/high texts mapping to the web component's i18n.confidence object, and updated the over-promising javadocs |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
| FormFieldMarker.add(element, fieldMarkerI18n); | ||
| // Applied on every mark so a refill without a source clears the | ||
| // indicator a previous fill set on the reused marker. | ||
| FormFieldMarker.setConfidence(element, getFieldSource(field) |
There was a problem hiding this comment.
Nit: we can maybe add a test for this. It seems like all the tests still pass without this line.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
|
This ticket/PR has been released with Vaadin 25.3.0-alpha9. |



Summary
Depends on #9900 and vaadin/web-components#12436.
Fixes https://github.com/orgs/vaadin/projects/103/views/3?pane=issue&itemId=226179925
🤖 Generated with Claude Code