[simlarsen]

What this does

Traces gets the AI section the Profiler got in #5228: an AI tab between Insights and Setup Guide, with Insights, Logs and Settings under a side menu of its own. Every hour OneUptime reads each service's last hour of spans and files three kinds of AI insight; OneUptime AI then looks into each and opens a pull request in the repository that holds the code. Nothing is ever merged.

Finding	Filed when	The pull request
Slow Operation	P99 of 1 s or more over 20+ spans, and the slowest request repeats a pattern that costs real time (N+1, sequential calls, one dominant call - the same rule as Fix performance with AI)	makes the operation faster
Failing Operation	20+ failed spans, 5% or more, and the newest failures say why (exception or error message)	fixes why it fails
Missing Tracing	500+ spans that are each the root of their trace; a slow request with 60%+ (and 500 ms+) in one span with nothing under it; or failures that record no message and no exception	adds the tracing

How it decides

• Server lane (Common/Types/AI/TelemetryAi): TelemetryAiLane.Trace, switch Project.enableAutomaticTraceRemediation (on for new projects, off for existing ones by migration 1807120000000-AddTraceAiSwitch), trigger entity Trace for AI rules (Match / Investigate / Fix, matched by service).
• Detector (TraceProblemDetector, pure gates in Common/Utils/Traces/TraceInsight): a projection-served first stage (TraceAggregationService.getServiceSpanHealth on proj_hist_by_minute, p99 from proj_agg_by_service), then at most 3 services of each kind and 2 operations each, read closely. One shared look per scan, three detectors.
• Verdict gate (generic, fixAfterVerdictInsightTypes): a failing operation's automatic pull request opens only once the triage classifies it code-fault; a failure from a dependency, the network or bad input gets none unless a person asks.
• No double work: a latency regression from AI Insights defers to this lane while it is on; one Code task per insight; the shared daily cap of 5 automatic pull requests (UTC) and each repository's open-PR cap still apply.
• Fix with AI on a finding (TelemetryAiFixStrip, POST /telemetry-ai/fix-suggestion and /telemetry-ai/fix): the person's own pull request, with a repository picker when the code was not found in one, and every blocker said in place.
• Open in Traces on a finding opens the Viewer on its service and operation (failed spans only for failures) for the hour it was read in.

Words and docs

• 29 new Dashboard strings, translated into the 16 other languages.
• The Traces Explorer docs page gets the AI tab and a "Find and fix slow and failing operations with AI" section in all 17 languages; TracesAiDocsFacts pins its numbers to TraceInsight.

Tests

• Unit and server: TraceInsight, TraceProblemDetector, TraceInsightFix, TelemetryAiFix, TelemetryAiFixSuggestion, the span health read, the lane's rules, routing, triage framing, the API routes, defaults and rule checks.
• Dashboard component tests: the tabs, the side menu, Fix with AI on a finding, the explorer link and the breadcrumbs.
• Docs: TracesAiDocsFacts plus the existing explorer facts and translation suites (all 426 docs suites pass).
• Offline browser suite test-traces-ai-ui (new, App Test group 1 since the UI review): 40 tests at 1280 and 390, light and dark, about 1.5 minutes with its bundle.
• Migration run from scratch on a throwaway Postgres, and the schema drift check: no drift.

Follow-ups (not in this PR)

• Create Monitor on a trace finding is disabled with the generic reason; a Failing Operation could prefill a Traces monitor.

UI review

Looked at, for real: Traces → AI → Insights (list, empty, empty while the switch is off, Enable AI off), a finding's page for each kind (slow, failing with its fix task, missing tracing) and every Fix with AI state (offered, a repository to pick, connect a repository, no AI credits, already running, refused, started), Logs (list, Load More, empty, failed), Settings (switch on and off, Enable AI off, no repository, no credits, More settings with the AI rules table, the create rule dialog step by step and the edit dialog), Monitor Create opened from a finding, and the Profiler's finding page beside it. At 1280 and 390 (and 768, 1024 and 1600 for the finding page), light and dark, in English, German, Persian (right to left), Japanese and Russian - the production pages in the offline fixture, with Inter; Monitor Create in the MonitorOverview fixture with a finding answered.

What changed, and why

• Create Monitor works on a failing operation. It was greyed out on all three kinds with "Monitors cannot be created from this kind of insight yet". A failing operation now opens Monitor Create on a Traces monitor that counts its failed spans - its span name, the Error status, its service, the past hour - at the detector's own floor of 20 failed spans, opening an incident (and clearing it) as every insight-made monitor does. The operation's name reaches only the span name filter, never the monitor's name, description or incident text (the rule the prefill already keeps for exception messages). A slow operation keeps the button, disabled, with a reason of its own: "Monitors cannot alert on how long an operation takes yet. To be alerted when it slows down, create a Metrics monitor on your service's request duration metric." Missing tracing draws no Create Monitor at all: a monitor could never count what a trace lacks, so a button that can never be pressed was only clutter (isAIInsightMonitorOffered).
• Monitor Create from any insight, link or template was broken on master. The criteria step drew "props.initialValue.toJSON is not a function" and an empty, required Default Monitor Status: since feat(values): archive and unarchive states, severities, monitor statuses, labels and incident roles #5161 the archived-values read called toJSON() on steps that arrive as plain JSON. It reads them as text now, whatever their shape (Components/Form/Monitor/MonitorSteps.tsx, one line). Found by rendering the new Traces monitor; the error-log spike prefill showed the same.
• The evidence a finding shows is the detector's own words. The offline fixture showed made-up prose, so the real text had never been looked at. It is now built by a pure buildTraceInsightDetail (Common/Utils/Traces/TraceInsight) that the detector writes with and the fixture draws with. What reading it found: a service's look wrote "p50 0 ms" (it reads only the p99), list items opened in lower case ("- the same database query ran..."), "Request read: trace ..." said nothing a person could act on (now "Slowest request" or "A failed request"), the code locations sat inside the repeated-pattern list, and "In 5 of the 5 most recent failures" reads "In each of the 5 most recent failures".
• Fix with AI in a finding page's narrow rail. With a repository to pick, the picker and the button sat side by side in the rail beside the evidence at 1280 and squeezed the button into "Fix / with / AI" and the placeholder into two lines - on the Profiler's finding page too. Both strips now lay out by their own width (a container query, FIX_STRIP_* in TelemetryAiFixCopy): stacked in the rail and on a phone, side by side where the strip is wide (1024), the button's words always on one line.
• No promise of a pull request for every problem. The switch and the More settings and AI rules lines said OneUptime AI "opens a pull request for each" problem, while a failing operation's waits for the code to be found at fault and a fix may find nothing to change: "OneUptime AI fixes what it can with pull requests", "...opens a pull request for each one it can fix". The docs page says it the same way.
• Docs (Traces Explorer, 17 languages): the AI rules line as above, and "On a failing operation, Create Monitor sets up a Traces monitor on its failed spans." TracesAiDocsFacts pins both to the code.
• Fixture fidelity: the failing finding carries its sample trace as the detector stores it, the slow one no failed-span count it never has, the log entry a real verdict ("code-fault"), and Monitor Create is a page to land on.

New words: 10 new strings (the Traces monitor and its two refusals) and 4 rewritten ones, translated in all 16 languages; the 4 replaced keys are gone from all 17 files. i18n:check and i18n:validate are clean.

Tests added or changed

• Common/Tests/Utils/Traces/TraceInsight.test.ts: the evidence per kind - facts first, code before the patterns, a capital only on OneUptime's own words, no "p50 0 ms", "In each of", "Slowest request" / "A failed request", escaping.
• App/Tests/Dashboard/AIInsightMonitorPrefill.test.ts: the failing operation's Traces monitor (scope, Error status, hour, floor, words, severity, no service id), a saveable prefill (fromJSON, validation, criteria pair, no telemetry text in any sentence), the two refusals, isAIInsightMonitorOffered, the hour is an option of the trace form.
• Common/Tests/App/Dashboard/AIInsightMonitorPrefillContracts.test.ts: the floor is the detector's (pickFailingOperations files at 20, not 19), the step query the worker compiles, TraceMonitorCriteria fires at 20 and clears at 19, and the steps form keeps the prefill as built.
• AIInsightViewCreateMonitor.test.tsx: failing - enabled, to Monitor Create with the finding; slow - disabled with its reason; missing tracing - no button.
• MonitorCreateFromAIInsight.test.tsx: the real Monitor Create page opens on the Traces monitor, and says the slow operation's reason instead of a form.
• MonitorStepsDefaultStatusAdvanced.test.tsx: steps prefilled as JSON load their statuses, with no error.
• TelemetryAiFixCopy.test.ts: the strip layout is keyed to its container, never the window; the button never wraps; both strips use it. TelemetryAiCopy.test.ts: no line promises a pull request for each problem. TraceProblemDetector.test.ts: the evidence line as written now.
• App/Tests/FeatureSet/Docs/TracesAiDocsFacts.test.ts: the two new docs statements, pinned to the copy and the prefill.
• Offline browser specs: E2E/TracesAi/TracesAi.spec.ts gains 3 tests at 1280 and 390 (the evidence as written, Create Monitor on each kind, the picker stacked in the rail and side by side at 1024) and the new copy; E2E/Profiles/ProfilesAi.spec.ts gains the Profiler finding page's picker. Traces AI: 40 tests, 1.5 minutes with its bundle locally (from 1.3). It now runs in App Test group 1, not 8: group 8 took 22.4 minutes on master's last green run without it (group 1: 14.9), so with it group 8 would have passed the 22-minute mark; group 1 is about 16.5 now. Profiles: about 3 seconds more in group 7.
• Mutation checks: 8 breaks, one at a time (the JSON steps read, the missing-tracing button, the slow reason, the failing monitor, its status filter, "p50 0 ms", the lower-case item, "each of"), each caught.

Verification (after merging master c2af2b0): type-checks clean for Common, Common compile-tests (all three programs), the Dashboard, App (its Tests included; --listFiles names nothing outside App and Common) and E2E. Common jest: the guard suites plus every suite that reads a changed file, the locale suites and the guard-like suites the batch named - 297 suites, 10,152 tests, all pass. App jest: the root-level guards, DashboardLocalesGuard, CardDescriptionBoilerplateGuard, the locale suites and the Dashboard and docs suites that read the changed files - 66 suites, 2,560 tests. Docs: the five suites that read the Traces Explorer page (2,155 tests) and the general Tests/FeatureSet/Docs/Docs suites (29 suites, 1,419 tests); docs:check-anchors, i18n:check and i18n:validate clean. Changed suites again with ee/ hidden (346 tests) and with BILLING_ENABLED=true (156). Repo-root Tests/Ops: 71 suites pass. Offline browsers: Traces AI 40/40, Profiles 71 passed (1 phone skip, as before). Prettier and eslint clean on every changed .ts/.tsx (the fixture .js files are eslint-ignored, as every fixture's). ReDoS checker: no new patterns. Then master moved (#5233): merged again (the 17 locale files by key - each side's new keys, none changed on both), i18n:check and i18n:extract clean, the Common, Dashboard and App type-checks clean, Common's guards, locale suites and the changed suites (222 suites, 7,926 tests) and App's root-level, locale, locale-guard and docs suites (65 suites, 4,485 tests) all pass, Tests/Ops 71 pass. Not run: a live stack.

Seen on master, not this PR's: #5233 (merged while this review ran) grew test-security-operations-ui to 92 tests, 9.9 minutes locally by its own account, in group 8 - which took 22.4 minutes before it. Group 8 may now run near the job's 30 minutes; worth watching on master's next App Test run.

Left as they are: Logs do not show an investigation's verdict (why a failing operation got no pull request) - the entry carries it, but showing it needs words for each verdict in both lanes: a follow-up. The Traces monitor type's own card says "Alert on span latency and error rate", while trace monitors count spans - not this PR's. Some AI Insights words are still English in most locales ("Needs Attention", "Fix Opened", the Details labels) - the AI Insights product's backlog, as the Profiler's review noted.

🤖 Generated with Claude Code

[simlarsen]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.