Releases: prometheus/prometheus
Releases · prometheus/prometheus
Release list
3.15.0-rc.0 / 2026-09-09
Immutable
release. Only release title and notes can be modified.
- [CHANGE] PromQL: A range query whose
endwas not aligned tostepcaused subqueries inside it to evaluate past the parent's last actual step, inflatingpeakSamplesin the query stats and against thequery.max-sampleslimit, and wasting storage I/O reading samples that were never used in the result. Add tests to prevent regression of the fix made in #18081. #18598 - [CHANGE] PromQL: Do not register a start timestamp reset if the start timestamp hasn't changed between subsequent samples. #19454
- [CHANGE] Logging: Deprecate
--log.level; useruntime.log_levelconfiguration to supply the default level. #19511 - [FEATURE] Configuration: Allow changing the process log level through
runtime.log_levelon configuration reload. #19511 - [FEATURE] Prometheus: Add
--auto-gomemlimit.refresh-intervalflag to periodically re-detect the container or system memory limit and updateGOMEMLIMITat runtime. #18843 - [FEATURE] Scraping: Add support for scraping targets via Unix Domain Sockets. #12024. #18091
- [FEATURE] scrape: Implement OM2.0 scrape format. #18606
- [ENHANCEMENT] Reduce TSDB head CPU utilization when initializing. #18001
- [ENHANCEMENT] Docker SD: Add labels
__meta_docker_container_imageand__meta_docker_container_image_id. #19386 - [ENHANCEMENT] Mixin: Add a p95/p99 remote-write send-batch latency panel to the remote-write dashboard. #19500
- [ENHANCEMENT] Mixin: Support native histograms in the remote-write send-batch latency panel. #19522
- [ENHANCEMENT] PromQL/TSDB: The
--enable-feature=st-storageflag now automatically enables XOR2 float chunk encoding and ST-capable histogram chunk encoding, so you no longer need to passxor2-encodingandhistograms-st-encodingalongside it. #19518 - [ENHANCEMENT] Remote write / Alertmanager: upgrade sigv4 to v0.5.0, adding
session_nameandtagsfields for STS AssumeRole sessions. The previously undocumentedservice_namefield is now also documented. #19569 - [ENHANCEMENT] Scraping: Support zstd-compressed scrape responses, enabled via feature flag
zstd-scrape. #19502 - [ENHANCEMENT] TSDB: Stabilize the XOR2 float chunk encoding.
--enable-feature=xor2-encodingis deprecated; usestorage.tsdb.chunk_encoding.floats: xor2instead. Check that other software reading the TSDB directly (e.g. Thanos sidecar) supports XOR2 before enabling. #19461 - [ENHANCEMENT] TSDB: add
prometheus_tsdb_head_appenders_created_totalmetric. #19411 - [ENHANCEMENT] Tracing: add more spans to scrapes, API queries and rule evaluations. #19410
- [ENHANCEMENT] UI: Show the effective configuration for each scrape pool on the Targets and Service Discovery pages. #19384
- [ENHANCEMENT] scrape: Enable start time synthesis for summary
_countand_sumseries in scrape appender v2. #19323 - [ENHANCEMENT] scrape: stop all pools in parallel for faster shutdowns. #19295
- [ENHANCEMENT] storage/remote: Add undocumented failed_request_logging config field to debug log remote write V2 requests on send errors. #19249
- [ENHANCEMENT] TSDB: Add fast path for XOR chunk decompression to speed up queries. #18049
- [ENHANCEMENT] UI: Improve native histogram table formatting and add a background bar indicating the bucket count. #19332
- [ENHANCEMENT] TSDB: Add
prometheus_tsdb_head_series_pending_commit_underflow_totalto report pending-sample reservation underflows. #19470 - [PERF] AWS SD: Build RDS cluster labels once per cluster instead of once per instance. #19504
- [PERF] AWS SD: Describe RDS instances of different clusters concurrently, bounded by
request_concurrency. #19506 - [PERF] AWS SD: Describe each ElastiCache resource once per refresh instead of twice. #19585
- [PERF] Remote read: Avoid cloning labels for sampled reads when no external labels are configured. #19503
- [PERF] Remote write: Reuse OTLP converter scratch state between requests. #19388
- [PERF] scrape: conversion from classic to native histograms should only parse start times when enabled. #19446
- [BUGFIX] PromQL: Fix
info()enrichment for composite expressions with mixed@/offset references or selector-free vector branches, preventing metadata from being evaluated at an unrelated timestamp. #19387 - [BUGFIX] PromQL: Fix info() enrichment when input series use different subsets of identifying labels. #19557
- [BUGFIX] PromQL: Preserve metric-name dropping through the info function when delayed name removal is enabled. #19413
- [BUGFIX] TSDB: Don't silently drop samples when head garbage collection removes a series while it is being appended to. #19272
- [BUGFIX] TSDB: Keep series with uncommitted samples during selected- and stale-series compaction, including when appenders overlap. #19470
- [BUGFIX] TSDB: Do not retain head series after a synthetic start-timestamp zero sample is rejected. #19470
- [BUGFIX] TSDB: Prevent query panics during series eviction after WAL replay. #19664
- [BUGFIX] TSDB: fix potential deadlock between mmapSeriesChunks and gcSeries. #19460
- [BUGFIX] TSDB: fix default block reload interval for custom options. #19368
- [BUGFIX] AWS SD: Do not crash on serverless MSK clusters or MSK clusters without Open Monitoring. #19194
- [BUGFIX] AWS SD: Do not panic when the ElastiCache API omits optional fields of a serverless cache or cache cluster. #19435
- [BUGFIX] AWS SD: Reject non-positive
request_concurrencyinstead of hanging service discovery indefinitely. #19524 - [BUGFIX] Discovery/AWS: Avoid a panic when discovering standalone ECS tasks with custom task groups. #19302
- [BUGFIX] Discovery: Do not panic in AWS Lightsail service discovery when an instance is missing optional fields such as availability zone, blueprint, bundle, name, state or support code. #19324
- [BUGFIX] Discovery: delete the stale
prometheus_sd_last_update_timestamp_secondsseries for a config that is removed on reload. #19131 - [BUGFIX] HTTP: Avoid truncating compressed responses when handlers set Content-Length. #19661
- [BUGFIX] IONOS SD: Do not panic when the API response omits the server, NIC or volume collections, or a server's properties. #19438
- [BUGFIX] Metadata will not affect the number of Remote Write v2 shards. #19218
- [BUGFIX] Mixins: Fix label mismatches that prevented PrometheusHAGroupNotIngestingSamples and PrometheusHAGroupCrashlooping from firing. #19444
- [BUGFIX] Native histograms:
DetectResetno longer misses a counter reset when a populated bucket behind an empty one disappears, which could make histogramrate()/increase()undercount. #19367 - [BUGFIX] Never skip histogram buckets for histogram_stddev and histogram_stdvar functions. #19521
- [BUGFIX] OTLP: Do not abort an entire OTLP payload ingestion if one metric has no datapoints. #19343
- [BUGFIX] PromQL: Fix FastRegexMatcher false-positive match when a capturing group is directly adjacent to a literal (e.g.
.*\|(foo)\|.*). #19516 - [BUGFIX] PromQL: Fix a panic in range selectors using the experimental
anchoredorsmoothedmodifier when the selected series has no samples inside the query window, for example a query evaluated inside a scrape gap. #19431 - [BUGFIX] PromQL: Fix empty results when a subquery with
@is used as the matrix argument of a call that is not step-invariant (for examplequantile_over_time(scalar(x), metric[...:...] @ T)). #19187 - [BUGFIX] PromQL: Make the "found duplicate series for the match group" many-to-many matching error message deterministic by sorting the two duplicate labels. #18810
- [BUGFIX] PromQL: Preserve parentheses around duration literals on Expr.String() round-trip. #19403
- [BUGFIX] PromQL: Reject duration-expression offsets and
@ start()/@ end()before range selectors, matching the existing rejection of literal offsets and@ <timestamp>. #19406 - [BUGFIX] PromQL: Report the position of the histogram argument rather than of a scalar argument in the native histogram NaN observation annotations of
histogram_quantileandhistogram_fraction. #19330 - [BUGFIX] PromQL: info() now applies the @ modifier/offset when evaluating the info series, so info(v @ T) enriches as of T consistently instead of depending on the query start/eval time. #19266
- [BUGFIX] Rules: Fix a panic when a rule manager created without a logger loads a rule file containing multiple YAML documents. #19433
- [BUGFIX] Scrape: Do not append a stale marker for a series that is still exposed when the storage returns a new series reference for it. #19328
- [BUGFIX] Scrape: JSON log formatter correctly format scrape target info. #19472
- [BUGFIX] TSDB: Fix in-order chunk ID overflow by wrapping head chunk IDs modulo 2^23 so they never collide with the out-of-order flag bit. #19450
- [BUGFIX] TSDB: Fix out-of-order chunk ID overflow by wrapping
firstOOOChunkIDmodulo 2^23 instead of growing unbounded. #19216 - [BUGFIX] UI: Remove an extraneous X-axis tick mark in the native histogram chart when using the "linear" display mode. #19326
- [BUGFIX] discovery/aws: Avoid a panic when an ElastiCache ARN is missing its resource ID. #19333
- [BUGFIX] discovery/aws: Do not panic on MSK clusters whose optional API fields are absent. #19584
- [BUGFIX] discovery/aws: Do not panic when the EC2 API omits optional instance fields. #19512
- [BUGFIX] discovery/aws: Don't panic on ECS tasks with absent optional fields. #19396
- [BUGFIX] discovery/aws: MSK Optional Custom Configuration Fields #19422. #19422
- [BUGFIX] discovery/ionos: Fix panic when the IONOS API omits optional server fields. #19418
- [BUGFIX] discovery/kubernetes: Populate
__meta_kubernetes_service_loadbalancer_ipfromstatus.loadBalancer.ingress, falling back to deprecatedspec.loadBalancerIP. #19404 - [BUGFIX] histogram: Fix Compact moving buckets to wrong indices, and producing negative bucket counts for integer histograms, when more than one span is merged in the same pass. #19312
- [BUGFIX] promtool: Fixed
tsdb dumpsilently dropping native histogram samples....
3.13.3 / 2026-09-07
Immutable
release. Only release title and notes can be modified.
- [SECURITY] Bump github.com/klauspost/compress to v1.18.7 (GO-2026-5841) and golang.org/x/crypto to v0.55.0 (GO-2026-6303).
- [BUGFIX] Docker SD: Fix panic in Docker Swarm service discovery when a service runs as a plugin or network-attachment. #19102
- [BUGFIX] PromQL: Fix case-insensitive regex label matchers silently dropping matching values. #19167
- [BUGFIX] Scrape: Fix scrape manager spinning at 100% CPU on shutdown. #19149
- [BUGFIX] Alerting: Fix 100% CPU usage on shutdown that could delay graceful shutdown and trigger timeout-based kills. #17859
- [BUGFIX] TSDB: Fix out-of-order queries blocking compaction for hours, causing memory usage to grow. #19013
- [BUGFIX] TSDB: Fix deleted series causing missing samples and errors after restart. #19140
- [BUGFIX] TSDB: Fix goroutine and file handle leaks when Prometheus fails to open a corrupt TSDB. On Windows, the leaked directory handle also prevented TSDB directory removal. #18291
3.14.0 / 2026-08-17
Immutable
release. Only release title and notes can be modified.
- [CHANGE] API: Deprecate the
statsquery parameter of/api/v1/queryand/api/v1/query_rangefor values other thantrueandall. Other values still enable basic statistics but now return a deprecation warning; they will be rejected in the next major release. #19124 - [CHANGE] API:
/api/v1/status/confignow correctly showsseparator: ""andreplacement: ""in relabel configs when explicitly set to empty, instead of omitting them. #18653 - [CHANGE] Discovery/Hetzner: Drop the
__meta_hetzner_datacenterlabel forhcloudtargets, following its removal from the Hetzner Cloud API. #19269 - [CHANGE] PromQL: Enable duration expressions by default. The
promql-duration-exprfeature flag is now a no-op. #19033 - [CHANGE] PromQL: Promote
first_over_timeto stable. It no longer requires thepromql-experimental-functionsfeature flag. #19093 - [FEATURE] Discovery: Add Oracle Cloud Infrastructure compute service discovery (
oci_sd_configs). #18919 - [FEATURE] PromQL: Add experimental
start_timestamp(instant-vector)function returning the start timestamp of each sample in the given vector. Requires theuse-start-timestampsfeature flag. #19089 - [FEATURE] PromQL: Allow
rate()andincrease()to use start timestamps as an alternative for rate extrapolation. Hidden behind theuse-start-timestampsfeature flag. #18619 - [FEATURE] TSDB: Add experimental support for encoding start timestamps in histograms and float histograms. Hidden behind the
histograms-st-encodingfeature flag. #18609 - [ENHANCEMENT] OTLP: Emit a warning when OTLP attribute names collide into the same Prometheus label after sanitization (e.g.
k8s.pod.nameandk8s_pod_nameboth becomek8s_pod_name), and expose theprometheus_api_otlp_translation_warnings_totalcounter labelled bycategoryto track such warnings. #18957 - [ENHANCEMENT] Promtool: Add
--remote-write.pathflag topush metricsfor backends that use a non-default remote-write endpoint. #19086 - [ENHANCEMENT] Remote write: Forward histogram start timestamps in the remote write V2 protocol. #18903
- [ENHANCEMENT] TSDB: Add
prometheus_tsdb_head_native_histogram_seriesandprometheus_tsdb_head_native_histogram_bucketsgauges tracking the number of native histogram series and buckets in the head. #19170 - [ENHANCEMENT] UI: Add syntax highlighting, autocompletion, and linting for PromQL duration expressions (
step(),range(),min_of(),max_of()) in range selectors and subqueries. #18625 - [ENHANCEMENT] UI: Add copy button next to rule names on the Rules and Alerts pages. #18706
- [ENHANCEMENT] UI: Improve rule group title contrast on the Rules page. #19181
- [PERF] Speed up regex label matchers matching a set of literal values (e.g.
{job=~"foo|bar|baz"}). #18833 - [PERF] Remote read: Improve remote read throughput by removing unnecessary per-write flushing. #18470
- [PERF] Scrape: Parse text and OpenMetrics formats without recursion, preventing stack overflow from deeply nested or malicious exposition input. #19143
- [PERF] Scrape: Reduce native histogram scrape parsing allocations by ~49%. #19282
- [PERF] TSDB: Speed up queries on series with many in-memory chunks. #18300
- [BUGFIX] Alerting: Fix 100% CPU usage on shutdown that could delay graceful shutdown and trigger timeout-based kills. #17859
- [BUGFIX] Discovery/AWS: Stop
promtool check configfrom making AWS metadata service (IMDS) network calls when theregionfield is omitted in EC2, ECS, RDS, MSK, ElastiCache, and Lightsail service discovery configs. #19037 - [BUGFIX] Discovery/Docker: Set a request timeout for
docker_sdanddockerswarm_sdonunix,npipe, andtcphosts. Previously an unresponsive daemon could freeze discovery indefinitely, silently pinning targets to a stale snapshot. #19237 - [BUGFIX] Discovery/Docker: Fix panic in Docker Swarm service discovery when a service runs as a plugin or network-attachment. #19102
- [BUGFIX] Discovery/Docker: Fix discovery of IPv6-only containers. #18778
- [BUGFIX] PromQL: Fix case-insensitive regex label matchers silently dropping matching values. #19167
- [BUGFIX] PromQL: Fix
mad_over_timereturning 0 instead ofNaNwhen the range contains aNaNsample. #19040 - [BUGFIX] Promtool: Accept
--enable-feature=promql-binop-fill-modifiersincheck rules, which previously rejected validfill()/fill_left()/fill_right()expressions. #19153 - [BUGFIX] Remote write: Respect the
AZURE_FEDERATED_TOKEN_FILEenvironment variable for workload identity authentication instead of hardcoding the token file path. #18973 - [BUGFIX] Rules: Clean up stale
rule_group_last_rule_duration_sum_secondsandrule_group_last_restore_duration_secondsseries when a rule group is removed or renamed on reload. Previously each reload leaked two series per dropped group, growing/metricscardinality over time. #19107 - [BUGFIX] Scrape: Fix scrape manager spinning at 100% CPU on shutdown. #19149
- [BUGFIX] TSDB: Fix silent data loss and potential crash loop when
stale_series_compaction_thresholdis used in the config file. #19016 - [BUGFIX] TSDB: Fix potential data loss on restart when out-of-order ingestion is enabled and blocks are compacted. #19016
- [BUGFIX] TSDB: Fix
prometheus_tsdb_head_stale_seriesover-counting and early eviction of series that change between float, integer histogram, and float histogram sample types. #19183 - [BUGFIX] TSDB: Fix goroutine and file handle leaks when Prometheus fails to open a corrupt TSDB. On Windows, the leaked directory handle also prevented TSDB directory removal. #18291
- [BUGFIX] TSDB: Fix out-of-order queries blocking compaction for hours, causing memory usage to grow. #19013
- [BUGFIX] TSDB: Fix deleted series causing missing samples and errors after restart. #19140
- [BUGFIX] TSDB: Fix native histogram data becoming incorrect after restart. #19202
- [BUGFIX] TSDB: Surface query errors that were previously silently discarded. #19120
- [BUGFIX] TSDB: Honour the configured float chunk encoding when compaction rewrites chunks; previously chunks encoded with
--enable-feature=xor2-encodingcould silently revert to XOR after compaction. #19145 - [BUGFIX] UI: Show the delete-series form on the TSDB Status page when
--web.enable-admin-apiis enabled. #19025
3.14.0-rc.0 / 2026-08-06
Immutable
release. Only release title and notes can be modified.
- [CHANGE] API: Deprecate the
statsquery parameter of/api/v1/queryand/api/v1/query_rangefor values other thantrueandall. Other values still enable basic statistics but now return a deprecation warning; they will be rejected in the next major release. #19124 - [CHANGE] API:
/api/v1/status/confignow correctly showsseparator: ""andreplacement: ""in relabel configs when explicitly set to empty, instead of omitting them. #18653 - [CHANGE] Discovery/Hetzner: Drop the
__meta_hetzner_datacenterlabel forhcloudtargets, following its removal from the Hetzner Cloud API. #19269 - [CHANGE] PromQL: Enable duration expressions by default. The
promql-duration-exprfeature flag is now a no-op. #19033 - [CHANGE] PromQL: Promote
first_over_timeto stable. It no longer requires thepromql-experimental-functionsfeature flag. #19093 - [FEATURE] Discovery: Add Oracle Cloud Infrastructure compute service discovery (
oci_sd_configs). #18919 - [FEATURE] PromQL: Add experimental
start_timestamp(instant-vector)function returning the start timestamp of each sample in the given vector. Requires theuse-start-timestampsfeature flag. #19089 - [FEATURE] PromQL: Allow
rate()andincrease()to use start timestamps as an alternative for rate extrapolation. Hidden behind theuse-start-timestampsfeature flag. #18619 - [FEATURE] TSDB: Add experimental support for encoding start timestamps in histograms and float histograms. Hidden behind the
histograms-st-encodingfeature flag. #18609 - [ENHANCEMENT] OTLP: Emit a warning when OTLP attribute names collide into the same Prometheus label after sanitization (e.g.
k8s.pod.nameandk8s_pod_nameboth becomek8s_pod_name), and expose theprometheus_api_otlp_translation_warnings_totalcounter labelled bycategoryto track such warnings. #18957 - [ENHANCEMENT] Promtool: Add
--remote-write.pathflag topush metricsfor backends that use a non-default remote-write endpoint. #19086 - [ENHANCEMENT] Remote write: Forward histogram start timestamps in the remote write V2 protocol. #18903
- [ENHANCEMENT] TSDB: Add
prometheus_tsdb_head_native_histogram_seriesandprometheus_tsdb_head_native_histogram_bucketsgauges tracking the number of native histogram series and buckets in the head. #19170 - [ENHANCEMENT] UI: Add syntax highlighting, autocompletion, and linting for PromQL duration expressions (
step(),range(),min_of(),max_of()) in range selectors and subqueries. #18625 - [ENHANCEMENT] UI: Add copy button next to rule names on the Rules and Alerts pages. #18706
- [ENHANCEMENT] UI: Improve rule group title contrast on the Rules page. #19181
- [PERF] Speed up regex label matchers matching a set of literal values (e.g.
{job=~"foo|bar|baz"}). #18833 - [PERF] Remote read: Improve remote read throughput by removing unnecessary per-write flushing. #18470
- [PERF] Scrape: Parse text and OpenMetrics formats without recursion, preventing stack overflow from deeply nested or malicious exposition input. #19143
- [PERF] Scrape: Reduce native histogram scrape parsing allocations by ~49%. #19282
- [PERF] TSDB: Speed up queries on series with many in-memory chunks. #18300
- [BUGFIX] Alerting: Fix 100% CPU usage on shutdown that could delay graceful shutdown and trigger timeout-based kills. #17859
- [BUGFIX] Discovery/AWS: Stop
promtool check configfrom making AWS metadata service (IMDS) network calls when theregionfield is omitted in EC2, ECS, RDS, MSK, ElastiCache, and Lightsail service discovery configs. #19037 - [BUGFIX] Discovery/Docker: Set a request timeout for
docker_sdanddockerswarm_sdonunix,npipe, andtcphosts. Previously an unresponsive daemon could freeze discovery indefinitely, silently pinning targets to a stale snapshot. #19237 - [BUGFIX] Discovery/Docker: Fix panic in Docker Swarm service discovery when a service runs as a plugin or network-attachment. #19102
- [BUGFIX] Discovery/Docker: Fix discovery of IPv6-only containers. #18778
- [BUGFIX] PromQL: Fix case-insensitive regex label matchers silently dropping matching values. #19167
- [BUGFIX] PromQL: Fix
mad_over_timereturning 0 instead ofNaNwhen the range contains aNaNsample. #19040 - [BUGFIX] Promtool: Accept
--enable-feature=promql-binop-fill-modifiersincheck rules, which previously rejected validfill()/fill_left()/fill_right()expressions. #19153 - [BUGFIX] Remote write: Respect the
AZURE_FEDERATED_TOKEN_FILEenvironment variable for workload identity authentication instead of hardcoding the token file path. #18973 - [BUGFIX] Rules: Clean up stale
rule_group_last_rule_duration_sum_secondsandrule_group_last_restore_duration_secondsseries when a rule group is removed or renamed on reload. Previously each reload leaked two series per dropped group, growing/metricscardinality over time. #19107 - [BUGFIX] Scrape: Fix scrape manager spinning at 100% CPU on shutdown. #19149
- [BUGFIX] TSDB: Fix silent data loss and potential crash loop when
stale_series_compaction_thresholdis used in the config file. #19016 - [BUGFIX] TSDB: Fix potential data loss on restart when out-of-order ingestion is enabled and blocks are compacted. #19016
- [BUGFIX] TSDB: Fix
prometheus_tsdb_head_stale_seriesover-counting and early eviction of series that change between float, integer histogram, and float histogram sample types. #19183 - [BUGFIX] TSDB: Fix goroutine and file handle leaks when Prometheus fails to open a corrupt TSDB. On Windows, the leaked directory handle also prevented TSDB directory removal. #18291
- [BUGFIX] TSDB: Fix out-of-order queries blocking compaction for hours, causing memory usage to grow. #19013
- [BUGFIX] TSDB: Fix deleted series causing missing samples and errors after restart. #19140
- [BUGFIX] TSDB: Fix native histogram data becoming incorrect after restart. #19202
- [BUGFIX] TSDB: Surface query errors that were previously silently discarded. #19120
- [BUGFIX] TSDB: Honour the configured float chunk encoding when compaction rewrites chunks; previously chunks encoded with
--enable-feature=xor2-encodingcould silently revert to XOR after compaction. #19145 - [BUGFIX] UI: Show the delete-series form on the TSDB Status page when
--web.enable-admin-apiis enabled. #19025
3.13.2 / 2026-07-29
Immutable
release. Only release title and notes can be modified.
What's Changed
- [SECURITY] Bump golang.org/x/text to v0.39.0 (CVE-2026-56852) and google.golang.org/grpc to v1.82.1 (GHSA-hrxh-6v49-42gf). #19290 by @krajorama
- [BUGFIX] PromQL: Preallocate the active query tracker file to avoid SIGBUS crashes when the data disk is full. #19289 by @akshajrawat
Full Changelog: v3.13.1...v3.13.2
3.13.1 / 2026-07-10
Immutable
release. Only release title and notes can be modified.
This is a bugfix release for 3.13 LTS.
- [BUGFIX] TSDB: Fix the head-chunk cache returning samples from the wrong chunk, or spurious not-found errors, to range queries after head-chunk truncation. #19134
3.5.5 / 2026-07-09
Immutable
release. Only release title and notes can be modified.
This release is built with Go 1.25.12 and fixes a security issue in a UI dependency.
- [SECURITY] UI: Bump
sanitize-htmlto v2.17.5 to fix CVE-2026-53606. #19060
3.13.0 / 2026-07-01
Immutable
release. Only release title and notes can be modified.
This is a Long Term Support LTS release.
- [SECURITY] UI: Bump
sanitize-htmlto fix a cross-site scripting vulnerability (CVE-2026-44990). #18697 - [CHANGE] UI: Third-party npm dependency licenses are now embedded in the Prometheus binary and served at
/assets/third-party-licenses.txt, replacing thenpm_licenses.tar.bz2archive previously shipped in release tarballs and container images. #18997 - [CHANGE] API: Use SHA-256 instead of SHA-1 to generate rule group pagination tokens. #18927
- [CHANGE] HTTP clients: Credentials (Authorization header, basic auth, bearer token, OAuth2, configured headers) are no longer forwarded when following a redirect to a different host; affects scraping, remote read/write, alerting, and service discovery. Via prometheus/common v0.69.0 (CVE-2025-4673 CVE-2023-45289). #18949
- [CHANGE] promtool: Relative file paths in the file passed to
--http.config.fileare now resolved relative to that config file's directory instead of its parent directory. Via prometheus/common v0.69.0. #18949 - [CHANGE] PromQL: Rename the
min()andmax()duration-expression functions (experimental feature flagexperimental-duration-expr) tomin_of()andmax_of()to avoid confusion with theminandmaxaggregate operators. #18687 - [FEATURE] API: Add experimental search endpoints to search metric names, label names, and label values. #18573
- [FEATURE] Discovery/AWS: Add ability to filter RDS instances. #18859
- [FEATURE] PromQL: Add
min_of(a, b)andmax_of(a, b)scalar experimental functions, returning the smaller or larger of two scalar values. #18687 - [FEATURE] PromQL: Add support for smoothed/anchored rate with native histograms. #18564
- [FEATURE] PromQL: Expose per-query
samplesRead(andsamplesReadPerStepwithstats=alland thepromql-per-step-statsfeature flag) in the query stats response, and add theprometheus_engine_query_samples_read_totalengine counter.samplesReadreflects storage I/O distinct fromtotalQueryableSamples, which counts samples loaded into the evaluator (and so over-counts when a sample is reused across multiple range-vector windows). #18081 - [FEATURE] Scrape: Add
__convert_classic_histograms_to_nhcb__internal label to allow per-target override ofconvert_classic_histograms_to_nhcbscrape configuration via relabeling. #18840 - [FEATURE] TSDB: Add
storage.tsdb.chunk_encoding.floatsconfiguration field to select float chunk encoding (xororxor2) at runtime, independently of the--enable-feature=xor2-encodingflag. #18769 - [FEATURE] remote_write: Add Certificate support for ingesting data into an Azure Monitor Workspace. #18217
- [FEATURE] Scrape: Add
__always_scrape_classic_histograms__and__scrape_native_histograms__internal labels to allow per-target override of thealways_scrape_classic_histogramsandscrape_native_histogramsscrape configuration via relabeling. #18929 - [ENHANCEMENT] Release: Container images are now also published to the GitHub Container Registry (ghcr.io). #18791
- [ENHANCEMENT] PromQL: Prettify
fill_left(x) fill_right(x)asfill(x)when both fill values are equal. #18851 - [ENHANCEMENT] UI: Improve autocompletion after closing a function bracket. #18894
- [PERF] Labels: Add case-insensitive prefix matching to speed up evaluation of long case-insensitive regular expressions (up to ~2x faster). #18540
- [PERF] TSDB: Reduce per-sample overhead in chunk population, speeding up affected queries by ~12-15% in benchmarks. #18699
- [PERF] TSDB: Eliminate unnecessary heap allocations in the V2 histogram WAL decoder, reducing allocations by up to 50% and memory by up to 10% for deployments using native histograms with created-timestamp storage enabled (
--enable-feature=created-timestamp-zero-ingestion). #18813 - [BUGFIX] Discovery/AWS: Fix failure when processing an AWS RDS cluster without instances. #18845
- [BUGFIX] Fix race condition in initTime that could cause ErrOutOfBounds. #18629
- [BUGFIX] PromQL: A range query whose
endwas not aligned tostepcaused subqueries inside it to evaluate past the parent's last actual step, inflatingpeakSamplesin the query stats and against thequery.max-sampleslimit, and wasting storage I/O reading samples that were never used in the result. #18081 - [BUGFIX] PromQL: A range query containing an at-modifier-unsafe function over a range-vector with an
@modifier (e.g.predict_linear(metric[60s] @ T, X)) silently under-countedtotalQueryableSamplesfor steps after step 0. #18081 - [BUGFIX] PromQL: Fix
fill_left/fill_rightproducing missing samples in range queries when usinggroup_left/group_right. #18850 - [BUGFIX] PromQL: Fix for resets() and changes() in anchored range extenders with histograms. #18906
- [BUGFIX] PromQL: Fix panic on
1[5m] smoothedand similar expressions when extended range selectors are enabled. #18764 - [BUGFIX] PromQL: Fix panic when a
smoothedinstant vector selector produces no samples for a series. #18943 - [BUGFIX] PromQL: Fix panic when using a parenthesised plain number as an offset (e.g.
foo offset -(5)). #18768 - [BUGFIX] promtool: Fix panic when parsing exposition text containing empty braces
{}. Via prometheus/common v0.69.0. #18949 - [BUGFIX] Promtool: Fix
check healthyandcheck readywhen--urlends with a trailing slash. #18854 - [BUGFIX] Rules: Close PromQL query after each rule evaluation to ensure resources are released. #18733
- [BUGFIX] Scaleway SD: Resolve VPC/IPAM-only instances that have no legacy
private_iporpublic_ipfield, but do have private NICs attached. #18772 - [BUGFIX] TSDB: Do not leak head series when an integer histogram append is rejected (e.g. out-of-order). #18838
- [BUGFIX] UI: Escape label values offered by PromQL autocomplete. #18658
- [BUGFIX] TSDB: Fix chunk snapshot encoding for EncXOR2 chunks, preventing corruption on TSDB restart when EncXOR2-encoded series were present. #18739
- [BUGFIX] TSDB: Store a millisecond timestamp (not a WAL segment number) in walExpiries when a series is evicted via CompactStaleHead/CompactSelectedSeries, so the series's label record is correctly retained in the next WAL checkpoint and replays cleanly. #18847
- [BUGFIX] TSDB: Prevent loss of samples at the chunk-range boundary when CompactSelectedSeries (and CompactStaleHead) evict the series — the per-slice compaction loop now runs one more iteration so the boundary timestamp is captured in a block before the in-memory copy is removed. #18849
3.13.0-rc.1 / 2026-06-22
Immutable
release. Only release title and notes can be modified.
Release notes of the 3.13-rc.1 release:
The 3.13.0-rc.0 release was only partially successful due to the migration from NPM to PNPM and subsequent CI issues, so most of the changes in this release candidate are CI/build-related. The only user-facing change is:
- [CHANGE] UI: Third-party npm dependency licenses are now embedded in the Prometheus binary and served at
/assets/third-party-licenses.txt, replacing thenpm_licenses.tar.bz2archive previously shipped in release tarballs and container images. #18997
Release notes of the 3.13-rc.0 release, as it was not published in partial state:
- [SECURITY] UI: Bump
sanitize-htmlto fix a cross-site scripting vulnerability (CVE-2026-44990). #18697 - [CHANGE] API: Use SHA-256 instead of SHA-1 to generate rule group pagination tokens. #18927
- [CHANGE] HTTP clients: Credentials (Authorization header, basic auth, bearer token, OAuth2, configured headers) are no longer forwarded when following a redirect to a different host; affects scraping, remote read/write, alerting, and service discovery. Via prometheus/common v0.69.0 (CVE-2025-4673 CVE-2023-45289). #18949
- [CHANGE] promtool: Relative file paths in the file passed to
--http.config.fileare now resolved relative to that config file's directory instead of its parent directory. Via prometheus/common v0.69.0. #18949 - [CHANGE] PromQL: Rename the
min()andmax()duration-expression functions (experimental feature flagexperimental-duration-expr) tomin_of()andmax_of()to avoid confusion with theminandmaxaggregate operators. #18687 - [FEATURE] API: Add experimental search endpoints to search metric names, label names, and label values. #18573
- [FEATURE] Discovery/AWS: Add ability to filter RDS instances. #18859
- [FEATURE] PromQL: Add
min_of(a, b)andmax_of(a, b)scalar experimental functions, returning the smaller or larger of two scalar values. #18687 - [FEATURE] PromQL: Add support for smoothed/anchored rate with native histograms. #18564
- [FEATURE] PromQL: Expose per-query
samplesRead(andsamplesReadPerStepwithstats=alland thepromql-per-step-statsfeature flag) in the query stats response, and add theprometheus_engine_query_samples_read_totalengine counter.samplesReadreflects storage I/O distinct fromtotalQueryableSamples, which counts samples loaded into the evaluator (and so over-counts when a sample is reused across multiple range-vector windows). #18081 - [FEATURE] Scrape: Add
__convert_classic_histograms_to_nhcb__internal label to allow per-target override ofconvert_classic_histograms_to_nhcbscrape configuration via relabeling. #18840 - [FEATURE] TSDB: Add
storage.tsdb.chunk_encoding.floatsconfiguration field to select float chunk encoding (xororxor2) at runtime, independently of the--enable-feature=xor2-encodingflag. #18769 - [FEATURE] remote_write: Add Certificate support for ingesting data into an Azure Monitor Workspace. #18217
- [FEATURE] Scrape: Add
__always_scrape_classic_histograms__and__scrape_native_histograms__internal labels to allow per-target override of thealways_scrape_classic_histogramsandscrape_native_histogramsscrape configuration via relabeling. #18929 - [ENHANCEMENT] Release: Container images are now also published to the GitHub Container Registry (ghcr.io). #18791
- [ENHANCEMENT] PromQL: Prettify
fill_left(x) fill_right(x)asfill(x)when both fill values are equal. #18851 - [ENHANCEMENT] UI: Improve autocompletion after closing a function bracket. #18894
- [PERF] Labels: Add case-insensitive prefix matching to speed up evaluation of long case-insensitive regular expressions (up to ~2x faster). #18540
- [PERF] TSDB: Reduce per-sample overhead in chunk population, speeding up affected queries by ~12-15% in benchmarks. #18699
- [PERF] TSDB: Eliminate unnecessary heap allocations in the V2 histogram WAL decoder, reducing allocations by up to 50% and memory by up to 10% for deployments using native histograms with created-timestamp storage enabled (
--enable-feature=created-timestamp-zero-ingestion). #18813 - [BUGFIX] Discovery/AWS: Fix failure when processing an AWS RDS cluster without instances. #18845
- [BUGFIX] Fix race condition in initTime that could cause ErrOutOfBounds. #18629
- [BUGFIX] PromQL: A range query whose
endwas not aligned tostepcaused subqueries inside it to evaluate past the parent's last actual step, inflatingpeakSamplesin the query stats and against thequery.max-sampleslimit, and wasting storage I/O reading samples that were never used in the result. #18081 - [BUGFIX] PromQL: A range query containing an at-modifier-unsafe function over a range-vector with an
@modifier (e.g.predict_linear(metric[60s] @ T, X)) silently under-countedtotalQueryableSamplesfor steps after step 0. #18081 - [BUGFIX] PromQL: Fix
fill_left/fill_rightproducing missing samples in range queries when usinggroup_left/group_right. #18850 - [BUGFIX] PromQL: Fix for resets() and changes() in anchored range extenders with histograms. #18906
- [BUGFIX] PromQL: Fix panic on
1[5m] smoothedand similar expressions when extended range selectors are enabled. #18764 - [BUGFIX] PromQL: Fix panic when a
smoothedinstant vector selector produces no samples for a series. #18943 - [BUGFIX] PromQL: Fix panic when using a parenthesised plain number as an offset (e.g.
foo offset -(5)). #18768 - [BUGFIX] promtool: Fix panic when parsing exposition text containing empty braces
{}. Via prometheus/common v0.69.0. #18949 - [BUGFIX] Promtool: Fix
check healthyandcheck readywhen--urlends with a trailing slash. #18854 - [BUGFIX] Rules: Close PromQL query after each rule evaluation to ensure resources are released. #18733
- [BUGFIX] Scaleway SD: Resolve VPC/IPAM-only instances that have no legacy
private_iporpublic_ipfield, but do have private NICs attached. #18772 - [BUGFIX] TSDB: Do not leak head series when an integer histogram append is rejected (e.g. out-of-order). #18838
- [BUGFIX] UI: Escape label values offered by PromQL autocomplete. #18658
- [BUGFIX] TSDB: Fix chunk snapshot encoding for EncXOR2 chunks, preventing corruption on TSDB restart when EncXOR2-encoded series were present. #18739
- [BUGFIX] TSDB: Store a millisecond timestamp (not a WAL segment number) in walExpiries when a series is evicted via CompactStaleHead/CompactSelectedSeries, so the series's label record is correctly retained in the next WAL checkpoint and replays cleanly. #18847
- [BUGFIX] TSDB: Prevent loss of samples at the chunk-range boundary when CompactSelectedSeries (and CompactStaleHead) evict the series — the per-slice compaction loop now runs one more iteration so the boundary timestamp is captured in a block before the in-memory copy is removed. #18849
3.13.0-rc.0 / 2026-06-18
Immutable
release. Only release title and notes can be modified.
- [SECURITY] UI: Bump
sanitize-htmlto fix a cross-site scripting vulnerability (CVE-2026-44990). #18697 - [CHANGE] API: Use SHA-256 instead of SHA-1 to generate rule group pagination tokens. #18927
- [CHANGE] HTTP clients: Credentials (Authorization header, basic auth, bearer token, OAuth2, configured headers) are no longer forwarded when following a redirect to a different host; affects scraping, remote read/write, alerting, and service discovery. Via prometheus/common v0.69.0 (CVE-2025-4673 CVE-2023-45289). #18949
- [CHANGE] promtool: Relative file paths in the file passed to
--http.config.fileare now resolved relative to that config file's directory instead of its parent directory. Via prometheus/common v0.69.0. #18949 - [CHANGE] PromQL: Rename the
min()andmax()duration-expression functions (experimental feature flagexperimental-duration-expr) tomin_of()andmax_of()to avoid confusion with theminandmaxaggregate operators. #18687 - [FEATURE] API: Add experimental search endpoints to search metric names, label names, and label values. #18573
- [FEATURE] Discovery/AWS: Add ability to filter RDS instances. #18859
- [FEATURE] PromQL: Add
min_of(a, b)andmax_of(a, b)scalar experimental functions, returning the smaller or larger of two scalar values. #18687 - [FEATURE] PromQL: Add support for smoothed/anchored rate with native histograms. #18564
- [FEATURE] PromQL: Expose per-query
samplesRead(andsamplesReadPerStepwithstats=alland thepromql-per-step-statsfeature flag) in the query stats response, and add theprometheus_engine_query_samples_read_totalengine counter.samplesReadreflects storage I/O distinct fromtotalQueryableSamples, which counts samples loaded into the evaluator (and so over-counts when a sample is reused across multiple range-vector windows). #18081 - [FEATURE] Scrape: Add
__convert_classic_histograms_to_nhcb__internal label to allow per-target override ofconvert_classic_histograms_to_nhcbscrape configuration via relabeling. #18840 - [FEATURE] TSDB: Add
storage.tsdb.chunk_encoding.floatsconfiguration field to select float chunk encoding (xororxor2) at runtime, independently of the--enable-feature=xor2-encodingflag. #18769 - [FEATURE] remote_write: Add Certificate support for ingesting data into an Azure Monitor Workspace. #18217
- [FEATURE] Scrape: Add
__always_scrape_classic_histograms__and__scrape_native_histograms__internal labels to allow per-target override of thealways_scrape_classic_histogramsandscrape_native_histogramsscrape configuration via relabeling. #18929 - [ENHANCEMENT] Release: Container images are now also published to the GitHub Container Registry (ghcr.io). #18791
- [ENHANCEMENT] PromQL: Prettify
fill_left(x) fill_right(x)asfill(x)when both fill values are equal. #18851 - [ENHANCEMENT] UI: Improve autocompletion after closing a function bracket. #18894
- [PERF] Labels: Add case-insensitive prefix matching to speed up evaluation of long case-insensitive regular expressions (up to ~2x faster). #18540
- [PERF] TSDB: Reduce per-sample overhead in chunk population, speeding up affected queries by ~12-15% in benchmarks. #18699
- [PERF] TSDB: Eliminate unnecessary heap allocations in the V2 histogram WAL decoder, reducing allocations by up to 50% and memory by up to 10% for deployments using native histograms with created-timestamp storage enabled (
--enable-feature=created-timestamp-zero-ingestion). #18813 - [BUGFIX] Discovery/AWS: Fix failure when processing an AWS RDS cluster without instances. #18845
- [BUGFIX] Fix race condition in initTime that could cause ErrOutOfBounds. #18629
- [BUGFIX] PromQL: A range query whose
endwas not aligned tostepcaused subqueries inside it to evaluate past the parent's last actual step, inflatingpeakSamplesin the query stats and against thequery.max-sampleslimit, and wasting storage I/O reading samples that were never used in the result. #18081 - [BUGFIX] PromQL: A range query containing an at-modifier-unsafe function over a range-vector with an
@modifier (e.g.predict_linear(metric[60s] @ T, X)) silently under-countedtotalQueryableSamplesfor steps after step 0. #18081 - [BUGFIX] PromQL: Fix
fill_left/fill_rightproducing missing samples in range queries when usinggroup_left/group_right. #18850 - [BUGFIX] PromQL: Fix for resets() and changes() in anchored range extenders with histograms. #18906
- [BUGFIX] PromQL: Fix panic on
1[5m] smoothedand similar expressions when extended range selectors are enabled. #18764 - [BUGFIX] PromQL: Fix panic when a
smoothedinstant vector selector produces no samples for a series. #18943 - [BUGFIX] PromQL: Fix panic when using a parenthesised plain number as an offset (e.g.
foo offset -(5)). #18768 - [BUGFIX] promtool: Fix panic when parsing exposition text containing empty braces
{}. Via prometheus/common v0.69.0. #18949 - [BUGFIX] Promtool: Fix
check healthyandcheck readywhen--urlends with a trailing slash. #18854 - [BUGFIX] Rules: Close PromQL query after each rule evaluation to ensure resources are released. #18733
- [BUGFIX] Scaleway SD: Resolve VPC/IPAM-only instances that have no legacy
private_iporpublic_ipfield, but do have private NICs attached. #18772 - [BUGFIX] TSDB: Do not leak head series when an integer histogram append is rejected (e.g. out-of-order). #18838
- [BUGFIX] UI: Escape label values offered by PromQL autocomplete. #18658
- [BUGFIX] TSDB: Fix chunk snapshot encoding for EncXOR2 chunks, preventing corruption on TSDB restart when EncXOR2-encoded series were present. #18739
- [BUGFIX] TSDB: Store a millisecond timestamp (not a WAL segment number) in walExpiries when a series is evicted via CompactStaleHead/CompactSelectedSeries, so the series's label record is correctly retained in the next WAL checkpoint and replays cleanly. #18847
- [BUGFIX] TSDB: Prevent loss of samples at the chunk-range boundary when CompactSelectedSeries (and CompactStaleHead) evict the series — the per-slice compaction loop now runs one more iteration so the boundary timestamp is captured in a block before the in-memory copy is removed. #18849