VictoriaMetrics 1.148.8 → 1.151.0
3 releases · 1 breaking change · 3 security fixes · 0 advisories fixed
3security1breaking0deprecated12feature29fix
Breaking changes
- breaking BUGFIX: vmsingle, vmselect in VictoriaMetrics cluster and vmctl: accept scientific notation with sub-second precision (e.g. 1.784144612388E9) for timestamp args such as start and end in /api/v1/query_range and --vm-native-filter-time-start and --vm-native-filter-time-end in vmctl. Previously,… in 1.149.0
Security changes
- security SECURITY: vmsingle, vmagent, vminsert and vmselect in VictoriaMetrics cluster: check HTTP Basic Auth credentials from the -httpAuth. command-line flags for cases that do not verify the corresponding *AuthKey command-line flag on its own. Previously, HTTP Basic Auth could be skipped for request… in 1.151.0
- security SECURITY: upgrade Go builder from Go1.26.5 to Go1.26.6. See the list of issues addressed in Go1.26.6. in 1.150.0
- security SECURITY: vmsingle and vmselect in VictoriaMetrics cluster: restrict /api/v1/admin/tsdb/delete_series, /tags/delSeries endpoints to POST method only to prevent some SSRF-based data deletion attacks. See. in 1.149.0
Every release in between
securitySECURITY: vmsingle, vmagent, vminsert and vmselect in VictoriaMetrics cluster: check HTTP Basic Auth credentials from the -httpAuth. command-line flags for cases that do not verify the corresponding *AuthKey command-line flag on its own. Previously, HTTP Basic Auth could be skipped for request…
featureFEATURE: all VictoriaMetrics components: add support for accepting HTTP requests over Unix domain socket at -httpListenAddr. For example, -httpListenAddr=unix:/run/victoria-metrics.sock. See VictoriaLogs#1618.
featureFEATURE: vmagent, vmsingle, vmstorage and vmselect in VictoriaMetrics cluster: expose the vm_app_prev_shutdown_unclean gauge. It is set to 1 when the previous process run didn't shut down cleanly. Added the UncleanShutdown alerting rule, which fires for 10 minutes after an unclean shutdown is…
featureFEATURE: vmui: show the selected time zone UTC offset next to the date/time controls and allow opening time zone settings from it. See.
otherFEATURE: vmsingle, vmagent, vmalert, and vmselect in VictoriaMetrics cluster: show how the default value is calculated for command-line flags which derive it from the number of available CPU cores. For example, -maxConcurrentInserts now prints (default 16 = 2*cgroup.AvailableCPUs()) in -help…
featureFEATURE: vmagent: introduce vmagent_kafka_consumer_broker_rtt_seconds and vmagent_kafka_consumer_fetch_queue_size metrics for Kafka consumer integration. These metrics help troubleshoot slow Kafka consumption. See.
fixBUGFIX: alerts: fix AlertingRuleResultsApproachingLimit and RecordingRuleResultsApproachingLimit alerting rules in vmalert alerting rules. The alerts could produce incorrect results or fail to fire when vmalert_group_rule_results_limit returned multiple samples per (group, file) label combination,…
fixBUGFIX: vmagent and vminsert in VictoriaMetrics cluster: fix infinite loop in the OpenTelemetry Firehose ingestion endpoint (/opentelemetry/api/v1/push) when receiving a malformed record with an incomplete varint in the data field. Previously this caused the goroutine to spin forever, permanently…
fixBUGFIX: vmalert-tool: reuse connections to -remoteWrite.url when writing the results of recording rules and alerts. Previously every series was sent over a new connection, which left a lot of sockets in TIME_WAIT state and could exhaust the ephemeral port range. The number of idle connections can…
fixBUGFIX: vmsingle and vmselect in VictoriaMetrics cluster: prevent process crash in sort_by_label_numeric() and sort_by_label_numeric_desc() when a label value contains a number with 309 or more digits. See.
fixBUGFIX: vmsingle and vmstorage in VictoriaMetrics cluster: fixes rare metrics metadata partial search response. See.
fixBUGFIX: vmctl: reuse connections in vm-native mode when --vm-concurrency exceeds 2. Previously the number of idle connections was limited to 2 per host, which was insufficient when --vm-concurrency was bigger than 2. See.
fixBUGFIX: vmsingle and vmselect in VictoriaMetrics cluster: properly re-use memory if query aggregation returns error. See.
fixBUGFIX: vmselect in VictoriaMetrics cluster: fail the query request directly when there is not enough disk space to store temporary search results. Previously, such queries could lead to vmselect crash. See.
fixBUGFIX: vmselect in VictoriaMetrics cluster: consistently re-use memory during storage blocks unpacking on parsing storage block error. See.
fixBUGFIX: vmalert: fix parsing of instant query sample timestamps from Prometheus-compatible data sources. Previously, decimal (e.g. 1786458420.123) or scientific-notation (e.g. 1.78645842E9) values were silently truncated to 0. See. Thanks to @missusk for the contribution.
fixBUGFIX: vmctl: properly release the export and import requests during migration requests fails in vm-native mode. Previously, failed export/import requests could have left hanging at the source or the destination. The fix is supposed to improve the resiliency of vmctl during long-running migrations.
fixBUGFIX: vmagent: include vm_account_id and vm_project_id labels in the instance identity used by monitoring data exchange filtering. Previously, when multitenant handlers were enabled, series from different tenants sharing the same job and instance labels could be misidentified as coming from a…
fixBUGFIX: vmsingle, vminsert in VictoriaMetrics cluster and vmagent: fix process crash at /api/v1/import/native endpoint when receiving a specially crafted ~45-byte request..
fixBUGFIX: vmsingle, vmagent and vmstorage in VictoriaMetrics cluster: allow access to /remotewrite-relabel-config, /api/v1/status/remotewrite-relabel-config, /remotewrite-url-relabel-config, /api/v1/status/remotewrite-url-relabel-config and /internal/log_new_series with only the corresponding…
securitySECURITY: upgrade Go builder from Go1.26.5 to Go1.26.6. See the list of issues addressed in Go1.26.6.
otherFEATURE: relabeling: reduce CPU usage up to 30% when matching relabeling rules with multiple if expressions containing exact metric names. Expressions for other metric names are now skipped before evaluating their remaining label filters. See. Thanks to @nevgeny for contribution.
featureFEATURE: vmagent and vmsingle: add support for linode_sd_configs for discovering scrape targets from Linode instances. See. Thanks to @cxdy for contribution.
otherFEATURE: vmalert: extend -replay.continueWithExecutionErr to also handle the 400 Bad Request response code, since it is used for Prometheus querying API requests when request parameters are missing or incorrect. See.
otherFEATURE: vmselect and vminsert in VictoriaMetrics cluster, and vmagent: set default value of -enableMultitenancyViaHeaders to true. This change enables support of multitenancy via headers for cluster and for vmagent by default, aligning VictoriaMetrics multitenancy behavior with multitenancy in…
featureFEATURE: vmui: add an option to customize the favicon color. This makes it easier to distinguish between different installations opened in multiple browser tabs. See.
fixBUGFIX: vmagent and vmsingle: avoid suggesting the unrelated -enableTCP6 command-line flag when scraping a target over a Unix domain socket fails. See. Thanks to @lwmacct for contribution.
fixBUGFIX: vmsingle and vminsert in VictoriaMetrics cluster: skip labels with empty name at /api/v1/import. Previously such a label replaced the metric name, so a series sent with "metric":{"name":"foo","":"bar"} was stored under the name bar. Other ingestion protocols already skip such labels. See.…
fixBUGFIX: vmsingle, vmselect in VictoriaMetrics cluster and vmctl: properly parse small fractional Unix timestamps in timestamp args such as start and end in /api/v1/query_range and --vm-native-filter-time-start and --vm-native-filter-time-end in vmctl. Previously, fractional Unix timestamps with…
fixBUGFIX: vmsingle, vmstorage and vmselect in VictoriaMetrics cluster: persist the previous working set cache during graceful shutdown when it is likely to contain the active working set. This prevents saving an empty or cold current cache right after split-mode cache rotation, which could otherwise…
fixBUGFIX: vmsingle and vmselect in VictoriaMetrics cluster: change the HTTP response code for Prometheus querying API requests from 422 Unprocessable Entity to 400 Bad Request when request parameters are missing or incorrect. See.
fixBUGFIX: vmui: respect the custom query step specified via g0.step_input when opening a URL. Previously, it could be reset to the automatically calculated step and potentially cause dashboards to freeze. See.
fixBUGFIX: vmagent and vmsingle: properly assign scrape target IP address at IPv6-only networks for docker_sd_configs. See.
fixBUGFIX: vmalert: rename vmalert_rule_group_results_limit back to vmalert_group_rule_results_limit. The metric was introduced in v1.147.0 but was accidentally given the wrong name. See.
fixBUGFIX: vmalert: properly update group-level eval_delay and eval_alignment for existing groups during runtime when config reload is triggered periodically or manually via /-/reload. Previously, these settings weren't updated after config reload during runtime. See.
fixBUGFIX: vmselect in VictoriaMetrics cluster: scale the default -search.maxConcurrentRequests with the number of available CPU cores instead of capping it at 16. See. Thanks to @Dhru1Tanna for contribution.
securitySECURITY: vmsingle and vmselect in VictoriaMetrics cluster: restrict /api/v1/admin/tsdb/delete_series, /tags/delSeries endpoints to POST method only to prevent some SSRF-based data deletion attacks. See.
featureFEATURE: dashboards: add Fsync avg duration panel to the Troubleshooting section of the single-node, cluster, and vmagent dashboards. This panel surfaces degradation of IO operation for faster incident triage. See.
featureFEATURE: vmagent: add name label identifying the corresponding -remoteWrite.url target to the vm_persistentqueue_* metrics exposed by persistent queue. See. Thanks to @tIGO for contribution.
featureFEATURE: vmagent: add -remoteWrite.obfuscateLabels flag for hashing values of the specified labels before sending metrics to the corresponding -remoteWrite.url. This allows sharing metrics with external systems while keeping sensitive label values hidden. See.
featureFEATURE: vmalert: add -replay.continueWithExecutionErr flag to allow continuing to replay other rules when a rule execution fails with a 422 response code, which can happen due to an expression syntax error or a resource limit being hit. See 11313.
featureFEATURE: vmalert: add template variable $interval to expose the alerting rule group's evaluation interval. This allows generating dashboard links with a lookback window relative to the rule's interval, for example &from={{ ($activeAt.Add (parseDurationTime (printf "-%s" .Interval))).UnixMilli…
featureFEATURE: vmbackupmanager and alerts: introduce vm_backup_last_success_at metric to track the last successful backup by type. Add alerting rules NoLatestBackupWithinLastDay, NoHourlyBackupWithinLastDay, NoDailyBackupWithinLast3Days, NoWeeklyBackupWithinLast14Days and NoMonthlyBackupWithinLast62Days…
otherFEATURE: vmctl: support Prometheus native histograms migration in remote read mode. Native histograms are converted into _count, _sum and _bucket series with vmrange labels in the same way as VictoriaMetrics converts native histograms received via Prometheus remote write protocol, except that for…
otherFEATURE: vminsert in VictoriaMetrics cluster: enable slowness-based re-routing by default. Previously, -disableRerouting defaulted to true, which limited ingestion throughput to the slowest vmstorage node. Now -disableRerouting defaults to false, so vminsert automatically routes data away from the…
otherFEATURE: vmui: persist the selected auto-refresh interval in the URL. See VictoriaLogs#1310.
fixBUGFIX: MetricsQL: properly drop data points filtered out by an inner comparison operation when its result is used on the right side of another comparison. Previously, queries like foo != (bar > 100) could return unexpected results because filtered-out data points are represented internally as…
fixBUGFIX: vmagent and vmsingle: ignore HTTP proxy environment variables when scraping targets over Unix domain sockets. See. Thanks to @lwmacct for contribution.
fixBUGFIX: vmalert: fixed the display of rule state badges on the Groups page in the web UI. See.
fixBUGFIX: vmbackupmanager: previously, vmbackupmanager was crashing on startup when it failed to restore backup state from remote storage, causing a crash loop. Now it logs the error and continues running, retrying the state restore before each scheduled backup. Added…
fixBUGFIX: stream aggregation: fix incorrect sum_samples_total results when enable_windows: true is set. See. Thanks to @beyond-infra for contribution.
breakingBUGFIX: vmsingle, vmselect in VictoriaMetrics cluster and vmctl: accept scientific notation with sub-second precision (e.g. 1.784144612388E9) for timestamp args such as start and end in /api/v1/query_range and --vm-native-filter-time-start and --vm-native-filter-time-end in vmctl. Previously,…