Logstash 9.3.5 → 9.5.0
13 releases · 5 breaking changes · 0 security fixes · 0 advisories fixed
0security5breaking2deprecated27feature31fix
Breaking changes
- breaking Drop support for Logstash 7.x by requiring elasticsearch gem >= 8. Logstash 8+ continues to work as before #213 in 9.5.0
- breaking Drop support for Logstash 7.x by requiring elasticsearch gem >= 8. Logstash 8+ continues to work as before #252 in 9.5.0
- breaking Remove explicit lz4-java dependency (now transitive from Kafka client) in 9.5.0
- breaking Breaking: partitioner options default and uniform_sticky are removed in 9.5.0
- breaking Remove duplicated deprecation log entry #208 in 9.5.0
Security changes
None in the release notes.
Deprecations
- deprecated Replace deprecated Aws::S3::Object#upload_file in favor of Aws::S3::TransferManager#upload_file #67 in 9.4.0
- deprecated Replace deprecated File.exists? with File.exist? for Ruby 3.4 (JRuby 10) compatibility #65 in 9.4.0
Every release in between
featureSend Logstash Metrics via OTLP #18857
featureAuto-reload pipelines on TLS certificate rotation #18978
featureExtend TLS auto-reload to CPM and monitoring service clients #19045
featureAdd support for cgroupv2 #18708
featureNew dead_letter_queue.flush_check_interval setting to control how frequently stale DLQ segment files are flushed #19036
featureRemoved legacy output concurrency mode. The deprecated concurrency :legacy option is no longer supported; outputs should use concurrency :single or concurrency :shared #19003
featureRemoved the pipe output plugin from the default plugin bundle #19159
fixPipeline metrics are now cleared when a pipeline fails to start, preventing stale metrics from being reported by the monitoring API #19091
fixFailures in plugin do_close no longer halt the pipeline shutdown sequence #19035
fixFixed forced-shutdown (double SIGINT) behavior for JRuby 10 #19017
fixAdded missed standard types during batch size estimation #19158
otherBump OpenTelemetry to 1.62.0 #19209
otherUpdate Netty dependency to 4.1.136.Final #575
otherSet SO_TIMEOUT on IOReactor to ensure NIO connections have a baseline socket timeout, preventing indefinite hangs when connection pool restore fails to apply the request-level timeout #480
otherSync up with Elasticsearch 9.5 branch to pull latest dependencies #476
fixFixes an issue where a field set by an integration pipeline to java.util.Date value-object representing a timestamp could not be converted to a timestamp #460
otherApplies Elasticsearch geoip module relocation changes #445
otherSupport Elastic Cloud API keys in the api_key option, which now accepts an id:api_key pair, its base64-encoded form, or an essu_ Cloud API key, and rejects an unrecognized format at startup #215
breakingDrop support for Logstash 7.x by requiring elasticsearch gem >= 8. Logstash 8+ continues to work as before #213
otherSupport Elastic Cloud API keys in the api_key option, which now accepts an id:api_key pair, its base64-encoded form, or an essu_ Cloud API key, and rejects an unrecognized format at startup #274
fixFix serverless request failure caused by conflicting compatible-with and Elastic-Api-Version headers when using elasticsearch-ruby v9 #269
breakingDrop support for Logstash 7.x by requiring elasticsearch gem >= 8. Logstash 8+ continues to work as before #252
fixFix to use the Content-type declared charset to decode the request body #230
otherUpdate Netty dependency to 4.1.136.Final #228
otherUpdate Netty dependency to 4.1.136.Final #281
fixFix connection leak on statement retry by opening JDBC connection once outside retry loop #201
fixFix sasl_jaas_config output configuration #245
otherUpdate Kafka client to 4.2.0 #243
breakingRemove explicit lz4-java dependency (now transitive from Kafka client)
otherDocument by_duration offset reset strategy (available since Apache Kafka 4.0.0)
otherMask sensitive sasl_jaas_config values in debug logs to prevent credential exposure #232
otherUpdate Kafka client to 4.1.0 and transitive dependencies #205
breakingBreaking: partitioner options default and uniform_sticky are removed
otherlinger_ms default value changed from 0 to 5
featureAdd group_protocols option for configuring Kafka consumer rebalance protocol
featureSetting group_protocol => consumer opts in to the new consumer group protocol
breakingRemove duplicated deprecation log entry #208
otherGenerate error events with _snmpfailure tag when all SNMP operations fail and the response data is empty (timeout for example) #92
featureHandle partial responses and errors gracefully: add tag_on_failure (default: ["_snmpfailure"]) to tag events when SNMP operations fail, and allow_partial_response (default: false) to preserve partial data from failed walk/table operations #91
featureAdded support for IPv6 addresses #16
otherUpdate bundled JDK to 21.0.12.1 build 1 #19508
otherUpdate bundled JRuby to 10.0.7.0 #19609
otherUpdate Jackson to 2.21.7 and jrjackson to 0.5.4 #19442 #19577
otherUpgrade rubyzip dependency to 3.x #19501
otherUpgrade zstd-jni dependency to 1.5.7-14 #19512
otherDocument that the lumberjack input plugin should not be used for Logstash-to-Logstash communication #19505
otherUpdate Netty dependency to 4.1.138.Final #579
otherUpdate Netty dependency to 4.1.138.Final #241
otherUpdate Netty dependency to 4.1.138.Final #290
otherUpgrade march_hare to consume 4.10+ #87
otherRequire httpclient5 5.6.4 or later #502
otherUpdate jackson dependency to 3.1.6 #498
otherUpdate proton-j to 0.35.0 #122
fixFix for processing the HA Proxy protocol, considering the line can be split in multiple buffers. #286
fixFix access method for large tables to avoid materializing the full result set in memory and triggering an out-of-memory error. #204
otherPrevent concurrent Sequel JDBC subadapter initialization races by preloading adapter at driver load time #203
otherUpgrade zstd-jni dependency #280
fixFix: terminate idle scheduler work threads on plugin close, which leaked one thread per pipeline reload #7
otherRequiere mail dependency 2.9 or later #72
featureInclude timestamp in health report snapshot API response #19422
fixFix a NoSuchFileException in the dead letter queue that could occur during oldest segment resolution when a segment file is concurrently removed #19409
otherUpdated concurrent-ruby gem #19428
fixFix NameError: uninitialized constant Bignum that prevented the codec from loading on Logstash 9.4+ #8
otherThe edn gem references the Bignum constant, which Ruby removed in 3.2 (shipped by the JRuby in Logstash 9.4+). This raised a NameError when the codec registered, so any pipeline using the edn codec failed to start. Alias the removed Fixnum/Bignum constants to Integer before requiring edn.
otherThe edn gem references the Bignum constant, which Ruby removed in 3.2 (shipped by the JRuby in Logstash 9.4+). This raised a NameError when the codec registered, so any pipeline using the edn_lines codec failed to start. Alias the removed Fixnum/Bignum constants to Integer before requiring edn.
fixFix NameError: uninitialized constant Fixnum when using the MURMUR3 algorithm on Logstash 9.4+ (Ruby 3.2+, which removed Fixnum in favor of Integer) #19
otherUpdate jackson dependency to 2.21.6 #121
otherUpdate Netty dependency to 4.1.137.Final #576
otherUpdate Netty dependency to 4.1.137.Final #239
featureAdd optional HTTP traffic logging (disabled by default). To enable it, turn on debug logging. #237
fixFix NameError: uninitialized constant Fixnum when reading a JMS MapMessage on Logstash 9.4+ #63
otherThe jruby-jms gem references the Fixnum constant, which Ruby removed in 3.2 (shipped by the JRuby in Logstash 9.4+). This raised a NameError while decoding a JMS MapMessage. Alias the removed Fixnum/Bignum constants to Integer before requiring jms.
otherUpdate Netty dependency to 4.1.137.Final #284
otherUpdate jackson dependency to 2.21.6 #275
otherUpdated JDK to 21.0.12 build 8 #19365
otherApply the default 30-second socket timeout to low-level HTTP connections so stalled connections time out consistently. #480
fixFixes an issue where failing to load a dictionary could cause the plugin to continue to run with a missing or partially-updated dictionary; this issue was especially noticeable when configured with refresh_behaviour => replace, which clears the dictionary before loading the replacement #112.
otherUpdated JRuby to 10.0.6.0 #19279
otherUpdated Jackson and jrjackson dependencies #19286
otherUpgraded Puma to 8.x #19234
otherSync up with Elasticsearch 9.4 branch to pull latest dependencies #477
otherUpdate jackson dependency to 3.1.4 #467
otherUpdate jackson dependency to 2.21.4 #118
fixWhen configured to use a port that is already in use, the failure is now propagated to the pipeline #537 This fixes an issue where a misconfigured input could retry indefinitely while Logstash's health report continued to report the pipeline as healthy.
fixWhen configured to use a port that is already in use, the failure is now propagated to the pipeline #221 This fixes an issue where a misconfigured input could retry indefinitely while Logstash's health report continued to report the pipeline as healthy.
otherUpdate jackson dependency to 2.21.4 #269
fixFix serverless compatibility: nil params in pool requests and unsupported template settings #1276
otherSupport Elastic Cloud API keys in the api_key option, which now accepts an id:api_key pair, its base64-encoded form, or an essu_ Cloud API key, and rejects an unrecognized format at startup #1274
feature[Doc] Add note for index option #1269
featureBatch size estimation improvements #19180
otherLogstash's file descriptors usage, limits and sizing guidance documented #19206
otherUpgrades jackson.core to 2.21.2 and nimbus-jose-jwt to 10.9 versions. #117
feature[DOCS] Added recommendation to migrate to the Kafka integration plugin #116
otherUpdate Netty dependency to 4.1.135.Final #567
otherUpdate Netty dependency to 4.1.135.Final #219
otherUpdate Netty dependency to 4.1.135.Final #274
fixFixes a metric leak in _node/stats when a pipeline repeatedly fails to start with config.reload.automatic: true. Previously, each retry left a fresh set of plugin metric entries in the collector, causing the stats payload to grow indefinitely #19120
otherUpdated jruby-openssl to 0.16.0 #19116
otherUpgraded jrjackson and fasterxml.jackson #19126
fixFixes an issue where a field set by an integration pipeline to java.util.Date value-object representing a timestamp could not be converted to a timestamp #462
otherInclude httpclient5/httpcore5 from the elasticsearch-java artifact #458
otherUpgrades jackson.core to 2.21.2 version #255
otherUpdate Netty dependency to 4.1.134.Final #541
otherUpdate Netty dependency to 4.1.134.Final #217
otherUpdate Netty dependency to 4.1.134.Final #257
other[DOC] Document command string form limitation #6
otherUpdate bundled JDK to 21.0.11 build 10.
otherUpdate Netty dependency to 4.1.133.Final #539
otherUpdate Netty dependency to 4.1.133.Final #216
otherUpdate Netty dependency to 4.1.133.Final #256
fixWhen configured to use a port that is already in use, the failure is now propagated to the pipeline. This fixes an issue where a misconfigured input could retry indefinitely while Logstash's health report continued to report the pipeline as healthy #250
otherImproves the logging experience when DLQ used #1253.
featureauto: enables pipeline crash recovery for pipelines backed by the persistent queue
featurefalse (default): do not automate recovery of crashed pipelines
featuretrue: enables pipeline crash recovery for any pipeline, even if backed by the ephemeral memory queue (risk: data loss)
featurePerformance improvements which saves ~40% CPU resource on DLQ segment file lookup operations 19013
otherUpdate JRuby to 10.0.5.0
otherUpdate bundled JDK to 21.0.10 build 7
featureFeature: Add a warning log message when the number of tasks stored in memory exceeds the configured threshold #125
fixUse milliseconds timestamp precision in S3 input to fix the skip backup and delete object issue in S3-compatible storage services #60
deprecatedReplace deprecated Aws::S3::Object#upload_file in favor of Aws::S3::TransferManager#upload_file #67
deprecatedReplace deprecated File.exists? with File.exist? for Ruby 3.4 (JRuby 10) compatibility #65
otherRe-packaging the plugin #63
featureAdd cutoff_second configuration option to S3 input plugin #59
featureAdd precision setting to support nanosecond precision timestamps #165
otherms (default): timestamps are stored with millisecond precision
otherit keeps the same behavior as before for backward compatibility
otherfractional seconds are truncated to 3 digits
othercustom parsing formats use joda-time library
otherns: timestamps are stored with nanosecond precision
otherfractional seconds support up to 9 digits
othercustom parsing formats use java.time
otherISO8601 now accepts up to 9 fractional-second digits
otherApply an 'error' tag to any event that fails the de-dotting process #26
featureAdd JRuby 10 support: replace removed NativeException with RaiseException, source JRuby from Logstash vendor directory instead of pinning Maven version #96
otherInclude Elasticsearch web-utils JAR into the plugin to keep registered_domain processor dependencies #397
fixFixed set_security_user processor to behave consistently with other unsupported processors (inference, enrich) by tagging events with _ingest_pipeline_failure #269
fixFix fingerprint instability for Hash and Array field values caused by JRuby 10 changing Hash#inspect formatting #79
fixFix a regression introduced in 11.8.7 where sasl_jaas_config was incorrectly typed in the output plugin, crashing Logstash on startup when a SASL Kafka output configuration was present #247
fixAdd JRuby 10 support: fix integer conversion precision loss beyond 2^53 caused by parse_signed_hex_str routing all strings through Float() #178
otherUpdates the gelf dependency #77
otherUpdated JRuby to 9.4.15.0 #19290
otherUpdated Jackson and jrjackson dependencies #19289
otherSync up with Elasticsearch 9.3 branch to pull latest dependencies #478
otherUpdate jackson dependency to 3.1.4 #470
otherUpdate jackson dependency to 2.21.4 #118
fixWhen configured to use a port that is already in use, the failure is now propagated to the pipeline #537 This fixes an issue where a misconfigured input could retry indefinitely while Logstash's health report continued to report the pipeline as healthy.
fixWhen configured to use a port that is already in use, the failure is now propagated to the pipeline #221 This fixes an issue where a misconfigured input could retry indefinitely while Logstash's health report continued to report the pipeline as healthy.
otherUpdate jackson dependency to 2.21.4 #269
fixFix serverless compatibility: nil params in pool requests and unsupported template settings #1276
otherSupport Elastic Cloud API keys in the api_key option, which now accepts an id:api_key pair, its base64-encoded form, or an essu_ Cloud API key, and rejects an unrecognized format at startup #1274
feature[Doc] Add note for index option #1269
otherUpgrade puma to 8.x #19228
otherNo user-facing changes in Logstash plugins.
featureBatch's size estimation improvements #19179
otherLogstash's file descriptors usage, limits and sizing guidance documented #19207
otherUpgrades jackson.core to 2.21.2 and nimbus-jose-jwt to 10.9 versions. #117
feature[DOCS] Added recommendation to migrate to the Kafka integration plugin #116
otherUpdate Netty dependency to 4.1.135.Final #567
otherUpdate Netty dependency to 4.1.135.Final #219
otherUpdate Netty dependency to 4.1.135.Final #274