ClickHouse 26.9.20.23 → 26.10.2.103
6 releases · 4 breaking changes · 0 security fixes · 0 advisories fixed
0security4breaking0deprecated41feature27fix
Breaking changes
- breaking Backported in: CREATE TABLE x AS src inherits the credentials embedded in the engine of src, which are masked in SHOW CREATE TABLE. It now requires the SELECT privilege on src when the inherited definition can carry credentials, instead of only SHOW COLUMNS. (Yarik Briukhovetskyi). in 26.10.2.103
- breaking Backported in: Remove object addresses from NATS, io_uring and HDFS messages. (Raúl Marín). in 26.9.24.8
- breaking Backported in: Remove object addresses from logical error messages. (Raúl Marín). in 26.9.24.8
- breaking Backported in: CREATE TABLE x AS src inherits the credentials embedded in the engine of src, which are masked in SHOW CREATE TABLE. It now requires the SELECT privilege on src when the inherited definition can carry credentials, instead of only SHOW COLUMNS. (Yarik Briukhovetskyi). in 26.9.21.9
Security changes
None in the release notes.
Every release in between
breakingBackported in: CREATE TABLE x AS src inherits the credentials embedded in the engine of src, which are masked in SHOW CREATE TABLE. It now requires the SELECT privilege on src when the inherited definition can carry credentials, instead of only SHOW COLUMNS. (Yarik Briukhovetskyi).
featureBackported in: Added the experimental Elasticsearch table engine and elasticsearch table function for reading documents from an Elasticsearch index as a JSON column, with the document id and index name available as virtual columns. Enable them with the enable_elasticsearch setting. (Kirill Kopnev).
featureThe setting use_query_condition_cache_for_time_conditions (using the query condition cache for conditions involving the current time) is now experimental and disabled by default. (Alexey Milovidov).
featureBackported in: Column statistics of MergeTree data parts are cached per data part and column in a new statistics cache (server settings statistics_cache_size, statistics_cache_policy, statistics_cache_size_ratio; SYSTEM DROP STATISTICS CACHE), loaded on demand and merged on the fly for every…
featureBackported in: Lazy materialization now works for LIMIT N queries over JOINs. Controlled by the new setting query_plan_lazy_materialization_for_join, enabled by default. . (Nikolai Kochetov).
featureBackported in: Sets of IN with a subquery spill to disk by default: max_bytes_ratio_before_external_set now defaults to 0.5, so a set spills once query memory exceeds half of the memory that the server or user memory limits leave. Set it to 0 to keep such sets in memory. Sets on disk are built and…
featureBackported in: Estimating subcolumn sizes for PREWHERE (for example m['key'] of a Map(LowCardinality(String), String) column) no longer reads the LowCardinality dictionary or the Variant discriminators header of every data part, which made planning slow on object storage, especially for each new…
featureBackported in: Improved automatic_parallel_replicas observability by adding outcome-specific ProfileEvents, split optimization/plan-build timing counters, and runtime statistics-collection timing in query_log. (Nikita Taranov).
featureBackported in: Queries with FINAL now use the virtual row optimization of reading in order. Supported for ReplacingMergeTree, CollapsingMergeTree, VersionedCollapsingMergeTree, SummingMergeTree, AggregatingMergeTree and CoalescingMergeTree. (Vladimir Cherkasov).
featureBackported in: Reduced the memory usage of incremental BACKUP: the base backup keeps only the sizes and checksums of its files instead of the full information about each file. For a base backup of millions of files this saves gigabytes. (Julia Kartseva).
featureBackported in: Compare TOTP codes and plaintext_password passwords in constant time, so that the response time does not depend on how many leading characters are correct or on which time window matched. (Vladimir Cherkasov).
featureBackported in: A replica that takes part in a parallel replicas read now follows the initiator even when its own automatic_parallel_replicas_mode is not 0, for example when allow_feature_tier drops the value the initiator sends, or when INSERT ... SETTINGS automatic_parallel_replicas_mode = 2…
featureBackported in: The "Format" button in the Web UI now supports inputs with multiple queries: each query is formatted separately and replaced in place, keeping the comments and delimiters between them. (Alexey Milovidov).
featureBackported in: Automatic parallel replicas settings (automatic_parallel_replicas_mode, automatic_parallel_replicas_min_bytes_per_replica) are now production tier instead of experimental. (Nikita Taranov).
featureBackported in: The nodes in the output of EXPLAIN AST graph = 1 are numbered sequentially instead of being identified by memory addresses, so the output is the same on every run. (Raúl Marín).
featureBackported in: Fatal error reports no longer contain runtime memory addresses: stack frames and the fault address are printed as offsets inside the executable or a library, which can be passed to addr2line, and other addresses are hidden. system.crash_log.fault_address now contains the offset…
featureBackported in: A JOIN whose ON condition has a constant-false conjunct now works with the experimental parallel_replicas_plan_based setting when the join uses grace_hash, spills to disk, or runs with join_use_nulls = 1, instead of terminating the server or failing with NOT_FOUND_COLUMN_IN_BLOCK.…
featureBackported in: Fix a possible crash or out-of-bounds memory access when reading from HDFS if a DataNode or NameNode sends malformed responses. (Alexey Milovidov).
featureBackported in: Adds a minimum threshold for spilling in-memory data when external aggregation is enabled. (Mikhail Artemenko).
featureBackported in: System tables no longer contain runtime memory addresses: stack frames outside the main binary are stored as 0, system.trace_log.ptr is an identifier instead of an address, and the ids in system.processors_profile_log are indices within the query. (Raúl Marín).
featureBackported in: Improve glue catalog cache. Let's cache only mapping column name -> timestamp, not the whole metadata. (Konstantin Vedernikov).
featureBackported in: The obfuscate table function and the clickhouse obfuscator tool now support LowCardinality, Enum, Date32, DateTime64, Decimal, wide integers, IPv4, IPv6, BFloat16, Tuple and Map columns. The exception for an unsupported type now names the type and the column. (Alexey Milovidov).
featureBackported in: Hardening: certificate chains containing the compromised eMudhra intermediate CAs or the roots that issued them (emSign Root TLS CA - G3, emSign ECC Root CA - G3) are rejected even if the trust store of the operating system still trusts them. The CA certificates embedded into the…
featureBackported in: This patch supports writeFileUsingBlobWritingFunction for disks with multiple configured locations. (Mikhail Artemenko).
featureNO CL ENTRY: 'Revert "Backport to 26.10: Lower Bound External Group By Spill"'. (Mikhail Artemenko).
featureBackported in: Make 03673_columns_description_cache and 04257_102926_columns_descriptions_cache_nested_eviction robust to asynchronous part removal, applying the fix already made to their sibling 04510_shared_part_schema_metadata_eviction. (Groene AI).
featureBackported in: Query cache: do not store a partial subquery result. (Konstantin Bogdanov).
featureBackported in: Rewrite arrayJoin(emptyArrayToSingle(x)) -> LEFT ARRAY JOIN x. (Yarik Briukhovetskyi).
featureBackported in: Fix h3CellAreaM2/h3CellAreaRads2 precision loss for cells crossing the antimeridian. (Konstantin Bogdanov).
featureBackported in: Fix flaky integration test test_prometheus_protocols/test_evaluation.py::test_multiple_instant_vectors, which compared the series of an instant vector in a fixed order. (Groene AI).
featureBackported in: Use node ids instead of addresses in pipeline diagnostics. (Raúl Marín).
featureBackported in: Exit early in sliceContains for short and early-hit searches. (Raúl Marín).
featureBackported in: Fix Broken pipe in test 05345_robin_hood_large_hash_tables. (Alexey Milovidov).
featureBackported in: Lazily replicated ARRAY JOIN passenger columns hold only the rows of their output block again, so materialized views, max_result_bytes and squashing size ARRAY JOIN output correctly. (Groene AI).
featureBackported in: With distributed_plan_join_runtime_filters, a runtime filter that is no longer needed by its receiver no longer produces EXCHANGE_PEER_DISCONNECTED errors in system.errors and in the log. (Alexander Gololobov).
featureBackported in: Rename integer division setting to sql_compatible_division_for_integers. (Ilya Yatsishin).
featureBackported in: Revert "SLRU filesystem cache: promote partially downloaded file segments". (Sema Checherinda).
featureBackported in: Pin the old server image of test_custom_s3_disk_remote_url_allow_hosts to 26.8.26.12. (Groene AI).
featureBackported in: Decline automatic parallel replicas when the coordinated read gets another read's analysis. (Nikita Taranov).
featureBackported in: Keep the sliceContains block loop independent of the first-vector test on AArch64. (Groene AI).
+61 more items in the release notesfeatureBackported in: Add the tcp_with_proxy_allow_interserver_mode server setting, which allows disabling the native protocol's interserver mode on tcp_with_proxy_port. (Pradeep Chhetri).
fixBackported in: Fix a possible crash or out-of-bounds memory access when reading from HDFS if a DataNode or NameNode sends malformed responses. (Alexey Milovidov).
otherBackported in: Improve glue catalog cache. Let's cache only mapping column name -> timestamp, not the whole metadata. (Konstantin Vedernikov).
fixBackported in: Fix a second-factor bypass in the MySQL protocol: a user configured with plaintext_password or double_sha1_password plus a time-based one-time password could authenticate with the password alone, because the mysql_native_password auth response was verified without checking the TOTP.…
fixBackported in: Fix query parameters in the filter of CREATE ROW POLICY and ALTER ROW POLICY being stored unsubstituted: a {db:Identifier} table qualifier was resolved in the current database of the query reading the table, and a value parameter made every read of the table fail. The parameters are…
fixBackported in: Fixed a row-policy bypass where an authenticated user could forge the result of a scalar subquery used in a row policy (e.g. USING has((SELECT ...), col)) by sending a Scalar packet (as with clickhouse-client --external ... --scalar) named with the subquery's query-tree hash, and…
fixBackported in: Creating a FileLog tables now require READ and WRITE privileges, analogous to File tables. (Robert Schulze).
fixBackported in: For users with a time-based one-time password, the error REQUIRED_SECOND_FACTOR is now returned whenever the code is missing, regardless of whether the password is correct. Previously a wrong password without a code returned AUTHENTICATION_FAILED, which allowed guessing the password…
fixBackported in: Keeper no longer pre-allocates memory for the array size declared in a request (for example the ACL list in Create) before it reads the array elements. Before this fix, a small malformed request could cause a large memory allocation. (Antonio Andelic).
fixBackported in: Fixed wrong strings, sometimes followed by an UNKNOWN_PACKET_FROM_SERVER error, when a sparse Nullable(String) column is sent over the native protocol to clickhouse-client or between servers by remote, cluster and Distributed, for example toString(nullIf(x, 0)) over a sparse column,…
fixBackported in: Fix polygonConvexHullCartesian returning garbage for a polygon with an empty exterior ring and a non-empty interior ring. (Alexey Milovidov).
fixBackported in: Fixes a crash in estimateCompressionRatio with codecs DoubleDelta and Gorilla. (Robert Schulze).
otherBackported in: Make 03673_columns_description_cache and 04257_102926_columns_descriptions_cache_nested_eviction robust to asynchronous part removal, applying the fix already made to their sibling 04510_shared_part_schema_metadata_eviction. (Groene AI).
otherBackported in: Do not skip the client output teardown when the terminal is gone. (Groene AI).
fixBackported in: Fixed a query silently returning fewer rows after an unrelated JOIN had run: a join runtime filter (enable_join_runtime_filters) moved into a read's PREWHERE made the read record a "no matching rows" verdict in the query condition cache for granules, or Parquet row groups, whose…
fixBackported in: Mask credentials embedded in HTTP URIs. (Han Fei).
fixBackported in: Fix a stack overflow on a deeply nested query plan received in the QueryPlan packet of the native protocol. (Alexey Milovidov).
fixBackported in: Fix use-after-free in complex key ssd_cache dictionaries when the cache file wraps around its partitions. Fix CANNOT_READ_ALL_DATA when reading long values of ssd_cache dictionaries from the cache file. (Alexey Milovidov).
fixBackported in: Fixed count() with WHERE d = toDate32('1900-01-01') (or !=) over a MergeTree Date32 column returning the number of 1970-01-01 rows when optimize_trivial_count_with_sparsity_filter is enabled (the default since 26.8). Closes. (Groene AI).
breakingBackported in: Remove object addresses from NATS, io_uring and HDFS messages. (Raúl Marín).
breakingBackported in: Remove object addresses from logical error messages. (Raúl Marín).
otherBackported in: Reduce repeated IN evaluation when many adjacent singleton row-mask updates are applied to one part. (Sergey Kuznetsov).
featureBackported in: Estimating subcolumn sizes for PREWHERE (for example m['key'] of a Map(LowCardinality(String), String) column) no longer reads the LowCardinality dictionary or the Variant discriminators header of every data part, which made planning slow on object storage, especially for each new…
otherBackported in: BACKUP uses less memory per file it backs up: an entry of the backup no longer keeps a copy of the description of the disk the file is on. A backup of a table with millions of files saves a few gigabytes. (Julia Kartseva).
otherBackported in: Fatal error reports no longer contain runtime memory addresses: stack frames and the fault address are printed as offsets inside the executable or a library, which can be passed to addr2line, and other addresses are hidden. system.crash_log.fault_address now contains the offset…
otherBackported in: A JOIN whose ON condition has a constant-false conjunct now works with the experimental parallel_replicas_plan_based setting when the join uses grace_hash, spills to disk, or runs with join_use_nulls = 1, instead of terminating the server or failing with NOT_FOUND_COLUMN_IN_BLOCK.…
fixBackported in: Fixed a read of uninitialized memory when a Variant, Dynamic or JSON column holding only NULL values is compressed, for example on insert into a Memory table created with compress = 1, or when a block is compressed for a JOIN or for the query result cache. The reported size of such…
fixBackported in: BACKUP and RESTORE to S3 retry requests that fail with ExpiredToken after refreshing credentials. The retry time limit is 120 seconds by default and can be changed with backup_s3_expired_token_retry_timeout_ms. (Nikolai Ovchinnikov).
fixBackported in: Fixed a server crash (or missing rows in the view, with a Null source table) on INSERT into the source table of a materialized view when a row policy or additional_table_filters of a table read by the view references the source table, e.g. USING x IN src. (Groene AI).
fixBackported in: Fix a segmentation fault in GROUP BY with the adaptive aggregator and external aggregation when a memory limit was hit while draining staged records. (Alexey Milovidov).
fixBackported in: Fix segfault when reading a MySQLDump file whose CREATE TABLE query has an ENUM column without a list of values or with non-string values. (Alexey Milovidov).
fixBackported in: Fixed a heap-use-after-free in the view dependency graph after renaming a materialized view whose query selects from itself. CREATE MATERIALIZED VIEW and ALTER TABLE ... MODIFY QUERY now reject a query that makes a materialized view select from itself. (Groene AI).
fixBackported in: Fix a stack overflow when a Redis server replies with deeply nested arrays. (Alexey Milovidov).
fixBackported in: Fix a stack overflow on a Protobuf schema with a very long chain of nested messages. (Alexey Milovidov).
otherBackported in: Not required. (Groene AI).
otherBackported in: An equality with a Bool value that no statistics can estimate, such as WHERE j.f = false on a JSON path, a Bool column without statistics, or IN with a Bool set, is now estimated to match half of the rows per value instead of 1%, so a join on such a filtered table builds its hash…
fixBackported in: Fixed crashes of the interactive line editor of clickhouse-client and clickhouse-local, which also crashed the server when used through the web terminal or an SSH session: after a history search that found nothing on an edited recalled query, when the prompt or the line given to an…
breakingBackported in: CREATE TABLE x AS src inherits the credentials embedded in the engine of src, which are masked in SHOW CREATE TABLE. It now requires the SELECT privilege on src when the inherited definition can carry credentials, instead of only SHOW COLUMNS. (Yarik Briukhovetskyi).
otherBackported in: Reduced the memory usage of incremental BACKUP: the base backup keeps only the sizes and checksums of its files instead of the full information about each file. For a base backup of millions of files this saves gigabytes. (Julia Kartseva).
otherBackported in: Compare TOTP codes and plaintext_password passwords in constant time, so that the response time does not depend on how many leading characters are correct or on which time window matched. (Vladimir Cherkasov).
otherBackported in: The nodes in the output of EXPLAIN AST graph = 1 are numbered sequentially instead of being identified by memory addresses, so the output is the same on every run. (Raúl Marín).
fixBackported in: Requesting SSH-key authentication over the native protocol no longer reveals whether the named user exists, or how it authenticates, before its signature is checked. (Shaohua Wang).
fixBackported in: Reject CR and LF characters in values sent as HTTP headers or in the request line of outgoing HTTP requests (BigQuery, YTsaurus, url, DataLake catalogs, GCP OAuth, AI functions, S3 storage class), which could be used to inject headers or additional requests. (Raúl Marín).
fixBackported in: The setting http_zlib_compression_level now applies only to the gzip and deflate HTTP response compression. The level of other methods (zstd, lz4, br, xz, bz2) is controlled by output_format_compression_level. Previously, a level suitable for zlib was passed to every method, which…
otherBackported in: Use node ids instead of addresses in pipeline diagnostics. (Raúl Marín).