min_* MergeTree table settings

ClickHouse MergeTree table settings in the min_* generated group.

These settings are available in system.merge_tree_settings and are autogenerated from ClickHouse source.

min_absolute_delay_to_close

Minimal absolute delay to close, stop serving requests and not return Ok during status check.

min_columns_to_activate_adaptive_write_buffer

Allow to reduce memory usage for tables with lots of columns by using adaptive writer buffers.

Compared against the number of streams a wide part writes, which can greatly exceed its number of columns: a Map with many buckets, or a deeply nested Array or Tuple, writes many streams for a single column, and one write buffer is allocated per stream.

Possible values:

• 0 - disabled
• 1 - always enabled

min_compress_block_size

Minimum size of blocks of uncompressed data required for compression when writing the next mark. You can also specify this setting in the global settings (see min_compress_block_size setting). The value specified when the table is created overrides the global value for this setting.

min_index_granularity_bytes

Min allowed size of data granules in bytes.

To provide a safeguard against accidentally creating tables with very low index_granularity_bytes.

min_marks_to_honor_max_concurrent_queries

The minimal number of marks read by the query for applying the max_concurrent_queries setting.

Queries will still be limited by other max_concurrent_queries settings.

Possible values:

• Positive integer.
• 0 — Disabled (max_concurrent_queries limit applied to no queries).

Example

<min_marks_to_honor_max_concurrent_queries>10</min_marks_to_honor_max_concurrent_queries>

min_merge_bytes_to_use_direct_io

The minimum data volume for merge operation that is required for using direct I/O access to the storage disk. When merging data parts, ClickHouse calculates the total storage volume of all the data to be merged. If the volume exceeds min_merge_bytes_to_use_direct_io bytes, ClickHouse reads and writes the data to the storage disk using the direct I/O interface (O_DIRECT option). If min_merge_bytes_to_use_direct_io = 0, then direct I/O is disabled.

min_partition_age_to_force_merge_seconds

Merge parts in a partition if every part in it is older than this value, i.e. the partition no longer receives inserts. Unlike min_age_to_force_merge_seconds with min_age_to_force_merge_on_partition_only, the partition does not have to fit into a single merge: each merge still respects max_bytes_to_merge_at_max_space_in_pool. Works for Simple and StochasticSimple merge selectors.

The age compared here is the age of the youngest part in the partition (now - modification_time, minimised over its parts), so the rule arms only once every part has aged past this value. Any new part resets it: an insert, a mutation, and also each merge this setting itself assigns, because the merged part is new. Forcing therefore disarms as soon as a forced merge lands and re-arms only after this value elapses again with no new parts, so a partition that needs several merges is compacted over successive rounds spaced by this interval rather than in one continuous pass.

Forcing works exactly like min_age_to_force_merge_seconds: it waives the size-ratio requirement that normally keeps an unbalanced merge from being assigned, and it also waives the min_parts_to_merge_at_once floor, so a forced merge can cover fewer parts than that floor asks for. No other heuristic is turned off: merge_selector_window_size still bounds which parts are examined, and merge_selector_enable_heuristic_to_remove_small_parts_at_right still trims a trailing small part from a selected range of three parts or more. Either of those that a workload needs off must be turned off explicitly through its own setting.

Under the Simple and StochasticSimple merge selectors, cannot be combined with min_age_to_force_merge_seconds together with min_age_to_force_merge_on_partition_only. That pair merges a whole partition at once, and only such a merge is marked final, which is what lets a ReplacingMergeTree run CLEANUP; forcing regular merges by partition age would pre-empt it. Use the pair for partitions that fit into a single merge and this setting for the ones that do not. Other selectors ignore this setting, so the combination is accepted with them.

Possible values:

• Positive integer.

min_parts_to_merge_at_once

Minimal amount of data parts which merge selector can pick to merge at once (expert level setting, don’t change if you don’t understand what it is doing). 0 - disabled. Works for Simple and StochasticSimple merge selectors.

min_replicated_logs_to_keep

Keep about this number of last records in ZooKeeper log, even if they are obsolete. It doesn’t affect work of tables: used only to diagnose ZooKeeper log before cleaning.

Possible values:

• Any positive integer.

min_unreserved_disk_space_for_merge

Keeps the specified amount of unreserved disk space (in bytes) out of reach of background merges, so that they cannot starve inserts. The amount is subtracted from the free space, and the limit on the total size of a merge’s source parts is derived from what remains with a safety factor on top, so the largest allowed merge stays well under that. The limit shrinks as the disk fills up and reaches zero once free space is down to this amount, at which point no background merge is selected at all. (0 means disabled)

Unlike keep_free_space_bytes, the protected space stays usable by ClickHouse itself - it is only kept out of merge scheduling. Mutations are not limited by it and can still consume the protected space, and the setting has no effect on disks with unlimited space (such as object storage).

Merges initiated by OPTIMIZE with FINAL or with an explicit PARTITION ignore this setting. Merges that drop wholly expired parts are selected whenever the limit is above zero, whatever their size, and nothing is selected at zero. Every merge reserves at least 1 MiB, so a merge started with less than 1 MiB left above the protected space can take up to 1 MiB of it. A ReplicatedMergeTree replica also runs queued drops when its own limit is zero, if a row TTL without WHERE is the table’s only TTL. With a GROUP BY, WHERE or column TTL such a drop rewrites the remaining rows: ReplicatedMergeTree postpones it by the size of its parts, but MergeTree does not and can write them into the protected space.

Once the limit is zero a plain OPTIMIZE assigns nothing: it is a no-op, or throws CANNOT_ASSIGN_OPTIMIZE with optimize_throw_if_noop = 1. Use FINAL or an explicit PARTITION to merge into the protected space. On ReplicatedMergeTree that exemption is recorded in the replication log entry only when the setting is non-zero on the replica that queues it, so an OPTIMIZE queued while it was 0 still runs under the headroom.

ALTER TABLE ... MODIFY SETTING is not replicated, so give every replica the same value: a replica with a larger headroom re-applies it to merges assigned elsewhere and can postpone them indefinitely. Keep the value at 0 until every replica runs a version that knows the setting - an older replica cannot parse the log entries written for the exempt merges and stops pulling any further entries for the table, including those for ordinary inserts.