You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: .github/pull_request_template.md
+1-1
Original file line number
Diff line number
Diff line change
@@ -4,4 +4,4 @@
4
4
## Checklist
5
5
-[ ] Delete items not relevant to your PR
6
6
-[ ] URL changes should add a redirect to the old URL via https://github.com/ClickHouse/clickhouse-docs/blob/main/docusaurus.config.js
7
-
-[ ] If adding a new integration page, also add an entry to the integrations list here: https://github.com/ClickHouse/clickhouse-docs/blob/main/docs/en/integrations/index.mdx
7
+
-[ ] If adding a new integration page, also add an entry to the integrations list here: https://github.com/ClickHouse/clickhouse-docs/blob/main/docs/integrations/index.mdx
Copy file name to clipboardExpand all lines: images/knowledgebase/connection_timeout_remote_remoteSecure.md
+1-1
Original file line number
Diff line number
Diff line change
@@ -5,7 +5,7 @@ description: "When using the `remote` or `remoteSecure` table functions on a nod
5
5
# Code: 279. DB::NetException: All connection tries failed.
6
6
7
7
**Problem**
8
-
[`remote()` or `remoteSecure()`](https://clickhouse.com/docs/en/sql-reference/table-functions/remote/) table function allows the access of remote table from another ClickHouse node.
8
+
[`remote()` or `remoteSecure()`](https://clickhouse.com/docs/sql-reference/table-functions/remote/) table function allows the access of remote table from another ClickHouse node.
9
9
10
10
When using these functions on a node that is located more than 100ms (latency wise) away from the remote node, it is common to encounter the following timeout error.
Copy file name to clipboardExpand all lines: images/knowledgebase/delete-old-data.md
+3-3
Original file line number
Diff line number
Diff line change
@@ -20,7 +20,7 @@ TTL can also be used to move data not only to [/dev/null](https://en.wikipedia.o
20
20
More details on [configuring TTL](../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-ttl).
21
21
22
22
## DELETE FROM
23
-
[DELETE FROM](/docs/en/sql-reference/statements/delete.md) allows standard DELETE queries to be run in ClickHouse. The rows targeted in the filter clause are marked as deleted, and removed from future result sets. Cleanup of the rows happens asynchronously.
23
+
[DELETE FROM](/docs/sql-reference/statements/delete.md) allows standard DELETE queries to be run in ClickHouse. The rows targeted in the filter clause are marked as deleted, and removed from future result sets. Cleanup of the rows happens asynchronously.
24
24
25
25
:::note
26
26
DELETE FROM is an experimental feature and must be enabled with:
@@ -31,7 +31,7 @@ SET allow_experimental_lightweight_delete = true;
31
31
32
32
## ALTER DELETE {#alter-delete}
33
33
34
-
ALTER DELETE removes rows using asynchronous batch operations. Unlike DELETE FROM, queries run after the ALTER DELETE and before the batch operations complete will include the rows targeted for deletion. For more details see the [ALTER DELETE](/docs/en/sql-reference/statements/alter/delete.md) docs.
34
+
ALTER DELETE removes rows using asynchronous batch operations. Unlike DELETE FROM, queries run after the ALTER DELETE and before the batch operations complete will include the rows targeted for deletion. For more details see the [ALTER DELETE](/docs/sql-reference/statements/alter/delete.md) docs.
35
35
36
36
`ALTER DELETE` can be issued to flexibly remove old data. If you need to do it regularly, the main downside will be the need to have an external system to submit the query. There are also some performance considerations since mutations rewrite complete parts even there is only a single row to be deleted.
37
37
@@ -49,4 +49,4 @@ More details on [manipulating partitions](../../sql-reference/statements/alter/p
49
49
50
50
It’s rather radical to drop all data from a table, but in some cases it might be exactly what you need.
51
51
52
-
More details on [table truncation](/docs/en/sql-reference/statements/truncate.md).
52
+
More details on [table truncation](/docs/sql-reference/statements/truncate.md).
# Execute SYSTEM statements on all nodes in ClickHouse Cloud
2
2
3
-
In order to execute the same [query](url) on all nodes of a ClickHouse cloud service, we can use [clusterAllReplicas](https://clickhouse.com/docs/en/sql-reference/table-functions/cluster/).
3
+
In order to execute the same [query](url) on all nodes of a ClickHouse cloud service, we can use [clusterAllReplicas](https://clickhouse.com/docs/sql-reference/table-functions/cluster/).
4
4
E.g. in order to get entries from a (node-local) system table from all nodes, you can use:
5
5
```
6
6
SELECT ... FROM clusterAllReplicas(default, system.TABLE) ...;
7
7
```
8
8
9
-
Similarly, you can execute the same [SYSTEM statement](https://clickhouse.com/docs/en/sql-reference/statements/system/) on all nodes with a single statement, by using the [ON CLUSTER](https://clickhouse.com/docs/en/sql-reference/distributed-ddl/) clause:
9
+
Similarly, you can execute the same [SYSTEM statement](https://clickhouse.com/docs/sql-reference/statements/system/) on all nodes with a single statement, by using the [ON CLUSTER](https://clickhouse.com/docs/sql-reference/distributed-ddl/) clause:
10
10
```
11
11
SYSTEM ... ON CLUSTER default;
12
12
```
13
13
14
-
For example for [dropping the filesystem cache](https://clickhouse.com/docs/en/sql-reference/statements/system/#drop-filesystem-cache) from all nodes, you can use:
14
+
For example for [dropping the filesystem cache](https://clickhouse.com/docs/sql-reference/statements/system/#drop-filesystem-cache) from all nodes, you can use:
Copy file name to clipboardExpand all lines: images/knowledgebase/improve-map-performance.md
+2-2
Original file line number
Diff line number
Diff line change
@@ -1,13 +1,13 @@
1
1
---
2
2
sidebar_position: 1
3
-
description: "Map lookups such as `a['key']' works with linear complexity (mentioned [here](https://clickhouse.com/docs/en/sql-reference/data-types/map)) and can be inefficient."
3
+
description: "Map lookups such as `a['key']' works with linear complexity (mentioned [here](https://clickhouse.com/docs/sql-reference/data-types/map)) and can be inefficient."
4
4
---
5
5
6
6
# Improving Map performance
7
7
8
8
**Problem**
9
9
10
-
Map lookup such as `a['key']` works with linear complexity (mentioned [here](https://clickhouse.com/docs/en/sql-reference/data-types/map)) and can be inefficient. This is because selecting a value with a specific key from a table would require iterating through all keys (~M) across all rows (N) in the Map column, resulting in ~MxN lookups.
10
+
Map lookup such as `a['key']` works with linear complexity (mentioned [here](https://clickhouse.com/docs/sql-reference/data-types/map)) and can be inefficient. This is because selecting a value with a specific key from a table would require iterating through all keys (~M) across all rows (N) in the Map column, resulting in ~MxN lookups.
11
11
12
12
A lookup using Map can be 10x slower than a String column. The experiment below also shows ~10x slowdown for cold query, and difference in multiple magnitudes of data processed (7.21 MB vs 5.65 GB).
Copy file name to clipboardExpand all lines: images/knowledgebase/ingest-parquet-files-in-s3.md
+4-4
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,7 @@ normal login users usually don't work since they may have been configured with a
10
10
The following is a very simple example that you can use to test the mechanics of accessing your parquet files successfully prior to applying to your actual data.
11
11
12
12
If you need an example of creating a user and bucket, you can follow the first two sections (create user and create bucket):
Copy file name to clipboardExpand all lines: images/knowledgebase/production.md
+1-1
Original file line number
Diff line number
Diff line change
@@ -60,5 +60,5 @@ Here is some guidance on how to choose between them:
60
60
Many teams who initially think that `lts` is the way to go often switch to `stable` anyway because of some recent feature that’s important for their product.
61
61
62
62
:::warning
63
-
One more thing to keep in mind when upgrading ClickHouse: we’re always keeping an eye on compatibility across releases, but sometimes it’s not reasonable to keep and some minor details might change. So make sure you check the [changelog](/docs/en/whats-new/changelog/index.md) before upgrading to see if there are any notes about backward-incompatible changes.
63
+
One more thing to keep in mind when upgrading ClickHouse: we’re always keeping an eye on compatibility across releases, but sometimes it’s not reasonable to keep and some minor details might change. So make sure you check the [changelog](/docs/whats-new/changelog/index.md) before upgrading to see if there are any notes about backward-incompatible changes.
Copy file name to clipboardExpand all lines: images/knowledgebase/time-series.md
+1-1
Original file line number
Diff line number
Diff line change
@@ -11,7 +11,7 @@ ClickHouse is a generic data storage solution for [OLAP](../../faq/general/olap.
11
11
12
12
First of all, there are **[specialized codecs](../../sql-reference/statements/create/table#specialized-codecs)** which make typical time-series. Either common algorithms like `DoubleDelta` and `Gorilla` or specific to ClickHouse like `T64`.
13
13
14
-
Second, time-series queries often hit only recent data, like one day or one week old. It makes sense to use servers that have both fast NVMe/SSD drives and high-capacity HDD drives. ClickHouse [TTL](/docs/en/engines/table-engines/mergetree-family/mergetree.md/##table_engine-mergetree-multiple-volumes) feature allows to configure keeping fresh hot data on fast drives and gradually move it to slower drives as it ages. Rollup or removal of even older data is also possible if your requirements demand it.
14
+
Second, time-series queries often hit only recent data, like one day or one week old. It makes sense to use servers that have both fast NVMe/SSD drives and high-capacity HDD drives. ClickHouse [TTL](/docs/engines/table-engines/mergetree-family/mergetree.md/##table_engine-mergetree-multiple-volumes) feature allows to configure keeping fresh hot data on fast drives and gradually move it to slower drives as it ages. Rollup or removal of even older data is also possible if your requirements demand it.
15
15
16
16
Even though it’s against ClickHouse philosophy of storing and processing raw data, you can use [materialized views](../../sql-reference/statements/create/view.md) to fit into even tighter latency or costs requirements.
## How to calculate the ratio of empty/zero values in every column in a table
13
13
14
-
If a column is sparse (empty or contains mostly zeros), ClickHouse can encode it in a sparse format and automatically optimize calculations - the data does not require full decompression during queries. In fact, if you know how sparse a column is, you can define its ratio using the [`ratio_of_defaults_for_sparse_serialization` setting](https://clickhouse.com/docs/en/operations/settings/merge-tree-settings#ratio_of_defaults_for_sparse_serialization) to optimize serialization.
14
+
If a column is sparse (empty or contains mostly zeros), ClickHouse can encode it in a sparse format and automatically optimize calculations - the data does not require full decompression during queries. In fact, if you know how sparse a column is, you can define its ratio using the [`ratio_of_defaults_for_sparse_serialization` setting](https://clickhouse.com/docs/operations/settings/merge-tree-settings#ratio_of_defaults_for_sparse_serialization) to optimize serialization.
15
15
16
16
This handy query can take a while, but it analyzes every row in your table and determines the ratio of values that are zero (or the default) in every column in the specified table:
Copy file name to clipboardExpand all lines: knowledgebase/how-to-check-my-clickhouse-cloud-sevice-state.mdx
+1-1
Original file line number
Diff line number
Diff line change
@@ -18,7 +18,7 @@ How do I check my ClickHouse Cloud Service state? I want to check if the Service
18
18
The [ClickHouse Cloud API](/cloud/manage/api/api-overview) is great for checking the status of a cloud service. You need to create an API Key in your service before you can use the Cloud API. You can do this in ClickHouse Cloud [clickhouse.cloud](https://console.clickhouse.cloud):
Check out the [docs](https://clickhouse.com/docs/en/operations/server-configuration-parameters/settings#max-thread-pool-size) for more details on the settings above and other settings that affect the Global Thread pool.
34
+
Check out the [docs](https://clickhouse.com/docs/operations/server-configuration-parameters/settings#max-thread-pool-size) for more details on the settings above and other settings that affect the Global Thread pool.
Copy file name to clipboardExpand all lines: knowledgebase/json_extract_example.mdx
+1-1
Original file line number
Diff line number
Diff line change
@@ -11,7 +11,7 @@ keywords: ['JSON', 'extract base types']
11
11
12
12
## JSON Extract example
13
13
14
-
This is just a short example that illustrates the use of [JSONExtract](https://clickhouse.com/docs/en/sql-reference/functions/json-functions) functions.
14
+
This is just a short example that illustrates the use of [JSONExtract](https://clickhouse.com/docs/sql-reference/functions/json-functions) functions.
Copy file name to clipboardExpand all lines: knowledgebase/kafka-clickhouse-json.mdx
+1-1
Original file line number
Diff line number
Diff line change
@@ -70,7 +70,7 @@ ENGINE = Kafka(
70
70
);
71
71
```
72
72
73
-
Note that we're using the [`JSONAsObject`](https://clickhouse.com/docs/en/interfaces/formats#jsonasobject) format, which will ensure that incoming messages are made available as a JSON object.
73
+
Note that we're using the [`JSONAsObject`](https://clickhouse.com/docs/interfaces/formats#jsonasobject) format, which will ensure that incoming messages are made available as a JSON object.
74
74
This format can only be parsed into a table that has a single column with the `JSON` type.
75
75
76
76
Next, we'll create the underlying table to store the Wiki data:
The `clickhouse-local` tool makes it quick and easy to read data from MySQL and output the data into lots of different formats, including Parquet, CSV, and JSON. We are going to:
15
15
16
-
- Use the [`mysql` table function](https://clickhouse.com/docs/en/sql-reference/table-functions/mysql) to read the data
16
+
- Use the [`mysql` table function](https://clickhouse.com/docs/sql-reference/table-functions/mysql) to read the data
17
17
- Use the `INTO OUTFILE _filename_ FORMAT` clause and specify the desired output format
18
18
19
19
The `clickhouse-local` tool is a part of the ClickHouse binary. Download it using the following:
Copy file name to clipboardExpand all lines: knowledgebase/olap.mdx
+1-1
Original file line number
Diff line number
Diff line change
@@ -38,7 +38,7 @@ All database management systems could be classified into two groups: OLAP (Onlin
38
38
39
39
In practice OLAP and OLTP are not categories, it’s more like a spectrum. Most real systems usually focus on one of them but provide some solutions or workarounds if the opposite kind of workload is also desired. This situation often forces businesses to operate multiple storage systems integrated, which might be not so big deal but having more systems make it more expensive to maintain. So the trend of recent years is HTAP (**Hybrid Transactional/Analytical Processing**) when both kinds of the workload are handled equally well by a single database management system.
40
40
41
-
Even if a DBMS started as a pure OLAP or pure OLTP, they are forced to move towards that HTAP direction to keep up with their competition. And ClickHouse is no exception, initially, it has been designed as [fast-as-possible OLAP system](https://clickhouse.com/docs/en/faq/general/why-clickhouse-is-so-fast) and it still does not have full-fledged transaction support, but some features like consistent read/writes and mutations for updating/deleting data had to be added.
41
+
Even if a DBMS started as a pure OLAP or pure OLTP, they are forced to move towards that HTAP direction to keep up with their competition. And ClickHouse is no exception, initially, it has been designed as [fast-as-possible OLAP system](https://clickhouse.com/docs/faq/general/why-clickhouse-is-so-fast) and it still does not have full-fledged transaction support, but some features like consistent read/writes and mutations for updating/deleting data had to be added.
42
42
43
43
The fundamental trade-off between OLAP and OLTP systems remains:
0 commit comments