Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions _partials/_not-supported-for-azure.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
<Highlight type="note">

This feature is on our roadmap for $CLOUD_LONG on Microsoft Azure. Stay tuned!

</Highlight>
3 changes: 3 additions & 0 deletions _partials/_prometheus-integrate.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import IntegrationPrereqs from "versionContent/_partials/_integration-prereqs.mdx";
import NotSupportedAzure from "versionContent/_partials/_not-supported-for-azure.mdx";

[Prometheus][prometheus] is an open-source monitoring system with a dimensional data model, flexible query language, and a modern alerting approach.

Expand All @@ -20,6 +21,8 @@ To follow the steps on this page:
- [Install Postgres Exporter][install-exporter].
To reduce latency and potential data transfer costs, install Prometheus and Postgres Exporter on a machine in the same AWS region as your $SERVICE_LONG.

<NotSupportedAzure />
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not visible in the review instance.


## Export $SERVICE_LONG telemetry to Prometheus

To export your data, do the following:
Expand Down
9 changes: 9 additions & 0 deletions integrations/aws.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,11 @@ keywords: [AWS, integrations]

import IntegrationPrereqsCloud from "versionContent/_partials/_integration-prereqs-cloud-only.mdx";
import TransitGateway from "versionContent/_partials/_transit-gateway.mdx";
import NotSupportedAzure from "versionContent/_partials/_not-supported-for-azure.mdx";

# Integrate Amazon Web Services with $CLOUD_LONG


[Amazon Web Services (AWS)][aws] is a comprehensive cloud computing platform that provides on-demand infrastructure, storage, databases, AI, analytics, and security services to help businesses build, deploy, and scale applications in the cloud.

This page explains how to integrate your AWS infrastructure with $CLOUD_LONG using [AWS Transit Gateway][aws-transit-gateway].
Expand All @@ -21,6 +23,8 @@ This page explains how to integrate your AWS infrastructure with $CLOUD_LONG usi

- Set up [AWS Transit Gateway][gtw-setup].

<NotSupportedAzure />

## Connect your AWS infrastructure to your $SERVICE_LONGs

To connect to $CLOUD_LONG:
Expand All @@ -33,6 +37,11 @@ To connect to $CLOUD_LONG:

You have successfully integrated your AWS infrastructure with $CLOUD_LONG.






[aws]: https://aws.amazon.com/
[aws-transit-gateway]: https://aws.amazon.com/transit-gateway/
[gtw-setup]: https://docs.aws.amazon.com/vpc/latest/tgw/tgw-getting-started.html
6 changes: 6 additions & 0 deletions integrations/cloudwatch.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ price_plans: [scale, enterprise]
keywords: [integrate]
---

import NotSupportedAzure from "versionContent/_partials/_not-supported-for-azure.mdx";
import IntegrationPrereqsCloud from "versionContent/_partials/_integration-prereqs-cloud-only.mdx";
import CloudWatchExporter from "versionContent/_partials/_cloudwatch-data-exporter.mdx";
import ManageDataExporter from "versionContent/_partials/_manage-a-data-exporter.mdx";
Expand All @@ -24,6 +25,8 @@ This pages explains how to export telemetry data from your $SERVICE_LONG into Cl

- Sign up for [Amazon CloudWatch][cloudwatch-signup].

<NotSupportedAzure />

## Create a data exporter

A $CLOUD_LONG data exporter sends telemetry data from a $SERVICE_LONG to a third-party monitoring
Expand All @@ -33,6 +36,9 @@ tool. You create an exporter on the [project level][projects], in the same AWS r

<ManageDataExporter />




[projects]: /use-timescale/:currentVersion:/security/members/
[pricing-plan-features]: /about/:currentVersion:/pricing-and-account-management/#features-included-in-each-plan
[cloudwatch]: https://aws.amazon.com/cloudwatch/
Expand Down
8 changes: 7 additions & 1 deletion integrations/corporate-data-center.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ keywords: [on-premise, integrations]

import IntegrationPrereqsCloud from "versionContent/_partials/_integration-prereqs-cloud-only.mdx";
import TransitGateway from "versionContent/_partials/_transit-gateway.mdx";
import NotSupportedAzure from "versionContent/_partials/_not-supported-for-azure.mdx";

# Integrate your data center with $CLOUD_LONG

Expand All @@ -18,6 +19,7 @@ This page explains how to integrate your corporate on-premise infrastructure wit
<IntegrationPrereqsCloud />

- Set up [AWS Transit Gateway][gtw-setup].
<NotSupportedAzure />

## Connect your on-premise infrastructure to your $SERVICE_LONGs

Expand All @@ -33,7 +35,11 @@ To connect to $CLOUD_LONG:

</Procedure>

You have successfully integrated your Microsoft Azure infrastructure with $CLOUD_LONG.
You have successfully integrated your corporate data center with $CLOUD_LONG.





[aws-transit-gateway]: https://aws.amazon.com/transit-gateway/
[gtw-setup]: https://docs.aws.amazon.com/vpc/latest/tgw/tgw-getting-started.html
Expand Down
6 changes: 6 additions & 0 deletions integrations/datadog.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ keywords: [integrate]
import IntegrationPrereqsCloud from "versionContent/_partials/_integration-prereqs-cloud-only.mdx";
import DataDogExporter from "versionContent/_partials/_datadog-data-exporter.mdx";
import ManageDataExporter from "versionContent/_partials/_manage-a-data-exporter.mdx";
import NotSupportedAzure from "versionContent/_partials/_not-supported-for-azure.mdx";

# Integrate Datadog with $CLOUD_LONG

Expand Down Expand Up @@ -36,6 +37,8 @@ This page explains how to:

- Install [Datadog Agent][datadog-agent-install].

<NotSupportedAzure />

## Monitor $SERVICE_LONG metrics with Datadog

Export telemetry data from your $SERVICE_LONGs with the time-series and analytics capability enabled to
Expand Down Expand Up @@ -132,6 +135,9 @@ metrics about your $SERVICE_LONGs.
Metrics for your $SERVICE_LONG are now visible in Datadog. Check the Datadog $PG integration documentation for a
comprehensive list of [metrics][datadog-postgres-metrics] collected.




[datadog]: https://www.datadoghq.com/
[datadog-agent-install]: https://docs.datadoghq.com/getting_started/agent/#installation
[datadog-postgres]: https://docs.datadoghq.com/integrations/postgres/
Expand Down
8 changes: 8 additions & 0 deletions integrations/google-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ keywords: [Google Cloud, integrations]

import IntegrationPrereqsCloud from "versionContent/_partials/_integration-prereqs-cloud-only.mdx";
import TransitGateway from "versionContent/_partials/_transit-gateway.mdx";
import NotSupportedAzure from "versionContent/_partials/_not-supported-for-azure.mdx";

# Integrate Google Cloud with $CLOUD_LONG

Expand All @@ -20,6 +21,7 @@ This page explains how to integrate your Google Cloud infrastructure with $CLOUD
<IntegrationPrereqsCloud />

- Set up [AWS Transit Gateway][gtw-setup].
<NotSupportedAzure />

## Connect your Google Cloud infrastructure to your $SERVICE_LONGs

Expand All @@ -37,6 +39,12 @@ To connect to $CLOUD_LONG:

You have successfully integrated your Google Cloud infrastructure with $CLOUD_LONG.







[google-cloud]: https://cloud.google.com/?hl=en
[aws-transit-gateway]: https://aws.amazon.com/transit-gateway/
[gtw-setup]: https://docs.aws.amazon.com/vpc/latest/tgw/tgw-getting-started.html
Expand Down
7 changes: 7 additions & 0 deletions integrations/microsoft-azure.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,11 @@ keywords: [Azure, integrations]

import IntegrationPrereqsCloud from "versionContent/_partials/_integration-prereqs-cloud-only.mdx";
import TransitGateway from "versionContent/_partials/_transit-gateway.mdx";
import NotSupportedAzure from "versionContent/_partials/_not-supported-for-azure.mdx";

# Integrate Microsoft Azure with $CLOUD_LONG


[Microsoft Azure][azure] is a cloud computing platform and services suite, offering infrastructure, AI, analytics, security, and developer tools to help businesses build, deploy, and manage applications.

This page explains how to integrate your Microsoft Azure infrastructure with $CLOUD_LONG using [AWS Transit Gateway][aws-transit-gateway].
Expand All @@ -20,6 +22,7 @@ This page explains how to integrate your Microsoft Azure infrastructure with $CL
<IntegrationPrereqsCloud />

- Set up [AWS Transit Gateway][gtw-setup].
<NotSupportedAzure />

## Connect your Microsoft Azure infrastructure to your $SERVICE_LONGs

Expand All @@ -37,6 +40,10 @@ To connect to $CLOUD_LONG:

You have successfully integrated your Microsoft Azure infrastructure with $CLOUD_LONG.





[aws-transit-gateway]: https://aws.amazon.com/transit-gateway/
[gtw-setup]: https://docs.aws.amazon.com/vpc/latest/tgw/tgw-getting-started.html
[azure]: https://azure.microsoft.com/en-gb/
Expand Down
4 changes: 3 additions & 1 deletion integrations/prometheus.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,9 @@ keywords: [integrate]
---

import PrometheusIntegrate from "versionContent/_partials/_prometheus-integrate.mdx";
import NotSupportedAzure from "versionContent/_partials/_not-supported-for-azure.mdx";

# Integrate Prometheus with $CLOUD_LONG

<PrometheusIntegrate />
<PrometheusIntegrate />

7 changes: 6 additions & 1 deletion use-timescale/data-tiering/about-data-tiering.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
title: About Tiger storage tiers
excerpt: Learn how Tiger helps you save on storage costs. The tiered storage architecture includes a high-performance storage tier and a low-cost object storage tier built on Amazon s3
products: [cloud]
price_plans: [scale, enterprise]
keywords: [tiered storage]
tags: [storage, data management]
cloud_ui:
Expand All @@ -11,6 +10,7 @@ cloud_ui:
---

import TieredStorageBilling from "versionContent/_partials/_tiered-storage-billing.mdx";
import NotSupportedAzure from "versionContent/_partials/_not-supported-for-azure.mdx";

# About storage tiers

Expand All @@ -31,8 +31,12 @@ $CLOUD_LONG high-performance storage comes in the following types:

## Low-cost storage

<Availability products={['cloud']} price_plans={['enterprise','scale']} />

Once you [enable tiered storage][manage-tiering], you can start moving rarely used data to the object tier. The object tier is based on AWS S3 and stores your data in the [Apache Parquet][parquet] format. Within a Parquet file, a set of rows is grouped together to form a row group. Within a row group, values for a single column across multiple rows are stored together. The original size of the data in your $SERVICE_SHORT, compressed or uncompressed, does not correspond directly to its size in S3. A compressed hypertable may even take more space in S3 than it does in $CLOUD_LONG.

<NotSupportedAzure />

Apache Parquet allows for more efficient scans across longer time periods, and $CLOUD_LONG uses other metadata and query optimizations to reduce the amount of data that needs to be fetched to satisfy a query, such as:

- **Chunk skipping**: exclude the chunks that fall outside the query time window.
Expand Down Expand Up @@ -121,6 +125,7 @@ The low-cost storage tier comes with the following limitations:
partitioned on more than one dimension. Make sure your hypertables are
partitioned on time only, before you enable tiered storage.


[blog-data-tiering]: https://www.timescale.com/blog/expanding-the-boundaries-of-postgresql-announcing-a-bottomless-consumption-based-object-storage-layer-built-on-amazon-s3/
[querying-tiered-data]: /use-timescale/:currentVersion:/data-tiering/querying-tiered-data/
[parquet]: https://parquet.apache.org/
Expand Down
16 changes: 9 additions & 7 deletions use-timescale/data-tiering/enabling-data-tiering.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ cloud_ui:
---

import TieredStorageBilling from "versionContent/_partials/_tiered-storage-billing.mdx";
import NotSupportedAzure from "versionContent/_partials/_not-supported-for-azure.mdx";

# Manage storage and tiering

Expand Down Expand Up @@ -54,7 +55,11 @@ This storage type gives you up to 16 TB of storage and is available under [all $

<Availability products={['cloud']} price_plans={['enterprise']} />

This storage type gives you up to 64 TB and 32,000 IOPS, and is available under the [$ENTERPRISE $PRICING_PLAN][pricing-plans]. To get enhanced storage:
This storage type gives you up to 64 TB and 32,000 IOPS, and is available under the [$ENTERPRISE $PRICING_PLAN][pricing-plans].

<NotSupportedAzure />

To get enhanced storage:

<Procedure>

Expand Down Expand Up @@ -87,6 +92,8 @@ You change from enhanced storage to standard in the same way. If you are using o

You enable the low-cost object storage tier in $CONSOLE and then tier the data with policies or manually.

<NotSupportedAzure />

### Enable tiered storage

You enable tiered storage from the `Overview` tab in $CONSOLE.
Expand All @@ -103,12 +110,6 @@ You enable tiered storage from the `Overview` tab in $CONSOLE.

</Procedure>

<Highlight type="note">

Data tiering is available in [Scale and Enterprise][pricing-plans] pricing plans only.

</Highlight>

### Automate tiering with policies

A tiering policy automatically moves any chunks that only contain data
Expand Down Expand Up @@ -286,6 +287,7 @@ If you no longer want to use tiered storage for a particular hypertable, drop th

</Procedure>


[data-retention]: /use-timescale/:currentVersion:/data-retention/
[console]: https://console.cloud.timescale.com/dashboard/services
[hypertable]: /use-timescale/:currentVersion:/hypertables/
Expand Down
27 changes: 23 additions & 4 deletions use-timescale/data-tiering/index.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
---
title: Tiered Storage
title: Storage on Tiger
excerpt: Save on storage costs by tiering older data to a low-cost bottomless object storage tier. Tiger tiered storage makes sure you cut costs while having data available for analytical queries
products: [cloud]
price_plans: [scale, enterprise]
price_plans: [scale, enterprise, performance]
keywords: [tiered storage]
tags: [storage, data management]
---

# Tiered storage
# Storage

<Tabs label="Tiger on AWS and Azure" persistKey="tiger-platform-clouds">

<Tab title="Tiger on AWS" label="aws-cloud">

Tiered storage is a [hierarchical storage management architecture][hierarchical-storage] for
[real-time analytics][create-service] $SERVICE_SHORTs you create in [$CLOUD_LONG](https://console.cloud.timescale.com/).
Expand Down Expand Up @@ -43,11 +47,26 @@ In this section, you:
* [Learn about replicas and forks with tiered data][replicas-and-forks]: understand how tiered storage works
with forks and replicas of your $SERVICE_SHORT.

</Tab>

<Tab title="Tiger on Azure" label="azure-cloud">

$CLOUD_LONG stores your data in high-performance storage optimized for frequent querying. Based on [AWS EBS gp3][aws-gp3], the high-performance storage provides you with up to 16 TB and 16,000 IOPS. Its [$HYPERCORE row-columnar storage engine][hypercore], designed specifically for real-time analytics, enables you to compress your data by up to 98%, while improving performance.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the recent changes in gp3, which now supports 64000 IOPS and 64TB, we need @rahilsondhi to reconsider external communications about it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$CLOUD_LONGs [$HYPERCORE ...


Coupled with other optimizations, $CLOUD_LONG high-performance storage makes sure your data is always accessible and your queries run at lightning speed.

</Tab>

</Tabs>


[about-data-tiering]: /use-timescale/:currentVersion:/data-tiering/about-data-tiering/
[enabling-data-tiering]: /use-timescale/:currentVersion:/data-tiering/enabling-data-tiering/
[replicas-and-forks]: /use-timescale/:currentVersion:/data-tiering/tiered-data-replicas-forks/
[creating-data-tiering-policy]: /use-timescale/:currentVersion:/data-tiering/enabling-data-tiering/#automate-tiering-with-policies
[querying-tiered-data]: /use-timescale/:currentVersion:/data-tiering/querying-tiered-data/
[add-retention-policies]: /api/:currentVersion:/continuous-aggregates/add_policies/
[create-service]: /getting-started/:currentVersion:/services/
[hierarchical-storage]: https://en.wikipedia.org/wiki/Hierarchical_storage_management
[hierarchical-storage]: https://en.wikipedia.org/wiki/Hierarchical_storage_management
[hypercore]: /use-timescale/:currentVersion:/hypercore
[aws-gp3]: https://docs.aws.amazon.com/ebs/latest/userguide/general-purpose.html
6 changes: 6 additions & 0 deletions use-timescale/data-tiering/querying-tiered-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ keywords: [ tiered storage, tiering ]
tags: [ storage, data management ]
---

import NotSupportedAzure from "versionContent/_partials/_not-supported-for-azure.mdx";

# Querying tiered data

Once rarely used data is tiered and migrated to the object storage tier, it can still be queried
Expand All @@ -24,6 +26,8 @@ Your hypertable is spread across the tiers, so queries and `JOIN`s work and fetc
By default, tiered data is not accessed by queries. Querying tiered data may slow down query performance
as the data is not stored locally on the high-performance storage tier. See [Performance considerations](#performance-considerations).

<NotSupportedAzure />

## Enable querying tiered data for a single query

<Procedure>
Expand Down Expand Up @@ -186,3 +190,5 @@ Queries over tiered data are expected to be slower than over local data. However

* Text and non-native types (JSON, JSONB, GIS) filtering is slower when querying tiered data.



Loading