Time Series Compression
With PowerTSM it is possible to store time series data compressed in the database to save storage space and speed up read and save operations.
In PowerTSM terminology, time series compression is a data storage technique in which individual time series values are combined into blocks of a specific size. The size of the blocks depends on the interval and the compression multiplier of the time series.
By default, a compression method called “Base64 Block Compression” is available, where a block of time series data is stored in the database as a Base64 string. The compression methods are extensible and thus custom compression methods can be implemented.
Compression Multiplier
Section titled “Compression Multiplier”The compression multiplier can be set on the time series and defines the block size of the compressed data (interval of the time series multiplied by the compression multiplier).
Example: If an hourly time series (Interval Type: Hour, Multiplier: 1) should be compressed into daily blocks, the compression multiplier must be set to 24.
The compression multiplier is an optional field and has the following defaults:
| Interval Type | Multiplier | Default Compression Multiplier |
|---|---|---|
| Second | 1 | 60 |
| Minute | 1 | 60 |
| Minute | 5 | 288 |
| Minute | 10 | 144 |
| Minute | 15 | 96 |
| Minute | 30 | 48 |
| Hour | 1 | 24 |
| Day | 1 | 30 |
| Month | 1 | 12 |
| else: | 100 |
How to Enable Compression
Section titled “How to Enable Compression”To enable compression for specific time series, the field CompressionMode has to be set:
var tsm = ITsm.Create();
var timeSeries = tsm.TimeSeries.Create( "My compressed time series", tsm.Intervals.Create(IntervalType.Hour()), true, FunctionType.StepRight(), compressionMode: CompressionMode.Base64BlockCompression());Reading and writing time series data can be done in the same way as for an uncompressed time series.
Why Use Compression
Section titled “Why Use Compression”Compression if properly utilized provides the following advantages:
- Reduced database storage usage. Example: a quarter hourly time series can potentially save 80% of the storage used by a non-compressed format.
- Increased data read speed. Example: in case of a quarter hourly time series data for one year the data rate is up to 20% higher compared to non-compressed format.
- Massively increased data write speed. Example: in case of a quarter hourly time series data for one year the data rate is up to 80% higher compared to non-compressed format.
Compression Use Cases
Section titled “Compression Use Cases”Performance gains are best when the time series values are accessed in similar (or the same) time blocks as those used in the compression. For example, in the case of quarter-hourly or hourly time series, reading whole days, months or years will give the best results.
In cases where only partitions of a compressed block are accessed (e.g. a quarter hour of a quarter hourly time series), compression will not result in a performance gain. On the contrary, an uncompressed time series will perform slightly better due to a small fraction of the overhead cost of compressing/decompressing a full block.
Recommendation
Section titled “Recommendation”Compression provides a performance gain if at least 75% of one or more blocks can be filled / read in one request. If this is not the case (e.g. when values are written one at a time in a given time series), compression costs will be slightly higher (around 10% of a very short processing time) compared to non-compressed time series.
Compressed time series are recommended in the following cases:
- When storage space utilization is important
- To boost save operations with volumes greater than 75% of a compression block
- To speed up read operations with volumes greater than 75% of a compression block