I agree to the use of cookies by the website owner in accordance with the Cookie Policy at tantorlabs.ru and tantorlabs.com and the Personal Data Processing and Protection Policy, including the transfer of cookie data to third-party analytics and advertising services as specified in the Cookie Policy at tantorlabs.ru.
Get in touch
Get in touch If you have any question, just let us know!
High-performance time series aggregation for PostgreSQL
Tantor PipelineDB
An open-source extension for Tantor Postgres DBMS or PostgreSQL for continuous execution of SQL queries on data streams with incremental result saving into tables
Download
Download
Documentation
Stores only the output of continuous queries
A standard PostgreSQL extension providing ETL capabilities
Real-time data processing using only SQL
High-performance time series data aggregation
Documentation
Streaming data is aggregated, filtered, and combined into summary data in real time using continuous SQL queries. The results are stored in Tantor PipelineDB.
Continuous aggregation
For streaming data analytics, context is often essential. The integrated relational storage engine of Tantor PipelineDB enables real-time correlation by joining streaming data with historical data.
Join streams on tables
You can run continuous queries with specified time intervals—such as second, minute, day, or 30 days. These queries are stored in Tantor PipelineDB, and raw data outside the defined intervals can be omitted.
Sliding windows
In real-time analytics, compromises in accuracy are often acceptable for the sake of speed. Tantor PipelineDB supports data structures and algorithms for approximate computations on streams: Bloom filters, count-min sketch, Filtered-Space-Saving top-k, HyperLogLog, and t-digest.
Probabilistic data structures
Tantor PipelineDB enables real-time data processing using only SQL. The complexity of computations on streaming data is handled entirely by the continuous query scheduler and execution engine.
Tantor PipelineDB eliminates the ETL layer. Query data is delivered directly to the database and continuously processed according to defined SQL queries. The database size remains unchanged as new data arrives.
Tantor PipelineDB is a standard extension for Tantor Postgres and PostgreSQL databases. It leverages the full capabilities of these DBMSs for high-performance streaming workloads.
No application code
No ETL layer
Works upon Tantor and PostgreSQL
How does Tantor PipelineDB work?
Examples of usage
Tantor PipelineDB is suitable for any applications that collect real-time analytics, provided that the queries are known in advance and can be implemented in SQL.
Example 1
Example 2
Calculate how many unique users clicked a given URL per day using a fixed amount of memory.
Example 3
Determine how many ad impressions occurred in the last five minutes.
Example 4
Join conversions to any prior clicks that may have caused them.
Example 5
Find the 90th, 95th, and 99th percentiles of server request latency.
Identify how much traffic is consumed by each of the top 10 IP addresses sending requests to the server.
A continuous view selects from a combination of streams and tables as its inputs and is incrementally updated in real-time as new data is written.
Tuples inserted into a stream are stored in a shared-memory ring buffer with parallel access.
Worker processes read microbatches from the stream buffer, aggregate them into intermediate results, and send them to combiner processes for processing.
Receives intermediate results from worker processes and continuously merges them with tuples stored on disk.
Glossary of streamed data processing
Continuous views
Stream buffer
Worker
Combiner
Get in touch
If you have any question, just let us know!