-4

I am working with Apache IoTDB 2.0.6 and trying to compute a rolling moving average over time-series data.

My requirement:

Calculate aggregates over a sliding window that always contains a fixed number of valid points (e.g., 5 points), even when some timestamps are missing due to holidays or device offline periods.

I have tried two built-in window functions but neither satisfies my needs:

  1. HOP time-based sliding window

It slides by time interval correctly, but when data is missing on holidays, some windows end up with fewer points than expected (e.g., only 4 points instead of 5), which makes the aggregation result unstable.

  1. CAPACITY count-based window

It ensures exactly N points per window, but it is not a sliding window --- it only groups data into non-overlapping batches. There is no SLIDE parameter to support overlapping windows.

After checking the official documentation, I found no built-in function for a count-based sliding window (fixed number of points + time-based slide).

Minimal Reproducible Example

Table Structure & Sample Data (with holiday gap)

  CREATE TABLE sensor_metrics (
  device_id STRING TAG,
  temperature FLOAT FIELD
  );
  
  +-----------------------------+----------+-----------+
  | time| device_id|temperature|
  +-----------------------------+----------+-----------+
  |2026-01-13T00:00:00.000+08:00| sensor_01| 25.1|
  |2026-01-14T00:00:00.000+08:00| sensor_01| 25.3|
  |2026-01-15T00:00:00.000+08:00| sensor_01| 24.9|
  |2026-01-16T00:00:00.000+08:00| sensor_01| 25.5|
  |2026-01-17T00:00:00.000+08:00| sensor_01| `null`|
  |2026-01-18T00:00:00.000+08:00| sensor_01| 26.0|
  |2026-01-19T00:00:00.000+08:00| sensor_01| 25.8|
  |2026-01-20T00:00:00.000+08:00| sensor_01| 25.2|
  +-----------------------------+----------+-----------+

Attempt 1: HOP TIME WINDOW

  SELECT
  window_start,
  window_end,
  device_id,
  AVG(temperature) AS avg_temp
  FROM HOP(
  DATA => (
  SELECT
  time,
  device_id,
  temperature
  FROM sensor_metrics
  WHERE time BETWEEN '2026-01-01' AND '2026-01-31'
  AND device_id = 'sensor_01'
  AND temperature IS NOT `NULL`
  ),
  TIMECOL => 'time',
  SLIDE => 1 day,
  SIZE => 5 days,
  ORIGIN => '2026-01-01'
  )
  `GROUP BY` window_start, window_end, device_id
  ORDER BY window_end DESC;

Result

Plain Text
  +-----------------------------+-----------------------------+-----------+------------------+
  | window_start| window_end| device_id| avg_temp|
  +-----------------------------+-----------------------------+-----------+------------------+
  |2026-01-16T00:00:00.000+08:00|2026-01-21T00:00:00.000+08:00| sensor_01| 25.50|
  |2026-01-15T00:00:00.000+08:00|2026-01-20T00:00:00.000+08:00| sensor_01| 25.36|
  |2026-01-14T00:00:00.000+08:00|2026-01-19T00:00:00.000+08:00| sensor_01| 25.44|
  +-----------------------------+-----------------------------+-----------+------------------+

Problem: Some windows contain only 4 valid points because of the holiday gap, so the average is unreliable.

Attempt 2: CAPACITY COUNT WINDOW

  SELECT
  window_index,
  device_id,
  AVG(temperature) AS avg_temp
  FROM CAPACITY(
  DATA => (
  SELECT
  time,
  device_id,
  temperature
  FROM sensor_metrics
  WHERE time BETWEEN '2026-01-01' AND '2026-01-31'
  AND device_id = 'sensor_01'
  AND temperature IS NOT `NULL`
  ),
  PARTITION BY device_id ORDER BY time DESC,
  SIZE => 5
  )
  `GROUP BY` window_index, device_id;

Result

Plain Text
  +-----------------------------+-----------+------------------+
  | window_index| device_id| avg_temp|
  +-----------------------------+-----------+------------------+
  | 0| sensor_01| 25.32|
  | 1| sensor_01| 25.50|
  +-----------------------------+-----------+------------------+

Problem: It groups data into non-overlapping batches and does not slide, so I cannot get a real-time rolling view.

My Question:

How can I implement a sliding window that always keeps a fixed number of recent valid points (ignoring null/missing data) in Apache IoTDB 2.0.6? Is there any syntax, function, or workaround to achieve this?

New contributor
Lefei Zhang is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.

0

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.