Time Series Collections

🎯 Why This Matters

The Problem: Time series data (IoT sensors, metrics, logs) generates massive volumes - often millions of documents per day. Regular MongoDB collections store each document separately, leading to:

The Solution: Time Series collections automatically bucket, compress, and optimize this data - giving you 90%+ storage savings and 10x faster queries with zero application changes.

🎯 Why This Matters

Buckets are the secret sauce. Instead of storing 1 million individual documents, MongoDB groups them into ~1,000 buckets. Each bucket holds documents with the same metadata within a time window. This means: fewer documents to scan, better compression, and indexes that are 1000x smaller. Understanding bucket structure helps you choose the right granularity and metadata fields.

📏 What Granularity Actually Controls

🔄 When Do Buckets Actually Close?

Under the hood, MongoDB stores time series data in a special system.buckets.<collection> collection. Each bucket is a single document with a specific structure:

{
  "_id": ObjectId("..."),           // Encodes bucket start time
  "control": {
    "version": 1,                     // Bucket format version
    "min": {
      "_id": ObjectId("..."),        // Min values for pruning
      "timestamp": ISODate("2024-01-01T10:00:00Z"),
      "temperature": 22.1
    },
    "max": {
      "timestamp": ISODate("2024-01-01T10:59:59Z"),
      "temperature": 28.7
    },
    "closed": false,               // Is bucket accepting writes?
    "count": 847                     // Number of measurements
  },
  "meta": {                            // Your metaField value
    "sensorId": "sensor_001",
    "location": "NYC"
  },
  "data": {                            // Columnar compressed data
    "timestamp": { "0": ISODate(...), "1": ISODate(...), ... },
    "temperature": { "0": 23.5, "1": 23.6, "2": 23.7, ... },
    "humidity": { "0": 45, "1": 46, "2": 44, ... }
  }
}

💰 Why This Matters

Compression directly impacts your costs. At scale, storing 1TB of raw sensor data might cost $100/month. With 90% compression, that drops to $10/month. Time series collections achieve this automatically through delta encoding - no application changes needed. Understanding how it works helps you design schemas that compress even better.

Time series data is highly compressible because consecutive values are often similar. MongoDB uses delta encoding to store only the differences between values:

⚡ Why This Matters

Memory management directly affects write performance. MongoDB keeps "open" buckets in RAM for fast inserts. If you have too many unique metadata combinations, you'll exhaust the bucket catalog and trigger constant disk I/O. Understanding the lifecycle helps you design metadata fields that don't explode your memory usage.

🧠 The Bucket Catalog

MongoDB maintains an in-memory Bucket Catalog that tracks all open buckets. This is crucial for high-performance writes - instead of searching disk for the right bucket, MongoDB keeps active buckets readily accessible in RAM.

🔄 Bucket State Machine

Every bucket transitions through distinct states. Understanding this lifecycle is key to optimizing time series performance.

What Triggers Bucket Closure?

When a bucket is closed, it flows through MongoDB's WiredTiger storage engine. Here's the complete journey from memory to disk:

Compression Pipeline Breakdown

// Create a time series collection
db.createCollection("sensorData", {
    timeseries: {
        timeField: "timestamp",        // Required: field containing timestamp
        metaField: "metadata",        // Optional: field for grouping
        granularity: "minutes"       // "seconds" | "minutes" | "hours"
    },
    expireAfterSeconds: 86400 * 30   // Optional: auto-delete after 30 days
});

// Insert a document
db.sensorData.insertOne({
    timestamp: new Date(),
    metadata: {
        sensorId: "sensor_001",
        location: "building_A"
    },
    temperature: 23.5,
    humidity: 45.2,
    pressure: 1013.25
});

Granularity Options

Write Path

Query Path

⚡ Why This Matters

Bucket pruning is why time series queries are fast. When you query "last 24 hours", MongoDB doesn't scan all your data. Each bucket has control.min and control.max fields that let MongoDB skip entire buckets that can't possibly contain matching documents. A year of data might have 8,760 buckets, but a "last hour" query only touches 1-2 buckets.

🔍 Step-by-Step: How a Read Query Executes

🎮 Interactive Query Analyzer

🔍 Why This Matters

Indexes on time series collections are 1000x smaller than on regular collections because they index buckets, not individual documents. A collection with 10 million documents might have only 10,000 buckets - so your index has 10,000 entries instead of 10 million. But they work differently: indexes are created on the internal system.buckets.* collection, and query planning uses bucket metadata for pruning.

Time series collections support secondary indexes, but they work differently than regular collections:

// Index on metadata subfields
db.sensors.createIndex({ "metadata.region": 1 })

// Compound with time (for range + filter)
db.sensors.createIndex({ 
  "metadata.sensorType": 1,
  "timestamp": -1 
})

// Index on measurement fields
db.sensors.createIndex({ "temperature": 1 })

🎯 Interactive Index Query Visualizer

Query Execution Flow

MongoDB provides powerful aggregation stages designed specifically for time series analysis:

{ $densify: {
  field: "timestamp",
  range: {
    step: 1,
    unit: "hour",
    bounds: "full"
  }
}}

{ $fill: {
  sortBy: { timestamp: 1 },
  output: {
    temperature: {
      method: "linear"
    }
  }
}}

{ $setWindowFields: {
  sortBy: { timestamp: 1 },
  output: {
    movingAvg: {
      $avg: "$temp",
      window: { range: [-1, 0], unit: "hour" }
    }
  }
}}

Time series collections have some restrictions to be aware of. Understanding these helps you design better schemas:

Estimate your storage savings by switching to time series collections: