Storage Configuration

Overview¶

Both InfluxDB and Loki support multiple storage backends for different use cases. This page covers storage configuration options and trade-offs.

Storage Backend Options¶

Local Filesystem¶

Description: Data stored on local disk attached to server

Pros: - Simple setup - Fast access - No external dependencies

Cons: - Limited scalability - No built-in replication - Capacity limited by local disk

Use case: Development, small deployments, low data volume

Configuration:

# InfluxDB
influxdb_storage_type: "local"
influxdb_data_path: "/var/lib/influxdb2"

# Loki
loki_storage_type: "filesystem"
loki_data_path: "/var/lib/loki"

NFS Mounts¶

Description: Data stored on network-attached storage via NFS

Pros: - Centralized storage - Easy to scale capacity - Shared access across hosts

Cons: - Network latency - NFS overhead - Single point of failure (without HA NFS)

Use case: Medium deployments with shared storage infrastructure

Configuration:

# Mount NFS first
- name: Mount NFS storage
  mount:
    path: /mnt/nfs/monitoring
    src: "storage.example.com:/exports/monitoring"
    fstype: nfs
    opts: "rw,sync"
    state: mounted

# Then configure applications
influxdb_data_path: "/mnt/nfs/monitoring/influxdb"
loki_data_path: "/mnt/nfs/monitoring/loki"

S3-Compatible Object Storage¶

Description: Data stored in S3-compatible object storage (AWS S3, MinIO, etc.)

Pros: - Virtually unlimited capacity - Cost-effective for large volumes - Built-in replication and durability - Separate compute from storage

Cons: - Slightly higher latency than local - Requires object storage infrastructure - Additional cost for object storage

Use case: Large deployments, long retention periods, cost optimization

InfluxDB with S3:

influxdb_storage_type: "s3"
influxdb_s3_endpoint: "storage.example.com:8010"
influxdb_s3_bucket: "influx11"
influxdb_s3_access_key: "{{ vault_s3_access }}"
influxdb_s3_secret_key: "{{ vault_s3_secret }}"
influxdb_s3_retention: "30d"

Loki with S3:

loki_storage_type: "s3"
loki_s3_endpoint: "storage.example.com:8010"
loki_s3_bucket: "loki11"
loki_s3_access_key: "{{ vault_s3_access }}"
loki_s3_secret_key: "{{ vault_s3_secret }}"

Storage Sizing¶

InfluxDB Storage¶

Factors affecting size: - Number of hosts - Metrics per host - Collection frequency - Retention period - Cardinality (unique tag combinations)

Estimation formula:

Storage = Hosts × Metrics × Frequency × Retention × Overhead

Example:
10 hosts × 100 metrics × 6 samples/min × 30 days × 10 bytes
= 10 × 100 × 8640 samples × 10 bytes
= 86.4 GB

Overhead factor: Add 50-100% for indexes and compaction

Loki Storage¶

Factors affecting size: - Number of hosts - Log volume per host (lines/sec) - Average log line size - Retention period - Compression ratio

Estimation formula:

Storage = Hosts × LogRate × LineSize × Retention / Compression

Example:
10 hosts × 10 lines/sec × 200 bytes × 30 days / 10
= 10 × 10 × 200 × 2592000 / 10
= 5.18 GB

Compression: Loki typically achieves 5-10x compression

Retention Policies¶

Time-Based Retention¶

InfluxDB:

influxdb_retention: "30d"    # Keep 30 days
influxdb_retention: "90d"    # Keep 90 days
influxdb_retention: "365d"   # Keep 1 year
influxdb_retention: "0"      # Keep forever (not recommended)

Loki:

loki_retention: "30d"
loki_retention: "90d"

Retention Best Practices¶

Short-term (7-30 days): Local storage, fast queries
Medium-term (30-90 days): S3 storage, cost-effective
Long-term (90+ days): S3 storage with longer retention
Archive: Export to cold storage for compliance

Retention Strategy Example¶

# Local storage for recent data (fast access)
influxdb_local:
  retention: "7d"
  path: "/var/lib/influxdb2"

# S3 storage for historical data (cost-effective)
influxdb_s3:
  retention: "365d"
  bucket: "influx-historical"

Performance Optimization¶

Local Storage Performance¶

Disk type: - SSD: Recommended for production - NVMe: Best performance for high write rates - HDD: Acceptable for low-volume or S3-backed deployments

Filesystem: - ext4: Good general-purpose choice - xfs: Better for large files - btrfs: Supports snapshots (useful for backups)

Mount options:

# /etc/fstab
/dev/sdb1  /var/lib/influxdb2  ext4  noatime,nodiratime  0  2

S3 Performance¶

Optimize chunk sizes:

# Loki
loki_max_chunk_age: "2h"        # Larger chunks = fewer S3 requests
loki_chunk_idle_period: "30m"

# InfluxDB
influxdb_cache_max_memory: "1g" # More caching = fewer S3 reads

Concurrent operations:

loki_max_concurrent_uploads: 10
influxdb_max_concurrent_compactions: 4

Storage Migration¶

Local to S3 Migration¶

InfluxDB: 1. Deploy new InfluxDB with S3 backend 2. Use influx backup to export data 3. Use influx restore to import to new instance 4. Update Telegraf clients to new endpoint 5. Decommission old instance

Loki: 1. Deploy new Loki with S3 backend 2. Update Alloy clients to send to both old and new 3. Wait for retention period on old instance 4. Remove old Loki endpoint from Alloy configs 5. Decommission old instance

Expanding Storage¶

Local storage:

# Add new disk
# Create filesystem
mkfs.ext4 /dev/sdc1

# Mount new disk
mount /dev/sdc1 /mnt/new_storage

# Stop services
systemctl stop influxdb loki

# Move data
rsync -av /var/lib/influxdb2/ /mnt/new_storage/influxdb/
rsync -av /var/lib/loki/ /mnt/new_storage/loki/

# Update mount points in systemd units
# Restart services
systemctl start influxdb loki

Backup Strategies¶

Local Storage Backup¶

# InfluxDB backup
systemctl stop influxdb
tar -czf influxdb-backup-$(date +%Y%m%d).tar.gz /var/lib/influxdb2/
systemctl start influxdb

# Loki backup
systemctl stop loki
tar -czf loki-backup-$(date +%Y%m%d).tar.gz /var/lib/loki/
systemctl start loki

S3 Storage Backup¶

When using S3 backend: - Data already stored in object storage (durable) - Enable S3 versioning for point-in-time recovery - Use S3 replication for disaster recovery - No application-level backup needed

Cost Optimization¶

Storage Tier Strategy¶

Hot tier (0-7 days): Local SSD for fast access
Warm tier (7-30 days): S3 standard storage
Cold tier (30+ days): S3 infrequent access or Glacier

Compression¶

Loki: Built-in compression (5-10x)

InfluxDB: Uses Snappy compression automatically

Retention Tuning¶

Balance retention with cost:

# Short retention for high-cardinality metrics
metrics_detailed:
  retention: "7d"

# Long retention for aggregate/summary metrics
metrics_summary:
  retention: "365d"

Monitoring Storage¶

Disk Usage¶

# Check disk usage
df -h /var/lib/influxdb2
df -h /var/lib/loki

# Check growth rate
du -sh /var/lib/influxdb2
du -sh /var/lib/loki

S3 Usage¶

Check bucket size via S3 API or web console.

Alerts¶

Set up alerts for: - Disk space < 20% free - Storage growth rate exceeding projections - S3 API errors - High storage latency

Reference Deployment¶

See Reference Deployments chapter for real-world S3 storage configuration: - monitor11.example.com - Both InfluxDB and Loki with S3 backend