Playbook · Log analytics

Ingest Nginx, JSON, syslog, or OTLP at line rate. Query hot logs in milliseconds. Cold-drop to cheap storage with ILM-free retention. This is the workload XERJ was originally built for.

Schema

$ curl -sX PUT http://localhost:8080/v1/indices/logs \
    -H 'Content-Type: application/json' \
    -d '{
      "fields": {
        "@timestamp":  "date",
        "service":     "keyword",
        "level":       "keyword",
        "host":        "keyword",
        "trace_id":    "keyword",
        "span_id":     "keyword",
        "status":      "integer",
        "latency_ms":  "integer",
        "bytes_out":   "integer",
        "client_ip":   "ip",
        "message":     "text"
      }
    }'

Ingest options

• Raw NGINX/Apache → POST /v1/indices/logs/logs auto-detects the format.
• Structured JSON → POST /v1/indices/logs/turbo-ingest NDJSON body.
• Syslog → POST /v1/indices/logs/syslog.
• OTLP → POST /v1/indices/logs/otlp.

Tail recent errors

{
  "query": {
    "bool": {
      "filter": [
        { "term":  { "level": "error" } },
        { "range": { "@timestamp": { "gte": "now-15m" } } }
      ]
    }
  },
  "size": 100,
  "sort": [ { "@timestamp": "desc" } ]
}

Top slow endpoints · percentiles

{
  "query": { "term": { "service": "api-gateway" } },
  "aggs": {
    "by_route": {
      "terms": { "field": "url_template", "size": 20 },
      "aggs": {
        "p": { "percentiles": { "field": "latency_ms", "percents": [50, 95, 99] } }
      }
    }
  },
  "size": 0
}

Retention

Per-index, in the config. No ILM policy DSL required.

[logs]
retention_days = 30
time_partition = "1h"

Dashboard

Open the playground and pick LOGS · OVERVIEW — nine panels with click-to-filter TopN, 1 ms date_histogram, heatmap, and drill-through to the SYSTEM · OVERVIEW dashboard.

Source · engine/crates/logs/src/lib.rs