Skip to content
LyftData Docs (Beta)

Telemetry

LyftData emits telemetry so you can understand what the server and workers are doing without SSH’ing into hosts. Telemetry includes job lifecycle events, worker heartbeats, logs and issues, and metrics suitable for dashboards and alerting.

What telemetry includes

  • Job lifecycle and health: staged/deployed/running outcomes, failures, retries, and warnings surfaced in the UI.
  • Worker heartbeats and system status: online/offline, basic host metrics, and runtime coordination signals.
  • Metrics: counters and gauges for server and worker behavior (for example throughput, error rates, backpressure).
  • Logs and issues: actionable warnings/errors you can use to triage failed runs or misconfiguration.

Where to view it

  • UI:
    • Dashboard for quick health and issues.
    • Jobs and Workers for run history and per-worker state.
    • Monitoring and Metrics Explorer for stored metrics.
  • Endpoints (advanced):
    • Worker telemetry surfaces under /api/workers (logs and metrics).
    • Stored metrics query surfaces under /api/metrics (used by the Metrics Explorer UI).

Access and security

Telemetry endpoints follow the same authentication and RBAC rules as the rest of the LyftData API.

  • Use a dedicated read-only token for dashboards and scrapers.
  • If you run Prometheus or log shippers from fixed IPs, the server supports IP allowlisting for selected read endpoints via --whitelist / LYFTDATA_API_WHITELIST. This bypass is limited and still requires the Authorization: WHITELIST header (see Security hardening and Configuration).

Outbound “phone-home” telemetry

LyftData can optionally send anonymized usage/health telemetry outbound when enabled. If you want to disable outbound telemetry entirely, start the server with --disable-telemetry or set LYFTDATA_DISABLE_TELEMETRY=true.

Operational tips

  • Start with Monitoring and Logs and issues to get a solid “day 2” operational baseline.
  • If live UI status appears stale, check that your environment allows long-lived HTTP connections (reverse proxies, VPNs, and corporate gateways can interrupt streaming updates).