Skip to content
Lyft Data Docs (Beta)

Troubleshooting

Use this page as a quick playbook when something goes sideways. Start with the symptom, work through the checks, and capture diagnostics before escalating.

How to use this page

  1. Find the symptom that best matches what you see.
  2. Run the quick checks in order; most issues are solved within the first two steps.
  3. Collect the diagnostics listed in the Capture before escalating section and attach them to support requests.

Connectivity issues

Cannot access the web UI

  • Symptom: Browser cannot reach the UI from another machine or the connection is refused.
  • Quick checks:
    • Start the server with a public bind address: lyftdata run server --bind-address 0.0.0.0:3000.
    • Or set LYFTDATA_BIND_ADDRESS=0.0.0.0:3000 in the environment.
    • Confirm firewalls or security groups allow inbound TCP 3000.
    • Default bind address is 127.0.0.1:3000 (local only).

”Login failed” or CLI cannot reach the server

  • Symptom: lyftdata login admin fails or CLI commands time out.
  • Quick checks:
    • Ensure the server is running and reachable at LYFTDATA_URL (default http://localhost:3000/).
    • If using a non-default port or address, export LYFTDATA_URL, for example http://server:3000.
    • For self-signed TLS in dev, pass the insecure TLS flag (--tls-insecure) when using the CLI.

Worker fails to register or stays offline

  • Symptom: Worker shows offline in the dashboard and does not receive jobs.
  • Quick checks:
    • Ensure LYFTDATA_URL points to the server API (include scheme and port).
    • Verify API key and worker ID (LYFTDATA_WORKER_API_KEY, LYFTDATA_WORKER_ID).
    • Confirm network reachability from the worker host to the server on TCP 3000.
  • Capture before escalating:
    • Worker logs (journalctl -u lyftdata-worker or console output).
    • Recent server logs showing registration attempts.

Licensing and onboarding

EULA prompt blocks automation

  • Set LYFTDATA_LICENSE_EULA_ACCEPT=yes in the environment for the first run to bypass the interactive prompt.
  • Apply the same variable when automating worker or CLI runs that would otherwise prompt.

Evaluation license expired or missing

  • Check the License page in the UI for status and expiry.
  • Reapply the license through the UI or restart the server with an updated LYFTDATA_LICENSE environment variable if you manage licenses non-interactively.

Installation and permissions

Server will not start because of staging directory permissions

  • Verify the service account owns the staging directory and it is writable.
  • Avoid running the server as root; create a dedicated user and fix ownership (chown -R lyftdata:lyftdata /var/lib/lyftdata).
  • If running on Windows, run the service as an account with Modify permissions on the staging directory.

Worker exits immediately on startup

  • Confirm the binary matches the host architecture.
  • Check that required environment variables are set (LYFTDATA_URL, LYFTDATA_WORKER_API_KEY).
  • Review stdout/stderr for panic messages; run with RUST_BACKTRACE=1 for extra context when filing a bug.

Jobs and pipeline execution

No data flowing through a job

  • Confirm the job is staged and deployed to an online worker.
  • Review Run Output and Logs for validation errors or connector failures.
  • Verify connector credentials and destination permissions.
  • Use the Issues pane to resolve schema or validation problems before redeploying.

UI actions do not take effect after staging or deploying

  • Check that the job version shows as Staged and the intended worker is connected.
  • For external workers, validate worker enrollment, API key, and LYFTDATA_URL.
  • Ensure the browser session is using the latest admin password; re-login if prompted.

Validation errors referencing missing fields

  • Ensure upstream jobs emit the fields referenced by Convert/Filter actions.
  • Use the Preview tab on each action to inspect incoming sample events.
  • When splitting pipelines, document channel schemas so downstream jobs expect the correct shape.

Performance and scaling

Backpressure or slow throughput

  • Monitor worker CPU and memory utilisation; scale horizontally by adding workers.
  • Use worker channels to fan out workloads and avoid single-job hotspots.
  • Apply rate limits on inputs where bursts cause congestion.
  • Inspect the worker backlog charts (UI) for long-running actions.

Disk usage high or logs purged unexpectedly

  • The server cleans up data when disk thresholds are exceeded.
  • Adjust retention: --db-retention-days or LYFTDATA_LOG_RETENTION_DAYS (default 30 days).
  • Adjust disk usage threshold: --disk-usage-max-percentage or LYFTDATA_DB_DISK_USE_MAX_PERCENT (default 80%).
  • Provision more disk for the staging directory volume.

Diagnostics and logs

  • Server staging directory: defaults to a per-user data path, configurable via --staging-dir or LYFTDATA_STAGING_DIR.
  • Worker job/data directory: LYFTDATA_JOBS_DIR or the worker cache directory if unset.
  • Collect server and worker logs, job run result JSON, and Issues list from the UI when escalating.
  • Review the Logs & Issues guide if warnings are not appearing where you expect them.
  • Record exact versions, host platform details, and recent configuration changes.

Additional resources