Worker Auto Enrollment

Auto enrollment lets you bootstrap external workers without copying API keys by hand. The server issues an API key the first time a worker connects with the shared secret, then the worker caches its credentials locally. Use this flow to scale fleets quickly, then disable it again once provisioning is complete.

Prerequisites

• A licensed deployment. Community Edition only supports the built-in worker and the enrolment endpoint returns 400 Bad Request.
• Administrative access to the control plane (JwtRoleAdmin). Issue an admin JWT with your LYFTDATA_JWT_PSK or reuse an existing admin token.
• A plan for distributing the auto-enrollment secret securely (VPN or trusted configuration channel). Treat it like a root password; anyone with the secret can register new workers until you rotate or disable it.

Configure the server

1. Generate a random shared secret:

   openssl rand -hex 32

2. Enable auto enrollment and store the secret:

   # ADMIN_JWT should contain an admin token; issue one with your LYFTDATA_JWT_PSK
   ADMIN_JWT=<paste-admin-jwt>
   curl -sS -X POST https://lyftdata.example.com/api/workers/auto-enrollment \
     -H "Authorization: Bearer $ADMIN_JWT" \
     -H "Content-Type: application/json" \
     -d '{
       "enabled": true,
       "enrollment_secret": "<paste-hex-secret>",
       "enrollment_api_key": null
     }'

   • enrollment_secret must be present when enabled is true.
   • enrollment_api_key is optional. Leave it null to let the server mint a fresh key on the first enrollment. That key is then reused for subsequent auto-enrolled workers until you rotate it.
3. Verify the configuration:

   curl -sS -H "Authorization: Bearer $ADMIN_JWT" \
     https://lyftdata.example.com/api/workers/auto-enrollment | jq

   The response mirrors the stored state (secret and API key are hidden).

Bootstrap a worker

1. Supply a unique worker name and the shared secret. The worker can run without a pre-assigned ID or API key:

   export LYFTDATA_URL=https://lyftdata.example.com
   export LYFTDATA_WORKER_NAME=ingest-us-east-01
   export LYFTDATA_AUTO_ENROLLMENT_KEY=<paste-hex-secret>
   
   lyftdata run worker --worker-jobs-dir /var/lib/lyftdata-worker

2. On startup the worker logs worker is using auto enrollment, computes an HMAC of the secret plus a random nonce, and calls POST /api/workers/enroll.
3. The server validates the HMAC, ensures the worker ID is unique (or generates one), issues an API key, and records the worker as auto_enrolled in the staging database.
4. The worker caches the response in its jobs directory:
   • LYFTDATA_WORKER_ID
   • .LYFTDATA_WORKER_API_KEY Future restarts reuse these files, so the auto-enrollment secret is no longer needed after the first success.
5. Confirm the worker joined the fleet:

   curl -sS -H "Authorization: Bearer $ADMIN_JWT" \
     https://lyftdata.example.com/api/workers | jq 'map({id: .worker, name: .name})'

Rotate or disable enrollment

• Disable once provisioning is finished to block surprise workers:

  curl -sS -X POST https://lyftdata.example.com/api/workers/auto-enrollment \
    -H "Authorization: Bearer $ADMIN_JWT" \
    -H "Content-Type: application/json" \
    -d '{"enabled": false}'

• Rotate the secret by posting a new enrollment_secret. Restart any workers that still rely on the old secret before it is disabled.
• Rotate the shared API key by supplying "enrollment_api_key": "<new-key>" in the POST body. Future enrollments return the new key.
• To force a specific worker to re-enroll, delete its LYFTDATA_WORKER_ID and .LYFTDATA_WORKER_API_KEY files and restart it with the current secret.

Troubleshooting

Symptom	Where to look	Likely cause
Worker exits immediately	Worker logs (journalctl -u lyftdata-worker)	Secret missing, worker name unset, or cached ID/API key mismatch
401 Unauthorized in worker logs	Server logs contain attempted auto enrollment, but secret was incorrect	Worker holds the wrong secret. Refresh LYFTDATA_AUTO_ENROLLMENT_KEY and restart
Community Edition only supports the built-in worker	Server log / API response	Instance is running Community Edition; external workers must use manual API keys
Repeated enrollments reuse the same API key	Expected. Server stores the generated key and hands it to subsequent workers until you rotate it
Worker never appears in /api/workers	Server log shows auto enrollment error, requested worker ID ... already exists	Worker tried to reuse an ID already registered; clear LYFTDATA_WORKER_ID or choose a different --worker-name

Best practices

• Use auto enrollment only on trusted networks and only for the time it takes to bring new workers online. Disable it afterwards.
• Store the secret in a secrets manager and inject it as an environment variable rather than baking it into images.
• Monitor /api/workers for unexpected entries and alert when workers appear with unfamiliar names.
• Combine enrollment with automation: Terraform or Ansible can set the shared secret, start new nodes, and immediately disable the feature again.
• Rotate the shared API key on a regular schedule so a leaked key cannot authenticate stale workers.

With these safeguards in place, auto enrollment speeds up provisioning without sacrificing traceability or control.