Skip to content
Lyft Data Docs (Beta)

Worker Installation

This guide assumes you completed the installation prerequisites, staged the binary, and have the Linux server configured and running.

Set up a new Worker as follows:

  1. Create a Worker ID and API key on the Server.

  2. Prepare the service account and data directory.

  3. Create a systemd service unit file.

  4. Start the Worker.

Generate an API key

A Worker ID and API key are required for the Worker before it can connect to the Server. This can be done either via the web-based UI or the CLI.

Via web-based UI

Log in to the Server. Go to Workers in the top navigation, then select NEW WORKER.

Create a new Worker with a specified name and ID.

The Worker name is a human-readable designation (a label), while the Worker ID will be used in the Worker configuration.

Next, create an API key for the Worker. Go to Manage > Keys in the top navigation.

Then create a name for the new API key, select Add and copy the key value for later use.

Create systemd files

Create a systemd service unit file:

sudo vi /etc/systemd/system/lyftdata-worker.service

The file must contain the following:

Terminal window
[Unit]
Description=lyftdata Worker
After=network.target auditd.service


[Service]
EnvironmentFile=/etc/default/lyftdata-worker
User=lyftdata-worker
Group=lyftdata-worker
ExecStart=/usr/sbin/lyftdata run worker
Restart=on-failure
RestartSec=60


[Install]
WantedBy=multi-user.target

Create an environment file for the EnvironmentFile setting:

Terminal window
sudo vi /etc/default/lyftdata-worker

Here, the Worker is configured through either lyftdata run worker options or environment variables. In this case, we’ll be using the latter.

At a minimum, the Worker needs to know:

  • A unique Worker ID (LYFTDATA_WORKER_ID).

  • An API key to authenticate against a Server (LYFTDATA_WORKER_API_KEY).

  • The Server URL (LYFTDATA_URL).

  • A data directory to store Job definitions and other state data (LYFTDATA_JOBS_DIR).

Therefore, the file should contain the following:

LYFTDATA_WORKER_ID=worker1
LYFTDATA_WORKER_API_KEY=<WORKER-API-KEY>
LYFTDATA_JOBS_DIR=/var/lib/lyftdata-worker
LYFTDATA_URL=http://<server>:3000
LYFTDATA_LICENSE_EULA_ACCEPT=yes

Optional settings

Change the LYFTDATA_WORKER_API_KEY to match the key you previously created.

Change the LYFTDATA_URL to the Server address or hostname (confirm that your DNS is configured).

Once you have saved the service unit file, reload systemd:

Terminal window
sudo systemctl daemon-reload

To start the Worker at boot, enable the service with:

Terminal window
sudo systemctl enable lyftdata-worker

Finally, start the Worker:

Terminal window
sudo systemctl start lyftdata-worker

Verify that the Worker started successfully:

Terminal window
systemctl status lyftdata-worker

It’s a good idea to inspect the startup output, which might contain an error or warn:

Terminal window
journalctl -u lyftdata-worker

A Worker should now be running. It will register with the Server, using the specified API key.

The Server should indicate the Worker status on the Dashboard. At this point, the Worker is ready to run received Jobs from the Server.