Workers
Workers execute jobs. The server stores job definitions, schedules runs, and deploys staged jobs to workers.
Built-in vs external workers
Section titled “Built-in vs external workers”- Built-in worker: Runs alongside the server and is the default for evaluation. If you followed the Evaluation Quickstart, you already have a built-in worker and do not need to install anything else to run jobs end-to-end.
- External workers: Run on additional hosts so you can scale out, isolate workloads, or place execution closer to sources/destinations. External worker support depends on your license; see Licensing and Community Edition.
What a worker needs to connect
Section titled “What a worker needs to connect”External workers use four core values for auto-enrollment:
LYFTDATA_WORKER_NAME: the worker’s human-readable name.LYFTDATA_AUTO_ENROLLMENT_KEY: the shared secret configured in Settings → Security.LYFTDATA_URL: the base URL of the server (how the worker reaches it).LYFTDATA_JOBS_DIR: the worker’s local data directory, which stores the cached identity and issued credential.
Follow Worker Auto Enrollment to enable the setting, then install the worker service on the target host. A key created in Settings → API Keys cannot enroll a Worker. Keep LYFTDATA_JOBS_DIR when reinstalling or moving the Worker: it holds the cached Worker ID and the issued credential. The enrollment secret cannot re-key a name that has already enrolled: if the directory is lost, that name cannot enroll again, and deleting the Worker in Workers does not free the name. Set LYFTDATA_WORKER_NAME to a name that has never enrolled on this Server, leave LYFTDATA_WORKER_ID and LYFTDATA_WORKER_API_KEY unset, put the enrollment secret back on the Worker, and make sure auto-enrollment is enabled in Settings → Security. The Worker joins as a new Worker with a new Worker ID. A static Worker group stores Worker IDs, and deleting a Worker does not change that list. On Workers, an administrator opens the new Worker, chooses Groups, and marks it a member of the static group, then opens the old Worker and clears that group. Plan and apply again. An administrator may then delete the old Worker in Workers.
Install an external worker
Section titled “Install an external worker”Choose the guide that matches the worker host OS:
- Linux: Worker installation
- macOS: Worker installation
- Windows: Worker installation
After the service starts, confirm the Worker appears as Connected in Workers and that its first-start log contains worker is using auto enrollment. If it does not, check the server URL, auto-enrollment secret, and firewall rules. See Networking & TLS and Troubleshooting.
After the worker is online
Section titled “After the worker is online”- Deploy a staged job to the worker to confirm end-to-end execution.
- Enable metrics collection for new workers (disabled by default) via Next steps.
- If you are scaling out, review System Requirements and the Scaling playbook.
Online means the server can see the worker heartbeat. It does not guarantee that every deployment or Provider Pack target job is serviceable. If a specific runtime reports degraded readiness, inspect the runtime-requirements report and target job status before restarting the worker. The common repairs are worker auth trust reset, worker settings or worker database ACL updates, and deployment runtime-requirements repair proposals. Follow Worker Recovery and Serviceability for the complete diagnostic and recovery sequence.