Databricks DevOps Setup

Sets up workload identity federation between Azure Pipelines and Databricks — so your pipelines authenticate with no personal access token stored anywhere — from a panel inside VS Code.

Use it

Databricks: Set up pipeline federation in the Command Palette.

The panel walks six steps:

1. Sign in — detects the Databricks and Azure CLIs and signs in with either.
2. Workspace — listed from your account, so the ID cannot be mistyped.
3. Pipelines — one federation policy is created per pipeline.
4. Service principal — reuse one, or create a Databricks-managed one.
5. Create policies — the principal, the policies, and the workspace grant.
6. Service connection — the Azure DevOps side, optional.

Every step checks before it acts, so re-running after adding a pipeline creates only that pipeline's policy.

What it needs

• The Databricks CLI, signed in as an account admin — or the Azure CLI, signed in with az login. Either works; the panel says which it is using.
• Nothing else. No token is stored, created, or asked for.

The Azure DevOps step needs no personal access token either: az account get-access-token issues one for Azure DevOps from your existing az login. A PAT is accepted where your Azure sign-in is not a member of the organisation; it is used for that one request and never written down.

Why a panel and not a pipeline task

Creating a service principal and its federation policies needs Databricks account admin. A deployment pipeline should never hold that, and the setup hub in Azure DevOps runs in a sandboxed browser tab with nowhere safe to keep it — so this half of the setup belongs on your machine, which is where you are already signed in.

Pre-filled forms

Open a folder containing a setup.json — the one the Azure DevOps setup hub puts in its downloadable kit — and the form opens with your organisation, project, pipelines and application ID already in place.

Related

The Azure Pipelines side ships separately as Databricks DevOps for Azure Pipelines, which contributes the service connection type and the DatabricksBundle / DatabricksCost tasks.