I was asked by a customer to create a dashboard in their custom solution for their different Azure Data Factory (ADF) pipelines, and the status of the pipeline runs with history.

ADF isn’t the tech I work with from day to day, but I have occasionally created some integrations utilizing the nice built-in features. I started off by looking into the documentation of an endpoint called queryPipelineRuns (Pipeline Runs – Query By Factory – REST API (Azure Data Factory) | Microsoft Learn), but quickly noticed that the security specifications said it only supported Flow: implicit.

Digging deeper into this I found some references to some examples having client credentials in their requests, which made me want to try to see if I could get the data without having an interactive browser flow, this led me to change the flow to client_credentials.

To be able to use the client_credentials flow i started of by creating an EntraID app registration, and added a secret for the integration to use. Next up I added RBAC permissions to the ADF instance for the app registration – I gave it the Azure Data Factory Reader permission so it had access to the list of pipeline jobs.

Next up I made a call to get the necessary token, notice the scope, and that the grant_type is set to client_credentials.

A small shout out to the team working on Bruno, which is a great lightweigth REST API client that runs completely offline so your information stays at your machine, you can get it here: GitHub – usebruno/bruno: Opensource IDE For Exploring and Testing Api’s (lightweight alternative to postman/insomnia)

curl --request POST \
  --url https://login.microsoftonline.com/<tenantId>/oauth2/v2.0/token \
  --header 'content-type: application/x-www-form-urlencoded' \
  --data 'client_id=<clientId>' \
  --data 'client_secret=<clientSecret>' \
  --data scope=https://management.azure.com/.default \
  --data grant_type=client_credentials

This gave me an access token I could use for the next call to try to fetch the status of the different ADF jobs. Before I could perform this next request I had to get some prerequisites from the Azure portal.

I performed a POST request to the queryPipelineRuns endpoint, and in return I got a nice json with a lot of different details of my pipeline runs. This is how my POST request looked:

curl --request POST \
  --url 'https://management.azure.com/subscriptions/<subscriptionId>/resourceGroups/<resourceGroup>/providers/Microsoft.DataFactory/factories/<factoryName>/queryPipelineRuns?api-version=2018-06-01' \
  --header 'authorization: Bearer eyJ0eXAiO....'

Based on the json response I could show information like who initiated the trigger, if the trigger ran successfully, when i started running and when it was finished to name a few.

{
  "value": [
    {
      "id": "<id>",
      "runId": "1b874496-2988-4-8a8bd393bcf2",
      "debugRunId": null,
      "runGroupId": "9b380116-a6a2-ba22-12925bbcf2ed",
      "pipelineName": "DataTransfer",
      "parameters": {},
      "invokedBy": {
        "id": "1972a3c95db441b34638506cba",
        "name": "Manual",
        "invokedByType": "Manual"
      },
      "runStart": "2025-01-15T10:45:43.7546197Z",
      "runEnd": "2025-01-15T10:45:44.9426601Z",
      "durationInMs": 1188,
      "status": "Succeeded",
      "message": "",
      "pipelineReturnValue": {},
      "lastUpdated": "2025-01-15T10:45:44.9427677Z",
      "annotations": [],
      "runDimension": {},
      "isLatest": true
    },
    {
      "id": "<id>",
      "runId": "c58d0974-deb3b73-b7a8661600cc",
      "debugRunId": null,
      "runGroupId": "3713e811-1-9272-1f43b6575c06",
      "pipelineName": "Transform pipeline",
      "parameters": {},
      "invokedBy": {
        "id": "4477dd67adb24eca9464be7e45e",
        "name": "Manual",
        "invokedByType": "Manual"
      },
      "runStart": "2025-01-15T10:45:55.1259805Z",
      "runEnd": "2025-01-15T10:45:56.6526949Z",
      "durationInMs": 1526,
      "status": "Failed",
      "message": "Operation on target Transfor failed failed: Transform failed",
      "pipelineReturnValue": {},
      "lastUpdated": "2025-01-15T10:45:56.6528279Z",
      "annotations": [],
      "runDimension": {},
      "isLatest": true
    }
  ]
}

Hope it can help anyone who stopped looking into it due to the implicit flow stated in the documentation.