A minimal Nextflow configuration for Azure Batch

Ask for help
1

The documentation states that:

A minimal Nextflow configuration for Azure Batch looks like the following snippet:

process {
    executor = 'azurebatch'
}

azure {
    storage {
        accountName = "<YOUR STORAGE ACCOUNT NAME>"
        accountKey = "<YOUR STORAGE ACCOUNT KEY>"
    }
    batch {
        location = '<YOUR LOCATION>'
        accountName = '<YOUR BATCH ACCOUNT NAME>'
        accountKey = '<YOUR BATCH ACCOUNT KEY>'
        autoPoolMode = true
    }
}

When running with valid values I get:

ERROR ~ Invalid Service Principal configuration - Make sure servicePrincipalId and servicePrincipalClient are set in nextflow.config or configured via environment variables

2

Which Nextflow version? Could you share a .nextflow.log file please?

3

I managed to figure out the problem by looking at the Nextflow source code.
I had an environment variable AZURE_TENANT_ID set even though it was not used in nextflow.config.

After I removed that variable the connection was successful.

2 Likes
4

Nextflow will find the following environment variables and try to use them:

  • AZURE_CLIENT_ID
  • AZURE_CLIENT_SECRET
  • AZURE_TENANT_ID
  • AZURE_STORAGE_ACCOUNT_KEY
  • AZURE_STORAGE_ACCOUNT_NAME
  • AZURE_STORAGE_ACCOUNT_SAS
  • AZURE_REGISTRY_USER_NAME
  • AZURE_REGISTRY_PASSWORD

We need to clarify this in the docs and improve the hierarchy to make sure it takes explicitly set variables first. Thanks for pointing this out @ebragge

1 Like
5

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.