What IP addresses does Sequera Tower Require to be allowed?

Ask for help
, ,
1

I am using Azure Batch with Seqera Tower. I have configured the network for Azure Batch. I have allowed all 5 ip’s described in https://cloud.seqera.io/ and get the error below:

{  "cloud.seqera.io": [
    "18.135.7.45/32",
    "18.169.21.18/32",
    "18.171.4.252/32"
  ],
  "licenses.seqera.io": [
    "35.176.121.51/32",
    "35.178.254.247/32"
  ] }

If I change Batch to public access, the workflow runs using Tower. Is Seqera tower using other IP addresses ?

Sep-25 05:07:47.678 [main] DEBUG nextflow.script.ScriptRunner - >
 Awaiting termination  Sep-25 05:07:47.678 [main] DEBUG
 nextflow.Session - Session await Sep-25 05:07:47.946 [Actor Thread 2]
 DEBUG nextflow.util.CacheHelper - Hash asset file sha-256:
 /.nextflow/assets/CDCgov/nf-core-pass/bin/nf.create_entry.py Sep-25
 05:07:48.023 [Task submitter] DEBUG n.c.azure.batch.AzBatchTaskHandler
 - [AZURE BATCH] Submitting task NFCORE_PASS:PASS:create_entry (SRR9686066) -
 work-dir=az://scratch/scratch/1BeU09LmRT8FLH/52/97c023dcec06233c26f512e774de58 Sep-25 05:07:48.105 [Task submitter] DEBUG
 n.cloud.azure.batch.AzBatchService - [AZURE BATCH] Checking VM pool
 id=standard_d4_v3; size=AzVmType(name:Standard_D4_v3,
 maxDataDiskCount:8, memory:16 GB, numberOfCores:4, osDiskSize:1023 GB,
 resourceDiskSize:100 GB) Sep-25 05:07:48.111 [Task submitter] DEBUG
 n.cloud.azure.batch.AzBatchService - [AZURE BATCH] Executor
 options=endpoint=https://nextflowc51d56ffeastus.eastus.batch.azure.com;
 account-name=nextflowc51d56ffeastus; account-key=(null) Sep-25
 05:07:48.111 [Task submitter] DEBUG n.cloud.azure.batch.AzBatchService
 - [AZURE BATCH] Creating Azure Batch client using Service Principal credentials Sep-25 05:07:48.387 [pool-5-thread-1] INFO 
 c.m.a.adal4j.AuthenticationAuthority - [Correlation ID:
 fe14708e-7e91-4669-8898-978551f2ef66] Instance discovery was
 successful Sep-25 05:07:48.752 [Task submitter] DEBUG
 nextflow.processor.TaskProcessor - Handling unexpected condition for  
 task: name=NFCORE_PASS:PASS:create_entry (SRR9686066);
 work-dir=az://scratch/scratch/1BeU09LmRT8FLH/52/97c023dcec06233c26f512e774de58   error [com.microsoft.azure.batch.protocol.models.BatchErrorException]:
 Status code 403, {  
 "odata.metadata":"https://nextflowc51d56ffeastus.eastus.batch.azure.com/$metadata#Microsoft.Azure.Batch.Protocol.Entities.Container.errors/@Element","code":"AuthorizationFailure","message":{
     "lang":"en-US","value":"This request is not authorized to perform this
 operation.\nRequestId:1618957d-48a3-474e-a5f7-14e157c6a813\nTime:2024-09-25T05:07:48.6957310Z"
 } } Sep-25 05:07:48.760 [Task submitter] ERROR
 nextflow.processor.TaskProcessor - Error executing process >
 'NFCORE_PASS:PASS:create_entry (SRR9686066)'
 
 Caused by:   Status code 403, {  
 "odata.metadata":"https://nextflowc51d56ffeastus.eastus.batch.azure.com/$metadata#Microsoft.Azure.Batch.Protocol.Entities.Container.errors/@Element","code":"AuthorizationFailure","message":{
     "lang":"en-US","value":"This request is not authorized to perform this
 operation.\nRequestId:1618957d-48a3-474e-a5f7-14e157c6a813\nTime:2024-09-25T05:07:48.6957310Z"
 } }
 
 com.microsoft.azure.batch.protocol.models.BatchErrorException: Status
 code 403, {  
 "odata.metadata":"https://nextflowc51d56ffeastus.eastus.batch.azure.com/$metadata#Microsoft.Azure.Batch.Protocol.Entities.Container.errors/@Element","code":"AuthorizationFailure","message":{
     "lang":"en-US","value":"This request is not authorized to perform this
 operation.\nRequestId:1618957d-48a3-474e-a5f7-14e157c6a813\nTime:2024-09-25T05:07:48.6957310Z"
 } }    at
 java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
 Method)    at
 java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
    at
 java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at
 java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
    at
 java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
    at
 com.microsoft.rest.ServiceResponseBuilder.build(ServiceResponseBuilder.java:122)
ns using Tower.**
2

Is Seqera tower using other IP addresses ?

These IP addresses are correct. You can always check the most up to date addresses here.

As for why you’re still running into this error, I’m not sure.

3

It looks like your Service Principal doesn’t have the required permissions. It should have Storage Blob Data Reader, Storage Blob Data Contributor and Batch Contributor:

Status code 403, {  
 "odata.metadata":"https://nextflowc51d56ffeastus.eastus.batch.azure.com/$metadata#Microsoft.Azure.Batch.Protocol.Entities.Container.errors/@Element","code":"AuthorizationFailure","message":{
     "lang":"en-US","value":"This request is not authorized to perform this
 operation.\nRequestId:1618957d-48a3-474e-a5f7-14e157c6a813\nTime:2024-09-25T05:07:48.6957310Z"

When running on Seqera Platform, you are likely to be using primary access keys which have no role attached so essentially can do anything in a storage or batch account so this doesn’t matter.

4

I am using https://cloud.seqera.io/. I am using the Azure credentials: batch account name/key + storage account name+key.

If I configure batch for public access the pipeline runs.

If I configure batch for my IP and run Nextflow from my desktop, the pipeline runs.

If I configure batch for private access + the 5 known ips associated with Tower, I get this error.

Therefore, this looks like some kind of firewall issue. However, Batch’s frontdoor is not attached to any firewall or network.

Are there other IPs associated with Tower ?

5

The logs clearly say you are using a Service Principal and the failure is while authenticating using that Service Principal, so something isn’t adding up here:

 - [AZURE BATCH] Creating Azure Batch client using Service Principal credentials Sep-25 05:07:48.387 [pool-5-thread-1] INFO 
 c.m.a.adal4j.AuthenticationAuthority - [Correlation ID:

Furthermore, the error is when accessing an Azure Batch resource:

 Caused by:   Status code 403, {  
 "odata.metadata":"https://nextflowc51d56ffeastus.eastus.batch.azure.com/$metadata#Microsoft.Azure.Batch.Protocol.Entities.Container.errors/@Element","code":"AuthorizationFailure","message":{
     "lang":"en-US","value":"This request is not authorized to perform this
 operation.\nRequestId:1618957d-48a3-474e-a5f7-14e157c6a813\nTime:2024-09-25T05:07:48.6957310Z"
 } }

Seqera Platform will launch a Nextflow task in Azure Batch which will then communicate with Azure Storage and the Batch service. Is the Azure Batch node capable of talking to these services or is the networking preventing these?

IP wise, the IPs listed above are the only ones the cloud hosted Seqera Platform uses.