Nf- core/sarek on aws batch error

Hello all I am a Genomics scientist new to Nextflow. I am trying to run the nf-core/sarek pipeline on an aws batch setup. I run the following command " nextflow run nf-core/sarek -profile awsbatch --awsqueue nf-var-queue --awsregion us-west-1 --input s3://my-bucket/input.csv --outdir s3://my-bucket/output --awscli /usr/bin/aws" and get the error below. I also looked at the AWS logs and the indicated that they could not find aws in a dependent container. See AWS log below also. Any ideas on how to solve?

[nf-core/sarek] Pipeline completed with errors
ERROR ~ Error executing process > 'NFCORE_SAREK:sarek:FASTP (Tumor1-1)'

Caused by:
  Task failed to start - CannotStartContainerError: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: exec: "/usr/local/env-ex

Command executed:

  [ ! -f Tumor1-1_1.fastq.gz ] && ln -sf ERR3239286_1.fastq.gz Tumor1-1_1.fastq.gz
  [ ! -f Tumor1-1_2.fastq.gz ] && ln -sf ERR3239286_2.fastq.gz Tumor1-1_2.fastq.gz

AWS LOG

@entity.Attributes.AWS.ServiceNameSource
ServerIamRole
@entity.Attributes.EC2.InstanceId
i-0d142dc187057eeb8
@entity.Attributes.PlatformType
AWS::EC2
@entity.KeyAttributes.Environment
ec2:default
@entity.KeyAttributes.Name
AWSBatchInstanceRole
@entity.KeyAttributes.Type
Service
@ingestionTime
1743716144388
@log
530856436170:/aws/batch/job
@logStream
nf-quay-io-biocontainers-gawk-5-1-0/default/3c43f62a100f47b4b0e245021c94324d
@message
bash: line 1: /usr/bin/aws: No such file or directory
@timestamp
1743716144275

Using a custom AMI solved the issue.

cf the slack conversation Slack

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.