RNA-Seq pipeline fails early

Ask for help
, ,
1

Hello all, I am attempting to run nf-core/rnaseq pipeline on GCP as a google batch job. Pipeline is running into an error quite early with nextflow.log indicating an issue possibly with the Trimgalore step or Google api authentication (see lines 608-611 of attached log file). Please help me understand how I can resolve this and get the pipeline to complete the job. Attaching nextflow.log file here.
nextflow_copy.log (226.1 KB)

2

Do any processes actually start? It looks like your jobs aren’t authenticated:

Sep-24 04:12:35.275 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
  task: name=NFCORE_RNASEQ:RNASEQ:FASTQ_QC_TRIM_FILTER_SETSTRANDEDNESS:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE (control_rep2); work-dir=gs://my-bucket-bckt/small_set/work/11/fc275887a707123091091a1ecb003f
  error [com.google.api.gax.rpc.UnauthenticatedException]: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Failed computing credential metadata
Sep-24 04:12:35.288 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'NFCORE_RNASEQ:RNASEQ:FASTQ_QC_TRIM_FILTER_SETSTRANDEDNESS:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE (control_rep2)'

Caused by:
  UNAUTHENTICATED: Failed computing credential metadata

com.google.api.gax.rpc.UnauthenticatedException: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Failed computing credential metadata
	at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:116)
	at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:41)
	at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:86)
	at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:66)
	at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
	at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:84)

Have you set the Google Cloud credentials correctly?

3

Thank you for pointing me to Google Cloud credentials! Yes, I believe I had set them up correctly using the instructions you have linked here. But I had forgotten to point to my creds.json file prior to this specific run (from which I got the errors I posted). Since I’m trying to run the pipeline from Google Cloud Shell, I believe I needed to complete the step of export GOOGLE_APPLICATION_CREDENTIALS=/path/to/creds.json each time I start the shell.

I’ve started the pipeline and it seems to be running.

4

I’m still unssuccessful in getting the pipeline to run. I was able to successfully run the nextflow-io/rnaseq-nf pipeline from my cloud shell with google batch settings in my nextflow.config file. But my run with nf-core/rnaseq stalls soon after it starts, does not generate any outputs for even the QC step and this is quite perplexing to me. It appears that Google cloud credentials shouldn’t be an issue since I’m able to run one of the pipelines. Please help me troubleshoot this.