How does aws.batch.maxSpotAttempts work?

Ask for help
,
1

Following up on Revant’s questions about spot instances & Nextflow, I have a few questions about how aws.batch.maxSpotAttempts works.

If I

  1. set aws.batch.maxSpotAttempts to 0,
  2. launch a Nextflow pipeline run with AWS Batch, and then
  3. manually initiate a Spot Instance interruption (AWS documentation link) while a Nextflow task is running on a Spot Instance worker,

what should I expect to see happen? Should the Nextflow task fail and require resubmission? Do those sorts of manual spot instance interruptions fall under “Spot Attempts” for Nextflow and therefore trigger use of aws.batch.maxSpotAttempts? What information would be captured in the Nextflow log files?

Thanks in advance! - Deepank

2

If that’s all you did, the task will fail and the pipeline will be interrupted due to a task failure.

Depending on the task, you should set its errorStrategy to ignore and then the pipeline will not be stopped upon failure. You could also set a retry policy but it wouldn’t matter here as you set aws.batch.maxSpotAttempts to 0.