Run the latest release of a pipeline automatically

Ask for help
1

Hi everyone!

I’d like to run the latest release of a pipeline automatically.

Currently, I use a combination of nextflow pull and parsing nextflow info for nextflow run -r.

Is there something more straightforward?

2 
nextflow run -latest <pipeline>
3

Thanks Mahesh!

Does this point to the latest release tag or the latest commit in the default branch?

4

The latest commit of a branch given with -r or the default branch if not specified.

Note, there is a page on CLI options in the documentation, although in this particular case it doesn’t answer your last question.
https://www.nextflow.io/docs/latest/reference/cli.html#run

5

Hmm, I think, this doesn’t solve my problem :thinking:

The latest commit on the default branch is not necessary the latest release tag. For example, I’d like to run -r 1.2.3 and -r 1.3.0 the next time (if there was a new release).

Yeah, I’m aware of the CLI documentation, but it’s lacking some details/explanations at some points.

6

Ah, then I misunderstood completely. I think then you need to use the Github API to query the latest tag, and use it that way.

E.g. ( replacing the repo path )

tag="$(curl -s https://api.github.com/repos/box/box-java-sdk/releases/latest | jq -r '.tag_name')"

from Getting latest tag on git repository · GitHub

You could then write a launch script to do this automatically if you want ( i.e. run this and then pass the env variable to -r

7

Just for completion an example bash script to run the latest version of nf-core/rnaseq with some extra features:
run_nextflow.sh:

#! /usr/bin/env bash
set -euo pipefail

# Select workflow (default: nf-core/rnaseq)
WORKFLOW="${WORKFLOW:-nf-core/rnaseq}"
# Select profile (default: standard)
PROFILE="${PROFILE:-standard}"
# Set working directory
WORKDIR="${WORKDIR:-work}"

# Check for params file as arg
WF_PARAMS=''
if test -f "${1:-''}"; then
    WF_PARAMS="-params-file $1"
fi

if test -f "$WORKFLOW"; then
    # File is a local script
    nextflow run "$WORKFLOW" \
        -work-dir "$WORKDIR" \
        -profile "$PROFILE" \
        -resume \
        $WF_PARAMS
else
    # Run latest release
    TAG=$( curl -s "https://api.github.com/repos/$WORKFLOW/releases/latest" | jq -r '.tag_name' )
    # Workflow version or branch to use (default: latest tag)
    BRANCH="${BRANCH:-$TAG}"

    # Workflow hosted on remote
    nextflow run "$WORKFLOW" \
        -work-dir "$WORKDIR" \
        -profile "$PROFILE" \
        -r "$BRANCH" \
        -latest \
        -resume \
        $WF_PARAMS
fi

# Clean up Nextflow cache to remove unused files
nextflow clean -f -before "$( nextflow log -q | tail -n 1 )"
# Clean up empty work directories
find "$WORKDIR" -type d -empty -delete

And can be called with:

PROFILE=test,docker bash run_nextflow.sh params.yml
1 Like
8

Nice, thanks for sharing!

I guess, I’ll switch to curl instead of parsing nextflow info output and nextflow run -latest to get the remote changes :+1:

9

(+1 to post this in the Tips & Tricks section :slight_smile: )

10

I didn’t know about the nextflow info. It seems that can be used too to get the latest tag.

nextflow info -o json nf-core/rnaseq | jq -r '.revisions.current'
11

I’m 75 % sure that nextflow info needs a nextflow pull before in order to get the latest version!

Or something like nextflow info -latest would probably solve that, but -latest is not available for nextflow info

1 Like
12

There’s no -latest, but there is -check-updates

But you’re correct. The workflow needs to have been pulled.

nextflow info nf-core/mag

fails immediately since it hasn’t been pulled.

1 Like
13

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.