# S3 work folder **URL:** \ **Category:** Ask for help\ **Tags:** platform\ **Created:** [February 4, 2025, 5:50am UTC](https://community.seqera.io/t/s3-work-folder/1699 "2025-02-04T05:50:58Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![Miguel\_Grau](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/miguel_grau/32/1420_2.png) [@Miguel\_Grau](https://community.seqera.io/u/Miguel_Grau)\ **Post date:** [February 4, 2025, 5:50am UTC](https://community.seqera.io/t/s3-work-folder/1699/1 "2025-02-04T05:50:58Z")
Hi, when trying to use a s3 folder as a work dir when launching a pipeline from seqera, I get this error: ```auto Pipeline work directory must start with slash character - offending value: s3://scratch/work ``` I tried defining `workDir` from the nextflow.config but it is still using the one defined during the lunch form (where I had to put a non-s3 one) ---
**Author:** ![robsyme](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/robsyme/32/342_2.png) [@robsyme](https://community.seqera.io/u/robsyme)\ **Post date:** [February 6, 2025, 7:53pm UTC](https://community.seqera.io/t/s3-work-folder/1699/2 "2025-02-06T19:53:39Z")
Hi Miguel I’m not sure I totally understand your situation. Some quick questions: 1. These are runs launched from Seqera Platform, correct? 2. What is the work directory defined in the Compute Environment (CE) in which you are executing the run? 3. What was the value you trying to set the work directory to? 4. It sounds like you’d like to override that work directory for a particular pipeline or a particular run. Is that correct? In general, if you want to override the CE for a particular run, it would be sensible to use the “Work directory” text field as shown below. Is that an option for your situation? ![CleanShot 2025-02-06 at 14.52.45@2x](https://europe1.discourse-cdn.com/flex013/uploads/seqera/original/2X/7/7644406cdf0f5b11f2418ce9abee2c74220055a3.png) ---
**Author:** ![Miguel\_Grau](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/miguel_grau/32/1420_2.png) [@Miguel\_Grau](https://community.seqera.io/u/Miguel_Grau)\ **Post date:** [February 7, 2025, 5:21am UTC](https://community.seqera.io/t/s3-work-folder/1699/3 "2025-02-07T05:21:09Z")
Hi robsyme, 1. Yes. 2. It is a general work folder in the netapp 3. The value is something similar to your screenshot: `s3://scratch/work/` 4. Yes That’s exactly my situation. When I try overriding using a s3 folder, I get the error: `Pipeline work directory must start with slash character - offending value: s3://scratch/work ` and I can’t submit it. I also tried not overriding the work folder in the “Run parameters” tab but in the Advanced settings, in the nextflow.config, but it doesn’t work either. ---
**Author:** ![robsyme](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/robsyme/32/342_2.png) [@robsyme](https://community.seqera.io/u/robsyme)\ **Post date:** [February 8, 2025, 2:45pm UTC](https://community.seqera.io/t/s3-work-folder/1699/4 "2025-02-08T14:45:48Z")
Thanks for the info, Miguel. Are you launching the run using a compute environment configured with an HPC executor (SLURM/PBS/SGE, etc)? Using an S3 path as the work directory only makes sense when using the AWS Batch executor. It would be more efficient to use a networked filesystems on your HPC rather than reading from and writing to a (potentially) remote object store like S3. ---
**Author:** ![Miguel\_Grau](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/miguel_grau/32/1420_2.png) [@Miguel\_Grau](https://community.seqera.io/u/Miguel_Grau)\ **Post date:** [February 11, 2025, 5:47am UTC](https://community.seqera.io/t/s3-work-folder/1699/5 "2025-02-11T05:47:33Z")
Hi robsyme, - Yes, we are using SLURM. - We are using [minio](https://www.nextflow.io/docs/latest/amazons3.html#s3-compatible-storage) configured in a local S3. Since it is local, the performance could be enough, right? ---
**Author:** ![Miguel\_Grau](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/miguel_grau/32/1420_2.png) [@Miguel\_Grau](https://community.seqera.io/u/Miguel_Grau)\ **Post date:** [February 12, 2025, 11:52am UTC](https://community.seqera.io/t/s3-work-folder/1699/6 "2025-02-12T11:52:42Z")
Just to clarify, I’m using Slurm with the Tower Agent ---
**Author:** ![Miguel\_Grau](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/miguel_grau/32/1420_2.png) [@Miguel\_Grau](https://community.seqera.io/u/Miguel_Grau)\ **Post date:** [July 9, 2025, 4:50pm UTC](https://community.seqera.io/t/s3-work-folder/1699/7 "2025-07-09T16:50:38Z")
Hi @robsyme. Any update on this? ---
**Author:** ![robsyme](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/robsyme/32/342_2.png) [@robsyme](https://community.seqera.io/u/robsyme)\ **Post date:** [July 9, 2025, 5:07pm UTC](https://community.seqera.io/t/s3-work-folder/1699/8 "2025-07-09T17:07:15Z")
Apologies, Miguel. When using a non-awsbatch executor, the work directory must be a path on a shared filesystem rather than an S3 bucket. It is difficult for Platform to verify that both the head and compute nodes will have the required permissions to read from and write to that remote location, and is likely to cause errors - even if the performance is sufficient. Is there a shared filesystem that you can use on this cluster? ---
**Author:** ![robsyme](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/robsyme/32/342_2.png) [@robsyme](https://community.seqera.io/u/robsyme)\ **Post date:** [July 9, 2025, 5:11pm UTC](https://community.seqera.io/t/s3-work-folder/1699/9 "2025-07-09T17:11:46Z")
I have also created a feature request on your behalf at: [Demote message about workdir prefix from error to warning when using s3 buckets as workdir on SLURM | Voters | Seqera Feedback Forum](https://feedback.seqera.io/feature-requests/p/demote-message-about-workdir-prefix-from-error-to-warning-when-using-s3-buckets) You can upvote and/or follow this feature for updates. ---
**Author:** ![Miguel\_Grau](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/miguel_grau/32/1420_2.png) [@Miguel\_Grau](https://community.seqera.io/u/Miguel_Grau)\ **Post date:** [July 9, 2025, 5:36pm UTC](https://community.seqera.io/t/s3-work-folder/1699/10 "2025-07-09T17:36:19Z")
Thanks, Rob! Already upvoted 🙂 Yes, we have a shared filesystem for the work folder (which we are using as the default one) but our input data is on a local s3 minio. It is a huge amount of data in some runs, so we were interested in using fusion to avoid the staging step and speed up the pipeline execution in general. ---
**Author:** ![Poshi](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/poshi/32/1126_2.png) [@Poshi](https://community.seqera.io/u/Poshi)\ **Post date:** [July 14, 2025, 9:13am UTC](https://community.seqera.io/t/s3-work-folder/1699/11 "2025-07-14T09:13:39Z")
Never used neither S3 nor the Tower environment, but did you try the obvious? Did you tried converting the relative path you are trying to use to an absolute one (one that starts with a slash)? ---
**Author:** ![Miguel\_Grau](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/miguel_grau/32/1420_2.png) [@Miguel\_Grau](https://community.seqera.io/u/Miguel_Grau)\ **Post date:** [July 15, 2025, 5:53am UTC](https://community.seqera.io/t/s3-work-folder/1699/12 "2025-07-15T05:53:18Z")
I am not using relative paths, @Poshi. These are s3 storage paths. ---
**Author:** ![Poshi](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/poshi/32/1126_2.png) [@Poshi](https://community.seqera.io/u/Poshi)\ **Post date:** [July 15, 2025, 9:43am UTC](https://community.seqera.io/t/s3-work-folder/1699/13 "2025-07-15T09:43:31Z")
Taking the first message you posted I saw: ```auto Pipeline work directory must start with slash character - offending value: s3://scratch/work ``` That is a relative path. Or at least, it is a relative path according to the standard. Maybe S3 works differently and don’t follow the standards, but I don’t think so. ---
**Author:** ![Miguel\_Grau](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/miguel_grau/32/1420_2.png) [@Miguel\_Grau](https://community.seqera.io/u/Miguel_Grau)\ **Post date:** [July 15, 2025, 9:59am UTC](https://community.seqera.io/t/s3-work-folder/1699/14 "2025-07-15T09:59:45Z")
Check how a s3 path looks, please. e.g. [File paths in Amazon S3 - Media2Cloud on AWS](https://docs.aws.amazon.com/solutions/latest/media2cloud-on-aws/file-paths-in-amazon-s3.html) ---
**Author:** ![Poshi](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/poshi/32/1126_2.png) [@Poshi](https://community.seqera.io/u/Poshi)\ **Post date:** [July 15, 2025, 10:12am UTC](https://community.seqera.io/t/s3-work-folder/1699/15 "2025-07-15T10:12:07Z")
Oh! Ok, I assumed that “scratch” was part of the path, not the bucket name. Then everything seems fine in the naming. Forgot my comment ☹