Bringing Pixi to Nextflow: A Smarter Way to Manage Environments
Reproducible environments are a cornerstone of modern bioinformatics workflows - and while Conda has long been the default in Nextflow pipelines, it’s not without challenges: slow environment resolution, inconsistent lockfile behavior, and bloated specs. Enter Pixi, a fast, modern, and TOML-based package and environment manager developed by prefix.dev.
Pixi builds on the conda-forge and bioconda ecosystems, but implements its own dependency resolver written in Rust for lightning-fast solves. It offers a structured, developer-friendly approach to managing environments. With a single pixi.toml
file, you define dependencies, platforms, tasks, and even multi-environment matrices - all version-locked and reproducible via pixi.lock
.
However, unlike Conda, Pixi is not yet a first-class citizen in Nextflow’s DSL - meaning you can’t simply use a pixi
directive inside your process
blocks.
But that doesn’t mean you can’t use it. This post shows how to integrate Pixi seamlessly into your Nextflow workflows using a combination of smart beforeScript
hooks and lightweight configuration - giving you all the benefits of Pixi without waiting on native support.
🧪 A Minimal Pixi + Nextflow Example
Let’s test it with a simple process that runs samtools --version
.
🧾 Step 1: Define pixi.toml
[project]
name = "test-pixi-env"
channels = ["conda-forge", "bioconda"]
platforms = ["linux-64"]
[dependencies]
samtools = "*"
🧬 Step 2: Create main.nf
process check_pixi_tool {
label 'pixi'
output:
path 'version.txt'
script:
"""
samtools --version > version.txt
"""
}
workflow {
check_pixi_tool()
}
⚙️ Step 3: Configure nextflow.config
params.pixi_env = './pixi.toml'
process {
withLabel: 'pixi' {
beforeScript = '''
[ -d .pixi ] && [ -f pixi.lock ] || pixi install
eval "$(pixi shell-hook)"
'''
}
}
This script:
- Checks if Pixi has already installed the environment (via
.pixi
andpixi.lock
) - Runs
pixi install
only if needed - Activates the environment via
pixi shell-hook
☁️ Deploying to AWS Batch
If you’re planning to use Pixi on AWS Batch, the key requirement is ensuring that your compute environment or container image includes Pixi itself - since Nextflow does not manage that installation for you.
✅ Requirements:
- ✅ Pixi installed (e.g., via
curl -fsSL https://pixi.sh/install.sh | bash
) - ✅ A
pixi.toml
file available inside the container or synced at runtime - ⚠️
nextflow.config
must reference the correct path topixi.toml
relative to the work directory
FROM ubuntu:22.04
# Install Pixi
RUN curl -fsSL https://pixi.sh/install.sh | bash
# Add Pixi to PATH
ENV PATH="/root/.pixi/bin:$PATH"
# Copy environment file (optional - you may also pull from S3)
COPY pixi.toml /workspace/pixi.toml
WORKDIR /workspace
⚠️ Note: In cloud environments like AWS Batch, you’ll likely need to make sure that
params.pixi_env
innextflow.config
resolves to the correct location inside the container (e.g.,/workspace/pixi.toml
or"$PWD/pixi.toml"
depending on mount context).
Summary
Feature | Pixi Support with Nextflow |
---|---|
Local development | ✅ Full support via beforeScript |
Dynamic install | ✅ Conditional logic included |
AWS Batch | ✅ With prebuilt container |
Native DSL syntax | ❌ Not supported (no pixi directive yet) |
💬 Final Thoughts
Pixi offers a lightweight, predictable, and developer-friendly way to manage environments - without the overhead often associated with Conda. While it isn’t yet supported as a native DSL directive in Nextflow, this guide shows how easy it is to integrate Pixi into your pipelines today using simple configuration patterns.
Whether you’re exploring reproducibility on your laptop or scaling to AWS Batch, Pixi + Nextflow is a powerful and modern combination worth trying.