Skip to content
@pipefunc

PipeFunc

Lightweight function pipeline (DAG) creation in pure Python for scientific workflows 🕸️🧪

PipeFunc: Automate Python DAGs for Scientific Workflows - From Local to HPC with N-D Parameter Sweeps

pipefunc is a Python library that simplifies function composition and pipeline creation for scientific computing and data processing. It automatically generates DAGs from Python functions, efficiently handles N-dimensional parameter sweeps, and scales seamlessly from local to HPC environments. Key features include auto-parallization, N-dimensional map-reduce, and type checking. pipefunc minimizes boilerplate code, allowing users to focus on logic while it manages function dependencies and execution order. Ideal for complex workflows in data science, ML, and scientific computing.

Pinned Loading

  1. pipefunc pipefunc Public

    Lightweight function pipeline (DAG) creation in pure Python for scientific workflows 🕸️🧪

    Python 186 8

Repositories

Showing 2 of 2 repositories
  • pipefunc Public

    Lightweight function pipeline (DAG) creation in pure Python for scientific workflows 🕸️🧪

    pipefunc/pipefunc’s past year of commit activity
    Python 186 MIT 8 17 22 Updated Oct 17, 2024
  • .github Public
    pipefunc/.github’s past year of commit activity
    0 0 0 0 Updated Sep 19, 2024

People

This organization has no public members. You must be a member to see who’s a part of this organization.

Top languages

Loading…

Most used topics

Loading…