DagsterDocs

Integrations#

Dagster is flexible and allows incremental adoption. It provides add-on libraries to integrate with your existing tools and infrastructure.

Guides#

This section includes guides on how to use Dagster with other tools.

NameDescription
Dagster with dbtOrchestrate dbt from Dagster.
Dagster with Great ExpectationsRun data quality tests using Great Expectations in a Dagster pipeline.
Dagster with PySparkDefine and execute spark jobs in Dagster.
Dagster with PandasHow Dagster works with Pandas.
Dagster with Jupyter/PapermillHow to orchestrate Jupyter notebooks from Dagster.
Dagster with AirflowUse Dagster in an Airflow cluster, or transform Airflow DAGs into Dagster pipelines.

Libraries#

Here is a complete list of Dagster's integration libraries. See full documentation in API Reference.

IntegrationLibrary
Airflowdagster-airflow
AWSdagster-aws
Azuredagster-azure
Celerydagster-celery
Celery + Dockerdagster-celery-docker
Crondagster-cron
Daskdagster-dask
Databricksdagster-databricks
Datadogdagster-datadog
Dockerdagster-docker
dbtdagster-dbt
GCPdagster-gcp
Great Expectationsdagster-ge
Githubdagster-github
Kubernetesdagster-k8s
MySQLdagster-mysql
PagerDutydagster-pagerduty
Pandasdagster-pandas
Papermilldagstermill
Papertraildagster-papertrail
PostgreSQLdagster-postgres
Prometheusdagster-prometheus
Pysparkdagster-pyspark
Shelldagster-shell
Slackdagster-slack
Snowflakedagster-snowflake
Sparkdagster-spark
SSH / SFTPdagster-ssh
Twiliodagster-twilio