Getting Started with Dagster#

Dagster is a data orchestrator for machine learning, analytics, and ETL

Or read about:

• Dagster’s Main Concepts
• How to deploy Dagster
• How to integrate Dagster with other tools
• Best practices guides
• Dagster’s full API Reference
• Dagster's open source Community

Quick Start#

Installing Dagster#

To install Dagster and Dagit into an existing Python environment, run:

pip install dagster

This will install the latest stable version of the core Dagster packages in your current Python environment.

Writing a Pipeline#

Let's get your first pipeline up and running.

from dagster import pipeline, solid


@solid
def get_name(_):
    return "dagster"


@solid
def hello(context, name: str):
    context.log.info("Hello, {name}!".format(name=name))


@pipeline
def hello_pipeline():
    hello(get_name())

Save the code above in a file named hello_world.py.

You can execute the pipeline in three different ways: Dagit, Dagster Python API, or Dagster CLI.

Running the Pipeline in Dagit#

It's highly recommended to use Dagit with Dagster. Dagit is a web-based interface for viewing and interacting with Dagster objects.

pip install dagit

To visualize your pipeline in Dagit, run the following command:

dagit -f hello_world.py

Then navigate to http://localhost:3000 to start using Dagit:

Click on the "Playground" tab, then press the "Launch Execution" button to execute the pipeline. You will then see Dagit launches a pipeline run:

Other Options to Run Dagster Pipelines#

You can also execute the pipeline without the UI in the following methods:

Dagster Python API

from dagster import execute_pipeline

if __name__ == "__main__":
    result = execute_pipeline(hello_pipeline)

Dagster CLI

dagster pipeline execute -f hello_world.py

If You Get Stuck#

If you have questions on getting started, we'd love to hear from you: