Skip to main content

Starter Pack

The runtime-starter-pack is a template repository with working examples that demonstrate the full dltHub Runtime workflow: extract, load, transform, validate, deploy, and monitor.

What's included​

Fruitshop pipeline (fruitshop_pipeline.py)​

A beginner-friendly batch pipeline using synthetic data -- no external API keys needed. It defines four resources (customers, inventory categories, inventory, and purchases) grouped into a fruitshop() source, and loads them into your configured destination.

Use it to test deploying and scheduling:

dlt runtime launch fruitshop_pipeline.py --follow
dlt runtime schedule fruitshop_pipeline.py "*/10 * * * *"

GitHub API pipeline (github-example/github_pipeline.py)​

A REST API pipeline that ingests commits and contributors from a GitHub repository. Demonstrates bearer token authentication, pagination handling, and multiple endpoint extraction.

Fruitshop notebook (fruitshop_notebook.py)​

An interactive Marimo notebook that attaches to the fruitshop pipeline and demonstrates:

  • Data quality checks -- validates product names, ensures prices are not null, flags negative prices
  • Transformations -- customer aggregations and grouping using Ibis expressions
  • Pipeline inspection -- schema inspection and data exploration

Use it to test serving and sharing:

dlt runtime serve fruitshop_notebook.py
dlt runtime publish fruitshop_notebook.py

Jaffle transformations (jaffle_transformations.py)​

Post-ingestion transformations using the @dlt.hub.transformation decorator. Shows customer order aggregation, payment joins, and multi-stage processing with Ibis expressions.

Using the starter pack as a template​

  1. Clone the repo and install dependencies:

    git clone https://github.com/dlt-hub/runtime-starter-pack.git my-project
    cd my-project
    uv sync

  2. Configure your credentials in .dlt/secrets.toml (see the Quickstart)

  3. Test locally first:

    python fruitshop_pipeline.py

  4. Deploy to Runtime:

    dlt runtime login
    dlt runtime launch fruitshop_pipeline.py --follow

  5. Modify the examples or add your own pipeline scripts -- Runtime will pick up any Python file you deploy.

Project structure​

runtime-starter-pack/
├── .dlt/                          # dlt configuration directory
│   ├── config.toml                # Pipeline settings
│   ├── secrets.toml               # Credentials (git-ignored)
│   ├── config.prod.toml           # Batch job profile config
│   └── secrets.access.toml        # Interactive job profile secrets
├── fruitshop_pipeline.py          # Batch pipeline example
├── fruitshop_notebook.py          # Interactive notebook example
├── jaffle_transformations.py      # Transformation examples
├── github-example/                # REST API pipeline example
│   └── github_pipeline.py
└── pyproject.toml                 # Python dependencies

For a detailed walkthrough of each example, see the Runtime tutorial in the dlt docs.