# Quarry - Complete + Onboarding

$300 one-time. All twelve, plus a 1:1 session.

## What this is

A working, pinned Docker Compose stack for self-hosted data engineering and
analytics. Not a link dump - the integration layer: services wired together,
environment template, quickstart, runbook, and a script that fetches the exact
upstream sources at pinned commits.

## Included

- All twelve components
- 1:1 onboarding session
- Lifetime updates

## Components

| Component | Licence | Language | Role |
|---|---|---|---|
| duckdb | MIT | C++ | In-process analytical database |
| polars | MIT | Rust | Fast dataframe library |
| airflow | Apache-2.0 | Python | Workflow orchestration |
| ClickHouse | Apache-2.0 | C++ | Columnar OLAP database |
| trino | Apache-2.0 | Java | Distributed SQL query engine |
| superset | Apache-2.0 | Python | Business intelligence and dashboards |
| streamlit | Apache-2.0 | Python | Data apps in pure Python |
| kafka | Apache-2.0 | Java | Event streaming platform |
| debezium | Apache-2.0 | Java | Change data capture |
| iceberg | Apache-2.0 | Java | Open table format |
| dagster | Apache-2.0 | Python | Data orchestration with typed assets |
| prefect | Apache-2.0 | Python | Workflow orchestration |

## Quickstart

```bash
cp .env.example .env
docker compose up -d
./verify.sh
```

Endpoints:
- `airflow` -> http://localhost:8080
- `dagster` -> http://localhost:3000
- `prefect` -> http://localhost:4200
- `kafka` -> http://localhost:9092
- `debezium` -> http://localhost:8083
- `clickhouse` -> http://localhost:8123
- `trino` -> http://localhost:8081
- `superset` -> http://localhost:8088

DuckDB, Polars, Iceberg and Streamlit are libraries rather than services. See
`docs/` for how to use each.

## Getting the sources

```bash
./fetch-sources.sh ./sources
```

All components are MIT or Apache-2.0. See `ATTRIBUTION.md`.

## Requirements

- Docker Engine 24+ with the Compose v2 plugin
- 4 GB RAM for the library tier, 8 GB with Airflow and ClickHouse,
  16 GB+ for the full stack including Kafka and Trino

## Support

See `RUNBOOK.md`.