Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Contributing

> **Work in progress.** A full contribution guide will be added in a follow-up PR.

Thank you for your interest in contributing to olap-sql!

## Quick start

1. Fork the repository and create a feature branch from `main`.
2. Make your changes with tests where applicable.
3. Run `go test ./...` to ensure all tests pass.
4. Open a pull request against `main` and describe your changes.

## Guidelines

- Follow standard Go conventions (`gofmt`, clear package names, documented exports).
- Keep commits focused and write meaningful commit messages.
- For larger changes, open an issue first to discuss the approach.

## Coming soon

- Detailed development environment setup
- Test strategy and coverage expectations
- Release process
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,9 +194,13 @@ fmt.Println(sql)

| Document | Description |
|----------|-------------|
| [Getting Started](./docs/getting-started.md) | Step-by-step guide to your first query |
| [Configuration](./docs/configuration.md) | Configure Manager, clients, and the OLAP dictionary |
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docs/examples.md does not exist in the repo — this link will 404. Please either add a stub file in this PR or remove the row until the file is ready.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added docs/examples.md as a stub file (commit 5781eab). It acknowledges the placeholder status and lists what will be covered in a follow-up PR. The link is now live.

| [Query](./docs/query.md) | Define metrics, dimensions, filters, orders, and limits |
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docs/architecture.md does not exist. Same as above — stub or remove.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added docs/architecture.md as a stub file (commit 5781eab). It includes a high-level component table and a "Coming soon" section for the deeper content. Link resolves now.

| [Result](./docs/result.md) | Parse and work with query results |
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CONTRIBUTING.md does not exist. Same as above — stub or remove.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added CONTRIBUTING.md at repo root as a stub file (commit 5781eab). Covers fork/branch/test/PR basics and flags a "Coming soon" section for deeper guidance. Link resolves now.

| [Examples](./docs/examples.md) | Common usage scenarios (ClickHouse joins, time filters, concurrency) |
| [Architecture](./docs/architecture.md) | Internal design for contributors |
| [Contributing](./CONTRIBUTING.md) | How to contribute to olap-sql |

---

Expand Down
22 changes: 22 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Architecture

> **Work in progress.** This file is a placeholder — a full architecture deep-dive will be added in a follow-up PR.

## High-level overview

olap-sql is organized around four core abstractions:

| Component | Responsibility |
|-----------|----------------|
| **Schema / Configuration** | Declares the virtual schema: sets, sources, dimensions, metrics, and joins |
| **Manager** | Loads configuration and owns the compiled schema at runtime |
| **Query** | Describes what data the caller wants (filters, grouping, ordering, pagination) |
| **Result** | Delivers the query output as structured rows |

See [Getting Started](./getting-started.md) for a walkthrough of how these fit together.

## Coming soon

- Component interaction diagrams
- Dependency graph design
- Extension points and custom adapters
12 changes: 12 additions & 0 deletions docs/examples.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Examples

> **Work in progress.** This file is a placeholder — detailed usage examples will be added in a follow-up PR.

For now, see the inline code snippets in [Getting Started](./getting-started.md) and the test files in the repository root for practical usage patterns.

## Coming soon

- SQLite quick-start example
- ClickHouse multi-tenant example
- Filtering, ordering, and pagination patterns
- Custom dictionary adapters
252 changes: 252 additions & 0 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,252 @@
# Getting Started with olap-sql

This guide walks you through everything you need to start generating OLAP SQL with olap-sql — from installation to running your first query.

---

## Table of Contents

1. [Prerequisites](#prerequisites)
2. [Installation](#installation)
3. [Core Concepts](#core-concepts)
4. [Step 1 — Define your schema](#step-1--define-your-schema)
5. [Step 2 — Create a Manager](#step-2--create-a-manager)
6. [Step 3 — Build and inspect SQL](#step-3--build-and-inspect-sql)
7. [Step 4 — Run a query](#step-4--run-a-query)
8. [Step 5 — Work with the result](#step-5--work-with-the-result)
9. [What's Next](#whats-next)

---

## Prerequisites

- **Go 1.22+** — olap-sql uses range-over-integer syntax (Go 1.22) and generics (Go 1.18+).
- A running database. The examples below use **SQLite** (zero config, no server required) so you can run them immediately. ClickHouse, MySQL, and PostgreSQL are all supported — swap out the `ClientsOption` when you are ready.

---

## Installation

```bash
go get github.com/awatercolorpen/olap-sql
```

---

## Core Concepts

Before writing any code it helps to know the four moving parts:

| Concept | What it is |
|----------------|---------------------------------------------------------------------|
| **Schema** | A TOML file describing your tables, metrics, and dimensions. |
| **Manager** | The main entry point. Holds DB clients + the schema dictionary. |
| **Query** | A struct (or JSON) you build to describe *what* you want. |
| **Result** | The structured response: a list of column names and row data. |

The flow is:

```
Query → Dictionary (schema) → Clause (IR) → SQL → DB → Result
```

You never write SQL by hand. You describe what you want and olap-sql figures out the SQL.

---

## Step 1 — Define your schema

Create a file called `olap-sql.toml` in your project root.

```toml
# Each "set" is a named query context (= a business data set).
sets = [
{name = "sales", type = "sqlite", data_source = "orders"},
]

# Each "source" maps to a real database table.
sources = [
{database = "", name = "orders", type = "fact"},
]

# Metrics define how numeric columns are aggregated.
metrics = [
{data_source = "orders", type = "METRIC_COUNT", name = "order_count", field_name = "*", value_type = "VALUE_INTEGER"},
{data_source = "orders", type = "METRIC_SUM", name = "revenue", field_name = "amount", value_type = "VALUE_FLOAT"},
{data_source = "orders", type = "METRIC_DIVIDE", name = "avg_order",
value_type = "VALUE_FLOAT", dependency = ["orders.revenue", "orders.order_count"]},
]

# Dimensions define how rows are grouped and filtered.
dimensions = [
{data_source = "orders", type = "DIMENSION_SINGLE", name = "date", field_name = "date", value_type = "VALUE_STRING"},
{data_source = "orders", type = "DIMENSION_SINGLE", name = "region", field_name = "region", value_type = "VALUE_STRING"},
]
```

**Key rules:**

- Each `sets` entry needs a `name` (used in queries), a `type` (database driver), and a `data_source` (references a `sources` entry).
- `metrics` with `type = "METRIC_DIVIDE"` don't have a `field_name` — they reference two other metrics via `dependency`.
- `dimensions` name must be unique within a `data_source`.

---

## Step 2 — Create a Manager

```go
package main

import (
"log"

olapsql "github.com/awatercolorpen/olap-sql"
"github.com/awatercolorpen/olap-sql/api/types"
)

func main() {
cfg := &olapsql.Configuration{
// ClientsOption: one entry per database connection.
// The key must match the "type" field in your sets (e.g. "sqlite").
ClientsOption: olapsql.ClientsOption{
"sqlite": {
DSN: "file::memory:?cache=shared",
Type: types.DBTypeSQLite,
},
},
// DictionaryOption: points to your schema TOML.
DictionaryOption: &olapsql.Option{
AdapterOption: olapsql.AdapterOption{Dsn: "olap-sql.toml"},
},
}

manager, err := olapsql.NewManager(cfg)
if err != nil {
log.Fatal(err)
}

_ = manager // use it in the next steps
}
```

> **Tip:** You can register multiple DB connections by adding more keys to `ClientsOption`.
> For example `"clickhouse"` and `"mysql"` can coexist — each set in the schema will route to its own client.

---

## Step 3 — Build and inspect SQL

Before executing anything, it is useful to see what SQL olap-sql would generate.

```go
query := &types.Query{
DataSetName: "sales",
TimeInterval: &types.TimeInterval{
Name: "date",
Start: "2024-01-01",
End: "2024-02-01",
},
Metrics: []string{"order_count", "revenue", "avg_order"},
Dimensions: []string{"date", "region"},
Orders: []*types.OrderBy{
{Name: "date", Direction: types.OrderDirectionAscending},
},
}

sql, err := manager.BuildSQL(query)
if err != nil {
log.Fatal(err)
}
fmt.Println(sql)
```

Expected output (SQLite):

```sql
SELECT
orders.date AS date,
orders.region AS region,
COUNT(*) AS order_count,
SUM(orders.amount) AS revenue,
(1.0 * SUM(orders.amount)) / NULLIF(COUNT(*), 0) AS avg_order
FROM orders AS orders
WHERE (orders.date >= '2024-01-01' AND orders.date < '2024-02-01')
GROUP BY orders.date, orders.region
ORDER BY orders.date ASC
```

`BuildSQL` never touches the database — it is safe to call at any time and is very useful for debugging.

---

## Step 4 — Run a query

Once you are happy with the SQL, run it:

```go
result, err := manager.RunSync(query)
if err != nil {
log.Fatal(err)
}
```

For **large result sets** where you don't want to buffer everything in memory, use the channel-based variant:

```go
result, err := manager.RunChan(query)
if err != nil {
log.Fatal(err)
}
```

Both methods return `*types.Result`.

---

## Step 5 — Work with the result

```go
import (
"encoding/json"
"fmt"
)

// result.Dimensions is a []string listing every column in order.
fmt.Println("columns:", result.Dimensions)

// result.Source is []map[string]any — one map per row.
for _, row := range result.Source {
fmt.Printf("date=%v region=%v revenue=%v avg_order=%v\n",
row["date"], row["region"], row["revenue"], row["avg_order"])
}

// Or marshal the whole result to JSON:
out, _ := json.MarshalIndent(result, "", " ")
fmt.Println(string(out))
```

Example JSON output:

```json
{
"dimensions": ["date", "region", "order_count", "revenue", "avg_order"],
"source": [
{"date": "2024-01-05", "region": "north", "order_count": 12, "revenue": 4320.0, "avg_order": 360.0},
{"date": "2024-01-05", "region": "south", "order_count": 7, "revenue": 1890.5, "avg_order": 270.07},
{"date": "2024-01-12", "region": "north", "order_count": 19, "revenue": 7600.0, "avg_order": 400.0}
]
}
```

---

## What's Next

| Topic | Where to look |
|-------|---------------|
| Full configuration reference | [docs/configuration.md](./configuration.md) |
| All query options (filters, orders, limits, raw SQL) | [docs/query.md](./query.md) |
| Result format details | [docs/result.md](./result.md) |
| Common usage scenarios | [docs/examples.md](./examples.md) |
| Architecture for contributors | [docs/architecture.md](./architecture.md) |
| Contribution guide | [CONTRIBUTING.md](../CONTRIBUTING.md) |
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ olap-sql 是一个 Go 的 OLAP 查询 SQL 生成库,支持 metrics、dimension

### 任务清单

- [ ] 重写 Getting Started(step-by-step,有完整可运行代码)
- [x] 重写 Getting Started(step-by-step,有完整可运行代码)
- [ ] API 文档:每个方法说明参数、返回值、使用场景
- [ ] 增加常见使用场景示例:
- ClickHouse 多表联查
Expand Down
Loading