diff --git a/docs/guides/production-deployment-guide/index.mdx b/docs/guides/production-deployment-guide/index.mdx new file mode 100644 index 00000000..c97e7142 --- /dev/null +++ b/docs/guides/production-deployment-guide/index.mdx @@ -0,0 +1,56 @@ +--- +title: "Building with OpenTDF: From Local Development to Production" +sidebar_label: "Production Deployment Guide" +sidebar_position: 2 +--- + +# Building with OpenTDF: From Local Development to Production + +This guide walks you through integrating a frontend and backend application with OpenTDF, then publishing it to the web. We use a medical practice as a running example: a clinic where patients encrypt health data and only authorized staff can decrypt it. + +:::note +This is a lightweight example using [Railway](https://railway.app) for hosting, aimed at developers new to working with Keycloak configuration and OpenTDF platform setup. In production, you would authenticate with JWTs from your own identity provider. Follow security best practices appropriate to your deployment environment and use case. +::: + +## What You're Building + +```text +┌─────────────────────────────────────────────────┐ +│ Your Application │ +│ (uses OpenTDF SDK to encrypt/decrypt) │ +└──────────┬──────────────────────┬───────────────┘ + │ gRPC/HTTP │ OIDC + ▼ ▼ +┌─────────────────────┐ ┌──────────────────────┐ +│ OpenTDF Platform │ │ Identity Provider │ +│ (single binary) │ │ (Keycloak) │ +│ │ │ │ +│ ┌───────────────┐ │ │ - User accounts │ +│ │ KAS │ │ │ - Roles & groups │ +│ │ Policy │ │ │ - JWT issuance │ +│ │ Authorization │ │ │ │ +│ │ Entity Res. │ │ └──────────────────────┘ +│ └───────────────┘ │ +└──────────┬──────────┘ + │ + ▼ +┌─────────────────────┐ +│ PostgreSQL │ +│ (policy storage) │ +└─────────────────────┘ +``` + +OpenTDF Platform is a **single Go binary** that bundles KAS (Key Access Server), Policy, Authorization, and Entity Resolution services. It connects to PostgreSQL for storage and an OIDC provider (Keycloak) for authentication. + +## Guide Structure + +1. **[Local Development](./production-deployment-guide/local-development)** — Set up OpenTDF locally, configure Keycloak, create attributes and policies, and integrate the Go and browser SDKs into your application. + +2. **[Production Deployment](./production-deployment-guide/production-deployment)** — What changes when you deploy to a real environment: TLS, secrets management, split-horizon DNS, container configuration, and security hardening. + +## See Also + +- [Subject Mapping Guide](/guides/subject-mapping-guide) — how identity claims map to attribute-based access control +- [GLP-1 Tracker demo](https://github.com/opentdf/demo-glp1-tracker) — a working example app built with this guide +- [opentdf.io/llms.txt](https://opentdf.io/llms.txt) — comprehensive LLM-optimized reference documentation +- [OpenTDF Discussion Forum](https://github.com/orgs/opentdf/discussions) — more examples and community Q&A diff --git a/docs/guides/production-deployment-guide/local-development.mdx b/docs/guides/production-deployment-guide/local-development.mdx new file mode 100644 index 00000000..6cf0eefd --- /dev/null +++ b/docs/guides/production-deployment-guide/local-development.mdx @@ -0,0 +1,406 @@ +--- +title: "Part 1: Local Development" +sidebar_label: "Local Development" +sidebar_position: 1 +--- + +# Part 1: Local Development + +Set up OpenTDF and build an application that encrypts data with attribute-based access control. + +## Prerequisites + +- A frontend application with OIDC authentication — the browser SDK needs a valid Keycloak token to decrypt. Use a library like [`keycloak-js`](https://www.keycloak.org/docs/latest/securing_apps/#_javascript_adapter) to handle login. See the [GLP-1 Tracker frontend](https://github.com/opentdf/demo-glp1-tracker/tree/main/frontend) for a working example. +- Docker and Docker Compose +- Go (check the platform's [`go.mod`](https://github.com/opentdf/platform/blob/main/service/go.mod) for the required version) +- OpenSSL (for generating KAS encryption keys) +- The platform source code: `git clone https://github.com/opentdf/platform.git` + +:::warning Why do I need the source code? +The `provision keycloak` command that bootstraps your identity provider requires files only in the source tree. The container image does not include them. Until provisioning is available as a standalone tool, you need the source checkout. +::: + +## Step 1: Start PostgreSQL and Keycloak + +Generate self-signed TLS certificates (used for HTTPS between platform services locally — not the KAS encryption keys, which are separate) and start infrastructure: + +```bash +cd platform +./.github/scripts/init-temp-keys.sh +docker compose up -d +``` + +Wait for both to be healthy: +```bash +curl -sf http://localhost:8888/auth/realms/master | head -1 +docker compose exec opentdfdb pg_isready +``` + +:::tip Apple M4 users +Keycloak (Java) may crash with SIGILL. Set `export JAVA_OPTS_APPEND="-XX:UseSVE=0"` before running Docker. +::: + +## Step 2: Provision the Identity Provider + +:::danger Order matters +The platform **cannot start** until the OIDC realm exists. Provision Keycloak BEFORE starting the platform. +::: + +```bash +go run ./service provision keycloak +``` + +This creates the `opentdf` realm, clients (`opentdf`, `opentdf-sdk`, `tdf-entity-resolution`, `cli-client`), roles (`opentdf-admin`, `opentdf-standard`), and sample users. + +### Create your application users + +For bulk user creation, see the [create-users.py](https://github.com/opentdf/demo-glp1-tracker/blob/main/scripts/create-users.py) script in the demo app. + +
+Keycloak user creation commands + +```bash +# Get admin token +ADMIN_TOKEN=$(curl -sf -X POST http://localhost:8888/auth/realms/master/protocol/openid-connect/token \ + -d "grant_type=password" -d "client_id=admin-cli" \ + -d "username=admin" -d "password=changeme" | python3 -c "import sys,json; print(json.load(sys.stdin)['access_token'])") + +# Create a user +curl -sf -X POST http://localhost:8888/auth/admin/realms/opentdf/users \ + -H "Authorization: Bearer $ADMIN_TOKEN" -H "Content-Type: application/json" \ + -d '{"username":"alice","enabled":true,"email":"alice@example.com","emailVerified":true}' + +# Set password +ALICE_ID=$(curl -sf "http://localhost:8888/auth/admin/realms/opentdf/users?username=alice" \ + -H "Authorization: Bearer $ADMIN_TOKEN" | python3 -c "import sys,json; print(json.load(sys.stdin)[0]['id'])") +curl -sf -X PUT "http://localhost:8888/auth/admin/realms/opentdf/users/$ALICE_ID/reset-password" \ + -H "Authorization: Bearer $ADMIN_TOKEN" -H "Content-Type: application/json" \ + -d '{"type":"password","value":"alice123","temporary":false}' + +# Add opentdf-standard role (required for SDK access) +STD_ROLE_ID=$(curl -sf http://localhost:8888/auth/admin/realms/opentdf/roles/opentdf-standard \ + -H "Authorization: Bearer $ADMIN_TOKEN" | python3 -c "import sys,json; print(json.load(sys.stdin)['id'])") +curl -sf -o /dev/null -X POST "http://localhost:8888/auth/admin/realms/opentdf/users/$ALICE_ID/role-mappings/realm" \ + -H "Authorization: Bearer $ADMIN_TOKEN" -H "Content-Type: application/json" \ + -d "[{\"id\":\"$STD_ROLE_ID\",\"name\":\"opentdf-standard\"}]" +``` + +
+ +**Important**: All users need the `opentdf-standard` realm role, or the browser SDK will get `403` when listing KAS servers. To make this automatic for new users: + +1. Keycloak admin → `opentdf` realm → Realm roles → `default-roles-opentdf` +2. Click **Assign role** → Filter by realm roles → Select `opentdf-standard` → **Assign** + +### Configure browser client + +For browser-based apps, update `cli-client` (the public client created by provisioning): + +In Keycloak admin → `opentdf` realm → Clients → `cli-client`: +- Valid redirect URIs: `http://localhost:*` +- Web origins: `+` + +## Step 3: Configure and Start the Platform + +Create `opentdf.yaml` (see the [demo app's local config](https://github.com/opentdf/demo-glp1-tracker/blob/main/opentdf.yaml) for a working example): + +
+Full opentdf.yaml for local development + +```yaml +logger: + level: info + type: text + output: stdout # Only 'stdout' is valid in the container image + +services: + kas: + registered_kas_uri: http://localhost:8080 + keyring: + - kid: e1 + alg: ec:secp256r1 + - kid: e1 + alg: ec:secp256r1 + legacy: true # Used for trial decryption of old TDFs that lack a kid field + - kid: r1 + alg: rsa:2048 + - kid: r1 + alg: rsa:2048 + legacy: true + entityresolution: + mode: claims # See "Understanding ERS Modes" below + url: http://localhost:8888/auth + clientid: "tdf-entity-resolution" + clientsecret: "secret" + realm: "opentdf" + legacykeycloak: true + inferid: + from: + email: true + username: true + +server: + tls: + enabled: false + auth: + enabled: true + enforceDPoP: false + audience: "http://localhost:8080" + issuer: http://localhost:8888/auth/realms/opentdf + policy: + extension: | + g, opentdf-admin, role:admin + g, opentdf-standard, role:standard + cors: + enabled: true + allowedorigins: + - "http://localhost:5173" # Vite dev server + - "http://localhost:3000" + allowedmethods: [GET, POST, PATCH, PUT, DELETE, OPTIONS] + allowedheaders: + - Accept + - Accept-Encoding + - Authorization + - Connect-Protocol-Version + - Content-Length + - Content-Type + - Dpop + - X-CSRF-Token + - X-Requested-With + - X-Rewrap-Additional-Context + exposedheaders: [Link] + allowcredentials: true + maxage: 3600 + grpc: + reflectionEnabled: true + cryptoProvider: + type: standard + standard: + keys: + - kid: r1 + alg: rsa:2048 + private: kas-private.pem + cert: kas-cert.pem + - kid: e1 + alg: ec:secp256r1 + private: kas-ec-private.pem + cert: kas-ec-cert.pem + port: 8080 +``` + +
+ +### Understanding ERS Modes + +The Entity Resolution Service mode controls **what subject mapping selectors match against**. + +| Mode | Selectors target | Use | +|------|-----------------|-----| +| `claims` | JWT token claims directly | Simpler — `.preferred_username`, `.client_id`, `.realm_access.roles[]` | +| `keycloak` (default) | Keycloak Admin API user object | When matching Keycloak user attributes: `.attributes.department[]` | + +**Use `claims` mode** unless you specifically need Keycloak Admin API data. See the [Subject Mapping Guide](/guides/subject-mapping-guide) for selector syntax. + +:::warning .sub selector doesn't work in claims mode +The `.sub` JWT claim resolves correctly in `otdfctl dev selectors generate` but does NOT evaluate in subject mapping conditions. Use `.preferred_username` instead. This is a [known issue](https://github.com/opentdf/platform/issues/3188). +::: + +### Start the platform + +```bash +go run ./service start +``` + +Verify: +```bash +curl -sf http://localhost:8080/healthz +# {"status":"SERVING"} +``` + +:::danger Don't run the platform in Docker for local dev +Running the platform in Docker alongside Keycloak causes split-horizon DNS — tokens from `localhost:8888` have different `iss` claims than tokens from `keycloak:8888`. Use `KC_HOSTNAME_URL` in production (see [Part 2](./production-deployment)), or run the platform on the host for local dev. +::: + +## Step 4: Set Up Attributes and Policy + +The authorization chain has four links — **ALL must be in place** before decrypt works: + +```text +1. Attribute exists in policy DB +2. KAS is registered with keys and granted to the attribute +3. Subject mapping connects identity claims to the attribute +4. Encrypted data is tagged with the attribute +``` + +### Create attributes + +Each `otdfctl` command below outputs an ID. Copy it — you'll need it in the next command. + +```bash +# Create namespace (must be a valid hostname, no scheme) +otdfctl policy attributes namespaces create \ + --host http://localhost:8080 \ + --with-client-creds '{"clientId":"opentdf","clientSecret":"secret"}' \ + --name "clinic.example" + +# Create attribute with values +otdfctl policy attributes create \ + --host http://localhost:8080 \ + --with-client-creds '{"clientId":"opentdf","clientSecret":"secret"}' \ + --namespace \ + --name staff_role --rule ANY_OF \ + --value doctor --value nurse --value admin +``` + +### Register KAS with keys + +
+KAS registration and key assignment commands + +```bash +# Register KAS +otdfctl policy kas-registry create \ + --host http://localhost:8080 \ + --with-client-creds '{"clientId":"opentdf","clientSecret":"secret"}' \ + --uri http://localhost:8080 \ + --public-key-remote http://localhost:8080/kas/v2/kas_public_key \ + --name local-kas + +# Create KAS key (base64-encode the PEM) +KAS_PUB_B64=$(curl -sf http://localhost:8080/kas/v2/kas_public_key | python3 -c "import sys,json,base64; print(base64.b64encode(json.load(sys.stdin)['publicKey'].encode()).decode())") +otdfctl policy kas-registry key create \ + --host http://localhost:8080 \ + --with-client-creds '{"clientId":"opentdf","clientSecret":"secret"}' \ + --kas --key-id "r1" --algorithm "rsa:2048" \ + --mode "public_key" --public-key-pem "$KAS_PUB_B64" + +# Assign key to attribute +otdfctl policy attributes key assign \ + --host http://localhost:8080 \ + --with-client-creds '{"clientId":"opentdf","clientSecret":"secret"}' \ + --attribute --key-id +``` + +
+ +### Create subject mappings + +One mapping per attribute value. See the [Subject Mapping Guide](/guides/subject-mapping-guide) for selector syntax, operators, and patterns. For programmatic creation of attributes and subject mappings, see the [policy manager](https://github.com/opentdf/demo-glp1-tracker/blob/main/backend/policymanager/manager.go) in the demo app. + +```bash +otdfctl policy subject-mappings create \ + --host http://localhost:8080 \ + --with-client-creds '{"clientId":"opentdf","clientSecret":"secret"}' \ + --attribute-value-id \ + --action read \ + --subject-condition-set-new '[{ + "condition_groups": [{ + "conditions": [{ + "subject_external_selector_value": ".preferred_username", + "operator": "SUBJECT_MAPPING_OPERATOR_ENUM_IN", + "subject_external_values": ["alice"] + }], + "boolean_operator": "CONDITION_BOOLEAN_TYPE_ENUM_AND" + }] + }]' +``` + +:::warning Duplicate subject mappings poison evaluation +If multiple mappings exist for the same attribute value and one has a non-matching condition, it can block other mappings from granting access. Always delete old mappings before creating replacements. See [issue #3190](https://github.com/opentdf/platform/issues/3190). +::: + +### Validate with SDK discovery + +Before encrypting, use the SDK's discovery helpers to verify your policy is configured: + +```go +// Check attribute exists +exists, err := sdk.AttributeValueExists(ctx, "https://clinic.example/attr/staff_role/value/doctor") + +// Validate all FQNs before encrypting +err := sdk.ValidateAttributes(ctx, fqns...) +``` + +Note: `AttributeValueExists` currently returns an error (not `false`) for non-existent values. See [issue #3191](https://github.com/opentdf/platform/issues/3191). + +## Step 5: Integrate the SDK + +### Go SDK (Backend — Encrypt Only) + +See the [demo app's TDF client](https://github.com/opentdf/demo-glp1-tracker/blob/main/backend/tdf/client.go) for a complete encrypt wrapper. + +```go +client, err := sdk.New( + "http://localhost:8080", // Must include http:// scheme + sdk.WithClientCredentials("opentdf-sdk", "secret", nil), + sdk.WithTokenEndpoint("http://localhost:8888/auth/realms/opentdf/protocol/openid-connect/token"), + sdk.WithInsecurePlaintextConn(), // Remove in production +) + +// Encrypt +var buf bytes.Buffer +_, err = client.CreateTDF(&buf, bytes.NewReader(plaintext), + sdk.WithDataAttributes("https://clinic.example/attr/staff_role/value/doctor"), + sdk.WithAutoconfigure(false), + sdk.WithKasInformation(sdk.KASInfo{URL: "http://localhost:8080"}), +) +``` + +### JavaScript/TypeScript (Browser SDK — Decrypt) + +For true per-user access enforcement, the **browser** decrypts using the user's own Keycloak token. The backend should only encrypt. See the demo app's [auth setup](https://github.com/opentdf/demo-glp1-tracker/blob/main/frontend/src/lib/auth.ts) and [TDF decrypt helper](https://github.com/opentdf/demo-glp1-tracker/blob/main/frontend/src/lib/tdf.ts). + +```typescript +import { AuthProviders, OpenTDF } from '@opentdf/sdk'; + +const authProvider = await AuthProviders.refreshAuthProvider({ + clientId: 'cli-client', // Must be a PUBLIC Keycloak client + exchange: 'refresh', + refreshToken: keycloak.refreshToken, + oidcOrigin: 'http://localhost:8888/auth/realms/opentdf', +}); + +const client = new OpenTDF({ + authProvider, + platformUrl: 'http://localhost:8080', +}); +await client.ready; + +// Decrypt +const decryptedStream = await client.read({ + source: { type: 'buffer', location: tdfBytes }, +}); +``` + +## Step 6: Test the Full Round-Trip + +```bash +# Encrypt +echo "secret data" | otdfctl encrypt \ + --host http://localhost:8080 \ + --with-client-creds '{"clientId":"opentdf","clientSecret":"secret"}' \ + --attr "https://clinic.example/attr/staff_role/value/doctor" \ + -o test.tdf + +# Decrypt as a user with the right entitlement +otdfctl decrypt \ + --host http://localhost:8080 \ + --with-access-token "$USER_TOKEN" \ + test.tdf +``` + +If decrypt fails with "could not perform access", check all four links in the authorization chain (Step 4). + +## Troubleshooting + +| Error | Cause | Fix | +|-------|-------|-----| +| Platform crashes: "failed to discover idp" | OIDC realm doesn't exist | Run `provision keycloak` BEFORE starting platform | +| SDK: "platform endpoint is malformed" | Missing `http://` scheme | Use `http://localhost:8080`, not `localhost:8080` | +| SDK: "unauthenticated" | Token endpoint unreachable | Use `WithTokenEndpoint()` to override Docker-internal URL | +| Container: "Config File not found" | Wrong mount path | Mount to `/home/nonroot/.opentdf/opentdf.yaml`, NOT `/app/` | +| Container: "invalid logger output: stderr" | Unsupported value | Use `output: stdout` | +| Browser SDK: "ListKeyAccessServers 403" | User missing role | Add `opentdf-standard` role to user | +| Encrypt works, decrypt 403 | Missing link in auth chain | Check: attribute exists, KAS registered + key assigned, subject mapping exists | +| `.sub` selector doesn't match | Known platform issue | Use `.preferred_username` instead | +| Duplicate mappings break access | Evaluation poisoning | Delete old mappings, create fresh | diff --git a/docs/guides/production-deployment-guide/production-deployment.mdx b/docs/guides/production-deployment-guide/production-deployment.mdx new file mode 100644 index 00000000..739e245c --- /dev/null +++ b/docs/guides/production-deployment-guide/production-deployment.mdx @@ -0,0 +1,325 @@ +--- +title: "Part 2: Production Deployment" +sidebar_label: "Production Deployment" +sidebar_position: 2 +--- + +# Part 2: Production Deployment + +Everything from [Part 1](./local-development) applies. This section covers what **changes** for production. + +## Key Differences from Local + +| Concern | Local Dev | Production | +|---------|-----------|------------| +| Database | Docker PostgreSQL, `changeme` | Managed PostgreSQL with SSL (`sslmode: require`) | +| TLS | Disabled | CA-signed certificates (or terminate at load balancer) | +| DPoP | `enforceDPoP: false` | `enforceDPoP: true` (proof-of-possession on tokens). Requires SDK and IdP support — verify both your browser SDK and backend SDK send DPoP proofs before enabling. | +| Identity Provider | Local Keycloak, default passwords | Production OIDC with strong secrets | +| KAS Keys | Generated locally in repo | Secrets manager or env vars, never committed | +| CORS | `allowedorigins: ["http://localhost:5173"]` | Specific frontend domain | +| Logging | `level: info`, `type: text` | `level: info`, `type: json` | +| gRPC Reflection | Enabled | Disabled | +| SDK Connection | `WithInsecurePlaintextConn()` | Remove — use TLS | +| Credentials | Hardcoded `secret` | Environment variables | +| Logger output | `stdout` | `stdout` (only valid value in container) | +| Keycloak | `start-dev` mode | `start` production mode | + +## Solving Split-Horizon DNS + +The #1 architectural issue for containerized deployment. When the platform and Keycloak run in separate containers, they reach each other via internal hostnames (`keycloak:8888`) but external clients use public URLs (`https://keycloak.example.com`). JWT tokens have different `iss` claims depending on which URL was used. + +**Solution**: Set `KC_HOSTNAME_URL` in Keycloak to the public URL. This forces all tokens to use the same issuer regardless of how Keycloak is accessed: + +```bash +KC_HOSTNAME_URL=https://keycloak.example.com/auth +``` + +## Keycloak Production Setup + +### Provisioning against a remote Keycloak + +```bash +# Stop local Keycloak first — provisioning defaults to localhost:8888 +docker compose down + +# Use -e flag to target remote Keycloak, -p for admin password +go run ./service provision keycloak \ + -e https://keycloak.example.com/auth \ + -u admin \ + -p +``` + +:::warning Provisioning gotchas +- The admin token expires in **60 seconds** — provisioning handles this, but some clients may be created while others aren't. Verify all expected clients exist. +- The provisioned `opentdf` client's audience defaults to `http://localhost:8080`. The platform's `auth.audience` config must match this, or you need to add an audience mapper. +::: + +### Required Keycloak fixes for production + +**Disable VERIFY_PROFILE**: Keycloak 25's Verify Profile required action can block login when user profiles are incomplete. Disable it: +```bash +# Get admin token (replace and with your values) +TOKEN=$(curl -sf -X POST "/realms/master/protocol/openid-connect/token" \ + -d "grant_type=password" -d "client_id=admin-cli" \ + -d "username=admin" -d "password=" | python3 -c "import sys,json; print(json.load(sys.stdin)['access_token'])") + +curl -X PUT "/admin/realms/opentdf/authentication/required-actions/VERIFY_PROFILE" \ + -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{"alias":"VERIFY_PROFILE","enabled":false,"defaultAction":false,"priority":90}' +``` + +**Add audience mapper to `cli-client`**: Browser tokens lack the platform audience by default. +1. Keycloak admin → `opentdf` realm → Clients → `cli-client` → Client scopes +2. Click `cli-client-dedicated` → Add mapper → By configuration → **Audience** +3. Included Custom Audience: must match your platform's `auth.audience` config. If you used the default provisioning, this is `http://localhost:8080` (see the note in "Config in Production" for why and how to change it). +4. Add to access token: ON + +**Add `opentdf-standard` to default roles**: So every new user automatically gets SDK access. +1. Realm roles → `default-roles-opentdf` → Associated roles → Add `opentdf-standard` + +### Missing clients after provisioning + +The 60-second token can cause provisioning to exit early. Verify these exist: +- `opentdf` — admin client +- `opentdf-sdk` — SDK client with `opentdf-standard` role +- `tdf-entity-resolution` — ERS client with `view-users`, `query-users` client roles +- `cli-client` — public client for browser auth + +## KAS Keys in Production + +Never commit private keys to a repository. Options: + +1. **Bake into Docker image** (simplest for demos): Copy keys into a custom Dockerfile +2. **Environment variables**: Base64-encode PEM files, decode at startup +3. **Secrets manager**: AWS Secrets Manager, GCP Secret Manager, HashiCorp Vault +4. **Volume mounts**: If your platform supports file mounts + +Generate keys: +```bash +openssl genpkey -algorithm RSA -out kas-private.pem -pkeyopt rsa_keygen_bits:2048 +openssl rsa -in kas-private.pem -pubout -out kas-cert.pem +openssl ecparam -name prime256v1 -out ecparams.tmp +openssl req -x509 -nodes -newkey ec:ecparams.tmp -subj "/CN=KAS" \ + -keyout kas-ec-private.pem -out kas-ec-cert.pem -days 365 +rm ecparams.tmp +``` + +## Container Registry + +The official image at `registry.opentdf.io/platform:latest` may not be accessible from all cloud providers (Railway returns 403 despite the registry being public). Workaround: mirror to GHCR or Docker Hub: + +```bash +docker pull --platform linux/amd64 registry.opentdf.io/platform:latest +docker tag registry.opentdf.io/platform:latest ghcr.io//opentdf-platform:latest +docker push ghcr.io//opentdf-platform:latest +``` + +**Important**: Specify `--platform linux/amd64` when pulling on ARM Macs — otherwise the image won't run on x86 cloud hosts. + +## Config in Production + +The platform reads config from `/home/nonroot/.opentdf/opentdf.yaml` in the container. For platforms that don't support file mounts (like [Railway](https://railway.app)), bake the config into a custom Docker image. See the demo app's [Railway deployment files](https://github.com/opentdf/demo-glp1-tracker/tree/main/deploy/railway) for a complete working example. + +```dockerfile +FROM --platform=linux/amd64 ghcr.io//opentdf-platform:latest +COPY opentdf.yaml /home/nonroot/.opentdf/opentdf.yaml +COPY keys/ /keys/ +CMD ["start"] +``` + +Parameterize all deployment-specific values with environment variables, and harden the settings that Part 1 left in dev mode: + +
+Full production opentdf.yaml + +```yaml +logger: + level: info + type: json # Structured logs for production + output: stdout + +db: + host: "${OPENTDF_DB_HOST}" + port: 5432 + database: "${OPENTDF_DB_DATABASE}" + user: postgres + password: "${OPENTDF_DB_PASSWORD}" + sslmode: require # Always use SSL for managed databases + +services: + kas: + registered_kas_uri: "${OPENTDF_KAS_URI}" + keyring: + - kid: e1 + alg: ec:secp256r1 + - kid: e1 + alg: ec:secp256r1 + legacy: true # Used for trial decryption of old TDFs that lack a kid field + - kid: r1 + alg: rsa:2048 + - kid: r1 + alg: rsa:2048 + legacy: true + entityresolution: + mode: claims + url: "${OPENTDF_KEYCLOAK_URL}" + clientid: "tdf-entity-resolution" + clientsecret: "${OPENTDF_ERS_SECRET}" + realm: "opentdf" + legacykeycloak: true + inferid: + from: + email: true + username: true + +server: + tls: + enabled: true # Enable TLS (or terminate at load balancer) + auth: + enabled: true + enforceDPoP: false # Set to true if your SDKs and IdP support DPoP + audience: "http://localhost:8080" # Must match provisioned audience (see note below) + issuer: "${OPENTDF_KEYCLOAK_URL}/realms/opentdf" + policy: + extension: | + g, opentdf-admin, role:admin + g, opentdf-standard, role:standard + cors: + enabled: true + allowedorigins: + - "${OPENTDF_CORS_ORIGIN}" # Specific frontend domain, NOT "*" + allowedmethods: [GET, POST, PATCH, PUT, DELETE, OPTIONS] + allowedheaders: + - Accept + - Accept-Encoding + - Authorization + - Connect-Protocol-Version + - Content-Length + - Content-Type + - Dpop + - X-CSRF-Token + - X-Requested-With + - X-Rewrap-Additional-Context + exposedheaders: [Link] + allowcredentials: true + maxage: 3600 + grpc: + reflectionEnabled: false # Disable in production + cryptoProvider: + type: standard + standard: + keys: + - kid: r1 + alg: rsa:2048 + private: /keys/kas-private.pem + cert: /keys/kas-cert.pem + - kid: e1 + alg: ec:secp256r1 + private: /keys/kas-ec-private.pem + cert: /keys/kas-ec-cert.pem + port: 8080 +``` + +
+ +:::tip About `audience: "http://localhost:8080"` +This looks wrong for production, but it works because `provision keycloak` hardcodes this audience in the `opentdf` client. The audience is a token validation string — the platform checks that the JWT's `aud` claim matches this value. It doesn't connect to this URL. + +You *can* change it to your actual domain (e.g., `https://platform.mydomain.com`). Just make sure all three places agree: +1. The `opentdf` client's audience in Keycloak +2. The `cli-client` audience mapper's "Included Custom Audience" value +3. The platform's `auth.audience` config + +If you used the default provisioning and didn't change the Keycloak client, keep it as `http://localhost:8080`. +::: + +:::tip TLS termination +If your platform (Railway, AWS ALB, etc.) terminates TLS at the load balancer, set `tls.enabled: false` and let the proxy handle certificates. The platform will receive plain HTTP on its internal port. +::: + +**Note**: If both YAML values and `OPENTDF_` env vars are set, YAML wins. Use env var references in the YAML (`${VAR}`) or remove the YAML values entirely and rely solely on env vars. + +## SDK Changes for Production + +The Part 1 SDK examples use dev-only settings. Here's what to change: + +### Go SDK + +```go +client, err := sdk.New( + "https://platform.example.com", // Use HTTPS + sdk.WithClientCredentials("opentdf-sdk", os.Getenv("CLIENT_SECRET"), nil), + sdk.WithTokenEndpoint("https://keycloak.example.com/auth/realms/opentdf/protocol/openid-connect/token"), + // No WithInsecurePlaintextConn() — TLS is required +) +``` + +Remove `sdk.WithInsecurePlaintextConn()` — this disables TLS and must never be used in production. + +### JavaScript/TypeScript (Browser SDK) + +```typescript +const authProvider = await AuthProviders.refreshAuthProvider({ + clientId: 'cli-client', + exchange: 'refresh', + refreshToken: keycloak.refreshToken, + oidcOrigin: 'https://keycloak.example.com/auth/realms/opentdf', +}); + +const client = new OpenTDF({ + authProvider, + platformUrl: 'https://platform.example.com', +}); +``` + +All URLs must use `https://`. The browser SDK doesn't have an insecure mode flag — it just needs HTTPS URLs and a valid Keycloak token. + +## Troubleshooting + +All [local development troubleshooting](./local-development#troubleshooting) applies, plus: + +| Error | Cause | Fix | +|-------|-------|-----| +| "Account is not fully set up" | `VERIFY_PROFILE` enabled | Disable it (see above) | +| Token audience mismatch (401) | Browser token missing platform audience | Add audience mapper to `cli-client` | +| "Exec format error" in container | Wrong CPU architecture | Use `--platform=linux/amd64` in Dockerfile | +| `provision keycloak` provisions wrong instance | Local Keycloak still running | Stop local first, use `-e` flag | +| Admin password doesn't work | Changed env var after first boot | Reset in Keycloak admin console — env var only works on first boot | +| Container can't pull `registry.opentdf.io` | Cloud provider blocks custom registries | Mirror to GHCR/Docker Hub | +| YAML config ignored, uses defaults | Config not at expected path | Mount to `/home/nonroot/.opentdf/opentdf.yaml` | +| `AttributeValueExists` errors on new values | SDK bug | Treat error as "not found" — [issue #3191](https://github.com/opentdf/platform/issues/3191) | + +## Advanced: Per-Field Encryption for Granular Sharing + +Instead of encrypting an entire record with one attribute, encrypt each data category separately. This enables granular sharing — share weight without sharing symptoms. + +### Compound attribute values + +Encode both user identity and data category in a single attribute value: + +```text +Attribute: patient_access (rule: ANY_OF) +Values: _ + +Example FQNs: + https://clinic.example/attr/patient_access/value/356eeb1968c94364b4227defd515db94_weight + https://clinic.example/attr/patient_access/value/356eeb1968c94364b4227defd515db94_symptoms +``` + +This provides per-user isolation AND per-category granularity in a single attribute. + +### Server-side vs client-side decrypt + +For true per-user cryptographic enforcement, the **browser** must decrypt using the user's own Keycloak token. The backend should only encrypt — never decrypt. This ensures the KAS checks each user's entitlements individually. + +For more on this pattern, see the [Subject Mapping Guide](/guides/subject-mapping-guide) and the [GLP-1 Tracker demo](https://github.com/opentdf/demo-glp1-tracker). + +## Next Steps + +- **Monitoring**: Add `server.trace` config with an OTLP endpoint for OpenTelemetry tracing +- **Key Rotation**: Add new keys with different `kid` values, update KAS grants +- **Scaling**: Multiple platform instances behind a load balancer (stateless — state lives in PostgreSQL) +- **Backup**: Regularly back up PostgreSQL — it contains all your policy configuration +- **Reference**: Read [opentdf.io/llms.txt](https://opentdf.io/llms.txt) for comprehensive LLM-optimized reference documentation +- **Community**: Visit the [OpenTDF Discussion Forum](https://github.com/orgs/opentdf/discussions) for more examples and to ask questions diff --git a/static/img/filecontents.svg b/static/img/filecontents.svg deleted file mode 100644 index 6059a070..00000000 --- a/static/img/filecontents.svg +++ /dev/null @@ -1,13 +0,0 @@ - - - - my_document.ext.tdf (Zip) - - - - manifest.json - - - - 0.payload (Encrypted) - \ No newline at end of file