Shadow Mode

Test on live data
without the risk.

Deploy a shadow flow that taps into your production pipeline. It processes real events with your new logic but doesn't write to your sinks. Compare outputs, validate fixes, then promote to production — zero downtime, zero risk.

Production Flow
Steam Reviews
v1
Mapper v1
ClickHouse Live data
Shadow Flow
Steam Reviews tap
v2
Mapper v2 your new logic
DryRun no write, just compare
Resilience

It handles failures
so you don't have to.

Built-in circuit breaker, global rate limiting, and dead letter queue. When Steam's API rate-limits you, Tsunagi backs off automatically. When ClickHouse is at 95% disk, it throttles ingestion. Failed events go to the Error Hub — fix, replay, done.

Feature What it does
Auto-pause on failure
Auto-stop a flow if error rate exceeds threshold, retry after cooldown
Governor
Global rate limiting shared across workers — no API bans
Failed event queue (DLQ)
Quarantine failed events with full payload and context
Replay
Re-process any failed event after fixing the pipeline
Bulk Start/Stop
Start or stop 50 workers in one click
Smart Alerting

Monitor data schema changes
in your pipelines.

The schema_monitor processor detects when fields are added or removed from your data, helping you track format stability and governance.

steam-pipeline.yaml
processors:
- type: schema_monitor
config:
blueprint_id: "steam-reviews"
mode: "alert"
sinks:
- type: when
config:
field: "metadata.schema_change.detected"
equals: "true"
then:
- type: webhook
config:
url: "https://hooks.slack.com/..."

Hash-Based Detection

Calculates hash of payload keys, compares with stored schema to detect added or removed fields.

Alert vs Strict Mode

Alert mode logs changes and continues processing. Strict mode blocks events when schema changes detected.

Conditional Alerting

Use conditional sinks to send alerts only when schema changes detected. Slack, Discord, or custom webhooks.

Schema Governance

Track data format stability over time. Store schema history in database for governance and compliance.

Monitor, alert, govern

Place the schema_monitor processor after sources or before sinks to track data format changes and maintain governance.

LLM Enrichment

AI directly in your pipeline.
One line of YAML.

Enrich events with AI directly in your pipeline. Sentiment, translation, summarization, classification — one line of YAML.

processor.yaml
- type: llm
config:
provider: anthropic
model: claude-3-haiku
prompt: "Sentiment of this review: {{.review}}"
target: ai_sentiment
daily_budget_usd: 5.0

How it works

1
Your data
2
Tsunagi processor
3
AI Service (FastAPI + LiteLLM)
4
Anthropic / OpenAI / Ollama
Enriched event

Built-in controls

Providers

Anthropic, OpenAI, Ollama (self-hosted)

Template engine

{{.field}} injects event data into prompts

Budget enforcement

Circuit breaker if daily_budget_usd exceeded

Cost tracking

Per-flow, per-event token usage and USD cost

Fallback

Default value if LLM fails

Retry + backoff

Built into the Go processor

Connectors

Gaming-native connectors.
Not generic social media scrapers.

Live streaming
Polling
History / Backfill (historical data)
Media
Messaging

Steam

Reviews Community

Discord

Bot History

Twitch

Chat Audio API

Reddit

Live History

YouTube

API Audio

Google Play

Scraper

App Store

Reviews

X

Live

Bluesky

Live

RSS

HTTP Poller

SQL Source

Backfill (historical)

S3 Source

Backfill (historical)

Kafka

Redpanda

Webhook

Receiver
+5

More coming

21 connectors available. New ones ship regularly.
Studio Desktop

A desktop that understands
your data pipelines.

Multi-instance. Native. Dark mode by default. Built for the team that ships at 2am.

Tsunagi Studio
Prod
Staging
Dev
Flows
steam_reviews
discord_messages
reddit_posts
twitch_chat
Tools
Dashboard
Genealogy
Data Forge
Error Hub
Run Logs
Liveboards
Templates
Playground
Events/min
12.4K
+8.2%
Active flows
8
of 12 configured
Success rate
99.7%
Last 24h
Failed events
23
Needs review
Data Forge
SELECT id, content, ai_sentiment, ai_summary FROM steam_reviews WHERE app_id = '730' LIMIT 5
id content ai_sentiment ai_summary timestamp
sr_001 Best update yet, love the new maps positive Player praises new maps 2m ago
sr_002 Servers are lagging after patch negative Reports performance issues 5m ago
sr_003 How do I unlock ranked mode? neutral Asks about ranked unlock 12m ago

Multi-instance

Prod, Staging, Dev — one click to switch

Live Dashboard

TPS, active flows, success rates in real-time

Event Genealogy

Trace any event through the entire pipeline

Error Hub

Click, replay, resolve — without leaving the app

Liveboards

Build dashboards with KPI cards, data tables, and ECharts visualizations. Drag, resize, save.

Templates

Start from 20+ pre-built blueprints. Source-only templates for the Data Forge.

Playground

Test your processors against mock JSON. See the output instantly. No deploy needed.

Liveboards

Analytics built into
your pipeline.

No BI tool needed. Query your live data directly where it flows — DuckDB handles the rest.

The old way

1 Call the API
2 Build a data pipeline
3 Load into DuckDB
4 Write SQL

Steampipe, Powerpipe, and friends.

query.sql
-- Live sentiment over the last 7 days
SELECT *
FROM tsunagi_live.gaming_sentiment
WHERE days = 7
-- That's it. No complex data processing. No external BI.

Live data

Query events as they flow through the pipeline

DuckDB-native

Embedded analytics engine, no external database

Zero setup

Works out of the box with your existing flows

Data Forge

Write a SQL query,
Tsunagi generates the pipeline.

Run it in sandbox, promote to production in one click.

SQL-to-Pipeline

Write your query, get a complete data pipeline

Sandbox Testing

Test with live data before promoting

One-Click Deploy

Promote to production instantly

data-forge.sql
-- Define what you want
SELECT
author, content, sentiment
FROM
steam.reviews
WHERE
app_id = '730'
AND
created_at > NOW() - INTERVAL '1 day'
-- Tsunagi generates the pipeline
✓ Pipeline created
✓ Running in sandbox
_