Object storage export

1. The gap this fills

Until now there was exactly one way from MousePlayer into a data warehouse, and it led to Google. If you run Snowflake, Redshift or ClickHouse, there was none - even though all three read from object storage without complaint.

So the bridge writes to object storage rather than to each warehouse in turn. Snowflake ingests from S3, Redshift ingests from S3, ClickHouse ingests from S3, and BigQuery ingests from Google Cloud Storage. One export reaches all of them.

Where	How it is addressed
Amazon S3	Signature Version 4, bucket in the host
Cloudflare R2	Signature Version 4 against your account endpoint
Google Cloud Storage	through its S3-compatible API, with an HMAC key
MinIO, Wasabi, Backblaze B2 and others	Signature Version 4 against your endpoint
Azure Blob Storage	a container SAS link - no signing on our side at all

2. Why Azure is the odd one, and why that is better

Azure Blob does not speak Signature Version 4; it signs with SharedKey. We could have built a second signing scheme. Instead the bridge takes a container SAS link - a URL you generate in your own storage account, with the signature already inside it.

That is less code on our side and less trust on yours. We never see an account key at all: what you hand over is a permission you scoped yourself, to one container, with an expiry date you chose, revocable at any time without touching anything else.

3. One object per goal, and the key is a date

Object storage has no append. A daily file would have to be rewritten whole every time, and two goals arriving at once would overwrite each other. So each fulfilled goal becomes its own object.

The price is many small objects; the key layout is what makes that workable:

prefix/YYYY/MM/DD/channel/session-goal.json

A warehouse reads one day as one partition, and a backfill is a prefix scan rather than a query. If you want to partition by project instead of by date, put the channel ID into the prefix.

Each object is one line of JSON

Deliberately not indented, so a whole prefix reads as NDJSON without opening the objects one at a time. Timestamp, project, session, customer ID, goal, value, currency and the link that plays the recording back.

An empty value is null, not zero

A goal without a monetary value writes null. In a warehouse a zero would sum into a total that never existed - the distinction between "no amount agreed" and "an amount of zero" survives the export.

4. If it answers 403, read this first

A rejected S3 request says almost nothing, and there is a reason the error message in our log says more than S3 does: the region goes into the signature. A wrong region and a wrong key produce the same 403, so the obvious suspicion - the key - is right only half the time.

Three things to check, in this order:

• The region, exactly as the bucket reports it. R2 and Google Cloud Storage take auto and do not check it; everyone else does.
• The prefix permission. A key scoped to mouseplayer/ cannot write to mouseplayer-goals/ - the trailing slash matters in a resource ARN.
• The endpoint, for R2 and S3-compatible stores. Paste the host without https://; we strip it either way, but a path appended to it is a different bucket.