feat(s3): add a blob-backed S3-compatible API

[pthmas]

Summary

• Add SQLite-backed S3-compatible API for bucket and object CRUD
• Celestia as verification layer, not storage: PutObject submits a small JSON CommitmentEnvelope (~200 bytes) containing SHA256 + object metadata to Celestia, not raw data
• SQLite caches raw object data; GetObject serves from the local cache; users can prove data authenticity by hashing a downloaded object and comparing against the on-chain SHA256 record
• All write operations (CreateBucket, DeleteBucket, PutObject, DeleteObject) are blocked with 405 Method Not Allowed when no Celestia submitter is configured (read-only mode)
• SigV4 AWS authentication with ±15 minute timestamp skew enforcement; optional — server warns at startup if unconfigured
• parsePath uses r.URL.RawPath to avoid double URL-decoding of %2F in object keys
• DeleteBucket wrapped in a transaction to prevent TOCTOU race between empty-check and delete
• maxObjectSize limit (2 MB) is independent of Celestia's blob size since only the envelope is submitted
• Migrations 004 and 005 squashed into a single clean migration with the final schema (includes sha256 column from the start, no AUTOINCREMENT overhead, single composite index)

Architecture

PUT /bucket/key
  → read body (max 2 MB)
  → compute SHA256 + MD5 (ETag)
  → marshal CommitmentEnvelope JSON
  → submit envelope to Celestia → get height + commitment
  → store raw data + metadata in SQLite

GET /bucket/key
  → serve raw data from SQLite cache
  → client can verify: sha256sum(body) == obj.SHA256 (from Celestia)

CommitmentEnvelope (submitted to Celestia)

{
  "version": 1,
  "bucket": "my-bucket",
  "key": "path/to/file.txt",
  "content_type": "text/plain",
  "size": 1234,
  "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "etag": "d41d8cd98f00b204e9800998ecf8427e"
}

Test plan

• just test passes with -race
• PUT object → verify obj.SHA256 non-empty → sha256sum of downloaded body matches
• PUT with no submitter → 405 Method Not Allowed
• PUT object > 2 MB → 413 Request Entity Too Large
• PUT object with key > 1024 bytes → 400 Bad Request
• Bucket name validation (3-63 chars, lowercase alphanum+hyphen, no IP)
• TestService_PutObject_WithSubmitter verifies submitted blob is a CommitmentEnvelope, not raw data
• TestObjectStore_ObjectCRUD verifies SHA256 round-trips through SQLite

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 6de85162-c09e-4def-bddc-8d4c5b69fd12

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch pthmas/s3-blob-export

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}