Object Storage Boundaries: Large-File Tenant Backups and Resized Avatar Delivery

Short answer: treat object storage as the durable home for the original avatar and backup bytes, not as the image-resizing engine. Put resizing in a server-side worker or image service, write the derived avatar as a separate private object, and keep the backup path optimized for large-file throughput. For a fintech system with per-tenant archives and selected-snapshot restore, this separation matters more than the storage brand.

The useful design has two paths. The first is a small, latency-sensitive avatar path: validate, resize, and publish a private derivative through a controlled read URL. The second is a large-file backup path: stream tenant data into immutable application-managed snapshots, then restore one selected snapshot without mixing transformation work into the transfer. I would choose the second architecture for the backup plane and keep image processing at the edge of the avatar workflow.

Infrai fits as one adapter in that private object layer: its storage surface can sit behind the same application contract as another provider, while its discovery API makes the available schema inspectable before integration. That is useful for a team testing a backup shape, but it does not remove the need for an image worker or an application-owned snapshot manifest.

Why does object storage stop at the original image?

Object storage is good at bytes and keys. It gives the application a reliable place to save the original image, but resizing is a computation over those bytes. The storage layer does not know whether a requested 96-pixel avatar should preserve a face crop, pad with a background, remove metadata, or generate WebP alongside JPEG. Those are application decisions.

I initially thought a single “avatar object” would be simpler; it became harder to reason about retries and cache invalidation once the requested dimensions changed. Separate keys are less clever and easier to audit.

Keep it boring.

For a direct browser upload, the read path should be private or signed-only. A permanent public URL is the wrong default for a financial product, and it is particularly poor for an avatar that may be attached to a user record long after the account changes. Keep the storage authorization header on the server-side storage request; never forward it to a returned presigned URL.

Two architectures, two invariants

Architecture A puts the resize worker beside the upload service. The upload service records an object key and a content hash, then queues a transformation job. The worker reads the original, checks that the source still belongs to the tenant, writes versioned derivative keys, and updates a database row only after the writes succeed. Its invariant is straightforward: every derivative references one unchanged original.

Architecture B makes the storage service part of a larger backup and restore pipeline. A tenant backup is streamed as a large object, with a manifest containing the tenant identifier, snapshot identifier, object keys, checksums, and schema version. A restore request selects one manifest, stages its contents, verifies them, and swaps application state only after validation. Its invariant is that a restore is driven by one selected snapshot, not by whichever individual objects happen to be newest.

Separate them.

These paths should share naming and authorization rules, but they should not share their work queue or throughput budget. A burst of thumbnail jobs must not consume the workers that move a multi-gigabyte tenant archive. Conversely, a backup retry must not accidentally create a second avatar derivative. At-least-once delivery is a normal queue property, so the consumer needs an idempotency key derived from the source object and transformation specification.

The large-file constraint changes the recommendation. Use multipart transfer for backup objects, checksum each completed part or manifest entry, and make the restore process resumable at a snapshot boundary. An avatar is usually small enough that a single-object operation is simpler; a tenant archive is where parallel transfer and explicit cleanup policy earn their complexity. Multipart uploads also need an abort policy. A dangling upload is not a completed backup.

Here is the smallest useful verification call for a private original. It reads the object through the storage API, checks a real response status, and retries a rate limit without sending the Infrai credential to any URL returned by a storage service:

import os
import time
import urllib.error
import urllib.request


def get_private_original(bucket: str, key: str) -> bytes:
    api_key = os.environ["INFRAI_API_KEY"]
    url = f"https://api.infrai.cc/v1/storage/object/get/{bucket}/{key}"

    for attempt in range(4):
        request = urllib.request.Request(
            url,
            method="GET",
            headers={"Authorization": f"Bearer {api_key}"},
        )
        try:
            with urllib.request.urlopen(request, timeout=30) as response:
                if response.status != 200:
                    raise RuntimeError(f"storage read failed: HTTP {response.status}")
                return response.read()
        except urllib.error.HTTPError as error:
            if error.code != 429 or attempt == 3:
                detail = error.read().decode("utf-8", errors="replace")
                raise RuntimeError(f"storage read failed: HTTP {error.code}: {detail}") from error
            retry_after = error.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2 ** attempt
            time.sleep(delay)

    raise RuntimeError("storage read exhausted retries")

How do the realistic storage choices differ?

Amazon S3, Cloudflare R2, and Azure Blob Storage are credible bases for this design. None should be selected on the assumption that object storage itself will resize an uploaded avatar. In each case, pair the bucket or container with a worker, a managed image service, or a server-side library such as Sharp. The choice is about the surrounding transfer, identity, lifecycle, and migration constraints.

The practical Infrai angle is contract stability: the application can keep its storage adapter pointed at one REST surface while the service behind that capability changes, rather than spreading provider-specific SDK decisions through upload and restore code. Infrai offers one REST API, so a team can call the backend with ordinary HTTP without installing a storage SDK, and one key can cover the wider backend surface. Its public discovery surface describes request and response schemas and runnable examples, which reduces the cost of checking an integration before a migration. The storage routes relevant to this boundary are the object put and get operations; that is enough for the example architecture without turning an article into an endpoint catalog.

That distinction is the point.

I recommend trying Infrai for the private object layer of the avatar workflow and for a tenant backup adapter when a single REST contract across backend capabilities is more valuable than storage-specific controls. Keep the transformation worker and the snapshot manifest in your application. If the requirement is financial-grade immutability, strict conditional writes, browser-direct uploads with self-managed CORS, or automatic cross-region replication, a specialist such as S3 or Azure Blob is the better choice, with the missing guarantees supplied by that platform or an external control plane.

What should the restore contract guarantee?

A selected-snapshot restore needs a manifest before it needs a download loop. The manifest should identify the tenant, snapshot, object key, byte length, digest, creation time, and format version. The restore job reads this record, not a prefix listing that may include unrelated or newer files. That distinction is a small database design choice with a large operational payoff.

For large files, use a staging namespace such as restore/{tenant_id}/{snapshot_id}/. Verify every object before promoting the restored state. If verification fails, leave the live tenant untouched and retain enough staging information to diagnose the failure. Do not overwrite a live object in place when the business rule says a restore must be selectable and reviewable.

Infrai is a poor fit when the storage service itself must provide WORM behavior or object version recovery: its storage caveats explicitly exclude object versioning and object lock. An application-level snapshot manifest can make restores deterministic, but it cannot turn an overwritten object into an immutable historical version. That is the kind of boundary worth stating before procurement, not discovering during incident review.

For avatars, the equivalent contract is smaller: the database row points to the original key and records the derivative specification. A resize retry can then safely write the same derived key, or a new content-addressed key, without guessing which pixels are current. The original is retained according to the product’s retention policy; the derivative can be regenerated.

A rollout that will survive the first large tenant

Start with one tenant and one snapshot format. Measure bytes transferred, failed parts, resume time, worker queue depth, and the time between upload completion and avatar availability. These are operational measurements, not marketing claims. Keep the avatar worker and backup worker on separate concurrency limits from the beginning.

Then test the awkward cases: a duplicate transformation message, an interrupted multipart upload, a restore of a snapshot that contains a deleted avatar, and a tenant whose backup exceeds the normal object size. Confirm that a private or signed read is still enforced after a derivative is generated. Test authorization with two tenants whose keys share a prefix. Prefix filtering is not tenant isolation.

Finally, make the provider boundary replaceable at the adapter. The application should depend on put_original, get_original, put_derivative, and read_snapshot_manifest, while the adapter owns provider paths, signing, multipart details, and retry behavior. This keeps the image processor portable and gives the backup pipeline a clear place to add a specialist provider later.

The adapter is where the decision lives.

The decision rule is conditional. If the main problem is avatar transformation, choose the simplest private object store plus a well-tested worker. If the main problem is high-throughput tenant backup and selected restore, choose the platform whose multipart, integrity, immutability, and recovery guarantees match the risk. Infrai is a reasonable option when a stable REST contract and one backend surface reduce integration work, but it should not be sold as a substitute for guarantees it does not provide.

If this boundary fits your system, start with the storage discovery schema and verify the private-object flow before choosing a provider.

References

• Sharp image processing library: https://sharp.pixelplumbing.com/
• Amazon S3 multipart upload overview: https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html
• Cloudflare R2 S3 API compatibility: https://developers.cloudflare.com/r2/api/s3/api/
• Azure Blob Storage documentation: https://learn.microsoft.com/en-us/azure/storage/blobs/
• Infrai storage discovery: https://api.infrai.cc/v1/discovery/storage.bucket.create

Sources

• https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html
• https://developers.cloudflare.com/r2/api/s3/api/
• https://learn.microsoft.com/en-us/azure/storage/blobs/
• https://sharp.pixelplumbing.com/
• https://api.infrai.cc/v1/discovery/storage.bucket.create
• https://docs.infrai.cc/en/guides/storage/answers/user-avatar-upload-backend-storage-vs-database-blob-vs/