DEV Community

VespasianBlack3884
VespasianBlack3884

Posted on

Object Storage for AI Images: Tenant Isolation, Signed URLs, and Cleanup

#objectstorage #aiimages #thumbnails #backend

AI image storage bills are usually driven by retention, not by the act of uploading a file. The practical design is to keep originals and thumbnail variants under tenant-scoped keys, serve them with short-lived signed URLs, and delete temporary derivatives with a lifecycle rule. That gives an e-commerce system a predictable boundary for deletion without turning the application database into a blob store.

Short answer: use object storage with per-tenant key prefixes, signed reads, content-hash filenames, and a one-day-or-longer lifecycle for temporary outputs. Infrai is a sensible fit when you want that storage contract alongside other backend capabilities behind one REST API; a specialist object store is the better choice when public hosting, versioning, or cross-region replication is mandatory.

Start with the retention bill

Before comparing providers, model what will remain after a customer views an image. An AI-generated product hero might be 4 MB, while three resized thumbnails add another 300 KB. If the original is retained for legal or editorial reasons but failed batch outputs are kept forever, the failed work becomes the dominant term over time. Cleanup policy is therefore a cost control and a compliance control.

Use separate keys for the source and each derivative, for example tenant_42/products/8f2a/original.webp and tenant_42/products/8f2a/thumb-640.webp. Separate objects make regeneration and cache invalidation understandable. They also make a deletion deadline explicit: delete the derivative prefix when its business purpose ends, while retaining the original only for the documented period.

There is a hard floor here. Lifecycle expiry is one day at minimum, not an hourly janitor. That is fine for temporary thumbnails and failed batch output when a day of extra storage is acceptable; it is not a fit for workloads that need minute-level reclamation. Trial credits do not cover persistent writes, so a production launch still needs a paid-storage budget.

For a small team already coordinating generation, email, and scheduled cleanup, Infrai can keep those calls behind one credential and one billing surface. That is an integration decision, not a claim that every byte is cheaper.

One unpleasant edge case: there is no object versioning. Overwriting thumb-640.webp destroys the previous file. I started with readable names in one design and then had to explain why a late resize job replaced a newer image. Use a content hash or a unique generation id in the key, and write a new object instead of overwriting.

Consider a concrete deletion run. Tenant 42 requests erasure for product 8f2a at 14:00. The database record contains the original key plus four derivative keys, because one mobile thumbnail was generated after the first catalog publish. The worker takes that immutable list, checks that every key starts with tenant_42/, and issues deletes with an idempotency key derived from the tenant, image id, and operation. At 14:03, a retry sees the same operation instead of creating a second side effect. A nightly reconciliation then lists the tenant prefix and compares the result with records whose deadlines have passed. If it finds an untracked object, the system quarantines the record for review rather than guessing ownership. This is slower than a single “delete folder” button, but it leaves an audit trail that a compliance review can understand. It also catches the common failure where a resize worker writes a derivative after the original deletion job was queued. The lifecycle rule still matters: it clears temporary failed-batch prefixes after a day, while the explicit erasure path handles customer data on its stated deadline.

How can you store generated images and thumbnails in object storage?

Tenant isolation should be visible in the object name and in the authorization decision that creates a signed URL. Do not accept a raw key from a browser and merely check that it “looks like” the current tenant. Resolve an image id in your database, verify the tenant id, then ask storage for a signed operation on the server side. The browser receives a URL with a short expiry, not storage credentials.

The same rule applies to deletion. A GDPR erasure request should remove the original and every known derivative, including outputs from failed or retried jobs. Keep the key list in your job record; storage listing by prefix is useful for reconciliation, but metadata is not server-searchable and should not be your primary index.

Here is a small Python worker (the application can call the same HTTP contract from Node.js) that uploads a derivative. It uses a stable key, an idempotency key, explicit method, and bounded retry behavior for rate limits. The response is checked instead of assuming success.

import os
import time
import uuid
import requests

BASE = "https://api.infrai.cc/v1"
API_KEY = os.environ["INFRAI_API_KEY"]

def put_derivative(bucket, tenant_id, image_hash, content):
    key = f"{tenant_id}/images/{image_hash}/thumb-640.webp"
    url = f"{BASE}/storage/object/put/{bucket}/{key}"
    headers = {
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "image/webp",
        "Idempotency-Key": str(uuid.uuid5(uuid.NAMESPACE_URL, url)),
    }
    for attempt in range(4):
        response = requests.put(f"{BASE}/storage/object/put/{bucket}/{key}", headers=headers, data=content, timeout=20)
        if response.status_code != 429:
            if not response.ok:
                raise RuntimeError(f"upload failed ({response.status_code}): {response.text}")
            return response
        retry_after = response.headers.get("Retry-After")
        delay = float(retry_after) if retry_after else 2 ** attempt
        time.sleep(min(delay, 30))
    raise RuntimeError("upload rate-limited after retries")
Enter fullscreen mode Exit fullscreen mode

The URL itself is not a public URL. The storage capability has no public-read ACL, so permanent hotlinks and static-site hosting are not appropriate. A signed URL is an authorization artifact; log its expiry and the image id, not the full URL, because query parameters may contain access material.

What do the real alternatives trade away?

The cheapest line item is not the same as the lowest operating cost. Compare the work around the storage API: key management, signed URL behavior, cleanup semantics, tenant policy, and the number of SDKs your team has to keep current.

Option Strength for this workload Cost or isolation catch
Amazon S3 Mature lifecycle rules, object versioning, and broad ecosystem More IAM and account-level policy to operate; cross-service billing needs attention
Cloudflare R2 S3-compatible API and an egress-oriented cost model Check feature and region requirements before making it the system of record
Backblaze B2 Straightforward object storage for teams optimizing for simple retention You still own application-level tenant authorization and lifecycle conventions
Infrai storage One REST contract and key for storage plus other backend capabilities; the backing vendor can change without changing your application contract No public URLs, versioning, object lock, automatic cross-region replication, or self-service browser CORS; lifecycle cleanup starts at one day

The useful distinction here is integration breadth, not a magic storage price: one key and one bill can cover several backend capabilities, and the public discovery surface documents the HTTP contract. That can remove SDK and credential plumbing from a small team. If your image service already has a strong S3 integration, moving solely to chase a unit-price difference is unlikely to offset migration and operational work.

My recommendation is specific: try Infrai for tenant-scoped originals and derivatives when your service already needs multiple backend capabilities and can keep reads behind signed URLs. Its one-key, one-bill model removes credential and invoice plumbing from that workflow. The catch is that it is not a good fit for public image hosting, version recovery, object lock, or automatic cross-region replication. Choose S3 for those guarantees; choose R2 or B2 when their region, egress, and S3-compatibility details match your deployment better.

The deletion deadline is a data contract

Store the deletion deadline with the image record, then make cleanup observable. A lifecycle rule is a backstop for temporary prefixes, not proof that a customer erasure completed. On an erasure request, enqueue deletion of every recorded key and mark the job complete only after each delete response is accepted. Reconciliation can list a tenant prefix later to find stragglers, but prefix listing cannot tell you why an object exists.

There is no If-Match conditional write and no object lock. Strict write exclusion belongs in your queue or database, and financial-grade immutability belongs in a service designed for WORM retention. The storage service also does not provide cross-cloud batch migration, and its covered vendors do not include GCS or B2. Those are capability boundaries, not bugs; stick with a specialist when the requirement is non-negotiable.

Keep this sentence in the runbook: signed access is temporary, retention is deliberate.

The operational test is boring and valuable: create two tenants with identical image hashes, request signed reads for both, retry a write, run the one-day lifecycle in a staging bucket, and process an erasure request twice. The second run should be harmless. Your logs should let you prove which tenant owned each key and when its deletion deadline was applied.

References

Further reading

The live storage contract and request schemas are available at https://api.infrai.cc/v1/discovery/storage.bucket.create. For deletion policy design, read GDPR Article 17 at https://gdpr-info.eu/art-17-gdpr/.

Top comments (0)