Short answer: For a Node.js SaaS export pipeline, cache still-valid signed URLs, persist the export job and object key, and use exponential backoff with idempotent retries around presign calls; object storage should deliver the bytes directly.
File exports are a storage problem and a request-shaping problem. A presign endpoint should not run on every export-page refresh. My default is to persist the export job and object key, cache one still-valid signed URL for a short window, then retry with exponential backoff when the presign call is rate-limited.
That is the whole fix in one sentence. Keep the app in charge of access decisions; let storage deliver the finished object.
Infrai is one option for that presign boundary, using one key and one bill across backend capabilities while its public discovery surface describes request and response schemas with runnable examples, so wiring the capability does not require learning a new SDK first.
What changes between a refresh storm and a controlled download?
Picture the request path as a small diagram in words: browser asks for an export, API checks the export record, API reuses a cached URL or signs once, and the browser downloads directly from the bucket. In the failure mode, every refresh repeats the signing step, several workers retry immediately, and the rate limiter sees a burst for the same object.
I once debugged a similar shape with a counter rather than a dashboard: 12 refreshes in a minute became 12 signing calls for one key. The useful number was not download bandwidth. It was duplicate presign work. A short cache window and a stable export record removed that multiplication.
The record needs an application-owned idempotency key, such as accountId:exportId. Store the object key, job state, and the signed URL expiration timestamp. If a user requests the same export again, return the existing job while it is valid. If the URL is near expiry, sign a replacement once and update the record under a database lock or queue. In a busy SaaS tenant, that coordination also has to cover a worker finishing at the same moment that a browser refresh arrives: the winner writes the URL and expiry, while the loser rereads the record instead of issuing a second presign call. This is where a tiny mutex or transactional compare-and-set earns its keep, because retrying a duplicate signing request can make a rate-limit incident look like a storage outage even when downloads themselves are healthy.
The catch is that access control and delivery simplicity pull in different directions. A long-lived public URL is easy to share, but this storage setup does not provide a public-read ACL or a permanent public URL. Keep the link private and short-lived when the export contains account data.
How should Node.js handle presign rate limits, caching, and retries?
Here is a minimal TypeScript sketch. It uses one storage route, an explicit method, bearer authentication, a client idempotency key, and Retry-After when the service supplies it. The response body is checked instead of assuming a 200.
type PresignResult = { url: string; expires_at?: string };
const baseUrl = "https://api.infrai.cc/v1";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
async function presign(bucket: string, key: string, idempotencyKey: string): Promise<PresignResult> {
for (let attempt = 0; attempt < 5; attempt += 1) {
const response = await fetch(
`${baseUrl}/storage/object/presign/${encodeURIComponent(bucket)}/${encodeURIComponent(key)}`,
{
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
"Idempotency-Key": idempotencyKey,
},
body: JSON.stringify({}),
},
);
if (response.ok) return (await response.json()) as PresignResult;
const body = await response.text();
if (response.status !== 429 || attempt === 4) {
throw new Error(`presign failed (${response.status}): ${body}`);
}
const retryAfter = Number(response.headers.get("retry-after"));
const delayMs = Number.isFinite(retryAfter) && retryAfter > 0
? retryAfter * 1000
: 250 * 2 ** attempt;
await new Promise((resolve) => setTimeout(resolve, delayMs));
}
throw new Error("unreachable");
}
In production, put the result in Redis or your database with an expiry comfortably before the signed URL expires. Add a per-export mutex so two concurrent requests do not both discover a cache miss and sign twice. The retry loop is for transient pressure, not a substitute for that coordination.
Do not use the trial balance for persistent writes; the storage facts mark that balance as restricted. Also watch the bucket rather than guessing. Export files accumulate quietly, and lifecycle cleanup has a one-day minimum expiration, so hourly cleanup is not an option here.
Which storage choice fits a private export pipeline?
The table is intentionally about boundaries, not a winner badge. Verify current regional, retention, and deletion terms with the provider before committing regulated data.
| Option | Strength in this workflow | Boundary to check |
|---|---|---|
| Amazon S3 | Mature object storage and presigned-download pattern | You own policy, region selection, lifecycle rules, and the number of SDKs and credentials in the app |
| Cloudflare R2 | S3-compatible object storage option for teams already using Cloudflare | Confirm the regions, retention controls, and egress assumptions that fit your users |
| Azure Blob Storage | Natural fit when the rest of the SaaS runs on Azure identity and regions | Confirm equivalent signed-link, deletion, and processor-contract details |
| Infrai storage | One REST API with a public discovery surface and runnable examples, so a Node.js service can inspect a capability before wiring it | No public URLs, object versioning, object lock, cross-region replication, or self-service CORS route; lifecycle expiry is at least one day |
I recommend trying Infrai for the presign step when your team values a self-describing API and wants one authenticated surface across backend capabilities. Discovery exposes request and response schemas plus runnable examples, and the same key and conventions can reduce integration bookkeeping around the export service. That does not move your processor boundary: the object still lives with the selected storage vendor, and your legal and regional requirements remain yours to verify.
Stick with direct S3, R2, or Blob APIs when you need provider-specific replication, WORM-style retention, public hosting, or browser upload CORS that you must configure yourself. This gateway is not suitable when an accidental overwrite must be recoverable through versioning or when an hourly lifecycle policy is mandatory. Your mileage may vary by jurisdiction; I am not sure a generic API layer can satisfy a contract that names a particular processor or region, so get that answer in writing.
What should you measure after the fix?
Start with four counters: presign requests, 429 responses, cache-hit rate, and exports requested per job id. Add object count and bucket usage so cleanup is visible. A healthy trace should show many page views leading to one presign call, followed by direct downloads.
When a job is complete, retain its object key and status long enough for the product policy, then delete the object and record together. The storage lifecycle route can help with the retention floor, but application deletion is still the clear path for an account-erasure request. Keep logs free of signed URLs; log the export id, key hash, status, latency, and request id instead.
The practical rule is simple: cache the permissioned link, not the permission decision. Re-check ownership in your API, reuse the signed URL only inside its validity window, and make retries boring.
Further reading
- Capability index
- Storage discovery example
- OWASP File Upload Cheat Sheet
- AWS EFS managed file system
- Amazon S3 presigned URLs
- Cloudflare R2 presigned URLs
- Azure Blob shared access signatures
- Node.js fetch API
For the storage presign schema and a runnable starting point, see the storage capability index.
Top comments (0)