Short answer: keep each user reminder as a durable record, use a periodic scan or delayed-message mechanism only to find work, and make email, SMS, and public webhook processing idempotent.
The least complex reliable design has a database as the schedule of record. A scheduler identifies reminders that are due, a queue carries small delivery jobs, and a worker writes a stable delivery key before calling a channel provider. That shape survives restarts, repeated messages, and a user who chose a local clock time instead of a UTC offset.
The timer is not the contract.
An in-process timer is tempting because the first version is tiny. It also disappears on deploy, has no durable cancellation record, and becomes ambiguous when more than one process is running. The same kind of ambiguity appears after a queue consumer has received a message but loses its acknowledgement: the broker is allowed to redeliver. Treating each component as at-least-once is less glamorous than promising exactly-once delivery, but it gives the operations team a state transition to inspect when a user asks where a reminder went.
How should reminder emails, SMS, time zones, and public webhook endpoints work?
Start with one record for one intended delivery. It needs a reminder ID, channel, due_at in UTC, the local wall-clock time selected by the user, an IANA time-zone name, a state, and an idempotency key. The key describes the intended message rather than the attempt: reminder:8842:sms:v1 remains stable while attempts change.
Then make every boundary repeatable. A scheduler can run late or overlap another scheduler. A queue can redeliver. A delivery provider can accept a request before the worker learns the result. A public webhook sender can retry when it does not receive a timely success response. None of these facts is exceptional; they are the normal operating conditions of distributed delivery.
The useful invariant is modest: repeated work must converge on the same recorded delivery, rather than create another send.
The incident path to design for is a partial handoff
Consider a bounded scan that claims 500 due rows. It publishes 499 jobs and stops before the last handoff is durably recorded. A later scan must be able to reclaim the unfinished item after its lease expires. At the same time, one of the 499 messages may be delivered to a worker twice after acknowledgement is interrupted. If the system treats queue publication as the only record of intent, the final row can vanish from the schedule. If it treats each attempt as a fresh delivery, the duplicated job can send twice. The safer result is deliberately boring: the durable row remains recoverable, and both workers contend for the same delivery key.
This is why a claim should be short-lived and why the database transaction should be small. On databases that support it, FOR UPDATE SKIP LOCKED lets concurrent dispatchers select different due rows without waiting on each other. An outbox row written in the same transaction as the state change closes the gap between claiming work and later publishing it. The publisher may still repeat after handing a message to the broker but before recording the fact, so consumers still need deduplication.
// Claim a bounded set of due reminders. The database is the schedule of record.
const claimDue = `
UPDATE reminder_deliveries d
SET state = 'claimed', claimed_at = now(), attempts = attempts + 1
FROM (
SELECT id
FROM reminder_deliveries
WHERE state = 'pending' AND due_at <= now()
ORDER BY due_at
FOR UPDATE SKIP LOCKED
LIMIT $1
) due
WHERE d.id = due.id
RETURNING d.id, d.channel, d.dedupe_key`
func (d *Dispatcher) Tick(ctx context.Context, batch int) error {
rows, err := d.db.QueryContext(ctx, claimDue, batch)
if err != nil {
return fmt.Errorf("claim due reminders: %w", err)
}
defer rows.Close()
for rows.Next() {
var job Job
if err := rows.Scan(&job.ID, &job.Channel, &job.DedupeKey); err != nil {
return err
}
if err := d.queue.Publish(ctx, job); err != nil {
return fmt.Errorf("publish reminder %s: %w", job.ID, err)
}
}
return rows.Err()
}
The catch is that this machinery is not suitable for a low-consequence application with a single process and no need for audit, cancellation, or retries. In that case, durable rows plus a modest delayed-job mechanism may be enough. Do keep duplicate suppression: a small deployment still restarts.
When should a reminder service use cron scans instead of delayed messages?
Cron is a reasonable discovery mechanism for periodic work. The crontab(5) format has minute granularity, so a one-minute scan cannot guarantee an exact second of delivery. It is a good fit for collecting due records, sweeping expired claims, and repairing missed publications.
Delayed messages are a better fit when a product requires finer-grained dispatch. They do not replace the reminder row; cancellation, rescheduling, auditing, and repair still need a durable source of truth. A delayed job with no corresponding record leaves an operator with little to reconcile after a deployment or a user edit.
Don't turn a minute tick into a workflow engine. A notification that needs a durable multi-step timer, compensation, and human approval should use an execution model designed to hold that state. The right latency target also matters: a high-volume batch can accept a scan interval that would be unacceptable for a reminder expected within seconds.
Local clocks need a daylight-saving policy
People set 09:00, not 2026-11-01T15:00:00Z. Store the original wall-clock time and the named IANA zone alongside the computed UTC due time. An offset alone cannot describe future rule changes, and a time-zone abbreviation can be ambiguous.
A spring transition can make a local time nonexistent. A fall transition can make it occur twice. Pick and document a policy, such as moving a nonexistent local time to the next valid minute and selecting the earlier occurrence for an ambiguous time, then put those cases in tests. Product semantics can justify a different rule, so the important part is making the choice explicit and visible to support staff.
For recurring reminders, calculate each future UTC instant from the local schedule and named zone. Adding 24 hours to yesterday's UTC timestamp drifts the wall-clock delivery time across a daylight-saving boundary. The materialization horizon is a trade-off: a short horizon reduces repair after time-zone-data changes; a longer one makes downstream load easier to forecast. Your mileage may vary with the number of recurring schedules and the operational cost of recalculation.
A public webhook is a repeatable, untrusted receipt
Verify a webhook signature over the raw request body before interpreting business data. When the sender includes a signed timestamp, enforce its replay window. Persist the sender's event ID behind a uniqueness constraint, enqueue the follow-up work after recording the receipt, and return success only for a receipt the endpoint can safely accept again.
func (h *ReceiptHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
body, err := io.ReadAll(io.LimitReader(r.Body, 64<<10))
if err != nil {
http.Error(w, "invalid body", http.StatusBadRequest)
return
}
signature := r.Header.Get("X-Webhook-Signature")
if !validSignature(body, signature, h.secret) {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
eventID := r.Header.Get("X-Event-ID")
inserted, err := h.store.InsertReceipt(r.Context(), eventID, body)
if err != nil {
http.Error(w, "unable to record receipt", http.StatusInternalServerError)
return
}
if inserted {
_ = h.queue.Publish(r.Context(), ReceiptJob{EventID: eventID})
}
w.WriteHeader(http.StatusNoContent)
}
The header names in the example are intentionally generic because webhook signatures are sender-specific. The operational pattern is not: verify first, cap input, record identity, and make follow-up handling idempotent. Log the reminder ID, delivery key, provider event ID, attempt number, queue age, and final state. Those fields turn a complaint into a traceable sequence instead of a search through unrelated logs.
Before release, exercise a duplicated queue job, a worker interrupted between send and acknowledgement, a reschedule during a claim, a nonexistent local time, an ambiguous local time, and a replayed signed webhook. Alert on the age of the oldest due record, expired claims, queue age, duplicate suppression, and delivery outcomes by channel. The runbook should say which state owns the next action; otherwise the system is difficult to repair under pressure.
References
- https://man7.org/linux/man-pages/man5/crontab.5.html
- https://www.rabbitmq.com/docs/confirms
- https://www.postgresql.org/docs/current/sql-select.html
- https://www.iana.org/time-zones
- https://www.rfc-editor.org/rfc/rfc9421.html
Top comments (0)