CloudWatch Logs retention policy for the whole account

A CloudWatch Logs retention policy is one of those settings nobody thinks about on the first deployment. Every Lambda function writes logs, and CloudWatch keeps them forever by default. After a few years of Serverless you end up with hundreds of log groups, and half of them belong to functions that no longer exist.

Since November 9, 2026, you can fix that centrally. An account-level retention policy defines how long logs stay in all log groups or in a selected subset. That sounds like a checkbox in the console. However, there is one detail in it that can delete data without warning. In this article I show you how the rules work, how to roll them out safely with a dry run, and how to manage them as code with AWS CDK.

In short

CloudWatch Logs can now set retention through policies for the whole account or for prefixes such as /aws/lambda. When several policies match, the longest one wins. The catch: a policy also replaces longer settings on individual log groups, so every change deserves a dry run first.

What you will learn in this article

• How account-level policies decide the retention of log groups
• Why a 90-day baseline can delete some logs earlier than you expect
• How to check the impact first with a dry run and a small TypeScript tool
• How to manage the policy as code with AWS CDK

Why log retention gets forgotten so often

CloudWatch Logs stores log data indefinitely by default. The documentation says so, and in day-to-day work you hardly notice. A Lambda function creates its log group /aws/lambda/<function-name> on the first invocation if you did not define it in your infrastructure code. It then says “Never expire”, and nobody ever looks at it again.

In addition, these log groups stay behind when you delete a stack. The function is gone, the logs remain. That costs storage, and it is also a privacy topic: logs like to contain email addresses, IP addresses or order numbers. The GDPR includes the principle of storage limitation (Art. 5(1)(e)), and “forever” is rarely a good answer to it. This is not legal advice, but it is a good reason to stop postponing the topic.

Until now you had two options. Either every team sets retention per log group in code. Or a platform group builds a Lambda function that catches new log groups via EventBridge and sets a retention afterwards. Both work, both have gaps. That is exactly where the new feature comes in.

How a CloudWatch Logs retention policy works

An account-level retention policy is an entry per account and Region that you create through the PutAccountPolicy API with the type RETENTION_POLICY. It has four parts:

You can create up to 20 of these policies per Region. The feature costs nothing extra, and according to the announcement it is available in all commercial Regions except Bahrain and the United Arab Emirates. Important: a policy only applies in the Region where you create it. If you work in eu-central-1 and eu-west-1, you need it twice.

The longest policy wins

Unlike other account-level policies in CloudWatch, retention policies may overlap. If several match a log group, the one with the longest retention applies. NEVER_EXPIRE counts as the longest. This leads to a nice pattern: one baseline for everything, longer periods for certain prefixes, and exceptions that never expire.

Example with three policies: the longest matching policy determines the effective retention.

The catch: longer settings get replaced too

Now comes the part you should not skim. A matching account-level policy fully replaces the setting of the log group. That works in both directions. If a log group keeps data for ten years and you create a 90-day baseline for everything, CloudWatch deletes everything in it that is older than 90 days. The documentation explicitly warns about this.

The opposite case is surprising as well. Many teams deliberately use 7 or 14 days for Lambda logs. Account-level policies, on the other hand, require at least 30 days. If such a log group falls under your baseline, it suddenly gets 30 or 90 days and costs more storage than before. So a baseline is a fixed value for everything it matches, and it is no upper limit.

RETROACTIVE or NEW_ONLY?

With RETROACTIVE, the policy applies to every matching log group, including existing ones. With NEW_ONLY, it only applies to log groups created after the policy. Existing ones keep their own setting. NEW_ONLY is the cautious start, yet it does not clean up the old leftovers. I would begin with a dry run for RETROACTIVE and decide after that.

Two timing details: a new or updated policy only takes effect after about 24 hours. Until then, the old value applies. Also, CloudWatch typically deletes expired events within 72 hours, in rare cases later. From the moment they expire, though, they no longer count towards storage costs.

🎬 How CloudWatch Logs determines the effective retention of a log group – this animated diagram only runs on my site: watch it on sascha-ms.de

Step by step: roll out retention with the AWS CLI

This is how I would introduce a CloudWatch Logs retention policy in an existing account. You need:

• AWS CLI v2 in a recent version whose put-account-policy command knows the --dry-run parameter
• the permissions logs:PutAccountPolicy and logs:PutRetentionPolicy, for deleting logs:DeleteAccountPolicy and logs:DeleteRetentionPolicy, for reading logs:DescribeAccountPolicies and logs:DescribeLogGroups
• an idea of which logs must stay longer (audit, billing, contracts)

1. Take stock

First, check which log groups never expire today and how much they store:

aws logs describe-log-groups \
  --query 'logGroups[?!retentionInDays].[logGroupName, storedBytes]' \
  --output table

The result is a table with names and stored bytes. Write down the log groups that should deliberately live longer. Those are exactly the ones that need their own policy in a moment.

2. Create the exceptions first

Create the longer periods before the baseline. That way there is no moment in which only the short baseline applies. In this example, the fictional bike rental keeps its audit logs for two years:

aws logs put-account-policy \
  --policy-name audit-2y \
  --policy-type RETENTION_POLICY \
  --policy-document '{"retentionInDays": 731, "enforcementMode": "RETROACTIVE"}' \
  --selection-criteria 'LogGroupNamePrefix="/bikerental/audit"'

3. Check the baseline with a dry run

Next comes the baseline for everything. With --dry-run, CloudWatch saves nothing. Instead, it reports how many log groups would keep data longer, shorter or unchanged:

aws logs put-account-policy \
  --policy-name baseline-90d \
  --policy-type RETENTION_POLICY \
  --policy-document '{"retentionInDays": 90, "enforcementMode": "RETROACTIVE"}' \
  --dry-run

The response contains a dryRunResult object with the fields decreasedRetentionLogGroupCount, increasedRetentionLogGroupCount and unchangedRetentionLogGroupCount. It also lists up to 20 of the largest affected log groups. For example, you immediately see whether a large archive group would get shorter. If decreasedRetentionLogGroupCount is greater than zero, look closely: those are the log groups that lose data.

4. Create it and check after 24 hours

If everything looks right, send the same command without --dry-run. After about a day, describe-log-groups shows the effective retention and the name of the policy that decides it:

aws logs describe-log-groups \
  --log-group-name-prefix /aws/lambda \
  --query 'logGroups[].[logGroupName, retentionInDays, retentionPolicyName]' \
  --output table

The retentionPolicyName field only shows up when an account-level policy applies. If NEVER_EXPIRE applies, retentionInDays is missing from the response. To list all policies, run aws logs describe-account-policies --policy-type RETENTION_POLICY.

A TypeScript tool for plan and dry run

The dry run returns counts and the 20 largest log groups. Often that is enough. But if you want to see the complete list of groups that lose data before an approval, a small tool helps. It loads all log groups and existing policies, recalculates the rules locally and then also asks for the real dry run.

Its core is pure logic without AWS calls and therefore easy to test. The effectiveRetention function implements both rules from above: the longest matching policy wins, and it replaces the log group’s own value.

// src/retention.ts
// Pure planning logic for account-level CloudWatch Logs retention policies.
// No AWS calls in here, so everything can be unit-tested offline.

/** Retention values (in days) that account-level policies accept. */
export const ALLOWED_DAYS = [
  30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1096, 1827, 2192, 2557, 2922,
  3288, 3653,
] as const;

export type Retention = number | "NEVER_EXPIRE";
export type EnforcementMode = "RETROACTIVE" | "NEW_ONLY";

export interface RetentionPolicy {
  name: string;
  /** Log group name prefix; undefined = all log groups in the account. */
  prefix?: string;
  retention: Retention;
  mode: EnforcementMode;
  /** When the policy was created (epoch ms). Needed for NEW_ONLY. */
  createdAt: number;
}

export interface LogGroupInfo {
  name: string;
  /** Retention set on the log group itself; undefined = never expire. */
  retentionInDays?: number;
  creationTime: number;
  storedBytes: number;
}

export type ChangeKind = "shorter" | "longer" | "unchanged";

export interface PlannedChange {
  group: LogGroupInfo;
  before: Retention;
  after: Retention;
  kind: ChangeKind;
  /** Name of the policy that decides the retention after the change. */
  decidedBy?: string;
}

/** Parses CLI input like "90" or "never" into a valid retention value. */
export function parseRetention(input: string): Retention {
  const value = input.trim().toLowerCase();
  if (value === "never" || value === "never_expire") return "NEVER_EXPIRE";
  const days = Number(value);
  if (!Number.isInteger(days) || !(ALLOWED_DAYS as readonly number[]).includes(days)) {
    throw new Error(
      `Invalid retention "${input}". Allowed: ${ALLOWED_DAYS.join(", ")} or "never".`,
    );
  }
  return days;
}

/** Builds the policyDocument string for PutAccountPolicy. */
export function buildPolicyDocument(retention: Retention, mode: EnforcementMode): string {
  return JSON.stringify({ retentionInDays: retention, enforcementMode: mode });
}

/** Builds the selectionCriteria string, or undefined for "all log groups". */
export function buildSelectionCriteria(prefix?: string): string | undefined {
  if (prefix === undefined || prefix === "") return undefined;
  if (prefix.includes('"')) throw new Error("Prefix must not contain double quotes.");
  return `LogGroupNamePrefix="${prefix}"`;
}

/** Reads a policy as returned by DescribeAccountPolicies. */
export function parsePolicy(p: {
  policyName?: string;
  policyDocument?: string;
  selectionCriteria?: string;
  lastUpdatedTime?: number;
}): RetentionPolicy {
  const doc = JSON.parse(p.policyDocument ?? "{}") as {
    retentionInDays?: Retention;
    enforcementMode?: EnforcementMode;
  };
  const match = /LogGroupNamePrefix\s*=\s*"([^"]*)"/.exec(p.selectionCriteria ?? "");
  return {
    name: p.policyName ?? "unknown",
    prefix: match?.[1],
    retention: doc.retentionInDays ?? "NEVER_EXPIRE",
    mode: doc.enforcementMode ?? "RETROACTIVE",
    // The API only returns lastUpdatedTime, which is the best approximation we have.
    createdAt: p.lastUpdatedTime ?? 0,
  };
}

/** Does a policy apply to a given log group? */
export function policyMatches(policy: RetentionPolicy, group: LogGroupInfo): boolean {
  const prefixOk = policy.prefix === undefined || group.name.startsWith(policy.prefix);
  const modeOk = policy.mode === "RETROACTIVE" || group.creationTime >= policy.createdAt;
  return prefixOk && modeOk;
}

/** Compares two retention values. NEVER_EXPIRE is the longest. */
export function compareRetention(a: Retention, b: Retention): number {
  const toNumber = (r: Retention) => (r === "NEVER_EXPIRE" ? Number.POSITIVE_INFINITY : r);
  return toNumber(a) - toNumber(b);
}

/**
 * Effective retention of a log group: if at least one account policy matches,
 * the longest matching policy wins and REPLACES the log group's own setting
 * (even if that setting was longer). Otherwise the log group's own value counts.
 */
export function effectiveRetention(
  group: LogGroupInfo,
  policies: RetentionPolicy[],
): { retention: Retention; decidedBy?: string } {
  const matching = policies.filter((p) => policyMatches(p, group));
  if (matching.length === 0) {
    return { retention: group.retentionInDays ?? "NEVER_EXPIRE" };
  }
  const winner = matching.reduce((best, p) =>
    compareRetention(p.retention, best.retention) > 0 ? p : best,
  );
  return { retention: winner.retention, decidedBy: winner.name };
}

/** Simulates adding (or replacing) a policy and lists the effect per log group. */
export function planChange(
  groups: LogGroupInfo[],
  existing: RetentionPolicy[],
  candidate: RetentionPolicy,
): PlannedChange[] {
  const after = [...existing.filter((p) => p.name !== candidate.name), candidate];
  return groups
    .map((group) => {
      const before = effectiveRetention(group, existing).retention;
      const next = effectiveRetention(group, after);
      const diff = compareRetention(next.retention, before);
      const kind: ChangeKind = diff < 0 ? "shorter" : diff > 0 ? "longer" : "unchanged";
      return { group, before, after: next.retention, kind, decidedBy: next.decidedBy };
    })
    .sort((a, b) => b.group.storedBytes - a.group.storedBytes);
}

/** Groups the plan by kind, e.g. for a short summary. */
export function summarize(plan: PlannedChange[]): Record<ChangeKind, number> {
  const summary: Record<ChangeKind, number> = { shorter: 0, longer: 0, unchanged: 0 };
  for (const change of plan) summary[change.kind] += 1;
  return summary;
}

/** Human-readable bytes (base 1024). */
export function formatBytes(bytes: number): string {
  const units = ["B", "KiB", "MiB", "GiB", "TiB"];
  let value = bytes;
  let unit = 0;
  while (value >= 1024 && unit < units.length - 1) {
    value /= 1024;
    unit += 1;
  }
  return `${value.toFixed(unit === 0 ? 0 : 1)} ${units[unit]}`;
}

The command line uses the AWS SDK for JavaScript v3. Starting with version 3.1165.0, @aws-sdk/client-cloudwatch-logs knows the dryRun parameter and the retentionPolicyName field. Install and run it like this:

npm install
npx tsx src/cli.ts inventory
npx tsx src/cli.ts plan --name baseline-90d --days 90
npx tsx src/cli.ts apply --name baseline-90d --days 90 --yes

The expected output of plan looks roughly like this. Every line with SHORTER is a log group that keeps less data after the change:

Local plan: 2 shorter, 41 longer, 12 unchanged
  SHORTER         3653 -> 90           8.0 GiB  /bookshop/archive
  SHORTER NEVER_EXPIRE -> 90           5.0 GiB  /aws/lambda/search
Dry run (AWS): matched 55, shorter 2, longer 41, unchanged 12, not analyzed 0

There are two limits you should know. If a policy already applies to a log group, the API only shows the policy value and no longer the group’s own one. And for NEW_ONLY, the tool uses the time of the last update as an approximation. If the numbers differ, trust the AWS dry run. The Vitest tests cover the rules, for example that NEVER_EXPIRE always wins.

Retention policy as code with AWS CDK

A setting that affects this many log groups belongs in version control. There is one catch: as of this writing (November 10, 2026), the CloudFormation resource AWS::Logs::AccountPolicy does not list the RETENTION_POLICY type yet. Until that changes, an AwsCustomResource helps in AWS CDK. It calls putAccountPolicy on create and update and deleteAccountPolicy on delete.

// lib/log-retention-stack.ts
// CloudFormation's AWS::Logs::AccountPolicy does not list RETENTION_POLICY yet,
// so this stack calls the CloudWatch Logs API through an AwsCustomResource.
import { Stack, type StackProps } from "aws-cdk-lib";
import * as iam from "aws-cdk-lib/aws-iam";
import {
  AwsCustomResource,
  AwsCustomResourcePolicy,
  PhysicalResourceId,
} from "aws-cdk-lib/custom-resources";
import { Construct } from "constructs";

export interface RetentionRule {
  /** Unique policy name, must not start with "aws/". */
  name: string;
  /** Log group name prefix; leave empty for all log groups. */
  prefix?: string;
  /** 30, 60, 90, ... 3653 or "NEVER_EXPIRE". */
  retention: number | "NEVER_EXPIRE";
  mode?: "RETROACTIVE" | "NEW_ONLY";
}

export class AccountRetentionPolicy extends Construct {
  constructor(scope: Construct, id: string, rule: RetentionRule) {
    super(scope, id);

    const params: Record<string, string> = {
      policyName: rule.name,
      policyType: "RETENTION_POLICY",
      policyDocument: JSON.stringify({
        retentionInDays: rule.retention,
        enforcementMode: rule.mode ?? "RETROACTIVE",
      }),
    };
    if (rule.prefix) {
      params.selectionCriteria = `LogGroupNamePrefix="${rule.prefix}"`;
    }

    const putCall = {
      service: "CloudWatchLogs",
      action: "putAccountPolicy",
      parameters: params,
      physicalResourceId: PhysicalResourceId.of(`retention-${rule.name}`),
    };

    new AwsCustomResource(this, "Policy", {
      onCreate: putCall,
      onUpdate: putCall,
      onDelete: {
        service: "CloudWatchLogs",
        action: "deleteAccountPolicy",
        parameters: { policyName: rule.name, policyType: "RETENTION_POLICY" },
      },
      policy: AwsCustomResourcePolicy.fromStatements([
        new iam.PolicyStatement({
          actions: [
            "logs:PutAccountPolicy",
            "logs:PutRetentionPolicy",
            "logs:DeleteAccountPolicy",
            "logs:DeleteRetentionPolicy",
          ],
          resources: ["*"],
        }),
      ]),
      installLatestAwsSdk: false,
    });
  }
}

export class LogRetentionStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    // Baseline for everything in this account and region.
    new AccountRetentionPolicy(this, "Baseline", { name: "baseline-90d", retention: 90 });

    // Longer retention for audit logs of the fictional bike rental.
    new AccountRetentionPolicy(this, "Audit", {
      name: "audit-2y",
      prefix: "/bikerental/audit",
      retention: 731,
    });
  }
}

Run npx cdk synth to check the template and npx cdk deploy to roll it out. Expected result: two resources of type Custom::AWS and one IAM policy with exactly the four Logs actions. If you delete the stack, the policies disappear as well. A dry run of delete-account-policy shows you beforehand what will then apply to each log group.

A tip from the architecture corner: once prefixes decide about retention, log group names suddenly become an architecture decision. Write the convention down, for example in an architecture decision record. Otherwise an audit log will eventually end up under a prefix that cleans up after 90 days.

Common mistakes with log retention

• Creating the baseline before the exceptions. The exceptions go in first, otherwise only the short period applies for a moment. The 24-hour delay softens that a bit, but I would not rely on it.
• Skipping the dry run. A look at decreasedRetentionLogGroupCount takes seconds. Deleted logs do not come back.
• Trusting the value in your code. If your team sets 14 days in a CDK stack, but an account-level policy applies, the policy value wins. Every team should know that.
• Forgetting Regions. Each Region needs its own policies, including the one where only “a little bit” runs.
• Granting too many permissions. AWS recommends giving logs:PutAccountPolicy only to administrators who own logging for the whole account.
• Starting several dry runs at once. Only one dry run can run per account at a time. Scripts that check in parallel will get errors.

Keeping an eye on cost and security

The policies themselves are free. What changes is the storage cost of log data, and it can move in both directions. Old log groups without expiry get cheaper, while short Lambda log groups below 30 days may get more expensive. How much that means for you follows from the inventory in step 1 together with the CloudWatch Logs price list.

Security is mostly about control. Whoever may change account-level policies can indirectly delete logs. Treat these permissions like delete permissions. For logs with a legal retention requirement, I would also check whether they belong in a central log archive account to which application teams have no write access.

My take

To be honest: I have been waiting for this for a long time. “Never expire” as the default was always a poor compromise for Serverless accounts with many small functions. Finally there is a built-in solution, without a custom cleanup Lambda.

Still, I would like different semantics. A baseline that replaces longer values on individual log groups is handy for platform teams and surprising for application teams. An option like “only set it if nothing is set” would save a lot of discussions. Until then, communication is the only fix: tell the teams that their value in code no longer has the last word. And put it into your runbook before someone searches for logs during an incident that no longer exist.

My suggestion for the start: a 90-day baseline with RETROACTIVE, exceptions for audit and contracts created first, everything checked with a dry run and managed as code. That takes an hour and cleans up years of leftovers.

FAQ

How long does CloudWatch Logs keep logs by default?

Indefinitely. Without its own setting and without an account-level policy, log events never expire.

Can I set a 7-day retention with an account-level policy?

No. Account-level policies require at least 30 days. Shorter periods still only work per log group, and only as long as no account-level policy applies to that group.

Does the retention policy cost extra?

No, according to AWS there are no additional costs for the policies. Only the storage costs of the log data change.

How do I delete a retention policy again?

Use aws logs delete-account-policy --policy-name <name> --policy-type RETENTION_POLICY. Add --dry-run to see the effect of the deletion first.

Your next step

Run the inventory from step 1 in your most important account today. The number of log groups set to “Never expire” is usually argument enough. If you are working on costs anyway, my article about NAT gateway costs with Lambda fits right in.

Now it is your turn: which baseline retention do you consider sensible, and which logs have to stay longer in your setup? Let me know in the comments.

All code from this article, ready to run: smirowstanitzok/sascha-ms-examples

Sources

• AWS What’s New: Amazon CloudWatch Logs announces account-level retention policies (November 9, 2026)
• Amazon CloudWatch Logs User Guide: Working with log groups and log streams
• Amazon CloudWatch Logs API Reference: PutAccountPolicy
• AWS CloudFormation: AWS::Logs::AccountPolicy
• npm: @aws-sdk/client-cloudwatch-logs
• Regulation (EU) 2016/679 (GDPR), Art. 5

This post was created with AI assistance.

👉 Next step: Curious what your own Lambda functions cost? Try my Lambda cost calculator – it runs right in your browser.

📬 Stay updated: New articles on serverless, AI and leading tech teams via RSS, browser push or the weekly newsletter (in German).

Originally published on sascha-ms.de. More articles on serverless, AI in everyday development and leading tech teams: sascha-ms.de/en