DEV Community

Discussion on: Watch out for Unexpected S3 cost when using Athena.

Collapse
 
francoislagier profile image
Francois LAGIER

I don't think it's possible to disable it. One solution would be to have a simple Lambda function to clean your files after a few minutes.

Collapse
 
starpebble profile image
starpebble

The Life Cycle transition is brilliant. The simple lambda interesting, too. Combine both with S3 object versioning to open a door!

Combining Life Cycles and object versioning can also reduce costs. Object versioning is a game changer. Two versions of a single query result may be stored with S3 object versioning. Each object with different attributes. The first object is the Athena generated object with a transition to a lower cost tier. Example: a csv and glacier. The second object could be the compressed version of the exact same object, at a more available S3 tier. Example: compressed at standard tier. Each version has a different version id. Create the compressed version automatically. Configure S3 to send an event when a query result is put in the bucket and trigger a lambda to compress the file and put the compressed file in the bucket as a second version of the query result. Combining life cycling and versioning is like opening up a door.