DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’»

Cover image for Reduct Storage Python SDK v0.3.0 has been released
Alexey Timin for Reduct Storage

Posted on • Updated on

Reduct Storage Python SDK v0.3.0 has been released

Good news everyone!

Reduct Storage Python SDK v0.3.0 has been released.

It supports the latest Reduct Storage HTTP API v0.6.0, but the most important change is data streaming. Let's see how it works with examples!

Prerequisites

This post isn't a tutorial, and I suppose you know how to launch an instance of Reduct Storage and create a bucket there. If you don't know, then a few links that can help:

Read Data By Chunks

In this release, we introduced a new method Bucket.read_by to read data by chunks with an asynchronous iterator:

 with open("file.bin", "wb") as file:
     async for chunk in bucket.read_by("entry-1", chunk_size=1024):
         file.write(chunk)
Enter fullscreen mode Exit fullscreen mode

It might be useful when you have a big record to download or need to read many records in the same time.

Write Data By Chunks

To write data by chunks, can be a bit more tricky. First, you have to pass an asynchronous iterator to the Bucket.write method. Then you have to specify the size of the data which you're writing. The storage engine requires the content-lenght header, and the user should know the size of written data in advance.

file_path = Path("./path/to/file")
async def sender():
    with open(file_path, "br") as file:
         while True:
             chunk = file.read(1024)
             if not chunk:
                 break
             yield chunk

content_length = file_path.stat().st_size
await bucket.write("entry-1", sender(), content_length=content_length)
Enter fullscreen mode Exit fullscreen mode

If you've got interested, learn more about the storage engine here


Cover Image Designed by roserodionova / Freepik

Top comments (0)

πŸ”  Find your favorite font

Β 
You can change your default font in Settings.