It has been a while, but I have been busy.
Well not with my home page, which was there and then no longer is...
But I come, bearing gifts.
I am currently working with quite big lists, I download from Azure via it's Graph API. Until recently I used the usual suspects to store the downloaded data ... well ... mostly a List.
And then I stumbled over IAsyncEnumerable<T>, which allows for using the data already, while it is only partially downloaded (yet).
The drawback: You will have to download it every time you access it.
Well in my case, I was going through the downloaded data multiple times and I thought I encapsulate it, to persists the downloaded data while downloading. So the second time I access it, it is already there.
It will depend on you use case, but I need one consistent set of multiple chunks/lists. So I either download everything once or do everything on the fly.
"Once" was my preference and I decided to create something that can support this use case.
Said and done. It took some time to make it thread safe. And since this introduces some runtime overhead, which is not always needed/wanted, I also decided to have two variants of it.
- Single Threaded Access - which gives you an Exception, when trying to create another Enumerator, while the "old" one is stil in use (or at least not yet disposed).
- The multi threaded one, which allows for multiple readers on the same underlying IAsyncEnumerable, persisting it's data along the way.
Where can I get it?
You can look into the sources here: GitHub - PersistingAsyncEnumerable
Or get the NuGet Package here: NuGet - PersistingAsyncEnumerable
What kind of licence is it?
MIT. I just thought, It might help some one and it was fun for me to create it.
Have fun with it and I would love some feedback.
Top comments (0)