DEV Community

Cover image for I just published my first Open Source library in Python and wanted to share it with you guys (video below)
Voy
Voy

Posted on

I just published my first Open Source library in Python and wanted to share it with you guys (video below)


https://www.youtube.com/watch?v=8XL7d_ehhsA

Databay is a scheduled data transfer interface, facilitating transfer of data from A to B on a scheduled interval. If your project requires custom data transfer, it would make my day if you gave Databay a look.

You can find Databay on GitHub, or its documentation on ReadTheDocs.

Some of Databay’s features are:

  • Simple, decoupled interface — Easily implement data production and consumption that fits your needs.
  • Granular control over data transfer — Multiple ways of passing information between producers and consumers.
  • Asyncio supported — You can produce or consume asynchronously.
  • We'll handle the rest — scheduling, startup and shutdown, exception handling, logging.
  • Support for custom scheduling — Use your own scheduling logic if you like.

I’m looking for someone who would like to do some code review on it (it’s relatively small), so if you’d feel like reading some code and helping out - drop me a message. Thanks!

This is the first time I’m publishing an open source library so would love to hear your feedback.

Thanks!
Voy

Top comments (8)

Collapse
 
omarkhatib profile image
Omar

Sounds interesting , Good Job 👏🏻

Collapse
 
voyz profile image
Voy

Thank you! Let me know if you have a chance to use it, would love to hear some feedback on it :)

Collapse
 
omarkhatib profile image
Omar

To be honest I didn't have a use case for it yet. But if I do for sure I will open a issue on github.👍

Thread Thread
 
voyz profile image
Voy

Appreciate it Omar! Have a good one!

Collapse
 
sakshatshinde profile image
Sakshat

Great work!

Collapse
 
voyz profile image
Voy

Thank you!!

Collapse
 
toanoop profile image
Anoop Sivadas

Will it be able to FTP. txt files to mainframes

Collapse
 
voyz profile image
Voy

I haven't tried doing this yet although at a glance I can't see why this wouldn't work. You're most welcome to give it a shot! Let me know how it goes, I'd be happy to help you build it.

You can implement it as follows:

  1. Create an Inlet that reads a directory's contents and produces records containing file paths to the .txt files
  2. Create an Outlet that expects records with file paths, picks the files specified and sends them over to your mainframe.
  3. Link the two nodes and schedule on the interval of your choice.

The basic_inlet and basic_outlet examples would be a good place to start if you'd like to see how inlets and outlets can be extended.

Keep me posted if you decide to try it out 😊