How did I come up with the idea?

I've been using instagram for quite a while posting stories of images and videos of all sorts of things so I thought to make an effect that would take my image and "recolorize" it in which would kind of spark life into a picture. Therefore I set myself to a task to create something as basic as "recolorizing" a picture.

Check the GitHub repository at https://github.com/slushnys/recolorize

Disclaimer: I'm not using any algorithms to bring colors to black and white images, all this does is reapplying existing ones back while creating a movie out of it.

Steps I had to take were (workflow):

1. Take a colorful image
2. Resize the image so it would take less time to process.
3. Research imageio library to help with image processing.
4. Save it to the disk
5. BONUS: Save it to google cloud storage.

Steps of my implementation

Getting familiar with Pillow library

First of all I just wanted to get to know how to manipulate the image and to do that I knew one way, to use Pillow library (fork of PIL) that has a lot of capabilities to work with image data. I would strongly suggest anyone interested in image manipulation to check it out. Check out some documentation at https://pillow.readthedocs.io/en/stable/index.html

You have to install with command:

python -m pip install Pillow

After trying out the library and reading some specific part of documentation I was able to open the image and reduce the color of it or increase it on demand.

# Open the image with the library
dir_name = os.path.dirname(__file__)
abs_path_to_image = os.path.join(dir_name, './test_image.jpeg')
image = Image.open(file)
color_percentage = 0.5 # 50%
enhanced_image = ImageEnhance.Color(image).enhance(color_percentage)
enhanced_image.show() # opens your enhanced image

Create a video out of this image enhancement

So what are videos? Videos are just a list of frames (images) that are ran fast enough so your eyes wouldn't see the jitter between images changing. That's about 24 frames per second. What we need to do is to multiply the frames per second needed by the seconds of the transition. I took 5 seconds for the transition therefore 5*24=106. Then for each frame in that range I took that image and enchanced it bit by bit to from 0% to 100% of colors so it would look like it's regaining color.

Ofcourse you need to write the manipulated image data into some sort of buffer so it would be possible to convert to a movie and this is where imageio came to help me. I found this to be very easy, fast and efficient with a easy API to work with. You can check the documentation at https://imageio.github.io/

total_frames = seconds * fps

color_percentage_for_each_frame = (100 / total_frames) / 100

write_to = 'output/{}.mp4'.format(output) # have a folder of output where output files could be stored.

writer = imageio.get_writer(write_to, format='mp4', mode='I', fps=fps)

for i in range(total_frames):
    if i < total_frames:
        processed = ImageEnhance.Color(im).enhance(
            color_percentage_for_each_frame * i)
        writer.append_data(np.asarray(processed))
    else:
        writer.append_data(np.asarray(im))
writer.close()

All this code does is creates a buffer in a variable of writer where all the image data is going to be stored and when you close the writer, it will create you the movie file from all the processed images.

Wrap this functionality in a server

To complete the implementation we simply wrapped up the functionality in a Flask webserver that would allow me to have a simple user interface of uploading a file and downloading a file. We used Flask because it's a lightweight server that can encapsulate simple tasks providing easy to access functionality.

Conclusion

I had fun trying to figure how to do this, and I think there could be a lot of improvements done in order to reduce the size of the video that's being outputted, also there should be performance gains when creating the video itself as currently, it is processor intensive.

BONUS

For the sake of integration, we have also an option to store the processed files in a buffer that would be later saved in google cloud storage on completion. If you're interested in this functionality, check out the Github repository.

For the future, I would like to encapsulate this functionality within a mobile app in order to make a simple filter for people to customize their pictures.