How to Develop a Timelapse Effect Generator

Background

Have you ever watched a video of the northern lights? Mesmerizing light rays that swirl and dance through the star-encrusted sky. It's even more stunning when they are backdropped by crystal-clear waters that flow smoothly between and under ice crusts. Complementing each other, the moving sky and water compose a dynamic scene that reflects the constant rhythm of the mother nature.

Now imagine that the video is frozen into an image: It still looks beautiful, but lacks the dynamism of the video. Such a contrast between still and moving images shows how videos are sometimes better than still images when it comes to capturing majestic scenery, since the former can convey more information and thus be more engaging.

This may be the reason why we sometimes regret just taking photos instead of capturing a video when we encounter beautiful scenery or a memorable moment.

In addition to this, when we try to add a static image to a short video, we will find that the transition between the image and other segments of the video appears very awkward, since the image is the only static segment in the whole video.

If we want to turn a static image into a dynamic video by adding some motion effects to the sky and water, one way to do this is to use a professional PC program to modify the image. However, this process is often very complicated and time-consuming: It requires adjustment of the timeline, frames, and much more, which can be a daunting prospect for amateur image editors.

Luckily, there are now numerous AI-driven capabilities that can automatically create time-lapse videos for users. I chose to use the auto-timelapse capability provided by HMS Core Video Editor Kit. It can automatically detect the sky and water in an image and produce vivid dynamic effects for them, just like this:

The movement speed and angle of the sky and water are customizable.

Now let's take a look at the detailed integration procedure for this capability, to better understand how such a dynamic effect is created.

Integration Procedure

Preparations

i. Configure necessary app information. This step requires you to register a developer account, create an app, generate a signing certificate fingerprint, configure the fingerprint, and enable the required services.
ii. Integrate the SDK of the kit.
iii. Configure the obfuscation scripts.
iv. Declare necessary permissions.

Project Configuration

i. Set the app authentication information. This can be done via an API key or an access token.

• Set an API key via the setApiKey method: You only need to set the app authentication information once during app initialization.

MediaApplication.getInstance().setApiKey("your ApiKey");

• Or, set an access token by using the setAccessToken method: You only need to set the app authentication information once during app initialization.

MediaApplication.getInstance().setAccessToken("your access token");

ii. Set a License ID. This ID should be unique because it is used to manage the usage quotas of the service.

MediaApplication.getInstance().setLicenseId("License ID");

iii. Initialize the runtime environment for the HuaweiVideoEditor object. Remember to release the HuaweiVideoEditor object when exiting the project.

• Create a HuaweiVideoEditor object.

HuaweiVideoEditor editor = HuaweiVideoEditor.create(getApplicationContext());

• Specify the preview area position. Such an area is used to render video images, which is implemented by SurfaceView created within the SDK. Before creating such an area, specify its position in the app first.

<LinearLayout    
    android:id="@+id/video_content_layout"    
    android:layout_width="0dp"    
    android:layout_height="0dp"    
    android:background="@color/video_edit_main_bg_color"    
    android:gravity="center"    
    android:orientation="vertical" />
// Specify the preview area position.
LinearLayout mSdkPreviewContainer = view.findViewById(R.id.video_content_layout);

// Specify the preview area layout.
editor.setDisplay(mSdkPreviewContainer);

• Initialize the runtime environment. If license verification fails, LicenseException will be thrown.

After it is created, the HuaweiVideoEditor object will not occupy any system resources. You need to manually set when the runtime environment of the object will be initialized. Once you have done this, necessary threads and timers will be created within the SDK.

try {
        editor.initEnvironment();
   } catch (LicenseException error) { 
        SmartLog.e(TAG, "initEnvironment failed: " + error.getErrorMsg());    
        finish();
        return;
   }

Function Development

// Initialize the auto-timelapse engine.
imageAsset.initTimeLapseEngine(new HVEAIInitialCallback() {
        @Override
        public void onProgress(int progress) {
            // Callback when the initialization progress is received.
        }

        @Override
        public void onSuccess() {
            // Callback when the initialization is successful.
        }

        @Override
        public void onError(int errorCode, String errorMessage) {
            // Callback when the initialization failed.
        }
});
// When the initialization is successful, check whether there is sky or water in the image.
int motionType = -1;
imageAsset.detectTimeLapse(new HVETimeLapseDetectCallback() {
        @Override
        public void onResult(int state) {
            // Record the state parameter, which is used to define a motion effect.
            motionType = state;
        }
});

// skySpeed indicates the speed at which the sky moves; skyAngle indicates the direction to which the sky moves; waterSpeed indicates the speed at which the water moves; waterAngle indicates the direction to which the water moves.
HVETimeLapseEffectOptions options = 
new HVETimeLapseEffectOptions.Builder().setMotionType(motionType)
        .setSkySpeed(skySpeed)
        .setSkyAngle(skyAngle)
        .setWaterAngle(waterAngle)
        .setWaterSpeed(waterSpeed)
        .build();

// Add the auto-timelapse effect.
imageAsset.addTimeLapseEffect(options, new HVEAIProcessCallback() {
        @Override
        public void onProgress(int progress) {
        }
        @Override
        public void onSuccess() {
        }
        @Override
        public void onError(int errorCode, String errorMessage) {
        }
});
// Stop applying the auto-timelapse effect.
imageAsset.interruptTimeLapse();

// Remove the auto-timelapse effect.
imageAsset.removeTimeLapseEffect();

Now, the auto-timelapse capability has been successfully integrated into an app.

Conclusion

When capturing scenic vistas, videos, which can show the dynamic nature of the world around us, are often a better choice than static images. In addition, when creating videos with multiple shots, dynamic pictures deliver a smoother transition effect than static ones.

However, for users not familiar with the process of animating static images, if they try do so manually using computer software, they may find the results unsatisfying.

The good news is that there are now mobile apps integrated with capabilities such as Video Editor Kit's auto-timelapse feature that can create time-lapse effects for users. The generated effect appears authentic and natural, the capability is easy to use, and its integration is straightforward. With such capabilities in place, a video/image app can provide users with a more captivating user experience.

In addition to video/image editing apps, I believe the auto-timelapse capability can also be utilized by many other types of apps. What other kinds of apps do you think would benefit from such a feature? Let me know in the comments section.