Implement Realistic 3D Rendering in Three Steps

The conventional method of rendering a 3D model and scene with realistic materials is often tedious, making it difficult to standardize into several, simple steps.

Let's see what this process requires:

First, a rich array of texture maps and material images should be prepared according to how detailed and complex the model and scene are. The more complex they are, the more time-consuming the process of collecting such maps and images will be. Of course you can turn to those available on the market, but they could be expensive.

Second, parameters that have to be created from scratch. Most 3D modeling programs require abundant parameters for rendering, and setting or controlling models via parameter adjustment.

Third, work experience. The demand for 3D modelers is increasing, emphasizing the skills and work experience necessary for 3D modeling. As models and scenes are becoming more diverse and delicate, the modeler has a bigger influence over the quality of the models.

Indeed, such requirements — if met — lead to a desirable result. They, however, are also daunting.

Fortunately, there is now an easier solution to this with the material generation capability from 3D Modeling Kit, which is easy-to-use and can create high-quality materials in just three steps.

The image on the left shows a complex scene containing multiple objects. Such a scene is common to see in a 3D game. To render it with materials, we just need to:

1. Use an RGB camera to collect images of real materials.

2.Input the images for material generation, which automatically generates the texture maps.

3.Use the maps for rendering.

The material generation capability from 3D Modeling Kit utilizes AI to streamline and simplify the strenuous process of material generation and rendering, to improve the efficiency of creating 3D models at a lower cost. Specifically speaking, this capability offers the template materials that are created from the experience and material creation specifications of technical artists, to make such valuable assets reusable.

Introduction to Material Generation

This capability enables your users to turn one or more RGB images into four physically based rendering (PBR) texture maps (the diffuse map, normal map, specular map, and roughness map), with just a tap, to help create true-to-life materials. This capability utilizes AI to create 3D materials for commercial use.

Specifications

•Software and hardware requirements: an Android device with a standard RGB camera, instead of an RGB-D or light detection and ranging (LiDAR) sensor

•Supported material types: concrete, marble, rock, gravel, brick, gypsum, clay, metal, wood, bark, leather, fabric, paint, plastic, composite material, and ground (including grass and sand)

•Supported input images: have a resolution within the range of 1K to 4K, containing no seam, light spot, shade, or reflection.

•Output texture maps: have a resolution of 1K (1024 x 1024 px) or 2K (2048 x 2048 px).

•SDK size: 88 KB

•Accuracy (SSIM): > 0.9
Application scenarios : The powerful material generation capability can be adopted in a wide range of scenarios, for example:

•E-commerce: Use this capability to create 3D product models, delivering a more immersive online shopping experience to users.

•Exhibition: Display lifelike 3D models of valuable items such as cultural exhibits.

•Game: Create realistic wooden floors, tables, and walls for indoor scenes, to better immerse players.

Development Preparations

Integrating the HMS Core SDK
Add the AppGallery Connect configuration file of your app.
Add the agconnect-services.json file to your app after you have enabled 3D Modeling Kit.

Step 1. Sign in to AppGallery Connect and click My projects .

Step 2. Find your project and click the app that needs to integrate the HMS Core SDK.

Step 3. Go to Project settings > General information . In the App information area, click agconnect-services.json to download the configuration file.

Step 4. Copy the file to the app's root directory.

Configuring the Maven Repository Address for the HMS Core SDK
Step 1. Open the project-level build.gradle file of your Android Studio project.

Step 2. Add the AppGallery Connect plugin and the Maven repository.

•Go to buildscript > repositories and configure the Maven repository address for the HMS Core SDK.

•Go to allprojects > repositories and configure the Maven repository address for the HMS Core SDK.

•Go to buildscript > dependencies and add the AppGallery Connect plugin configuration, if the agconnect-services.json file has been added to the app.

buildscript { 
    repositories { 
        google() 
        jcenter() 
        // Configure the Maven repository address for the HMS Core SDK.
        maven {url 'https://developer.huawei.com/repo/'} 
    } 
    dependencies { 
        ... 
        // Add the AppGallery Connect plugin configuration.
        classpath 'com.huawei.agconnect:agcp:1.4.2.300' 
    } 
} 
allprojects { 
    repositories { 
        google() 
        jcenter() 
        // Configure the Maven repository address for the HMS Core SDK.
        maven {url 'https://developer.huawei.com/repo/'} 
    } 
} 

Note that the Maven repository address cannot be accessed from a browser. It can only be configured in the IDE. If there are multiple Maven repositories, add the Maven repository address of Huawei as the last one.

Adding Build Dependencies

Step 1. Open the app-level build.gradle file.

Step 2. Add build dependencies in the dependencies block.

dependencies { 
    implementation 'com.huawei.hms:modeling3d-material-generate:{version}' 
}

Note: Replace {version} with the actual SDK version number. For details about the version number, please refer to Version Change History.

For example: implementation 'com.huawei.hms:modeling3d-material-generate:1.0.0.300'.

Step 3. Add the AppGallery Connect plugin configuration in either of the following methods:

•Method 1: Add the following information under the declaration in the file header:

apply plugin: 'com.huawei.agconnect'

•Method 2: Add the plugin configuration in the plugins block.

plugins { 
    id 'com.android.application' 
    // Add the following configuration.
    id 'com.huawei.agconnect' 
}

Defining Multi-language Settings

•If your app supports all the languages supported by the HMS Core SDK, skip the operation procedure in this section.

•If your app supports only some of these languages, follow the operation procedure in this section to complete the required configuration.
a. Open the app-level build.gradle file.

b.Go to android > defaultConfig, add resConfigs, and configure the supported languages as follows:

android { 
    defaultConfig { 
        ... 
        resConfigs "en", "zh-rCN", "Other languages supported by your app" 
    } 
}  

For details about the languages supported by the HMS Core SDK, please refer to Languages Supported by the HMS Core SDK.

Synchronizing the Project

After completing the preceding configuration, click the synchronization icon on the toolbar to synchronize the Gradle files.

If an error occurs, check the network connection and the configuration in the build.gradle files.

Configuring Obfuscation Scripts

Before building the APK, configure the obfuscation configuration file to prevent the HMS Core SDK from being obfuscated.

Step 1. Open the app-level obfuscation configuration file proguard-rules.pro of your project and add configurations to exclude the HMS Core SDK from obfuscation.

-ignorewarnings 
-keepattributes *Annotation* 
-keepattributes Exceptions 
-keepattributes InnerClasses 
-keepattributes Signature 
-keepattributes SourceFile,LineNumberTable 
-keep class com.huawei.hianalytics.**{*;} 
-keep class com.huawei.updatesdk.**{*;} 
-keep class com.huawei.hms.**{*;}

Step 2. Add the trustlist of AndResGuard to the app-level build.gradle file in your project, if you're using AndResGuard.

"R.string.hms*", 
"R.string.connect_server_fail_prompt_toast", 
"R.string.getting_message_fail_prompt_toast", 
"R.string.no_available_network_prompt_toast", 
"R.string.third_app_*", 
"R.string.upsdk_*", 
"R.layout.hms*", 
"R.layout.upsdk_*", 
"R.drawable.upsdk*", 
"R.color.upsdk*", 
"R.dimen.upsdk*", 
"R.style.upsdk*",  
"R.string.agc*"

Adding Permissions

To use the material generation capability, declare the following permissions in the AndroidManifest.xml file:

<!-- Write texture map files to storage and read data to be processed from storage. --> 
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" /> 
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" /> 
<!-- Connect to Internet for uploading images or downloading texture maps. --> 
<uses-permission android:name="android.permission.INTERNET" />

Development Procedure

Before developing your app, you need to make necessary preparations. Ensure that the Maven repository address for the HMS Core SDK has been configured in your project, and the SDK of this service has been integrated.

1.Set an access token or use the API key in agconnect-services.json during app initialization for your app authentication.
•(Recommended) Use the setAccessToken method to set an access token during initialization when the app is started. The access token does not need to be set again.

MaterialGenApplication.getInstance().setAccessToken("your AccessToken");

For details about how to obtain the access token, please refer to "Client Credentials" in OAuth 2.0-based Authentication.

•Use the setApiKey method to set an API key during initialization when the app is started. The API key does not need to be set again.

MaterialGenApplication.getInstance().setApiKey("your api_key");

When you create an app in AppGallery Connect, an API key will be assigned to your app.

2.Create a material generation engine and configurator, and initialize the engine.

// Create a material generation engine and pass the current context.
Modeling3dTextureEngine engine = Modeling3dTextureEngine.getInstance(context); 
// Create a material generation configurator.
Modeling3dTextureSetting setting = new Modeling3dTextureSetting.Factory() 
// Set the working mode to AI.
.setTextureMode(Modeling3dTextureConstants.AlgorithmMode.AI) 
.create();

3.Create a listener callback to process the image upload result.

Modeling3dTextureUploadListener uploadListener = new Modeling3dTextureUploadListener() { 
    public void onResult(String taskId, Modeling3dTextureUploadResult result, Object ext) { 
        // Obtain the image upload result.
        if (result.isComplete()) { 
            // Process the upload result.
        } 
    } 
    @Override 
    public void onError(String taskId, int errorCode, String message) { 
        // Callback when an error occurs during upload.
    } 
    @Override 
    public void onUploadProgress(String taskId, double progress, Object ext) { 
        // Reserved.
    } 
};

4.Upload the collected images to the cloud.

// Obtain the ID of a material generation task and pass it to the configurator.
Modeling3dTextureInitResult modeling3dTextureInitResult = engine.initTask(setting); 
String taskId = modeling3dTextureInitResult.getTaskId(); 
if (taskId == null || taskId.equals("")) { 
    Log.e("", "get taskId error " + modeling3dTextureInitResult.getRetMsg()); 
} else { 
    // Set the upload listener.
    engine.setTextureUploadListener(uploadListener); 
    // Upload the images in asynchronous mode, passing the task ID and directory of the images.
    engine.asyncUploadFile(taskId, filePath); 
}

5.Query the progress of an on-cloud material generation task.

// Create a material generation task processing instance and pass the current context.
Modeling3dTextureTaskUtils taskUtils = Modeling3dTextureTaskUtils.getInstance(context); 
// Query the material generation progress.
Modeling3dTextureQueryResult queryResult = taskUtils.queryTask(taskId);

6.Create a listener callback to process the download result of generated texture maps.

Modeling3dTextureDownloadListener downloadListener = new Modeling3dTextureDownloadListener() { 
    public void onResult(String taskId, Modeling3dTextureDownloadResult result, Object ext) { 
        // Obtain the download result of generated texture maps.
        if (result.isComplete()) { 
            // Process the download result.
        } 
    } 
    @Override 
    public void onError(String taskId, int errorCode, String message) { 
        // Callback when an error occurs during download.
    } 
    @Override 
    public void onDownloadProgress(String taskId, double progress, Object ext) { 
        // Reserved.
    } 
};

7.Download the generated texture maps.

// Set the download listener.
engine.setTextureDownloadListener(downloadListener); 
// Download the texture maps, passing the task ID and texture map path.
engine.asyncDownloadTexture(taskId, savePath);

8.Call the synchronous API to obtain the generated texture maps in real time.

// Call the synchronous API, passing the image file path, texture map path, and configurator.
int result = engine.syncGenerateTexture(filePath, downloadPath, setting);

9.Delete the material generation task.

int ret = taskUtils.deleteTask(taskId);

Amazing capability, right? To learn more about 3D Modeling Kit, check out:

References

Official website of 3D Modeling Kit
Development guide to the kit
HMS Core section on HUAWEI Developer Forum
GitHub and Gitee to obtain the open-source code for the kit
Stack Overflow where you can find solutions to integration-related problems