Building AR Experiences with React Native: Bridging the Gap with Kotlin

Introduction

Recently I got assigned an interesting project which required having AR capabilities in a React Native app for cross platform purposes. The initial idea was integrating ViroReact, but this very quickly turned out to be a bad idea. Missing features for the 3D-Models (model animations, ...) and it's general incompatibility with the newest React Native Version (0.72) made it unsufficient for our use case. So the decision was made to implement the functionality via native modules. And so began my descent into dependency hell 💾🔥...

This is part one of a multi part article series about the integration of AR services via native modules into a React Native app. The first half of this series will be about the integration into an Android app, using ARCore and SceneView. In this part, you will learn how to set up Kotlin to write native modules.

• Part Two: Integrating ARCore
• Part Three: Setting up iOS with Objective-C
• Part Four: Integrating ARKit

You can follow along or download the full code of part one here.

Coding

Prerequisites

I am using the React Native CLI for this project on a real device (simulators don't work well with AR APIs). Make sure to have finished the setup for new React Native apps on Android. Also download Android Studio. You will need it.

Setting up a new React Native project

Open up your favorite terminal on the OS of your choice (I am on Ubuntu 22.04.3 LTS) and use the following command to setup a new React Native project:

npx react-native init RN3DWorldExplorer

Then add the folder to the workspace of your favorite editor and change into the directory via cd RN3DWorldExplorer.

Kotlin

When writing native modules, you have the choice to use Kotlin or Java. We will be using Kotlin for this tutorial.

Kotlin is a modern but already mature programming language. It's concise, safe, interoperable with Java and other languages, and provides many ways to reuse code between multiple platforms for productive programming.

Unfortunately, it does not work out of the box (This may change in future React Native versions). We have to do the following steps to make it work with React Native:

• Add kotlinVersion = "1.8.0" to the buildscript { ext section of your android/build.gradle nad classpath("org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlinVersion") to the dependencies section. It should look like this:
  

• Add apply plugin: 'kotlin-android' to the top of your android/app/build.gradle

• Add implementation "org.jetbrains.kotlin:kotlin-stdlib:$kotlinVersion" to the dependencies` section of the same file

• Sync the project with Gradle via the elephant button in the upper right

Now try running the app via npm run start and type a for Android. The app should build and run now.

Bridging native modules to React Native

Creating a native module

Following the steps outlined from the docs, we will be creating our AR module.

For this, open up Android Studio, navigate via the project pane (upper left) to the app/java/com.rn3dworldexplorer folder, right click on it, select new -> Kotlin Class and name it ARModule.kt. Then add the following code to the file:
`

package com.rn3dworldexplorer

import android.content.Intent
import com.facebook.react.bridge.ReactApplicationContext
import com.facebook.react.bridge.ReactContextBaseJavaModule
import com.facebook.react.bridge.ReactMethod

class ARModule(private val reactContext: ReactApplicationContext) : ReactContextBaseJavaModule(reactContext) {
    override fun getName(): String {
        return "ARModule"
    }

    @ReactMethod
    fun showAR(promise: Promise) {
       promise.resolve("Hello ARWorld")
    }
}

• With the getName() function, we will be able to access our module in our React Native code.
• The showAR() function takes a Promise as a parameter for now. The reason is that these methods can't return values directly to JavaScript due to the asynchronous nature of the bridge between native code and JavaScript. In part 2, we will integrate a custom activity to display a 3D-model instead.

Registering the native module

Now we need to register the module with React Native:

In order to do so, you need to add your native module to a ReactPackage and register the ReactPackage with React Native. During initialization, React Native will loop over all packages, and for each ReactPackage, register each native module within.

Create a new Kotlin file called ArPackage.kt and copy this code into it:

package com.rn3dworldexplorer

import android.view.View
import com.facebook.react.ReactPackage
import com.facebook.react.bridge.NativeModule
import com.facebook.react.bridge.ReactApplicationContext
import com.facebook.react.uimanager.ReactShadowNode
import com.facebook.react.uimanager.ViewManager

class GLBPackage : ReactPackage {
    override fun createViewManagers(
        reactContext: ReactApplicationContext
    ): MutableList<ViewManager<View, ReactShadowNode<*>>> = mutableListOf()


    override fun createNativeModules(
        reactContext: ReactApplicationContext
    ): MutableList<NativeModule> = listOf(GLBModule(reactContext)).toMutableList()
}

This file imports the created native AR - module and instantiates it via the createNativeModules function and returns it as a list of NativeModules to register.

Now we just must add the ARPackage.kt to ReactNativeHost's getPackages() method. Open up MainApplication.java and add this inside the method: packages.add(new ARPackage());

It should look like this:

This bridges our native module with the JavaScript part of our application. Sync the project one last time.

Add the native module to the app

It is best practice to create a file for every native module in your code. So in the root of your project, create a new folder modules and inside it a file named ARModule.tsx. Copy this code inside it:

import { NativeModules } from "react-native";
const { ARModule } = NativeModules;

interface ARModuleInterface {
  showAR(): Promise<string>;
}

export default ARModule as ARModuleInterface;

Doing it this way allows us to properly type our module and be able to import it anywhere simply via import ARModule from "./modules/ARModule";

Update App.tsx

Right now we have the boilerplate setup for new React Native projects in our App.tsx. Update the file with this code instead:

import React from "react";
import type { PropsWithChildren } from "react";
import {
  SafeAreaView,
  ScrollView,
  StatusBar,
  StyleSheet,
  Text,
  View,
  Button,
} from "react-native";
import ARModule from "./modules/ARModule";

type SectionProps = PropsWithChildren<{
  title: string,
  description: string,
}>;

const Section: React.FC<SectionProps> = ({ children, title, description }) => (
  <View style={styles.sectionContainer}>
    <Text style={[styles.sectionTitle]}>{title}</Text>
    <Text style={styles.sectionDescription}>{description}</Text>
    {children}
  </View>
);

const App: React.FC<{}> = () => (
  <SafeAreaView>
    <StatusBar barStyle="light-content" />
    <ScrollView contentInsetAdjustmentBehavior="automatic">
      <View>
        <Section title="AR" description="Render the native AR module">
          <Button
            title="Show AR"
            onPress={async () => {
              const res = await ARModule.showAR();
              console.log(res);
            }}
          />
        </Section>
      </View>
    </ScrollView>
  </SafeAreaView>
);

const styles = StyleSheet.create({
  sectionContainer: { marginTop: 32, paddingHorizontal: 24 },
  sectionTitle: { fontSize: 24, fontWeight: "600" },
  sectionDescription: {
    marginTop: 8,
    marginBottom: 16,
    fontSize: 18,
    fontWeight: "400",
  },
});

export default App;

Run the app now, press the "Show AR" button and you should see a console.log "Hello ARWorld" in your terminal.

Conclusion

This concludes the first part of this series on integrating AR services into React Native. This part focused on setting up a Kotlin environment to write native modules, a crucial step in bridging the gap between the high-level React Native framework and the low-level efficiency of native Android development. Looking ahead, the established groundwork here will allow us to dive into the world of AR development in Android. The next part of this series will explore the integration of ARCore and SceneView.