DEV Community

Cover image for Navigating the AR Landscape in React Native with Kotlin
Pascal C
Pascal C

Posted on • Updated on

Navigating the AR Landscape in React Native with Kotlin


In the first part of this series, we laid the groundwork by setting up Kotlin within our React Native environment, a crucial step in bringing augmented reality (AR) to our applications. This segment is designed for developers who are ready to take the next step in their AR journey, willing to learn how to integrate ARCore into their hybrid apps.

This is part two of a multi part article series about the integration of AR services via native modules into a React Native app. You will learn how to integrate ARCore and how to display a 3D-model in this article.

The full code for this part is here.

What is ARCore

ARCore is Google's platform for building augmented reality (AR) experiences. It enables phones to understand and interact with the real world. Using three key capabilities – motion tracking, environmental understanding, and light estimation – ARCore makes it possible for apps to merge virtual content with the real world in a seamless and realistic way.

Add dependencies to the project

First we need to add the necessary dependencies to our project. Go into your android/app/build.gradle and add the following implementations to your dependencies section:

// AR
implementation ''
Enter fullscreen mode Exit fullscreen mode

It should look like this:

app level build.gradle file

Then we have to bump our minSdkVersion to 24 in our android/build.gradle. Afterwards sync the project.

Download a model

Now we need a 3D model to display. Download one from here for example. Make sure to select the GLB version!

Add file to Android

To add a GLB model to your Android project in Android Studio, you should create an assets folder and place your GLB file there. Here's how to do it:

  1. Create the assets Folder:
  • Right-click on the app folder in the Android Studio project view.
  • Go to New > Folder > Assets Folder.
  • Click Finish in the dialog that appears to create the folder.
  1. Add Your GLB File:
  • Copy your GLB model file.
  • Paste it into the assets folder you just created.

Create an Activity XML file

Now we need to create the layout file where our model will be displayed:

  1. Locate the res (Resources) Folder: In your project hierarchy in Android Studio, it is typically found under app > src > main.

  2. Create the layout Folder:

  • Right-click on the res folder.
  • Choose New > Directory.
  • Name the directory layout.
  1. Add Your XML File:
  • Right-click on the newly created layout folder.
  • Choose New > Layout resource file.
  • Name the file ar_activity.xml and click 'OK'.

Android Studio will open the file in Design mode, but we will need to change into Code mode. In order to do this, click on the Code button (should be in the upper right and looks like 5 vertical lines). Copy the following code into the file:

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android=""

        tools:layout_editor_absoluteY="16dp" />

        app:layout_constraintTop_toTopOf="@+id/sceneView" />

        tools:layout_editor_absoluteX="365dp" />
Enter fullscreen mode Exit fullscreen mode

If there is a warning / error about a missing package `androidx.constraintlayout.widget.ConstraintLayout´ right click on it, select Show Context Actions and select Add dependency. Android Studio will then add an import and sync the project.

The button at the end is so that we can close the screen, when we want to return to the non-ar part of our app. In order for this button to be displayed correctly, we need to

Update the ARModule file

Now we can reference the created layout file in our module. Open the file ARModule.kt and replace the showAR method with the following:

  fun showAR(fileName: String) {
      val assetPath = "file:///android_asset/$fileName"
      val intent = Intent(reactContext,
        intent.putExtra("MODEL_PATH", assetPath)

        // If starting the activity from a non-activity context, set this flag

Enter fullscreen mode Exit fullscreen mode

We will pass the filename of our model to our native code, which will in turn load it from the assets folder and start a custom Activity to display the model. In simple terms, an activity represents a single screen with a user interface.

Create the Activity file

Right click on the com.rn3dworldexplorer folder and select New > Kotlin Class. Name the file ModelDisplayActivity.kt. Copy the following code into it:

package com.rn3dworldexplorer

import android.os.Bundle
import android.util.Log
import android.view.View
import android.widget.Button
import android.widget.ProgressBar
import android.widget.Toast
import io.github.sceneview.math.Position
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.launch

class ModelDisplayActivity : AppCompatActivity() {
    private lateinit var progressBar: ProgressBar
    private lateinit var closeButton: Button
    private lateinit var sceneView: ArSceneView
    private lateinit var modelNode: ArModelNode

    override fun onCreate(savedInstanceState: Bundle?) {

        // Initialize views
        closeButton = findViewById(
        progressBar = findViewById(
        sceneView = findViewById<ArSceneView>( {
            this.lightEstimationMode = Config.LightEstimationMode.DISABLED
        sceneView.onArSessionFailed = { exception: Exception ->
            Log.e("ARModule", "${exception.message}")
            // If AR is not available, we add the model directly to the scene for a 3D only usage

        // Set close button listener
        closeButton.setOnClickListener {

        // Load model if path is provided
        intent.getStringExtra("MODEL_PATH")?.let { modelPath ->

        } ?: showMessage("Model path is null")

    private fun loadModel(glbFileLocation: String) {
        Log.d("ARModule", "loadModel path: $glbFileLocation")
        progressBar.visibility = View.VISIBLE

        CoroutineScope(Dispatchers.Main).launch {
            try {
                modelNode = ArModelNode(sceneView.engine, PlacementMode.INSTANT).apply {
                        glbFileLocation = glbFileLocation,
                        scaleToUnits = 1f,
                        centerOrigin = Position(0.0f)
                        sceneView.planeRenderer.isVisible = true
                        val materialInstance = it.materialInstances[0]
                sceneView.planeRenderer.isVisible = false
            } catch (e: Exception) {
                showMessage("Error loading model: ${e.message}")
                Log.e("ARModule", "Error occurred: ${e.message}")
            } finally {
                progressBar.visibility = View.GONE

    private fun showMessage(message: String) {
        Toast.makeText(this, message, Toast.LENGTH_LONG).show()
Enter fullscreen mode Exit fullscreen mode

Here is a brief overview of the file:


  • UI Elements: The class defines UI elements like a ProgressBar, a Button (closeButton), and an ArSceneView (for AR content display).
  • Model Node: modelNode of type ArModelNode is used to represent the 3D model in AR.

onCreate Method

  • UI Setup: In the onCreate method (called when the activity is starting), it sets up the user interface from a layout resource (R.layout.ar_activity).
  • Initializing Views: Finds and initializes views like the close button, progress bar, and AR scene view.
  • AR Session Failure Handling: Sets an error handling mechanism for the AR session. If AR is not available, it directly adds the model to the scene for non-AR 3D viewing.
  • Close Button Listener: Implements an action for the close button - when clicked, it finishes (closes) the activity.
  • Model Loading: Checks if a model path is provided via the intent; if so, it calls loadModel to load the model.

loadModel Method

  • Functionality: This method loads the 3D model (GLB format) for display in the AR scene.
  • Coroutine: Uses a coroutine for asynchronous execution to avoid blocking the UI thread.
  • Model Node Setup: Configures the modelNode with the model path, scale, and origin. Once the model is loaded, it is anchored to a position in the AR scene.
  • Visibility Settings: Manages the visibility of the plane renderer and progress bar based on the model loading status.
  • Error Handling: Catches exceptions, logs errors, and shows error messages if model loading fails.

showMessage Method

Functionality: A utility method to show a toast message on the screen. It's used to provide feedback to the user (like successful model loading or error messages).

Added the necessary permissions to the Androidmanifest.xml file

Replace the contents of your AndroidManifest.xml with this one:

<manifest xmlns:android="" xmlns:tools="">

    <uses-permission android:name="android.permission.INTERNET" />
    <uses-permission android:name="android.permission.CAMERA" /> <!-- You may need these if doing any screen recording from within the app -->
        android:version="1" />

    <uses-permission android:name="android.permission.NFC" />
    <uses-permission android:name="android.permission.WAKE_LOCK" /> <!-- Other camera related features -->
    <uses-feature android:name="" />
        tools:replace="required" />
    <uses-feature android:name="" /> <!-- Specifying OpenGL version or requirements -->
        tools:replace="required" /> <!-- Usage of accelerometer and gyroscope -->
        tools:replace="required" />
        tools:replace="required" />

            <action android:name="android.intent.action.MAIN" />
            <category android:name="android.intent.category.LAUNCHER" />
        <activity android:name=".ModelDisplayActivity" />

            tools:replace="android:value" />
Enter fullscreen mode Exit fullscreen mode

We request several device features needed for AR like the gyroscope as well as register our custom activity via this line: <activity android:name=".ModelDisplayActivity" />. Not all of them may be used in the end, so it is your responsibility to choose carefully. This list just aims to be as comprehensive as possible out of convenience.

Adjust our JavaScript React code

We are finally back in the cozy surroundings of your JavaScript code. All that is left is to adjust our ARModule.tsx and our App.tsx. For the ARModule file, adjust your showAR function to this showAR(path: string): Promise<void>; to reflect the new type signature. After that, in the App.tsx you can adjust the onPress function of the button to pass the name of your chosen model. For me it is like this:
onPress={async () => await ARModule.showAR("AR-model.glb")}.

Now just press the button and it should look similar to this:

3D Sphere displayed on mobile phone


As we conclude this second part of our series, we have navigated through the integration of ARCore in a React Native environment via Kotlin. It can be pretty overwhelming if you haven't delved too deep into native code integration, but hopefully, it was manageable. From here on out, you can expand on the basic functionality and create your own features.

Moving forward, we will be doing the setup for native modules using Objective C in iOS in the next entry.

Top comments (0)