Detección de estado de salud con AR Engine

Introducción

AR Engine tiene soporte para detectar tu rostro y obtener datos saludables como la frecuencia cardíaca o tu edad.

¿Qué es el motor HUAWEI AR?

HUAWEI AR Engine es una plataforma para crear aplicaciones de realidad aumentada (AR) en teléfonos inteligentes Android. Se basa en el conjunto de chips HiSilicon e integra algoritmos centrales de AR para proporcionar capacidades AR básicas como seguimiento de movimiento, seguimiento del entorno, seguimiento del cuerpo y seguimiento de rostros, lo que permite que su aplicación conecte el mundo virtual con el mundo real, para una visión completamente nueva. experiencia de usuario interactiva.
Actualmente, HUAWEI AR Engine ofrece tres tipos de capacidades, que incluyen seguimiento de movimiento, seguimiento de entorno y seguimiento de rostro y cuerpo humano.

Ejemplo de aplicación de Android

Para este ejemplo, trabajaremos en el seguimiento del entorno para que podamos detectar una mano e interactuar con ella.

Proceso de desarrollo

Crear una aplicación
Cree una aplicación siguiendo las instrucciones de Creating an AppGallery Connect Project y Adding an App to the Project.
Platforma: Android
Dispositivo: Mobile phone
Categoria de la App: App or Game

Integración de HUAWEI AR Engine SDK
Antes del desarrollo, integre HUAWEI AR Engine SDK a través del repositorio de Maven en su entorno de desarrollo.
Abra el archivo build.gradle en el directorio raíz de su proyecto de Android Studio.

apply plugin: 'com.android.application'
android {
compileSdkVersion 30
buildToolsVersion "30.0.1"
defaultConfig {
applicationId "com.vsm.myarapplication"
minSdkVersion 27
targetSdkVersion 30
versionCode 1
versionName "1.0"
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
}
buildTypes {
release {
minifyEnabled false
proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
}
}
}
dependencies {
implementation fileTree(dir: "libs", include: ["*.jar"])
implementation 'androidx.appcompat:appcompat:1.2.0'
implementation 'androidx.constraintlayout:constraintlayout:2.0.1'
testImplementation 'junit:junit:4.12'
//
implementation 'com.huawei.agconnect:agconnect-core:1.4.1.300'
//
implementation 'com.huawei.hms:arenginesdk:2.15.0.1'
//
implementation 'de.javagl:obj:0.3.0'
androidTestImplementation 'androidx.test.ext:junit:1.1.2'
androidTestImplementation 'androidx.test.espresso:espresso-core:3.3.0'
}
apply plugin: 'com.huawei.agconnect'

Creas tu actividad en la que trabajarás (activity_health.xml):

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".healt.HealtActivity">
<android.opengl.GLSurfaceView
        android:id="@+id/healthSurfaceView"
        android:layout_width="fill_parent"
        android:layout_height="fill_parent"
        tools:ignore="MissingConstraints" />
<LinearLayout
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:orientation="horizontal">
<ImageView
            android:layout_width="1dp"
            android:layout_height="match_parent"
            android:layout_gravity="start"
            android:layout_weight="1"
            android:scaleType="fitXY"
            android:src="@drawable/face_bg_fill" />
<ImageView
            android:id="@+id/health_fresh_face"
            android:layout_width="wrap_content"
            android:layout_height="match_parent"
            android:layout_gravity="center_horizontal"
            android:adjustViewBounds="true"
            android:scaleType="fitCenter"
            android:src="@drawable/face_img_mask" />
<ImageView
            android:layout_width="1dp"
            android:layout_height="match_parent"
            android:layout_gravity="end"
            android:layout_weight="1"
            android:scaleType="fitXY"
            android:src="@drawable/face_bg_fill" />
    </LinearLayout>
<TextView
        android:id="@+id/health_check_status"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text=""
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        tools:ignore="MissingConstraints" />
<ProgressBar
        android:id="@+id/health_progress_bar"
        style="?android:attr/progressBarStyleHorizontal"
        android:layout_width="200dp"
        android:layout_height="20dp"
        android:layout_centerInParent="true"
        android:max="100"
        android:progress="0"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent"
        app:layout_constraintVertical_bias="0.6" />
<TextView
        android:id="@+id/process_tips"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text=""
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toBottomOf="@+id/health_progress_bar"
        tools:ignore="MissingConstraints" />
<TableLayout
        android:id="@+id/health_param_table"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginBottom="50dp"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"></TableLayout>
</androidx.constraintlayout.widget.ConstraintLayout>

AR Engine no es para todos los dispositivos, por lo que primero debemos validar si el dispositivo es compatible con AR Engine y está disponible, aquí está la lista de dispositivos compatibles

Utilizará estos métodos para comprobar si un dispositivo es compatible:

private boolean arEngineAbilityCheck() {
boolean isInstallArEngineApk = AREnginesApk.isAREngineApkReady(this);
if (!isInstallArEngineApk && isRemindInstall) {
Toast.makeText(this, "Please agree to install.", Toast.LENGTH_LONG).show();
finish();
}
Log.d(TAG, "Is Install AR Engine Apk: " + isInstallArEngineApk);
if (!isInstallArEngineApk) {
startActivity(new Intent(this, ConnectAppMarketActivity.class));
isRemindInstall = true;
}
return AREnginesApk.isAREngineApkReady(this);
}

MessageWhenError

private void setMessageWhenError(Exception catchException) {
if (catchException instanceof ARUnavailableServiceNotInstalledException) {
startActivity(new Intent(getApplicationContext(), ConnectAppMarketActivity.class));
} else if (catchException instanceof ARUnavailableServiceApkTooOldException) {
message = "Please update HuaweiARService.apk";
} else if (catchException instanceof ARUnavailableClientSdkTooOldException) {
message = "Please update this app";
} else if (catchException instanceof ARUnSupportedConfigurationException) {
message = "The configuration is not supported by the device!";
} else {
message = "exception throw";
}
}

En nuestro HealthActivity.java llamará a la detección de superficie:

public class HealtActivity extends AppCompatActivity {
private static final String TAG = HealtActivity.class.getSimpleName();
private static final int MAX_PROGRESS = 100;
private GLSurfaceView mGlSurfaceView;
private ARSession mArSession;
private ARFaceTrackingConfig mArFaceTrackingConfig;
private String mMessage;
private boolean isRemindInstall = false;
private HealthRenderManager mHealthRenderManager;
private DisplayRotationManager mDisplayRotationManager;
private ProgressBar mHealthProgressBar;
private TextView mProgressTips;
private TextView mHealthCheckStatusTextView;
@Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_healt);
        AppCompatDelegate.setCompatVectorFromResourcesEnabled(true);
        mHealthProgressBar = findViewById(R.id.health_progress_bar);
        mGlSurfaceView = findViewById(R.id.healthSurfaceView);
        mProgressTips = findViewById(R.id.process_tips);
        mHealthCheckStatusTextView = findViewById(R.id.health_check_status);
        mDisplayRotationManager = new DisplayRotationManager(this);
mGlSurfaceView.setPreserveEGLContextOnPause(true);
// Set the OpenGLES version.
        mGlSurfaceView.setEGLContextClientVersion(2);
// Set the EGL configuration chooser, including for the
        // number of bits of the color buffer and the number of depth bits.
        mGlSurfaceView.setEGLConfigChooser(8, 8, 8, 8, 16, 0);
        mHealthRenderManager = new HealthRenderManager(this, this);
        mHealthRenderManager.setDisplayRotationManage(mDisplayRotationManager);
        TableLayout mHealthParamTable = findViewById(R.id.health_param_table);
        mHealthRenderManager.setHealthParamTable(mHealthParamTable);
        mGlSurfaceView.setRenderer(mHealthRenderManager);
        mGlSurfaceView.setRenderMode(GLSurfaceView.RENDERMODE_CONTINUOUSLY);
    }
@Override
    protected void onResume() {
        super.onResume();
        mMessage = null;
        if (mArSession == null) {
            try {
                if (!arEngineAbilityCheck()) {
                    finish();
                    return;
                }
                mArSession = new ARSession(this);
                mArFaceTrackingConfig = new ARFaceTrackingConfig(mArSession);
                mArFaceTrackingConfig.setEnableItem(ARConfigBase.ENABLE_HEALTH_DEVICE);
                mArSession.configure(mArFaceTrackingConfig);
                setHealthServiceListener();
            } catch (ARUnavailableServiceNotInstalledException capturedException) {
                startActivity(new Intent(this, ConnectAppMarketActivity.class));
            } catch (ARUnavailableServiceApkTooOldException capturedException) {
                mMessage = "Please update HuaweiARService.apk";
            } catch (ARUnavailableClientSdkTooOldException capturedException) {
                mMessage = "Please update this app";
            } catch (ARUnSupportedConfigurationException capturedException) {
                mMessage = "The configuration is not supported by the device!";
            } catch (Exception capturedException) {
                mMessage = "unknown exception throws!";
            }
            if (mMessage != null) {
                stopArSession();
                return;
            }
        }
        try {
            mArSession.resume();
        } catch (ARCameraNotAvailableException e) {
            Toast.makeText(this, "Camera open failed, please restart the app", Toast.LENGTH_LONG).show();
            mArSession = null;
            return;
        }
        mDisplayRotationManager.registerDisplayListener();
        mHealthRenderManager.setArSession(mArSession);
        mGlSurfaceView.onResume();
    }
private void stopArSession() {
        Log.i(TAG, "Stop session start.");
        Toast.makeText(this, mMessage, Toast.LENGTH_LONG).show();
        if (mArSession != null) {
            mArSession.stop();
            mArSession = null;
        }
        Log.i(TAG, "Stop session end.");
    }
/**
     * Check whether HUAWEI AR Engine server (com.huawei.arengine.service) is installed on the current device.
     * If not, redirect the user to HUAWEI AppGallery for installation.
     *
     * @return true:AR Engine ready
     */
    private boolean arEngineAbilityCheck() {
        boolean isInstallArEngineApk = AREnginesApk.isAREngineApkReady(this);
        if (!isInstallArEngineApk && isRemindInstall) {
            Toast.makeText(this, "Please agree to install.", Toast.LENGTH_LONG).show();
            finish();
        }
        Log.d(TAG, "Is Install AR Engine Apk: " + isInstallArEngineApk);
        if (!isInstallArEngineApk) {
            startActivity(new Intent(this, ConnectAppMarketActivity.class));
            isRemindInstall = true;
        }
        return AREnginesApk.isAREngineApkReady(this);
    }
@Override
    protected void onPause() {
        Log.i(TAG, "onPause start.");
        super.onPause();
        if (mArSession != null) {
            mDisplayRotationManager.unregisterDisplayListener();
            mGlSurfaceView.onPause();
            mArSession.pause();
            Log.i(TAG, "Session paused!");
        }
        Log.i(TAG, "onPause end.");
    }
@Override
    protected void onDestroy() {
        Log.i(TAG, "onDestroy start.");
        super.onDestroy();
        if (mArSession != null) {
            mArSession.stop();
            mArSession = null;
        }
        Log.i(TAG, "onDestroy end.");
    }
@Override
    public void onWindowFocusChanged(boolean isHasFocus) {
        Log.d(TAG, "onWindowFocusChanged");
        super.onWindowFocusChanged(isHasFocus);
        if (isHasFocus) {
            getWindow().getDecorView()
                    .setSystemUiVisibility(View.SYSTEM_UI_FLAG_LAYOUT_STABLE | View.SYSTEM_UI_FLAG_LAYOUT_HIDE_NAVIGATION
                            | View.SYSTEM_UI_FLAG_LAYOUT_FULLSCREEN | View.SYSTEM_UI_FLAG_HIDE_NAVIGATION
                            | View.SYSTEM_UI_FLAG_FULLSCREEN | View.SYSTEM_UI_FLAG_IMMERSIVE_STICKY);
        }
    }
private void setHealthServiceListener() {
        mArSession.addServiceListener(new FaceHealthServiceListener() {
            @Override
            public void handleEvent(EventObject eventObject) {
                if (!(eventObject instanceof FaceHealthCheckStateEvent)) {
                    return;
                }
                final FaceHealthCheckState faceHealthCheckState =
                        ((FaceHealthCheckStateEvent) eventObject).getFaceHealthCheckState();
                runOnUiThread(new Runnable() {
                    @Override
                    public void run() {
                        mHealthCheckStatusTextView.setText(faceHealthCheckState.toString());
                    }
                });
            }
@Override
            public void handleProcessProgressEvent(final int progress) {
                mHealthRenderManager.setHealthCheckProgress(progress);
                runOnUiThread(new Runnable() {
                    @Override
                    public void run() {
                        setProgressTips(progress);
                    }
                });
            }
        });
    }
private void setProgressTips(int progress) {
        String progressTips = "processing";
        if (progress >= MAX_PROGRESS) {
            progressTips = "finish";
        }
        mProgressTips.setText(progressTips);
        mHealthProgressBar.setProgress(progress);
    }
}

Conclusión

Puede detectar la frecuencia cardíaca, conocer la edad de su rostro y obtener más datos sobre la salud para múltiples propósitos de una manera sencilla.
Documentation
Codelab
Code Sample

Explora más artículos en el Huawei Developer Forum.
Conviértete en Huawei Developer, es gratis.
Conviértete en Huawei Developer, es gratis.