Stefanos Kouroupis

Posted on Jul 25, 2019

Getting started with native openGL Android App

#java #android #opengl

This post is yet another transfer from Medium. I decided a while ago to delete my account there, but before that It would be a waste not to transfer (and improve) some of my posts from there. This is one of them.

During the holidays (christmas 2018) I wanted to try something different. I never did much mobile development in my life. But since I had a bit of time I thought I ought to give it a go.

But what's the fun of doing something simple, like a form app. I wanted to learn something, so I decided to do a simple OpenGL app without using any third party libraries. Everything from scratch.

Bear in my mind that, once upon a time I was kind of comfortable with OpenGL development (using C++).

I took a look around for tutorials, I found bits and pieces here and there and finally managed to create what I wanted.

These were the things I wanted to achieve:

Draw a circle
Make the circle move as you tilt your phone.

Here is what I had in mind

Which was a great opportunity to learn two things

How sensors work
How OpenGL works on Android.

Setting up the project

So I downloaded Android Studio and Created an Empty Project.

At this point I’ve read some articles and tutorials so I had some vague Idea what I had to do. I had to create a View and a Renderer.

My OpenGLView needs to extend GLSurfaceView.

public class OpenGLView extends GLSurfaceView {

    public OpenGLView(Context context) {
        super(context);
        init();
    }

    public OpenGLView(Context context, AttributeSet attrs) {
        super(context, attrs);
        init();
    }

    private void init(){
        setEGLContextClientVersion(2);  // OpenGL ES Version
        setPreserveEGLContextOnPause(true);
        setRenderer(new OpenGLRenderer());
    }
}

Then my OpenGLRenderer had to implement GLSurfaceView.Renderer.

public class OpenGLRenderer implements GLSurfaceView.Renderer {
    @Override
    public void onSurfaceCreated(GL10 gl, EGLConfig config) {
        GLES20.glClearColor(0.9f, 0.9f,0.9f,1f);
    }
   @Override
    public void onSurfaceChanged(GL10 gl, int width, int height) {
    }
    @Override
    public void onDrawFrame(GL10 gl) {
        GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);
    }
}

The Renderer at this point doesn’t do anything apart from setting up a background color.

Then we need to go to our layout and add the view we just created. So that the view element would fill our screen.

<net.something.OpenGLPlayground.OpenGLView
        android:id="@+id/openGLView"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

Finally in our MainActivity (which I named MainGameActivity) should look like this:

public class MainGameActivity extends AppCompatActivity {
private OpenGLView openGLView;
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main_game);
        openGLView = findViewById(R.id.openGLView);
    }
    @Override
    protected void onResume(){
        super.onResume();
        openGLView.onResume();
    }
    @Override
    protected void onPause(){
        super.onPause();
        openGLView.onPause();
    }
    @Override
    public void onPointerCaptureChanged(boolean hasCapture) {
    }
}

…and if we execute the code. we would get a view with whatever color we defined. The color I defined is nearly white, so there shouldn’t be anything. 🎉

Implementing SensorEventListener

Before we draw our circle lets set up the SensorEventListener to listen to the Accelerometer. Ok, the accelerometer might not be the best sensor for what we try to achieve (cause it will only work when you are not moving) so we could switch to the gyroscope but for the time, I guess, it's fine.

public class MainGameActivity extends AppCompatActivity implements SensorEventListener {
...
    @Override
    public void onSensorChanged(SensorEvent event) {
        if (event.sensor.getType() == Sensor.TYPE_ACCELEROMETER) {
        }
    }
    @Override
    public void onAccuracyChanged(Sensor sensor, int accuracy) {

    }
}

The accelerometer has 3 values

value[0] affects the x axis
value[1] affects the y axis
value[2] affects the z axis

We will only use x and y. We will also round the values on the second decimal since, we specifically, don’t want high accuracy. Our function will look like:

@Override
    public void onSensorChanged(SensorEvent event) {
        if (event.sensor.getType() == Sensor.TYPE_ACCELEROMETER) {
          float x = Math.round(event.values[0] * 100.0) / 100f;
          float y = Math.round(event.values[1] * 100.0) / 100f;
    }
}

Drawing the circle

So we have some coordinates for our circle. Now we need to use the powers of OpenGL to draw the circle.

We are going to create a Circle Class with the following functions.

public class Circle {
    // basically a circle is a linestring so we need its centre
    // radius and how many segments it will consist of
    public Circle(float cx, float cy, float radius, int segments) {
    }

    // calculate the segments
    public void calculatePoints(float cx, float cy, float radius, int segments) {
    }
    // actuall openGL drawing
    public void draw() {
    }
}

In our Circle Class we are going to add a function that compiles our shape shader. Basically shaders needs to be compiled by openGL.

public static int loadShader(int type, String shaderCode){
    int shader = GLES20.glCreateShader(type);
    GLES20.glShaderSource(shader, shaderCode);
    GLES20.glCompileShader(shader);
    return shader;
}

In the Circle Class we will define two shaders (which don’t do much).
Vertex Shader : for rendering the vertices of a shape.
Fragment Shader : for rendering the face of a shape with colors or textures.

private final String vertexShaderCode =
        "attribute vec4 vPosition;" +
                "void main() {" +
                "  gl_Position = vPosition;" +
                "}";

private final String fragmentShaderCode =
        "precision mediump float;" +
                "uniform vec4 vColor;" +
                "void main() {" +
                "  gl_FragColor = vColor;" +
                "}";

It time to calculate the points of the circle. We will store these points in a FloatBuffer.

A simple glance of the following code will reveal something a bit weird. That’s the DisplayMetrics. The issue here is that’s due to OpenGL canvas is square, and the mapping of the screen coordinates span from -1 to 1. If we just drew the circle it would end up distorted. We need the width and height of the screen to calculate the aspect ratio so we can squeeze one dimension, and produce an actual circle.

private FloatBuffer vertexBuffer;
private static final int COORDS_PER_VERTEX = 3;
public void CalculatePoints(float cx, float cy, float radius, int segments) {
    DisplayMetrics dm = Resources.getSystem().getDisplayMetrics();

    float[] coordinates = new float[segments * COORDS_PER_VERTEX];

    for (int i = 0; i < segments * 3; i += 3) {
        float percent = (i / (segments - 1f));
        float rad = percent * 2f * (float) Math.PI;

        //Vertex position
        float xi = cx + radius * (float) Math.cos(rad);
        float yi = cy + radius * (float) Math.sin(rad);

        coordinates[i] = xi;
        coordinates[i + 1] = yi / (((float) dm.heightPixels / (float) dm.widthPixels));
        coordinates[i + 2] = 0.0f;
    }

    // initialise vertex byte buffer for shape coordinates
    ByteBuffer bb = ByteBuffer.allocateDirect(coordinates.length * 4);
    // use the device hardware's native byte order
    bb.order(ByteOrder.nativeOrder());

    // create a floating point buffer from the ByteBuffer
    vertexBuffer = bb.asFloatBuffer();
    // add the coordinates to the FloatBuffer
    vertexBuffer.put(coordinates);
    // set the buffer to read the first coordinate
    vertexBuffer.position(0);
}

Time to implement the constructor. In cases of:

let's just want to draw a shape and be over.

We wouldn't need to check if the app has being initialised, but because we intend to update the objects coordinates every time we get a sensor event. We shouldn't load the shader/app/link more than once.

private int app = -1;
public Circle(float cx, float cy, float radius, int segments) {
    CalculatePoints(cx, cy, radius, segments);
    if (app == -1) {
        int vertexShader = OpenGLRenderer.loadShader(GLES20.GL_VERTEX_SHADER,
                vertexShaderCode);
        int fragmentShader = OpenGLRenderer.loadShader(GLES20.GL_FRAGMENT_SHADER,
                fragmentShaderCode);

        // create empty OpenGL ES Program
        app = GLES20.glCreateProgram();

        // add the vertex shader to program
        GLES20.glAttachShader(app, vertexShader);

        // add the fragment shader to program
        GLES20.glAttachShader(app, fragmentShader);

        // creates OpenGL ES program executables
        GLES20.glLinkProgram(app);
    }
}

The draw function

public void draw() {

    int vertexCount = vertexBuffer.remaining() / COORDS_PER_VERTEX;

    // Add program to the environment
    GLES20.glUseProgram(app);

    // get handle to vertex shader's vPosition member
    int mPositionHandle = GLES20.glGetAttribLocation(app, "vPosition");

    // Enable a handle to the triangle vertices
    GLES20.glEnableVertexAttribArray(mPositionHandle);

    // Prepare the triangle coordinate data
    GLES20.glVertexAttribPointer(mPositionHandle, COORDS_PER_VERTEX,
        GLES20.GL_FLOAT, false,
        vertexStride, vertexBuffer);

    // get handle to fragment shader's vColor member
    mColorHandle = GLES20.glGetUniformLocation(app, "vColor");

    // Draw the triangle, using triangle fan is the easiest way
    GLES20.glDrawArrays(GLES20.GL_TRIANGLE_FAN, 0, vertexCount);

    // Disable vertex array
    GLES20.glDisableVertexAttribArray(mPositionHandle);

    // Set color of the shape (circle)
    GLES20.glUniform4fv(mColorHandle, 1, new float[]{0.5f, 0.3f, 0.1f, 1f}, 0);
}

Finally we go back to our Renderer and add a new circle object. We will draw our circle initially at x = 0, y = 0, radius = 0.1 with segments = 55. *objectsReady * switch will come in handy when we will start updating the object from the sensor events.

public class OpenGLRenderer implements GLSurfaceView.Renderer {
    private Circle circle;
    public boolean objectsReady = false;
    public Circle getCircle() {
        return circle;
    }
    @Override
    public void onSurfaceCreated(GL10 gl, EGLConfig config) {
        GLES20.glClearColor(0.9f, 0.9f,0.9f,1f);
        circle = new Circle(0,0, 0.1f, 55);
        objectsReady = true;
    }
...
}

At this point if we run our app we should get a brown circle in the middle of our screen. So our onSensorChanged will become. SCALE is used to map the sensor data we need (-4, 4) to our OpenGL view (-1,1).

private final static int SCALE = 4;
@Override
    public void onSensorChanged(SensorEvent event) {
        if (event.sensor.getType() == Sensor.TYPE_ACCELEROMETER) {
          float x = Math.round(event.values[0] * 100.0) / 100f;
          float y = Math.round(event.values[1] * 100.0) / 100f;
if (openGLView.renderer.objectsReady) {
              openGLView.renderer.getCircle().CalculatePoints(x /    SCALE, y / SCALE, 0.1f, 55);
              openGLView.requestRender();
          }
    }
}

Finally our circle, is alive and well, but its motion is jitterish.

Normalise sensor data

SensorEventListener fires thousand events per second with a huge accuracy, so in order to make movement smooth, we need to normalise our data by using some statistical method. The obvious choice with this problem (at least obvious to me) is to use moving average.

The moving average is nothing more than the average value of the x last readings.
This is easily done. We only need to add the following to our MainActivity.

private final static int OVERFLOW_LIMIT = 20;
private float[][] movingAverage = new float[2][OVERFLOW_LIMIT];
@Override
public void onSensorChanged(SensorEvent event) {
    if (event.sensor.getType() == Sensor.TYPE_ACCELEROMETER) {
        float x = Math.round(event.values[0] * 100.0) / 100f;
        float y = Math.round(event.values[1] * 100.0) / 100f;

        movingAverage[0][overflow] = x;
        movingAverage[1][overflow] = y;

        float s1 = calculateAverage(movingAverage[0]);
        float s2 = calculateAverage(movingAverage[1]);

        if (openGLView.renderer.objectsReady) {
            openGLView.renderer.getCircle().CalculatePoints(s1 / SCALE, s2 / SCALE, 0.1f, 55);
            openGLView.requestRender();
         }
    }
    overflow += 1;
    if (overflow >= OVERFLOW_LIMIT) {
        overflow = 0;
    }
}
private float calculateAverage(float[] input) {
    DoubleStream io = IntStream.range(0, input.length)
            .mapToDouble(i -> input[i]);
    float sum = (float)io.sum();
    return sum/OVERFLOW_LIMIT;
}