OpenGL and Android
In this chapter we will talk about working with OpenGL on the Android platform. You will know the background of OpenGL and OpenGL ES the variant that Android supports. In the current Android SDK, doucmentation on how to get started with Open GL is almost non existent. The few online resources available on OpenGL with Android assume a lot of knowledge of Open GL. In this chapter we will help you with both these problems. With few pre-requisites, we will walk you through to create an OpenGL ES test harness to start drawing and experimenting with the Open GL ES api. In the process we will also draw attention to the basics of Open GL and point you to resources available online to explore Open GL in further detail. By the end of this chapter we are confident that we will leave you well equipped with the ideas of drawing in three dimensions, setting up the camera, setting up the viewing volume (also called frustum). We will do this by introducing minimal or almost no mathematics which is normally the vogue with many OpenGL books.
OpenGL Background
"OpenGL" is a 2D and 3D graphics api, originally developed by SGI for their Unix workstations. It is widely adapted now on all operating systems. This standard forms the basis of much of the gaming industry, CAD, and even Virtual Reality. Although the SGI OpenGL has been around for a longer time period, the standardized first spec of OpenGL first showed up in 1992. The OpenGL standard is currently being managed by an industry consortium called the "Khronos group(http://www.khronos.org)". This group was founded in 2000 by companies such as NVIDIA, Sun, ATI, and SGI. There are a number of resources at this website related to OpenGL. You can learn more about the OpenGL spec at
http://www.khronos.org/opengl/
The official documentation page for OpenGL is available at
http://www.opengl.org/documentation/
As you could see from this page there are very many books and online resources available on OpenGL. Among those the book titled "The official Guide to Learning OpenGL, Version 1.1" is the gold standard. This book is also referred to as the "red book" of OpenGL. An online version of this book is available at
http://www.glprogramming.com/red/
DirectX 3D is a competing standard to OpenGL in the windows world. It came to Microsoft in 1996 and is programmed using COM interfaces. OpenGL on the otherhand uses language bindings that look similar to their "c" language counterparts. As time passes both these standards are converging in their capabilities. However the learning curve between the two could be different as one is a "c" api and the other a "COM" interface, not to mention the rendering semantics. One key evolutionary difference between the two is DirectX3D is designed with hardware accelerators in mind where as OpenGL could use hardware accelerators but can fall back to software acceleration.
You can get a bigger picture of their subtle and not so subtle differences at the following URL
http://en.wikipedia.org/wiki/Comparison_of_OpenGL_and_Direct3D
OpenGL ES Background
Khronos group is also responsible two additional standards that are tied to Open GL: Open GL ES, and EGL. "Open GL ES" is a smaller version of Open GL intended for Embedded Systems. Because Open GL and Open GL ES are general purpose interfaces for drawing, each operating system or environment needs to provide a hosting environment for Open GL and Open GL ES. You will get to know this need in very practical terms later in the chapter. Khronos tried to define this Operating System to Open GL bridge through a standard called "EGL".
The targets for Open GL ES include such handhelds as, cell phones, appliances, and even vehicles. OpenGL ES is much smaller than OpenGL. Many of the functions have been removed. For example drawing rectangles is not directly allowed in OpenGL ES. You will have to draw two triangles to make a rectangle.
At the time of this writing there are two versions of OpenGL ES. The 1.x versions support fixed function hardware accelerated GPUs (Dedicated Graphics Processing Units). The 2.x versions support more programmable 3D chips just like regular CPUs. Android supports OpenGL ES 1.0 in the current SDK. It has no support yet for OpenGL ES 2.x.
As we start exploring Android support for OpenGL we will be focusing primarily on OpenGL ES and its bindings to Android OS through Java and EGL.
The documentation or "man" pages for OpenGL ES can be found at
http://www.khronos.org/opengles/documentation/opengles1_0/html/index.html
This is a very useful resource as it lays out every OpenGL ES api name, its arguments and some explanation. These are similar to Java apis. However you will need a lot of OpenGL background to understand all thse APIs. You will also see that programming in OpenGL is like programming in an assembly language.
OpenGL ES java binding
OpenGL ES like OpenGL is a "c" based flat api. Android SDK being a Java based programming API, needs a Java binding to the OpenGL ES. Jva ME has already defined this binding through JSR 239 for OpenGL ES. This JSR 239 itself is based on JSR 231 that is a Java binding for OpenGL 1.5. JSR 239 could have been strictly a subset of JSR 231 but that is not the case because there are some extensions to the OpenGL ES that are not in OpenGL 1.5.
The Java binding for the OpenGL ES is documented at
http://java.sun.com/javame/reference/apis/jsr239/
The defined packages include
javax.microedition.khronos.egl javax.microedition.khronos.opengles javax.microedition.khronos.nio
The "nio" package is necessary because the OpenGL ES implementations take only byte streams as inputs for efficiency reasons. "nio" defines a lot of utility buffers for this purpose. You will see some of this in action in later part of this chapter.
Android documents their support for this Java binding at
http://code.google.com/android/toolbox/apis/opengl.html
On this page Android documentation indicates that the Android implementation may diverge from Open GL ES at a few places.
M3G: Another Java ME 3D graphics standard
JSR 239 is a merely a java binding on a native OpenGL ES standard. Java has provided another API to work with 3D on mobile devices. This object oriented standard is called "Mobile 3D Graphics API" (M3G). This standard is defined in JSR 184. As per JSR 184, M3G is targeted as a lightweight, object oriented, interactive 3D graphics API for mobile devices.
The object oriented nature of M3G separates it from OpenGL ES. There is also another primary difference. M3G has two modes. In one mode called "Immediate" it looks very similar to OpenGL ES. In a mode called "retain" mode it allows to store rendered objects as a "scene graph" that could be saved on a disk as a file. This file can be transported between design tools and run time tools. In this mode a high level design tool can render a complex 3D object which can be imported to a cell phone and be animated.
The homepage for JSR 184 can be found at
http://www.jcp.org/en/jsr/detail?id=184
The apis for M3G are available in the java package
javax.microedition.m3g.*;
M3G is a higher level API compared to OpenGL ES and should be easier to learn. However the jury is out how well it will perform on handhelds. As of now Android does not support M3G.
Exercising OpenGL ES
Now that you know the problem space of OpenGL let us proceed on a plan to draw some simple figures using OpenGL ES API. As you go through the rest of the chpater you may want to keep the following references handy
A basic book on OpenGL
http://www.glprogramming.com/red/
OpenGL ES Java Api reference
http://java.sun.com/javame/reference/apis/jsr239/
OpenGL ES "C" api page at Khronos Group
http://www.khronos.org/opengles/documentation/opengles1_0/html/index.html
Keep in mind that, as you start using the OpenGL books or documentation as a reference for OpenGL ES, a good number of APIs are not available in OpenGL ES. This is where the OpenGL ES API reference from the Khronos group come handy.
Our goal in this chapter is to take a couple of simple OpenGL example and let that example demonstrate you the basic concepts of OpenGL and also how to use Android for OpenGL. We will start with a simple triangle. As we know, a triangle can be described using three points in a coordinate system. The code to draw a triangle, although still a bit involved, is the easiest to understand in OpenGL. Once we have this basic code planned out, we have to find a place for that code to live in the Android SDK. As it turns out this is not a simple task. You will need a SurfaceView to effectively do 3D work. A SurfaceView will require a secondary thread to draw on it. And then you have to initialize the OpenGL ES environment (This is sufficiently involved as well). Based on an example provided in the Android SDK we have created a test harness to abstract all this code and we will show you how. Once you get this understanding how to test this basic drawing code, we will go into the difficult task of explaing the idea of Camera, Frustum, Viewports and Model coordinates to give you the back ground necessary to be on your own as you go exploring the rest of OpenGL. We would like to point out however that the primary goal is to show you how to run OpenGL programs in Android and a deeper learning of OpenGL is left to outside resources. At the same time you will be able to follow the code here with out needing to refer to any OpenGL book. You will need to refer to those resources only when you want to sprout wings and want to go further with OpenGL.
Drawing a triangle
Let us start with drawing a triangle. OpenGL has mechanisms to draw higher level polygons directly such as a rectnagle. As OpenGL reduces the number of APIs it leaves only a more primitive level drawing apis. To draw a triangle in OpenGL ES you will need to pass three points in a buffer to OpenGL ES and ask it to draw those points, assuming they form a triangle. Same approach is used to draw points, or lines, or more complex figures that are combination of any of these. You pass a series of point data in a long buffer and then tell OpenGL ES to draw either points, or lines, or triangles. OpenGL ES will interpret the points depending on the nature of what is to be drawn. If you tell OpenGL ES to draw triangles, it will assume that every three points form a triangle and accordingly it will divide the series of points accordingly. If you tell OpenGL ES to draw lines, then every two points will become a line segment. With this knowledge you can refer to the OpenGL redbook to find all the options available and how each options intepret the series of data points that are passed in. In this book we will focus on drawing triangles. We can use the glDrawElements method to draw a series of points as a triangle. Here is a code snippet using this method to draw a single triangle.
//Clear the surface of any color
gl.glClear(gl.GL_COLOR_BUFFER_BIT);
//Set the current color
gl.glColor4f(1.0f, 0, 0, 0.5f);
//There are three points (floats) in the specificed buffer pointer
gl.glVertexPointer(3, GL10.GL_FLOAT, 0, mFVertexBuffer);
//Draw them assuming they form triangles
//there are 3 points
gl.glDrawElements(GL10.GL_TRIANGLE_STRIP, 3,
GL10.GL_UNSIGNED_SHORT, mIndexBuffer);
In this code the variable "gl" represents the OpenGL ES interface, specifically javax.microedition.khronos.opengles.GL10. We will show you later how you would go about getting a reference to this "gl" object.
In this code snippet we start out by using "glClear" to clean up or erase the drawing surface. Using this method you can reset not only color, but also depth and the type of stencils used. Which one to reset is specified by the passed in constant. The constants include GL_COLOR_BUFFER_BIT, GL_DEPTH_BUFFER_BIT, GL_STENCIL_BUFFER_BIT. In the code here we have reset the color.
Then we proceed to set "red" as the active drawing color. This api sets the active color based on 4 arguments (red, green, blue, alpha). The starting values for each are (1,1,1,1). In this case we have set the color to red with a half a gradient (specified by the last alpha argument). Let us take a moment and point out to the curious structure of "gl" method call names. They are all preceeded by "gl". They have then a basename describing what the method is for and then optionally a number such as "4" pointing to either the number of dimensions such as (x,y,z) etc or in this case number of color arguments. The method then is followed by a data type such as "f" for float. You can refer to OpenGL resources to find the various combinations. For example if a method takes an argument either as a "byte (b)" or a "float (f)" then the method will have two names, one ending with "b" and one ending with "f".
After we cleared the color and set the color to red, we will supply OpenGL ES with a buffer that contains three points using the method "glVertexPointer". This method is responsible for specifying an array of points to be drawn. Each point is specified in three dimensions. So each point will have three values: x, y, and z. Let us see how we can specify these points in an array
float[] coords = {
-0.5f, -0.5f, 0, //p1: (x1,y1,z1)
0.5f, -0.5f, 0, //p2: (x1,y1,z1)
0.0f, 0.5f, 0 //p3: (x1,y1,z1)
};
You will first ask what are the units of these coordinates. Short answer is anything you want. You will need to specify the bounding volume (or a bounding box) that quantifies these coordinates. For example you can specify the bounding box as a 5 inch sided cube or a 2 inch sided cube. These coordinates are also refered to sometimes as "world" coordinates. We will further explain that later in this chapter. For now assume that you are using a cube that is 2 units across all its sides. You can also assume that the origin is at the center of visual display. The "z" axis will be negative going into the display and positive coming out of the display. "x" will go positive as you move right and will be negative as you move left. However these will also be dependent on from what direction you are viewing the scene. Again, I will go into that later.
We take these points and then pass them to OpenGL ES as part of a vertex buffer. For efficiency reasons the method glVertexPointer does not take an arry of floats but a native buffer. For this we need to convert the array of floats to an acceptable native buffer. Here is the sample code to do that
jva.nio.ByteBuffer vbb = java.nio.ByteBuffer.allocateDirect(3 * 3 * 4);
vbb.order(ByteOrder.nativeOrder());
java.nio.FloatBuffer mFVertexBuffer = vbb.asFloatBuffer();
The byte buffer is a buffer of memory ordered into bytes. The number of bytes we need to hold each point is 3 floats because of the three axes. Each float is 4 bytes. So together we get 3 * 4 bytes for each point. There are 3 points in a triangle. Altogether we need then 3 * 3 * 4 bytes to hold 3 float points of a triangle. Now that we have all the necessary data we can call the "glVertexPointer" as
gl.glVertexPointer(3, GL10.GL_FLOAT, 0, mFVertexBuffer);
You will need to pay special attention to the first argument. This argument tells OpenGL ES how many dimensions are there in a point or a vertex. In this case we have specified x, y, and z. We could also have specified just x,y and omitting z. "z" will then be assumed to be zero. In that case we need to pass the first argument as "2". This argument is not the count of number of points in the buffer. So if you pass 20 points to draw a number of triangles you will not pass "20" as the first argument.
The second argument indicates that the coordinates need to be interpreted as "floats". The 3rd argument is called a "stride". "stride" points to the number of bytes separating each point. In this case it is zero because one point follows the other right away. Sometimes you can add color attributes as part of the buffer after each point. In that case you can use "stride" to skip those as part of the vertex specification. The last argument is the pointer to the buffer containing the mathematical points.
Once we announce the points to OpenGL we are almost ready to draw. Do know that "method" affects in OpenGL are cumulative. This means we don't need to specify the vertices as part of "glDrawElements" because OpenGL(like a state machine) remembers the previous values of vertex specification.
We use "glDrawElements" to draw using the points that are already specified. One curious feature of "glDrawElements" is it allows you to reuse point specification. Let us talk about what this reuse is. For example in a square there are 4 points. Each square can be drawn as a combination of two triangles. If we want to draw two triangles to make up the square, do we have to specify 6 points? we don't have to. we can just specify 4 points and refer to them 6 times to draw two triangles. This later process is called indexing into the point buffer.
Here is an example
Points: (p1, p2, p3, p4)
Draw indices (p1, p2, p3, p2,p3,p4)
Notice how the first triangle is made of "p1,p2,p3" and the second one is made of "p2,p3,p4". With this knowledge on our sleeve we are ready to use the "glDrawElements" method
gl.glDrawElements(GL10.GL_TRIANGLE, 3,
GL10.GL_UNSIGNED_SHORT, mIndexBuffer);
The first argument is telling OpenGL ES to consider the points to draw a triangle. The possible options are Points Only(GL_POINTS), Line strips (GL_LINE_STRIP), Lines only (GL_LINES), Line Loops (GL_LINE_LOOP), Triangles only (GL_TRIANGLES), Triangle Strips (GL_TRIANGLE_STRIP), Triangle Fans (GL_TRIANGLE_FAN). The concept of a "STRIP" is to add new points while making use of the old ones and avoiding specifying all the points for each new object. We will leave this as an exercise to you to vary these parameters and see how each is drawn. The idea of a "FAN" applies to triangles where the very first point is used as a starting point for all subsequent triangles and in the process making a "FAN or Circle" like object with the first vertex in the middle.
The second argument identifies how many indeces are there in the index buffer. In our case there are only 3. The third argument points to the type of values in the index array whether they are unsigned shorts (GL_UNSIGNED_SHORT), or unsigned bytes (GL_UNSIGNED_BYTE). The last argument points to the index buffer. We can construct an unsigned short index buffer as follows
//get a short buffer
java.nio.ShortBuffer mIndexBuffer;
//Allocate 2 bytes each for each point
ByteBuffer ibb = ByteBuffer.allocateDirect(3 * 2);
ibb.order(ByteOrder.nativeOrder());
mIndexBuffer = ibb.asShortBuffer();
//Figure out your drawing scheme
short[] myIndecesArray = {0,1,2};
//stuff that into the buffer
for (int i=0;i<3;i++)
{
mIndexBuffer.put(myIndecesArray[i]);
}
With the index buffer in place you can make a call to draw the triangle
gl.glDrawElements(GL10.GL_TRIANGLE_STRIP, 3,
GL10.GL_UNSIGNED_SHORT, mIndexBuffer);
Getting an OpenGL ES context to draw
So far you have understood the basics of what it takes to draw a simple triangle in OpenGL ES. But we are far from being able to test this code. Before we draw we need to get an OpenGL context, bind it to a view, clear the view, and setup the camera, and also respond to window size changing events.
Let us start this journey by first obtaining an OpenGL ES context. This is where the second API of OpenGL ES implemented in javax.microedition.khronos.egl.EGL10 is utilized. You can read about using this package in more detail at
http://java.sun.com/javame/reference/apis/jsr239/javax/microedition/khronos/egl/EGL10.html
High level steps of getting an EGL context include
1. Get an implementation of EGL10
2. Get a display to use
3. Initialize the display
4. Specify a device specific configuration to EGL
5. Use an initialized display, and a configuration to get a context
Once you have the context, you can tell the context to use a window surface evertime a window is created or changed and then tear it down at the end. We will look at preparing the window surface and tearing it down in the next section.
Let us, first, take a look at some boiler plate code to get an EGL Context.
//Ask for an implementation of EGL10
EGL10 mEgl = (EGL10) EGLContext.getEGL();
//get the defalut display
EGLDisplay mEglDisplay = mEgl.eglGetDisplay(EGL10.EGL_DEFAULT_DISPLAY);
//initialize the display
int[] version = new int[2];
mEgl.eglInitialize(mEglDisplay, version);
//config spec
int[] configSpec = {
EGL10.EGL_DEPTH_SIZE, 0,
EGL10.EGL_NONE
};
EGLConfig[] configs = new EGLConfig[1];
int[] num_config = new int[1];
mEgl.eglChooseConfig(mEglDisplay, configSpec, configs, 1,
num_config);
mEglConfig = configs[0];
mEglContext = mEgl.eglCreateContext(mEglDisplay, mEglConfig,
EGL10.EGL_NO_CONTEXT, null);
This code is pretty standard except for the part where a drawing configuration could be different depending on an application. The method "getEGL" returns an implementation for the interface "EGL10". The rest of the methods uses this interface in an implementation independent manner to get to the context.
The method "eglGetDisplay" will return a default display to connect to, if what is passed in is a "EGL_DEFAULT_DISPLAY". The "eglInitialize" method initializes the display and returns a major and minor version numbers of the OpenGL implementation.
The next method eglChooseConfigSpec is more involved. This method wants you to specify the type of things that are critical to you as you draw. Some example config specs are
A configuration suitable for 8 bit depth color sizes
int[] configAttrs = { EGL10.EGL_RED_SIZE, 8,
EGL10.EGL_GREEN_SIZE, 8,
EGL10.EGL_BLUE_SIZE, 8,
EGL10.EGL_ALPHA_SIZE, EGL10.EGL_DONT_CARE,
EGL10.EGL_DEPTH_SIZE, EGL10.EGL_DONT_CARE,
EGL10.EGL_STENCIL_SIZE, EGL10.EGL_DONT_CARE,
EGL10.EGL_NONE
};
A suitable configuration where depth is zero
int[] configSpec = {
EGL10.EGL_DEPTH_SIZE, 0,
EGL10.EGL_NONE
};
Refer to an OpenGL book to get a better understanding of "configuration management" under OpenGL. Based on these configuration specs, EGL10 implementation returns a series of suitable EGLConfig references. In this case the first configuration is chosen.
Finally to get the EGL context we pass an EGLDisplay and an EGLConfig to "eglCreateContext". The third argument of this method indicates sharing. Here we pointed out there is no other context we want to share objects with. The last argument is a set of additional attributes which we are specified here as null.
Preparing a window to paint
Once we have an EGL context we don't have to recreate it as windows are drawn. This means we create a Context once and keep it for the life of that Activity. However, we need to bind the drawing window to the EglContext using the following method sequence everytime the window is created or changed in some manner.
android.view.SurfaceHolder holder = surfaceView.getHolder();
mEglSurface = mEgl.eglCreateWindowSurface(mEglDisplay,
mEglConfig, holder, null);
mEgl.eglMakeCurrent(mEglDisplay, mEglSurface, mEglSurface,
mEglContext);
GL gl = mEglContext.getGL();
We finally have seen a reference to the "GL" interface that allowed us to draw our triangle using OpenGL ES commands. We also need to disassociate window surface as window changes. The code for this is
mEgl.eglMakeCurrent(mEglDisplay, EGL10.EGL_NO_SURFACE,
EGL10.EGL_NO_SURFACE, EGL10.EGL_NO_CONTEXT);
mEgl.eglDestroySurface(mEglDisplay, mEglSurface);
Closing out the OpenGL at the end of the program
Here is the sequence you will need to use to close out the OpenGL ES resources at the end of your program
//Destory surface
mEgl.eglMakeCurrent(mEglDisplay, EGL10.EGL_NO_SURFACE,
EGL10.EGL_NO_SURFACE,
EGL10.EGL_NO_CONTEXT);
mEgl.eglDestroySurface(mEglDisplay, mEglSurface);
//Destroy context
mEgl.eglDestroyContext(mEglDisplay, mEglContext);
//Disassoicate display
mEgl.eglTerminate(mEglDisplay);
OpenGL Test Harness
Unfortunately our journey to draw an OpenGL triangle is not yet at an end. If the OpenGL examples from Android SDK are any indication, they all use a "SurfaceView" as the surface to draw OpenGL. A "SurfaceView" unlike a regular "View" uses a separate thread to draw on its surface. However it is more involved. To understand all of the examples that come with the Android SDK, let us brave the waters and understand how a SurfaceView is tamed to the benefit of OpenGL. You also might ask, do I have to do all this everytime I want to use OpenGL. The answer lies in encapsulating all of this code in a set of reusable classes so that you can focus on OpenGL drawing.
Towards that goal we will build in this chapter an OpenGL test harness that you could use for most of your OpenGL needs. At the minimum it will be a great tool to test your OpenGL ideas. To create the test harnness we will start with the OpenGL examples. All of the OpenGL examples use a similar pattern and uses a class called "com.example.android.apis.graphics.GLSurfaceView".
There are some issues to use this as a simple test harness. Other than the source code there is no documentation at all how the code works. Using what you have learned so far you can kind of follow the code of this class. But it is still a bit difficult as there are a number of inner classes crowding the basic idea. Moreover the GLSurfaceView assumes that there is an animation for everything we draw. For example if we were to place the triangle sample code above, the GLSurfaceView will draw this code again and again and again in a loop whether a redraw is needed or not. Obviously we don't need to do that unless we need animation.
We have done a few things in this chapter to help you with this. We have broken that class down into independent classes and stripped it down to the basics by removing unnecessary code. We have also drawn a class diagram identifying classes and their responsibilities. Removed the animation so that it is easy for you to test your code. We have also explained every part of the code so that you will know how this code works. Importantly once this test harness is in place, you rarely have to modify this and you can focus on your OpenGL code.
Basic Idea of Test Harness
The general idea of the test harness is depicted here in the following diagram. The view we want to draw on will be represented by OpenGLTestHarness inheriting from SurfaceView. The drawing itself will happen in a separate co-operating thread called OpenGLDrawingThread. This thread needs to be alive for the life of the SurfaceView. As events happen on or to the SurfaceView it needs to inform the drawing thread of those events so that drawing can takes place. As you can see from the diagram a number of these calls are delegated to the OpenGLDrawingThread. The thread itself needs to have an OpenGL context at the begining and tear it down at the end. As the window comes into existence and changes its size, the thread needs to bind and unbind the OpenGL context to this window. OpenGLDrawingThread uses a utility class called EGLHelper to help with this work. In the end the test harness assumes that the variable parts of the OpenGL drawing are concentrated in a class implementing the "Renderer" interface. An implementation of the renderer is responsible for such things as setting the camera, positioning the camera, and setting coordniates for the viewing box. If you agree upon a given size or volume of a box then you can even further abstract this class out and leave only the "drawing" portion to the leaf level implementation class. We have created a class called "AbstractRenderer" that abstracts out these operations. That leaves us to focus on the "triangle drawing" that we have started this chapter with.
Figure 10.1 OpenGL Test Harness Class Diagram
Let us now consider the source code for each class in the test harness. You can build your test harness by taking these classes and building them into your own project. Each source listing is followed by commentary on the important parts of the source code.
SimpleTriangleRenderer
Let us show you first the class you would have to write to draw the OpenGL triangle described so far. This will give you an indication of the abstraction available in the test harness.
public class SimpleTriangleRenderer extends AbstractRenderer
{
//Number of points or vertices we want to use
private final static int VERTS = 3;
//A raw native buffer to hold the point coordinates
private FloatBuffer mFVertexBuffer;
//A raw native buffer to hold indices
//allowing a reuse of points.
private ShortBuffer mIndexBuffer;
public SimpleTriangleRenderer(Context context)
{
ByteBuffer vbb = ByteBuffer.allocateDirect(VERTS * 3 * 4);
vbb.order(ByteOrder.nativeOrder());
mFVertexBuffer = vbb.asFloatBuffer();
ByteBuffer ibb = ByteBuffer.allocateDirect(VERTS * 2);
ibb.order(ByteOrder.nativeOrder());
mIndexBuffer = ibb.asShortBuffer();
float[] coords = {
-0.5f, -0.5f, 0, // (x1,y1,z1)
0.5f, -0.5f, 0,
0.0f, 0.5f, 0
};
for (int i = 0; i < VERTS; i++) {
for(int j = 0; j < 3; j++) {
mFVertexBuffer.put(coords[i*3+j]);
}
}
short[] myIndecesArray = {0,1,2};
for (int i=0;i<3;i++)
{
mIndexBuffer.put(myIndecesArray[i]);
}
mFVertexBuffer.position(0);
mIndexBuffer.position(0);
}
//overriden method
protected void draw(GL10 gl)
{
gl.glColor4f(1.0f, 0, 0, 0.5f);
gl.glVertexPointer(3, GL10.GL_FLOAT, 0, mFVertexBuffer);
gl.glDrawElements(GL10.GL_TRIANGLES, VERTS,
GL10.GL_UNSIGNED_SHORT, mIndexBuffer);
}
}Notice how focused and "bare" this code is. As you create your new OpenGL experiments, this level of simplicity allows you to be very productive. We set up the draw method based on the principles we have covered for drawing a triangle. To reiterate quickly here ,we have identified our points, we transported point coordinates to a buffer. We did the same with indices for those points. Then we drew the triangle using glDrawElements. Let us now see how we take this simple renderer and set it as an activity of the OpenGL Test Harness. The "AbstractRenderer" is covered later in this section.
OpenGLTestHarnessActivity
public class OpenGLTestHarnessActivity extends Activity {
private OpenGLTestHarness mTestHarness;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
mTestHarness = new OpenGLTestHarness(this);
mTestHarness.setRenderer(new SimpleTriangleRenderer(this));
setContentView(mTestHarness);
}
@Override
protected void onResume() {
super.onResume();
mTestHarness.onResume();
}
@Override
protected void onPause() {
super.onPause();
mTestHarness.onPause();
}
}
As you come up with new OpenGL renderers all you have to do is instantiate an OpenGLTestHarness and set it into an activity as a view. You know now, how to create a new renderer to test your OpenGL drawing code and use it in an Activity. Let us now look at the rest of the classes that go into the implementation of the harness.
OpenGLTestHarness
public class OpenGLTestHarness extends SurfaceView
implements SurfaceHolder.Callback
{
public static final Semaphore sEglSemaphore = new Semaphore(1);
public boolean mSizeChanged = true;
public SurfaceHolder mHolder;
private OpenGLDrawingThread mGLThread;
public OpenGLTestHarness(Context context) {
super(context);
init();
}
public OpenGLTestHarness(Context context, AttributeSet attrs) {
super(context, attrs);
init();
}
private void init() {
mHolder = getHolder();
mHolder.addCallback(this);
mHolder.setType(SurfaceHolder.SURFACE_TYPE_GPU);
}
public SurfaceHolder getSurfaceHolder() {
return mHolder;
}
public void setRenderer(Renderer renderer) {
mGLThread = new OpenGLDrawingThread(this,renderer);
mGLThread.start();
}
public void surfaceCreated(SurfaceHolder holder) {
mGLThread.surfaceCreated();
}
public void surfaceDestroyed(SurfaceHolder holder) {
mGLThread.surfaceDestroyed();
}
public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {
mGLThread.onWindowResize(w, h);
}
public void onPause() {
mGLThread.onPause();
}
public void onResume() {
mGLThread.onResume();
}
@Override public void onWindowFocusChanged(boolean hasFocus) {
super.onWindowFocusChanged(hasFocus);
mGLThread.onWindowFocusChanged(hasFocus);
}
@Override
protected void onDetachedFromWindow() {
super.onDetachedFromWindow();
mGLThread.requestExitAndWait();
}
}
This is the SurfaceView class that provides the surface for drawing. All this class does is to be a vehicle for transporting window events to the support OpenGLDrawingThread. This class creates the OpenGLDrawingThread using an renderer that is passed in. This happens in the public setRenderer() method. This part of the code is highlighted in the code above.
OpenGLDrawingThread
class OpenGLDrawingThread extends Thread
{
private boolean mDone, mPaused, mHasFocus;
private boolean mHasSurface, mContextLost, mSizeChanged;
private int mWidth,mHeight;
private Renderer mRenderer;
private EglHelper mEglHelper;
private OpenGLTestHarness pSv = null;
OpenGLDrawingThread(OpenGLTestHarness sv, Renderer renderer) {
super();
mDone = false; mWidth = 0; mHeight = 0;
mRenderer = renderer; mSizeChanged = false;
setName("GLThread");
pSv = sv;
}
@Override
public void run() {
try {
try {
OpenGLTestHarness.sEglSemaphore.acquire();
} catch (InterruptedException e) {
return;
}
guardedRun();
} catch (InterruptedException e) {
// fall thru and exit normally
} finally {
OpenGLTestHarness.sEglSemaphore.release();
}
}
private void guardedRun() throws InterruptedException {
mEglHelper = new EglHelper();
int[] configSpec = mRenderer.getConfigSpec();
mEglHelper.start(configSpec);
GL10 gl = null;
boolean tellRendererSurfaceCreated = true;
boolean tellRendererSurfaceChanged = true;
while (!mDone)
{
int w, h;
boolean changed;
boolean needStart = false;
synchronized (this) {
if (mPaused) {
Log.d("x", "Paused");
mEglHelper.finish();
needStart = true;
}
if(needToWait()) {
while (needToWait()) {
wait();
Log.d("x", "woke up from wait");
}
}
if (mDone) {
break;
}
changed = pSv.mSizeChanged;
w = mWidth;
h = mHeight;
pSv.mSizeChanged = false;
this.mSizeChanged = false;
}
if (needStart) {
Log.d("x", "Need to start");
mEglHelper.start(configSpec);
tellRendererSurfaceCreated = true;
changed = true;
}
if (changed) {
Log.d("x", "Change");
gl = (GL10) mEglHelper.createSurface(pSv.mHolder);
tellRendererSurfaceChanged = true;
}
if (tellRendererSurfaceCreated) {
Log.d("x", "Render Surface created");
mRenderer.surfaceCreated(gl);
tellRendererSurfaceCreated = false;
}
if (tellRendererSurfaceChanged) {
Log.d("x", "Render Surface changed");
mRenderer.sizeChanged(gl, w, h);
tellRendererSurfaceChanged = false;
}
if ((w > 0) && (h > 0)) {
Log.d("x", "Drawing frame now");
mRenderer.drawFrame(gl);
mEglHelper.swap();
}
}
mEglHelper.finish();
}
private boolean needToWait() {
return ((!mSizeChanged) || mPaused || (! mHasFocus) || (! mHasSurface) || mContextLost)
&& (! mDone);
}
public void surfaceCreated() {
synchronized(this) {
mHasSurface = true;
mContextLost = false;
notify();
}
}
public void surfaceDestroyed() {
synchronized(this) {
mHasSurface = false;
notify();
}
}
public void onPause() {
synchronized (this) {
mPaused = true;
}
}
public void onResume() {
synchronized (this) {
mPaused = false;
notify();
}
}
public void onWindowFocusChanged(boolean hasFocus) {
synchronized (this) {
mHasFocus = hasFocus;
if (mHasFocus == true) {
notify();
}
}
}
public void onWindowResize(int w, int h) {
synchronized (this) {
mWidth = w;
mHeight = h;
pSv.mSizeChanged = true;
this.mSizeChanged = true;
Log.d("x","window size changed. w, h:" + w + "," + h);
if (w > 0)
{
notify();
}
}
}
public void requestExitAndWait()
{
synchronized(this) {
mDone = true;
notify();
}
try {
join();
} catch (InterruptedException ex) {
Thread.currentThread().interrupt();
}
}
}
OpenGLDrawingThread is a critical class to understand the interaction between Android SDK and the OpenGL ES. The OpenGLTestHarness class starts this thread as soon as a renderer is set in the harness. According to the Android documentation the first thing the "run" method needs to do is to wait for any previous instances to close and ensure that there is only one activity that is running. This is because there are timing issues between onDestroy() and onCreate(). Eitherway the practice is to go and ensure exclusive access through this semaphore.
Once the thread starts running it will get the "configspec" from the renderer and use that config spec to start the EGLHelper. EGLHelper will be used to obtain an EGLContext. This is where EGLHelper initializes display and uses the config to create the context.
Once the context is available, the thread has to wait for a window to be created before drawing. With out a window you do not have anything to bind to the OpenGL context. In effect the while loop goes into a wait mode. When a window is created or resized the corresponding call forward methods from the OpenGLTestHarness SurfaceView wakes up the thread using "notify". The thread will then bind the OpenGL context to the window and then draw. Once the drawing is complete the thread uses the "swap" buffers method of EGLHelper to tranfer the paint buffers to the screen.
Once the drawing is complete, the thread will have to go back to wait so that it can respond to further events on the screen such as resize or pause or resume etc. This is why this "while" loop needs to be "forever". Pay special attention to the "needToWait()" function and the corresponding "notify" events. The "needToWait()" halts the thread if the size hasn't changed, or surface hasn't been created, or the focus is not there. You can enable this thread for animation if you remvove chcking for the flag which indicates size change. This will allow redraw even when the size doesn't change which essentially is the basis of animation.
EGLHelper
We have already explained the basics of what happens in the EGL Helper in the begining of this chapter. The code here just consolidates those snippets into a whole class.
public class EglHelper
{
EGL10 mEgl; EGLDisplay mEglDisplay; EGLSurface mEglSurface;
EGLConfig mEglConfig; EGLContext mEglContext;
public EglHelper(){}
public void start(int[] configSpec)
{
mEgl = (EGL10) EGLContext.getEGL();
mEglDisplay = mEgl.eglGetDisplay(EGL10.EGL_DEFAULT_DISPLAY);
int[] version = new int[2];
mEgl.eglInitialize(mEglDisplay, version);
EGLConfig[] configs = new EGLConfig[1];
int[] num_config = new int[1];
mEgl.eglChooseConfig(mEglDisplay, configSpec, configs, 1,
num_config);
mEglConfig = configs[0];
mEglContext = mEgl.eglCreateContext(mEglDisplay, mEglConfig,
EGL10.EGL_NO_CONTEXT, null);
mEglSurface = null;
}
public GL createSurface(SurfaceHolder holder) {
if (mEglSurface != null) {
mEgl.eglMakeCurrent(mEglDisplay, EGL10.EGL_NO_SURFACE,
EGL10.EGL_NO_SURFACE, EGL10.EGL_NO_CONTEXT);
mEgl.eglDestroySurface(mEglDisplay, mEglSurface);
}
mEglSurface = mEgl.eglCreateWindowSurface(mEglDisplay,
mEglConfig, holder, null);
mEgl.eglMakeCurrent(mEglDisplay, mEglSurface, mEglSurface,
mEglContext);
GL gl = mEglContext.getGL();
return gl;
}
public boolean swap() {
mEgl.eglSwapBuffers(mEglDisplay, mEglSurface);
return mEgl.eglGetError() != EGL11.EGL_CONTEXT_LOST;
}
public void finish() {
if (mEglSurface != null) {
mEgl.eglMakeCurrent(mEglDisplay, EGL10.EGL_NO_SURFACE,
EGL10.EGL_NO_SURFACE,
EGL10.EGL_NO_CONTEXT);
mEgl.eglDestroySurface(mEglDisplay, mEglSurface);
mEglSurface = null;
}
if (mEglContext != null) {
mEgl.eglDestroyContext(mEglDisplay, mEglContext);
mEglContext = null;
}
if (mEglDisplay != null) {
mEgl.eglTerminate(mEglDisplay);
mEglDisplay = null;
}
}
}
Renderer
Let us now turn our attention to the Renderer interface
public interface Renderer {
int[] getConfigSpec();
void surfaceCreated(GL10 gl);
void sizeChanged(GL10 gl, int width, int height);
void drawFrame(GL10 gl);
}
The getConfigSpec() method is responsible for returning the OpenGL configuration necessary to construct an OpenGL context. In the surfaceCreated() method the implementer is responsible for unbinding and binding the OpenGL context to the surface or window. You will need to set the ViewPort and Zoom in the sizeChanged. drawFrame() is responsible for drawing the model objects.
AbstractRenderer
The way you set bind and unbind to the surface and also the way you set the viewport, camera etc could be quite common for a number of scenarios. With this in mind we have abstracted out this further and created an abstract class to deal with these variations.
public abstract class AbstractRenderer implements Renderer
{
public int[] getConfigSpec() {
int[] configSpec = {
EGL10.EGL_DEPTH_SIZE, 0,
EGL10.EGL_NONE
};
return configSpec;
}
public void surfaceCreated(GL10 gl) {
gl.glDisable(GL10.GL_DITHER);
gl.glHint(GL10.GL_PERSPECTIVE_CORRECTION_HINT,
GL10.GL_FASTEST);
gl.glClearColor(.5f, .5f, .5f, 1);
gl.glShadeModel(GL10.GL_SMOOTH);
gl.glEnable(GL10.GL_DEPTH_TEST);
}
public void sizeChanged(GL10 gl, int w, int h) {
gl.glViewport(0, 0, w, h);
float ratio = (float) w / h;
gl.glMatrixMode(GL10.GL_PROJECTION);
gl.glLoadIdentity();
gl.glFrustumf(-ratio, ratio, -1, 1, 3, 7);
}
public void drawFrame(GL10 gl)
{
gl.glDisable(GL10.GL_DITHER);
gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT);
gl.glMatrixMode(GL10.GL_MODELVIEW);
gl.glLoadIdentity();
GLU.gluLookAt(gl, 0, 0, -5, 0f, 0f, 0f, 0f, 1.0f, 0.0f);
gl.glEnableClientState(GL10.GL_VERTEX_ARRAY);
draw(gl);
}
protected abstract void draw(GL10 gl);
}
It is worth pointing out here about some of the OpenGL apis that are used. This discussion will allow you to vary these and see what that will do to your test programs. Viewing an OpenGL drawing is similar to taking a picture with a camera. You first position the camera, like on a tripod. This is done through "gluLookAt()". Then you adjust the zoom or the distances that you are interested in. This is done through "glFrustm()". You can choose the size of the photograph. This is done through "glViewPort()".
Understanding Camera and Coordinates in OpenGL
You won't be able to program anything in OpenGL unless you understand the implications of these three apis. gluLookAt gluFrustum glViewPort
Let us elaborate on the Camera symbolism further to explain how these three apis effect what you see on an OpenGL screen.
Imagine you are going on a trip to take some pictures of a landscape involving flowers, trees, streams and mountains. You get to the meadow. What lies before you, the scene, is equivalent to what you draw. You can make these drawings as big, like the mountains, or as small, like the flowers as long as they are proportional to each other. These coordinates are called world coordinates. So I could say a line to be 4 units long on the x-axis by setting my points as (0,0,0 to 4,0,0).
As you are getting ready to take a picture you find a spot to place your tripod. Then you hook up the Camera to the tripod. This point where your camera is located becomes the origin of your world. So you will need to take a piece of paper and write this location down. If you don't say anything this camera is located at (0,0,0) which is in the smack middle of your screen. Usually you want to step away from the origin so that you can see the (x,y) plane that is sitting at the origin of z=0. For argument let us say we we position the camera at (0,0,5). This would move the Camera off your screen towards you by 5 units.
Once you placed the camera, you start looking ahead to see which portion of the scene you want to take picture off. You will position the camera in the direction you are looking at. This far off point that you are looking at is called a "view" point. This point is really a direction. So If I specify my view point as (0,0,0) then the Camera is looking along z axis toward the origin from a distance of 5 units.
Imagine further that there is a rectangular building at the origin, you want to look at it not in a portrait fashion but in a "landscape" fashion. What do you have to do? You obviously can leave the camera in the same location and still point it towards the origin but now you have to turn the Camera by 90 degrees. This is the orientation of the camera as the camera is fixed at a given and looking at a specific point or direction. This orientation is called the "up" vector. This up vector simply identifies the orientation of the camera such as up, down, left, right or at an angle. This orientation of the camera is specified using a point as well. You will imagine a line from the origin to this point. Whatever angle that line subtends in three dimensions at the origin is the orientation of camera. For example an up vector for a camera may look like (0,1,0) or even (0,15,0) both of which would have the same affect. The point (0,1,0) is a point on the "y" axis away from the origin. This means you point the camera upright. If you use (0,-1,0) you would have hosted the camera upside down. Still in both cases the camera is still at the same point (0,0,5) and looking at the same origin (0,0,0). These three coordinates can now be summarized as follows
(0,0,5) - eye point (location of the camera)
(0,0,0) - look at point (which direction the camera is pointing at)
(0,1,0) - "up" vector whether camera is up or down or slanted
You will specify these three points to the "gluLookAt" function in that order. Here is the code
GLU.gluLookAt(gl, 0,0,5, 0,0,0, 0,1,0);
You will be wondering how come this function is not a function against "gl" like below
gl.gluLookAt(0,0,5, 0,0,0, 0,1,0);
but a utility function in the accompanying non standard GLU class. As it turns out no new method is required in "gl" to simulate this by applying the proper scaling and rotation to the standard world coordinates assuming the camera is always positioned at origin and also assuming always up and also looking in the negative z direction. This utility function does exactly that but is much easier to understand conceptually.
If you notice none of these points that described the camera position deal with size. They only deal with direction. How can we tell the camera where to focus? What is distance of the subject we are trying to capture? Is it too near or is it too far and how near and how far. And also how wide and how tall. These aspects of the camera is described by using an imaginary rectangular like box that bounds your subject. Anthing outside the box is clipped and ignored. So how do we specify the box. You can first decide how far from the camera the box starts. This is called the near point. You can decide how far from the camera the box ends. This is called the far point. The distance between near and far is the depth of the box along the "z" axis. If you say near is 50 and far is 200, then you will be able to capture everythign that happens between 50 and 200 which is 150 deep. Under this scenario if the Camera is at the orgin then you will miss everything that happens in the origin plane of "z=0". These also give you a sense of units that you can use to draw your objects. You will also need to specify the left side of the box, right side of the box, top of the box and bottom of the box with respect to the imaginary "ray" that joins the camera to the look at point. In a sense the box is the enclosing volume of this ray.
In OpenGL there are two ways this box can be imagined. One is a cube like rectangular box or it can be like a pyramid with camera at its origin. The second one is called a persepctive view and suited for natural camera like function. This pyramidal structure is called a "frustm". The first one is called orthographic projection which is suited for geometrical drawings that need to preserve sizes despite the distance from the camera.
Let us now go ahead and see how this "frustum" of a box is specified for our example
float ratio = (float) w / h;
gl.glMatrixMode(GL10.GL_PROJECTION);
gl.glLoadIdentity();
gl.glFrustumf(
-ratio, // Left side of the viewing box
ratio, // right side of the viewing box
1, // top of the viewing box
-1, // bottom of the viewing box
3, // how far is the front of the box from the camera
7); // how far is the back of the box from the camera
Because we set the top to 1 and bottom to -1, we have set the size of the box to 2 units. You could have specified 50 and -50 if you wanted to. You will proportionatley decide the size for left and right so that you maintain the aspect ration of the figures. This is why you used the window height and width to figure out the proportion. You have focussed the area of action to be between 3 and 7 units along the z axis. Anything that is drawn outside these coordinates (after looking at where the camera is) wont be visible.
Because we set the camera to (0,0,5) and pointing at (0,0,0), 3 units from the camera towards the origin will be (0,0,2) and 7 units from the camera will be (0,0,-2). This leaves the origin plane right smack in the middle of your three dimensional box.
We still have one more job to do to nail down the real coordinates of what we are drawing and how big or small they show up on the screen. Somewhere we need to point out how big our window is. This is done through "glViewPort".
Here is the code
gl.glViewport(0, 0, w, h);
If your window size is 100 pixels and your "frustum" height is 10 units, then every logical unit of 1 translates to 10 pixels.
Having a bit of fun with Camera settings
To understand these coordinates better let us experiment with these camera related methods and see what happens to the triangle that we drew.
Start with what we have for these three methods for a triangle at (-1,-1, 1,-1, 0,1) (assuming z is zero)
//Look at the screen (origin) from 5 units away from the front of the screen
GLU.gluLookAt(gl, 0,0,5, 0,0,0, 0,1,0);
//Set the height to 2 units and depth to 4 units
gl.glFrustumf(-ratio, ratio, -1, 1, 3, 7);
//normal window stuff
gl.glViewport(0, 0, w, h);
You should see an upright triangle taking up most of the screen. Now if I change my "up" vector of the camera towards the negative "y" direction as below
GLU.gluLookAt(gl, 0,0,5, 0,0,0, 0,-1,0);
you will see an upside down triangle. If you increase the viewing boxes hight and width 4 times by doing the following
gl.glFrustumf(-ratio * 4, ratio * 4, -1 * 4, 1 *4, 3, 7);
You will see the triangle shrink because the triangle stayed at the same units but our viewing box is bigger now. If you change the camera position so that it looks at the screen from behind the screen you will see your coordinates reversed in the x-y plane. You can set this up by doing
GLU.gluLookAt(gl, 0,0,-5, 0,0,0, 0,1,0);
Using Indices to add another triangle
Let us conclude the examples by actually inheriting from the abstract renderer and create another triangle by simply adding another point and using indices.
Conceptually we will define the four points as (-1,-1, 1,-1, 0,1, 1,1)
And we will ask OpenGL to draw these as (0,1,2 0,2,3)
Here is the code to do this
public class SimpleTriangleRenderer2 extends AbstractRenderer
{
private final static int VERTS = 4;
private FloatBuffer mFVertexBuffer;
private ShortBuffer mIndexBuffer;
public SimpleTriangleRenderer2(Context context)
{
ByteBuffer vbb = ByteBuffer.allocateDirect(VERTS * 3 * 4);
vbb.order(ByteOrder.nativeOrder());
mFVertexBuffer = vbb.asFloatBuffer();
ByteBuffer ibb = ByteBuffer.allocateDirect(6 * 2);
ibb.order(ByteOrder.nativeOrder());
mIndexBuffer = ibb.asShortBuffer();
float[] coords = {
-1.0f, -1.0f, 0, // (x1,y1,z1)
1.0f, -1.0f, 0,
0.0f, 1.0f, 0,
1.0f, 1.0f, 0
};
for (int i = 0; i < VERTS; i++) {
for(int j = 0; j < 3; j++) {
mFVertexBuffer.put(coords[i*3+j]);
}
}
short[] myIndecesArray = {0,1,2, 0,2,3};
for (int i=0;i<6;i++)
{
mIndexBuffer.put(myIndecesArray[i]);
}
mFVertexBuffer.position(0);
mIndexBuffer.position(0);
}
protected void draw(GL10 gl)
{
gl.glColor4f(1.0f, 0, 0, 0.5f);
gl.glVertexPointer(3, GL10.GL_FLOAT, 0, mFVertexBuffer);
gl.glDrawElements(GL10.GL_TRIANGLES, 6,
GL10.GL_UNSIGNED_SHORT, mIndexBuffer);
}
}
Implementing Animation
Altering this code to allow for animation is quite simple. Update the "OpenGLDrawingThread" guardedRun() method so that the while loop won't wait to redraw as long as the width and height are valid. This will allow continues redraws() even when there is no resize(). Once a draw method is called multiple times you can use the matrix methods to rotate, scale, and move. At that point the ideas are similar to what we have presented in the Animation chapter.
Summary
In this chapter we have covered the background and basics of OpenGL support in Android. To cover OpenGL in depth is not a goal of this chapter. We have provided resources to learn more about OpenGL. We have explored how Android wants to use OpenGL ES on its SDK. We have covered extensivley an OpenGL paradigm using SurfaceView that is also used in many of the Android samples. We have also given you a very simplified Test harness that you could use to explore OpenGL further using the OpenGL resources identified.
Android comes with at least 5 simples covering a lot of ground in OpenGL. However without having a background in OpenGL it is almost impossible to understand any of these samples. We believe we have given you enough background and resources in this chapter to get you started with these Android samples. We will encourage you to explore each of those samples to gain a full understanding of OpenGL capabilities. Once you explore the samples you can get into advanced game development using OpenGL.