Alexandr u Io an Cuza University fr om Ia³i [618647]

"Alexandr u Io an Cuza" University fr om Ia³i
F a cul ty of Ma thema tics
OpenGL es for Android. Applica tion
Dissertation
Scien tic co ordinator:
Lect. Dr. Apetri i MariusStuden t:
Negr uµ Andreea
JUL Y, 2020
Ia³i

Abstract
The purp ose of this thesis is to enable the reader to dev elop kno wledge, understanding, capabilities
and attitudes to create an Android application using Op enGL ES. It's divided in t w o parts: a
theoretical part,the rst c hapter and a practical part, the second c hapter, that will describ e the
application.

Contents
In tro duction 2
1 Op en Graphics Library for Em b edded Systems 3
1.1 Building Ob jects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2 Shaders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Op enGL Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4 Color and Shade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.5 Camera Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.6 Mapping Co ordinates for Dra wn Ob jects . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.6.1 Pro jection and camera view in Op enGL ES 2.0 . . . . . . . . . . . . . . . . . 12
1.7 3D T ransforms and Matrix Represen tations . . . . . . . . . . . . . . . . . . . . . . . 13
1.7.1 Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.7.2 T ranslation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.7.3 Rotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.8 Orthographic Pro jections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.9 P ersp ectiv e Pro jections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2 Application 20
Bibliograph y 23
1

Introduction
The Android framew ork pro vides a lot of standard to ols for creating functional,attractiv e graphical
user in terfaces. Android includes supp ort for high p erformance 2D and 3D graphics with the Op en
Graphics Library , sp ecically , the Op enGL ES API.
Op enGL is a cross-platform graphics API that sp ecies a standard soft w are in terface for 3D
graphics pro cessing hardw are. This cross-platform w as originally dev elop ed b y Silicon Graphics
in 1992. Op enGL has b een used on the desktop for a while no w, and the mobile platforms use a
sp ecial em b edded v ersion kno wn as Op enGL ES. The rst v ersion of Op enGL ES brough t 3D to
mobile, whic h w as v ery p opular with dev elop ers b ecause it w as easy to learn and b ecause it had a
w ell-dened feature set. Ho w ev er, this feature set w as also limited, and it w asn't able to k eep up
with the latest and greatest features from the most p o w erful smartphones and tablets.
Most of the old APIs w ere completely remo v ed and replaced with new programmable APIs,
whic h mak es it m uc h easier to add sp ecial eects and tak e adv an tage of what the latest devices
ha v e to oer. These devices can no w pro duce graphics that riv al consoles from just a few y ears ago.
In August 2012, the Khronos Group nalized the sp ecication for the next v ersion, Op enGL ES
3.0, whic h is fully compatible with 2.0, extending it with a few adv anced features.
The Op enGL ES APIs pro vided b y the Android framew ork oers a set of to ols for displa ying
high-end, animated graphics that are limited only b y imagination and can also b enet from the
acceleration of graphics pro cessing units (GPUs) pro vided on man y Android devices.
Op enGL ES 2.0 is standard in mobile devices and has pro v ed its success as a 3D rendering
tec hnology .
Op enGL ES is widely used in 3D gaming dev elopmen t and the computer graphics imagery . It
is used in animation mo vies and sp ecial eects for b oth mo vies and series.
2

Chapter 1
Open Graphics Library for Embedded
Systems
Android includes supp ort for high p erformance 2D and 3D graphics with the Op en Graphics Library
(Op enGL R
), sp ecically , the Op enGL ES API. Op enGL is a cross-platform graphics API that
sp ecies a standard soft w are in terface for 3D graphics pro cessing hardw are. Op enGL ES is a a v or
of the Op enGL sp ecication in tended for em b edded devices.
Op enGL ES is designed in a w a y that it uses a normalized cub e. In this cub e the 3D scene
will b e rendered. In this mo de, it is determine what is visible and what is not. Op enGL ES will
ignore an ything outside of the cub e. The main dierences b et w een Op enGL ES 1.1 and Op enGL
Figure 1.1: Normalized cub e
ES 2.0 is that the 1.1 implemen ts a xed function pip eline while the Op enGL ES 2.0 sp ecication
implemen ts a programmable graphics pip eline (Figure 1.2). The Op enGL ES 2.0 is deriv ed from
the Op enGL 2.0 sp ecication.
3

Op en Gr aphics Libr ary for Emb e dde d Systems
Figure 1.2: Op enGL ES 2.0 Graphics Pip eline
The term pip eline refers to is all the op erations happ ening from the dra wing command in
Op enGL ES un til the 3D are fully dra wn on the displa y . The term dra w in Op enGL ES is usually
called rendering. Op enGL ES is con tin uously dra wing in a similar w a y than a video is con tin uously
running and the dieren t images b eing displa y ed are called frames.
Op enGL ES 1.1 uses a xed rendering pip eline. So, the data submitted to Op enGL ES will
b e rendered in a w a y that it Op enGL ES will do all the op erations and computations needed in
order to render the scene without ha ving an y input from the user ab out ho w those op erations and
computations m ust b e p erformed. As an example, if taking ligh ting, once the ligh t is dened with its
p ositioning and its strength, it is Op enGL ES the one to manage ho w the reections are rendered,
ho w the shades will b e, etc. This v ersion oers for the dev elop er an easy w a y to render ob jects
but, it also puts up some limits.
Op enGL ES 2.0 p ermit the dev elop er to write co de in the form of small programs that will run
on the GPU. Suc h programs cannot b e written in Ob jectiv e-C, C or C++. These programs m ust b e
written with the purp ose of talking to the GPU in a sp ecic language designed . That language
is the Op enGL Shader Language (Op enGL SL or OGSL).
The programs written in Op enGL SL are called shaders. Are t w o t yp es: v ertex shaders and
fragmen t shaders. Shaders are small programs that run on the GPU. They are the elemen t that
mak es the programmable pip eline receiv e the programmable attribute.
There are t w o foundational classes in the Android framew ork that let create and manipulate
graphics with the Op enGL ES API: GLSurface View and GLSurface View.Renderer.
GLSurface View
GLSurface View tak es care of the grittier asp ects of Op enGL initialization, suc h as conguring the
displa y and rendering on a bac kground thread. This dra wn is done on a sp ecial area of the displa y ,
called a surface. This also referred to as a viewp ort.
In Android, activities can b e created and destro y ed, and they are also paused and resumed
when the user switc hes to another activit y and later return. The GLSurface View class mak es easy
4

Op en Gr aphics Libr ary for Emb e dde d Systems
to gestionate the standard Android activit y life cycle. GLSurface View pro vides help er metho ds
for release Op enGL's resources when activit y are paused. F or capture touc h screen ev en ts, m ust
extended the GLSurface View class to implemen t the touc h listeners.
GLSurface View.Renderer
This in terface denes the metho ds required for dra wing graphics in a GLSurface View. Must pro-
vided an implemen tation of this in terface as a separate class and attac hed it to an GLSurface View
instance using GLSurface View.setRenderer(). The GLSurface View.Renderer in terface requires that
implemen t the follo wing metho ds:
1. onSurfaceCreated(): The system calls this metho d once, when creating the GLSurface View.This
metho d is used to p erform actions that need to happ en only once, suc h as setting Op enGL
en vironmen t parameters or initializing Op enGL graphic ob jects.
2. onDra wF rame(): The system calls this metho d on eac h redra w of the GLSurface View. This
metho d is used as the primary execution p oin t for dra wing (and re-dra wing) graphic ob jects.
3. onSurfaceChanged(): The system calls this metho d when the GLSurface View geometry c hanges,
including c hanges in size of the GLSurface View or orien tation of the device screen. F or ex-
ample, the system calls this metho d when the device c hanges from p ortrait to landscap e
orien tation. This metho d is used to resp ond to c hanges in the GLSurface View con tainer.
1.1 Building Ob jects
F or building ob jects is used a set of indep enden t p oin ts. This p oin ts are kno wn as v ertices. F or
dra wing ob jects is used shaders. Ev ery ob ject are built b y joining together v ertices in to p oin ts,
lines, and triangles. All these primitiv es get dra wn b y using shaders.
F or dra wing an ob ject, rst, the structure of ob ject m ust dene in a form that Op enGL under-
stands. In Op enGL, the structure of ev erything b egins with a v ertex.
A v ertex is a p oin t represen ting one corner of a geometric ob ject. That p oin t a v e v arious
attributes asso ciated. The most imp ortan t attribute is the p osition. This represen ts where this
v ertex is lo cated in space.
P oin ts, Lines, and T riangles
In Op enGL only p oin ts, lines, and triangles can b e dra w.
The triangle is the most basic geometric shap e around. P oin ts and lines can b e used for certain
eects, but only triangles can b e used to construct an en tire scene of complex ob jects and textures.
T riangles are built in Op enGL b y grouping individual v ertices together, and then it should b e
men tioned literally ho w Op enGL connect the dots. Ev erything what is build needs to b e dened in
terms of these p oin ts, lines, and triangles, and if is builded more complex shap es, suc h as an arc h,
then is needed enough p oin ts to appro ximate the curv e.
5

Op en Gr aphics Libr ary for Emb e dde d Systems
The main problem is the language of en vironmen t. The en vironmen t where co de runs and the
en vironmen t where Op enGL runs don't sp eak the same language. Op enGL runs directly on top of
the hardw are as a nativ e system library . There's no virtual mac hine, and there's no garbage col-
lection or memory compaction. When are used metho ds from the andr oid.op engl.GLES20 pac k age,
the SDK is actually using Ja v a Nativ e In terface (JNI) b ehind the scenes to call the nativ e system
library .
After the structure of the ob ject is dened, and the data is copied to nativ e memory , where
Op enGL will b e able to access it, b efore they are dra w to the screen they are sen t through the
Op enGL pip eline. T o do this, small subroutines kno wn as shaders, are used. These shaders tell the
graphics pro cessing unit (GPU) ho w to dra w the ob ject.
1.2 Shaders
In Op enGL ES are t w o t yp es of shaders : v ertex shaders and fragmen t shaders . Shaders don't
ha v e strict return v alues. Some output v ariables m ust ha v e a v alue written b efore the end of the
shader's main, but from a practical p oin t of view, these v ariables w ork as required return v alues.
1. A v ertex shader generates the nal p osition of eac h v ertex and is run once p er v ertex. Once
the nal p ositions are kno wn, Op enGL will tak e the visible set of v ertices and assem ble them
in to p oin ts, lines, and triangles.
2. A fragmen t shader generates the nal color of eac h fragmen t of a p oin t, line, or triangle and
is run once p er fragmen t. A fragmen t is a small, rectangular area of a single color, analogous
to a pixel on a computer screen.
Once the nal colors are generated, Op enGL will write them in to a blo c k of memory kno wn as
the frame buer, and Android will then displa y this frame buer on the screen. The shaders can
shoud b e used to con trol ho w eac h v ertex gets dra wn to the screen, and they can also con trol ho w
eac h fragmen t of ev ery p oin t, line, and triangle gets dra wn. It is p ossible to do p er-pixel ligh ting
and other neat eects, lik e carto on-cel shading. It is p ossible to add an y custom eect, as long as
it can express in the shader language.
As its name sho ws, the v ertex shader, is where the v ertex pro cessing is p erformed. T o pro cess a
v ertex means: scaling, rotating ob jects , sim ulating p ersp ectiv es, transforming texture co ordinates.
T o group them, v ertex pro cessing ma y b e understo o d as an y calculation that aects v ertices directly
or indirectly through sets of data stored on a p erv ertex basis. A v ertex shader is called once for
ev ery v ertex. It use the v ertex shader's output for the new p ositioning of eac h v ertex.
A v ertex shader needs certain data as input: attributes, uniforms, samplers (a sp ecic t yp e
of uniforms that represen t textures used b y the v ertex shader), samplers in a v ertex shader are
optional, shader Program (v ertex shader program source co de or executable that describ es the
op erations that will b e p erformed on the v ertex).
6

Op en Gr aphics Libr ary for Emb e dde d Systems
A ttributes are p er-v ertex data supplied using v ertex arra ys and uniforms are constan t data used
b y the v ertex shader.
Figure 1.3: V ertex Shader
The v ertex shader returns the so called v arying v ariables. These v ariables are assigned to eac h
v ertex in a primitiv e rasterization stage. Using in terp olation, v arying v ariables are passed in as
inputs to the fragmen t shader.
The Rasterizer
There are t w o stages in Figure 1.2 in b et w een the v ertex and fragmen t stages: primitiv e assem bly
and rasterization. Op enGL creates an image that w e can map on to the pixels of mobile displa y
b y breaking do wn eac h p oin t, line, and triangle in to a bunc h of small fragmen ts through a pro cess
kno wn as rasterization. As it generates the fragmen t job it calculates the in terp olated v alues for
v aryings using something called barycen tric co ordinates. In the example b elo w, on the left side is
a triangle with red, green, and blue sp ecied as the colour attributes for the three v ertices. On the
righ t the colours ha v e b een in terp olated across the triangle.
Figure 1.4: In terp olating p er-v ertex colours across a triangle
In next gure is an example of ho w Op enGL migh t rasterize a line on to a set of fragmen ts.
The displa y system usually maps these fragmen ts directly to the pixels on the screen so that one
fragmen t corresp onds to one pixel. Ho w ev er, this isn't alw a ys true: a sup er high-res device migh t
w an t to use bigger fragmen ts so that the GPU has less w ork to do.
7

Op en Gr aphics Libr ary for Emb e dde d Systems
Figure 1.5: Rasterization: generating fragmen ts
In the dra wing op eration, the fragmen t shader runs once for eac h fragmen t. A fragmen t includes
ev erything that could aect the nal color of a pixel. F ragmen t shaders are also called pixel shaders.
Lik e ths v ertex shaders, fragmen t shaders ha v e an input and an output. The output is the nal
color for the pixel, gl _FragColor , corresp onding to the fragmen t. The fragmen t shader do esn't
run for that pixel if is nothing in the virtual w orld that ma y ha v e an eect on a particular screen
pixel.
The p ossible inputs of the fragmen t shader are follo ws: uniforms (constan t data used b y the
fragmen t shader), v arying v ariables (outputs of the v ertex shader that are generated b y the ras-
terization unit for eac h fragmen t using in terp olation), samplers (a sp ecic t yp e of uniforms that
represen t textures used b y the fragmen t shader), shader program (fragmen t shader program source
co de or executable that describ es the op erations that will b e p erformed on the fragmen t).
Figure 1.6: F ragmen t Shader
1.3 Op enGL Programs
An Op enGL program is simply one v ertex shader and one fragmen t shader link ed together in to
a single ob ject. V ertex shaders and fragmen t shaders alw a ys go together. There ma y b e man y
programs implemen ted in an Op enGL ES application but only one program can run at a time.
8

Op en Gr aphics Libr ary for Emb e dde d Systems
Without a v ertex shader, Op enGL w ouldn't kno w where to dra w the fragmen ts and without a
fragmen t shader, Op enGL w ouldn't kno w ho w to dra w the fragmen ts. F ragmen ts mak e up eac h
p oin t, line, and triangle.
The same shader can b e used in more than one program at a time.
The dev elop er can apply dieren t eects to ob jects in one single scene, using man y programs.
In order to use a program it m ust b e created and the shaders forming it m ust b e loaded and com-
piled some time b et w een the application start and the dra wing but it is a m ust that these op erations
happ en b efore the dra wing b ecause loading and compiling shaders ma y use man y resources.
T o load a program the steps are: load the shader source co de in to memory , create an
empt y shader ob ject, pass the source co de to the new empt y shader ob ject, compile
the shader, c hec k the compile status and that the shader compiled correctly , link the
program, use the program, and, nally , delete the program from memory when done .
The application is ready to start with the dra wing, after loading a program. T o ac hiev e the
dra wing step successfully the follo wing steps are required: use glViewp ort to dene the size
of the screen where Op enGL ES will b e displa y ed, use glClearColor to set an R GB
clear color, use glClear to clear the screen, use GLoat arra y for the v ertices to cre-
ate the geometry , use glV ertexA ttribP oin ter to load the v ertex data in memory , use
glDra wArra ys ( or glDra wElemen ts or glDra wRangeElemen ts) to dra w .
1.4 Color and Shade
Op enGL uses the additiv e R GB color mo del. So, w orks with just the three primary colors: blue,
green, and red. By mixing these primary colors together in v arious prop ortions, man y colors can
b e created.
Op enGL assumes that these colors all ha v e a linear relationship with eac h other: a red v alue of
0.5 should b e t wice as brigh t as a red v alue of 0.25, and a red v alue of 1 should b e t wice as brigh t
as a red v alue of 0.5. These primary colors are clamp ed to the range [0, 1], with 0 represen ting the
absence of that particular primary color and 1 represen ting the maxim um strength for that color.
Using enough triangles, eac h with a dieren t color, it can fo ol the view er in to seeing a complex
and nice scene with a ric h v ariation of colors. If is use a dieren t color at eac h p oin t of a triangle
and blended these colors across the surface of the triangle, it w ould end up with a smo othly shaded
triangle.
Op enGL giv es a w a y to smo othly blend the colors at eac h v ertex across a line or across the
surface of a triangle. Exist the p osibilities to add a color attribute to eac h p oin t. In the case of
triangles the order of co ordinates is: X, Y, R, G, B . The next step will b e to remo v e the color
uniform from the shader and replace it with an attribute. F or colors to v ary across the surface of a
triangle is necessary to use a sp ecial v ariable t yp e kno wn as a v arying.
A v arying is a sp ecial t yp e of v ariable that blends the v alues giv en to it and sends these v alues
9

Op en Gr aphics Libr ary for Emb e dde d Systems
to the fragmen t shader. If the fragmen t b elongs to a line, then Op enGL will use the t w o v ertices
that mak e up that line to calculate the blended color. If the fragmen t b elongs to a triangle, then
Op enGL will use the three v ertices that mak e up that triangle to calculate the blended color.
Ho w Do es a V arying Get Blended at Eac h F ragmen t?
Exist the p osibilit y to blend more than just colors, sending an y v alue to a v arying, and Op enGL
will tak e the t w o v alues b elonging to a line, or the three b elonging to a triangle, and smo othly blend
these v alues across the primitiv e, with a dieren t v alue for eac h fragmen t. This blending is done
using linear in terp olation.
Line ar Interp olation A long a Line
The blended colors along a line with a red v ertex and a green v ertex, w ould lo ok something lik e
this:
Eac h color is scales linearly along the length of the line. Since the left v ertex of the line is red
and the righ t v ertex is green, the left end of the line should b e 100 p ercen t red, the middle should
b e 50 p ercen t red, and the righ t should b e 0 p ercen t red:
The same thing happ ens with green. Since the left v ertex is red and the righ t v ertex is green,
the left end of the line will b e 0 p ercen t green, the middle will b e 50 p ercen t green, and the righ t
will b e 100 p ercen t green:
A dding the t w o together, is obtined a blended line.
The strength of eac h color dep ends on the distance of eac h fragmen t from the v ertex con taining
that color. The distance ratio is simply a ratio b et w een 0 and 100 p ercen t, with 0 p ercen t b eing
the left v ertex and 100 p ercen t b eing the righ t v ertex.
1.5 Camera Space
EYE, A T, and UP are three parameters that sp ecis the camera p ose, as sho wn on the left of Fig.
1.7.
10

Op en Gr aphics Libr ary for Emb e dde d Systems
* EYE is the camera p osition in the w orld space.
* A T is a reference p oin t in the scene at whic h the camera is p oin ting. (In the example, A T is
lo cated at the tip of the teap ot's mouth.)
* UP is a v ector that roughly describ es where the top of the camera is p oin ting. (In most cases,
UP is set to the v ertical axis of the w orld space, i.e., the y-axis.)
These parameters are used to build the camera space, whic h is also called the view space or ey e
space. Its origin is EYE. The b o x in the middle of Fig. 1.7, sho ws the sequen tial pro cess of building
the basis, {u, v, n}, of the camera space.
-> rst, the v ector connecting A T and EYE is normalized to comp ose a unit v ector n
-> the cross pro duct of UP and n is normalized to dene another unit v ector u
-> the cross pro duct of n and u determines a unit v ector v.
{u, v, n} is an orthonormal basis. The camera space is denoted as {u, v, n, EYE}.
Figure 1.7: The camera space {u, v, n, EYE}
In image, with resp ect to the camera space, the camera is lo cated at the origin and p oin ts in
the n direction.
1.6 Mapping Co ordinates for Dra wn Ob jects
Android devices screens can v ary in size and shap e. This is one of the basic problems in displa ying
graphics on Android devices. Op enGL assumes a square, uniform co ordinate system and, b y default,
happily dra ws those co ordinates on to t ypically non-square screen as if it is p erfectly square.
11

Op en Gr aphics Libr ary for Emb e dde d Systems
Figure 1.8: (left) Default Op enGL co ordinate system mapp ed to (righ t) a t ypical Android device
screen
On the left, the gure sho ws the uniform co ordinate system assumed for an Op enGL frame.
On the righ t, the gure sho ws ho w these co ordinates actually map to a t ypical device screen in
landscap e orien tation . So on, to solv e this problem, Op enGL pro jection mo des and camera views
to transform co ordinates so graphic ob jects ha v e the correct prop ortions on an y displa y .
In order to apply pro jection and camera views, a pro jection matrix and a camera view matrix
are created and are applied to the Op enGL rendering pip eline. The pro jection matrix recalculates
the co ordinates of graphics so that they map correctly to Android device screens. The camera view
matrix creates a transformation that renders ob jects from a sp ecic ey e p osition.
1.6.1 Pro jection and camera view in Op enGL ES 2.0
In the ES 2.0 API, pro jection and camera view is apply b y rst adding a matrix mem b er to the v ertex
shaders of graphics ob jects. With this matrix mem b er added, the generate and apply pro jection
and camera viewing matrices is p ossible to ob jects. Steps are:
1. the matrix is added to v ertex shaders – a v ariable is created for the view pro jection matrix;
the v ariable is included as a m ultiplier of the shader's p osition
2. the shader matrix is accessed – after the creating a ho ok in v ertex shaders to apply pro jection
and camera view, access that v ariable to apply pro jection and camera viewing matrices is p os-
sible. T o access the matrix v ariable dened in the v ertex shader ab o v e, the onSurfaceCreated()
metho d of a GLSurface View.Renderer implemen tation is mo died
3. the pro jection and camera viewing matrices are created- the pro jection and viewing matri-
ces are generated to b e applied the graphic ob jects. The onSurfaceCreated() and onSur-
faceChanged() metho ds of a GLSurface View.Renderer implemen tation are mo died to create
camera view matrix and a pro jection matrix based on the screen asp ect ratio of the device
4. the pro jection and camera viewing matrices are applied – to apply the pro jection and camera
view transformations, the matrices are m ultiplyied together and then set them in to the v ertex
12

Op en Gr aphics Libr ary for Emb e dde d Systems
shader. The onDra wF rame() metho d of a GLSurface View.Renderer implemen tation is mo di-
ed to com bine the pro jection matrix and camera view and then is applied it to the graphic
ob jects to b e rendered b y Op enGL.
1.7 3D T ransforms and Matrix Represen tations
In Op enGL, one of the most imp ortan t uses of a matrix is in setting up orthographic and p ersp ectiv e
pro jections. T o do a pro jection just in v olv es a bunc h of adds and m ultiplies o v er a set of data.
GPUs are v ery fast to do this.
A v ector is a one-dimensional arra y of elemen ts. In Op enGL, a p osition is usually a four-elemen t
v ector, as is a color.
A matrix is a t w o-dimensional arra y of elemen ts. In Op enGL, generally matrices are used
to pro ject v ectors using an orthographic or p ersp ectiv e pro jection, and they also are used to do
rotations, translations, and scaling of an ob ject. This is do es b y m ultiplying the matrix with eac h
v ector that w e w an t to transform.
Left-hand System and Righ t-hand System
A 3D Cartesian co ordinate system can b e lik e in Fig. 1.9. In the righ t-hand system, the th um b
of the righ t hand p oin ts to w ard the p ositiv e end of the z -axis and the other four ngers curl from
the x -axis to the y-axis. In the left-hand system, the same rule is applied with the left hand.
Op enGL uses therigh t-hand system.
Figure 1.9: Righ t-hand system v ersus left-hand system
Matrix-V ector Multiplication
2
666664xxxyxzxw
yxyyyzyw
zxzyzzzw
wxwywzww3
7777752
666664x
y
z
w3
777775=2
666664xxx+xyy+xzz+xww
yxx+yyy+yzz+yww
zxx+zyy+zzz+zww
wxx+wyy+wzz+www3
777775
All four comp onen ts of the rst ro w of the matrix will aect the resulting x, all four comp onen ts
of the second ro w will aect the resulting y , and so on.
13

Op en Gr aphics Libr ary for Emb e dde d Systems
1.7.1 Scaling
Figure 1.10: Scaling examples
-> the rst ob ject is a uniform scaling in that all scaling factors are iden tical
-> the second is a non-uniform scaling
The scaling matrix applied to a 3D v ector:
2
666664sx0 0 0
0sy0 0
0 0 sz0
0 0 0 13
7777752
666664x
y
z
13
777775=2
666664sxx
syy
szz
13
777775
1.7.2 T ranslation
(dx; dy; dz) is the translation v ector. This is inserted in to the fourth column of the 44 iden tit y
matrix. This matrix can b e m ultiplied with the homogeneous co ordinates of the 3D p oin t:
2
6666641 0 0 dx
0 1 0 dy
0 0 0 dz
0 0 0 13
7777752
666664x
y
z
13
777775=2
666664x+dx
y+dy
z+dz
13
777775
1.7.3 Rotation
T o gure out ho w an ob ject w ould rotate around a giv en axis, the righ t-hand rule is used: the
th um b in the direction of the p ositiv e axis and the curl of ngers sho ws ho w an ob ject w ould rotate
around that axis, giv en a p ositiv e angle of rotation.
Matrix rotation uses the trigonometric functions of sine and cosine to con v ert the rotation angle
in to scaling factors. The follo wing is the matrix denition for a rotation around the x -axis, y -axis
and z -axis:
14

Op en Gr aphics Libr ary for Emb e dde d Systems
Figure 1.11: (a)Rz(900):(b)Rx(900):(c)Ry(900)
2
6666641 0 0 0
0cos(a)sin(a) 0
0sin(a)cos(a) 0
0 0 0 13
777775;2
666664cos(a) 0 sin(a) 0
0 1 0 0
sin(a) 0 cos(a) 0
0 0 0 13
777775;2
666664cos(a)sin(a) 0 0
sin(a)cos(a) 0 0
0 0 1 0
0 0 0 13
777775
Rotation around the x-axis as a test:
2
6666641 0 0 0
0cos(90)sin(90) 0
0sin(90) cos(90) 0
0 0 0 13
7777752
6666640
1
0
13
777775=2
6666640
0
1
13
777775
The p oin t has b een mo v ed from (0, 1, 0) to (0, 0, 1).
P anning the Camera Around the Scene
P anning the camera around the scene can easily do b y listening for touc h ev en ts and using those
touc h ev en ts to rotate the scene.
A touc h listener is dened that will measure ho w far are dragged nger b et w een eac h successiv e call
to onT ouc h(). A t rst touc h of screen, the curren t touc h p osition will b e recorded in previousX and
previousY. As nger are draged across the screen, a bunc h of drag ev en ts are got ,and eac h time is
do, the dierence b et w een the new p osition and the old p osition is tak ed and are stored that in to
deltaX and deltaY, and then previousX and previousY are up dated. The deltas will b e passed in to
the particle renderer b y calling handleT ouc hDrag(). The metho d will tak e in the distance the user
dragged in eac h direction and add it to xRotation and yRotation, whic h represen t the rotation in
degrees.
15

Op en Gr aphics Libr ary for Emb e dde d Systems
1.8 Orthographic Pro jections
An orthographic pro jection maps part of 3D w orld on to the screen. It do es so in a w a y suc h that
ev erything app ears in the same size (no matter ho w near or far it is). This t yp e of pro jection is
great for doing 2D games and art.
Constructing GL _PROJECTION matrix for orthographic pro jection is m uc h simpler than
p ersp ectiv e mo de.
Allxe ,ye andze comp onen ts in EYE space are linearly mapp ed to NDC. Just is needed to scale
a rectangular v olume to a cub e, then m ust mo v ed to the origin.
A basic denition of an orthographic pro jection matrix is:
2
6666642
rightleft0 0 right +left
rightleft
02
topbottom0top+bottom
topbottom
0 02
farnearfar+near
farnear
0 0 0 13
777775
All co ordinates b et w een left and righ t, near and far, b ottom and top will b e mapp ed in to
normalized device co ordinates. An ything within this range will b e visible on the screen.
The matrix b y passing in -1 for left, b ottom, and near, and +1 for righ t, top, and far:
2
6666642
110 01+1
11
02
1101+1
11
0 02
111+1
11
0 0 0 13
777775=)2
6666642
1+10 011
1+1
02
1+1011
1+1
0 02
1+111
1+1
0 0 0 13
777775=)2
6666641 0 0 0
0 1 0 0
0 01 0
0 0 0 13
777775
It lo oks almost exactly lik e the iden tit y matrix. This happ ens b ecause normalized device co-
ordinates range from -1 to 1 on eac h axis, so when w e also pass in -1 and 1 as our ranges, w e are
16

Op en Gr aphics Libr ary for Emb e dde d Systems
essen tially asking for an orthographic pro jection that lea v es its co ordinates unc hanged, just lik e an
iden tit y matrix. The main dierence is that the z-axis is in v erted.
Creating a Regular Orthographic Pro jection
A matrix that will map [0,1] on to the range [-1, 1] for the x, y , and z comp onen ts w ould lo ok lik e:
2
6666642
100 01+0
10
02
1001+0
10
0 02
101+0
10
0 0 0 13
777775=)2
6666642 0 01
0 2 01
0 021
0 0 0 13
777775
T o pro v e that this matrix will indeed transform a co ordinate in the range [0;1] to the range [1;1]
w e'll start with a v ector at the minim um range, with all comp onen ts at 0:
2
6666642 0 01
0 2 01
0 021
0 0 0 13
7777752
6666640
0
0
13
777775=2
6666641
1
1
13
777775
As exp ected, ev erything maps to -1 in normalized device co ordinates.
1.9 P ersp ectiv e Pro jections
P ersp ectiv e pro jections map part of the virtual w orld on to the screen, but they do so b y using
p ersp ectiv e: the further a w a y something is, the smaller it will app ear on the screen. The pro jection
matrix uses a fourth comp onen t w, in conjunction with the p ersp ectiv e divide.
17

Op en Gr aphics Libr ary for Emb e dde d Systems
In p ersp ectiv e pro jection, a 3D p oin t in a truncated p yramid frustum (EYE co ordinates) is
mapp ed to a cub e (NDC). The range of x-co ord. from [l; r] to[1;1] , the y co ord. from [b; t] to
[1;1] and the z-co ord. from [n;f] to[1;1] .
The EYE co ordinates are dened in the righ t-handed co ordinate system. NDC uses the left-
handed co ordinate system. Meaning, the camera at the origin is lo oking along Z axis in EYE
space, but it is lo oking along +Z axis in NDC.
A v ery basic p ersp ectiv e pro jection matrix, with the near plane b eginning at a z of -1:
2
6666641 0 0 0
0 1 0 0
0 012
0 01 03
777775
? The rst t w o ro ws of this matrix simply cop y o v er the x andy comp onen ts
? The third ro w copies o v er the z comp onen t, in v erting it at the same time. The -2 in the fourth
column will b e m ultiplied with a v ertex's w, whic h defaults to 1, so this ro w of the matrix
ends up setting the nal z to z2 . F or example, a z of -1 will b e set to -1, a z of -2 will b e
set to 0, a z of -3 will b e set to 1, and so on.
With a -1 in the third column, a -2 in the fourth column is simply the n um b er that will map
the near plane of the frustum to the near plane in normalized device co ordinates (-1 will map
to -1); and since the same z of -1 will create a w of 1, the z v alue will remain -1 after the
p ersp ectiv e divide.
? The fourth ro w sets the nal w to the negativ e z. Once Op enGL do es the p ersp ectiv e divide,
this will ha v e the eect of shrinking ob jects that are further a w a y .
Multiply this matrix with -> a p oin t on the near end of the frustum:
2
6666641 0 0 0
0 1 0 0
0 012
0 01 03
7777752
6666641
1
1
13
777775=2
6666641
1
1
13
777775
With a z of -1, the result has a z of -1 and a w of 1.
-> a p oin ts further a w a y than the previous one:
18

Op en Gr aphics Libr ary for Emb e dde d Systems
2
6666641 0 0 0
0 1 0 0
0 012
0 01 03
7777752
6666641
1
3
13
777775=2
6666641
1
1
33
777775
As the p oin t gets further a w a y , b oth z and w get larger and larger.
Dividing b y W
W e'v e explored the rst part of the magic of p ersp ectiv e pro jection: dening a matrix that will
create a w v alue that increases as distance increases. After Op enGL divides eac h comp onen t b y w,
w e'll end up with the follo wing results:
2
6666641
1
1
13
777775=)2
6641
1
1
1
1
13
775=2
6641
1
13
775
2
6666641
1
1
33
777775=)2
6641
3
1
3
1
33
775=2
6640:33
0:33
0:333
775
2
6666641 0 0 0
0 1 0 0
0 012
0 01 03
7777752
6666641
1
10000
13
777775=2
6666641
1
9998
100003
777775=)2
6641
10000
1
10000
9998
100003
775=2
6640:0001
0:0001
0:99983
775
After Op enGL divides ev erything b y w, the relationship b et w een input z and output z is no longer
linear. Going from an input z of -1 to -2 increases the output z from -1 to 0, but going from an input
z of -10,000 to -20,000 only has a v ery small eect on the output z. This nonlinear relationship has
imp ortan t implications when it comes to rendering ob jects at dieren t distances in scene.
19

Chapter 2
Application
Creating our o wn application can b e the b est indicator in terms of understanding and the lev el
of kno wledge acquired. Op enGL ES 2.0 oers a lot of freedom and dieren t applications can
b e created. Starting from simple applications suc h as: creating a free scene, dra wing an ob ject,
applying dieren t transformations on a created ob ject, w e can get to dra w a complex scene, with
dieren t ob jects that can b e easily confused with realit y . The application that will b e presen ted
b elo w started from the idea of a famous pain ter, Segi Cadenas.
Segi Cadenas w as b orn in Girona in 1972. The artist in him expressed itself when he w as 30 and
he turned in to an artist. He nev er got to formally study dra wing or pain ting. He learned ho w to do
oil pain ting and fo cused on p ortraits. His creativ e mind w asn't con ten t with mere pain ting p ortraits,
so he disco v ered new w a ys of creating. It w as through exp erimen tation that he created his unique
st yle all b y himself, where in his 3D oil pain tings c hange according to the view er's p ersp ectiv e. And
the result w as his incredible 3D w orks in oil.
What's surprising is this artist is totally self-taugh t.
The application is divided in to t w o parts:
-> the rst part is the preparation of the scene in whic h the pain ting will b e dra wn
-> the second part is related to the transformations applied to the pain ting and the camera: dis-
tance, approac h, rotation, etc.
20

Applic ation
In order to stop Go ogle Pla y users who do not ha v e devices that supp ort the Op enGL ES v ersion
needed for installing the app, is added the follo wing < usesfeature > tag in the pro ject's manifest
le:
< usesfeatureandroid :required = "true"android :glEsV ersion = "0x00020000" >
The application is designed to in teract in landscap e mo de when running, b y calling the MainA ctivit y
class:
< activityandroid :screenOrientation = "sensorLandscape "android :name = ":MainActivity ">
InMainActivity:class is o v erride te metho d onCreate () for create a GLSurface View instance and
set it as View for glSurface ViewLa y out:
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
glView = new MyGLSurfaceView(this);
RelativeLayout layout = (RelativeLayout) findViewById(R.id.glSurfaceViewLayout);
RelativeLayout.LayoutParams glParams = new RelativeLayout.LayoutParams
(RelativeLayout.LayoutParams.MATCH_PARENT, RelativeLayout.LayoutParams.MATCH_PARENT);
layout.addView(glView, glParams);
}
This metho d is call bac k when the activit y is started, to initialize the view.
Also here a ob ject of MyGLSurface View' s class is initialize, ob ject that will represen t the scene:
private MyGLSurfaceView glView;
The screen is divided in t w o: a RelativLa y out (glSurface ViewLa y out) and a LinearLa y out (con-
tains dieren t buttons):
…
<LinearLayout
android:layout_width="match_parent"
android:layout_height="match_parent"
android:weightSum="100"
android:orientation="horizontal">
<RelativeLayout
android:id="@+id/glSurfaceViewLayout"
21

Applic ation
android:layout_height="match_parent"
android:layout_width="match_parent"
android:layout_weight="25"/>
<LinearLayout
android:layout_width="match_parent"
android:layout_height="match_parent"
android:layout_weight="75"
android:orientation="vertical">
<LinearLayout
android:layout_width="match_parent"
android:layout_height="75dp"
android:layout_weight="15"
android:orientation="horizontal">
…
In MyGLSurface View.class a ob ject of MyGLRenderer' s class is initialize:
private final MyGLRenderer renderer;
The constuctor of MyGLSurface View's class is dened:
public MyGLSurfaceView(Context context){
super(context);
// Create an OpenGL ES 2.0 context
setEGLContextClientVersion(2);
renderer = new MyGLRenderer(context);
// Set the Renderer for drawing on the GLSurfaceView
setRenderer(renderer);
setCameraPosition();
// Render the view only when there is a change in the drawing data
setRenderMode(GLSurfaceView.RENDERMODE_WHEN_DIRTY);
}
The metho d setCameraPosition () is v ery imp orten t b ecause it calculate the camera's p osition in
Cartesian co ordinates:
public void setCameraPosition(){
double y = lookY + CameraR * Math.sin(CameraPhi);
double hyp = CameraR * Math.cos(CameraPhi);
double x = lookX + hyp * Math.cos(CameraTheta);
22

Applic ation
double z = lookZ + hyp * Math.sin(CameraTheta);
renderer.setCameraPosition((float)x,(float)y,(float)z,lookX,lookY,lookZ);
requestRender();
}
23

Bibliography
[1] Kevin Brothaler, Op enGL ES 2 for A ndr oid , 2013
[2] JungHyun Han, Intr o duction to Computer Gr aphics with Op enGL ES , 2018
[3] h ttps://dev elop er.android.com/training/graphics/op engl
[4] Prateek Meh ta, L e arn Op enGL ES for mobile Gr aphis Development , 2013
[5] Munshi A., Ginsburg D., Sc hreiner D. Op enGL ES 2.0 Pr o gr amming Guide , 2008
[6] Aaftab Munshi, Jon Leec h, Op enGL ES Common Pr ole Sp e cic ation V ersion 2.0.25 (F ul l
Sp e cic ation) , 2010
[7] h ttp://barnadas.com/en/w ork/aging-100-x-100-cm/
24