Alexandr u Io an Cuza University fr om Ia³i [618648]

"Alexandr u Io an Cuza" University fr om Ia³i
F a cul ty of Ma thema tics
OpenGL es for Android. Applica tion
Dissertation
Scien tic co ordinator:
Lect. Dr. Apetri i MariusStuden t:
Negr uµ Andreea
JUL Y, 2020
Ia³i

Abstract
The purp ose of this thesis is to enable the reader to dev elop kno wledge, understanding, capabilities
and attitudes to create an Android application using Op enGL ES. It's divided in t w o parts: a
theoretical part,the rst c hapter and a practical part, the second c hapter, that will describ e the
application.

Contents
In tro duction 2
1 Op en Graphics Library for Em b edded Systems 3
1.1 Building Ob jects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2 Shaders in Op enGL ES 2.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Op enGL Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4 Color and Shade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.5 Camera Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.6 Mapping Co ordinates for Dra wn Ob jects . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.6.1 Pro jection and camera view in Op enGL ES 2.0 . . . . . . . . . . . . . . . . . 12
1.7 3D T ransforms and Matrix Represen tations . . . . . . . . . . . . . . . . . . . . . . . 13
1.7.1 Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.7.2 T ranslation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.7.3 Rotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.8 The Math Behind Orthographic Pro jections . . . . . . . . . . . . . . . . . . . . . . . 16
1.9 The Math Behind P ersp ectiv e Pro jections . . . . . . . . . . . . . . . . . . . . . . . . 17
2 Application 20
Bibliograph y 20
1

Introduction
Android has just gone through an incredible p erio d of gro wth. Along with Apple, Android also has
a cen tralized mark et a v ailable on ev ery Android phone and tablet, called Go ogle Pla y . With this
mark et installed on ev ery Android device, there's nev er b een a b etter opp ortunit y for an y one who's
ev er had a dream to publish his or her o wn game or liv e w allpap er.
On Android, as w ell as on Apple's iOS and man y other mobile platforms, dev elop ers create 2D
and 3D graphics through a cross-platform application programming in terface called Op enGL. This
cross-platform w as originally dev elop ed b y Silicon Graphics bac k in 1992. Op enGL has b een used
on the desktop for a while no w, and the mobile platforms use a sp ecial em b edded v ersion kno wn as
Op enGL ES. The rst v ersion of Op enGL ES brough t 3D to mobile, whic h w as v ery p opular with
dev elop ers b ecause it w as easy to learn and b ecause it had a w ell-dened feature set. Ho w ev er, this
feature set w as also limited, and it w asn't able to k eep up with the latest and greatest features from
the most p o w erful smartphones and tablets.
Most of the old APIs w ere completely remo v ed and replaced with new programmable APIs,
whic h mak es it m uc h easier to add sp ecial eects and tak e adv an tage of what the latest devices
ha v e to oer. These devices can no w pro duce graphics that riv al consoles from just a few y ears ago.
In August 2012, the Khronos Group nalized the sp ecication for the next v ersion, Op enGL ES
3.0, whic h is fully compatible with 2.0, extending it with a few adv anced features.
Op enGL ES 2.0 has pro v ed its success as a 3D rendering tec hnology b y b eing the standard in
mobile devices.
F urthermore, Op enGL ES 2.0 is based in Op enGL, whic h is widely used in 3D gaming dev el-
opmen t and the computer graphics imagery used in animation mo vies and sp ecial eects for b oth
mo vies and series.
2

Chapter 1
Open Graphics Library for Embedded
Systems
Android includes supp ort for high p erformance 2D and 3D graphics with the Op en Graphics Library
(Op enGL R
), sp ecically , the Op enGL ES API. Op enGL is a cross-platform graphics API that
sp ecies a standard soft w are in terface for 3D graphics pro cessing hardw are. Op enGL ES is a a v or
of the Op enGL sp ecication in tended for em b edded devices.
Op enGL ES is designed in a w a y that it uses a normalized cub e where the 3D scene will b e
rendered. That cub e will determine what is visible and what is not since Op enGL ES will ignore
an ything outside of it, therefore, not b eing dra wn.
Figure 1.1: Normalized cub e
The main dierences b et w een Op enGL ES 1.1 and Op enGL ES 2.0 is that the 1.1 implemen ts
a xed function pip eline and is deriv ed from the Op enGL 1.5 sp ecications while the Op enGL ES
2.0 sp ecication implemen ts a programmable graphics pip eline (Figure 1.2) and is deriv ed from the
Op enGL 2.0 sp ecication.
3

Op en Gr aphics Libr ary for Emb e dde d Systems
Figure 1.2: Op enGL ES 2.0 Graphics Pip eline
The term pip eline refers to is all the op erations happ ening from the dra wing command in
Op enGL ES un til the 3D are fully dra wn on the displa y . The term dra w in Op enGL ES is usually
called rendering. Op enGL ES is con tin uously dra wing in a similar w a y than a video is con tin uously
running and the dieren t images b eing displa y ed are called frames.
Op enGL ES 1.1 (and 1.0) uses a xed rendering pip eline. This means that the data submitted
to Op enGL ES will b e rendered in a w a y that it Op enGL ES will p erform all the op erations and
computations needed in order to render the scene without ha ving an y input from the user ab out
ho w those op erations and computations m ust b e p erformed. That is the case for all the elemen ts
rendered in Op enGL ES 1.1. T aking ligh ting as an example, once the ligh t is dened with its
p ositioning, its strength, etc. it is Op enGL ES the one to manage ho w the shades will b e, ho w the
reections are rendered, etc. This oers an easy w a y for the dev elop er to render ob jects but it
also puts up some limits.
Op enGL ES 2.0 force the dev elop er to write co de in the form of small programs that will run
on the GPU but suc h programs cannot b e written in Ob jectiv e-C, C or C++. These programs are
v ery p o w erful but they m ust b e written in a sp ecic language designed with the purp ose of talking
to the GPU. That language is the Op enGL Shader Language (Op enGL SL or OGSL).
The programs written in Op enGL SL are called shaders and there are t w o t yp es: v ertex shaders
and fragmen t shaders. Shaders are small programs compiled at run time that run on the GPU in
order to p erform certain tasks and they are the elemen t that mak es the programmable pip eline
receiv e the programmable attribute.
There are t w o foundational classes in the Android framew ork that let create and manipulate
graphics with the Op enGL ES API: GLSurface View and GLSurface View.Renderer.
GLSurface View
GLSurface View tak es care of the grittier asp ects of Op enGL initialization, suc h as conguring the
displa y and rendering on a bac kground thread. This rendering is done on a sp ecial area of the
displa y , called a surface; this is also sometimes referred to as a viewp ort.
4

Op en Gr aphics Libr ary for Emb e dde d Systems
The GLSurface View class also mak es it easy to handle the standard Android activit y life cycle.
In Android, activities can b e created and destro y ed, and they are also paused and resumed when
the user switc hes to another activit y and later returns. GLSurface View pro vides help er metho ds
for release Op enGL's resources when activit y are paused. F or capture touc h screen ev en ts, m ust
extended the GLSurface View class to implemen t the touc h listeners.
GLSurface View.Renderer
This in terface denes the metho ds required for dra wing graphics in a GLSurface View. Must pro-
vided an implemen tation of this in terface as a separate class and attac hed it to an GLSurface View
instance using GLSurface View.setRenderer(). The GLSurface View.Renderer in terface requires that
implemen t the follo wing metho ds:
1. onSurfaceCreated(): The system calls this metho d once, when creating the GLSurface View.This
metho d is used to p erform actions that need to happ en only once, suc h as setting Op enGL
en vironmen t parameters or initializing Op enGL graphic ob jects.
2. onDra wF rame(): The system calls this metho d on eac h redra w of the GLSurface View. This
metho d is used as the primary execution p oin t for dra wing (and re-dra wing) graphic ob jects.
3. onSurfaceChanged(): The system calls this metho d when the GLSurface View geometry c hanges,
including c hanges in size of the GLSurface View or orien tation of the device screen. F or ex-
ample, the system calls this metho d when the device c hanges from p ortrait to landscap e
orien tation. This metho d is used to resp ond to c hanges in the GLSurface View con tainer.
1.1 Building Ob jects
F or building ob jects are use a set of indep enden t p oin ts kno wn as v ertices, and for dra wing these
ob jects are use shaders. Ev ery ob ject are built b y joining together v ertices in to p oin ts, lines, and
triangles, and these primitiv es all get dra wn b y using shaders.
The rst step for dra wing an ob ject is to dene the structure of ob ject in a form that Op enGL
understands. In Op enGL, the structure of ev erything b egins with a v ertex.
A v ertex is simply a p oin t represen ting one corner of a geometric ob ject, with v arious attributes
asso ciated with that p oin t. The most imp ortan t attribute is the p osition, whic h represen ts where
this v ertex is lo cated in space.
P oin ts, Lines, and T riangles
In Op enGL only p oin ts, lines, and triangles can b e dra w.
The triangle is the most basic geometric shap e around. P oin ts and lines can b e used for certain
eects, but only triangles can b e used to construct an en tire scene of complex ob jects and textures.
T riangles are built in Op enGL b y grouping individual v ertices together, and then it should b e
men tioned literally ho w Op enGL connect the dots. Ev erything what is build needs to b e dened in
5

Op en Gr aphics Libr ary for Emb e dde d Systems
terms of these p oin ts, lines, and triangles, and if is builded more complex shap es, suc h as an arc h,
then is needed enough p oin ts to appro ximate the curv e.
The main problem is that the en vironmen t where co de runs and the en vironmen t where Op enGL
runs don't sp eak the same language. Op enGL runs directly on top of the hardw are as a nativ e system
library . There's no virtual mac hine, and there's no garbage collection or memory compaction. When
are used metho ds from the andr oid.op engl.GLES20 pac k age, the SDK is actually using Ja v a Nativ e
In terface (JNI) b ehind the scenes to call the nativ e system library .
After the structure of the ob ject is dened, and the data is copied to nativ e memory , where
Op enGL will b e able to access it, b efore they are dra w to the screen they are sen t through the
Op enGL pip eline. T o do this, small subroutines kno wn as shaders, are used. These shaders tell the
graphics pro cessing unit (GPU) ho w to dra w the ob ject.
1.2 Shaders in Op enGL ES 2.0
There are t w o t yp es of shaders for Op enGL ES: v ertex shaders and fragmen t shaders . Shaders
don't ha v e strict return v alues; instead there are some output v ariables that m ust ha v e a v alue
written b efore the end of the shader's main but from a practical p oin t of view, these v ariables w ork
as required return v alues.
1. A v ertex shader generates the nal p osition of eac h v ertex and is run once p er v ertex. Once
the nal p ositions are kno wn, Op enGL will tak e the visible set of v ertices and assem ble them
in to p oin ts, lines, and triangles.
2. A fragmen t shader generates the nal color of eac h fragmen t of a p oin t, line, or triangle and
is run once p er fragmen t. A fragmen t is a small, rectangular area of a single color, analogous
to a pixel on a computer screen.
Once the nal colors are generated, Op enGL will write them in to a blo c k of memory kno wn as
the frame buer, and Android will then displa y this frame buer on the screen. The shaders can
shoud b e used to con trol ho w eac h v ertex gets dra wn to the screen, and they can also con trol ho w
eac h fragmen t of ev ery p oin t, line, and triangle gets dra wn. It is p ossible to do p er-pixel ligh ting
and other neat eects, lik e carto on-cel shading. It is p ossible to add an y custom eect, as long as
it can express in the shader language.
The v ertex shader, as its name sho ws, is where the v ertex pro cessing is p erformed. This means
not only scaling, or rotating ob jects but also sim ulating p ersp ectiv es or transforming texture co or-
dinates. T o sum up, v ertex pro cessing ma y b e understo o d as an y calculation that aects v ertices
directly or indirectly through sets of data stored on a p erv ertex basis. A v ertex shader is called
once for ev ery v ertex, using the v ertex shader's output for the new p ositioning of eac h v ertex.
A v ertex shader needs certain data as input: attributes (p er-v ertex data supplied using v ertex
arra ys), uniforms (constan t data used b y the v ertex shader), samplers (a sp ecic t yp e of uniforms
6

Op en Gr aphics Libr ary for Emb e dde d Systems
that represen t textures used b y the v ertex shader), samplers in a v ertex shader are optional, shader
Program (v ertex shader program source co de or executable that describ es the op erations that will
b e p erformed on the v ertex).
Figure 1.3: Op enGL ES 2.0 V ertex Shader
As for the outputs, the v ertex shader returns the so called v arying v ariables. These v ariables are
assigned to eac h v ertex in a primitiv e rasterization stage and then v arying v ariables are passed in
as inputs to the fragmen t shader using in terp olation.
The Rasterizer
There are t w o stages in Figure 1.2 in b et w een the v ertex and fragmen t stages: primitiv e assem bly
and rasterization. Op enGL creates an image that w e can map on to the pixels of mobile displa y
b y breaking do wn eac h p oin t, line, and triangle in to a bunc h of small fragmen ts through a pro cess
kno wn as rasterization. As it generates the fragmen t job it calculates the in terp olated v alues for
v aryings using something called barycen tric co ordinates. In the example b elo w, on the left side is
a triangle with red, green, and blue sp ecied as the colour attributes for the three v ertices. On the
righ t the colours ha v e b een in terp olated across the triangle.
Figure 1.4: In terp olating p er-v ertex colours across a triangle
In next gure is an example of ho w Op enGL migh t rasterize a line on to a set of fragmen ts.
The displa y system usually maps these fragmen ts directly to the pixels on the screen so that one
fragmen t corresp onds to one pixel. Ho w ev er, this isn't alw a ys true: a sup er high-res device migh t
w an t to use bigger fragmen ts so that the GPU has less w ork to do.
7

Op en Gr aphics Libr ary for Emb e dde d Systems
Figure 1.5: Rasterization: generating fragmen ts
The fragmen t shader runs once for ev ery fragmen t in the dra wing op eration. A fragmen t includes
ev erything that could p oten tially ha v e eects on the nal color of a pixel. F ragmen t shaders are also
called pixel shaders. F ragmen t shaders are also required to get an input and ha v e an output. The
output is the nal color for the pixel corresp onding to the fragmen t . Ho w ev er, if there is nothing
in the virtual w orld that ma y ha v e an eect on a particular screen pixel, then the fragmen t shader
do esn't run for that pixel.
As for the inputs of the fragmen t shader, in they are describ ed as follo ws: v arying v ariables
(outputs of the v ertex shader that are generated b y the rasterization unit for eac h fragmen t using
in terp olation), uniforms (constan t data used b y the fragmen t shader), samplers (a sp ecic t yp e of
uniforms that represen t textures used b y the fragmen t shader), shader program (fragmen t shader
program source co de or executable that describ es the op erations that will b e p erformed on the
fragmen t).
Figure 1.6: Op enGL ES 2.0 F ragmen t Shader
1.3 Op enGL Programs
An Op enGL program is simply one v ertex shader and one fragmen t shader link ed together in to
a single ob ject. V ertex shaders and fragmen t shaders alw a ys go together. There ma y b e man y
8

Op en Gr aphics Libr ary for Emb e dde d Systems
programs implemen ted in an Op enGL ES application but only one program can run at a time.
Without a fragmen t shader, Op enGL w ouldn't kno w ho w to dra w the fragmen ts that mak e up eac h
p oin t, line, and triangle; and without a v ertex shader, Op enGL w ouldn't kno w where to dra w these
fragmen ts.
Ha ving man y programs allo w the dev elop er to apply dieren t eects to ob jects in one single
scene.
In order to use a program it m ust b e created and the shaders forming it m ust b e loaded and com-
piled some time b et w een the application start and the dra wing but it is a m ust that these op erations
happ en b efore the dra wing b ecause loading and compiling shaders ma y use man y resources.
The follo wing steps are required in order to load a program: load the shader source co de in to
memory , create an empt y shader ob ject, pass the source co de to the new empt y shader
ob ject, compile the shader, c hec k the compile status and that the shader compiled
correctly , link the program, use the program, delete the program from memory when
done .
After loading a program, the application is no w ready to start with the dra wing. The follo wing
steps are required to ac hiev e the dra wing step successfully: use glViewp ort to dene the size
of the screen where Op enGL ES will b e displa y ed, use glClearColor to set an R GB
clear color, clear the screen using glClear, create the geometry using GLoat arra y for
the v ertices, use glV ertexA ttribP oin ter to load the v ertex data in memory , dra w using
glDra wArra ys, glDra wElemen ts or glDra wRangeElemen ts .
1.4 Color and Shade
Op enGL uses the additiv e R GB color mo del, whic h w orks with just the three primary colors: red,
green, and blue. Man y colors can b e created b y mixing these primary colors together in v arious
prop ortions.
Op enGL assumes that these colors all ha v e a linear relationship with eac h other: a red v alue of
0.5 should b e t wice as brigh t as a red v alue of 0.25, and a red v alue of 1 should b e t wice as brigh t
as a red v alue of 0.5. These primary colors are clamp ed to the range [0, 1], with 0 represen ting the
absence of that particular primary color and 1 represen ting the maxim um strength for that color.
Using enough triangles, eac h with a dieren t color, it can fo ol the view er in to seeing a nice,
complex scene with a ric h v ariation of colors. If is use a dieren t color at eac h p oin t of a triangle
and blended these colors across the surface of the triangle, it w ould end up with a smo othly shaded
triangle.
Op enGL giv es a w a y to smo othly blend the colors at eac h v ertex across a line or across the
surface of a triangle. Exist the p osibilities to add a color attribute to eac h p oin t. In the case of
triangles the order of co ordinates is: X, Y, R, G, B . The next step will b e to remo v e the color
uniform from the shader and replace it with an attribute. F or colors to v ary across the surface of a
9

Op en Gr aphics Libr ary for Emb e dde d Systems
triangle is necessary to use a sp ecial v ariable t yp e kno wn as a v arying.
A v arying is a sp ecial t yp e of v ariable that blends the v alues giv en to it and sends these v alues
to the fragmen t shader. If the fragmen t b elongs to a line, then Op enGL will use the t w o v ertices
that mak e up that line to calculate the blended color. If the fragmen t b elongs to a triangle, then
Op enGL will use the three v ertices that mak e up that triangle to calculate the blended color.
Ho w Do es a V arying Get Blended at Eac h F ragmen t?
Exist the p osibilit y to blend more than just colors, sending an y v alue to a v arying, and Op enGL
will tak e the t w o v alues b elonging to a line, or the three b elonging to a triangle, and smo othly blend
these v alues across the primitiv e, with a dieren t v alue for eac h fragmen t. This blending is done
using linear in terp olation.
Line ar Interp olation A long a Line
The blended colors along a line with a red v ertex and a green v ertex, w ould lo ok something lik e
this:
Eac h color is scales linearly along the length of the line. Since the left v ertex of the line is red
and the righ t v ertex is green, the left end of the line should b e 100 p ercen t red, the middle should
b e 50 p ercen t red, and the righ t should b e 0 p ercen t red:
The same thing happ ens with green. Since the left v ertex is red and the righ t v ertex is green,
the left end of the line will b e 0 p ercen t green, the middle will b e 50 p ercen t green, and the righ t
will b e 100 p ercen t green:
A dding the t w o together, is obtined a blended line.
The strength of eac h color dep ends on the distance of eac h fragmen t from the v ertex con taining
that color. The distance ratio is simply a ratio b et w een 0 and 100 p ercen t, with 0 p ercen t b eing
the left v ertex and 100 p ercen t b eing the righ t v ertex.
10

Op en Gr aphics Libr ary for Emb e dde d Systems
1.5 Camera Space
The camera p ose is sp ecied in terms of three parameters, EYE, A T, and UP , as sho wn on the left
of Fig. 1.7.
* EYE is the camera p osition in the w orld space.
* A T is a reference p oin t in the scene at whic h the camera is p oin ting. (In the example, A T is
lo cated at the tip of the teap ot's mouth.)
* UP is a v ector that roughly describ es where the top of the camera is p oin ting. (In most cases,
UP is set to the v ertical axis of the w orld space, i.e., the y-axis.)
These parameters are used to build the camera space, whic h is also called the ey e space or view
space. Its origin is EYE. The b o x in the middle of Fig. 1.7, sho ws the sequen tial pro cess of building
the basis, {u, v, n}, of the camera space. First, the v ector connecting A T and EYE is normalized to
comp ose a unit v ector n. The cross pro duct of UP and n is normalized to dene another unit v ector
u. Finally , the cross pro duct of n and u determines a unit v ector v. {u, v, n} is an orthonormal
basis. The camera space is denoted as {u, v, n, EYE}.
Figure 1.7: EYE, A T, and UP dene the camera space {u, v, n, EYE}. With resp ect to the camera
space, the camera is lo cated at the origin and p oin ts in the n direction.
1.6 Mapping Co ordinates for Dra wn Ob jects
One of the basic problems in displa ying graphics on Android devices is that their screens can v ary
in size and shap e. Op enGL assumes a square, uniform co ordinate system and, b y default, happily
dra ws those co ordinates on to t ypically non-square screen as if it is p erfectly square.
11

Op en Gr aphics Libr ary for Emb e dde d Systems
Figure 1.8: Default Op enGL co ordinate system (left) mapp ed to a t ypical Android device screen
(righ t)
The illustration ab o v e sho ws the uniform co ordinate system assumed for an Op enGL frame on
the left, and ho w these co ordinates actually map to a t ypical device screen in landscap e orien tation
on the righ t. T o solv e this problem, Op enGL pro jection mo des and camera views to transform
co ordinates so graphic ob jects ha v e the correct prop ortions on an y displa y .
In order to apply pro jection and camera views, a pro jection matrix and a camera view matrix
are created and are applied to the Op enGL rendering pip eline. The pro jection matrix recalculates
the co ordinates of graphics so that they map correctly to Android device screens. The camera view
matrix creates a transformation that renders ob jects from a sp ecic ey e p osition.
1.6.1 Pro jection and camera view in Op enGL ES 2.0
In the ES 2.0 API, pro jection and camera view is apply b y rst adding a matrix mem b er to the v ertex
shaders of graphics ob jects. With this matrix mem b er added, the generate and apply pro jection
and camera viewing matrices is p ossible to ob jects. Steps are:
1. the matrix is added to v ertex shaders – a v ariable is created for the view pro jection matrix
and include it as a m ultiplier of the shader's p osition
2. the shader matrix is accessed – after the creating a ho ok in v ertex shaders to apply pro jection
and camera view, access that v ariable to apply pro jection and camera viewing matrices is
p ossible. The onSurfaceCreated() metho d of a GLSurface View.Renderer implemen tation is
mo died to access the matrix v ariable dened in the v ertex shader ab o v e
3. the pro jection and camera viewing matrices are created- the pro jection and viewing matri-
ces are generated to b e applied the graphic ob jects. The onSurfaceCreated() and onSur-
faceChanged() metho ds of a GLSurface View.Renderer implemen tation are mo died to create
camera view matrix and a pro jection matrix based on the screen asp ect ratio of the device
4. the pro jection and camera viewing matrices are applied – to apply the pro jection and camera
view transformations, the matrices are m ultiplyied together and then set them in to the v ertex
12

Op en Gr aphics Libr ary for Emb e dde d Systems
shader. The onDra wF rame() metho d of a GLSurface View.Renderer implemen tation is mo di-
ed to com bine the pro jection matrix and camera view and then is applied it to the graphic
ob jects to b e rendered b y Op enGL.
1.7 3D T ransforms and Matrix Represen tations
So m uc h of Op enGL w orks with v ectors and matrices, and one of the most imp ortan t uses of a
matrix is in setting up orthographic and p ersp ectiv e pro jections. One of the reasons for this is
that at the heart of things, using a matrix to do a pro jection just in v olv es a bunc h of adds and
m ultiplies o v er a set of data in sequence, and mo dern GPUs are v ery fast at that sort of thing.
A v ector is a one-dimensional arra y of elemen ts. In Op enGL, a p osition is usually a four-elemen t
v ector, as is a color.
A matrix is a t w o-dimensional arra y of elemen ts. In Op enGL, generally matrices are used
to pro ject v ectors using an orthographic or p ersp ectiv e pro jection, and they also are used to do
rotations, translations, and scaling of an ob ject. This is do es b y m ultiplying the matrix with eac h
v ector that w e w an t to transform.
Righ t-hand System v ersus Left-hand System
A 3D Cartesian co ordinate system is either righ t-handed or left-handed, assho wn in Fig. 5. In
the righ t-hand system (RHS), the th um b of the righ thand p oin ts to w ard the p ositiv e end of the z
-axis when the other four ngerscurl from the x -axis to the y-axis. In the left-hand system (LHS),
the same rule is applied with the left hand. Op enGL uses the RHS, but Direct3D uses the LHS.
Figure 1.9: Righ t-hand system v ersus left-hand system
Matrix-V ector Multiplication
2
666664xxxyxzxw
yxyyyzyw
zxzyzzzw
wxwywzww3
7777752
666664x
y
z
w3
777775=2
666664xxx+xyy+xzz+xww
yxx+yyy+yzz+yww
zxx+zyy+zzz+zww
wxx+wyy+wzz+www3
777775
All four comp onen ts of the rst ro w of the matrix will aect the resulting x, all four comp onen ts
of the second ro w will aect the resulting y , and so on.
13

Op en Gr aphics Libr ary for Emb e dde d Systems
1.7.1 Scaling
Three-dimensional scaling is represen ted b y a 33 matrix:
2
664sx0 0
0sy0
0 0 sz3
775
Figure 1.10: 3D scaling examples: Ev ery v ertex of a p olygon mesh is m ultiplied b y the same scaling
matrix
The rst is a uniform scaling in that all scaling factors are iden tical, and the second is a non-
uniform scaling.
The scaling matrix applied to a 3D v ector:
2
666664sx0 0 0
0sy0 0
0 0 sz0
0 0 0 13
7777752
666664x
y
z
13
777775=2
666664sxx
syy
szz
13
777775
1.7.2 T ranslation
In order to dene the 3D translation that handles homogeneous co ordinates, the translation v ector,
(dx; dy; dz) , is inserted in to the fourth column of the 44 iden tit y matrix. It can b e m ultiplied with
the homogeneous co ordinates of the 3D p oin t:
2
6666641 0 0 dx
0 1 0 dy
0 0 0 dz
0 0 0 13
7777752
666664x
y
z
13
777775=2
666664x+dx
y+dy
z+dz
13
777775
1.7.3 Rotation
T o gure out ho w an ob ject w ould rotate around a giv en axis, the righ t-hand rule is used: the
th um b in the direction of the p ositiv e axis and the curl of ngers sho ws ho w an ob ject w ould rotate
14

Op en Gr aphics Libr ary for Emb e dde d Systems
around that axis, giv en a p ositiv e angle of rotation.
Figure 1.11: Rotations ab out the principal axes: (a)Rz(900):(b)Rx(900):(c)Ry(900)
Matrix rotation uses the trigonometric functions of sine and cosine to con v ert the rotation angle
in to scaling factors. The follo wing is the matrix denition for a rotation around the x-axis, y-axis
and z-axis:
2
6666641 0 0 0
0cos(a)sin(a) 0
0sin(a)cos(a) 0
0 0 0 13
777775;2
666664cos(a) 0 sin(a) 0
0 1 0 0
sin(a) 0 cos(a) 0
0 0 0 13
777775;2
666664cos(a)sin(a) 0 0
sin(a)cos(a) 0 0
0 0 1 0
0 0 0 13
777775
Rotation around the x-axis as a test:
2
6666641 0 0 0
0cos(90)sin(90) 0
0sin(90) cos(90) 0
0 0 0 13
7777752
6666640
1
0
13
777775=2
6666640
0
1
13
777775
The p oin t has b een mo v ed from (0, 1, 0) to (0, 0, 1).
P anning the Camera Around the Scene
P anning the camera around the scene can easily do b y listening for touc h ev en ts and using those
touc h ev en ts to rotate the scene.
A touc h listener is dened that will measure ho w far are dragged nger b et w een eac h successiv e call
to onT ouc h(). A t rst touc h of screen, the curren t touc h p osition will b e recorded in previousX and
previousY. As nger are draged across the screen, a bunc h of drag ev en ts are got ,and eac h time is
do, the dierence b et w een the new p osition and the old p osition is tak ed and are stored that in to
deltaX and deltaY, and then previousX and previousY are up dated. The deltas will b e passed in to
the particle renderer b y calling handleT ouc hDrag(). The metho d will tak e in the distance the user
dragged in eac h direction and add it to xRotation and yRotation, whic h represen t the rotation in
degrees.
15

Op en Gr aphics Libr ary for Emb e dde d Systems
1.8 The Math Behind Orthographic Pro jections
An orthographic pro jection maps part of our 3D w orld on to the screen, and it do es so in a w a y suc h
that ev erything app ears the same size, no matter ho w near or far it is. F or this reason, this t yp e of
pro jection is great for doing 2D games and art.
Creating an Iden tit y Orthographic Pro jection
The follo wing is a basic denition of an orthographic pro jection matrix:
2
6666642
rightleft0 0 right +left
rightleft
02
topbottom0top+bottom
topbottom
0 02
farnearfar+near
farnear
0 0 0 13
777775
Giv en this matrix, all co ordinates b et w een left and righ t, b ottom and top, and near and far will
b e mapp ed in to normalized device co ordinates, and an ything within this range will b e visible on
the screen.
The matrix b y passing in -1 for left, b ottom, and near, and +1 for righ t, top, and far:
2
6666642
110 01+1
11
02
1101+1
11
0 02
111+1
11
0 0 0 13
777775=)2
6666642
1+10 011
1+1
02
1+1011
1+1
0 02
1+111
1+1
0 0 0 13
777775=)2
6666641 0 0 0
0 1 0 0
0 01 0
0 0 0 13
777775
It lo oks almost exactly lik e the iden tit y matrix. This happ ens b ecause normalized device co-
ordinates range from -1 to 1 on eac h axis, so when w e also pass in -1 and 1 as our ranges, w e are
essen tially asking for an orthographic pro jection that lea v es its co ordinates unc hanged, just lik e an
iden tit y matrix. The main dierence is that the z-axis is in v erted.
16

Op en Gr aphics Libr ary for Emb e dde d Systems
Creating a Regular Orthographic Pro jection
A matrix that will map [0,1] on to the range [-1, 1] for the x, y , and z comp onen ts w ould lo ok lik e:
2
6666642
100 01+0
10
02
1001+0
10
0 02
101+0
10
0 0 0 13
777775=)2
6666642 0 01
0 2 01
0 021
0 0 0 13
777775
T o pro v e that this matrix will indeed transform a co ordinate in the range [0;1] to the range [1;1]
w e'll start with a v ector at the minim um range, with all comp onen ts at 0:
2
6666642 0 01
0 2 01
0 021
0 0 0 13
7777752
6666640
0
0
13
777775=2
6666641
1
1
13
777775
As exp ected, ev erything maps to -1 in normalized device co ordinates.
1.9 The Math Behind P ersp ectiv e Pro jections
P ersp ectiv e pro jections w ork somewhat dieren tly . They also map part of the virtual w orld on to
the screen, but they do so b y using p ersp ectiv e: the further a w a y something is, the smaller it will
app ear on the screen. The pro jection matrix can't do this b y itself, so it uses a fourth comp onen t,
w, in conjunction with the p ersp ectiv e divide.
The follo wing is a v ery basic p ersp ectiv e pro jection matrix. This matrix assumes a righ t-handed
17

Op en Gr aphics Libr ary for Emb e dde d Systems
co ordinate space, with the near plane b eginning at a z of -1.
2
6666641 0 0 0
0 1 0 0
0 012
0 01 03
777775
? The rst t w o ro ws of this matrix simply cop y o v er the x and y comp onen ts
? The third ro w of this matrix copies o v er the z comp onen t, in v erting it at the same time. The
-2 in the fourth column will b e m ultiplied with a v ertex's w, whic h defaults to 1, so this ro w
of the matrix ends up setting the nal z to z2 . F or example, a z of -1 will b e set to -1, a
z of -2 will b e set to 0, a z of -3 will b e set to 1, and so on.
With a -1 in the third column, a -2 in the fourth column is simply the n um b er that will map
the near plane of the frustum to the near plane in normalized device co ordinates (-1 will map
to -1); and since the same z of -1 will create a w of 1, the z v alue will remain -1 after the
p ersp ectiv e divide.
? The fourth ro w sets the nal w to the negativ e z. Once Op enGL do es the p ersp ectiv e divide,
this will ha v e the eect of shrinking ob jects that are further a w a y .
Multiply this matrix with -> a p oin t on the near end of the frustum:
2
6666641 0 0 0
0 1 0 0
0 012
0 01 03
7777752
6666641
1
1
13
777775=2
6666641
1
1
13
777775
With a z of -1, the result has a z of -1 and a w of 1.
-> a p oin ts further a w a y than the previous one:
2
6666641 0 0 0
0 1 0 0
0 012
0 01 03
7777752
6666641
1
3
13
777775=2
6666641
1
1
33
777775
As the p oin t gets further a w a y , b oth z and w get larger and larger.
Dividing b y W
W e'v e explored the rst part of the magic of p ersp ectiv e pro jection: dening a matrix that will
create a w v alue that increases as distance increases. After Op enGL divides eac h comp onen t b y w,
18

Op en Gr aphics Libr ary for Emb e dde d Systems
w e'll end up with the follo wing results:
2
6666641
1
1
13
777775=)2
6641
1
1
1
1
13
775=2
6641
1
13
775
2
6666641
1
1
33
777775=)2
6641
3
1
3
1
33
775=2
6640:33
0:33
0:333
775
2
6666641 0 0 0
0 1 0 0
0 012
0 01 03
7777752
6666641
1
10000
13
777775=2
6666641
1
9998
100003
777775=)2
6641
10000
1
10000
9998
100003
775=2
6640:0001
0:0001
0:99983
775
After Op enGL divides ev erything b y w, the relationship b et w een input z and output z is no longer
linear. Going from an input z of -1 to -2 increases the output z from -1 to 0, but going from an input
z of -10,000 to -20,000 only has a v ery small eect on the output z. This nonlinear relationship has
imp ortan t implications when it comes to rendering ob jects at dieren t distances in scene.
19

Chapter 2
Application
20

Bibliography
[1] Kevin Brothaler, Op enGL ES 2 for A ndr oid , 2013
[2] JungHyun Han, Intr o duction to Computer Gr aphics with Op enGL ES , 2018
[3] h ttps://dev elop er.android.com/training/graphics/op engl
[4] Prateek Meh ta, L e arn Op enGL ES for mobile Gr aphis Development , 2013
[5] Munshi A., Ginsburg D., Sc hreiner D. Op enGL ES 2.0 Pr o gr amming Guide , 2008
[6] Aaftab Munshi, Jon Leec h, Op enGL ES Common Pr ole Sp e cic ation V ersion 2.0.25 (F ul l
Sp e cic ation) , 2010
21