International Journal of Computer Applications (0975 8887) [609362]

International Journal of Computer Applications (0975 – 8887)
Volume 168 – No.1 1, June 2017
7 An Overview of Traffic Signs Recognition Methods
Prachi Dewan
Research Scholar
EECE Dept.
The North Cap
University
Gurugram, Haryana
Rekha Vig, PhD
EECE Dept.
The North Cap
University
Gurugram, Haryana

Neeraj Shukla ,
PhD
EECE Dept.
The North Cap
University
Gurugram, Haryana
B. K. Das
Professor
EECE Dept.
The North Cap
University
Gurugram, Haryana

ABSTRACT
Image processing has many areas of applications like in
metrological prediction of weather conditions, medical
science, artificial intelligence, robotics etc. Traffic signs
recognition system is in fact the most happening area of
research these days. As the world is moving towards the
driverless vehicles, a more automated world is the utmost
requirement for improving the road safety. Such systems can
help the drivers on signs that they may not have noticed
beforehand. These systems are so designed s o that they
consume less power and hence can be efficiently implemented
on hardware. FPGA ’s are preferred over CPU and GPU due
to its low cost and power, prototyping applications and next
levels to ASIC’s development. In this paper we have quoted
some basic challenges in traffic signs recognition methods
and summarized the various detection a nd recognition
techn iques for traffic signs. This paper divided the various
methods into three categories: color -based, shape -based and
learning – based. We have concluded that the Xilinx System
Generator is the best tool while implementing on FPGA’s. It
is the fastest resource estimation tool in order to take full
advantage of FPGA’s resources. Finally, the hardware
perspective of traffic signs implementation is briefly
examined.
General Terms
Image pre-processing, Traffic Signs, Shape Extraction ,
Algorithm, Driver Assistance system
Keywords
Traffic Sign Recognition method , Field programmable gate
array (FPGA), Xilinx system generator (XSG) , Advanced
Driver assistance system (ADAS) , Machine learning
techniques.
1. INTRODUCTION
Advanced Driver assistance system is a smart system
designed to improve the safety and comfort of drivers. Every
year thousan ds of traffic accidents occur around the world
that causes loss of lives. When driving on a congested road, it
is sometimes difficult to ke ep a track of the oncoming traffic
and also what is behind us, while trying to maintain speed.
Hence a system should be designed to detect all the traffic
signs beforehand for display console in front of the driver.
The increase in computing power of mac hines has made the
real time detection and recognition of traffic signs possible.

Recognition of speed limit sign can inform the driver about
the present speed limit and it can also alert the driver if a car
is being driven faster than the speed limit . This appears to be
easy as drivers can easily recognize all the signs because the
color and shapes of the signs are very different from the
natural environment. They are designed with specific colors
and shapes, with a text or symbol in a high contrast t o
background. But the reality during sign detection by machines
is different due to many limitations regarding shape, size and
orientation of road signs .Sometimes they are partially hidden
and this can lead to false detection The color s that are mostly
used are blue and red for all warning and prohibitory signs.
Moreover, the shapes of the signs are triangle or circle which are uncommon in the nature. These two unique features of
traffic signs give motivation to design efficient algorithms for
detection and recognition. In this paper, we have tried to
compare all the technologies and tools used in designing a
robust traffic signs detection system. Both the image
processing and machine learning algorithms have to be
improved to implement the real time con straints of traffic
signs recognition system. Figure 1 shows the various
prohibitory and warning road signs.

Figure 1:Various Warning and Prohibitory Signs

However, there are some problems while processing the
images of signs. Some of the main problems are low camera
resolution, imperfect sign state, illumination, and occlusion ,
weather conditions such as rain, snow or fog . All of these can
affect the color analysis and shape extraction of signs. In
order to correctly recognize the signs, the above mentioned
challenges should be properly addressed . In most of the
published papers, the system is designed using a three stage
sequential approach i) preprocessing stage and then d etection
stage ii) recognition stage and iii) finally transmission( fro m
one vehicle to another) stage. The various stages of traffic
signs recogni tion system is shown in figure 2 .

Figure 2: Traffic signs preprocessing, detection,
recognition and tran smission stages
During preprocessing , the image quality is enhanced so that it
can be efficiently delivered to the subsequent stages. In the
detection stage, the image is searched for different traffic
signs and hence regions of interest ( ROI) are calculated by
using color and s hape features of the sign. These ROI’s are
then analyzed in the recognition stage. After the signs are

International Journal of Computer Applications (0975 – 8887)
Volume 168 – No.1 1, June 2017
8 correctly recognized, they are then transmitted from one
vehicle to another wirelessly.
In this paper, we have done a comp arative study of all the
methods of detection and recognition algorithms used in
designing traffic sign recognition system. We have addressed
all the issues and challenges in designing the traffic signs
detection system. Table 1 discusses the various techn iques
used by various traffic sign recognition (TSR) system from
preprocessing to recognition stage.
Table 1: Techniques used by various TSR system
Stages Methods
Preprocessing Camera calibration and its resolutio n

Detection Color
based Color Thresholding
Color Learning

Shape
Based Edge Detection
Template Algorithm
Machine learning
techniques
Hough Transform
Recognition Template Matching
Machine Learning

2. ISSUES AND CHALLENGES
The various methods for which traffic signs detection must
take into account the following:
Type of input – Video or Static images.
Weather conditions -Are the images taken in daylight or at
night or in adverse conditions like rain, fog or snow.
Type of sensor – High or low resolution camera, grayscale or
color sensor.
Processing method – Real time image capture or offline.
There are many issues that must be considered while
designing any traffic signs recognition system. They are
acquisition vision system, conditions of road, surrounding
conditions, traffic sign state and size of image.
2.1 Acquisition System
The fundamental function of acquisition system is to capture
the image or video so that it can be further processed. In fact ,
the camera should be properly calibr ated and its dynamic
range should be adjusted before capturing any scene.
Recently , high dynamic range cameras (HDR) based on
CMOS technology has begun to develop in field of driver
assistance systems since they provide high contrast images
during night. T he position and resolution of cameras
acquiring the images of signs are the main concern. If the
cameras are placed too far away , then the image captured will
be blurred and cannot be detected accurately. But if the
camera is placed too close (maybe on bum per of car) ; it will
capture images which can be affected by the climatic
conditions. Hence the distance of camera from the image
determines the quality of image. The traffic sign detected in the image is small and noisy if captured with a low resolution
camera and the image would not be detected efficiently.
Hence many researchers have tried to use cameras of
resolution 640*480 pixels. But practically this resolution is
insufficient while dealing with signs which are far away from
the camera. There can al so be excessive tilt in camera plane
while capturing the image. All of these factors can make it
impossible to recover the useful information from the images.
2.2 Surrounding Conditions
The detected color of the image of the roadside sign can be
different from the actual color due to illumination effects. The
shape of image is also affected due to occlusion and weather
conditions such as rain, snow or fog. Surrounding objects like
buildings or trees may directly or indirectly affect the
detection process.
2.3 Traffic Sign State
The traffic sign on the roadside may be damaged due to
physical conditions of environment. This degradation of sign
can be due to ultra violet radiations or due to inappropriate
material used in making sign. There could be mechanical
deformations like folding of edges. These could affect the
shape and color of sign and can make detection process
tedious.
2.4 Size of Image
The size of image appearing in the scene has an impact on the
accuracy of detection and recognition process. If the signs are
too small then they would not be detected as picking up of
color and shape would be challenging task even for best
computer vision algorithms. A large size image could provide
enough and good quality information but requires a very large
processing time. Hence a system should be designed so that it
keeps track of signs from the point it become visible until a
reasonable size that would be easily detected.
3. TRAFFIC SIGN STAGES
3.1 Preprocessing Techniques
Preprocessing stage is essential for making the detection
process faster. It not only considers the camera calibration but
also enhance the quality of image. High dynamic range
(HDR) cameras should be tested for sign detection for good
quality images. Cmos Video Cameras should b e used with
high HDR. The authors in [1] proposed the techniques for
both reducing the HDR and temporal effects. The image
captured from the camera is of RGB nature. These images are
very sensitive to lighting variations. To reduce the effect of
lightning a proper color space conversion has to be done. The
most important color spaces used for detection purpose are
HIS, YC bCr, HSV , YUV and RGB. The YUV color space
give good results for red/yellow/orange signs, but it fails for
white/black or low resolution colors. The authors in [2] used
the ratios between the intensity of a given channel and the
sum of all RGB channel intensities. The HSV (Hue –
Saturation -Value) color space has been used in many works
presented by Waite and Oruk lu in [3] and authors in [4, 5].
This color space is preferred as it is ineffective to illumination
effects. The interconversion formulas of RGB -HSV are non –
linear and hence the computational cost is very high.
Moreover hue component changes with distance, dust and
age. The YC bCr color model has been used by authors in [6].
The chrominance component (Y) was used to detect the signs.
They worked on grayscale images as it presented good results
regardless of lighting conditions (day/night/rainy/wet). Fang,
Chen and Fuh in [7] used the concept of pre -stored hues to
classify colors The reason was to store the hue values
beforehand and the color label was calculated by calculating

International Journal of Computer Applications (0975 – 8887)
Volume 168 – No.1 1, June 2017
9 the similarity with all available hues, so that the classification
that is most similar is chosen. Hence depending on the
application the color space conversion can be chosen.
3.2 Detection Techniques
The detection process sends ROI’s from the captured image
to the recognition stage suppressing the background regions.
There are many methods for correct detection of signs
through color, shape and by using many machine learning
techniques.
3.2.1 Detection based on color
Color segmentation involves either the use of any particular
color space or any of the color learning methods.

3.2.1.1 Techniques based on color thresholding
Color based detection techniques aim to search the area of
interest based on colors of interest using color based
thresholding or segmentation techniques. Different color
spaces can be used to detect the colo r of interest and separate
it from background. Several models have been designed. In
mid -1990s, several researchers used Hue -Saturation -Intensity
(HIS) color spa ce model. The authors in [8] used color
thresholding to segment the image. They implemented th e
thresholding using a 16 -bit look -up-table.
C.H Lai and C.C.Yu in [9] performed the color detection in
the HSV color space. They concluded that the color saturation
shows significant change when the videos are taken from
different devices. Some authors [10] studied the effect of
light on the color of traffic signs during day and night and
concluded that the color of roadside image could get distorted
due to light and this may affect the quality of images.
However, they emphasized that outdoor illuminatio n does not
affect the RGB component differences for traffic signs. The
authors in [11] emphasize d on color segmentation as
detection of shape is unreliable in urban areas. Unfortunately,
all the segmentation algorithms based on color thresholding
have to a djust their thresholds again and again. Hence color
segmentation always requires a fixation of threshold.
3.2.1.2 . Techniques based on color learning
methods
Various learning methods have been implemented for traffic
signs. Aoyagi and Asakura in [12] were the first ones, who
worked in area of genetic algorithm (GA) for the sign
detection. They detected the speed limit signs. They used
smoothing filter and Laplacian filter to remove noise and then
used genetic algorithm for sign detection. An algorithm bas ed
on Support Vector Machines (SVM) was presented by
authors in [13] to classify pixels using color information.
The authors concluded that the segmentation based on
learning algorithms give better results as compared to color
thresholding segmentation . In [14], the detection is carried
out using Haar wavelets obtained from Ada Boost algorithm.
The authors have used the wavelet feature to solve many
practical problems in real time.
3.2.2 Detection based on shape
Color based detection techniques are only useful if w e are
using high resolution CMOS color camera. Color based
detection would not be effective during night when all the
artificial lights are present which as confirmed by authors in
[15]. Detection can be done by shape also . Shape detection is
effective only when light illumination changes frequently.
Moreover shape based detection works on grayscale images and this may help to meet global cost requirements. The shape
is usually extracted either by their edges, gradient features,
template matchin g or by Hough Transform.
3.2.2.1 Shape detection using ed ges
Most of the edge detection techniques depend on the contour
or edge information of signs. The authors in [16] explained
the detection of signs by using rectangular pattern matching
algorithm. The areas detected as signs were defined under one
scan window. The ellipse equation was also used to detect the
circle in an image [17]. The authors in [18] explained the
detection of rectangular and circular images of traffic sign
using rectangular and circular detection algorithm. However,
shape detection using edge features can only give accurate
results in daytime only.
3.2.2.2 Shape detection using templ ate matching
Shape can also be detected using template matching
techniques. The captured image is matched with the images of
known good traffic signs in the database. The matching is
done by calculating the Hausdroff distance between the image
captured and each template as imple mented by authors in
[19]. However, this technique is not suitable for real time
applications as it requires many computations between
template and ROIs for sign detection. Sheldon Waite and
Oruklu in [20] applied this technique in face recognition and
object matching. The main advantage of using template
approach is that it can be modified to detect any objects.
3.2.2.3 Detection based on machine learning
techniques
Machine learning algorithms like suppor t vector machines
(SVMs) and neural networks ( NNs) can be used to extract
shapes. SVMs and NNs are used for signs detection because
of their ability to detect shapes accurately. Neural networks
are trained for each set of signs. However, addition of more
signs implies again training of network and hence manual
selection of training samples. SVM ’s are invariant to rotation,
translation and partial occlusions of road signs. MSERs
(maximally stable extremal regions) work on the traffic
symbols with white backg round. Although MSERs are a
robust form of sign detection in complex scenes but they are
computationally expensive . The authors in [21] explained the
detection process by calculating the threshold at different
levels ranging from 70 to 190 and their MSERs.
3.2.2.4 Shape detection based on hough t ransform
Hough transform is used to detect circles and lines based on
curve fitting algorithm. The advantage of using Hough
transform is that it is not sensitive to imperfect data and noise
and also manages to detect occluded images. The authors in
[22] used circular Hough transform for detection of European
speed limit signs. However, the authors used rectangular edge
detecti on based algorithm for detecting U.S speed limit signs.
3.3 Recognition Techniques
The recognition stage receives the ROIs from the detection
stage that possibly contains one or more signs. Since there is
huge number of traffic signs and moreover there can be
distortions and occlusions on the signs, the recognition
system should always be reliable. Traffic signs can be
recognized by mainly two main approaches: template
matching or machine learning techniques.

International Journal of Computer Applications (0975 – 8887)
Volume 168 – No.1 1, June 2017
10 3.3.1 Re cognition based on templa te matchin g
This algorithm stores some sample sign images in database.
Template matching was used by authors in [23] for
classification. They calculated the distance between candidate
regions came from detection stage and different sizes of
template images in database. The template having minimum
distance is the matched sign. However, template matching
algorithm fails for tilted, rotated and partially hidden traffic
signs.
3.3.2 Recognition ba sed on m achine learning
techniques
There are many machine learning techniques like SVMs,
Neural networks which can be used to recognize signs. The
authors in [21] used the cascaded structure of support vectors
classifiers and HOG features to recognize th e signs. SURF
(Speeded up Robust Feature) algorithm was used to match
two traffic signs even if they are slightly rotated. The authors
in [11] created a SURF database of various traffic signs
templates. However, a fast internet connection is needed for
better performance of tool. A mobile based application”
Mansalakuna” was developed by authors in [23 ] that would
detect signs through mobile phones. They depend on Global
positioning system (GPS) and digital road maps. But again a
fast internet connection is needed for application based traffic
sign detection and recognition system.
4. HARDWARE PERSPECTIVE OF
TRAFFIC SIGNS
IMPLEMENTATION
While considering the hardware aspec ts, some algorithms
have to be modified to overcome the real time constraint. To
implement the TSR system to hardware, researchers are
using Simulink graphical tool for modeling , simulation
which is suitable for making image processing b lock. The
authors in [24] had proposed FPGA based hardware
implementation of road signs. The authors have used
Matlab/Xilinx system Generator as it has a big advantage in
terms of the conception time .This tool provide the user some
special Xilinx building blocks to create the optimized designs
for Xilinx FPGA’s. Moreover, XSG operate on fixed point
data which represent the wide range of formats for hardware
implementation.
5. CONCLUSION AND FUTURE SCOPE
This research has divided the TSR system in three main
stages as preprocessing; detection and recoginition.It
describe each stages showing the various methods of
detection and recognition and some techniques that can be
used effectively for its hardware implementation. It was
shown by some of the authors that the hardwa re
implementation of TSR system can be optimized by using
XSG tool. The use of this tool has a benefit in terms of
conception time and hence can be embedded in any car
equipped with high resolution cameras and GPS receiver.
Table 2 illustrates the various sign detection and recognition
methods used by researchers from the last 15 years.
6. REFERENCES
[1] Stefano Marsi , Gaetano Impoco, Anna Ukovich, Sergio
Carrato and Giovanni Ramponi ,” Video Enhancement
and Dynamic Range Control of HDR Sequenc es for
Automotive Applications”, EURASIP Journal on
Advances in Signal Processing , Vol.2007 Article ID
80971.
[2] Arturo E scalera, Lius Moreno, Miguel Salichs and Jose
Armingol ,”Road traffic Sign Detection and Classification”, IEEE T ransactions on Industrial
Electronics, Vol.44, No.6 , pp.848 -859, December 1997 .
[3] Sheldon Waite and Erdal Oruklu,” FPGA -Based Traffic
Sign Recognition for Advanced Driver Assistance
Systems ”, Journal of Transportation Technologies ,
Vol.3, pp.1 -16, November 2012.
[4] Chokri Souani, Hassene Faiedh and Kamel Besbes ,”
Efficient algorithm for automatic road sign recognition
and its hardware implementation ”, Journal of Real Time
Image Processing, Vol.9, pp.79 –93,April 2013.
[5] Karla Brkic, "An overview of traffic sign detection
methods ”, Department of Electronics, Microelectronics,
Computer and Intelligent Systems Faculty of Electrical
Engineering and Computing Unska , 2010.
[6] Ching -Hao Lai and Chia -Chen Yu, "An efficient real –
time traffic sign recognition system for intelli gent
vehicles with smart phones ", In Technologies and
Applications of Artificial Intelligence (TAAI), 2010
IEEE International Conference , pp. 195 -202, 2010.
[7] C.Y.Fang , S.W Chen and C.S. Fuh,” Road sign
detection and tracking ”, IEEE Transactions on
Vehicular Technology, Vol. 52, No. 5, pp. 1329 –
1341. September 2003.
[8] Arturo Escalera, Luis E. Moreno, Miguel Angel S alichs
and José Maria Armingol " Road traffic si gn det ection
and classification " ,IEEE transactions on Industrial
Electronics , Vol. 44, No. 6, pp. 848 -859,December
1997.
[9] Ching – Hao Lai, and Yu Chia -Chen "An efficient real –
time traffic sign recognition system for intelli gent
vehicles with smart phones ", In IEEE International
Conference on Technologies and Applications of
Artificial Intelligence (TAAI), pp. 195 -202, 2010.
[10] M. Benallal and J. Meunier, ”Real-time color
segmentation of road signs”, IEEE Canadian
Conference on Electrical and Computer Engineering,
Vol.3 , pp.1823 -1826,2003.
[11] E. Oruklu, D. Pesty, J. Neveu x and J. E .Guebey ,” Real
Time Traffic Sign Detection and Recognition for in
car Driver Assistance Systems ”, IEEE I nternational
Symposium on Circuits and System, pp.976 -979,2012.
[12] Y. Aoyagi and T . Asakura ,“ A study on traffic sign
recognition in scene image using genetic algorithm and
neural networks ”, IEEE International Conference on
Industrial Electronics, Control and Instrumentation ,Vol.
3, pp. 1838 -1843, 1996 .
[13] H.G. Moreno , S.M.Bascón , P.G.Jiménez.and
S.F.Arroyo ,” Goal Evaluation of Segmentation
Algorithm s for Traffic Sign Recognition”, IEEE
Transaction on Intelligent Transportation Systems , Vol.
11, No. 4, pp.917 – 930,2010.
[14] Claus Bahlmann, Ying Zhu, Ramesh Visvanathan, M.
Pellkofer and K.Thorsten, ” A system for Traffic Sign
detection ,tracking and recognition Using color, shape
and motion Information”, In IEEE proceedings of
Intell igent Vehicles Symposium, pp. 255 -260,2005.
15] Miguel Angel García -Garrido , Miguel Angel Sotelo and
Ernesto Martin – Gorostiza ,” Fast traffic sign detection
and recognition under changing lighting conditions ”, In
IEEE Intelligent Transportation Systems Conference, pp.
811-816, 2006.

International Journal of Computer Applications (0975 – 8887)
Volume 168 – No.1 1, June 2017
11 [16] Anh-Tuan Hoang, Tetsush Koide and Masaharu
Yama moto, ” Low Cost Hardware Implementation for
Traffic Sign Detecti on system”, IEEE Asia Pacific
Conference on Circuit and System, pp.363 -366, 2014 .
[17] Aurore Arlicot, Bahman Soheilian and Nicolas
Paparoditis,” Circular road sign extrac tion from street
level images u sing color, shape and texture databases
maps ”, Titled in International Archives of
Photogrammetry, Remote Sensing and Spatial
Informational services in IAPRS,Vol.38, pp. 205 -210,
2009.
[18] Fabien Moutarde, Alexandre Bargeton, Anne Herbin and
Lowik Chanussot,” Robust on -vehicle real -time visual
detection of American and European speed limit signs
with a modular Tr affic Signs Recognition system”, In
IEEE Intelligent Vehicles Symposium, Istanbul, Turk ey,
pp.1122 – 1126 ,June 2007.
[19] De La Escalera, J. Ma Armingol and Mario Mata, Traffic
Sign Recognition and Analysis for intelligent vehicles”,
Journal of image and vision computing, Vol.21, No.3,
pp.247 -258, 2003. [20] S.Waite and E. Oruklu, ” FPGA -Based Traffic Sign
Recognition for Advanced Driver Assistance Systems ”,
Journal of Transportation Technologies , Vol.3, pp.1 –
16,2013
[21] J.Greenhalgh and M. M ehdi,” Real-Time Detection
and recognition of road traffic signs ”, IEEE
Transactions on intelligent transportation systems,
Vol.13, No.4, pp.1498 -1506 , 2012
[22] Par Karem and Oguz Tosun,” Real-time traffic sign
recognition with map fusion on multicore/many core
architectures ”, In Acta Polytechnica Hungarica, Vol.9 ,
No.2, pp.231 -250, 2012.
[23] A. Karunalithika,R.P.Jayasundra,M.A.Rasamjan ,D.N.
Senayanke and V.N.Vithana,” Road sign identification
application using image processing and augmented
reality”, International journal of advanced computer
technology , Vol. 4,Issue.11.pp.79 -93,2015.
[24] Rihab Hamida, Abdessalam Abdelali and Abdellatif
Mtibaa,” Hardware implementation and validation of a
traffic road sign detection and identification system”,
Journal of Real Time I mage processing, p p. 1-18, 2016

7. APPE NDIX

Table 2: Summary of various detection and rec ognition methods

Authors Year Detection Algorithm Recognition Algorithm
Eclasera and Salichs [2] 1997 Color thresholding Neural network
Benallal and Meunier
[10] 2003 Color segmentation –
Escelera [19] 2003 Genetic algorithm Neural network
Claus Bahlaam [14] 2006 Ada Boost algorithm. Bayesian generative modeling.
Mourtarde [18 ] 2007 Shape detection through
Hough transform Neural network
Arlicot [17] 2009 color segmentation SVM
Oruklu [11] 2013 Color segmentation. Template matching and Neural network
Chokri Souani [4] 2013 Color segmentation Neural network
A.T .Hoang [16] 2014 Shape detection through
rectangle matching algorithm –
Karunalithika[23] 2015 OpenCV Library is used for
detection Mobile based application
Rihab Hamida [24 ] 2016 Color segmentation based on
Xilinx System Generator Simulink model based on Xilinx System
Generator
IJCATM : www.ijcaonline.org