Fuzzy Control

Table of Contents

Chapter 1 Introduction

1.1 Motivation

1.2 Case Study on Current Research

1.2.1 Computer Vision

1.2.2 Image Processing

1.2.3 Control Methods

1.2.4 Robot Control Architectures

1.3 Objectives and Project Specifications

Chapter 2 Theory Fundamentals

2.1 Computer Vision

2.1.1 Color Models and Color Spaces

2.1.2 RGB Color Model

2.1.3 HSV Color Space

2.2 Processing Vision Data

2.3 Fuzzy Control

2.3.1 Foundations of Fuzzy Logic

2.3.2 Fuzzy Inference Process

2.3.3 Types Of Fuzzy Inference Systems

2.4 System Architectures

2.4.1 Introduction

2.4.2 Component Based Software Engineering in Robotics

2.4.3 SmartSoft Component Model

Chapter 3 Solution

3.1 Analysis and Design

3.1.1 OpenCV and EMGU CV

3.1.2 Managed Extensibility Framework

3.1.3 TPL Dataflow Library

3.1.4 Architecture

3.2 Implementation

3.2.1 Setting up EMGU CV in Microsoft Windows

3.2.2 Solution Structure and Architecture

3.2.3 Implementation Details

3.2.4 HSV Filter

3.2.5 Fuzzy Controller

Chapter 4 Testing and Validation

4.1 Obstacle Detection

4.1.1 Detecting Obstacles in a Video

4.1.2 Detecting Obstacles in Pictures

4.2 Control

4.2.1 Speed

4.2.2 Control Values

Chapter 5 Conclusions

Bibliography

Chapter 1 INTRODUCTION

1.1 Motivation

Very few tasks in the modern world can be accomplished without using robots. From factories to other planets, robots are irreplaceable in modern man's life, to perform tasks with speed and precision impossible to achieve by any human being. Robots are replaceable, easy to repair and maintain, powerful, and capable to perform tasks in hostile environments for humans.

Limited only by their creator's imagination, one robot can be relatively adaptable to perform a large amount of tasks. For example, a single robotic arm can be programmed to perform various tasks from mounting components of a car, plane, or any other product on an assembly line or grasp rock s from the Mars ground.

In most current application, robots are programmed to perform a sequence of predefined movements, without the need to have much information about the environment. But the humanity is moving forward from assembling cars and, at the moment, a robot rover is exploring Mars, without too much help from human operators, and without knowing what might come in front of it at the next step. Without means of sensing the environment, it would be impossible for a relatively autonomous robot to make a single safe step forward.

Robots are very close to living beings by means of sensing, because nature was the only example of what can be done in the field. Ultrasonic, infrared, vision sensors are some of the examples of sensing elements used with robots that can also be found in nature, with various beings, including humans. Distance is an important description of the environment, no matter if it refers to an object to be approached or avoided. But distance can be computed by means of a line, and, when it comes to real environments, we talk in three dimensions.

The most complex sensor that nature evolved and mankind managed to reproduce to a certain scale is the eye. Vision can offer complex and useful information about the size, shape, color, texture, distance of an object. None of these details can be provided by range sensors.

Vision is one of the most powerful sensing methods due to its ability to provide detailed information about the environment, allowing the intelligent system to perform many tasks in a new, dynamic environment, like obstacle avoidance, path planning, object recognition. Another advantage of vision sensors is the relative small cost compared to other types of sensors like infrared or ultrasonic.

But these descriptions of the environment are impossible to achieve directly. As we know with the eye, vision comes by light projected on a sensor and interpreted forward by the brain to provide more details about the sensed data. Vision must be processed.

When comes to exploring new environments, on which even the creators of the robot have very little information, robots must have a certain degree of intelligence. Artificial intelligence is not a new concept, but, even if much research has been done in the field, robots are far from competing with a human or, with many other living beings, at dealing with the unknown. That is because life has had millions of years to evolve, to “learn” how to process the inputs from its various sensors: vision, hearing, touch etc. Or, by that matter, to “grow” the actual sensors needed to perceive the environment.

As hard as it can be to obtain and process information from the environment, it is useless without using it. A robot is used to perform a task and, either by human operator, by a predefined series of movements and sense based decisions or by real time control performed by its own brain.

After exposing the necessity of having autonomous robots, and ways to make them perceive, process the environment and take decisions based on what they sense, depending on the task, this paper is meant to explore the relatively new field of computer vision and intelligent control with autonomous robots and try to implement a solution for sensing, processing and control on an autonomous robot arm.

1.2 Case Study on Current Research

This chapter is intended to provide an insight about some of the research done in computer vision and autonomous navigation, following a logical and natural approach:

Sensing the environment; gather data from a web cam mounted on the robot

Processing the sensor data; transform matrix data received from the web cam in logical data, field specific, in terms of obstacles

Apply control algorithms based on the detected obstacles to determine the desired direction of movement in order to avoid collision

1.2.1 Computer Vision

For humans, perceiving the three-dimensional structure of the world is relatively easy. We can easily distinguish shape, translucency, texture by processing with our brain the patterns of light and shade on the objects and approximate with relative precision the distance to and size of objects, thanks to our stereoscopic vision. We can extract great amounts of information from a picture in no time, by simply looking at it: we can distinguish objects from background, count people and tell their emotions from their facial expressions.

The researches in computer vision have succeeded in developing mathematical algorithms to extract precious information from an image, trying to follow the human's visual perception. But this algorithms need great computing power and, even with the powerful today's computers we are still far away from reaching the capabilities of the human eye. Capabilities like reconstructing the 3D structure of an environment can be reached from a large set of overlapping images, while the human vision can perform this task almost instantaneously.

Computers today have reached a great computational power, and even a computer that fits in a pocket is capable of running such algorithms that can extract various information from an image or, for tasks like 3D reconstruction, from a set of images: reconstruct the 3-dimensional shape of objects, reconstruct the 3D model of an environment, detect faces, distinguish emotions, track moving objects from a background.

Far from reaching the desired capabilities, computer vision has been successfully implemented and is in continuous development for a large variety of applications, as enumerated by [SZ2010]:

3D model reconstruction or photogrammetry – construction of 3D models from aerial photographs

Medical imaging – imagery before and during surgical interventions are of real help to modern medicine. 3D reconstruction of organs from a set of imaged obtained by certain technologies (RMN, computerized tomography), reconstruction of the human brain, detection of tumors by image segmentation etc.

Automotive

– detecting obstacles such as pedestrians, traffic sign recognition are first steps in developing a fully automated driving

– surveillance and traffic monitoring

– mechanical inspection

CGI and movie industry – live camera footage with tracked 3D points merged with computer generated image (CGI) are widely used in movie and computer games industry to generate realistic character movements

Fingerprints recognition

3D modeling – a set of images of an object can be converted into a 3D model of the object

Face detection and visual authentication

Exposure bracketing – merging multiple exposures obtained under challenging lighting conditions into a better image

Morphing – transform an image into another

1.2.2 Image Processing

This section presents generalities about some of the most useful computer vision and image processing methods and algorithms and analyzes a bit more detailed techniques that can be used in obstacle avoidance problem.

Image Formation

Geometric primitives and transformations

Points, lines and planes are the basic primitives used when talking about image formation and how 3D features are projected in 2D. [SZ2010]

2D points in computer vision represent pixel coordinates in an image and are denoted using a pair of values or a vector:

or

Linear algebra is used to perform operations on lines and planes of an image, such as rotations, translations, projections etc.

Photometric image formation

This section describes the basic interactions between light and optical elements such as lens and sensors that contribute to image formation process.

Although geometric primitives are used to project 3D features into 2D features of an image, images are composed of discrete color and intensity values. Light is indispensable when talking about vision and optics.

In Figure 1.1 is represented a simplified model of image formation.

Figure 1.1 Photometric image formation [SZ2010]

The digital camera

After reflecting from one or more sources, light passes through the optics of the capture system (usually a kit of lens) and reaches the sensor (most used are CCD and CMOS sensors). The continuous information (photons) that reaches the sensor is sampled into discrete values, like the known RGB system.

Image Processing

After image reaches a sensor and is sampled into a color space, we have a matrix of pixels holding information about color and light intensity of tiny regions of an image. But the obtained digital information represents only a first step in image analysis. The image needs further processing, similar to any kind of signal processing taught in electrical engineering departments.

Point operators

This type of image transformations manipulate single pixels, independently of neighbors. In some cases, they can take into account global parameters. Brightness, contrast, color correction are examples of such operators.

Neighborhood operators

Refer to transformations on pixels depending on a small number of neighbor pixels. Smoothing, sharpening, blurring are some effected obtained with neighborhood operators.

Linear filtering

Is a subtype of neighborhood operators. Consists on transformations of weighted combinations of pixels. The output pixel is a result of a weighted sum of input pixels. The weights or coefficients are grouped in a matrix called kernel or mask. Gaussian kernel is an example of such operator.

Fourier transforms

Are also a subtype of neighbor operators. These operators can provide information about the frequency characteristics of an image and filtering results for different levels of frequency.

Pyramids and wavelets

Neighbor operators can be cascaded into pyramids and wavelets, useful for image analysis at different resolutions or scales. Some applications of grouping an image into a pyramid of different scales are finding an object in an image without knowing at what scale the object could appear, speeding up recognition algorithms, or performing edit op transformations manipulate single pixels, independently of neighbors. In some cases, they can take into account global parameters. Brightness, contrast, color correction are examples of such operators.

Neighborhood operators

Refer to transformations on pixels depending on a small number of neighbor pixels. Smoothing, sharpening, blurring are some effected obtained with neighborhood operators.

Linear filtering

Is a subtype of neighborhood operators. Consists on transformations of weighted combinations of pixels. The output pixel is a result of a weighted sum of input pixels. The weights or coefficients are grouped in a matrix called kernel or mask. Gaussian kernel is an example of such operator.

Fourier transforms

Are also a subtype of neighbor operators. These operators can provide information about the frequency characteristics of an image and filtering results for different levels of frequency.

Pyramids and wavelets

Neighbor operators can be cascaded into pyramids and wavelets, useful for image analysis at different resolutions or scales. Some applications of grouping an image into a pyramid of different scales are finding an object in an image without knowing at what scale the object could appear, speeding up recognition algorithms, or performing edit operations on the image at different scales, while maintaining details. In Figure 1.2 is an example of image pyramid for 3 scales.

Figure 1.2 Image pyramid [SZ2010]

Wavelets and pyramids are very close related. However, a difference between them is that wavelets are more orientation selective than pyramids and are used in object detection for objects that might appear with different orientation.

Geometric transformations

Rotations, shears and perspective deformations can be performed for certain processing needs. Figure 1.3 presents some of the main geometric transformations.

Figure 1.3 Geometric transformations

Feature Detection and Matching

Feature detection and matching are essential for many computer vision problems and applications. Features are elements of interest in an image. They can be points (key point features, corners), edges (usually used to mark object boundaries) or lines (a subgroup of edges that can be curves or straight lines inside an image). Usual applications are object matching (an object from a reference picture can be found in another picture or set of pictures), or image alignment based on correspondences (to form a mosaic, a panorama or even a 3D reconstruction).

There isn't a precise definition of what a feature may represent, but if a point, curve or region is a feature or not depends on the type of the application. For simplicity, a feature may be called an "interesting" part of the image an is a starting point for many computer vision algorithms. A quality criteria of a feature detector may be its repeatability, i.e. whether or not the same feature will be detected in more images of the same scene.

Feature detection is a low level processing operation, usually performed as the first operation on an image. It examines every pixel to see if the pixel represents or is a component of a feature. Usually feature detection is a component of a larger algorithm (object detection, obstacle avoidance etc.). After the feature detection is performed, the algorithm will further analyze the image in the regions with features.

As a prerequisite of feature detection, usually some of the image details need to be discarded. This is done by means of a Gaussian kernel, which smoothens the image before the actual feature detection takes place. Sometimes feature detection can be computationally expensive and the main algorithm that uses the feature detector may guide the process over certain areas of the image.

Being used as the initial step by many image processing algorithms, many feature detectors have been developed and they can be divided into four groups: edges, corners/interest points, blobs/regions of interest or interest points, ridges.

Points and patches

Corners or interest points derive from the edge detection by finding rapid changes in direction of the detected edges. Some algorithms were developed to discover only the corners, with no need to look first for the edges, by looking for gradients with high levels of curvature. Some points that cannot be traditionally called "corners" are detected by the corner detectors. For example, bright points on a dark background. These points are known as interest points.

The main advantage of point features is that they can be matched even in scale and orientation changes. Keypoint detection and matching can be divided into four stages [SZ2010]: feature detection, feature description, feature matching and feature tracking. An example of object found using SURF (Speeded Up Robust Feature) feature detector/descriptor is provided in Figure 1.4.

Figure 1.4 Feature detection and matching

Edges

Edges are points representing boundaries between two regions of an image. They are usually identified by strong magnitude of the gradient. While point features are important for finding or matching images in 2D, edges can provide other useful information, like object boundaries.

Blobs

Blobs are descriptors of an image in terms of regions, unlike edges or corners which are point-like features. Usually blobs can contain a preferred point, called local maximum or center of gravity. Therefore blobs detectors can be considered interest points detectors also.

Lines

While edges are useful for describing contours, objects usually consist of linear construction, and lines inside objects can be helpful in a variety of applications like architectural modeling or pose estimation in urban environments [SZ2010].

Ridges

Ridges can be seen as axis of symmetry or medial axis. The algorithms which extract ridges are harder to implement than edge/corner/blobs detectors on most classes of images. However, ridge detectors are used in road detection in aerial images or blood vessel detection in medical images.

Feature extraction

Once a feature has been detected, the algorithm performs other processing tasks over the feature to provide local histograms (graphical representations of the distributions of data – color or intensity values – of portions of the image; in feature detection, these histograms are realized for an image patch around the feature). Other details such as edge orientation or gradient magnitude, together with the histogram and the feature itself are called feature descriptors.

In Table 1.1 is a classification of common feature detectors:

Table 1.1 Common feature detectors [WPfd]

SURF (Speeded-Up Robust Feature)

SURF (Speeded Up Robust Features) is a robust feature detecting algorithm introduced by Herbert Bay in 2006. It can be used in tasks such as object recognition and 3D reconstruction. It is partially inspired by SIFT descriptor (Scale Invariant Feature Transform), another well known algorithm (patented by Universty Of British Columbia).

Both SIFT and SURF algorithms are scale and rotation invariant detectors and descriptors. SURF is claimed by its authors to be more robust (and faster) than SIFT in different image transformations.

The claimed characteristics of SURF algorithm are achieved by using integral images for convolutions. It uses a Hessian matrix based measure for the detector and distribution based descriptor. [Bay2008]

Figure 1.5 Detail of the Graffiti scene showing the size of oriented

descriptor window at different scales [Bay2008]

1.2.3 Control Methods

When it comes to control, there are many possibilities and approaches, each one with advantages and disadvantages related to others. Some may be even not suitable at all for the problem in discussion, but in most of the cases, a control method by itself can't achieve a 100 % success without being combined with other methods.

When it comes to inputs like images, classical control from control theory like PID controllers may not be suited at all. Or they could be used only to a subsystem of the application.

Two approaches used in controlling dynamic systems with complex inputs coming from complex environments are fuzzy logic and, as a subset of the field of artificial intelligence, neural networks. Or a combination of both: neuro-fuzzy control.

1.2.4 Robot Control Architectures

The term robot architecture is used to refer to how a system is divided into subsystems and how those subsystems interact. Robot architectures are distinguished from other software architectures because of the special needs of robot systems. [KELLEHER]

Besides the fact that robots are complex systems, composed from both hardware and software, they also have to perform complex real-time tasks in dynamic environments. Robots have to process data from sensors, control actuators, react to unexpected situations in a concurrent and asynchronous manner.

In Figure 1.5 is Shakey robot, the first general purpose mobile robot able to reason about its own actions. [WPshakey]

Figure 1.5 Shakey [WPshakey]

Hierarchical Architecture

SPA (Sense – Plan – Act) architecture consists of three functional elements, as the name implies. The main flow is the following: Sensing is reading sensor data used to update the environment model [PASSOW], compute a solution and plan the commands then act, or send the commands to the actuators. In Figure 1.6 is a sketch of the SPA paradigm.

Figure 1.6 Sense – Plan -Act [KELLEHER]

The main characteristic of the SPA control architecture is that it uses the sensors to create a world model, then plans the next steps based on the created model without considering any sensor data read meanwhile. Although straight forward, and goal directed [PASSOW], this characteristic is also the drawback of the architecture. While constructing the environment model and computing the decisions, the world is likely to change, leading to delayed response to the dynamic behavior of the environment. In conclusion, the SPA solutions fails in the presence of uncertainty. [PASSOW]

The Motor Schema Architecture

The Motor Schema Architecture was proposed by Ronald Arkin, at Georgia Tech in 1987. It is biologically inspired, massively parallel and mimics muscle memory. [PASSOW]

Independent behaviors are represented by a motor, leading to scaled decomposition of tasks. Although scalable, this solution does not solve the problem of perception and can be highly complex. [PASSOW]

In Figure 1.7 is presented the Motor Schema Architecture.

Figure 1.7 Motor Schema Architecture [PASSOW]

Finite State Machines

Finite State Machines is a mathematical technique to order behaviors synchronously. [PASSOW] They are represented by states (behaviors) and transitions.

In Figure 1.8 is an example of finite state machine for obstacle detection with a randomly planed goal.

Figure 1.8 Finite State Machine [PASSOW]

Finite State Machines are models simple to design and powerful but not scalable to very large systems and code is not running continuously but in discrete moments of time.

The Subsumption Architecture

The Subsumption Architecture is a layered architecture proposed by Rodney A. Brooks at MIT in 1985. According to Brooks, layers of control system are built to let the robot operate at increasing levels of competence. [Brooks1985] At the base of each layer reside asynchronous modules, communicating with each other. Higher level layers can suppress the output of lower levels. The lower layers can function without change when higher levels are added. As a result, the system is robust and flexible.

In Figure 1.8 is represented an example of subsumption architecture layers, as opposed to Sense-Plan-Act horizontal flow of control.

Figure 1.8 Subsumption Architecture – example of task decomposition [Brooks1985]

The control problem is decomposed in task achieving behaviors. [Brooks1985] Each horizontal slice representing a behavior is implemented separately, then the whole system is tied together. According to [Brooks1985], there are four main requirements of the system: multiple goals, multiple sensors, robustness, additivity.

As stated in the introduction of the subsumption architecture, it is based on modules communicating with eachother, higher level modules being able to suppress lower level ones. In Figure 1.9 is an example of how a module can be pictured, as a block with input and output lines. Input signals can be suppressed and replaced with the suppresing signal. Output signals can be inhibited. [Brooks1985]

Figure 1.9 Subsumption Architecture module [KELLEHER]

Hybrid Paradigm

According to [KELLEHER], hybrid architectures combine the planning capabilities of early architectures (Sense-Plan-Act) and the reactivity of the behavior based architectures (Subsumption architecture). Hybrid paradigm also keeps the layered approach of the behavioral architectures.

One of the most popular variant of hybrid architecture are the 3-tiers (layers) architectures. [KELLEHER]

One example of 3-layered robot control system has the following layers: Controller (Reactive layer – low level control; tight sensor-action loop [KELLEHER]), Sequencer (Executive layer – intermediate layer that sequences or chooses between planner directives for the reactive layer [KELLEHER]), Planner (Deliberative layer – solves complex tasks, heavy computational components [KELLEHER]).

1.3 Objectives and Project Specifications

The main objective of this paper is an attempt to understand the world of vision based autonomous robots in terms of software architecture, vision data processing, control methods and algorithms.

The project follows a logical line of understanding the field and developing an application that uses some of the information gathered to demonstrate how a computer, representing the brain of a robot system, can be programmed to process environment information obtained from a web camera, to detect obstacles in the vision field and finally, apply control algorithms to avoid the obstacles.

Chapter 2 THEORY FUNDAMENTALS

This chapter is intended to provide basic theoretical aspects on each subpart of the autonomous robot control system implemented in this work. The structure of this chapter follows, more or less, the outline of the case study sub-chapter in chapter 1. That is, trying to follow a logical and natural approach to the system, in terms of information and control flow:

Sensing the environment

Processing sensor data

Applying control to the robot

The last sub-chapter presents the architectural approach of the system, from both information flow and software engineering perspective.

2.1 Computer Vision

In the previous chapter was shortly presented the way light reaches a sensor in a digital camera, by means of optics (lens). The webcam used as a vision sensor can be considered as a type of digital camera, recording images at a frame rate of 30 fps, in most of the cases.

After a field of light reaches the sensor and falls onto the active sense areas, the photons arriving at each active cell are integrated and digitized. (SZ) In most sensing devices, the light from different parts of the spectrum is integrated into the discrete red, green and blue (RGB) color values. [SZ2010]

2.1.1 Color Models and Color Spaces

A color model is a mathematical manner of describing the way colors are represented as tuples of numbers, typically three or four values or color components. [WPcs]

For a color model to be useful, it is mandatory that a mapping function exists to map between the color model and an absolute color space.

An absolute color space is a color space in which the difference perceived between colors is directly related to distances between colors, represented as points in the color space. [WPacs]

The colors chosen to form a color model are called primary colors and can be mixed to obtain new ones.

Red, green and blue are additive primary colors and can be added to produce cyan, magenta, yellow, white and all other colors that can be seen on our TV sets and monitors. [SZ2010]

Another set of colors that can be combined to obtain many others and used by printers instead of classic RGB are cyan, magenta and yellow. Black is also often used in four color printing. These are called subtractive primaries and form the CMYK color model (cyan, magenta, yellow, black). [SZ2010]

In Figure 2.1 the combination of primaries, both additive and subtractive, can be seen. In Figure 2.1 a) additive colors are combined to produce cyan, magenta, yellow and white. In Figure 2.1 b) subtractive colors are mixed to produce red, green, blue and black.

Figure 2.1 a) Additive primary colors Figure 2.1 b) Subtractive primary colors

[SZ2010]

2.1.2 RGB Color Model

The RGB color model is an additive color model in which red, green and blue light are mixed in various ways to reproduce a broad array of colors. [WPRGB]

RGB color model is widely used for sensing, representation and displaying images in electronic systems. [WPRGB]

The motivation behind choosing RGB primaries as a color model is the phisiology of the human eye. They maximize the difference between the response of the cone cells of the human retina to light of different wavelengths. [WPRGB]

In 1930s, the Commission Internationale d'Eclairage (CIE) standardized the RGB representation by performing color matching experiments using the primary colors of red (700.0nm wavelength), green (546.1nm) and blue (435.8nm). [SZ2010]

In most digital systems displayed in RGB, each pixel is represented by 3 bytes of data, 1 byte for each channel. That is, each of the three colors is represented by an integer between 0-255. [AIScs]

In Figure 2.2 is represented a color image and each individual channel. Each channel is grayscaled because it has only 1 byte for each pixel.

Figure 2.2 The original color image and the individual Red, Green and Blue channels [AIScs]

2.1.3 HSV Color Space

HSV (hue-saturation-value) color space is one of the most common cylindrical-coordinate representations of points in an RGB color model. [WPHSV]

Developed for computer graphics applications, HSV color space rearranges the geometry of RGB in an attempt to be more intuitive and perceptually relevant than the cartesian (cube) representation, by mapping the values into a cylinder, inspired by the traditional color wheel. [WPHSV]

HSV color space comes handy when trying to identify intensity component of the light, which is impossible to define by means of colors combinations. For example, distinguishing between different intensities and saturation of the same color can be handy when trying to identify objects of a specific color, seen under different lighting conditions.

In Figure 2.3 can be seen the geometrical representation of the HSV color space.

Figure 2.3 HSV geometry

The HSV color space has 3 channels, but not defined by color names that could be combined to obtain other colors.

The Hue channel represents the color (for example, red).

The Saturation channel represents the "amount" of color (differentiates between pale and pure green, for example).

The Value or Intensity is the brightness of the color. (light blue or dark blue, for example). [AIScs]

2.2 Processing Vision Data

As presented in previous case study chapter, there are many algorithms and techniques in computer vision that can be used alone or combined to accomplish various tasks. Only by experience and trial one can choose and combine a subset of existing algorithms to fit the needs of the project. Only by a very good understanding of computer vision and existing algorithms, together with a good mathematical background, one can develop new algorithms, if none of the existing completely fits the needs.

There isn't an universal algorithm to solve all the computer vision problems. Even choosing an algorithm from the existing ones can be difficult, because it requires decisions and compromises to be taken regarding the capabilities of system.

For demonstration purposes of computer vision capabilities, hsv color filtering and image segmentation based on color space was chosen for detecting obstacles in the vision field provided by the webcam sensor.

2.3 Fuzzy Control

The term "fuzzy logic" was introduced with the 1965 proposal of fuzzy set theory by Lotfi A. Zadeh. It has been applied in many fields from control theory to artificial intelligence. [WPFL]

Fuzzy logic has many common applications from consumer products such as cameras, camcorders, washing machines, microwave ovens to industrial process control, medical instrumentation, decision-support systems. [MWFL]

Fuzzy Logic is, in a narrow sense, a form of logical system having many values, in contrast with Boolean logic which has only 2 values. In a wider sense it can be said that it is somehow synonymous with the theory of fuzzy sets. Fuzzy set theory deals with classes of objects with unshaped boundaries, which makes fuzzy logic a "reasoning that's robust and approximate, rather than brittle and exact". [ITVBOAT2011]

Fuzzy logic deals with degrees of membership and degrees of truth. It uses a continuum of values between 0 and 1, 0 being completely false and 1 completely true. The values of fuzzy variables are more like linguistic terms than numerical values.

Fuzzy logic is relatively easy to implement and provides more consistent results than conventional control methods. In combination with neural networks, fuzzy logic can be a rather powerful tool, because it has a flavor of human brain by dealing with some levels of uncertainties and vagueness. Fuzzy logic takes some imprecise inputs and turns them into precise outputs.

Some of the reasons to use fuzzy logic, and also some scenarios when other approaches may be better, are presented bellow, according to [MWFL]:

Reasons to use FL

– conceptually easy to understand

– flexible

– deals very well with imprecise data

– can model nonlinear, complex systems

– builds on experience of engineers

– conventional control methods can be mixed with fuzzy logic

– based on natural language

When FL may not be the best approach

Some of the advantages of FL can also be disadvantages, depending on the situation:

– it is dependent on designer's experience

– it doesn't provide one receipt of a successful design; fuzzy logic has many paths that can be chosen towards the same goal; it also provides different mathematical tools to use in the process of fuzzification, membership computation of inputs (that is, deciding the level of 'truth' of the inputs), output computation

2.3.1 Foundations of Fuzzy Logic

Similar to Boolean logic, fuzzy logic has the same purpose: to map an input to an output. As a distinction, fuzzy uses a list of if-then statements called rules, which is an approach closer human reasoning.

In Figure 2.3.1, is a diagram of the fuzzy inference process: the general description to the left and an example to the right:

Figure 2.3.1 The fuzzy inference process [MWFL]

Fuzzy Sets

Unlike classical sets, fuzzy sets contain elements without clearly defined boundaries, having a partial degree of membership to the particular set. As a result, with fuzzy logic, the truth of a statement is defined as a degree, rather that a true or false value. As an example, a certain temperature can be seen as both HOT and MEDIUM in a certain degree, (or even LOW), depending of the system in which is evaluated.

Membership Functions

A Membership Function is a mathematical representation (numerical and graphical) of the degree of membership of input elements to a particular set, having a value anywhere between 0 and 1.

Logical Operations

According to [MWFL], fuzzy logical resoning is a superset of the boolean logic. That is, for degrees of membership of 0 (completely false) or 1 (completely true), standard logical operations will have the same results.

Operations that could be applied to the interval (0, 1) and keep the boolean logic meanings for the extremes, where found to be: min(A, B) as a substitute of AND, max(A, B) for OR and 1-A for NOT.

The Figure 2.3.2 shows how the three operations specified above work over a continuous range.

Figure 2.3.2 Boolean logic operators applied to a continuous range of values [MWFL]

Additional Fuzzy Operators

The classical Boolean operators AND, OR, NOT can be customized in fuzzy logic.

For example, the intersection (AND) of two fuzzy sets can be represented as a binary mapping which aggregates two membership functions, as follows:

The T operator could be, for example, a multiplication.

The Boolean union (OR) could be represented by another operator, for example an addition:

If-Then Rules

As explained above, the rules of fuzzy logic are very similar to the human reasoning, in terms of linguistic values (A, B), attributes or ranges (X, Y), antecedent or premise ("if X is A") and consequent or conclusion ("then B is Y").

According to [MWFL], interpreting if-then rules, is a three part process:

1. Fuzzify inputs: All fuzzy statements in the premise are assigned a degree of membership between 0 and 1.

2. Apply fuzzy operator to multiple part antecedents: Apply fuzzy logic operators to get a single value from all the parts of the premise.

3. Apply implication method: The output of a fuzzy logic system is also a fuzzy set, with a defined membership function. The value obtained by applying fuzzy operators to inputs, is translated into a value in the output fuzzy set.

2.3.2 Fuzzy Inference Process

The inference process refers to the steps developed in a fuzzy system that map a given input to an output.

In the Figure 2.3.3 is described an example of fuzzy inference process of a decision system for the value of the tip in a restaurant given two fuzzy inputs: service quality and food quality.

Figure 2.3.3 Fuzzy inference process [MWFL]

According to [MWFL], the basic layers of a fuzzy logic inference process are:

– Fuzzification of the input variables

– Application of the fuzzy operator (AND or OR) in the premise

– Implication from the antecedent to the consequently

– Aggregation of the consequents across the rules

– Defuzzification

In the Figure 2.3.4 is a fuzzy inference diagram.

Figure 2.3.4 Fuzzy inference diagram [MWFL]

Bellow, the main three steps (fuzzification, inference mechanism and defuzzification) are explained.

Fuzzification

Fuzzification is the translation of input data from a process into fuzzy variables (a "normalized domain"). This operation consists in two parts:

1.1 The first part defines the fuzzy variables that correspond to the inputs from the process to be controlled.

1.2 The second part defines sets of fuzzy input variables and their membership functions or operators on the fuzzy set. Membership functions do not have very rigid boundaries and they can cross each other's borders. Each membership function may be triangular, trapezoidal or bell shaped.

Fuzzy logic uses intersection (or minimuum), union (or maximum) and complement to represent standard logic operations like AND, OR and NOT. These operators are defined as follows:

NOT x = (1 – truth(x))

x AND y = minimum(truth(x), truth(y))

x OR y = maximum(truth(x), truth(y))

The triangular function can be defined as follows:

[WPFL]

The trapezoidal function can be represented as:

[WPFL]

Inference Mechanism

The generation of fuzzy rules is a process that depends of knowledge and experience of human experts (domain experts), the fuzzy model of the plant to be controlled and an analysis of the system. Because the appropriate fuzzy operator may not be known or, it may be defined different by the domain experts involved in the process, fuzzy logic uses a simple set of rules called the IF-THEN rules, or equivalent structures, such as fuzzy associative matrices.

The rule base is composed of two parts: the IF part and the THEN part.

The IF part is the antecedent part where rules are defined to describe the system state in terms of fuzzy propositions.

The THEN part is the consequent part which forms the desired conclusion of the output variable.

The IF-THEN rules for a temperature fuzzy controller may be defined as follows:

IF temperature IS very cold THEN stop fan

IF temperature IS cold THEN turn down fan

IF temperature IS normal THEN maintain level

IF temperature IS hot THEN speed up fan

[WPFL]

All the conditions are verified in the IF-THEN block therefore there is no ELSE statement.

Being the intermediate layer between fuzzification and deffuzzification, the inference mechanism provides ways to form the rules conclusion. The main methods used by the inference mechanism are: Max-Min inference and Max-Product inference.

Max-Min inference uses the Minimum operator to combine the antecedent of the IF-THEN rules, producing modified fuzzy sets for the ouputs. Then these sets are combined using the Maximum operator and the Max-Product inference

Deffuzification

Deffuzzification is the process of mapping the fuzzy result of the infered action to a non-fuzzy control action having a value that best fits the infered control action. This value is calles the crisp value. There are several methods to compute this value such as Centre-of-Area, Centre-of-Largest-Area, Centre-of-Sums and Mean-of-Maximum. This methods are based on two mechanisms: Centroid and Maximum. Centroid methods are based on finding a balance point while the maximum methods search for the peak value of weight (area) of the fuzzy set.

Centre-of-Sum performs faster than many deffuzzification methods and does not restrict symmetric membership functions over the fuzzy set. This method performs the algebraic sum of individual output fuzzy sets instead of their union.

2.3.3 Types of Fuzzy Inference Systems

There are two most common types of inference systems: Mamdani and Sugeno.

Mamdani's inference system is the most common and was the first fuzzy logic used in control systems.

The Sugeno (or Takagi-Sugeno-Kang) technique is very similar to the Mamdani inference system, but the main difference, according to [MWFL], is that Sugeno output membership functions are either linear or constant.

Below are a few advantages of each type of inference system, according to [MWFL]:

Advantages of Sugeno method

– computationally efficient

– can be integrated with linear techniques like PID control

– works well with optimization and adaptive techniques

– has guaranteed continuity of the output surface

Advantages of Mamdani method

– intuitive

– widespread acceptance

– well suited to human input

2.4 System Architecture

2.4.1 Introduction

Vital functions of robots are provided by software and software dominance is still growing. Mastering the software complexity is not only demanding but also indispensable task towards an operational robot. [Comp] A well designed software architecture can provide a robust and extensible robot control system, as well as possibility of a distributed implementation of different parts of the system.

2.4.2 Component Based Software Engineering in Robotics

According to [Comp], Component-Based Software Engineering shifts the emphasis in system-building from traditional requirement analysis, system design and implementation to composing software systems from a mixture of reusable off-the-shelf and custom-built components.

Component based software can be considered a natural approach to designing a software robot control system. Robotic systems were implemented long before software reached the current expansion in every aspect of our life. When looking to a hardware system, like a robot, with very little to no software components, one can see that the whole hardware structure, from mechanics to electronics, is based on small components communicating with each other, either through mechanical joints or electric signal traveling through wires and pins. And this approach of building a system from small, more or less independent components cooperating, proved to be an innate and feasible solution for most of the applications.

Components represent distribution of tasks into logic unities of software that can be developed more or less independently and tied together to build the complexity that is a robot system.

A software component should be designed and implemented to satisfy a set of requirements, without worrying about the other parts of the system. The only restrictions regarding the system should refer to the interfaces and data types the component provides and requires to link to other components in the system.

According to [Comp], the following are the technical requirements when developing a software component:

Dynamic wiring – allows changes to connections between components at runtime

Component interfaces – supports loosely coupled components and should follow standardized interface semantics

Asynchronicity – components run independently and communicate with each other

Component internal structures – internal structure of components shouldn't be a restriction; it depends on the specific technical challenges of the functionality a component provides

Transparency – details of the internal structure should be hidden but at a certain level functional requirements of a component should be provided (for example initialization parameters, configuration etc)

Legacy code – a component should provide a way to use legacy code and software libraries inside its internal structure

Easy usage – a component can be seen as a framework; it should provide an easy learning curve of usage

Stable interfaces – interfaces provided by the component should be stable and designed following the requirements of the domain (system), independent of the technologies used

State-of-the-art software technology – a base framework on top of which a robot system builder could easily integrate components without being an expert software engineer

2.4.3 SmartSoft Component Model

SmartSoft is a component approach for robotics software based on communication patterns as core of a robotics component model. [SRUlm] SmartSoft provides a framework on top of which a robot system builder can create standardized components, with semantic interfaces based on communication patterns.

The main parts of the SmartSoft model are:

Components can be seen as processes from a technical point of view. They run asynchronously, can contain several thread, are wired dynamically with other components and communicate through predefined communication patterns.

Communication Patterns assist the component builder and provide standard ways to define component interfaces based on the predefined communication patterns. A communication pattern defines the communication mode, provides predefined access methods and hides all the communication and synchronization issues. They are based on two complementary parts named service requestor and service provider. [SRUlm] The two parts of the communication pattern are connected by client/server, publisher/subscriber or master/slave relationship.

In Figure 2.4 are sketched three components and the way they communicate by means of the communication patterns.

Figure 2.4 Components and communication patterns [SRUlm]

Communication Objects represent the data transmitted through the communication patterns. They can be used to parameterize generic communication patterns classes.

Services are instances of specific communication patterns. They can be seen as actual communication ports of the component defined by the communication pattern they implement and the data parameter.

In Table 2.1 are the communication patterns provided by the SmartSoft component model.

Table 2.1 Communication Patterns [SRUlm]

Chapter 3 SOLUTION

The solution implemented as an example of robot control system is using the C# language and Microsoft .NET framework. The project runs in a Microsoft Windows operating system, tested in both Windows 7 and Windows 8.

The C# language was chosen over other languages benefiting of OpenCV interfaces like C++ or Java considering the following aspects:

– C# is purely object-oriented, unlike C++, that has a mixture of OOP and procedural components

– C# is more type-safe

– memory is managed by the CLR, and there are less risks of memory leaks

– C# has many language constructs like delegates (function pointers), lambda expressions etc.

– threading and multitasking supported by a mature API; from .NET 4.5 multitasking is supported by language keywords like async/await

– the .NET framework has the benefits of being developed and supported by the same entity, unlike Java, that includes many third party libraries that sometimes can be a burden to be configured and integrated

-user interface for desktop is well suited for rich GUI applications and easy to use and implement by using both designer and code behind. Supports architectural patterns like MVVM (Model-View-Viewmodel) and MVC (Model-View-Controller). The two main UI technologies for desktop applications are WPF (Windows Presentation Foundation) and Windows Forms.

– benefits from a professional and comprehensive IDE, with great and easy to use debugging tools and can be enhanced with productivity tools like Resharper [Resharper] for coding assistance, fast and smart navigation, automated code refactoring etc.

The .NET framework can be ported to other platforms using the Mono cross platform .NET development framework. [Mono]

3.1 Analysis and Design

3.1.1 OpenCV and EMGU CV

OpenCV

OpenCV (Open Source Computer Vision) is a library of computer vision functions and algorithms developed by Intel and now supported by Willow Garage and Itseez. The library is cross-platform and focuses on real time image processing. [WPOCV]

According to [OCV_WG] OpenCV library is released under a BSD license, is free for both academic and commercial use. The library has interfaces written in C++, C, Java and Python and supports operating systems like Linux, Windows, Mac OS, iOS and Android. It is written in optimized C/C++ and has a strong support for multi-core processing.

OpenCV is used in many fields involving computer vision, like advanced robotics, gaming, interactive art, mines inspection, medical imaging.

The current version of the library is 2.4.9 and can be downloaded for free at [OCV]

According to [WPOCV], OpenCV is supporting most of in current research and development computer vision applications like: egomotion estimation (estimate the camera motion relative to the scene), facial and gesture recognition, human-computer interaction, motion understanding, object identification, segmentation and recognition, stereo vision, structure from motion, motion tracking, augmented reality. Also, OpenCV includes machine learning algorithms to support the latest researches in the field of robotics.

According to [OCV_ref], the main parts of the library are:

– core – the core functionality

– imgproc – image processing

– highgui – high level GUI and Media I/O

– video – video analysis

– calib3d – camera calibration and 3D reconstruction

– features2d – 2D features framework

– objdetect – object detection

– ml – machine learning

– flann – clustering and search in multi dimensional spaces

– gpu – GPU-accelerated computer vision

– photo – computational photography

– stiching – images stitching

EMGU CV

According to [EMGU_CV], Emgu CV is a cross platform .NET wrapper for OpenCV. Being a .NET wrapper, the library allows OpenCV functions to be called from .NET languages like C#, VB, VC++ etc. Emgu is cross-platform because is can be compiled in Mono, to run on non Microsoft systems like Linux, Mac OS X, iPhone, iPad and Android.

Emgu allows systems to be implemented using modern programming models like Object Oriented Programming and consequently supports many kind of architectures and programming paradigms. The OpenCV functions and algorithms are encapsulated in classes and interfaces, which makes it easier to work with.

Emgu CV has two layers:

– Layer 1: functions, structures and enumeration mappings reflecting those in OpenCV

– Layer 2: OpenCV data structures, functions and algorithms encapsulated in classes

In Figure 3.1 is a diagram of the two layers of Emgu CV, containing the examples of main components.

Figure 3.1 Emgu CV main components [EMGU_CV]

The current version of Emgu CV is 2.4.9 and can be downloaded for free at [EMGU_CV_SF].

3.1.2 MEF

The Managed Extensibility Framework or MEF is a library for creating lightweight, extensible applications. It allows developers to discover and use extensions with no configuration required. It also helps to easily encapsulate code, avoid hard dependencies and to reuse extensions both within and across applications. [MEF]

Extensibility is a very important factor in developing software for a robot system. Both the component based architecture and the possibility for future improvements as adding new sensors or processing components are a motivation for choosing a framework that deals with exactly these kind of problems.

The software application that drives the autonomous robot includes a potentially large number of smaller components and is responsible for creating and running them. Adding these components directly as source code and calling them has obvious drawbacks. First of all, existent code has to be modified to call the new components.

In a large and complex application, adding new components can be a complicated and risky task. Although the issue can be solved by providing decoupling through means of interfaces, the loaded components still have to be registered explicitly into a configuration file or by other means, which leads further to a complex maintenance of the system.

Instead of explicit registration of available components, MEF provides a way to discover them implicitly, via composition. [MEF]

There are two base attributes required by a MEF component: [Import] and [Export]. They refer to the component's dependencies (imports) and capabilities (exports). [MEF]

The composition container manages the creation of objects and fills the dependencies discovered by exports declared by other parts. The composition container can be set to look for exported parts in the current assembly, specified assemblies or even folders. The parts can be composed when the composing method is called and any other time during application lifetime by calling a refresh method on the container. This can lead to discover new parts added in specified folders during runtime.

MEF is part of .NET framework 4 and can be used with any type of applications whether Windows Forms, WPF or web technologies using ASP.NET.

3.1.3 TPL Dataflow Library

Choosing the actor model at the base of the software components leads inevitably to concurrency and asynchronous data flows.

The Task Parallel Library (TPL), available for .NET 4.5, is a library that provides software components to assist concurrent-enabled data flows in the application. It can be installed using NuGet Package Manager in Visual Studio 2012 and searching for Microsoft.Tpl.Dataflow.

TPL is ideal for applications that process data in multiple steps, as it provides support for creating pipelines of processing blocks that can link and communicate with each other in a parallel manner. Parallelism at any degree, is indispensable in an application based on tasks that perform CPU-intensive and time expensive processing, like image processing. [TPL]

The library also offers a great control over the manner data is buffered and moved in the system and handles many problems of parallelism and data sharing like synchronization and data availability.

The main components of the library are dataflow blocks that have the capability to buffer and process data. The library offers by means of interfaces, three types of blocks: source, target and propagator blocks, defined by interfaces:

ISourceBlock<TOutput> – interface that is implemented by source blocks

ITargetBlock<TInput> – interface implemented by target blocks

IPropagatorBlock<TInput,TOutput> – interface implemented by blocks that can act both as source and target.

A source block can be linked to a target block using ISourceBlock<TOutput>.LinkTo method. The basic link rules are: source blocks are linked to target or propagator blocks and type parameters of generic interfaces must match. As long as the basic rules are respected, blocks can be linked in any kind of combination to form dataflow and processing pipelines.

Main methods that are called to send messages to target blocks are Post<TInput> and DataflowBlock.SendAsync.

Multitasking concepts like completion and cancellation are also supported by the library.

The main predefined Dataflow block types are:

– Buffering blocks: BufferBlock<T>, BroadcastBlock<T>, WriteOnceBlock<T>

Buffering blocks hold data for use by data consumers. [TPL]

– Execution blocks: ActionBlock<T>, TransformBlock<TInput,Toutput>,

TransformManyBlock<TInput, TOutput>

Execution blocks call a user-provided delegate for each piece of received data. Every execution block object buffers input messages until the block is ready to process them. By default, these classes process messages in the order they are received, on message at a time. The degree of parallelism can be specified to enable execution blocks objects to process multiple messges concurrently. [TPL]

– Grouping blocks: BatchBlock<T>, JoinBlock<T1, T2>, JoinBlock<T1, T2, T3>,

BatchedJoinBlock<T1, T2>, BatchedJoinBlock<T1, T2, T3>

Grouping blocks combine data from one or more sources and under various constraints. [TPL]

Additional options can be set by providing a DataflowBlockOptions object to the constructor of dataflow block types.

3.1.4 Architecture

In Chapter 2, component model of a robot system software architecture was discussed. Also, in Chapter 1, some robot control architectures were presented, from the control point of view, with emphasis on the subsumption architecture.

3.2 Implementation

3.2.1 Setting up Emgu CV in Microsoft Windows

Emgu CV is fairly easy to setup and include in the projects. The latest version of the library can be downloaded from [EMGU_CV_SF] as an executable.

Emgu CV can be installed in a folder at choice. The OpenCV library and Emgu wrappers can be found as dll's (Dynamic Link Libraries) in bin folder under the root installation folder. Dll's can be copied in a project folder or referenced directly from the installation folder.

In the current solution all OpenCV and Emgu dll's where copied in the lib/Emgu folder under the solutions's root folder. Only Emgu dll's (Emgu.*.dll) must be referenced in the projects but they have strong dependencies to OpenCV dll's (opencv*.dll). In order to resolve all the current or future dependencies all the OpenCV dll's where copied in the application's output folder. The Emgu libraries are copied automatically when building the projects that reference them.

In most of the situations, only Emgu.CV.dll, Emgu.CV.UI.dll and Emgu.Util.dll are referenced.

3.2.2 Solution Structure and Architecture

In Figure 3.1 is presented an approach of 3-tiered architecture (the UI host is excluded from the robot control system layers), described by National Instruments in an article on their website [NIRArch2012].

Figure 3.2 Three layered architecture [NIRArch2012]

The structure above was used as a guide for the solution structure.

The solution is structured into folder regions, making it easy to follow the architectural design. Each folder may have one ore more class library projects. Only the WinForms project, which makes the application UI is alone and in the solution's root folder.

The projects are listed bellow, along with the folders they reside in and a short explanation, following a top-down layering approach:

ApplicationUI – WinForms project

ApplicationServices – folder

ApplicationServices.Contracts – class library project

ApplicationServices – class library project

SystemIntegrator – folder

SystemIntegrator.Contracts – class library project

SystemIntegrator – class library project

UILayer – folder

UILayer.Contracts – class library project

UILayer.Components – class library project

AlgorithmLayer – folder

AlgorithmLayer.Contracts – class library project

AlgorithmLayer.Components – class library project

PlatformLayer – folder

PlatformLayer.Contracts – class library project

PlatformLayer.Components – class library project

DriverLayer – folder

DriverLayer.Contracts – class library project

DriverLayer.Components – class library project

CommonInfrastructure – folder

CommonInfrastructure – class library project

CommonInfrastructure.Emgu – class library project

CommonInfrastructure.UI – class library project

ComponentInfrastructure – folder

ComponentInfrastructure.Contracts – class library project

ComponentInfrastructure.Base – class library project

In Figure 3.3 is a diagram of architecture of the entire system containing the projects of the system and relationships between them:

Figure 3.3 System architecture

3.2.3 Implementation Details

The main sections of this chapter are the component infrastructure, i.e. the common framework provided for the software components of the system and the implementation details of the components as following a common pattern as well as individual aspects of each component, in the layered manner of the system's architecture.

Component Infrastructure

The component framework consists of two projects:

ComponentInfrastructure.Contracts

This project contains interfaces (contracts) for the backbone of the system that are cross-cutting all the other parts of the system, being implemented by both main system classes (like ComponentPool or SystemIntegrator) and individual components.

Main contracts in this project are:

IComponent – implemented by each system component, in order to be automatically discovered by the SystemIntegrator, using MEF. This interface also provides methods to be implemented by components in order to be recognized and run automatically by the system.

IComponentPool – provides interfaces that allow adding and storing components in the same collection and retrieving by name.

IModule – a contract implemented by classes that need initialization and can be initialized in the same workflow.

ISystemIntegrator – provides contracts for actions involving the entire system, from component discovery, initialization, to component retrieval based on component name.

ComponentInfrastructure.Base

This project provides implementations for some of the interfaces in the Contracts project:

– Component: the abstract that should be extended by all system's components to make use of common implementation of some component methods (for example RunAsync() method is implemented in the abstract class and runs the Run() method by starting a new Task).

– ComponentPool is also implemented in this project and provides implementation for methods exposed by IComponentPool interface.

Components

In this chapter the system's components will be presented, in a layered approach, corresponding to the system's architecture.

The components part of the solution is divided in 8 projects: 4 contract projects and 4 implementation projects, as follows:

– Driver Layer – 2 projects

– Platform Layer – 2 projects

– Algorithm Layer – 2 projects

– UI Layer – 2 projects

The components make use of TPL Dataflow library to communicate with each other and all component contracts in the *.Contracts projects should expose interfaces with properties of IDataflowBlock type (parameterized interfaces that extend IDataflowBlock: ISourceBlock<T>, ITargetBlock<T>, IPropagatorBlock<Tin, Tout> etc.).

All the .Contracts projects contain two main folders: DataContracts (contains classes used as data parameters for the communication dataflow blocks) and Components (contains interfaces implemented by components in the component implementation projects). All interfaces in Components folder should extend IComponent interface in ComponentInfrastructure.Contracts and all component implementations should implement their specific interfaces and extend the Component abstract class in ComponentInfrastructure.Base project.

The components are retrieved by the system integrator based on name as IComponent and cast to the interfaces from .Contracts they implement to be linked to other components using the source/target blocks (ports) they expose.

Driver Layer

The driver layer contains components interacting with hardware like sensors or robot actuators. These components aren't exactly drivers for hardware, that is they don't interact directly with hardware but act as drivers for the implemented robotic software system.

WebCamComponent

The WebCamComponent interacts with the webcam acting as a vision sensor through the Emgu CV class Capture. This component can be replaced with any other component that can retrive and process vision data as an image or stream of images for testing purposes.

The WebCamComponent implements the interface IWebCamComponent in DriverLayer.Contracts project that exposes a property of type ISourceBlock<WebCamData>. This component can be linked to any other component that exposes a property of type ITargetBlock<WebCamData> through Link(ITargetBlock<T> target, DataflowBlockOptions options) method exposed by ISourceBlock<T>. The WebCamData class is a data contract class found in DriverLayer.Contracts.DataContracts namespace. It contains two parameterized properties of type Image<TColor, TDepth> found in Emgu.CV namespace: Image<Gray, Byte> GrayImage and Image<Bgr, Byte> ColorImage. This data contract class is used to transport either color or gray image data between components.

Platform Layer

This layer contains components dealing with platform specific computation algorithms like image processing and obstacle detection.

HsvObjectDetectorComponent

This component implements the IHsvObjectDetectorComponent interface and extends Component abstract class. Through the IHsvObjectDetectorComponent it exposes two dataflow communication blocks: ITargetBlock<HsvFilterConfiguration> (receives configuration data from UI) and IPropagatorBlock<WebCamData, DetectedObjects> (transforms image data received from the web cam through WebCamData objects into DetectedObjects data that contains information about the vision field – width, height and the detected areas – XY coordinates, width, height etc. of the rectangles defined as a contour for the detected objects via HSV filtering).

Algorithm Layer

ObstacleAvoidanceControllerComponent

This component received data about the detected objects and computes a command for the robot in the form of a negative or positive value. A negative value means the robot should take left to avoid the obstacle, while a positive value means the robot should take right. The value represents a real number with the meaning of the command amplitude (when transmitted to the robot this could mean the speed of the movement).

The obstacle for which are computed the fuzzy inputs is chosed by an instance of the ObstacleSelector class. The algorithm of obstacle selection computes the weight of each obstacle, which is equal to the weighted average of the distance related to the field of vision and distance to the center of the obstacle.

This component calls the fuzzy inference system implemented in Matlab and described later in this paper.

The ObstacleAvoidanceControllerComponent implements the interface IObstacleAvoidanceControllerComponent and extends the Component base class for all components.

Through the IObstacleAvoidanceControllerComponent interface, the controller exposes the following dataflow blocks:

– 2 x ITargetBlock<DetectedObjects> – there are two ports of this type exposed because one of them responds to the input with an asynchronous action. These ports receive data describing the positions of the detected obstacles in the vision plane (image).

– ITargetBlock<FuzzyConfiguration> – this port receives a FuzzyConfiguration object with the parameters for the fuzzy controller, like the range of distance and angle and the weights of the distance and how central an object is, used by the algorithm that selects the main obstacle.

UI Layer

The UI Layer contains components exposing both dataflow communication blocks and methods called by the UI through application services. This layer receives and serves data from and to application UI and sends data input by user like configuration parameters to subsequent layers.

WebCamDisplayComponent

This component implements IWebCamDisplayComponent and extends Component abstract class. It exposes two dataflow blocks of type ITargetBlock<WebCamData>. Both blocks are initialized as ActionBlock<WebCamData>. One block is initialized with a void method and one with an asynchronous method, that returns a Task object. The IWebCamDisplayComponent also exposes a method to register UI controls of type PictureBox used to display the data received from the webcam. PictureBox controls in different forms, like the main form or the hsv filter configuration form can be registered in this component and the data receied from the web cam is displayed on every registered control.

HsvFilterUIComponent

This component is a link component between the application UI Form for HSV filter configuration and the HsvObjectDetectorComponent. The UI Form interacts with this component through application services. It extends the Component abstract class and implements IHsvFilterUIComponent interface.

The IHsvFilterUIComponent interface provides methods that can be called by application services and two dataflow blocks: ISourceBlock<HsvFilterConfiguration> and ITargetBlock<WebCamData>. The first block is used to send configuration data to the HsvObjectDetectorComponent and the second dataflow block is used to receive webcam data that is processed with the same algorithm that HsvObjectDetectorComponent uses, in order to display the processing results like the filtered image and the detected objects with a red rectangle contour.

The methods exposed by the IHsvFilterUIComponent provide means to get and set the current hsv filter configuration and register PictureBox controls on which the component displays processing results: filtered image and detected filtered objects.

CommandSimulatorComponent

This component displays data from commands received from controllers. It is used to simulate commands further sent to actuators through a component that should be placed in PlatformLayer. This component isn't implemented in the current application as it is not needed for the demonstration purpose of the project.

The CommandSimulatorComponent is placed in the UILayer because interacts with a PictureBox component in the main window of the application. The PictureBox component is registered through the RegisterCommandDisplay(PictureBox pictureBox) contract exposed by the ICommandSimulatorComponent interface and called by the main form through a method with the same signature exposed by the IControlSimulatorService and implemented by the ControlSimulatorService in ApplicationServices project.

The component also exposes a dataflow block property ITargetBlock<ControlCommand>. The controllers can link to this port if they expose an ISourceBlock<ControlCommand> port.

In Figure 3.4 is an example of command data displayed on the CommandSimulatorPictureBox in the main form.

Figure 3.4 Simulated command

Application UI

The user interface project, called ApplicationUI resides in the root folder of the solution and is a WinForms project. However, the system could be provided other implementations, like WPF (Windows Presentation Foundation) or even a web interface (although real-time processing and displaying of webcam data could suffer delays because of transmission of data through web protocols).

The project sends and receives data to/from the components using application services through means of interfaces defined in ApplicationServices.Contracts project. At the moment services are instantiated inside the contructor of each form, as required and, as a result, the user interface project is dependent of ApplicationServices implementations project. The services could be later instantiated by means of a service locator, and this dependency removed.

The project also has a reference to the ComponentInfrastructure.Contracts project and to the implementation of ISystemIntegrator. The Initialize() method is called on the singleton instance of SystemIntegrator inside the Main() method of the static Program class, just before the call Application.Run(new Form1()), in order to ensure initialization of components before the application starts.

The application's user interface is simplistic, for demonstration purposes. It consists of three forms:

– Form1 – the main form. Has a PictureBox component that displays the current analyzed image, a PictureBox control to display the command received from the controller and two buttons: Config HSV (opens the configuration form for the hsv filter) and Test Image (opens the form connected to the component used to open and send to processing pipeline an image from disk instead of using the webcam or the test animation video).

– HsvColorDetectorView – the HSV filter configuration form. Contains three PictureBox controls: original image, filtered image (black background and white for the colors in the configured hsv range), detected objects (original image with filtered objects with red rectangular contour); the form also contains controls for the HSV filter configuration that is sent to the HsvFilterUIComponent in real time to apply the new values to the processed image.

– TestPicture – this form allows the user to browse an image from disk and send it to the processing pipeline for testing purposes.

In the figures bellow are the three forms of the application.

Figure 3.5 Main form

Figure 3.6 HSV configuration form

Figure 3.7 Open/Send test image form

Application Services

The application services are contracts and classes that provide means for the user interface to communicate with the system's components. They act as a separation layer between the user interface and the rest of the system, in both logical and architectural manner. This layer contains two projects, that reside under the ApplicationServices folder:

– ApplicationServices.Contracts – data and service contracts

– ApplicationServices – implementations of the service contracts in the .Contracts project

Common projects

At the moment of writing this paper, there are three common projects used through the application by all components, as helpers for different tasks such as image processing or displaying an image on a PictureBox asynchronosly control using the Invoke method of the UI control. These projects are:

– CommonInfrastructure

– constant classes like ComponentConstant (used to store component names)

– Timer helper to measure and display the duration of execution of tasks like detecting obstacles

– CommonInfrastructure.Emgu – utilities classes that contain algorithms and objects that use Emgu features (like HsvColorFilter, ContourFinder, ContourPainter etc.)

– CommonInfrastructure.UI – contains UI helper classes like extensions of PictureBox control to display Emgu IImage object.

3.2.4 HSV filter

A reference image (webcam, test video, test image) is sent to and processed by the HsvObjectDetectorComponent by using two classes in the CommonInfrastructure.Emgu project:

– HsvColorFilter – method Image<Gray, Byte> Filter(Image<Bgr, Byte> image, HsvFilterParameters parameters) receives the image to process and parameters like: hue, saturation, value, enabled parameter for the previous ones and returns a filtered image with black background and white elements in HSV range defined by the parameters method parameter.

– ContourFinder – used to find contours of areas filtered by the HSV filter and returns a list of rectangles representing these contours.

3.2.5 Fuzzy controller

The control process of the obstacle avoidance component consists of a controller implemented in Matlab and integrated into the .NET solution.

In this chapter the steps of implementing a fuzzy controller in Matlab are described, with examples from the implemented obstacle avoidance controller used in the project.

Tools used to implement the fuzzy controller are Matlab with Fuzzy Logic Toolbox and the Matlab deployment tool, used to generate .NET assemblies from the Matlab files, to be used in the implemented solution.

The type of fuzzy inference system used to implement the controller is Mamdani, as it is the most common and intuitive technique.

Fuzzy controller in Fuzzy Toolbox

The Fuzzy Toolbox can be accesed either from the Matlab Applications menu or by typing the command fuzzy in the command line.

The graphical tools provided by Fuzzy logic toolbox to implement a fuzzy logic system, are:

– Fuzzy Inference System Editor – handle high level parameters of the system: the number of inputs/outputs, variable names, etc.

– Membership Function Editor – define the types, ranges and shapes of membership functions associated with each variable

– Rule Editor – create the set of rules that define the behavior of the system

– Rule Viewer – can be used as a diagnostic tool; the values of input variables can be changed in order to see how they influence the output of the system

– Surface Viewer – plots a surface map of the output relative to the inputs of the system

In the Figure 3.8 are displayed the 5 graphical tools described above:

Figure 3.8 Fuzzy Toolbox GUI Editors [MWFL]

Fuzzy Inference System

The main parameters of the FIS are set up in the FIS Editor, which is the main window that opens when accesing the Fuzzy Toolbox.

Variables can be added to the system by accessing the menu: Edit -> Add Variables. Either input or output variables can be added by this method.

The designed fuzzy controller has two inputs: distance and angle and one output command.

Both input and output variables have predefined ranges in the fuzzy inference system. The input values representing the distance and the angle of the closest, most central object detected are normalized to these ranges before they are sent to the controller.

The distance represents a normalized number between 1 and 100 computed relative to the vision field height. Objects with the base of the contour higher in the reference image are at a higher distance than objects closer to the base of the reference image. In Figure 3.9 is represented the way the distance of an object is perceived.

Figure 3.9 Detected objects and their perceived distances

The formula used to compute the normalized value of the distance input is:

where

Fd – Fuzzified distance

Vd – vision distance: computed difference between the height of the fision field and the Y coordinate of the object.

division – value of a fuzzified division of the vision field

A division's value is computed related to the height of the vision field:

The angle input represents the perceived distance of the object relative to the center of the vision field. The angle has a predefined range of the form: [-x, x]. Negative values of the angle represent objects with the central point to the left of the median axis of the vision field and angles with positive values are objects to the right. In Figure 3.10 is an example of how the angle of an object relative to the central axis of the image is deduced:

Figure 3.10 Angle of a detected object

The angle value is computed the same way as the distance taking into account X-coordinates of the of the central points of the object and vision field, with the observation that the result has either positive or negative sign, depending of the position of the object relative to the center of the field, as specified above.

The formula used to fuzzify the angle:

where:

– Fa – fuzzy angle input

– Xc – X coordinate of the object's center

– Xvfc – X coordinate of the vision field's center

– division – the value of an angle division computed by dividing the field width to the double of the maximum angle value (2*180 – there are 360 divisions between -180 and 180).

The ranges of each variable of the inference system are:

– distance: [0 100]

– angle: [-180 180]

– command: [-10 10]

In the case of both angle and command value, negative values represent LEFT and positive values represent RIGHT.

The Figure 3.11 shows the FIS Editor with the designed system.

Figure 3.11 FIS Editor

Membership Functions

The Membership Functions window is a tool used to define the parameters of the membership functions associated with each variable like type, range, shape. This tool can be accessed either from the menu Edit -> Membership Functions or by double click one of the system's variables in the FIS Editor.

The Membership Functions editor has predefined membership functions for each variable. In order to define membership functions, it is recommended to start from scratch and remove the predefined functions. This can be achieved by selecting the desired variable and from the menu Edit -> Remove All MFs.

After removing the predefined membership functions, the user can define its own either by choosing the one toolbox's functions or custom ones.

To add membership functions for a variable, using the toolbox's functions, one should access the menu Edit -> Add MFs. This way, a window opens and provides selectable types of functions and the number of functions to add. The window is presented in Figure 3.12.

Figure 3.12 Add membership functions

Some of the types of membership functions that can be added, are:

– trimf: triangular shape

– trapmf: trapezoidal shape

– gausmf: gaussian functions

There are many other types of selectable types provided by the toolbox, each with its own shape and mapping function between variable values and the membership of the value.

In the next figures are the membership functions for each variable of the system and the editable parameters:

Figure 3.13 Distance Membership Functions

Figure 3.14 Angle Membership Functions

Figure 3.15 Command Membership Functions

Rule Editor

The Rule Editor can be accessed by double click the rectangle in the FIS Editor that represents the connection box between the inputs and outputs or by selecting from the menu Edit -> Rules.

In this window are created the rules of the system that transform the inputs into the ouputs. As described in the chapter dedicated to Fuzzy Logic, these rules are if-then rules.

The Rule Editor is intuitive and provides means to create rules as required by the fuzzy mechanism, with inputs in the premise and outputs in the conclusion. Fuzzy logic operands between the inputs and weight of each rule can also be defined in this tool.

Before defining the rules of the system, a table with the mapping of the outputs related to the inputs was implemented:

Table 3.1 Fuzzy Rules Mapping

The meanings of the abbreviations are explained bellow:

Distance input:

Table 3.2 Distance variables

Angle input:

Table 3.3 Angle Variables

Command output:

Table 3.4 Command variables

In the Figure 3.16 is the Fuzzy Rule Editor with the rules defined for the implemented obstacle avoidance controller.

Figure 3.16 The Rule Editor

Rule Viewer

The Rule Viewer displays graphs of all the rules and the computed output, with possibility to change the input values either by dragging horizontal bars or changing values in the input vector. Both input values and the output values are described by red horizontal bars and the degree of membership for each function by a colored fill inside the graphs.

The Rule Viewer and some input values with the result value are exemplified in Figure 3.17.

Figure 3.17 The Rule Viewer

Surface Viewer

The surface viewer is a 3D plot of the output as a function of the inputs, described in Figure 3.18 bellow.

Figure 3.18 Surface Viewer

Generate .NET assemblies

In order to be used from the implemented .NET solution, the fuzzy controller must be exported as a .NET assembly to be referenced from the project using it.

Integration of fuzzy controller with the .NET project

The exported fuzzy controller is used in the AlgorithmLayer project, class FuzzyController, called by the ObstacleAvoidanceComponent.

In order to generate assemblies that can be used from a .NET project, several steps must be completed, according to [MWmtodll]:

Open the Deployment Project window

The Deployment Project window can be opened either by accessing Matlab applications menu or by using the command deploytool in Matlab command window.

In Figure 3.19 is the Deployment Project window.

Figure 3.19 Deployment Project window

Create a Matlab project

An existing Matlab project can be selected or a new project created using the Deployment Window.

In the Type dropdown, select .NET Assembly, give a name and select a path to the new project. After clicking OK, a new window opens, with project pre-build configuration.

Add files and create classes

In Figure 3.20 is an example of a created class FuzzyControl, with methods generated from the file GetFuzzyOutput.m. Bellow the created class is the new class form.

Figure 3.20 Add files and classes

The GetFuzzyOutput.m file is a Matlab function used to read the FIS.mat file that contains the structure of the fuzzy inference system, calls the fuzzy system for the input array and return the output value.

The readfis command

The FIS.mat file is an exported file for a structure generated in Matlab Command Window by calling the command readfis('FuzzyObstacleAvoidance.fis').

In the Figure 3.21 is shown how the readfis command was called to generate the FIS variable in Matlab workspace.

Figure 3.21 Read the Fuzzy Inference System

The FIS variable was exported to FIS.mat by using the Save As command from the context menu of the FIS variable.

The GetFuzzyOutput function in the file GetFuzzyOutput.m is shown in Figure 3.22.

Figure 3.22 The function calling the Fuzzy Inference System

Add MCR (Matlab Compiler Runtime)

If the project is run on a machine that hasn't an installation of Matlab, the Matlab Compiler Runtime can be added to the generated project when building on by hand. The MCR can be found at [MWMCR]. The version of the MCR should match the version of the Matlab installation used to generate the project, in this case R2012b, Windows x64.

Build the project

Click the Build icon at right side of the project path bar to build the project. This step is exemplified in Figure 3.23.

Figure 3.23 Build

In Figure 3.24 is the window that shows the buiding project, opening after clicking Build.

Figure 3.24 Building the project

The structure of the generated project can be seen in Figure 3.25 bellow.

Figure 3.25 Project structure

Add references to MWArray.dll and to the generated assembly

The assembly generated can be added as a reference to the project by the project context menu option Add Reference. In this case, a reference should be added to the assembly FuzzyObstacleAvoidance.dll. Also, the FIS.mat file should be copied to the output folder (Application/x64 folder, which is the output folder set for al projects). The data structures used to interact with the generated methods can be found in the MWArray.dll assembly. This assembly can be found either in Matlab or in MCR installation folder (search subfolders for assembly).

Use the generated classes

The class that calls the Matlab generated assembly is FuzzyController, that inplements the interface IFuzzyController and is called by the ObstacleAvoidanceControllerComponent.

In Figure 3.26 is shown how the MWArray Fuzzy.GetFuzzyOutput(MWArray input) method is used in the FuzzyController class.

Figure 3.26 Use the Matlab generated class

Chapter 4 TESTING AND VALIDATION

In this chapter are summarized the results of the tests applied to the application, from 2 perspectives: obstacle detection and control algorithm.

4.1 Obstacle detection

The algorithm used to detect obstacles (the HSV filter) in the field of vision was tested using two methods: a video of some obstacles modeled in Blender [Blender], and images with rectangles of different colors drew in MS Paint.

4.1.1 Detecting obstacles in a video

This test was performed mainly to test the speed of the algorithm and to simulate a real scenario representing a camera capture in a field with obstacles. The reason for the use of a video with motion through a field with obstacles instead of a webcam is simplicity and ease of the method. It is more convenient to construct and record obstacles in a CGI scene than to use real objects and move the camera while connected to the computer. The tested scene consists of gray objects and background and is important mainly for testing how fast the application detects obstacles. The .NET Stopwatch class was used for this purpose.

In Figure 4.1 bellow is shown the detection of obstacles in a recorded video of a moving through objects in a 3D scene.

Figure 4.1 Detecting obstacles

4.1.2 Detecting obstacles in pictures

Unlike the previous test, in this case where used images with rectangular objects of different colors. The purpose of this test is to observe how the configuration of the HSV filter, together with the color of the background affects the detection of obstacles of different colors.

In the figures bellow are the results of detecting squares of different colors on a white background, for different hue values of the HSV configuration. The saturation and value parameters are disabled. On the PictureBox of the original image (most left one) can be seen on one of the squares a red circle whose position is computed by the obstacle selector class, indicating the most eligible obstacle.

Figure 4.2 One obstacle detected; hue range: 79-125

Figure 4.2 Three obstacles detected; hue range: 100-180

Figure 4.4 Four obstacles detected; hue range: 50-180

Figure 4.5 Five obstacles detected; hue range: 20-180

Figure 4.6 Six obstacles detected; hue range: 1-180

4.2 Control

4.2.1 Speed

The duration of the computing a control value was measured and displayed in console using the video test method. The start moment is the moment when the controller received data through the ITargetBlock<DetectedObjects> property. The timer stops before the method returns the computed control value. The .NET Stopwatch class was used for this purpose.

In the Figure 4.7 are the some values logged in the console representing the execution of the ApplyControl(DetectedObjects objects) method of the ObstacleAvoidanceControllerComponent.

Figure 4.7 Control command speed

4.2.2 Control values

The control values computed by the fuzzy controller are displayed in a box bellow the current computed image PictureBox on the main form. The negative values are displayed as take left commands, while positive values are displayed as take right commands. The magnitude of the control depends on the distance of the obstacle and how central the obstacle is. In the figures bellow are shown some detected obstacles (marked with red circle on the analyzed image) and the generated control command.

Figure 4.8 Values of take left control commands

Figure 4.9 Values of take right control commands

Chapter 5 CONCLUSIONS

In latest years the development of image processing algorithms and implemented solutions of computer vision libraries available for free, like Open CV, led to a vast range of applications in robotics, implemented by large companies, universities or even hobbyists. The computation power of small and cheap computers of the size of a credit card like Raspberry Pi [RaspPi] or BeagleBone [Beagle] makes it easy to implement computer vision applications in robotics without the use of larger computers.

As seen in this paper, there is a large amount of solutions to computer vision and obstacle avoidance problems, that can be selected or combined to cover each algorithm's limitations or required purpose: background segmentation, obstacle detection, motion detection, reference object detection etc.

When it comes to control, there are also lots of possibilities to choose from, but, as demonstrated in the current solution, Fuzzy control is one of the easiest and intuitive, with a great support from tools like Matlab. Other approaches like artificial intelligence using neural networks or combined neuro-fuzzy algorithms could be better choices in more complex requirements of the application.

Regarding the architectural design of the solution, the application shows the advantages of a component model with actor-based communication between components, that makes it easy to implement components of the system independently with care only for data contracts between connection points.

The obstacle avoidance based on computer vision problem may be considered one of the fundamental topics when implementing a robotic system based on both present and future developments in the field of robotics.

BIBLIOGRAPHY

[SZ2010] Richard Szeliski, "Computer Vision: Algorithms and Applications", Springer, 2010

[ITVBOAT2011] Mehmet Serdar Guzel and Robert Bicker (2011). Vision Based Obstacle Avoidance Techniques, Recent Advances in Mobile Robotics, Dr. Andon Topalov (Ed.), ISBN: 978-953-307-909-7, InTech, Available from: http://www.intechopen.com/books/recent-advances-in-mobile-robotics/vision-based-obstacle-avoidancetechniques

[PASSOW] Benjamin N. Passow, „Robot Architectures”, De Montfort University, UK

[KELLEHER] John Kelleher, „Robot Control Architectures”, Dublin Institute of Technology

[NIRArch2012] „A Layered Approach to Designing Robot Software”, http://www.ni.com/white-paper/13929/en/, 2012

[Brooks1985] Rodney A. Brooks, „A Robust Layered Control System for a Mobile Robot”, A.I. Memo 864, MIT, 1985

[Comp] Christian Schlegel, Andreas Steck, Alex Lotz, „Model-Driven Software Development în Robotics: Communication Patterns aș Key for a Robotics Component Model”, University of Applied Sciences Ulm, Germany

[Gonzales-Taha2007] Julirose Gonzales and Zahari Taha, "Mobile Robot Navigation Using Open Computer Vision with Fuzzy Controller", Department of Engineering Design & Manufacture, University Malaya, Kuala Lumpur, Malaysia, 2007

[Bay2008] Herbert Bay, Andreas Ess, Tinne Tuytelaars, Luc Van Gool, "SURF: Speeded Up Robust Features", Computer Vision and Image Understanding (CVIU), Vol. 110, No. 3, pp. 346–359, 2008

[WPfd] http://en.wikipedia.org/wiki/Feature_detection_(computer_vision)

[WPcs] http://en.wikipedia.org/wiki/Color_space

[Wpacs] http://en.wikipedia.org/wiki/Absolute_color_space

[WPRGB] http://en.wikipedia.org/wiki/RGB_color_model

[WPHSV] http://en.wikipedia.org/wiki/HSL_and_HSV

[AIScs] http://www.aishack.in/2010/01/color-spaces/

[WPFL] http://en.wikipedia.org/wiki/Fuzzy_logic

[TPL] http://msdn.microsoft.com/en-us/library/hh228603(v=vs.110).aspx

[MEF] http://msdn.microsoft.com/en-us/library/dd460648(v=vs.110).aspx

[WPshakey] http://en.wikipedia.org/wiki/Shakey_the_robot

[SRUlm] http://servicerobotik-ulm.de/drupal/?q=node/19

[OCV_WG] https://www.willowgarage.com/pages/software/opencv

[OCV] http://opencv.org/

[OCV_ref] http://docs.opencv.org/modules/refman.html

[WPOCV] http://en.wikipedia.org/wiki/OpenCV

[EMGU_CV] http://www.emgu.com/wiki/index.php/Main_Page

[EMGU_CV_SF] http://sourceforge.net/projects/emgucv/

[Resharper] http://www.jetbrains.com/resharper/

[Mono] http://www.mono-project.com/Main_Page

[MWFL] http://www.mathworks.com/help/fuzzy/index.html

[MWmtodll] https://www.mathworks.com/matlabcentral/answers/uploaded_files/5510/MATLAB -C%23 interfacing.pdf

[MWMCR] http://www.mathworks.com/products/compiler/mcr/

[Blender] http://www.blender.org/

[RaspPi] http://www.raspberrypi.org/

[Beagle] http://beagleboard.org/

BIBLIOGRAPHY

[SZ2010] Richard Szeliski, "Computer Vision: Algorithms and Applications", Springer, 2010

[ITVBOAT2011] Mehmet Serdar Guzel and Robert Bicker (2011). Vision Based Obstacle Avoidance Techniques, Recent Advances in Mobile Robotics, Dr. Andon Topalov (Ed.), ISBN: 978-953-307-909-7, InTech, Available from: http://www.intechopen.com/books/recent-advances-in-mobile-robotics/vision-based-obstacle-avoidancetechniques

[PASSOW] Benjamin N. Passow, „Robot Architectures”, De Montfort University, UK

[KELLEHER] John Kelleher, „Robot Control Architectures”, Dublin Institute of Technology

[NIRArch2012] „A Layered Approach to Designing Robot Software”, http://www.ni.com/white-paper/13929/en/, 2012

[Brooks1985] Rodney A. Brooks, „A Robust Layered Control System for a Mobile Robot”, A.I. Memo 864, MIT, 1985

[Comp] Christian Schlegel, Andreas Steck, Alex Lotz, „Model-Driven Software Development în Robotics: Communication Patterns aș Key for a Robotics Component Model”, University of Applied Sciences Ulm, Germany

[Gonzales-Taha2007] Julirose Gonzales and Zahari Taha, "Mobile Robot Navigation Using Open Computer Vision with Fuzzy Controller", Department of Engineering Design & Manufacture, University Malaya, Kuala Lumpur, Malaysia, 2007

[Bay2008] Herbert Bay, Andreas Ess, Tinne Tuytelaars, Luc Van Gool, "SURF: Speeded Up Robust Features", Computer Vision and Image Understanding (CVIU), Vol. 110, No. 3, pp. 346–359, 2008

[WPfd] http://en.wikipedia.org/wiki/Feature_detection_(computer_vision)

[WPcs] http://en.wikipedia.org/wiki/Color_space

[Wpacs] http://en.wikipedia.org/wiki/Absolute_color_space

[WPRGB] http://en.wikipedia.org/wiki/RGB_color_model

[WPHSV] http://en.wikipedia.org/wiki/HSL_and_HSV

[AIScs] http://www.aishack.in/2010/01/color-spaces/

[WPFL] http://en.wikipedia.org/wiki/Fuzzy_logic

[TPL] http://msdn.microsoft.com/en-us/library/hh228603(v=vs.110).aspx

[MEF] http://msdn.microsoft.com/en-us/library/dd460648(v=vs.110).aspx

[WPshakey] http://en.wikipedia.org/wiki/Shakey_the_robot

[SRUlm] http://servicerobotik-ulm.de/drupal/?q=node/19

[OCV_WG] https://www.willowgarage.com/pages/software/opencv

[OCV] http://opencv.org/

[OCV_ref] http://docs.opencv.org/modules/refman.html

[WPOCV] http://en.wikipedia.org/wiki/OpenCV

[EMGU_CV] http://www.emgu.com/wiki/index.php/Main_Page

[EMGU_CV_SF] http://sourceforge.net/projects/emgucv/

[Resharper] http://www.jetbrains.com/resharper/

[Mono] http://www.mono-project.com/Main_Page

[MWFL] http://www.mathworks.com/help/fuzzy/index.html

[MWmtodll] https://www.mathworks.com/matlabcentral/answers/uploaded_files/5510/MATLAB -C%23 interfacing.pdf

[MWMCR] http://www.mathworks.com/products/compiler/mcr/

[Blender] http://www.blender.org/

[RaspPi] http://www.raspberrypi.org/

[Beagle] http://beagleboard.org/