Feature and Performance Comparison [627555]

Feature and Performance Comparison
of the V-REP, Gazebo and ARGoS Robot
Simulators
Lenka Pitonakova(B), Manuel Giuliani, Anthony Pipe, and Alan Winfield
Bristol Robotics Laboratory, University of the West of England, Bristol, Bristol, UK
[anonimizat]
Abstract. In this paper, the characteristics and performance of three
open-source simulators for robotics, V-REP, Gazebo and ARGoS, are
thoroughly analysed and compared. While they all allow for program-
ming in C++, they also represent clear alternatives when it comes tothe trade-off between complexity and performance. Attention is given
to their built-in features, robot libraries, programming methods and the
usability of their user interfaces. Benchmark test results are reported inorder to identify how well the simulators can cope with environments of
varying complexity. The richness of features of V-REP and the strong
performance of Gazebo and ARGoS in complex scenes are highlighted.Various usability issues of Gazebo are also noted.
1 Introduction
Simulation is a useful scientific tool that can complements more traditional
experimental approaches [ 1]. Choosing a suitable simulator is important, as
different simulation environments offer different performance, model detail and
built-in features, all of which may affect the success and the merit of a simulation-based study. In this paper, features and performance of three open-source simula-
tors for robotics, V-REP [ 2], Gazebo [ 3]a n dA R G o S[ 4], are thoroughly analysed
and suggestions about what types of projects they are suitable for are provided.All of the three simulators allow for programming in C++, allowing us to make
reasonable comparisons between their features and performance. At the same
time, the simulators represent clear alternatives when it comes to the trade-off
between simulation complexity and speed.
Other simulators, such as Matlab [ 5,6], Webots [ 5,6], Player/Stage [ 5,7–9],
Gazebo [ 5,7–9], USARSim [ 5,8,9], SARGE [ 8] and TeamBots [ 7], have previously
been described and compared. While some comparisons were rather informal
[6,8], others involved ranking simulators based on specific evaluation criteria.
For example, in [ 9], programming language support, documentation, user inter-
face and debugging techniques of simulators were evaluated. In [ 5], the evaluation
criteria included simulator physical fidelity, functional fidelity, ease of develop-ment and cost.
To the best of our knowledge, no formal comparison between V-REP, ARGoS
and Gazebo has been conducted before. Inspired by [ 5,7,9], it is therefore our
c/circlecopyrtSpringer International Publishing AG, part of Springer Nature 2018
M. Giuliani et al. (Eds.): TAROS 2018, LNAI 10965, pp. 357–368, 2018.https://doi.org/10.1007/978-3-319-96728-8
_30

358 L. Pitonakova et al.
aim to provide a ranked evaluation of the simulator characteristics in Sect. 2.
Moreover, similarly as in [ 9], the simulator performance is evaluated on a set of
benchmark experiments in Sect. 3.
2 Simulator Characteristics
All tests were performed in the 64-bit Ubuntu Linux 16.04 environment running
on a computer with 4x Intel Core i7 2.2 Gz processor, 8 GB RAM and Intel HD
Graphics 6000 graphics card. The built-in capabilities, model library, program-ming methods and user interface of the simulators are compared in Table 1.
Table 1. Simulator characteristics ranked as rich (green), neutral (white) and poor
(red). Ranking was performed by considering characteristics in the same row (i.e., of
the same type) relatively to each other.
V-REP Gazebo ARGoS
Available for MacOS,
Linux and Windows.Binary packages are avail-able for all platforms.Available for MacOS,Linux and Windows.A binary package isonly available for LinuxDebian. Installed viathe command line usingthird-party package man-agers on other systems.Available for MacOS andLinux. Binary packagesare available for Linux.On MacOS, ARGoS is in-stalled via the commandline using a third-partypackage manager.
Built-in capabilities
Default physics engines
include: Bullet 2.78, Bul-
let 2.83, ODE, Vortex and
Newton.Only the ODE physics
engine is available by de-
fault. It is, however, possi-
ble to build Gazebo fromsource with a differentphysics engine.A 2D and a 3D custom-
built physics engines
with very limited capa-
bilities are available bydefault.
Includes a code and a
scene editor .Includes a code and a
scene editor .Includes a Lua script ed-
itor but no scene editor .
Meshes can be manipu-
lated (e.g., cut) by robots
in real time.Nomesh manipulation
is available.No mesh manipula-
tion is available.
Scene objects can be
fully interacted with
(e.g., moved or added) by
the user during simula-
tion. The world returns
to its original state whenthe simulation is reset.Scene objects can be
fully interacted with
(e.g., moved or added)
by the user during sim-
ulation. The world does
not return to its originalstate when the simulationis reset.Scene objects can be
moved by the user dur-
ing simulation.
(continued )

Feature and Performance Comparison 359
Table 1. (continued )
The default models are
very detailed and there-fore appropriate for high-precision simulations. It ispossible to simplify themodels directly in V-REP.The default modelsare fairly simple and aretherefore more appropri-ate for computationallycomplex simulations.The default modelsare fairly simple and aretherefore more appropri-ate for computationallycomplex simulations.
Meshes are imported
as collections of sub-components. It is there-fore possible manipulate
individual parts of an
imported model and to
change their textures,materials and otherproperties.Meshes are imported
as single objects. Modelsthat contain multiple sub-components have to be as-
sembled in Gazebo from
multiple DAE files, each
corresponding to one sub-component.Mesh importing is
not available. Objectrepresentations areprogrammed using
OpenGL.
It is possible to simplify,
split and combine
meshes . This makes
it possible to optimise
the triangle count of
imported models and tomanipulate meshes (e.g.,to cut them) with robot
actuators.Imported meshes can-
not be changed .A
model therefore has to
be optimised in third-
party 3D modeling soft-
ware. This may be diffi-cult during iterative de-velopment.N/A
Programming methods
A scene is saved in
a special V-REP format.All scene editing therefore
has to be done using the
V-REP interface.A scene is saved as an
XML file. This makes itpossible to e.g. create a
bash script that changes
the scene and then runs a
simulation.A scene is saved as an
XML file. This makes itpossible to e.g. create a
bash script that changes
the scene and then runs
a simulation.Outputs include video,
custom data plots andtext files.Outputs include simula-
tion log files, video framesas pictures and text files.Outputs include video
frames as pictures andtext files.
Includes particle systems. No particle systems areavailable. No particle systems areavailable.
Robot and other models
Provides a large varietyofrobots , including bi-
pedal, hexapod, wheeled,
flying and snake-likerobots. Also provides alarge number of robot
actuators and sensors.A less diverse library
of default robots , that
mostly includes wheeled
and flying robots. Third-party robot models areavailable, but their docu-
mentation is often poor.A fairly small library
ofrobots , only includ-
ing the e-puck [11], eye-
bot [13], Kilobot [17],marXbot [12] and Spiri[14] robots.
(continued )

360 L. Pitonakova et al.
Table 1. (continued )
“CustomUI” API, based
on QT[15], is used to cre-ate custom interfaces .
Custom UI controllers canbe attached to individ-ual robots. For example,it is possible to displaya robot’s camera outputwhen the robot is clickedon.Custom interfaces can
be created as plug-ins byusing the default QT API[15]. However, the inter-faces can only be attachedto the whole scene and notto individual robots.Custom interfaces can
be created in C++ bysub-classing an ARGoSAPI class. The interfacescan be attached to thewhole scene or to individ-ual robots.
All scripts and plug-ins
provided with the de-fault robot models and
example scenes worked.Many plug-ins pro-
vided with the defaultrobot models did
not work. Some on-lineexamples were difficultto run due to a largenumber of dependenciesand differences in ROSversions.A few examples are
provided on the AR-
GoS website, all of whichworked.
Good API documenta-
tion , a large library of tu-
torials and code examplesand a large user commu-nity are available. Regularupdates have been pro-
vided since 2013.A fairly comprehensivedocumentation ,s t e p –
by-step tutorials and alarge user communityare available. Gazebo islikely to be supported
in the future, since adevelopment road map isavailable on the website.Good documentation ,
but a small user commu-nity are available. Devel-opment has not been reg-ular.
User interface (UI)There are various optionsforprogramming func-
tionality, including scriptsattached to robots, plug-ins, ROS nodes [10] orseparate programs thatconnect to V-REP via theRemoteAPI.Functionality can be pro-
grammed either as com-
piled C++ plug-ins or asROS [10] programs. Lackof scripting makes it dif-ficult to run quick testswith ad-hoc solutions.Robots can be pro-
grammed either
through Lua scripts orin C++.
Scripts can be included inrobot models and are of-ten used to describe the
models and their capa-
bilities.It is difficult to recognisehow a third-party robotmodel works or what
plug-ins it uses. The plug-in list is only available inthe Model Editor.Some documentation
of the robots is providedin ARGoS, but most ofhow a robot works needsto be deducted fromcode examples.
(continued )

Feature and Performance Comparison 361
Table 1. (continued )
The model library is
distributed with V-REP
and it is thus always avail-able regardless of Internetconnectivity.The model library
is not distributed with
Gazebo, and it is insteadavailable on-line. Onmultiple occasions, thelibrary could not be
accessed because Gazebo
could not connect to its
server, even though the
computer was connected
to the Internet.The robot models are
distributed with ARGoS
and it is thus alwaysavailable regardless of In-ternet connectivity.
The model library is
organised into folders
based on model category.
User can create their
own folders for their own
models.The model library is
a long list of models and
particular model types
(e.g., robots) can be dif-
ficult to find.Different robot types
are natively a part of AR-
GoS and can be found
by querying command-
line documentation.No freezing issues with
the interface were experi-enced.The interface froze a
number of times and theprogram, and sometimesthe computer, had to berestarted. This occurred,e.g., when editing robotmodels, starting or stop-ping the simulation, andin other instances.No freezing issueswith the interface wereexperienced.
All functionality is
fairly intuitive and
follows general conven-
tions known from similarapplications.The UI usability is rel-
atively low. For exam-
ple, the top application
tool bar sometimes disap-pears, it is not possible
to copy and paste multi-
ple objects, or to save ascene into the same file af-
ter making changes to it.The UI is very limited,
but all functionality is
fairly intuitive and fol-
lows general conventionsknown from similar ap-
plications.
V-REP offers the largest repertoire of features including, most notably, a
scene editor, 3D model importing, mesh manipulation, video recording and anAPI for remotely connecting to a simulation. Also, its model library is relatively
large and well-documented. Gazebo also offers a scene editor and 3D model
importing, however, no mesh editing is available and the imported models needto be optimised in third-party software. Gazebo relies on ROS [ 10] when it
comes to remote connectivity. Notably, Gazebo crashed a number of times on our
computer system and its interface was generally slow during testing (see details
in Table 1). Furthermore, and again only tested on our computer system, some of
its example code could not be compiled or did not run properly. Finally, ARGoS

362 L. Pitonakova et al.
offers the smallest amount of features compared to the other two simulators.
It has no scene editor and 3D models cannot be imported into it. Also, bycomparison, its robot library and documentation are limited.
One advantage of Gazebo and ARGoS over V-REP is the ability to define a
scene in a XML file. This is convenient, for example, when multiple experimentswith varying parameter values need to be generated and run automatically. On
the other hand, a V-REP experiment can only be specified in a V-REP scene file
via the V-REP graphical interface. It is therefore difficult to vary experimentalparameters, especially when running V-REP from the command line. While V-
REP offers up to nine optional command-line arguments that can be supplied to
a simulation, a more involved parameter specification would have to be handled,for example, by a plug-in that could parse parameter text files. Such a plug-in
is currently not distributed with V-REP.
3 Simulator Performance
3.1 Methods
There were two types of benchmark test performed with each simulator:
–The GUI benchmark involved running a simulation of robots that moved
in a straight line and avoided obstacles in real-time. The simulators were run
along with their user interfaces. Each simulation took one minute.
–The headless benchmark involved running the same simulation as in the
GUI benchmark, that lasted five minutes. The simulators were run from the
command line without their user interfaces.
There were two types of simulation environment:
–Small scene , where robots were put on a large 2D plane.
–Large scene , where an industrial building model with approximately 41,6000
vertices was imported into the simulator (Fig. 1a). Since it was not possible to
import the model into ARGoS, 5,200 boxes, corresponding to 41,6000 vertices,
were randomly placed in the environment (Fig. 1b).
Robot models were selected from a library of models available in each sim-
ulator, so that a sensible similarity, in terms of robot geometry and controller
capabilities, was achieved (Fig. 2). In ARGoS, a flying instead of a wheeled robot
was used, since all wheeled robots in ARGoS are handled by a relatively simple
2D physics engine. In V-REP, the original e-puck model was simplified in order
to decrease its vertex count. Each benchmark test was run with 1, 5, 10 and 50robots in both environments.
At the beginning of each benchmark experiment, a simulator was restarted.
The computer was not running any other applications, apart from those normallyrequired by its operating system, and a simple CPU and memory monitoring
application. V-REP was running in “threaded rendering” mode during the GUI
benchmarks.

Feature and Performance Comparison 363
Fig. 1. The “Large scene” environment in (a) V-REP and Gazebo, and (b) ARGoS.
Fig. 2. (a) The simplified (left) and the original (right) e-puck [ 11]m o d e l si nV – R E P .
(b) The Pioneer 3AT [ 16] model in Gazebo. (c) The eye-bot [ 13] model in ARGoS.
The simulation in both V-REP and ARGoS was updated 10 times per sec-
ond, i.e., dt= 100 ms. In Gazebo, 10 times more simulation steps per second
were needed ( dt= 10 ms), otherwise some robots exhibited strange movement
dynamics, such as rocking behaviour on a straight horizontal plane.
Three performance metrics were used to evaluate the simulator performance.
Real-time factor ,R= simulated time/real time, the amount of CPU usage ,C,
and the amount of memory usage ,M. When R>1, a simulation could run
faster than real time and vice versa. When C>100%, a simulator could utilise
multiple processor cores. Two values for Cand Mwere noted for Gazebo GUI
experiments, corresponding to the usage of gzclient (visualisation) and gzserver
(simulation), respectively.
3.2 Results
ARGoS achieved the highest simulation speed in the GUI experiments with up
to 50 robots in the Small scene and with up to 5 robots in the Large scene,
while utilising the smallest amount of resources (Table 2). Gazebo outperformed
ARGoS in other experiments, especially when the Large scene was used in the
Headless benchmark (Tables 2and3). However, Gazebo usually required the

364 L. Pitonakova et al.
largest amount of memory when it was running in the GUI mode, and a median
amount in the Headless mode. V-REP combined with ODE usually achievedthe lowest simulation speed. Using Bullet 2.78 often significantly increased the
performance of V-REP.
Table 2. Simulator performance in the GUI mode. The best (green) and the worst
(red) performance are indicated.
V-REP +
Bullet 2.78V-REP +ODE Gazebo +ODE ARGoS +Point-Mass3D
1 robot +Small scene R≥1
C= 180%
M= 235 MBR≥1
C= 190%
M=2 2 5M BR≥1
C=1 0 0+9 %
M=2 2 5 + 5 8M BR≥1
C=7 %
M=8 5M B
5 robots +
Small sceneR=0.52
C= 395%
M= 380 MBR=0.37
C= 395%
M=3 6 0M BR≥1
C=1 0 0+1 9 %
M=3 0 5 + 5 8M BR≥1
C=1 0 %
M=8 8M B
10 robots +
Small sceneR=0.11
C= 400%
M= 536 MBR=0.099
C= 400%
M=5 3 0M BR≥1
C=1 0 0+3 0 %
M=4 0 2 + 5 8M BR≥1
C=1 3 %
M=8 9M B
50 robots +
Small sceneNot feasible Not feasible R=0.87
C= 100 + 105%
M= 1410 + 358
MBR=0.9
C= 103%
M=9 3M B
1 robot +
Large sceneR=0.96
C= 205%
M= 235 MBR=0.53
C= 200%
M=2 2 5M BR≥1
C=1 0 0+1 0 %
M=2 6 4 + 5 8M BR≥1
C=3 2 %
M=9 0M B
5 robots +
Large sceneR=0.18
C= 400%
M= 325 MBR=0.1
C= 400%
M=3 1 0M BR≥1
C=1 0 0+2 5 %
M=3 3 3 + 5 8M BR≥1
C=6 0 %
M=9 7M B
10 robots +
Large sceneR=0.052
C= 400%
M= 433 MBR=0.036
C= 400%
M=4 6 0M BR≥1
C=1 0 0+4 0 %
M=4 2 5 + 5 8M BR=0.86
C= 120%
M=9 7M B
50 robots +
Large sceneNot feasible Not feasible R=0.57
C= 100 + 100%
M= 1450 + 426
MBR=0.052
C= 107%
M= 106 MB

Feature and Performance Comparison 365
In general, R≥1 could be achieved by all simulators until all available CPU
power was used. The cut-off point, in terms of the number of robots, was alwaysthe lowest for V-REP, i.e. the simulation in V-REP slowed down in smaller
experiments, compared to the other simulators. Furthermore, the simulation
speed decrease due to insufficient CPU power was generally less severe for Gazebothan for V-REP and ARGoS.
Running Gazebo and ARGoS in the Headless mode (Table 3) increased Rin
environments where maximum CPU power was utilised by the GUI mode. Onthe other hand, Rwas often smaller in the Headless mode of V-REP, compared
Table 3. Simulator performance in the Headless mode. The best (green) and the worst
(red) performance are indicated.
V-REP +
Bullet 2.78V-REP +ODE Gazebo +ODE ARGoS +Point-Mass3D
1 robot +Small scene R=4.1
C= 200%
M= 165 MBR=3.12
C= 200%
M=1 6 0M BR=4 2.85
C=1 0 0 %
M=1 0 7M BR=3 0 0
C=6.3%
M=1 8M B
5 robots +
Small sceneR=0.38
C= 400%
M= 320 MBR=0.32
C= 400%
M=3 2 0M BR=1 0
C=1 0 0 %
M=1 3 0M BR=1 5 0
C= 100%
M=2 0M B
10 robots +
Small sceneR=0.09
C= 400%
M= 470 MBR=0.08
C= 400%
M=4 8 0M BR=5.26
C= 100%
M=1 5 0M BR=2 1.42
C= 144%
M=2 0M B
50 robots +
Small sceneNot feasible Not feasible R=1.06
C= 100%
M=3 5 6M BR=0.52
C= 103%
M=2 5M B
1 robot +
Large sceneR=1.91
C= 200%
M= 165 MBR=0.58
C= 200%
M=1 6 0M BR=1 8.75
C=1 0 0 %
M=1 7 4M BR=1 5.78
C= 139%
M=3 1M B
5 robots +
Large sceneR=0.2
C= 400%
M= 270 MBR=0.11
C= 400%
M=2 5 0M BR=5.88
C= 100%
M=1 9 2M BR=5.45
C= 157%
M=4 5M B
10 robots +
Large sceneNot feasible Not feasible R=3.09
C= 100%
M=2 1 1M BR=1.59
C= 130%
M=4 7M B
50 robots +
Large sceneNot feasible Not feasible R=0.60
C= 100%
M=4 2 3M BR=0.03
C= 105%
M=5 5M B

366 L. Pitonakova et al.
to its GUI mode. There were two factors that contributed to this result. Firstly,
Rof the Headless mode was calculated over a longer period of time (simulated
five rather than one minute) and its average, rather than its maximum, value
was reported. Secondly, Rof the Headless mode took into account not only
the actual simulation time, but also the time needed to initialise and close thesimulation. This in some cases took a significant amount of time, especially when
ODE physics engine was used with V-REP.
Finally, V-REP demonstrated the most optimal CPU utilisation. It automat-
ically spawned new threads when it was necessary and it could thus fully utilise
all available CPU cores. Gazebo only utilised a single CPU core per process. In
the GUI mode, Gazebo ran two processes, gzclient and gzserver, that could eachutilise a maximum of 100% of CPU power. In the Headless mode, only a single
core was utilised, as only the gzserver process was running. The multi-threaded
core utilisation by ARGoS worked in general but problems were experienced in
larger experiments. The CPU usage was notably smaller when more robots were
added to the environment. Furthermore, unlike V-REP, ARGoS requires the userto specify the desired number of threads, rather than automatically spawning
new threads when it is necessary.
It is notable that the 3D models used in ARGoS and Gazebo were fairly sim-
ple compared to those used in V-REP, even though effort was made to simplify
the V-REP robot model (Fig. 2). Moreover, the ARGoS physics engine was much
simpler than those used by V-REP and Gazebo. It is therefore expected thatusing third-party libraries to cope with various aspects of the simulation that
are currently not covered in ARGoS, such as calculating more complex physics
dynamics, or working with imported 3D meshes, would decrease the simula-tor’s performance. Similarly, it is expected that more complex 3D models would
decrease the performance of Gazebo compared to V-REP.
In order to confirm that the mesh complexity played a major role in V-REP,
we ran experiments with the Kilobot robots [ 17], that could only move forward
and did not have the sensing or controller capabilities of e-pucks, but consisted
of very simple 3D meshes. In the Kilobot experiments, up to six times higher R
could be achieved, using only about an eighth of the computer’s resources. In
another experiment, the building model in the Large scene was simplified fromaround 41,600 to around 2,000 vertices. This increased Rby up to 66%. Another
kind of optimisation involved decreasing the number of simulation update loops
per second from ten to two (i.e., dt= 500 ms). This increased Rby up to 150%,
while decreasing memory usage by up to 15%. These results suggest that it is
possible to significantly increase the performance of V-REP by carefully setting
simulation parameters and by optimising 3D models used in the simulation.
4 Conclusion
V-REP is the most complex and the most resource-hungry of the three simu-lators. However, it offers a number of useful features, such as multiple physics
engines, a comprehensive model library, the ability of a user to interact with
the world during simulation and, most importantly, mesh manipulation and

Feature and Performance Comparison 367
optimisation. Moreover, V-REP automatically spawns new threads on multi-
ple CPU cores and therefore utilises the full amount of CPU power when it isnecessary. It is therefore suitable for high-precision modelling of robotic appli-
cations such as object transportation or area surveillance, as well as of various
industrial applications, where only a few robots are required to operate at thesame time.
ARGoS, on the other hand, is a suitable choice for simulations of swarm
robotics tasks, such as collective foraging, flocking, or area coverage. Comparedto V-REP, ARGoS trades-off robot, environment and physics complexity for
superior performance. An XML-based simulation settings file is also very conve-
nient, especially when a large variety of simulations need to be generated auto-matically. However, there are multiple important features missing from ARGoS,
most notably the ability to import 3D meshes into the simulator. Currently,
users that are not willing to spend time and effort on programming new robot
models in OpenGL, have fairly limited choices.
Gazebo occupies the space between V-REP and ARGoS. While it is much
closer to V-REP in terms of features, its interface and default robot models are
much simpler and resemble those found in ARGoS. It is notable that Gazebo
outperformed ARGoS in the larger simulation environments studied here, whichsuggests that it is a more suitable choice for large swarm robotics experiments.
However, our experiments showed that the usability of Gazebo is relatively poor.
Firstly, while it can import 3D meshes, there are no editing options, making itdifficult to alter and optimise models. Another problem is the interface that has
a number of issues and fails to follow established conventions. Finally, difficulties
were noted when installing dependencies for Gazebo and for many of its third-party models. While not necessarily severe by themselves, these issues together
could have a negative impact on a research project.
In the future, it would be interesting to conduct a user survey regarding the
features and simulator usability in order to obtain a more robust comparison. It
would also be interesting to evaluate the extent of the reality gap in the default
simulator robot models.
Acknowledgements. This work was supported by EPSRC Programme Grant
“Robotics for Nuclear Environments”, grant ref: EP/P01366X/1. We would like to
thank Farshad Arvin, Tareq Assaf, Giovanni Beltrame, Paul Bremner, Ales Leonardis,
Carlo Pinciroli, Chie Takahashi, Simon Watson and Craig West for sharing their opin-ions and insights into various aspects of simulation work. We would also like to thank
Xavier Poteau, who created the sample industrial building CAD model.

368 L. Pitonakova et al.
References
1. Andrews, P.S., Stepney, S., Timmis, J.: Simulation as a scientific instrument. In:
Stepney, S., Andrews, P.S., Read, M.N. (eds.) Proceedings of the 2012 Workshop
on Complex Systems Modelling and Simulation, pp. 1–10. Luniver Press (2012)
2. Rohmer, E., Singh, S.P.N., Freese, M.: V-REP: a versatile and scalable robot simu-
lation framework. In: Proceedings of the 2013 IEEE/RSJ International Conference
on Intelligent Robots and Systems (IROS 2013). IEEE Press, Piscataway (2013).https://doi.org/10.1109/IROS.2013.6696520
3. Koenig, N., Howard, A.: Design and use paradigms for Gazebo, an open-source
multi-robot simulator. In: Proceedings of the 2004 IEEE/RSJ International Con-ference on Intelligent Robots and Systems (IROS 2004), pp. 2149–2154. IEEE
Press, Piscataway (2004)
4. Pinciroli, C., Trianni, V., et al.: ARGoS: a modular, parallel, multi-engine simulator
for multi-robot systems. Swarm Intell. 6(4), 271–295 (2012)
5. Craighead, J., Murphy, R., Burke, J., Goldiez, B.: A survey of commercial & open
source unmanned vehicle simulators. In: Proceedings of the 2007 IEEE Interna-
tional Conference on Robotics and Automation (ICRA 2007), pp. 852–857. IEEE,
Piscataway (2007)
6.ˇZlajpah, L.: Simulation in robotics. Math. Comput. Simul. 79, 879–897 (2008)
7. Kramer, J., Scheutz, M.: Development environments for autonomous mobile robots:
a survey. Autonom. Robot. 22(2), 101–132 (2007)
8. Harris, A., Conrad, J.M.: Survey of popular robotics simulators, frameworks, and
toolkits. In: Proceedings of the 2011 IEEE Southeastcon, pp. 243–249 (2011)
9. Staranowicz, A., Mariottini, G.L.: A survey and comparison of commercial and
open-source robotic simulator software. In: Proceedings of the 4th International
Conference on PErvasive Technologies Related to Assistive Environments (PETRA
2011). ACM, New York (2011). https://doi.org/10.1145/2141622.2141689
10. Quigley, M., Gerkey, B., et al.: ROS: an open-source robot operating system. In:
Proceedings of the ICRA 2009 Workshop on Open Source Software (2009)
11. Mondada, F., Bonani, M., et al.: The e-puck, a robot designed for education in engi-
neering. In: Goncalves, P.J.S., Torres, P.J.D., Alves, C.M.O. (eds.) Proceedings of
the 9th Conference on Autonomous Robot Systems and Competitions (ROBOT-
ICA 2009), vol. 1, no. 1, pp. 59–65 (2009)
12. Bonani, M., Longchamp, V., et al.: The MarXbot, a miniature mobile robot open-
ing new perspectives for the collective-robotic research. In: Proceedings of the
2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS2010), pp. 4187–4193. IEEE Press, Piscataway (2010)
13. Ducatelle, F., Di Caro, G.A., et al.: Self-organized cooperation between robotic
swarms. Swarm Intell. 5(2), 73–96 (2011)
14. Spiri Specifications. http://pleiadesrobotics.com . Accessed 18 Apr 2018
15. QT. https://www.qt.io . Accessed 18 Apr 2018
16. Pioneer 3AT Specifications. http://bit.ly/2D2VfSR . Accessed 18 Apr 2018
17. Rubenstein, M., Ahler, C., Nagpal, R.: Kilobot: a low cost scalable robot system for
collective behaviors. In: Proceedings of the 2012 IEEE International Conference onRobotics and Automation (ICRA 2012), pp. 3293–3298. Computer Society Press
of the IEEE, Washington, D.C. (2012)