A Survey of Computer Vision Methods for 2D Object Detection from Unmanned Aerial Vehicles

in Journal of Imaging (2020), 6(8), 78

The spread of Unmanned Aerial Vehicles (UAVs) in the last decade revolutionized many applications fields. Most investigated research topics focus on increasing autonomy during operational campaigns ... [more ▼]

The spread of Unmanned Aerial Vehicles (UAVs) in the last decade revolutionized many applications fields. Most investigated research topics focus on increasing autonomy during operational campaigns, environmental monitoring, surveillance, maps, and labeling. To achieve such complex goals, a high-level module is exploited to build semantic knowledge leveraging the outputs of the low-level module that takes data acquired from multiple sensors and extracts information concerning what is sensed. All in all, the detection of the objects is undoubtedly the most important low-level task, and the most employed sensors to accomplish it are by far RGB cameras due to costs, dimensions, and the wide literature on RGB-based object detection. This survey presents recent advancements in 2D object detection for the case of UAVs, focusing on the differences, strategies, and trade-offs between the generic problem of object detection, and the adaptation of such solutions for operations of the UAV. Moreover, a new taxonomy that considers different heights intervals and driven by the methodological approaches introduced by the works in the state of the art instead of hardware, physical and/or technological constraints is proposed. [less ▲]

A Real-Time Approach for Chance-Constrained Motion Planning with Dynamic Obstacles

in IEEE Robotics and Automation Letters (2020), 5(2), 3620-3625

Uncertain dynamic obstacles, such as pedestrians or vehicles, pose a major challenge for optimal robot navigation with safety guarantees. Previous work on motion planning has followed two main strategies ... [more ▼]

Uncertain dynamic obstacles, such as pedestrians or vehicles, pose a major challenge for optimal robot navigation with safety guarantees. Previous work on motion planning has followed two main strategies to provide a safe bound on an obstacle's space: a polyhedron, such as a cuboid, or a nonlinear differentiable surface, such as an ellipsoid. The former approach relies on disjunctive programming, which has a relatively high computational cost that grows exponentially with the number of obstacles. The latter approach needs to be linearized locally to find a tractable evaluation of the chance constraints, which dramatically reduces the remaining free space and leads to over-conservative trajectories or even unfeasibility. In this work, we present a hybrid approach that eludes the pitfalls of both strategies while maintaining the original safety guarantees. The key idea consists in obtaining a safe differentiable approximation for the disjunctive chance constraints bounding the obstacles. The resulting nonlinear optimization problem is free of chance constraint linearization and disjunctive programming, and therefore, it can be efficiently solved to meet fast real-time requirements with multiple obstacles. We validate our approach through mathematical proof, simulation and real experiments with an aerial robot using nonlinear model predictive control to avoid pedestrians. [less ▲]

Real-Time Human Head Imitation for Humanoid Robots

in Proceedings of the 2019 3rd International Conference on Artificial Intelligence and Virtual Reality (2019, July)

The ability of the robots to imitate human movements has been an active research study since the dawn of the robotics. Obtaining a realistic imitation is essential in terms of perceived quality in human ... [more ▼]

The ability of the robots to imitate human movements has been an active research study since the dawn of the robotics. Obtaining a realistic imitation is essential in terms of perceived quality in human-robot interaction, but it is still a challenge due to the lack of effective mapping between human movements and the degrees of freedom of robotics systems. If high-level programming interfaces, software and simulation tools simplified robot programming, there is still a strong gap between robot control and natural user interfaces. In this paper, a system to reproduce on a robot the head movements of a user in the field of view of a consumer camera is presented. The system recognizes the presence of a user and its head pose in real-time by using a deep neural network, in order to extract head position angles and to command the robot head movements consequently, obtaining a realistic imitation. At the same time, the system represents a natural user interface to control the Aldebaran NAO and Pepper humanoid robots with the head movements, with applications in human-robot interaction. [less ▲]

A Real-Time 3D Path Planning Solution for Collision-Free Navigation of Multirotor Aerial Robots in Dynamic Environments

in Journal of Intelligent and Robotic Systems (2019), 93(1-2), 33-53

Deliberative capabilities are essential for intelligent aerial robotic applications in modern life such as package delivery and surveillance. This paper presents a real-time 3D path planning solution for ... [more ▼]

Deliberative capabilities are essential for intelligent aerial robotic applications in modern life such as package delivery and surveillance. This paper presents a real-time 3D path planning solution for multirotor aerial robots to obtain a feasible, optimal and collision-free path in complex dynamic environments. High-level geometric primitives are employed to compactly represent the situation, which includes self-situation of the robot and situation of the obstacles in the environment. A probabilistic graph is utilized to sample the admissible space without taking into account the existing obstacles. Whenever a planning query is received, the generated probabilistic graph is then explored by an A$^{\star}$ discrete search algorithm with an artificial field map as cost function in order to obtain a raw optimal collision-free path, which is subsequently shortened. Realistic simulations in V-REP simulator have been created to validate the proposed path planning solution, integrating it into a fully autonomous multirotor aerial robotic system. [less ▲]

Towards trajectory planning from a given path for multirotor aerial robots trajectory tracking

in 2018 International Conference on Unmanned Aircraft Systems (ICUAS), Dallas 12-15 June 2018 (2018, June)

Planning feasible trajectories given desired collision-free paths is an essential capability of multirotor aerial robots that enables the trajectory tracking task, in contrast to path following. This ... [more ▼]

Planning feasible trajectories given desired collision-free paths is an essential capability of multirotor aerial robots that enables the trajectory tracking task, in contrast to path following. This paper presents a trajectory planner for multirotor aerial robots carefully designed considering the requirements of real applications such as aerial inspection or package delivery, unlike other research works that focus on aggressive maneuvering. Our planned trajectory is formed by a set of polynomials of two kinds, acceleration/deceleration and constant velocity. The trajectory planning is carried out by means of an optimization that minimizes the trajectory tracking time, applying some typical constraints as m-continuity or limits on velocity, acceleration and jerk, but also the maximum distance between the trajectory and the given path. Our trajectory planner has been tested in real flights with a big and heavy aerial platform such the one that would be used in a real operation. Our experiments demonstrate that the proposed trajectory planner is suitable for real applications and it is positively influencing the controller for the trajectory tracking task. [less ▲]

A Multi-Layered Component-Based Approach for the Development of Aerial Robotic Systems: The Aerostack Framework

in Journal of Intelligent and Robotic Systems (2017), 88(2), 638-709

To achieve fully autonomous operation for Unmanned Aerial Systems (UAS) it is necessary to integrate multiple and heterogeneous technical solutions (e.g., control-based methods, computer vision methods ... [more ▼]

To achieve fully autonomous operation for Unmanned Aerial Systems (UAS) it is necessary to integrate multiple and heterogeneous technical solutions (e.g., control-based methods, computer vision methods, automated planning, coordination algorithms, etc.). The combination of such methods in an operational system is a technical challenge that requires efficient architectural solutions. In a robotic engineering context, where productivity is important, it is also important to minimize the effort for the development of new systems. As a response to these needs, this paper presents Aerostack, an open-source software framework for the development of aerial robotic systems. This framework facilitates the creation of UAS by providing a set of reusable components specialized in functional tasks of aerial robotics (trajectory planning, self localization, etc.) together with an integration method in a multi-layered cognitive architecture based on five layers: reactive, executive, deliberative, reflective and social. Compared to other software frameworks for UAS, Aerostack can provide higher degrees of autonomy and it is more versatile to be applied to different types of hardware (aerial platforms and sensors) and different types of missions (e.g. multi robot swarm systems). Aerostack has been validated during four years (since February 2013) by its successful use on many research projects, international competitions and public exhibitions. As a representative example of system development, this paper also presents how Aerostack was used to develop a system for a (fictional) fully autonomous indoors search and rescue mission. [less ▲]

Human-Robot Cooperation in Surface Inspection Aerial Missions

Scientific Conference (2017, September 21)

The goal of the work presented in this paper is to facilitate the cooperation between human opera- tors and aerial robots to perform surface inspec- tion missions. Our approach is based on a model of ... [more ▼]

The goal of the work presented in this paper is to facilitate the cooperation between human opera- tors and aerial robots to perform surface inspec- tion missions. Our approach is based on a model of human collaborative control with a mixed ini- tiative interaction. In the paper, we present our human-robot cooperation model based on the combination of a supervisory mode and an as- sistance mode with a set of interaction patterns. We developed a software system implementing this interaction model and carried out several real flight experiments that proved that this ap- proach can be used in aerial robotics for sur- face inspection missions (e.g., in vision based indoor missions). Compared to a conventional tele-operated inspection system, the solution pre- sented in this paper gives more autonomy to the aerial systems, reducing the cognitive load of the operator during the mission development. [less ▲]

A flight altitude estimator for multirotor UAVs in dynamic and unstructured indoor environments

in 2017 International Conference on Unmanned Aircraft Systems (ICUAS) (2017, June)

A robust real-time path planner for the collision-free navigation of multirotor aerial robots in dynamic environments

in 2017 International Conference on Unmanned Aircraft Systems (ICUAS) (2017, June)

A reliable open-source system architecture for the fast designing and prototyping of autonomous multi-uav systems: Simulation and experimentation

in Journal of Intelligent and Robotic Systems (2016), 84(1-4), 779-797

During the process of design and development of an autonomous Multi-UAV System, two main problems appear. The first one is the difficulty of designing all the modules and behaviors of the aerial multi ... [more ▼]

During the process of design and development of an autonomous Multi-UAV System, two main problems appear. The first one is the difficulty of designing all the modules and behaviors of the aerial multi-robot system. The second one is the difficulty of having an autonomous prototype of the system for the developers that allows to test the performance of each module even in an early stage of the project. These two problems motivate this paper. A multipurpose system architecture for autonomous multi-UAV platforms is presented. This versatile system architecture can be used by the system designers as a template when developing their own systems. The proposed system architecture is general enough to be used in a wide range of applications, as demonstrated in the paper. This system architecture aims to be a reference for all designers. Additionally, to allow for the fast prototyping of autonomous multi-aerial systems, an Open Source framework based on the previously defined system architecture is introduced. It allows developers to have a flight proven multi-aerial system ready to use, so that they can test their algorithms even in an early stage of the project. The implementation of this framework, introduced in the paper with the name of ``CVG Quadrotor Swarm'', which has also the advantages of being modular and compatible with different aerial platforms, can be found at \url{https://github.com/Vision4UAV/cvg_quadrotor_swarm} with a consistent catalog of available modules. The good performance of this framework is demonstrated in the paper by choosing a basic instance of it and carrying out simulation and experimental tests whose results are summarized and discussed in this paper. [less ▲]