RAIDER

In order for theoretical algorithms to reach a state of deployment, an acceleration and optimization process must convert a computationally intensive technique into a real-time system.  The UD Vision Lab has provided the platform and sensors required for a real-time prototype system.  We will refer to this system as the Robust Artificial Intelligence-based Defense Electro Robot or simply RAIDER.  The RAIDER research can be categorized into the following five major groups:

1. Sensor Integration and Robot Design
2. Obstacle Avoidance and Vision-Based Navigation for Robotic Navigation
3. Autonomous Facial Recognition for Robot Based Surveillance System
4. Human Re-identification from Shape and Clothing by Security Robot  
5. Human Behavior and Activity Classification for Robot as a Watchdog 

1. Sensor Integration and Robot Design

 Objective: To design, integrate, and synchronize the robot mechanics with a variety of sensors

With more than 50 components and sensors on the RAIDER, this multidisciplinary effort calls for an integration of all the sensors with the algorithm processing unit in an optimal way.   The electrical aspect of this project focuses on efficiently designing the communication interface amongst onboard sensors.  The major challenges includes efficient utilization of the bandwidth as well as management of the energy source consumption.  The mechanical aspect of the design deals with the kinematics and stabilization of the RAIDER.   Along with the installation of sensors at optimal location, mechanical engineering students ensure that the weight distribution allow for smooth and energy efficient navigation.  Figure 1 illustrates some of the key components and system modules within the RAIDER.

2. Obstacle Avoidance and Vision-Based Navigation for Robotic Navigation

 Objective: To understand depths of the surrounding environment and to maneuver in between stationary and moving obstacles.

The focus of this project is for RAIDER to autonomously navigate within a complex environment.  The fundamentals of navigation through an unknown space are understanding the locations and depths of surrounding objects as well as correctly determining a collision-free path to take. A 3D reconstruction algorithm enable the RAIDER to see the surrounding environment through a stereo vision system.  Similarly to the eyes of a human, the RAIDER utilizes two vintage points to perform triangulation and depth analysis to create a 3D model of the environment.  Preliminary results showing the reconstruction of lobby elevators in Kettering Laboratories is shown in Figure 2. 

Additionally, the RAIDER’s navigation controls system must be robust to obstacles. As shown in Figure 3, the RAIDER must be able to follow a particular person within a crowd or if they were to turn the corner. An algorithm that predicts the movement of the person of interest through a complex environment will utilize predictive filtering in 3D space as well as optimal path planning.  

3. Autonomous Facial Recognition for Robot Based Surveillance System

 Objective: To develop and implement real-time facial identification on a mobile platform

In order for the RAIDER to autonomously interact with its surrounding, an automated face recognition system must be developed.  In this project, the robot would differentiate between people through detection, characterization and recognition of facial features.  A team of students will design and tune a computer vision algorithm that recognizes multiple individuals irrespective of the viewing angle or the lighting conditions.  As shown in Figure 4, pose-invariant face detection is an ongoing research problem due to the infinite possible poses of a face within an image.  The UD Vision Lab has been developing pose invariant face detection algorithms that will need to be accelerated, integrated and tested to be a pivotal part of an autonomous surveillance system.  

4. Human Re-identification from Shape and Clothing by Security Robot

 Objective: To detect and characterize unknown human based on clothing and shape

Humans have an instinctive way for physically describing and remembering other people based on clothing and shape.  This project focuses building and implementing an algorithm that would allow the RAIDER to enter a room full of unknown people and automatically describe and “remember” them.   This algorithm will attempt to recreate the human ability to identify someone without a formal introduction.   An algorithmic textural and shape descriptor is equivalent to describing someone as “male wearing a striped shirt and jeans” or “lady wearing a red dress”.  As shown in Figure 5, the human description algorithm can distinguish between varieties of clothing textures in multiple environmental conditions. 

In order to test a human detection security system, the project lab space must allow for the RAIDER to capture the full human body within its field of view.  The success of the project will be measured by the RAIDER’s ability to re-identify a person previously seen.  Since clothing textures are not uniform and the challenge becomes in recognizing a person irrespective of the viewing angle.   As shown in Figure 6, the descriptor must be robust to recognizing people in multiple positions and orientation.

5. Behavior and Activity Classification for Robot as a Watchdog

 Objective: To track the motion of human limbs and classify the sequences into categories of behavior and activity.

The key to achieve human-machine interfacing is to train the RAIDER to understand human behavior, especially to identify abnormal activities. Human behavior can be classified into distinct categories by tracking features on the human body.  As shown in Figure 7, basic hand action such as “come here”, “go there” or “stop” can be classified by tracking the hand features through space in relation to the rest of the body.   Similarly, tracking body features is also applied to classify more complex actions such as “running” or “bending”.  In the case of classifying complex behaviors, such as “arguing” or “whispering,” the RAIDER will consider additional body features such as facial expressions and tones of voice. A cascade of actions is then defined and classified as an activity. RAIDER will have the ability to detect and identify abnormal behaviors and activities. The project lab space will allow for the RAIDER to capture the full human body within its field of view to analyze and classify human activities.