Skip to main content

Deep Learning Right-of-Way

Deep Learning Based Automated Right-of-Way Surveillance

Rapid advances made in the area of camera and sensor technology has enabled the use of video acquisition systems to monitor the right-of-ways (ROW) of oil and gas pipelines. Huge amount of data is thus made available for analysis. However, it would be very expensive to employ analysts to scan through the data and identify threats to the right-of-ways in the vast amount of wide area imagery. This warranted the deployment of an automated mechanism that is able to detect threats to the right-of-way and send out warnings in the event of detection of a threat. The current research in the area of neural network based deep learning methodologies for object detection, recognition, identification, and tracking has shown significant progress in the performance of computer vision applications. Deep learning is a machine learning method based on learning data characteristics and representations. Deep learning models are inspired by the structural and functional properties of the biological nervous systems. The deep learning based computer vision system adopts the visual perception behavior of the human visual system. We propose to develop a software package for automated threat detection on pipeline right-of-ways by employing the advanced deep learning based artificial intelligence technologies. The main objective of the proposed pipeline right-of-way surveillance system and threat recognition software is to identify, localize, and track potential threats to the pipeline right of ways. This will be achieved via deep learning based artificial intelligence algorithms, which will analyze high-resolution imagery captured by cameras located on small aircrafts flying over the pipeline right-of-ways.

Developing a real-time threat detection solution that will identify and display potential threats to the operator along the right-of-way during the survey operation

The proposed deep learning based artificial intelligence methodology for machinery threat detection on pipeline right-of-ways include several computational units viz. key frame selection, background region elimination, object feature extraction, object detection, object localization, and object tracking. The overall deep learning architecture for the proposed threat object detection system is shown in Figure 1.

ImageFigure 1. Deep learning approach for object detection and classification.

We propose a neural network architecture named as the deep perceptual neural network (DPNN), which is a convolutional neural network optimized for detection and classification. The network is comprised of four different parts: backbone, feature fusion, classification, and regression. These parts of the DPNN work together for extracting relevant features and determining the bounding box and class of the detection.



We have used the Aberdeen dataset collected by Pipeline Research Council International (PRCI) as an initial dataset for testing and evaluation. The dataset has 4380 images with a ground sample distance of ~3 inches. Figure 2 shows some images of the detections made by the DPNN algorithm. It is found that the DPNN works very well in detection and classification of the machinery and vehicles in the dataset and will generalize fairly well in different environments.
ImageFigure 2. Initial detection results of machinery and vehicles on the PRCI Aberdeen dataset

We have also conducted initial evaluations of the DPNN architecture using the vehicle detection in aerial imagery (VEDAI) dataset, which has several vehicle classes and multiple complex scenarios. This dataset is lower resolution than the given PRCI data. Figure 3 shows various detection on the VEDAI dataset using the proposed DPNN architecture. It is found that the DPNN architecture provides accurate detection and classification of vehicles in both simple and complex backgrounds.

ImageFigure 3. Initial detection results of different classes of vehicles on the VEDAI dataset


Vision Lab, Dr. Vijayan Asari, Director

Kettering Laboratories
300 College Park
Dayton, Ohio 45469 - 0232