Vision-Based Target Detection and Localization via a Team of Cooperative UAV and UGVs

Sara Minaeian, Jian Liu, Young-Jun Son

Research output: Contribution to journalArticle

66 Citations (Scopus)

Abstract

Unmanned vehicles (UVs) play a key role in autonomous surveillance scenarios. A major task needed by these UVs in undertaking autonomous patrol missions is to detect the targets and find their locations in real-Time. In this paper, a new vision-based target detection and localization system is presented to make use of different capabilities of UVs as a cooperative team. The scenario considered in this paper is a team of an unmanned aerial vehicle (UAV) and multiple unmanned ground vehicles (UGVs) tracking and controlling crowds on a border area. A customized motion detection algorithm is applied to follow the crowd from the moving camera mounted on the UAV. Due to UAVs lower resolution and broader detection range, UGVs with higher resolution and fidelity are used as the individual human detectors, as well as moving landmarks to localize the detected crowds with unknown independently moving patterns at each time point. The UAVs localization algorithm, proposed in this paper, then converts the crowds' image locations into their real-world positions, using perspective transformation. A rule-of-Thumb localization method by a UGV is also presented, which estimates the geographic locations of the detected individuals. Moreover, an agent-based simulation model is developed for system verification, with different parameters, such as flight altitude, number of landmarks, and landmark assignment method. The performance measure considered in this paper is the average Euclidean distance between the estimated locations and simulated geographic waypoints of the crowd. Experimental results demonstrate the effectiveness of the proposed framework for autonomous surveillance by UVs.

Original languageEnglish (US)
Article number7330001
Pages (from-to)1005-1016
Number of pages12
JournalIEEE Transactions on Systems, Man, and Cybernetics: Systems
Volume46
Issue number7
DOIs
StatePublished - Jul 1 2016

Fingerprint

Unmanned vehicles
Ground vehicles
Unmanned aerial vehicles (UAV)
Target tracking
Cameras
Detectors

Keywords

  • Algorithms
  • automation cooperative systems
  • geographic information systems (GISs)
  • position measurement

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this

Vision-Based Target Detection and Localization via a Team of Cooperative UAV and UGVs. / Minaeian, Sara; Liu, Jian; Son, Young-Jun.

In: IEEE Transactions on Systems, Man, and Cybernetics: Systems, Vol. 46, No. 7, 7330001, 01.07.2016, p. 1005-1016.

Research output: Contribution to journalArticle

@article{6342f74722834bcaa3d32c5f6542628c,
title = "Vision-Based Target Detection and Localization via a Team of Cooperative UAV and UGVs",
abstract = "Unmanned vehicles (UVs) play a key role in autonomous surveillance scenarios. A major task needed by these UVs in undertaking autonomous patrol missions is to detect the targets and find their locations in real-Time. In this paper, a new vision-based target detection and localization system is presented to make use of different capabilities of UVs as a cooperative team. The scenario considered in this paper is a team of an unmanned aerial vehicle (UAV) and multiple unmanned ground vehicles (UGVs) tracking and controlling crowds on a border area. A customized motion detection algorithm is applied to follow the crowd from the moving camera mounted on the UAV. Due to UAVs lower resolution and broader detection range, UGVs with higher resolution and fidelity are used as the individual human detectors, as well as moving landmarks to localize the detected crowds with unknown independently moving patterns at each time point. The UAVs localization algorithm, proposed in this paper, then converts the crowds' image locations into their real-world positions, using perspective transformation. A rule-of-Thumb localization method by a UGV is also presented, which estimates the geographic locations of the detected individuals. Moreover, an agent-based simulation model is developed for system verification, with different parameters, such as flight altitude, number of landmarks, and landmark assignment method. The performance measure considered in this paper is the average Euclidean distance between the estimated locations and simulated geographic waypoints of the crowd. Experimental results demonstrate the effectiveness of the proposed framework for autonomous surveillance by UVs.",
keywords = "Algorithms, automation cooperative systems, geographic information systems (GISs), position measurement",
author = "Sara Minaeian and Jian Liu and Young-Jun Son",
year = "2016",
month = "7",
day = "1",
doi = "10.1109/TSMC.2015.2491878",
language = "English (US)",
volume = "46",
pages = "1005--1016",
journal = "IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans.",
issn = "1083-4427",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "7",

}

TY - JOUR

T1 - Vision-Based Target Detection and Localization via a Team of Cooperative UAV and UGVs

AU - Minaeian, Sara

AU - Liu, Jian

AU - Son, Young-Jun

PY - 2016/7/1

Y1 - 2016/7/1

N2 - Unmanned vehicles (UVs) play a key role in autonomous surveillance scenarios. A major task needed by these UVs in undertaking autonomous patrol missions is to detect the targets and find their locations in real-Time. In this paper, a new vision-based target detection and localization system is presented to make use of different capabilities of UVs as a cooperative team. The scenario considered in this paper is a team of an unmanned aerial vehicle (UAV) and multiple unmanned ground vehicles (UGVs) tracking and controlling crowds on a border area. A customized motion detection algorithm is applied to follow the crowd from the moving camera mounted on the UAV. Due to UAVs lower resolution and broader detection range, UGVs with higher resolution and fidelity are used as the individual human detectors, as well as moving landmarks to localize the detected crowds with unknown independently moving patterns at each time point. The UAVs localization algorithm, proposed in this paper, then converts the crowds' image locations into their real-world positions, using perspective transformation. A rule-of-Thumb localization method by a UGV is also presented, which estimates the geographic locations of the detected individuals. Moreover, an agent-based simulation model is developed for system verification, with different parameters, such as flight altitude, number of landmarks, and landmark assignment method. The performance measure considered in this paper is the average Euclidean distance between the estimated locations and simulated geographic waypoints of the crowd. Experimental results demonstrate the effectiveness of the proposed framework for autonomous surveillance by UVs.

AB - Unmanned vehicles (UVs) play a key role in autonomous surveillance scenarios. A major task needed by these UVs in undertaking autonomous patrol missions is to detect the targets and find their locations in real-Time. In this paper, a new vision-based target detection and localization system is presented to make use of different capabilities of UVs as a cooperative team. The scenario considered in this paper is a team of an unmanned aerial vehicle (UAV) and multiple unmanned ground vehicles (UGVs) tracking and controlling crowds on a border area. A customized motion detection algorithm is applied to follow the crowd from the moving camera mounted on the UAV. Due to UAVs lower resolution and broader detection range, UGVs with higher resolution and fidelity are used as the individual human detectors, as well as moving landmarks to localize the detected crowds with unknown independently moving patterns at each time point. The UAVs localization algorithm, proposed in this paper, then converts the crowds' image locations into their real-world positions, using perspective transformation. A rule-of-Thumb localization method by a UGV is also presented, which estimates the geographic locations of the detected individuals. Moreover, an agent-based simulation model is developed for system verification, with different parameters, such as flight altitude, number of landmarks, and landmark assignment method. The performance measure considered in this paper is the average Euclidean distance between the estimated locations and simulated geographic waypoints of the crowd. Experimental results demonstrate the effectiveness of the proposed framework for autonomous surveillance by UVs.

KW - Algorithms

KW - automation cooperative systems

KW - geographic information systems (GISs)

KW - position measurement

UR - http://www.scopus.com/inward/record.url?scp=84976472942&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84976472942&partnerID=8YFLogxK

U2 - 10.1109/TSMC.2015.2491878

DO - 10.1109/TSMC.2015.2491878

M3 - Article

VL - 46

SP - 1005

EP - 1016

JO - IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans.

JF - IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans.

SN - 1083-4427

IS - 7

M1 - 7330001

ER -