Integrated guidance and control for pinpoint mars landing using reinforcement learning

Brian Gaudet, Richard Linares, Roberto Furfaro

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Future Mars missions will require advanced guidance, navigation, and control algorithms for the powered descent phase in order to target specific surface locations and achieve pinpoint accuracy (landing error ellipse < 5m radius). This requires both a navigation system capable of estimating the lander’s state in real-time and a guidance and control system that can map the estimated lander state to body-frame actuator commands. In this paper we present a novel integrated guidance and control algorithm designed by applying the principles of reinforcement learning theory. The key innovation is the use of reinforcement learning to learn a policy mapping the lander’s estimated state directly to actuator commands, with the policy resulting in accurate and fuel efficient trajectories. Specifically, we use proximal policy optimization, a policy gradient method, to learn the policy. We present simulation results demonstrating the guidance and control system’s performance in a 6-DOF simulation environment, and demonstrate robustness to noise and system parameter uncertainty.

Original languageEnglish (US)
Title of host publicationAAS/AIAA Astrodynamics Specialist Conference, 2018
EditorsBelinda G. Marchand, Ryan M. Weisman, Brandon A. Jones, Puneet Singla
PublisherUnivelt Inc.
Pages3135-3154
Number of pages20
ISBN (Print)9780877036579
StatePublished - Jan 1 2018
EventAAS/AIAA Astrodynamics Specialist Conference, 2018 - Montreal, Canada
Duration: Aug 19 2018Aug 23 2018

Publication series

NameAdvances in the Astronautical Sciences
Volume167
ISSN (Print)0065-3438

Conference

ConferenceAAS/AIAA Astrodynamics Specialist Conference, 2018
CountryCanada
CityMontreal
Period8/19/188/23/18

Fingerprint

Mars landing
Reinforcement learning
reinforcement
Landing
learning
Mars
Actuators
Control systems
Gradient methods
Navigation systems
commands
Navigation
navigation
Innovation
Trajectories
control system
learning theory
actuators
Mars missions
environment simulation

ASJC Scopus subject areas

  • Aerospace Engineering
  • Space and Planetary Science

Cite this

Gaudet, B., Linares, R., & Furfaro, R. (2018). Integrated guidance and control for pinpoint mars landing using reinforcement learning. In B. G. Marchand, R. M. Weisman, B. A. Jones, & P. Singla (Eds.), AAS/AIAA Astrodynamics Specialist Conference, 2018 (pp. 3135-3154). (Advances in the Astronautical Sciences; Vol. 167). Univelt Inc..

Integrated guidance and control for pinpoint mars landing using reinforcement learning. / Gaudet, Brian; Linares, Richard; Furfaro, Roberto.

AAS/AIAA Astrodynamics Specialist Conference, 2018. ed. / Belinda G. Marchand; Ryan M. Weisman; Brandon A. Jones; Puneet Singla. Univelt Inc., 2018. p. 3135-3154 (Advances in the Astronautical Sciences; Vol. 167).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Gaudet, B, Linares, R & Furfaro, R 2018, Integrated guidance and control for pinpoint mars landing using reinforcement learning. in BG Marchand, RM Weisman, BA Jones & P Singla (eds), AAS/AIAA Astrodynamics Specialist Conference, 2018. Advances in the Astronautical Sciences, vol. 167, Univelt Inc., pp. 3135-3154, AAS/AIAA Astrodynamics Specialist Conference, 2018, Montreal, Canada, 8/19/18.
Gaudet B, Linares R, Furfaro R. Integrated guidance and control for pinpoint mars landing using reinforcement learning. In Marchand BG, Weisman RM, Jones BA, Singla P, editors, AAS/AIAA Astrodynamics Specialist Conference, 2018. Univelt Inc. 2018. p. 3135-3154. (Advances in the Astronautical Sciences).
Gaudet, Brian ; Linares, Richard ; Furfaro, Roberto. / Integrated guidance and control for pinpoint mars landing using reinforcement learning. AAS/AIAA Astrodynamics Specialist Conference, 2018. editor / Belinda G. Marchand ; Ryan M. Weisman ; Brandon A. Jones ; Puneet Singla. Univelt Inc., 2018. pp. 3135-3154 (Advances in the Astronautical Sciences).
@inproceedings{9697a49c13164a5695017d0125fbbca2,
title = "Integrated guidance and control for pinpoint mars landing using reinforcement learning",
abstract = "Future Mars missions will require advanced guidance, navigation, and control algorithms for the powered descent phase in order to target specific surface locations and achieve pinpoint accuracy (landing error ellipse < 5m radius). This requires both a navigation system capable of estimating the lander’s state in real-time and a guidance and control system that can map the estimated lander state to body-frame actuator commands. In this paper we present a novel integrated guidance and control algorithm designed by applying the principles of reinforcement learning theory. The key innovation is the use of reinforcement learning to learn a policy mapping the lander’s estimated state directly to actuator commands, with the policy resulting in accurate and fuel efficient trajectories. Specifically, we use proximal policy optimization, a policy gradient method, to learn the policy. We present simulation results demonstrating the guidance and control system’s performance in a 6-DOF simulation environment, and demonstrate robustness to noise and system parameter uncertainty.",
author = "Brian Gaudet and Richard Linares and Roberto Furfaro",
year = "2018",
month = "1",
day = "1",
language = "English (US)",
isbn = "9780877036579",
series = "Advances in the Astronautical Sciences",
publisher = "Univelt Inc.",
pages = "3135--3154",
editor = "Marchand, {Belinda G.} and Weisman, {Ryan M.} and Jones, {Brandon A.} and Puneet Singla",
booktitle = "AAS/AIAA Astrodynamics Specialist Conference, 2018",

}

TY - GEN

T1 - Integrated guidance and control for pinpoint mars landing using reinforcement learning

AU - Gaudet, Brian

AU - Linares, Richard

AU - Furfaro, Roberto

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Future Mars missions will require advanced guidance, navigation, and control algorithms for the powered descent phase in order to target specific surface locations and achieve pinpoint accuracy (landing error ellipse < 5m radius). This requires both a navigation system capable of estimating the lander’s state in real-time and a guidance and control system that can map the estimated lander state to body-frame actuator commands. In this paper we present a novel integrated guidance and control algorithm designed by applying the principles of reinforcement learning theory. The key innovation is the use of reinforcement learning to learn a policy mapping the lander’s estimated state directly to actuator commands, with the policy resulting in accurate and fuel efficient trajectories. Specifically, we use proximal policy optimization, a policy gradient method, to learn the policy. We present simulation results demonstrating the guidance and control system’s performance in a 6-DOF simulation environment, and demonstrate robustness to noise and system parameter uncertainty.

AB - Future Mars missions will require advanced guidance, navigation, and control algorithms for the powered descent phase in order to target specific surface locations and achieve pinpoint accuracy (landing error ellipse < 5m radius). This requires both a navigation system capable of estimating the lander’s state in real-time and a guidance and control system that can map the estimated lander state to body-frame actuator commands. In this paper we present a novel integrated guidance and control algorithm designed by applying the principles of reinforcement learning theory. The key innovation is the use of reinforcement learning to learn a policy mapping the lander’s estimated state directly to actuator commands, with the policy resulting in accurate and fuel efficient trajectories. Specifically, we use proximal policy optimization, a policy gradient method, to learn the policy. We present simulation results demonstrating the guidance and control system’s performance in a 6-DOF simulation environment, and demonstrate robustness to noise and system parameter uncertainty.

UR - http://www.scopus.com/inward/record.url?scp=85069482375&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85069482375&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85069482375

SN - 9780877036579

T3 - Advances in the Astronautical Sciences

SP - 3135

EP - 3154

BT - AAS/AIAA Astrodynamics Specialist Conference, 2018

A2 - Marchand, Belinda G.

A2 - Weisman, Ryan M.

A2 - Jones, Brandon A.

A2 - Singla, Puneet

PB - Univelt Inc.

ER -