Actor-critic reinforcement learning approach to relative motion guidance in near-rectilinear orbit

Andrea Scorsoglio, Roberto Furfaro, Richard Linares, Mauro Massari

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper aims a developing a new feedback guidance algorithm for docking maneuvers in the cislunar environment. In particular, the goal is to create an algorithm that is lightweight, closed-loop and capable of taking path constraints into account. The problem has been solved starting from the well know Zero-Effort-Miss/Zero-Effort-Velocity (ZEM/ZEV) guidance using machine learning to improve its capabilities and widen its field of application. The algorithm has been developed in the circular restricted three body problem (CRTBP) framework for Near Rectilinear Orbits (NRO) in the Earth-Moon system but the results can be easily generalized to many more guidance problems. The results are satisfactory and show that reinforcement learning can be effectively used to solve constrained relative spacecraft guidance problems.

Original languageEnglish (US)
Title of host publicationSpaceflight Mechanics 2019
EditorsFrancesco Topputo, Andrew J. Sinclair, Matthew P. Wilkins, Renato Zanetti
PublisherUnivelt Inc.
Pages1737-1756
Number of pages20
ISBN (Print)9780877036593
StatePublished - Jan 1 2019
Event29th AAS/AIAA Space Flight Mechanics Meeting, 2019 - Maui, United States
Duration: Jan 13 2019Jan 17 2019

Publication series

NameAdvances in the Astronautical Sciences
Volume168
ISSN (Print)0065-3438

Conference

Conference29th AAS/AIAA Space Flight Mechanics Meeting, 2019
CountryUnited States
CityMaui
Period1/13/191/17/19

Fingerprint

Reinforcement learning
reinforcement
learning
Orbits
orbits
spacecraft guidance
Earth-Moon system
machine learning
Electronic guidance systems
three body problem
Moon
maneuvers
Learning systems
spacecraft
Earth (planet)
Feedback

ASJC Scopus subject areas

  • Aerospace Engineering
  • Space and Planetary Science

Cite this

Scorsoglio, A., Furfaro, R., Linares, R., & Massari, M. (2019). Actor-critic reinforcement learning approach to relative motion guidance in near-rectilinear orbit. In F. Topputo, A. J. Sinclair, M. P. Wilkins, & R. Zanetti (Eds.), Spaceflight Mechanics 2019 (pp. 1737-1756). [AAS 19-441] (Advances in the Astronautical Sciences; Vol. 168). Univelt Inc..

Actor-critic reinforcement learning approach to relative motion guidance in near-rectilinear orbit. / Scorsoglio, Andrea; Furfaro, Roberto; Linares, Richard; Massari, Mauro.

Spaceflight Mechanics 2019. ed. / Francesco Topputo; Andrew J. Sinclair; Matthew P. Wilkins; Renato Zanetti. Univelt Inc., 2019. p. 1737-1756 AAS 19-441 (Advances in the Astronautical Sciences; Vol. 168).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Scorsoglio, A, Furfaro, R, Linares, R & Massari, M 2019, Actor-critic reinforcement learning approach to relative motion guidance in near-rectilinear orbit. in F Topputo, AJ Sinclair, MP Wilkins & R Zanetti (eds), Spaceflight Mechanics 2019., AAS 19-441, Advances in the Astronautical Sciences, vol. 168, Univelt Inc., pp. 1737-1756, 29th AAS/AIAA Space Flight Mechanics Meeting, 2019, Maui, United States, 1/13/19.
Scorsoglio A, Furfaro R, Linares R, Massari M. Actor-critic reinforcement learning approach to relative motion guidance in near-rectilinear orbit. In Topputo F, Sinclair AJ, Wilkins MP, Zanetti R, editors, Spaceflight Mechanics 2019. Univelt Inc. 2019. p. 1737-1756. AAS 19-441. (Advances in the Astronautical Sciences).
Scorsoglio, Andrea ; Furfaro, Roberto ; Linares, Richard ; Massari, Mauro. / Actor-critic reinforcement learning approach to relative motion guidance in near-rectilinear orbit. Spaceflight Mechanics 2019. editor / Francesco Topputo ; Andrew J. Sinclair ; Matthew P. Wilkins ; Renato Zanetti. Univelt Inc., 2019. pp. 1737-1756 (Advances in the Astronautical Sciences).
@inproceedings{a33828c1030b441ca2af29be79ab3a7b,
title = "Actor-critic reinforcement learning approach to relative motion guidance in near-rectilinear orbit",
abstract = "This paper aims a developing a new feedback guidance algorithm for docking maneuvers in the cislunar environment. In particular, the goal is to create an algorithm that is lightweight, closed-loop and capable of taking path constraints into account. The problem has been solved starting from the well know Zero-Effort-Miss/Zero-Effort-Velocity (ZEM/ZEV) guidance using machine learning to improve its capabilities and widen its field of application. The algorithm has been developed in the circular restricted three body problem (CRTBP) framework for Near Rectilinear Orbits (NRO) in the Earth-Moon system but the results can be easily generalized to many more guidance problems. The results are satisfactory and show that reinforcement learning can be effectively used to solve constrained relative spacecraft guidance problems.",
author = "Andrea Scorsoglio and Roberto Furfaro and Richard Linares and Mauro Massari",
year = "2019",
month = "1",
day = "1",
language = "English (US)",
isbn = "9780877036593",
series = "Advances in the Astronautical Sciences",
publisher = "Univelt Inc.",
pages = "1737--1756",
editor = "Francesco Topputo and Sinclair, {Andrew J.} and Wilkins, {Matthew P.} and Renato Zanetti",
booktitle = "Spaceflight Mechanics 2019",

}

TY - GEN

T1 - Actor-critic reinforcement learning approach to relative motion guidance in near-rectilinear orbit

AU - Scorsoglio, Andrea

AU - Furfaro, Roberto

AU - Linares, Richard

AU - Massari, Mauro

PY - 2019/1/1

Y1 - 2019/1/1

N2 - This paper aims a developing a new feedback guidance algorithm for docking maneuvers in the cislunar environment. In particular, the goal is to create an algorithm that is lightweight, closed-loop and capable of taking path constraints into account. The problem has been solved starting from the well know Zero-Effort-Miss/Zero-Effort-Velocity (ZEM/ZEV) guidance using machine learning to improve its capabilities and widen its field of application. The algorithm has been developed in the circular restricted three body problem (CRTBP) framework for Near Rectilinear Orbits (NRO) in the Earth-Moon system but the results can be easily generalized to many more guidance problems. The results are satisfactory and show that reinforcement learning can be effectively used to solve constrained relative spacecraft guidance problems.

AB - This paper aims a developing a new feedback guidance algorithm for docking maneuvers in the cislunar environment. In particular, the goal is to create an algorithm that is lightweight, closed-loop and capable of taking path constraints into account. The problem has been solved starting from the well know Zero-Effort-Miss/Zero-Effort-Velocity (ZEM/ZEV) guidance using machine learning to improve its capabilities and widen its field of application. The algorithm has been developed in the circular restricted three body problem (CRTBP) framework for Near Rectilinear Orbits (NRO) in the Earth-Moon system but the results can be easily generalized to many more guidance problems. The results are satisfactory and show that reinforcement learning can be effectively used to solve constrained relative spacecraft guidance problems.

UR - http://www.scopus.com/inward/record.url?scp=85072959449&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072959449&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85072959449

SN - 9780877036593

T3 - Advances in the Astronautical Sciences

SP - 1737

EP - 1756

BT - Spaceflight Mechanics 2019

A2 - Topputo, Francesco

A2 - Sinclair, Andrew J.

A2 - Wilkins, Matthew P.

A2 - Zanetti, Renato

PB - Univelt Inc.

ER -