# Dynamic Voltage and Frequency Scaling in NoCs with Supervised and Reinforcement Learning Techniques

Quintin Fettes, Mark Clark, Razvan Bunescu, Avinash Karanth, Ahmed Louri

Research output: Contribution to journalArticle

2 Citations (Scopus)

### Abstract

4Network-on-Chips (NoCs) are the de facto choice for designing the interconnect fabric in multicore chips due to their regularity, efficiency, simplicity, and scalability. However, NoC suffers from excessive static power and dynamic energy due to transistor leakage current and data movement between the cores and caches. Power consumption issues are only exacerbated by ever decreasing technology sizes. Dynamic Voltage and Frequency Scaling (DVFS) is one technique that seeks to reduce dynamic energy; however this often occurs at the expense of performance. In this paper, we propose LEAD Learning-enabled Energy-Aware Dynamic voltage/frequency scaling for multicore architectures using both supervised learning and reinforcement learning approaches. LEAD groups the router and its outgoing links into the same V/F domain and implements proactive DVFS mode management strategies that rely on offline trained machine learning models in order to provide optimal V/F mode selection between different voltage/frequency pairs. We present three supervised learning versions of LEAD that are based on buffer utilization, change in buffer utilization and change in energy/throughput, which allow proactive mode selection based on accurate prediction of future network parameters. We then describe a reinforcement learning approach to LEAD that optimizes the DVFS mode selection directly, obviating the need for label and threshold engineering. Simulation results using PARSEC and Splash-2 benchmarks on a <formula><tex>$4 \times 4$</tex></formula> concentrated mesh architecture show that by using supervised learning LEAD can achieve an average dynamic energy savings of 15.4% for a loss in throughput of 0.8% with no significant impact on latency. When reinforcement learning is used, LEAD increases average dynamic energy savings to 20.3% at the cost of a 1.5% decrease in throughput and a 1.7% increase in latency. Overall, the more flexible reinforcement learning approach enables learning an optimal behavior for a wider range of load environments under any desired energy vs. throughput tradeoff.

Original language English (US) IEEE Transactions on Computers https://doi.org/10.1109/TC.2018.2875476 Accepted/In press - Jan 1 2018 Yes

### Fingerprint

Supervised learning
Reinforcement learning
Supervised Learning
Reinforcement Learning
Voltage
Scaling
Throughput
Energy conservation
Energy
Energy Saving
Latency
Buffer
Electric potential
Chip
Routers
Leakage currents
Learning systems
Scalability
Labels
Transistors

### Keywords

• Dynamic Voltage and Frequency Scaling (DVFS)
• Machine Learning (ML)
• Reinforcement Learning
• Ridge Regression

### ASJC Scopus subject areas

• Software
• Theoretical Computer Science
• Hardware and Architecture
• Computational Theory and Mathematics

### Cite this

Dynamic Voltage and Frequency Scaling in NoCs with Supervised and Reinforcement Learning Techniques. / Fettes, Quintin; Clark, Mark; Bunescu, Razvan; Karanth, Avinash; Louri, Ahmed.

In: IEEE Transactions on Computers, 01.01.2018.

Research output: Contribution to journalArticle

@article{a552d4fbc26840e8a75ae62cf0cc3344,
title = "Dynamic Voltage and Frequency Scaling in NoCs with Supervised and Reinforcement Learning Techniques",
abstract = "4Network-on-Chips (NoCs) are the de facto choice for designing the interconnect fabric in multicore chips due to their regularity, efficiency, simplicity, and scalability. However, NoC suffers from excessive static power and dynamic energy due to transistor leakage current and data movement between the cores and caches. Power consumption issues are only exacerbated by ever decreasing technology sizes. Dynamic Voltage and Frequency Scaling (DVFS) is one technique that seeks to reduce dynamic energy; however this often occurs at the expense of performance. In this paper, we propose LEAD Learning-enabled Energy-Aware Dynamic voltage/frequency scaling for multicore architectures using both supervised learning and reinforcement learning approaches. LEAD groups the router and its outgoing links into the same V/F domain and implements proactive DVFS mode management strategies that rely on offline trained machine learning models in order to provide optimal V/F mode selection between different voltage/frequency pairs. We present three supervised learning versions of LEAD that are based on buffer utilization, change in buffer utilization and change in energy/throughput, which allow proactive mode selection based on accurate prediction of future network parameters. We then describe a reinforcement learning approach to LEAD that optimizes the DVFS mode selection directly, obviating the need for label and threshold engineering. Simulation results using PARSEC and Splash-2 benchmarks on a $4 \times 4$ concentrated mesh architecture show that by using supervised learning LEAD can achieve an average dynamic energy savings of 15.4{\%} for a loss in throughput of 0.8{\%} with no significant impact on latency. When reinforcement learning is used, LEAD increases average dynamic energy savings to 20.3{\%} at the cost of a 1.5{\%} decrease in throughput and a 1.7{\%} increase in latency. Overall, the more flexible reinforcement learning approach enables learning an optimal behavior for a wider range of load environments under any desired energy vs. throughput tradeoff.",
keywords = "Dynamic Voltage and Frequency Scaling (DVFS), Machine Learning (ML), Reinforcement Learning, Ridge Regression",
author = "Quintin Fettes and Mark Clark and Razvan Bunescu and Avinash Karanth and Ahmed Louri",
year = "2018",
month = "1",
day = "1",
doi = "10.1109/TC.2018.2875476",
language = "English (US)",
journal = "IEEE Transactions on Computers",
issn = "0018-9340",
publisher = "IEEE Computer Society",

}

TY - JOUR

T1 - Dynamic Voltage and Frequency Scaling in NoCs with Supervised and Reinforcement Learning Techniques

AU - Fettes, Quintin

AU - Clark, Mark

AU - Bunescu, Razvan

AU - Karanth, Avinash

AU - Louri, Ahmed

PY - 2018/1/1

Y1 - 2018/1/1

N2 - 4Network-on-Chips (NoCs) are the de facto choice for designing the interconnect fabric in multicore chips due to their regularity, efficiency, simplicity, and scalability. However, NoC suffers from excessive static power and dynamic energy due to transistor leakage current and data movement between the cores and caches. Power consumption issues are only exacerbated by ever decreasing technology sizes. Dynamic Voltage and Frequency Scaling (DVFS) is one technique that seeks to reduce dynamic energy; however this often occurs at the expense of performance. In this paper, we propose LEAD Learning-enabled Energy-Aware Dynamic voltage/frequency scaling for multicore architectures using both supervised learning and reinforcement learning approaches. LEAD groups the router and its outgoing links into the same V/F domain and implements proactive DVFS mode management strategies that rely on offline trained machine learning models in order to provide optimal V/F mode selection between different voltage/frequency pairs. We present three supervised learning versions of LEAD that are based on buffer utilization, change in buffer utilization and change in energy/throughput, which allow proactive mode selection based on accurate prediction of future network parameters. We then describe a reinforcement learning approach to LEAD that optimizes the DVFS mode selection directly, obviating the need for label and threshold engineering. Simulation results using PARSEC and Splash-2 benchmarks on a $4 \times 4$ concentrated mesh architecture show that by using supervised learning LEAD can achieve an average dynamic energy savings of 15.4% for a loss in throughput of 0.8% with no significant impact on latency. When reinforcement learning is used, LEAD increases average dynamic energy savings to 20.3% at the cost of a 1.5% decrease in throughput and a 1.7% increase in latency. Overall, the more flexible reinforcement learning approach enables learning an optimal behavior for a wider range of load environments under any desired energy vs. throughput tradeoff.

AB - 4Network-on-Chips (NoCs) are the de facto choice for designing the interconnect fabric in multicore chips due to their regularity, efficiency, simplicity, and scalability. However, NoC suffers from excessive static power and dynamic energy due to transistor leakage current and data movement between the cores and caches. Power consumption issues are only exacerbated by ever decreasing technology sizes. Dynamic Voltage and Frequency Scaling (DVFS) is one technique that seeks to reduce dynamic energy; however this often occurs at the expense of performance. In this paper, we propose LEAD Learning-enabled Energy-Aware Dynamic voltage/frequency scaling for multicore architectures using both supervised learning and reinforcement learning approaches. LEAD groups the router and its outgoing links into the same V/F domain and implements proactive DVFS mode management strategies that rely on offline trained machine learning models in order to provide optimal V/F mode selection between different voltage/frequency pairs. We present three supervised learning versions of LEAD that are based on buffer utilization, change in buffer utilization and change in energy/throughput, which allow proactive mode selection based on accurate prediction of future network parameters. We then describe a reinforcement learning approach to LEAD that optimizes the DVFS mode selection directly, obviating the need for label and threshold engineering. Simulation results using PARSEC and Splash-2 benchmarks on a $4 \times 4$ concentrated mesh architecture show that by using supervised learning LEAD can achieve an average dynamic energy savings of 15.4% for a loss in throughput of 0.8% with no significant impact on latency. When reinforcement learning is used, LEAD increases average dynamic energy savings to 20.3% at the cost of a 1.5% decrease in throughput and a 1.7% increase in latency. Overall, the more flexible reinforcement learning approach enables learning an optimal behavior for a wider range of load environments under any desired energy vs. throughput tradeoff.

KW - Dynamic Voltage and Frequency Scaling (DVFS)

KW - Machine Learning (ML)

KW - Reinforcement Learning

KW - Ridge Regression

UR - http://www.scopus.com/inward/record.url?scp=85054637525&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85054637525&partnerID=8YFLogxK

U2 - 10.1109/TC.2018.2875476

DO - 10.1109/TC.2018.2875476

M3 - Article

AN - SCOPUS:85054637525

JO - IEEE Transactions on Computers

JF - IEEE Transactions on Computers

SN - 0018-9340

ER -