NDLI: Warped-slicer:efficient intra-SM slicing through dynamic resource partitioning for GPU multiprogramming

Content Provider	ACM Digital Library
Author	Annavaram, Murali Xu, Qiumin Jeon, Hyeran Kim, Keunsoo Ro, Won Woo
Abstract	As technology scales, GPUs are forecasted to incorporate an ever-increasing amount of computing resources to support thread-level parallelism. But even with the best effort, exposing massive thread-level parallelism from a single GPU kernel, particularly from general purpose applications, is going to be a difficult challenge. In some cases, even if there is sufficient thread-level parallelism in a kernel, there may not be enough available memory bandwidth to support such massive concurrent thread execution. Hence, GPU resources may be underutilized as more general purpose applications are ported to execute on GPUs. In this paper, we explore multiprogramming GPUs as a way to resolve the resource underutilization issue. There is a growing hardware support for multiprogramming on GPUs. Hyper-Q has been introduced in the Kepler architecture which enables multiple kernels to be invoked via tens of hardware queue streams. Spatial multitasking has been proposed to partition GPU resources across multiple kernels. But the partitioning is done at the coarse granularity of streaming multiprocessors (SMs) where each kernel is assigned to a subset of SMs. In this paper, we advocate for partitioning a single SM across multiple kernels, which we term as intra-SM slicing. We explore various intra-SM slicing strategies that slice resources within each SM to concurrently run multiple kernels on the SM. Our results show that there is not one intra-SM slicing strategy that derives the best performance for all application pairs. We propose Warped-Slicer, a dynamic intra-SM slicing strategy that uses an analytical method for calculating the SM resource partitioning across different kernels that maximizes performance. The model relies on a set of short online profile runs to determine how each kernel's performance varies as more thread blocks from each kernel are assigned to an SM. The model takes into account the interference effect of shared resource usage across multiple kernels. The model is also computationally efficient and can determine the resource partitioning quickly to enable dynamic decision making as new kernels enter the system. We demonstrate that the proposed Warped-Slicer approach improves performance by 23% over the baseline multiprogramming approach with minimal hardware overhead.
Starting Page	230
Ending Page	242
Page Count	13
File Format	PDF
ISSN	01635964
DOI	10.1145/3007787.3001161
Journal	ACM SIGARCH Computer Architecture News (CARN)
Volume Number	44
Issue Number	3
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	1981-04-01
Publisher Place	New York
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Multi-kernel Gpus Scheduling Multiprogramming Resource management
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Warped-slicer: efficient intra-SM slicing through dynamic resource partitioning for GPU multiprogramming

Kernelet: High-Throughput GPU Kernel Executions with Dynamic Slicing and Scheduling

A processor allocation method for time-sharing

A Lock-Controlled Session Table Partitioning Scheme with Dynamic Resource Balancing for Multi-Core Architecture

1Kernelet: High-Throughput GPU Kernel Executions with Dynamic Slicing and Scheduling

A Dynamic Resource Management and Scheduling Environment for Embedded Multimedia and Communications Platforms

1 Kernelet: High-Throughput GPU Kernel Executions with Dynamic Slicing and Scheduling

Softshell: dynamic scheduling on GPUs

PRO: Progress Aware GPU Warp Scheduling Algorithm

Warped-slicer:efficient intra-SM slicing through dynamic resource partitioning for GPU multiprogramming

Similar Documents

Warped-slicer: efficient intra-SM slicing through dynamic resource partitioning for GPU multiprogramming

Kernelet: High-Throughput GPU Kernel Executions with Dynamic Slicing and Scheduling

A processor allocation method for time-sharing

A Lock-Controlled Session Table Partitioning Scheme with Dynamic Resource Balancing for Multi-Core Architecture

1Kernelet: High-Throughput GPU Kernel Executions with Dynamic Slicing and Scheduling

A Dynamic Resource Management and Scheduling Environment for Embedded Multimedia and Communications Platforms

1 Kernelet: High-Throughput GPU Kernel Executions with Dynamic Slicing and Scheduling

Softshell: dynamic scheduling on GPUs

PRO: Progress Aware GPU Warp Scheduling Algorithm

Warped-slicer:efficient intra-SM slicing through dynamic resource partitioning for GPU multiprogramming