![]() ![]() Averaging across all SMs gives the overall achieved occupancy, which is shown alongside theoretical occupancy in the experiment details pane.Īchieved occupancy cannot exceed theoretical occupancy, so the first step toward increasing occupancy should be to increase theoretical occupancy by adjusting the limiting factors. Dividing by the SM's maximum supported number of active warps gives the achieved occupancy per SM averaged over the duration of the kernel, which is shown in the Achieved Occupancy Chart. These counts are then summed across all warp schedulers on each SM and divided by the clock cycles the SM is active to find the average active warps per SM. Achieved occupancy is measured on each warp scheduler using hardware performance counters to count the number of active warps on that scheduler every clock cycle. Maintaining as many active warps as possible (a high occupancy) throughout the execution of the kernel helps to avoid situations where all warps are stalled and no instructions are issued. To sufficiently hide latencies between dependent instructions, each scheduler must have at least one warp eligible to issue an instruction every clock cycle. Each warp scheduler attempts to issue instructions from a warp on each clock cycle. ![]() As explained in Issue Efficiency, an SM contain one or more warp schedulers. Theoretical occupancy shows the upper bound active warps on an SM, but the true number of active warps varies over the duration of the kernel, as warps begin and end. Similarly, 16 active blocks with 128 threads per block (4 warps per block) would also result in 64 active warps, and 100% theoretical occupancy. For example, on a GPU that supports 64 active warps per SM, 8 active blocks with 256 threads per block (8 warps per block) results in 64 active warps, and 100% theoretical occupancy. If this factor is limiting active blocks, occupancy cannot be increased. ![]() Since occupancy is the ratio of active warps to maximum supported active warps, occupancy is 100% if the number of active warps equals the maximum. The SM has a maximum number of warps that can be active at once. Thus, the upper limit for active warps can be raised by increasing the number of warps per block (defined by block dimensions), or by changing the factors limiting how many blocks can fit on an SM to allow more active blocks. The upper limit for active warps is the product of the upper limit for active blocks and the number of warps per block. The number of blocks which can execute concurrently on an SM is limited by the factors listed below. ![]() A block is considered active from the time its warps begin executing to the time when all warps in the block have exited from the kernel. Each block of a kernel launch gets distributed to one of the SMs for execution. There is an upper limit for active warps, and thus also for occupancy, derivable from the launch configuration, compile options for the kernel, and device capabilities. An early step of kernel performance analysis should be to check occupancy and observe the effects on kernel execution time when running at different occupancy levels. When occupancy is at a sufficient level to hide latency, increasing it further may degrade performance due to the reduction in resources per thread. Low occupancy results in poor instruction issue efficiency, because there are not enough eligible warps to hide latency between dependent instructions. Occupancy varies over time as warps begin and end, and can be different for each SM. Occupancy is defined as the ratio of active warps on an SM to the maximum number of active warps supported by the SM. There is a maximum number of warps which can be concurrently active on a Streaming Multiprocessor (SM), as listed in the Programming Guide's table of compute capabilities. A warp is considered active from the time its threads begin executing to the time when all threads in the warp have exited from the kernel. The CUDA C Programming Guide explains how a CUDA device's hardware implementation groups adjacent threads within a block into warps. Additional graphs show achieved occupancy per SM, and illustrate how occupancy can be controlled by varying compiler and launch parameters. The Achieved Occupancy Profile mode experiment measures occupancy during execution of the kernel, and adds the achieved values to the Occupancy experiment detail pane alongside the theoretical values. (Undefined variable: MyVariables.NsightVSEMainHeader)įor all CUDA kernel launches recorded in both Profile and Trace modes, the Occupancy experiment detail pane shows "Theoretical Occupancy", the upper limit for occupancy imposed by the kernel launch configuration and the capabilities of the CUDA device. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |