How can a GPU work on many calculations together?

Many graphics and scientific calculations apply similar operations to large collections of values. For example, changing every pixel’s brightness can involve the same arithmetic at many locations. A GPU organizes many threads so that such work can progress in parallel.

A program assigns work to threads, often giving each one an index into an array. Hardware schedules groups of threads on its execution resources. In NVIDIA’s model, a warp contains threads that execute instructions through a shared scheduling arrangement, while each thread has its own logical state and data.

Parallel work is easiest when many elements can be processed independently and access memory efficiently. Dependencies require coordination. If threads in a group follow different branches, their different paths can reduce efficiency; a large number of threads alone is not a promise of high speed.

A GPU also spends time moving data, waiting for memory and coordinating work. CPUs and GPUs therefore suit different mixes of tasks, and often work together. The useful principle is matching an algorithm’s structure to parallel resources, rather than assuming that a GPU makes every program faster.