News › performance
By Zayden R., August 17, 2026
Sankalp's innovative approach in a GPU Mode contest led to a 232x faster kernel using Codex for QR factorization. This significant speedup in linear algebra computation offers valuable insights for engineers working with CUDA matrices.
In a recent contest hosted by GPU Mode, in collaboration with Core Automation, an impressive achievement was made by a participant named Sankalp. He managed to achieve a 232x speedup over the baseline solution in the implementation of batched square compact-Householder QR factorization. This was part of GPU Mode's Linear Algebra Kernels in the Age of Research series. For engineers working with CUDA matrices, this breakthrough provides a valuable perspective on optimizing performance.
The challenge involved processing a batch of square FP32 CUDA matrices, which required returning a compact Householder QR representation. This is akin to what torch.geqrf(A) accomplishes, producing an H matrix with an upper triangle R and a lower triangle storing Householder vectors, along with a tau vector of reflector coefficients. The contest's checker verified the results by rebuilding Q with torch.linalg.householder_product(H, tau) and ensuring the QR decomposition satisfied FP32-style checks.
Participants were ranked based on runtime, measured by the geometric mean across various matrix shapes and conditioning cases. The matrices ranged in size, with significant emphasis on batched squares like 512x512, and extending to larger cases such as 1024, 2048, and 4096. While low-bit FP16, FP8, or NVFP4 was permissible internally, the outputs had to pass FP32 checks.
Sankalp's method, which he describes as a form of 'auto-research' or 'loop engineering', focused on optimizing the QR decomposition process. By introducing idea diversity to escape local maxima, he significantly enhanced kernel performance. This approach could be particularly beneficial for engineers looking to maximize computational efficiency in similar contexts.
This contest and its outcomes underscore the potential of innovative problem-solving in computational mathematics, particularly for those immersed in GPU-accelerated tasks. Sankalp's success not only highlights the power of Codex in optimizing GPU kernels but also opens up new avenues for performance improvements in linear algebra computations.
The Linux Camp teaches these topics as hands-on labs on real virtual machines, verified as you type.