NVIDIA has joined hands with Microsoft to build one of the most powerful AI supercomputers in the world, powered by Microsoft Azure’s advanced supercomputing infrastructure combined with NVIDIA GPUs, networking and full stack of AI software to help enterprises train, deploy and scale AI, including large, state-of-the-art models.

Azure’s cloud-based AI supercomputer includes powerful and scalable ND- and NC-series virtual machines optimized for AI distributed training and inference. It is the first public cloud to incorporate NVIDIA’s advanced AI stack, adding tens of thousands of NVIDIA A100 and H100 GPUs, NVIDIA Quantum-2 400Gb/s InfiniBand networking and the NVIDIA AI Enterprise software suite to its platform.

As part of the multi-year collaboration, NVIDIA will utilize Azure’s scalable virtual machine instances to research and further accelerate advances in generative AI, a rapidly emerging area of AI in which foundational models like Megatron Turing NLG 530B are the basis for unsupervised, self-learning algorithms to create new text, code, digital images, video or audio.

The companies will also collaborate to optimize Microsoft’s DeepSpeed deep learning optimization software. NVIDIA’s full stack of AI workflows and software development kits, optimized for Azure, will be made available to Azure enterprise customers.

Additionally, the platform will support a broad range of AI applications and services, including Microsoft DeepSpeed and the NVIDIA AI Enterprise software suite.

Microsoft DeepSpeed will leverage the NVIDIA H100 Transformer Engine to accelerate transformer-based models used for large language models, generative AI and writing computer code, among other applications. This technology applies 8-bit floating point precision capabilities to DeepSpeed to dramatically accelerate AI calculations for transformers — at twice the throughput of 16-bit operations.

NVIDIA AI Enterprise — the globally adopted software of the NVIDIA AI platform — is certified and supported on Microsoft Azure instances with NVIDIA A100 GPUs. Support for Azure instances with NVIDIA H100 GPUs will be added in a future software release.