H100 SXM
NVIDIA · NVIDIA Hopper™ · Released 2022-03-27 · 16,896 CUDA Cores · 80 VRAM · Up to 700W (configurable) TDP
OVERVIEW
The NVIDIA H100 SXM variant features exceptional performance and scalability for a wide range of workloads. It includes fourth-generation Tensor Cores and a Transformer Engine with FP8 precision, providing up to 4X faster training over the prior generation for large language models.
WORKLOAD SUITABILITY
The H100 SXM variant is ideal for high-performance computing (HPC) applications, large language model training, and data analytics that require significant compute power and memory bandwidth.
- High-Performance Computing (HPC)
- Large Language Model Training
- Data Analytics
SPECIFICATIONS
WHAT THIS GPU IS GOOD AT
The H100 SXM excels in AI and machine learning workloads, particularly in training large neural networks and performing inference at scale. It offers significant performance improvements over its predecessors due to its advanced architecture and increased memory bandwidth. The H100 is also well-suited for high-performance computing (HPC) applications, providing exceptional computational power and efficiency.
SERVER OPTIONS
The NVIDIA H100 SXM is available in high-performance server platforms such as NVIDIA's own DGX systems and HGX platforms. It is also integrated into OEM servers from major vendors like Dell PowerEdge, HPE ProLiant, and Supermicro. Cloud providers such as AWS, Azure, and Google Cloud offer instances powered by the H100, providing scalable options for various workloads.
POWER, THERMALS & NOISE
The H100 SXM has a high TDP, typically around 700 watts, necessitating robust cooling solutions, often liquid cooling, to maintain optimal performance. Its thermal design is optimized for datacenter environments, ensuring efficient heat dissipation. Noise is generally not a concern in datacenter settings, but adequate cooling infrastructure is essential to manage the heat output.
COMPATIBILITY & SYSTEM FIT
The H100 SXM uses the SXM form factor, which is designed for high-density server environments. It supports NVLink for high-speed GPU-to-GPU communication, enhancing multi-GPU configurations. The platform requires a compatible motherboard with SXM slots and a power supply capable of supporting its high power demands.
LIMITATIONS & KNOWN TRADE-OFFS
One limitation of the H100 SXM is its high power consumption, which may not be suitable for all datacenter environments. Additionally, its reliance on specific server platforms and cooling solutions can limit deployment flexibility. Availability can be constrained due to high demand and production capacities, potentially leading to longer lead times for procurement.
PRICING
NOTES
Ideal for high-performance computing (HPC) applications and AI workloads.
"Preliminary specifications. May be subject to change."