NVIDIA T4 GPUs Come to Google Cloud for High Speed Machine Learning

Print Friendly, PDF & Email

NVIDIA T4 GPU

Today the Google Cloud announced Public Beta availability of NVIDIA T4 GPUs for Machine Learning workloads. Starting today, NVIDIA T4 GPU instances are available in the U.S. and Europe as well as several other regions across the globe, including Brazil, India, Japan and Singapore.

The T4 joins our NVIDIA K80, P4, P100, and V100 GPU offerings, providing customers with a wide selection of hardware-accelerated compute options,” said Chris Kleban, Product Manager at Google Cloud. “The T4 is the best GPU in our product portfolio for running inference workloads. Its high-performance characteristics for FP16, INT8, and INT4 allow you to run high-scale inference with flexible accuracy/performance tradeoffs that are not available on any other accelerator.”

NVIDIA T4 GPUs are designed to accelerate diverse cloud workloads, including high-performance computing, deep learning training and inference, machine learning, data analytics, and graphics. NVIDIA T4 is based on NVIDIA’s new Turing architecture and features multi-precision Turing Tensor Cores and new RT Cores.

According to Google, each T4 GPU is equipped with 16GB of GPU memory, delivering 260 TOPS of computing performance. The devices can be used for as low as $0.29 per hour per GPU on Preemptible VM instances. “On-demand instances start at $0.95 per hour per GPU, with up to a 30 percent discount with sustained use discounts,” Kleban said.

The Google Cloud AI team has published a technical blog to help developers make the most out of T4 GPUs and the NVIDIA TensorRT platform. In this post, the team describes how to run deep learning inference on large-scale workloads with NVIDIA TensorRT 5 running on NVIDIA T4 GPUs on the Google Cloud Platform.

An ideal place to download software to run on the new T4 instance type is NGC, NVIDIA’s catalog of GPU-accelerated software for AI, machine learning, and HPC. NGC features a large variety of ready-to-run containers with GPU-optimized software such as the TensorFlow AI framework, RAPIDS for accelerated data science, the above-mentioned NVIDIA TensorRT and ParaView with NVIDIA OptiX, and much more.

Sign up for our insideHPC Newsletter