Nvidia announces TensorRT 8, slashes BERT inference times down to a millisecond

Providing over twice the precision and inference speed compared to the last generation, Nvidia's new TensorRT 8 deep learning SDK clocked in a time of 1.2 ms in BERT-Large's inference.