AMD Zendnn 5.0.1 published to help the Epyc inference for recommendation systems & LLMS

The last year shortly after the EPYC 9005 “Turin” processor start was Zendnn 5.0 for Zen 5 optimized CPU infection with pytorch and tensorflow. According to AMD engineers, Zendnn 5.0 delivers an output of up to 400%. Nowadays, Zendnn 5.0.1 is with further optimizations, especially with regard to recommendation engines and large language models (LLMS).

Zendnn 5.0.1 is a small update that is aimed at further advance the power cover for recommendation systems and large voice models for AMD -EPYC CPUs. Zendnn 5.0.1 is a small update that is aimed at further advance the power cover for recommendation systems and large voice models for AMD -EPYC CPUs. There is now support for int8- and int4 quantized deep learning recommendation models (DLRM) for faster design and lower memory consumption compared to the preferred BF16 precision of Zendnn.

That was with regard to the listed changes for Zendnn 5.0.1. There were no published benchmark numbers of AMD of the effects that they see from this Zendnn point publication, but BF16 to Int4/Int8 can achieve a considerable thrust. In any case, those who want to use Zendnn 5.0.1 for optimized CPU-based inferences with Pytorch and Tensorflow on AMD-Zen-processors can find the new version via Github. The Zendnn library remains available under an open source license from Apache 2.0.