ggml/llama.cpp releases: b12000
Sentiment: neutral
TL;DR
The ggml/llama.cpp project released version b12000 with support for OCP FP8 E4M3 optimization and fixes in type-checking and data conversion, aiming to enhance performance and compatibility. This update is significant as it improves handling of specific model conversions, potentially boosting efficiency in AI model deployment.
Detailed Summary
The ggml/llama.cpp project has released version b12000, which includes experimental support for OCP FP8 E4M3 format and fixes issues with Python type-checking and input scale materialization. This update addresses a bug where per-row FP8 data incorrectly converted to NVFP4. The broader impact is enhanced flexibility in model precision handling, potentially improving performance on specific hardware configurations.
Key Points
- • WIP OCP FP8 E4M3 support
- • Fix python type-checker
- • Materialize input_scales also for FP8 when converting
- • Per row FP8 wrongly fell to NVFP4 conversion, as NVFP4 check