A team of engineers at AI inference technology company BitEnergy AI reports a method to reduce the energy needs of AI applications by 95%. The group has published a paper describing their new technique on the arXiv preprint server.
In other words… This method of computation could save 95% of the energy spent on floating point multiplication (and save 80% on dot products)… Not 95% of total energy.
It’s an improvement (potentially), but I don’t see any analysis of how this would impact total energy.
Good point. Though, the vast majority of ML training and use is tensor math on floating points, so largely dot and cross products, among other matrix operations.
This is an extremely misleading headline.
From the abstract:
In other words… This method of computation could save 95% of the energy spent on floating point multiplication (and save 80% on dot products)… Not 95% of total energy.
It’s an improvement (potentially), but I don’t see any analysis of how this would impact total energy.
I’d say it’s not just misleading but incorrect if it says “integer” but it’s actually floats.
Good point. Though, the vast majority of ML training and use is tensor math on floating points, so largely dot and cross products, among other matrix operations.