Addition is All You Need for Energy-efficient Language Models

great stuff!

also a typo on page 3, 2nd block of text: i believe "save size" should be "same size" right?

Nice idea! You could try combining it with the paper Scaling FP8 training to trillion-token LLMs to train models using far less energy than usual.

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

• Floating-Point Multiply-Add with Approximate Normalization for Low-Cost Matrix Engines (2024)
• ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models (2024)
• Efficient Arbitrary Precision Acceleration for Large Language Models on GPU Tensor Cores (2024)
• MobileQuant: Mobile-friendly Quantization for On-device Language Models (2024)
• LUT Tensor Core: Lookup Table Enables Efficient Low-Bit LLM Inference Acceleration (2024)

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

Great paper BitEnergy team! More power to energy efficient AI and edge computing ⚡
Here's a quick summary: https://soessentially.substack.com/p/gpu-gang-better-watch-out

Great work, thanks. a lot! Here's my summary:

💥 𝐋-𝐌𝐮𝐥: 𝐀𝐝𝐝𝐢𝐭𝐢𝐨𝐧-𝐎𝐧𝐥𝐲 𝐌𝐮𝐥𝐭𝐢𝐩𝐥𝐢𝐜𝐚𝐭𝐢𝐨𝐧 𝐜𝐚𝐧 𝐬𝐥𝐚𝐬𝐡 𝐜𝐨𝐦𝐩𝐮𝐭𝐚𝐭𝐢𝐨𝐧𝐚𝐥 𝐜𝐨𝐬𝐭𝐬 𝐛𝐲 𝟖𝟎%!

Microsoft researchers dropped a groundbreaking technique that could slash the energy use in transformer computations : their novel "linear-complexity multiplication" (L-Mul) algorithm approximates floating-point multiplication using energy-efficient integer addition instead of costly multiplications.

💡 Quick reminder on how floats are coded on 8 bits (FP8):
In the e4m3 FP8 standard, you encode a number as:
Sign (1 bit) | Exponent (4 bits) | Mantissa (3 bits)
Example: 0 (positive) | 1000 (8) | 101 (1/2 + 1/8 = 0.625)
Calculation: you add one to the mantissa, and multiply it by 2 power (the exponent - a bias term which is 7 for e4m3):

➡️ You get (1 + 0.625) × 2^(8-7) = 3.25

Now back to the paper. 𝗞𝗲𝘆 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:

⚡️ Multiplication is extremely energy-intensive compared to addition. For 32-bit operations, multiplication (3.7 pJ) uses 37x more energy than addition (0.1 pJ)!

🧮 Traditional floating-point multiplication go like (noting xm the mantissa and xe the exponent): Mul(x,y) = (1 + xm) · 2^xe · (1 + ym) · 2^ye = (1 + xm + ym + xm · ym) · 2^(xe+ye)

💡 L-Mul cleverly approximates this as: L-Mul(x,y) = (1 + xm + ym + 2^-l(m)) · 2^(xe+ye), eliminating the costly xm · ym term

🔧 l(m) term is adaptively set based on mantissa size for optimal accuracy

📊 Benchmarks on the Llama-3.1-8B-Instruct model show L-Mul preserves precision across various NLP tasks, with performance nearly identical to full BFloat16 precision

💬 Authors claim: "We can achieve the same model inference performance while reducing the energy cost of attention computations by 80%."

This breakthrough is still theoretical and would need implementation on dedicated hardware to confirm real-world gains, but it’s a really exciting path for more sustainable AI! 🌱

I got confused by the "no rounding needed" claim

Let's say x = 1.5, y = 1.75 then the result should be
$$(1+x_m+y_m)\times 2^{x_e+y_e}=(1+0.5+0.75)\times 2^{0+0}=2.25\times 2^0=1.125\times 2^1(1\; +\; x\_m\; +\; y\_m)\; \backslash times\; 2^\{x\_e\; +\; y\_e\}\; =\; (1\; +\; 0.5\; +\; 0.75)\; \backslash times\; 2^\{0\; +\; 0\}\; =\; 2.25\; \backslash times\; 2^0\; =\; 1.125\; \backslash times\; 2^1$$
Consider the uint add without rounding: x_m = (10...0)_2, y_m = (11...0)_2
x_m + y_m = (01...0)_2 with carry 1
new mantissa (01...0)_2 represents 0.25
the exponent add by carry 1
Thus, the result without rounding will be
$$1.25\times 2^{1}1.25\; \backslash times\; 2^1$$

I think a right shift is required.