Tanh shows up everywhere in neural networks and audio processing. That smooth S-shaped function maps any real number into the range between -1 and 1. Standard library implementations give you accuracy, but they're computationally expensive. When inference calls tanh millions of times per forward pass, or audio processing demands real-time performance at 44.1 kHz sample rates, that cost adds up fast.
A technical survey by J. Tom Schroeder details four approximation strategies with Rust implementations. The fastest option grabs just the first several terms of a Taylor series expansion. You lose precision but gain speed. For better accuracy, Padé approximants divide one polynomial by another. The JUCE audio framework relies on a [7/6] variant with a 7th-degree numerator and 6th-degree denominator, and the survey adapts this in code. Splines take yet another approach, splitting the input range into subintervals where each segment gets its own cubic polynomial with tuned coefficients, based on work by Simos and Tsitouras.
The wildest technique bypasses floating-point math entirely. K-TanH, a hardware-efficient algorithm from a paper on deep learning optimization, manipulates IEEE-754 floating-point bits directly using only integer operations and a 512-bit lookup table. It reads the exponent and mantissa bits of the input, indexes into a table of 32 pre-computed parameters, then reconstructs the output through bit shifts and additions. The lookup table fits in a single AVX-512 register. Research from Nicol Schraudolph explores similar bitwise manipulation approaches.
These optimizations aren't academic exercises. Frameworks like llama.cpp and bitsandbytes rely on lookup tables for tanh and other non-linear activations when running quantized models on edge hardware. Low-precision formats like INT4 struggle to represent both the linear region and saturation tails of tanh, so quantization-aware training often learns specific clipping ranges to prevent accuracy degradation. When you're trying to run a model locally, every microsecond saved on activation functions matters.