New exponent functions that make SiLU and SoftMax 2x faster, at full accuracy(github.com)
368 points by weinzierl | 72 comments
GitHub pull request to llama.cpp by jart that upstreams llamafile's vectorized expf() functions to compute SiLU and softmax faster and more accurately on CPU.