Oops, you're a little early!

We're still working on this page. But in a few days, you will find our latest findings regarding a new state-of-the-art FFN activation function outperforming SwiGLU. Sign up below and we'll let you know first!