fc(1) fc(4) fc(16) Identity GELU ReLU RReLU PReLU SiLU ELU SELU GELU CELU Mish Softplus Hardswish LeakyReLU LeakyReLU(0.1) LeakyReLU(0.5) LeakyReLU(5.0) Softsign Tanh Hardtanh sigmoid Hardsigmoid Softshrink Hardshrink Tanhshrink abs atan tan sinc sin sin(πx) ExpLog IdGELU IdGELUsin Idsin Softmax Softmin LogSigmoid LogSoftmax exp log sqrt x^2 0 0.5 0 1