In the following illustration we render the gradient of some $loss(x,y)$ with respect to $x$ where
⋅ $x∈ℝ^2$ is the value to be optimized,
⋅ $y∈ℝ^2$ is the target value (the target is controller by the mouse.
grid size
amplitude
noise amplitude
noise frequence space
noise frequence time
display mode
motion length
motion speed
color (phase=|loss|)
color speed
$MSE(x,y) = mean( ⟨x-y|x-y⟩ )$
mouse controls $y$, gradient (wrt $x$) for all $x$
same but this is $log(MSE(x,y))$
$CosSim(x,y) = \frac{⟨x|y⟩}{|x||y|}$, we must minimize $1-CosSim(x,y)$.
mouse controls $y$, gradient (wrt $x$) for all $x$
same but this is $log(1-CosSim(x,y))$
$DotSim(x,y) = \frac{⟨x|y⟩}{|y||y|} = \frac{⟨x|y⟩}{⟨y|y⟩}$
mouse controls $y$, gradient (wrt $x$) for all $x$
mouse controls $x$, gradient (wrt $y$) for all $y$
$GenSim(x,y) = \frac{⟨x|y⟩}{|x|^α|y^{2-α}|}$ with $α∈[0,2]$ generalizes $DotSim$ and $CosSim$
we minimize $MSE(GenSim(x,y)_α,GenSim(y,y)_α)$
α
mouse controls $y$, gradient (wrt $x$) for all $x$
mouse controls $x$, gradient (wrt $y$) for all $y$
Let's mix both previous losses : $SumSim = 1 - CosSim(x,y) + MSE(1,DotSim(x,y))$
mouse controls $y$, gradient (wrt $x$) for all $x$
mouse controls $x$, gradient (wrt $y$) for all $y$
$1 - CosSim(x,y) + MSE(x,y)$
$1 - CosSim(x,y) + MSE(1.,DotSim(x,y))$