Visualised behaviour of dynamic neural networks
27th July, 2017The dynamics of simple nonlinear systems can be dramatically beautiful. I visualised a large number of randomly-generated systems as an exploration.
The dynamics of simple nonlinear systems can be dramatically beautiful. I visualised a large number of randomly-generated systems as an exploration.
In the last post, we saw that spike generation can cause major issues with gradient descent, by completely zeroing the gradients. We also learned how to get around this issue by faking the neuron output by making a spiking surrogate.
In the last post, we learned about how a spiking neuron differs from a common-or-garden Artificial Neuron. TL;DR: Spiking neurons understand time and have internal dynamics, Artificial Neurons don’t.