Popular Posts

Wednesday, June 24, 2015

Idea of modeling non-linearity as learning distrtibution over function spaces in feedforward neural network

I was thinking about relationship in graphical context of graphical models and feedforward neural network. On one hand, Feedforward Neural Network is a graph of deterministic functions and on the other hand, Graphical models are graph of dependence of random variable which are uncertain. Then I thought what if deterministic non-linearities can be replace with random process which generates functions and shared non-linearity can be inferred.

An interesting idea would be to learn a distribution over function space(which will be used as non-linearity in Feedforward Neural Networks) jointly with backpropagation in an EM like fashion.

To summarize, We want to replace the deterministic function with a learned function by modeling the distribution over function space and inferring the shared non-linearity in neural network.

No comments:

Post a Comment