http://www.manifestdensity.net/2007/04/18/two-neuroscience-links/ Just another WordPress weblog Wed, 08 Feb 2012 17:42:35 +0000 hourly 1 http://wordpress.org/?v=3.3 http://www.manifestdensity.net/2007/04/18/two-neuroscience-links/comment-page-1/#comment-569 son1 Wed, 18 Apr 2007 19:21:17 +0000 http://127.0.0.1:8888/?p=206#comment-569 I don't think that neural networks are 'failed AI technology', really (although maybe the Numenta people think so, who knows). As I understand it, the issue is really that a lot of neural-network and perceptron-like learning machines are really just poor or obscure implementations of 'gaussian processes.' For instance, see <a href="http://citeseer.ist.psu.edu/mackay97gaussian.html" rel="nofollow">here</a> and <a href="http://www.inference.phy.cam.ac.uk/mackay/abstracts/gpB.html" rel="nofollow">here</a>; <a href="http://www.gaussianprocess.org/" rel="nofollow">this</a> is a good index site on GPs. Gaussian processes have their own strengths and weaknesses -- plenty of people use them (and similar 'nonparametric' techniques) today. NNs just got a bit of a bad rap because all the "it's like a brain!" hype sort of obscured the fact that they were special cases of a more general framework that was already well-understood. But once you leave behind all the brain analogies, and understand the math behind them, they're a reasonable technology to use for some approaches/problems. This also isn't a knock against a lot of 'connectionist' approaches to learning -- plenty of those methods are still popular and in-use today. Numenta's just an example of someone trying to commercialize one of those approaches. It smells a bit fishy to <i>me</i>, but I could easily be wrong... I don’t think that neural networks are ‘failed AI technology’, really (although maybe the Numenta people think so, who knows).
As I understand it, the issue is really that a lot of neural-network and perceptron-like learning machines are really just poor or obscure implementations of ‘gaussian processes.’
For instance, see here and here; this is a good index site on GPs.
Gaussian processes have their own strengths and weaknesses — plenty of people use them (and similar ‘nonparametric’ techniques) today. NNs just got a bit of a bad rap because all the “it’s like a brain!” hype sort of obscured the fact that they were special cases of a more general framework that was already well-understood. But once you leave behind all the brain analogies, and understand the math behind them, they’re a reasonable technology to use for some approaches/problems.
This also isn’t a knock against a lot of ‘connectionist’ approaches to learning — plenty of those methods are still popular and in-use today. Numenta’s just an example of someone trying to commercialize one of those approaches. It smells a bit fishy to me, but I could easily be wrong…

]]>