djete: July 2013

Google X's neural network

At a time when Research & Development divisions are frequently seen as luxury for companies, Google had established one of them a few years ago in the most secret way possible. Google X Lab, the young R&D division of Google, is charged with confidential projects such as Google Glass or driverless cars, and only little information is filtered. Even though those two projects sound very cool and will probably be part of our future, I would like to bring your attention to another project which is the Google Brain project. It consists of creating large scale artificial neural networks, exploiting the ridiculous amount of processing power that Google owns.

Using 1000 machines composed of 16 cores each, Google X Lab created an unsupervised self-learning neural network [1] which was fed with pictures extracted from Youtube videos. After training, the neural network managed to categorise images such as human faces or cats with a satisfying percentage of success. In other words, Google X’s neural network was able to create new concepts without any previous data or human intervention during the learning process, which could mean that the artificial neural network was able to generate new knowledge by itself and somehow “understand” its environment. Of course, this kind of conclusion can sound a bit hasty and ambitious but the results are quite surprising, considering that a normal human brain has around 100 billion neurons for 100 trillion synapses while this network had only 1 million synapses.

It is important to emphasise the fact that the neural network was asked to learn from visual inputs while it would have been easier to make it learn some concepts from audio inputs, which would lead to the possibility of translating concepts from a language to another, instead of translating literally. Nonetheless, Google X Lab is now using the same neural network for speech recognition system so we can expect them to work on such an idea.

We could then wonder what the result would be if they could build a neural network composed of 100 trillion synapses, even though it is true that the cognitive model of artificial neurons does not properly match that of biological neurons.