Neural Networks

I don’t know if it’s just me, but all things I study start converging into a single monolith. If I have studied 10 different ideas then you can be absolutely that I would have found some sort of skeleton architecture into which all of them would fit more or less seamlessly. It is an obsession with me to build these structures of unified thought.

Given this, you will understand why going through IEEE transactions on Neural Networks absolutely disgusts me. Make no mistake; I think IEEE is a great organization which has done a lot to further the interests of science and technology. But what passes as research in these transactions is so inherently inimical to all that I seek and desire from study that it becomes difficult to digest.

The problem, as far as I am concerned, is that all these brilliant people from all over the world keep confusing theory with reality. Mathematical modeling of all phenomena is fine thing and it helps us gain a better understanding of systems, but only as long as you don’t start taking the m0odel for the real thing. Going through any of the IEEE documents on the subject I find a plethora of such arcane variations on such obscure schemes that I have to stop and ask myself after every 2 pages-“Ok, so what are we trying to do here?”

The avowed aim of the study of neural networks is to understand and implement systems which emulate the working of the human brain. The first few generations seem to have worked with this definition in mind. Concepts like Perceptron, Hebbian Learning, Adalines are a product of this line of thought. They correspond to an actual mapping between the real brain processes and the mathematical models being developed. With the advent of recurrent networks, however, the field has been abandoned to mathematical chaos.

When half of today’s journals are filled with description of n different kinds of Neural Network schemes, each more convoluted than the previous one and the other half with schemes to get these schemes to work in a better way, we have to enquire about the vision guiding these efforts. Have these guys ever talked to a biology student to know about the processes that they are trying to model. How is the ‘Radial basis function NN’ or the ‘Epoch based real time recurrent NN’ suppose to carry out the tasks of the brain if its makers have no idea of how things happen in the brain. What they is a lot of complicated equations which have only an incidental relation with the real entity they are trying to model.

The human is a generic machine. Although the tasks are divided among the different parts of the brain in a general sort of way (e.g. listening recall etc.), these individual parts are themselves generically structured to carry out the tasks pertaining to their domain. In the hearing part of the brain, the whole part is responsible for all the tasks given to it. This must be so because of the way we evolve. Evolution is not in parts but as a whole. Our bodies did not evolve different parts in an independent fashion to satisfy each of our requirements. Had this been so, the distinct division of brain into domains of expertise would not have occurred. What I believe happens is that first a primitive neural structure evolves, which takes on more and more complexity as more and more diverse tasks are given to it.

So in the part of our brain which handles visual acuity, there aren’t 10 different architectures to identify 10 different poses or directions of illumination. The process, I believe is more cogent and cohesive than that.

So the aim of our research should be to find generic models or principles which have their corresponding counterparts in human brain and on a firmer classification and understanding of the tasks that the brain must handle. Once this classification is done, we can undertake the modeling of the various processes.

Any mathematical models without clear biological grounding can not come under the purview of neural networks because although they use a weighted added (the neuron), the paradigm they are employing has no well defined relation with the brain. Such systems are not concepts-they are heuristics. This is not to say that they are not useful for certain tasks, but through a blind pursuit of them, we lose direction in the quest to find true AI.