I am currently taking a course called "Introduction to Machine Learning with ENCOG 3", and I have a question about how well the Artificial Intelligence (AI) algorithm for a "neural network" corresponds with how an actual neuron works.
In the course, they model a neuron like this: 
x1, x2, etc. are voltage inputs, the wij are weights. The inputs are multiplied by these weights and summed up by the neuron. The neuron then has an "activation function" which then takes the sum of weighted inputs and calculates an output, typically a value between 0 and 1 (or between -1 and 1). You can think of the wij as representing dendrites (a higher weight means a more dense and thus conductive dendrite), and the output of the activation function as the voltage that gets sent down an axon.
The AI neural network algorithm creates a kind of intelligence by modifying the weights (wij shown in the picture).
My first questions is: Is this a good approximation as to how neurons actually work? That is, do our neurons "learn" by changing the weights (dendrite density, conductivity)? Or, is there some other mechanism that is more important (e.g. do neurons learn by changing their activation or summation functions?)
My second question is: If neurons really do learn by changing the density of dendrites, then how fast does this happen? Is this a fast process like DNA replication? Does the neuron quickly generate (or decrease) dendrite density when it receives some kind of biochemical signal that it needs to learn now?
I understand that much of this might not yet be known, but would like to know how well the AI algorithm corresponds with current theories on biological neural networks.
Answer
With respect to your first question, that model isn't intended to take time into account, but is based on Hebbian learning with the goal of computability. It's generally used in simple pattern recognition situations where each case has no bearing on the next. The learning portion is performed ahead of time during the training phase. For example, a deterministic perceptron isn't permitted to change after training. In contrast, the Biological Neuron Model is much more complex, and uses a variety of ordinary differential equations to integrate the cumulative behavior of neurons over time. As such, those models are non-deterministic and don't see as much practical use outside of experimentation.
To address your second question, neurons themselves don't "learn." A single neuron is essentially meaningless. Learning is an emergent process created by the interaction of several systems at once. While one influencing factor is connectivity (zero for no connection, non-zero for the inhibitory or excitatory connection which emulates both synaptic and non-synaptic), what you might call short term learning can also be performed by existing clusters of neurons, without any change in connectivity. Biologically, this is what must occur before any of the comparably slow process of tissue remodelling can take place, and the computationally equivalent process is only possible in time-aware models like the aforementioned Biological Neuron Model.
Take, for example, someone who wishes to learn to play guitar. When they begin playing, existing clusters emulate the desired behavior as best as they can. These neurons act as the functional scaffold that initiates and drives the neuroplastic process. Playing becomes easier because this scaffold becomes more efficient as new connections (shortcuts) are created, existing connections are strengthened, and irrelevant connections are inhibited. The improved scaffold in turn allows further improvements. Newborn neurons may also migrate to the area, though the how, why, and when of that process is unclear to me. This "behavior emulation" or "short term learning" process used in practicing the guitar, or whenever a novel situation is encountered, must be primarily governed by excitatory and inhibitory neurons' influence. Otherwise the whole process cannot even begin.
Further reading
No comments:
Post a Comment