Biological Inspiration: The Neuron

While Artificial Neural Networks (ANNs) are fundamentally mathematical models implemented in software, their original conception drew significant inspiration from the structure and function of the brain's primary processing unit: the biological neuron. Understanding this biological counterpart helps appreciate the design choices made in early ANNs and provides a useful mental model, even though modern deep learning has evolved significantly.

The Brain's Processing Unit

The human brain contains billions of neurons, forming an extensive, interconnected network. Each neuron acts as a tiny information processor. Though biochemically complex, we can simplify its function for our purposes. A typical neuron consists of three main parts involved in signal transmission:

1. Dendrites: These tree-like branches act as the input channels. They receive electrochemical signals from other neurons through connections called synapses.
2. Soma (Cell Body): The central part of the neuron. It collects and sums the incoming signals received by the dendrites. If the combined signal strength surpasses a certain activation threshold within a short period, the soma generates an electrical pulse.
3. Axon: A long, cable-like projection that transmits the electrical pulse (the "output signal") away from the soma. The axon terminates in branches that form synapses with the dendrites of other neurons, passing the signal along.

Synapses are the junctions between neurons. They are not just passive connections; the strength or "weight" of a synapse determines how much influence the signal from the presynaptic (sending) neuron has on the postsynaptic (receiving) neuron. This synaptic strength can change over time, which is believed to be the basis of learning and memory in the brain.

In essence, a biological neuron receives multiple weighted inputs (signals modulated by synaptic strength), integrates them in the cell body, and if the total input exceeds a threshold, it "fires," sending a signal down its axon to potentially activate other neurons.

A simplified comparison between a biological neuron's components and the elements of an artificial neuron.

From Biology to Computation

The artificial neuron, which we will define mathematically in the next section, is a highly simplified abstraction inspired by this biological process:

• Inputs: Correspond to signals received by dendrites.
• Weights: Analogous to synaptic strength, determining the influence of each input.
• Summation: Mimics the integration of signals in the soma.
• Activation Function: Represents the threshold firing mechanism of the soma. If the summed weighted input exceeds a certain value, the artificial neuron produces an output signal.
• Output: Corresponds to the signal transmitted down the axon.

It's important to remember that this is an inspiration, not a direct replica. Artificial neural networks do not model the complex electrochemical dynamics, neurotransmitters, precise timing of spikes, or the intricate 3D structure of the brain. They capture a core computational principle: combining weighted inputs and applying a non-linear activation. This simplification makes ANNs computationally feasible and effective for solving complex problems, even if the underlying mechanism differs significantly from biological reality. Understanding this origin provides context as we move towards the mathematical formulation of the artificial neuron.