Neural Nebula: Multi-Layer Perceptrons

High in the upper atmosphere of the Archipelago lies the Neural Nebula. Single-layer decision boundaries fail here because signal streams are entangled in non-linearly separable XOR patterns.

MISSION OBJECTIVES:

Configure 2 hidden layer neurons ($h_1, h_2$) using non-linear ReLU activations.
Combine the two linear half-spaces at output neuron $y$.
Achieve 100% accuracy across all 4 XOR truth table signals.

THEORY NOTE: THE XOR PROBLEM & HIDDEN LAYERS

In 1969, Minsky & Papert proved that a single Perceptron cannot solve XOR ($0,0\to 0$, $0,1\to 1$, $1,0\to 1$, $1,1\to 0$) because no single straight line can separate the points.

y = Step( wOut1 · ReLU(W₁ x + b₁) + wOut2 · ReLU(W₂ x + b₂) + bOut )

MULTI-LAYER PERCEPTRON (MLP) CALIBRATOR

Non-Linear XOR Hidden Layer Resolver

ACTIVATION FUNCTION:

HIDDEN NEURON H1 (Half-Space 1)h1 = ReLU(1.5x1 + -1.5x2 + -0.5)

w11: 1.5

w12: -1.5

b1: -0.5

HIDDEN NEURON H2 (Half-Space 2)h2 = ReLU(-1.5x1 + 1.5x2 + -0.5)

w21: -1.5

w22: 1.5

b2: -0.5

OUTPUT NEURON Y (Combiner)y = Step(1.5h1 + 1.5h2 + -0.5)

wOut1: 1.5

wOut2: 1.5

bOut: -0.5

XOR SIGNAL MATRIX CHECKLIST100% ACCURACY

x1	x2	h1	h2	Expected	Pred	Status
0	0	0.0	0.0	0	0	✓
0	1	0.0	1.0	1	1	✓
1	0	1.0	0.0	1	1	✓
1	1	0.0	0.0	0	0	✓