I've been doing some reading about artificial intelligence recently and have been fascinated by some of the advances in the field.  Many of the white papers and journal articles are difficult to get through and require some advanced math to fully understand, so I spent some time looking for some beginner articles.  I also needed to limit my scope to start, so I wanted a single topic.  My workplace recently received some copiers that do a not horrible job of OCR, so I focused in on Neural Networks and how they are created, trained, and used.  One atricle that was probably the most helpful for me was from AI Junkie and can be found here.  In fact, if you wanted to just read that and skip this I would totally understand.
I learn best by doing, so I decided to write my own Neural Network and genetic algorithm to train it.  The experience has been an enlightening one.  I also am learning Haxe with a focus on HaxeFlixel, so that was my language and my target platform, mostly so I could play with it in a browser.


My goals for this application were:
  • Write a neural network
  • Write a genetic algorithm
  • Write a demo app that I could distribute to teach others and have a real life example

Neurons: Building blocks of the neural network

Neural networks are actually very simple.  The smallest element of a neural network is a Neuron.

A neuron simply takes a number of inputs, decides how important each input is by multiplying it by a weight (more on this later), adds everything up, and passes it on as an output.  A neuron doesn't know what the data it is getting is or what the output will be used for.  It doesn't need to. The intelligence of a neural network isn't in any of the individual neurons.  The intelligence comes out of the interactions between them.
Here's the guts of the code for my neuron.  It has other functions to create and modify weights, but this is the important part to see how it works:
class Neuron
public  var weights:Array<Float>;
public function fire(inputs:Array<Float>):Float {
var out:Float = 0;
if (inputs.length == inputs.length) {
for (i in 0...inputs.length)
out += inputs[i] * weights[i];
trace("Error with neuron");
return out;
The guts are in the function called fire.  It takes a number of inputs, multiplies them all by a different weight, and then adds them up and returns them.  So if I had a neuron with 3 inputs (1, 2, 3) and weights of .5, it would go (1 * .5) + (2 * .5) + (3 * .5) = 3.  So the output would be 3.
On its own, this neuron doesn't help the computer recognize my handwriting at all which is the goal of this whole nonsense, but it will get me there.  It is how we will organize the neurons that will get us the solutions we need.
Read Part 2 here.

Add comment

Security code