Learn How To Build A Provider Framework For .NET

• Download source files - 100 Kb

Contents

1. The Problem
2. The Solution
3. So, The Framework
4. Designing the Framework
5. Implementing The Framework
   • Creating the IPublisher Interface
   • Creating the PublisherFactory
   • Creating the PublisherLoader
6. Developing A Simple Plug-in
7. An Application To Test The Framework
8. Design Variations
9. Conclusion

Solution Architect: We should start concentrating more on frameworks!!

Dumb Developer: Well, I think I know about .NET framework. You mean, something like that?

After reading this article, you'll be able to

• Change your mindset a little bit, and start thinking about 'frameworks' instead of just 'code'
• Understand a lot about practically applying provider pattern in your projects
• Gain much knowledge regarding how xml configuration files and providers work (especially in context of .NET)

1) The Problem

You are the Solution Architect of a BIG credit card company. You need to publish your credit card data to various banks, in order to get it approved. Basically, you send the card information (say, as an XML file) to a bank, the bank may or may not approve each card request, and you need to get the result back. The format you need to publish your card data is different from bank to bank, and you need a separate interfacing standard for each bank. Some banks support only FTP, and some other banks support only web services - and so on. Basically,

• You want to fetch your card data from a data source
• Convert it to the bank specific format (like, an XML file or a CSV file)
• Push it to a bank for processing
• Synchronously or asynchronously pull it back/get the result
• Save the result for each card in your data base

You may do this process based on a scheduler - because you need to send card data to banks few times a day (once or twice), and to pull the results back.

And to make matters more interesting, more banks will be added to your system in future.

Well, now you need to propose a solution. Interesting.. huh?

2) The Solution

Let us see what all solutions are not good enough.

• You should not think about developing your own scheduler - because you can use an existing scheduler to run your application to push/pull data to/from each banks.
• You should not think about developing a separate application for synchronizing with each bank - because it is difficult to create, maintain, and deploy.

Let us see what all decisions are good design decisions.

• You may develop an application to communicate with all banks. The application will load a list of plug-ins to push/pull data to/from banks from a configuration file
• Your application can be scheduled using Windows Scheduler the way you like.

So finally, let us decide to

• Build a 'Publisher' application, with some capability to load plug-ins and push/pull data

Now, you spoke with some of your friends and they asked you.. "Oh dude! Why you can't make it a generic interfacing Framework, so that we can re-use it?" - "Quite a good idea" you feel .. "But man, how exactly I make it a Framework?" :|

There we are :). Let us see.

3) So, The Framework

Basically a framework is

• Something you can re-use (Give me the ready-made pack, I'll use it. Or else, get away).
• A set of libraries and support programs (like, the .NET framework)
• A set of pre-solved problems, to help you build solutions faster..

Solution Architect 1: "The day we complete building frameworks for all human work flows happening in this world, Solution Architects may go out of job".

Solution Architect 2: "No, Still we can design and build frameworks to fix our existing frameworks".

Solution Architect 3: "Dude, What about a framework to build frameworks?. Probably a meta-meta framework or a meta-meta-meta framework?".

Dumb Programmer (Thinks) : "All solution architects are dumb, lazy and crazy. They just draw diagrams and talk Greek. The whole project is in my head".

Let us come back to our topic. How to create a framework for solving the above design issues?. Here are some final thoughts regarding the framework we are going to build.

• There should be a 'Publisher' framework to load plug-ins
• The Publisher will call certain functions in each plug-in (say, Put and Get ) to push the data and to pull the data
• A simple front end application will use the Publisher framework to load plug-ins from an XML file, and to push/pull data
• We may use windows scheduler to schedule the front end application.

Now, as you see, we need to load plug-ins, after reading the information from an xml configuration file. Essentially, we need to separate some kind of functionality from the actual application - i.e., the publishing logic (push/pull data to/from bank) should be decoupled from the actual application (loader/scheduler) through a plug-in model - so that we can add/remove and configure plug-ins at a later stage very easily.

Hence,

• When a new bank comes in, we can develop a simple plug-in for data push/pull and deploy it easily.
• Modify the configuration file, so that the loader will load it.

Well, this may ring the bell. You suddenly think.. "Separating functionality using a plug-in based model is nothing new - and an existing design pattern should be in existence for this". Well, that is what (actually, some what :-) ) provider pattern is all about. Let us see how we can design our framework, around the provider pattern, to solve the design issue.

If you are not so familiar with design patterns, you are lagging at least one step. Stop here, go and read my articles on 'Applying Design Patterns' - Part 1 and Part 2, Then come back.

4) Designing The Framework

Altogether, the idea about 'provider pattern' is quite simple. There are various ways of implementation - here is a quite simple one.

• You have a well defined plug-in interface - public interface IMyInterface , or even an abstract class, in your Framework project
• Your plug-in classes will implement this interface or abstract class. The trick is, your plug-in classes that implements your interface will be residing (most probably) in separate library files (say, dll files)
• At run time, your framework will read the list of available plug-ins from a configuration file (most probably, an XML file)
• You load those plug-in classes using Reflection mechanism (System.Reflection in .NET) and create an object of the class
• You can now call the methods inside your class, through your interface

Much like this.

• IPublisher is our interface in the Framework, which defines our abstract functions.
  • The Init function will be called initially when a publisher is instantiated, to pass the settings to the newly loaded publisher
  • Other functions like Get and Put are used for specific purposes - say, to push and pull data to/from the bank.
• PublisherFactory is our factory class - which takes two parameters as input (Don't confuse this with Factory pattern or something. Now, just consider it as a class name)
  • The assembly name (from where we should load the plug-in class)
  • The settings to pass to the publisher when it is loaded.

FTPPublisher and SOAPPublisher are two plug-in classes, which implements the IPublisher interface. KeyValueCollection is a set of name-value pair, to pass some settings (like Host=myhost, Port=30).

5) Implementing The Framework

Now the easiest part - implementing the framework (Unfortunately, most people start thinking from this phase only).

Credits to Sandeep (sandeepkumar@sumerusolutions.com) for implementing the actual code, based on the design

5.a) Creating The IPublisher Interface

As we discussed, the interface definition is pretty simple. The type SettingCollection is synonymous to KeyValue collection we discussed above.

/// <summary>
/// The interface 
for an IPublisher
/// </summary>
public interface IPublisher



    {




/// <summary>
/// To publish data
/// </summary>
void Put();
/// <summary>
/// To retreive data
/// </summary>
void Get();

/// <summary>
/// Function called 
by publisher factor to pass the settings (settings is a NameValue collection)
/// </summary>
/// <param name="settings">
/// </param>
void Init(Sumeru.Publisher.Framework.Data.SettingCollection settings);


}
  

5.b) Creating The PublisherFactory

The publisher factory has a very simple static function, CreatePublisher.

Publisher factory simply loads the type (i.e, our plug-in class) from the assembly, creates an object of it, and return it. Looks so simple, huh? If the type we loaded is not an IPublisher, an error may occur at the line ipub=(IPublisher) pub because explicit casting may fail. In this way, we make sure that all plug-ins we load should obey the IPublisher interface. It will also call the init function to pass the parameters to the plug-in after loading it.

/// <summary>
/// 
Summary description for PublisherFactory.
/// </summary>
public abstract 
class 
PublisherFactory
{
 
/// <summary>
/// Creates the 
publisher and return a type of IPublisher
/// </summary>
/// <param name="assembly">The path to the assembly</param>
/// <param name="className">The class name to instantiate</param>
/// <param name="settings">The settings collection</param>
/// <returns></returns>
public static IPublisher CreatePublisher(string assemblyName, string 
className,
                     
Sumeru.Publisher.Framework.Data.SettingCollection settings) 

{
try 

{

IPublisher ipub;

Assembly 
asm=Assembly.LoadFrom(assemblyName); 
System.Type 
type=asm.GetType(className);
object pub=Activator.CreateInstance(type); 

ipub=(IPublisher) pub;
ipub.Init(settings);
return ipub;
}
catch (Exception ex) 
{
throw 
ex;
}
} 


}

5.c) Creating The PublisherLoader Class

If you remember, one of our requirements were to load the plug-in information from an XML file. PublisherLoader class is exactly for that. This class is not there in the above UML diagram - pardon me, but still it is a part of the Framework. It will load an XML configuration file, load each plug-in specified, instantiate it by calling the PublisherFactory, and call other functions in the plug-in like Get() and Put(). Before we go to the class definition, here is some more meat regarding how to create your own xml configuration files and how to actually read them.

The XML Configuration File

For eg, consider that we should allow our Framework to load plug-ins by reading a configuration file like this.

<Publishers>
<!-- FTP Publisher -->
<Publisher AssemblyName="Sumeru.Publisher.FTPPublisher.dll" 
                
ClassName="Sumeru.Publisher.FTPPublisher.FTPPublisherPlugin" Name="FTP">
<Setting Key="Host" Value="myhost1"/>
<Setting Key="Port" Value="21"/>
<Setting Key="Password" Value=""/>
<Setting Key="UserName" Value=""/>
<Setting Key="FileToDownload" 
Value=""/>
<Setting Key="DownloadToPath" 
Value=""/>
<Setting Key="FileToUpload" 
Value=""/>
</Publisher> 

<!-- SOAP Publisher -->
<Publisher AssemblyName="Sumeru.Publisher.SOAPPublisher.dll" 

               
ClassName="Sumeru.Publisher.SOAPPublisher.SOAPPublisherPlugin" Name="SOAP">
<Setting Key="URL" Value="http://www.myhost.asmx"/>
<Setting Key="Port" Value="80"/>
</Publisher> 


</Publishers> 

Each Publisher has various properties like AssemblyName and ClassName for specifying the plug-in assembly path (dll) and the name of the plug-in class. Also, each publisher has a set of settings, like Host and Port. These settings are passed to the plug-in class by the PublisherFactory class when we instantiate an object of the plug-in, by calling the Init function. How this works? Read on.

How to read a configuration file like above? Fortunately it is very simple in .NET. The steps are,

<ul> <li>Create an XML schema to describe the configuration file format (I.e, to describe the elements and attributes in your configuration file) </li> <li>Create a set of classes from the XML Schema (to serialize and de-serialize an xml document specific to this schema) </li> <li>Use these classes to read a configuration file like above, to an object model. </li> </ul>

Creating The XML Schema

So, first let us create the XML schema.

<?xml version="1.0" encoding="utf-8" ?>
<xs:schema id="Publisher" 
targetNamespace="http://tempuri.org/Publisher.xsd" elementFormDefault="qualified"
xmlns="http://tempuri.org/Publisher.xsd" xmlns:mstns="http://tempuri.org/Publisher.xsd"
xmlns:xs="http://www.w3.org/2001/XMLSchema">


<xs:element name="Publishers">
<xs:complexType>
<xs:sequence>
    <xs:element ref="Publisher" maxOccurs="unbounded" />
</xs:sequence>
    
<xs:attribute name="Version" type="xs:string" />
</xs:complexType>
</xs:element>



<xs:element name="Publisher">
<xs:complexType>
<xs:sequence>
    <xs:element ref="Setting" maxOccurs="unbounded" />
</xs:sequence>
    <xs:attribute name="Name" type="xs:string" />
    <xs:attribute name="AssemblyName" type="xs:string" />
    <xs:attribute name="ClassName" type="xs:string" />
    <xs:attribute name="Version" type="xs:string" />
</xs:complexType>
</xs:element>



<xs:element name="Setting">
<xs:complexType>
<xs:sequence></xs:sequence>
    <xs:attribute name="Key" type="xs:string" />
    <xs:attribute name="Value" type="xs:string" />
</xs:complexType>
</xs:element>



</xs:schema>  

If you see the above schema, there are three elements, Publishers, Publisher and Setting. It is easy to understand, still, if you see, the Publishers element can contain more than on Publisher element. A Publisher element has four attributes - Name, AssemblyName, ClassName and Version - and a Publisher element can contain more than one Setting element. A Setting element has two attributes- a Key and a Value. This schema exactly describes the configuration document structure we need. Save the schema to a file, named Publisher.xsd

Creating A Set Of Classes

Now, to create a data structure out of our schema, I've several options. My favorite option is to use XsdObjectGenerator. It will generate a set of classes, which is capable of representing my above schema. I.e., it will create classes to create an object model of the schema, to which I can de-serialize my configuration file.

Let us create the classes from the command line, using xsdobjectgen.

xsdobjectgen Publisher.xsd /n:Sumeru.Publisher.Framework.Data

This will create an object model based on elements in Publisher.xsd (i.e, a set of classes like Publishers, Publisher, Setting etc along with attributes) to describe the elements in Publisher.xsd. The /n switch specifies the namespace for the newly created classes.

And this created a file Sumeru.Publisher.Framework.Data.cs with all the classes - please see the attached source code if required. There'll be a Sumeru.Publisher.Framework.Data.Publishers Class (If you remember, Publishers is the root element according to our schema), and we can use an object of this class to de-serialize an xml document.

Load The Configuration File

This part is pretty simple. We just create an object of Publishers class, and de serialize the xml file to it.

Sumeru.Publisher.Framework.Data.Publishers pubs;
XmlSerializer xsr=new XmlSerializer(typeof(Framework.Data.Publishers));
pubs=(Framework.Data.Publishers) xsr.Deserialize(new StreamReader("c:\config.xml"));

If you remember, Publishers element has a collection of Publisher element, and each Publisher element has few attributes like Class Name and Assembly Name.

We iterate the Publisher elements, to pass the class name and assembly name to the PublisherFactory we wrote earlier. This will dynamically load a publisher, and may return us an IPublisher object - if the assembly name and class name is specified correctly in the XML file. Then, we may call our custom functions

The PublisherLoader Class

The PublisherLoader class does all this work (though it is very simple)

/// <summary>
/// Summary description for PublisherLoader.
/// </summary>
public class PublisherLoader
{

/// <summary>
/// Load all the publishers from the configuration, and create an instance
/// using the factory. Also call our custom functions.
/// </summary>
/// <param name="path"></param>
public void LoadPublisher(string path) 
{
Framework.Data.Publishers pubs;
XmlSerializer xsr=new XmlSerializer(typeof(Framework.Data.Publishers));
pubs=(Framework.Data.Publishers) xsr.Deserialize(new StreamReader(path));
foreach (Framework.Data.Publisher p in pubs) 
{
Framework.IPublisher ipub=Framework.PublisherFactory.CreatePublisher(p.AssemblyName,
                                      p.ClassName,p.SettingCollection);
ipub.Get();
ipub.Put(); 
}
}
}

Here, as you see, we are loading the configuration, iterate each publisher element, and passes the class name and assembly name to the PublisherFactory for creating an IPublisher object. Then we are calling the functions Get() and Put().

If you notice, we are also passing the Setting collection of a publisher element to the PublisherFactory, and the PublisherFactory will in-turn pass this Setting collection by calling the Init(..) function in a Publisher object during the time of instantiation (See the code of PublisherFactory).

Now, we are almost done. Let us write a simple plug-in for our framework now.

6) Developing A Simple Plug-In

To develop a plug in, I've done the following steps.

• Created a class library project named Sumeru.Publisher.FTPPublisher
• Added a reference to our Framework project
• Created a class named FTPPublisherPlugin by implementing the IPublisher interface in our framework.

Here is the FTPPublisherPlugin Class. If you see, it just implements all the functions defined in IPublisher.

You may need to add your own code for putting and getting files to/from ftp server, in place of showing message boxes.

/// <summary>
/// A Simple Plugin Class
/// </summary>
public class FTPPublisherPlugin : Sumeru.Publisher.Framework.IPublisher 
{

private System.Collections.Hashtable ht=new System.Collections.Hashtable();

/// <summary>
/// To publish 
data
/// </summary>
public void Put() 
{
//Put your data
//Note: Implement proper error handling for invalid 
parameters
string host=(string)ht["Host"];
string port=(string)ht["Port"];
System.Windows.Forms.MessageBox.Show("Putting File To " + 
host + "," + port); 
//Write Your own ftp 
code to put file
}
/// <summary>
/// To retreive data
/// </summary>
public void Get() 
{
//Get your data
//Note: Implement proper error handling for invalid 
parameters
string host=(string)ht["Host"];
string port=(string)ht["Port"];
System.Windows.Forms.MessageBox.Show("Getting File from " 
+ host + "," + port); 

//Write Your own 
ftp code to get file
 
}

/// <summary>
/// Init Function called by publisher factory to pass the settings
/// </summary>
/// <param name="settings">
/// </param>
public void Init(Sumeru.Publisher.Framework.Data.SettingCollection settings)
{
//Keep our settings locally 
in a hash table
//Implement proper error handling for duplicate keys
foreach(Framework.Data.Setting s in settings) 
{
ht.Add(s.Key,s.Value);
}

}

}

7) An Application To Test The Framework

Now, let us develop a very simple Loader application, which consumes our Framework. It basically invokes the LoadPublisher function in PublisherLoader class we created earlier. I created a form, with a text box which has the path to the Xml configuration file.

The XML Configuration file has information to load our newly written Plug-in. I'm just loading the same plug-in twice with different settings. Try developing yet another plug-in, and add a new publisher element to load that. Make sure that you are copying the dll to the path of this Loader app - if you are not explicitly specifying the full path for the dll.

<?xml version="1.0" encoding="utf-8" ?>
<Publishers xmlns="http://tempuri.org/Publisher.xsd">
<!-- FTP 
Publisher - A simple ftp transaction to host 1 -->
<Publisher AssemblyName="Sumeru.Publisher.FTPPublisher.dll" 
ClassName="Sumeru.Publisher.FTPPublisher.FTPPublisherPlugin" 
Name="FTP">
<Setting Key="Host" Value="myhost1"/>
<Setting Key="Port" Value="21"/>
<Setting Key="Password" Value=""/>
<Setting Key="UserName" Value=""/>
<Setting Key="FileToDownload" Value=""/>
<Setting Key="DownloadToPath" Value=""/>
<Setting Key="FileToUpload" Value=""/>
</Publisher> 
<!-- FTP 
Publisher - A simple ftp transaction to host 2 -->
<Publisher AssemblyName="Sumeru.Publisher.FTPPublisher.dll" 
ClassName="Sumeru.Publisher.FTPPublisher.FTPPublisherPlugin" 
Name="FTP">
<Setting Key="Host" Value="myhost2"/>
<Setting Key="Port" Value="21"/>
<Setting Key="Password" Value=""/>
<Setting Key="UserName" Value=""/>
<Setting Key="FileToDownload" Value=""/>
<Setting Key="DownloadToPath" Value=""/>
<Setting Key="FileToUpload" Value=""/>
</Publisher> 
 
</Publishers>
  


In the Load button click, I have the code to load the plug-ins in this 
configuration file, like


private void btnLoad_Click(object sender, System.EventArgs e)
{
PublisherLoader pl=new PublisherLoader();
pl.LoadPublisher(this.txtFileToLoad.Text);
}
 

As you see, we are using the PublisherLoader class, to read the configuration file, and to load the plug-ins as explained earlier. Now, if you click Load button, the plug-ins will be loaded, and you may see the message boxes from the plug-in we developed. (Let us keep it simple ;)

---

8) Design Variations

Here are some interesting design variations so that you can implement for practice, and for some additional brainstorming.

• Yesterday I spend some time with Abhilash - and he was mentioning how he is planning to use a script engine in his provider-like framework. Instead of developing separate classes as plug-ins, there will be a core engine, and it will load script files which can be interpreted by the core engine from a configuration file. That framework is mainly for data mapping between protocols.
• Right now, the framework is not passing any data to the plug-ins (other than the settings). Some times (most of the times), this framework may need to pass data to/from plug-in functions (like, data to put and get). If the data passed from Framework to put/get functions are standardized (i.e., a pre-defined schema), everything is fine - but what if the data format is not pre-defined? How will you solve this issue? Post your comments here :)

---

9) Conclusion

Hope you enjoyed it - I tried to keep it very simple.

Some more information if you are health conscious.

• When you work with your comps, normally your entropy will go high (just if you are a smart programmer), and this may decrease the level of water in your body. Make sure that you are drinking lots of water - this will eliminate a lot of stress, strain and depression.

And finally, I'll also recommend an Art Of Living Part I workshop for you (See http://www.artofliving.org/ and have a look at the Courses). It is an interactive workshop of 18 hours spread over 6 days. As it did for me, I hope that it may help you to find the right balance between your work and life - to improve the mental strength and clarity of your mind, and to improve the quality of your life.

Here are some other popular articles I wrote.

• Design Your Soccer Engine, and Learn How To Apply Design Patterns (Observer, Decorator, Strategy and Builder Patterns) - Part I and II
• Design Your Soccer Engine, and Learn How To Apply Design Patterns (Observer, Decorator, Strategy and Builder Patterns) - Part III and IV
• Article Name: BrainNet Neural Network Library - Part I - Learn Neural Network Programming step by step And Develop a Simple Handwriting Detection System

You can visit my own website at http://amazedsaint.blogspot.com/ for more articles, projects and source code

Also, you may view my tech-bits blog , download my opersource projects. Also, if you have some time to read some insights, see my intuitions blog

Thanks a lot :)

---

BrainNet II - Creating A Neural Network Library

.NET

• Download source files and projects - 300 Kb

This article will explain the actual concepts of Backward Propagation Neural Networks - in such a way that even a person with zero knowledge in neural networks can understand the required theory and concepts very easily. The related project demonstrates the designing and implementation of a fully working 'BackProp' Neural Network library, i.e, the Brain Net library as I call it. You can find the theory, illustration and concepts here - along with the explanation of the neural network library project - in this article. Also, find the full source code of the library and related demo projects (a simple pattern detector, a hand writing detection pad, an xml based neural network processing language etc) in the associated zip file.

---

Contents

• 1. Overview
• 2. Before We Begin.
• 3. Understanding Neural Networks
  • 3.1 Biological Neurons
  • 3.2 Artificial Neurons
• 4. How A Neural Network Actually Works
  • 4.1. Training Phase
  • 4.2. Running The Network
• 5. Designing BrainNet Neural Network Library
  • 5.1. The UML Model
  • 5.2. A Neuron In BrainNet Library
  • 5.3. The Strategy Of A Neuron
  • 5.4. A Neural Network In BrainNet library
  • 5.5. Training The Network
  • 5.6. Running The Network
  • 5.7. Creating A Network
• What is Next
• Appendix A: Small Dose Of Spiritual Programming!!

---

1. Overview

• Solution Architect: "Well, you learned something about neural networks?"
• (Dumb?) Developer: "No, I'm smart enough. I love using other's code."
• Solution Architect: "But, if you don't understand the concepts, how you can optimize and re-use other's code?"
• (Dumb?) Developer: "Err.. I feel that most others can code better than me, so why should I optimize?"

In my previous article, the focus was on what a neural network can do. In this article, we will see what a neural network is, and how to create one yourself. I will go a little deeper. After reading this article, you will be able to

• Understand the basic theory behind neural networks (backward propagation neural networks in particular)
• Understand how neural networks actually 'work'
• Understand in more detail, the design and source code of BrainNet library.
• Understand in more detail, how to use BrainNet Library in your projects.
• Think about new possibilities of neural network programming
• Put forward some concepts to optimize and generalize BrainNet library.

Now, let me answer some questions I got in past.

• Q) Why you selected an object oriented programming model for this Neural Network Library?
  • Answer - The focus is on the understandability of basic concepts, not on performance.
• Q) Is this neural network library fully optimized?
  • Answer - Not yet, we are still in the beta stage. The focus is on readability, so the code is flattened so that even a beginner can understand it. Suggestions and modifications are always welcome. Send your modifications, hacks and suggestions to amazedsaint@gmail.com
• Q) Whether this library can be used in projects?
  • You can use it - as long as your usage confronts to the specifications in the associated license notice (see the source code). Anyway, I request you to send me a notification (and the modified code), if you hack it or use it in any of your projects.

2. Before We Begin.

This article is complete by itself. It explains what is a neural network, and how to create one your own. How ever, to get an idea regarding what a neural network can do, and to get a user level experience - please read the first part of this article.

The first article in this article series is titled "BrainNet Neural Network Library - Part I - Learn Neural Network Programming step by step And Develop a Simple Handwriting Detection System". You can read it and download the source code, either from Code Project [ Click Here ] or from my Website [ Click Here ].

If you are really a beginner, it will help you a lot, and may provide you a step by step approach towards understanding neural networks.

This is my second article about Neural Networks in general and the BrainNet Neural Network Library in particular. This article explains Neural Networks and their working in more detail, and in a very simple way. Then I will explain the design concepts of BrainNet library.

Tip - In this article, the theory about neural networks is explained in the most simplest (human readable) manner, so that even a person with zero background in neural networks can understand it. So, if you already know some theory about neural networks, you may consider skipping the theory part, and go ahead to the designing part which describes the design concepts of BrainNet library.

3. Understanding Neural Networks

One fascinating thing about artificial neural networks is that, they are mainly inspired by the human brain. This doesn't mean that Artificial Neural Networks are exact simulations of the biological neural networks inside our brain - because the actual working of human brain is still a mystery. The concept of artificial neural networks emerged in its present form our very limited understanding about our own brain ("I know that I know nothing").

Brain Net Neural Network library is designed and implemented using Object Oriented Concepts.

Alert - You can visit the articles section of my website here - http://amazedsaint-articles.blogspot.com/ - for reading other articles in this series. You can also see there a full tutorial about OOP concepts in .NET and VB.NET programming. This may help you to understand this article better - especially if you don't have a good understanding regarding the Object Oriented Programming concepts. Also, read about design patterns there.

Before understanding how neurons and neural networks actually work, let us revisit the structure of a neural network. As I mentioned earlier, a neural network consists of several layers, and each layer has a number of neurons in it. Neurons is one layer is connected to multiple or all neurons in the next layer. Input is fed to the neurons in input layer, and output is obtained from the neurons in the last layer.

Fig: A Fully Connected 4-4-2 neural network with 4 neurons in input layer, 4 neurons in hidden layer and 2 neurons in output layer.

An artificial neural network can learn from a set of samples.

For training a neural network, first you provide a set of inputs and outputs. For example, if you need a neural network to detect fractures from an X-Ray of a born, first you train the network with a number of samples. You provide an X-Ray, along with the information that whether that particular X-Ray has a fracture or not. After training the network a number of times with a number of samples like this (probably thousands of samples), it is assumed that the neural network can 'detect' whether a given X-Ray indicates a fracture in the born (This is just an example). The concept of training a network is detailed in my first article. Later, in this article, we will discuss the theory behind network learning.

As we already discussed, the basic component in a neural network is a neuron. First of all, let us have a very brief look towards biological neurons, and their corresponding artificial models.

3.1 Biological Neurons

First of all, let us have a look at a biological neuron. Frankly, I don't have much knowledge regarding the actual structure of a biological neuron - how ever, the following information is more than enough at this stage for us to get in to the groove. A biological neuron will look some what similar to this.

The four basic components of a biological neuron are

• Dendrites - Dendrites are hair like extensions of a neuron, and each dendrite can bring some input to the neuron (from neurons in the previous layer). These inputs are given to the soma.

• Soma - Soma is responsible for processing these inputs, and the output is provided to other neurons through the axon and synapses.

• Axon - The axon is responsible for carrying the output of soma to other neurons, through the synapses

• Synapses - Synapses of one neuron is connected to the dendrites of neurons in the next layer. The connections between neurons is possible because of synapses and dendrites.

A single neuron is connected to multiple neurons (mostly, all neurons) in the next layer. Also, a neuron in one layer can accept inputs from more than one neuron (mostly, all neurons) in the previous layer.

3.2 Artificial Neurons

Now, let us have a look at the model of an artificial neuron.

An artificial neuron consists of various inputs, much like the biological neuron. Instead of Soma and Axon, we have a summation unit and a transfer function unit. The output of one neuron can be given as input to multiple neurons.

Please note that for an artificial neuron, we have a weight value associated with each input. Now, let us have a look at the working of a neuron.

Summation Unit

• When inputs are fed to the neuron, the summation unit will initially find the net-value. For finding the Net Value, the product of each input value and corresponding connection weight is calculated.

• i.e, input value x(i) of each input to the neuron is multiplied with the associated connection weight w(i). In simplest case, these products are summed and fed to the transfer function. See the pseudo code below, it is simpler to understand.

Also, a neuron has a bias value, which affects the net value. A bias of a neuron is set to a random value, when the network is initialized. We will change the connection weights and bias of all neurons in the network (other than neurons in the input layer), during training phase.

I.e, if x is the input, and w is the associated weight, then pseudo code for net value calculation is as follows.

netValue=0

for i=0 to neuron.inputs.count-1
   netValue=netValue + x(i) * w(i)
next

netValue=netValue + Bias

Transfer Function

Transfer function is a simple function, that uses the net value to generate an output. This output is then propagated to the neurons in the next layer. We can use various types of transfer functions as shown below.

Hard Limit Transfer Function: For example, a simple hard limit function will output 1 if net value is greater than 0.5, and will output 0 if the net value is lesser than 0.5 - as shown.

if (netValue<0.5)
     output = 0
else
     output = 1

Sigmoid Transfer Function: Another type of transfer function is a sigmoid transfer function. A sigmoid transfer function will take a net value as input and produce an output between 0 and 1 as shown.

output = 1 / (1 + Exp(-netValue))

The implementation of summation unit and transfer function unit may vary in different networks.

This, a neural network is constructed from such basic models, called neurons, arranged together in layers, and connected to each other as explained earlier. Now let us see how all these neurons work together, inside a neural network.

4. How A Neural Network Actually 'Works'

Working with a neural network includes

• Training the network - by providing inputs and corresponding outputs.
  • In this phase, we train a neural network with samples to perform a particular task.
• Running the network - by providing the input to obtain the output.
  • In this phase, we will provide an input to the network, and obtain the output. The output may not be accurate always. Generally speaking, the accuracy of the output during running phase depends a lot on the samples we provided during the training phase, and the number of times we trained the network.

4.1. Training Phase

This section explains how the training takes place, in a back ward propagation neural network. In a backward propagation neural network, there are several layers, and each neuron in each layer is connected to all neurons in the next layer. For each connection, a random weight is assigned when the network is initialized. Also, a random bias value is assigned to each neuron during initialization.

Training is the process of adjusting the connection weights and bias of all neurons in the network (other than neurons in the input layer), to enable the network to produce expected output for all input sets.

Now, let us see how the training actually happens. Consider a small 2-2-1 network. Now, we are going to train this network with AND truth table. As you know, AND truth table is

AND TRUTH TABLE A B Output 0 0 0 0 1 0 1 0 0 1 1 1

Fig: A 2-2-1 Neural Network and Truth Table Of AND

In the above network, N1 and N2 are neurons in input layer, N3 and N4 are neurons in hidden layer, and N5 is the neuron in output layer. The inputs are fed to N1 and N2. Each neuron in each layer is connected to all neurons in next layer. We call the above network a 2-2-1 network, based on the number of neurons in each layer.

Tip - The concepts we are going to discuss here is largely biased towards a commonly used neural network model called Backward Propagation Neural Networks. How ever, you should understand that various other models also exist - like Counter Propagation Neural Networks, Kohanen's Self Organizing Maps etc.

The above diagram will be used to illustrate the process of training.

First, let us see how we train our 2-2-1 network, the first condition in the truth table, i.e, when A=0, B=0 then output=0.

Step 1 - Feeding The Inputs

Initially, we will feed the inputs to the neural network. This is done by simply setting the output of neurons in Layer 1, as the input values we need to feed. I.e, as per the above example, our inputs are 0,0 and output is 0. we will set the output of Neuron N1 as 0, and the output of N2 is set to 0.

Have a look at this pseudo code, and it will make things clear. Inputs is the input array. The number of elements in Input array should match the number of neurons in input layer.

       
i = 0
For Each neuron In InputLayer
    someNeuron.OutputValue = Inputs(i)
    i = i + 1
Next

Step 2 - Finding the output of the network

We have already seen how we calculate the output of a single neuron. As per our above example, the output of neurons N1 and N2 will act as the inputs of N3 and N4.

Finding the output of neural network involves, calculating the outputs of all hidden layers and output layer. As we discussed earlier, a neural network can have a number of hidden layers.

 'Find output of all neurons in all hidden layers
 For each layer in HiddenLayers
    For Each neuron In layer.Neurons
        neuron.UpdateOutput()
    Next
 Next

 'Find output of all neurons in output layer   
 For Each neuron In OutputLayer.Neurons
        neuron.UpdateOutput()
 Next

UpdateOutput() function of a single neuron works exactly as we discussed earlier. First, net value is calculated by the summation unit, and then it is provided to a transfer function to obtain the output of the neuron. Pseudo code is again shown below.

Summation Unit works like this:

Dim netValue As Single = bias

For Each InputNeuron connected to ThisNeuron
    netValue = netValue + (Weight Associated With InputNeuron * Output of InputNeuron)
Next

I.e, as per our above example, let us calculate the net value of neuron N3. We know that N1 and N2 are connected to N3

• Net Value Of N3 = N3.Bias + (N1.Output * Weight Of Connection From N1 to N3) + (N2.Output * Weight Of Connection From N2 to N3)

Similarly, to calculate the net value of N4,

• Net Value Of N4 = N4.Bias + (N1.Output * Weight Of Connection From N1 to N4) + (N2.Output * Weight Of Connection From N2 to N4)

Activation Unit Or Transfer Unit:

Now, let us see how we are generating the output, using Transfer unit. Here, we are using the sigmoid transfer function. This is exactly as we discussed earlier.

Output of Neuron = 1 / (1 + Exp( - NetValue )

Now, the output of N3 and N4 will be passed to each neuron in the next layer as inputs. This process of propagating the output of one layer as the input to the next layer is called forward propagation part in the training phase.

Thus, after step 2, we just found the output of each neuron in each layer - starting from the first hidden layer to the output layer. The output of the network is simply the output of all neurons in the output layer.

Step 3 - Calculating The Error or Delta

In this step, we will calculate the error of the network. Error or Delta can be stated as the difference between the expected output and the obtained output. For example, when we find the output value of the network for the first time, most probably the output will be wrong. We need to get 0 as the output for inputs A=0 and B=0. But the output may be, some other value like 0.55, based on the random values assigned to the bias and connection weights of each neuron.

Now let us see, how we can calculate the error. Let us see how to calculate the error or delta of each neuron in all the layers.

• First we will calculate the error or delta of each neuron in the output layer.
• The delta value thus calculated will be used to calculate the error or delta of neurons in the previous layer (i.e, the last hidden layer)
• The delta value of all neurons in the last hidden layer is used to calculate the error or delta of all neurons in the previous layer (i.e, second last hidden layer)
• This process is continued, till we reach the first hidden layer (delta of input layer is not calculated).

Please note one interesting point. In Step 2, we are propagating values forward - starting from the first hidden layer to the output layer, for finding the output. In Step 3, we are starting from the output layer, and propagating the error values backward - and hence, this neural network is called as a Backward Propagation neural network.

Time to see how things actually work. The general equation for finding the delta of a neuron is

Neuron.Delta = Neuron.Output * (1 - Neuron.Output) * ErrorFactor

Now, let us see how the error factor is calculated for each neuron. The Error Factor of neurons in output layer can be calculated directly (since we know the expected output of each neuron in output layer).

For a neuron in output layer,

ErrorFactor Of An Output Layer Neuron = ExpectedOutput - Neuron's Actual Output

i.e, with respect to our above example, if the output of N5 is 0.5 and the expected output is 0, then error factor = 0 - 0.5 = - 0.5

For a neuron in hidden layer, error factor calculation is some what different. To calculate the error factor of a neuron in hidden layer,

• First the delta of each neuron to which this neuron is connected is multiplied with the weight of this connection
• These products are summed up together to obtain the error factor of a hidden layer neuron

Simply speaking, a neuron in a hidden layer is using the delta of all connected neurons in next layer, along with the corresponding connection weights, to find the error factor. This is because, we don't have any direct parameters for calculating the error of neurons in the hidden layer (as we did in the output layer neurons).

Remember - To calculate the output of a neuron, we used the outputs of connected neurons in previous layer, along with the corresponding connection weights.

 'Calculating the error factor of a neuron in a hidden layer
      
 For Each Neuron N to which ThisNeuron Is Connected
   'Sum up all the delta * weight
   errorFactor = errorFactor + (N.DeltaValue * Weight Of Connection From ThisNeuron To N)
 Next

To illustrate this, consider a neuron x1 (ThisNeuron), which is a hidden layer neuron. X1 is connected to neurons y1, y2, y3 and y4 - and these are neurons in next layer.

i.e, to make things simple,

• Error Factor of X1 = (Y1.Delta * Weight Of Connection From X1 To Y1) + (Y2.Delta * Weight Of Connection From X1 To Y2) + (Y3.Delta * Weight Of Connection From X1 To Y3) + (Y4.Delta * Weight Of Connection From X4 To Y4)

Now, as we discussed earlier, the Delta of a X1 can be calculated as,

• X1.Delta = X1.Output * (1 - X1.Output) * ErrorFactor Of X1

Thus, after finishing step 3, we have the Delta of all neurons.

Step 4 - Adjusting The Weights and Bias

After calculating the delta of all neurons in all layers, we should correct the weights and bias with respect to the error or delta, to produce a more accurate output next time. Connection Weights and Bias, together are called free parameters. Remember that a neuron should update more than one number of weights - because, as we already discussed, there is a weight associated with each connection to a neuron.

See the pseudo code for updating the free parameters of all neurons in all layers

 'Update free parameters of all neurons in hidden layer
 For each layer in HiddenLayers
    For Each neuron In layer.Neurons
        neuron.UpdateFreeParams()
    Next
 Next

 'Update free parameters of all neurons in output layer   
 For Each neuron In OutputLayer.Neurons
        neuron.UpdateFreeParams()
 Next

UpdateFreeParams() function simply does two things.

• Find the new bias of a neuron, based on the delta we calculated above
• Update the connection weights based on the delta we calculated above

Finding the new bias value of a neuron is pretty simple. See the pseudo code. If Learning Rate is a constant (for e.g, Learning Rate=0.5)

New Bias Value = Old Bias Value + LEARNING_RATE * 1 * Delta

Now let us see how to update the connection weights. The new weight associated with an input neuron can be calculated as shown below.

New Weight  = Old Weight +  LEARNING_RATE * 1 * Output Of InputNeuron * Delta

As a neuron can have more than one input, the above step should be performed for all input neurons connected to this neuron.

I.e,

For Each InputNeuron N connected to ThisNeuron
    New Weight of N = Old Weight of N + LEARNING_RATE * 1 * N.Output * ThisNeuron.Delta
Next

Now, after step 4, we have a better network. This process is repeated for all other entries in the AND truth table - for probably more than thousand number of times, to train the network 'well'.

4.2. Running The Network

Running the network involves,

• Providing the inputs to the network exactly as described earlier in Step 1 above
• Calculating the outputs as explained in Step 2 above

How ever, it is important to note that the network should be trained with sufficient samples (and sufficient number of times), to obtain desired results. Anyway, it is almost impossible to say that the output of a neural network will be 100% accurate for any input.

Now, let us see how these concepts are implemented in BrainNet Neural Network Library.

5. Designing BrainNet Neural Network Library

The fundamental challenge for any solution developer is to create, build or assemble a working program from his abstract concepts about a system. The quality of this transformation depends a lot on how well he understand the system. At this point, I would like to mention that Brain Net Library is actually not designed after a complete and thorough understanding of various existing neural network models and emerging possibilities in the area of neural networks. Hence, I suspect that the present design of this framework is mostly biased towards Backward Propagation systems I explained earlier - though it can be modified to create other neural network models also.

We are simply mapping the above concepts to the library. Hence, the following code and explanation is very easy to understand, if you read the above concepts regarding Neural Networks.

5.1. The UML Model

Now, let us have a look at some of the interfaces and classes in BrainNet library.

Remember - If you need to brush up some Object Oriented Designing and UML concepts, have a look at my article regarding design patterns [ Click Here ].

Have a look at this model below. Please not that this model holds only the major interfaces and classes with in the model.

Fig: An Partial Model of BrainNet Framework

As we discussed earlier, a Neural Network consists of various Neuron Layers, and each Neuron Layer has various Neurons. A Neuron has a strategy - which decides how it should perform tasks like summation, activation, error calculation, bias adjustment, weight adjustment etc.

To brief the UML diagram above,

• INeuron, INeuronStrategy, INeuralNetwork and INetworkFactory are interfaces

• A Neuron should implement the INeuron interface

• A Neural Network should implement the INeuralNetwork interface

• A Neuron has a strategy, and a strategy should implement the INeuronStrategy interface. We have a concrete implementation of INeuronStrategy, called BackPropNeuronStrategy (for a backward propagation neural network).

• A Neural Network is initialized and connections betweens layers are made by a neural network factory. A Factory should implement the INetworkFactory interface. We have a concrete implementation of INetworkFactory, called BackPropNetworkFactory, for creating Backward Propagation neural networks.

The major interfaces in the model are briefed below.

INetworkFactory	An interface to define a neural network factory
INeuron	The interface for defining a neuron
INeuronStrategy	The interface for defining the strategy of the neuron
INeuralNetwork	The interface for defining a neural network

The major classes in the model are briefed below.

BackPropNeuronStrategy A backward propagation neuron strategy. This is a concrete implementation of INeuronStrategy NetworkHelper The class is to help the user to initialize and train the network. It maintains a list of training data elements. NeuralNetwork A generic neural network. This is a concrete implementation of INeuralNetwork NeuralNetworkCollection A collection of neural networks Neuron A concrete implementation of INeuron NeuronCollection A collection of INeurons NeuronConnections This is a hash table to keep track of all neurons connected to/from a neuron, along with the related weights

5.2. A Neuron In BrainNet Library

The INeuron interface provides an abstract interface that should be implemented to create a concrete neuron. I request you to refresh the concepts of an artificial neuron we discussed earlier.

The elements in INeuron interface is detailed below.

'The interface for defining a neuron 
Public Interface INeuron

    'The current bias this neuron
    Property BiasValue() As Single
    'The current output this neuron
    Property OutputValue() As Single
    'The current delta value this neuron
    Property DeltaValue() As Single
    'A list of neurons to which this neuron is connected
    ReadOnly Property ForwardConnections() As NeuronCollection
    'Gets a list of neurons connected to this neuron
    ReadOnly Property Inputs() As NeuronConnections
    'Gets or sets the strategy of this neuron
    Property Strategy() As INeuronStrategy
    'Method to update the output of a neuron
    Sub UpdateOutput()
    'Method to find new delta value
    Sub UpdateDelta(ByVal errorFactor As Single)
    'Method to update free parameters
    Sub UpdateFreeParams()

End Interface

A concrete neuron will implement the INeuron interface. Neuron class is a concrete implementation of INeuron. The Strategy property of a Neuron holds its current strategy. Inputs property holds the references of Neurons (in previous layer) connected to this neuron. ForwardConnections holds references to the neurons (in next layer) to which this neuron is connected.

Now, have a look at the Neuron class by extracting the source code zip of BrainNet library. Let us inspect three major functions implemented in the Neuron class - UpdateOutput, UpdateDelta and UpdateFreeParams. These functions are called by the NeuralNetwork class, by training and running the network. We will see later how the functions in NeuralNetwork class call these functions.

These functions uses the current strategy object of the neuron to perform operations.

• UpdateDelta - Find the new delta of this neuron using the current strategy. Error factor (remember that this will vary based on the layer of a neuron) will be passed to the UpdateDelta function, from the functions in Neural Network class.
• UpdateOutput - Find the new output of the neuron, by finding the net value, and then by invoking the activation function - as defined in the current strategy.
• UpdateFreeParams - Updating free parameters includes calling the functions according to the current strategy of this neuron to find new bias and to update weights.

    'Calculate the error value 
    Public Sub UpdateDelta(ByVal errorFactor As Single) Implements _
                                      NeuralFramework.INeuron.UpdateDelta

        If _strategy Is Nothing Then Throw New StrategyNotInitializedException("", Nothing)

        'Error factor is found and passed to this
        DeltaValue = Strategy.FindDelta(OutputValue, errorFactor)
    End Sub

    'Calculate the output 
    Public Sub UpdateOutput() Implements NeuralFramework.INeuron.UpdateOutput

        If _strategy Is Nothing Then Throw New StrategyNotInitializedException("..", Nothing)


        Dim netValue As Single = Strategy.FindNetValue(Inputs, BiasValue)
        OutputValue = Strategy.Activation(netValue)
    End Sub

    'Calculate the free parameters 
    Public Sub UpdateFreeParams() Implements NeuralFramework.INeuron.UpdateFreeParams

        If _strategy Is Nothing Then Throw New StrategyNotInitializedException("..", Nothing)


        BiasValue = Strategy.FindNewBias(BiasValue, DeltaValue)
        Strategy.UpdateWeights(Inputs, DeltaValue)

    End Sub

5.3. The Strategy Of A Neuron

How a Neuron actually functions is decided by the strategy of a neuron. A concrete strategy should implement the INeuronStrategy interface. This interface is shown below. BackPropNeuronStrategy is a concrete implementation of INeuronStrategy interface.

The elements in INeuronStrategy interface, along with description is given below.

'The interface for defining the strategy of a neuron 
Public Interface INeuronStrategy

    'Function to find the delta or error rate of this INeuron 
    Function FindDelta(ByVal output As Single, ByVal errorFactor As Single) As Single

    'Activation Function, or ThreshHold function
    Function Activation(ByVal value As Single) As Single

    'Summation Function for finding the net value
    Function FindNetValue(ByVal inputs As NeuronConnections, ByVal bias As Single) As Single

    'Function for calculating new bias
    Function FindNewBias(ByVal bias As Single, ByVal delta As Single) As Single

    'Function for updating weights
    Sub UpdateWeights(ByRef connections As NeuronConnections, ByVal delta As Single)

End Interface

Have a look at the BackPropNeuronStrategy class, in the code, and see how these functions are implemented as we described earlier. It is pretty easy to understand.

5.4. A Neural Network In BrainNet library

Now, let us see how the Neural Network is implemented. Any concrete neural network should implement the INeuralNetwork interface. INeuralNetwork interface is shown below.

Public Interface INeuralNetwork

    'Method to train a network     
    Sub TrainNetwork(ByVal t As TrainingData)
    'This function can be used for connecting two neurons together 
    Sub ConnectNeurons(ByVal source As INeuron, ByVal destination As INeuron, ByVal weight As Single)
    'This function can be used for connecting two neurons together with random weight 
    Sub ConnectNeurons(ByVal source As INeuron, ByVal destination As INeuron)
    'This function can be used for connecting neurons in two layers together with random weights 
    Sub ConnectLayers(ByVal layer1 As NeuronLayer, ByVal layer2 As NeuronLayer)
    'This function can be used for connecting all neurons in all layers together 
    Sub ConnectLayers()
    'This function may be used for running the network 
    Function RunNetwork(ByVal inputs As ArrayList) As ArrayList
    'This function may be used to obtain the output list 
    Function GetOutput() As ArrayList
    ReadOnly Property Layers() As NeuronLayerCollection
    'Gets the first (input) layer
    ReadOnly Property InputLayer() As NeuronLayer
    'Gets the last (output) layer
    ReadOnly Property OutputLayer() As NeuronLayer

End Interface

There are two interesting functions, TrainNetwork and RunNetwork, for training and running the network. The input to the TrainNetwork function is an object of TrainingData class. The TrainingData class has two properties of type ArrayList - Inputs and Outputs. To train the network, we put the input values to the Inputs array list, and corresponding output values are filled to the Outputs array list.

5.5. Training The Network

First of all, feed the inputs to all the neurons in the input layer. Then, the algorithm is like

• Step1: Find the output of hidden layer neurons and output layer neurons
• Step2: Finding Delta
  • 2.1) find the delta (error rate) of output layer
  • 2.2) Calculate delta of all the hidden layers, backwards
• Step3: Update the free parameters of hidden and output layers

Have a look at how this goes, inside TrainNetwork function in the NeuralNetwork class, it is commented heavily. Some part of TrainNetwork function is shown below.

         
         
           Dim i As Long
           Dim someNeuron As INeuron


            i = 0
          
     'Give our inputs to the first layer. t is an object of TrainingData class
   
            For Each someNeuron In InputLayer
                someNeuron.OutputValue = t.Inputs(i)
                i = i + 1
            Next

            'Step1: Find the output of hidden layer neurons and output layer neurons

            Dim nl As NeuronLayer
            Dim count As Long = 1

            For count = 1 To _layers.Count - 1
                nl = _layers(count)
                For Each someNeuron In nl
                    someNeuron.UpdateOutput()
                Next
            Next

            'Step2: Finding Delta


            '2.1) Find the delta (error rate) of output layer



            i = 0
            For Each someNeuron In OutputLayer
                'Find the target-output value and pass it
                someNeuron.UpdateDelta(t.Outputs(i) - someNeuron.OutputValue)
                i = i + 1
            Next

            '2.2) Calculate delta of all the hidden layers, backwards

            Dim layer As Long
            Dim currentLayer As NeuronLayer

            For i = _layers.Count - 2 To 1 Step -1

              
                currentLayer = _layers(i)

                For Each someNeuron In currentLayer
                    Dim errorFactor As Single = 0
                    Dim connectedNeuron As INeuron

                    For Each connectedNeuron In someNeuron.ForwardConnections
                        'Sum up all the delta * weight
                        errorFactor = errorFactor + (connectedNeuron.DeltaValue * _
                                            connectedNeuron.Inputs.Weight(someNeuron))
                    Next

                    someNeuron.UpdateDelta(errorFactor)
                Next

            Next


            'Step3: Update the free parameters of hidden and output layers

            For i = 1 To _layers.Count - 1
                For Each someNeuron In _layers(i)
                    someNeuron.UpdateFreeParams()
                Next
            Next

5.6. Running The Network

Running the network is pretty simple. For running the network, we just feed the inputs to the first layer, and calculate the outputs, just as explained earlier during the training phase. Here is some part of the RunNetwork function.

          
          
            Dim someNeuron As INeuron

            Dim i As Long = 0
            For Each someNeuron In InputLayer
                someNeuron.OutputValue = CType(inputs(i), System.Single)
                i += 1
            Next

            'Step1: Find the output of each hidden neuron layer


            Dim nl As NeuronLayer

            For i = 1 To _layers.Count - 1

                nl = _layers(i)
                For Each someNeuron In nl
                    someNeuron.UpdateOutput()
                Next
            Next

5.7. Creating A Network

Now, let us see how you can create a network easily. Here is a simple code that shows how to create a network. Let us assume that the input to the method is an array list which holds a list of long values that represent the number of neurons in each layer.

  
    'Demo Routine to create a network. The input parameter is a list of 
    'long values that represent the number of neurons in each layer
  
    Public Sub CreateNetwork(ByVal neuronsInLayers As ArrayList)  
        Dim bnn As New NeuralNetwork()
        Dim neurons As Long

        Dim strategy As New BackPropNeuronStrategy()

        'NeuronsInLayers is an arraylist which holds 
        'the number of neurons in each layer
      
        For Each neurons In neuronsInLayers
            Dim layer As NeuronLayer
            Dim i As Long

            layer = New NeuronLayer()

            'Let us add
            For i = 0 To neurons - 1
                layer.Add(New Neuron(strategy))
            Next

            bnn.Layers.Add(layer)
        Next

        'Connect all layers together
        bnn.ConnectLayers()
      
        'Now the network is ready, do other stuff here


    End Function
  
  

Or better, you can use the BackPropNetworkFactory class to create a network easily. Have a look at the BackPropNetworkFactory class. It has two overloaded CreateNetwork functions, for creating a neural network.

Some notes.

• This article is much like a 'Developers Guide' of BrainNet neural network library.
• Have a look at my previous article if you haven't done that yet. It is more or less a 'user's guide' for this library - for more information regarding how to use this BrainNet Library in your own projects, and to see the demo projects in action.

What is Next?

Cheers!! Thus, we finished the second article about Neural Networks. Just turn back and make sure that you understood all the points clearly.

Experiment yourself with the library, and try to optimize it a little bit, or even better, create a neural network yourself using this as an example. In my next article,

• I will explain how to create an XML based language yourself, for creating, training and processing neural networks.
• Explain the concept of some classes in the framework that I haven't mentioned in this article (like NXML interpreter, NetworkSerializer etc).

There are some 'Easter Eggs' along with the BrainNet library source code, that I haven't mentioned right now. For example, If you are smart enough, start playing with the nxml tool, already included in the associated zip. The zip file holds the whole code. nxml is a command line tool which may help you to create, train and run a neural network using xml. I'll explain it in detail, in my next article. Anyway, after compiling the project, typing nxml in the command prompt will reveal its usage :) - just if you can't wait till my next article. Another demo project is a simple Handwriting detection pad, which is also available in the source code zip.

Also,

• You may visit my website http://amazedsaint.blogspot.com/ for a lot of tech resources, code and projects
• Read all the articles I published so far here, http://amazedsaint-articles.blogspot.com/. - You'll find articles about Design Patterns, Neural Networks, Security, Hacking and more.
  • You can subscribe to the XML atom feed of my technical articles blog, for tracking new posts. Click Here for the XML Atom Feed.

When you play with the library, if you come across any bugs, please report it.

Consider A Donation:

Your contributions to the Amazed saint blog and BrainNet Library will help us to bring out more open source projects like BrainNet - along with other well written articles, projects and tutorials in emerging areas.

• Hence, we humbly request you to consider a small donation here.

---

Appendix A: Small Dose Of Spiritual Programming!!

This paragraph is about the life of a programmer, and not about technology.

Always, we are seeking for joy in life, but few people find it permanently. We need to be intense in our life. When we were kids, we had a lot of intensity, grace and bliss in our approach, and we were capable of extracting the whole juice and we were 100% in all our activities. But we lost our innocence, bliss and intensity in between. Satisfaction and smile is the cornerstone and symbol of success.

Here are some tips for my fellow programmers.

• You may go to http://www.artofliving.org/ and attend the Art Of Living healing breath workshop by contacting a center near you - and learn some Yoga and breathing practices to clear your body and mind. It is amazing.
• Drink a lot of water in between when you are with your computer, because if you don't have enough water in your body, you'll get depressed very easily. Drink two/three glasses of water each day, when you get up. Normally, when you are working, your entropy goes high, and a lot of water loss is happening.
• Take some Ayurveda medicines, this can stabilize your body and mind, to improve the quality of your intellect and clarity of your mind.

After spending quite a few years as a programmer/solution architect, I realized that practicing a little bit of Yoga, Sudarshana Kriya, Pranayamas etc (I learned these processes after attending a program from the Art Of Living Foundation) can change my attitude and behavior a lot, so that I can create a lot of harmony and contentment inside and outside - to improve my productivity leaps and bounds. This prompted me to recommend the Art Of Living Healing Breath Workshop for all my friends and fellow programmers.

Have a look at my Inspiring Intuitions blog at http://amazedsaint-intutions.blogspot.com/ for some more thoughts. Have a great time, enjoy coding - and don't forget to enjoy life. And what I recommend to you is some spiritual programming!!

Again, to conclude with some technology - you may proceed and read all the articles I published so far here, at http://amazedsaint-articles.blogspot.com/