Learn How To Build A Provider Framework For .NET

Download source files - 100 Kb

The Problem
The Solution
So, The Framework
Designing the Framework
Implementing The Framework
Developing A Simple Plug-in
An Application To Test The Framework
Design Variations
Conclusion

Solution Architect: We should start concentrating more on frameworks!!

Dumb Developer: Well, I think I know about .NET framework. You mean, something like that?

After reading this article, you'll be able to

Change your mindset a little bit, and start thinking about 'frameworks' instead of just 'code'
Understand a lot about practically applying provider pattern in your projects
Gain much knowledge regarding how xml configuration files and providers work (especially in context of .NET)

1) The Problem

You are the Solution Architect of a BIG credit card company. You need to publish your credit card data to various banks, in order to get it approved. Basically, you send the card information (say, as an XML file) to a bank, the bank may or may not approve each card request, and you need to get the result back. The format you need to publish your card data is different from bank to bank, and you need a separate interfacing standard for each bank. Some banks support only FTP, and some other banks support only web services - and so on. Basically,

You want to fetch your card data from a data source
Convert it to the bank specific format (like, an XML file or a CSV file)
Push it to a bank for processing
Synchronously or asynchronously pull it back/get the result
Save the result for each card in your data base

You may do this process based on a scheduler - because you need to send card data to banks few times a day (once or twice), and to pull the results back.

And to make matters more interesting, more banks will be added to your system in future.

Well, now you need to propose a solution. Interesting.. huh?

2) The Solution

Let us see what all solutions are not good enough.

You should not think about developing your own scheduler - because you can use an existing scheduler to run your application to push/pull data to/from each banks.
You should not think about developing a separate application for synchronizing with each bank - because it is difficult to create, maintain, and deploy.

Let us see what all decisions are good design decisions.

You may develop an application to communicate with all banks. The application will load a list of plug-ins to push/pull data to/from banks from a configuration file
Your application can be scheduled using Windows Scheduler the way you like.

So finally, let us decide to

Build a 'Publisher' application, with some capability to load plug-ins and push/pull data

Now, you spoke with some of your friends and they asked you.. "Oh dude! Why you can't make it a generic interfacing Framework, so that we can re-use it?" - "Quite a good idea" you feel .. "But man, how exactly I make it a Framework?" :|

There we are :). Let us see.

3) So, The Framework

Basically a framework is

Something you can re-use (Give me the ready-made pack, I'll use it. Or else, get away).
A set of libraries and support programs (like, the .NET framework)
A set of pre-solved problems, to help you build solutions faster..

Solution Architect 1: "The day we complete building frameworks for all human work flows happening in this world, Solution Architects may go out of job".

Solution Architect 2: "No, Still we can design and build frameworks to fix our existing frameworks".

Solution Architect 3: "Dude, What about a framework to build frameworks?. Probably a meta-meta framework or a meta-meta-meta framework?".

Dumb Programmer (Thinks) : "All solution architects are dumb, lazy and crazy. They just draw diagrams and talk Greek. The whole project is in my head".

Let us come back to our topic. How to create a framework for solving the above design issues?. Here are some final thoughts regarding the framework we are going to build.

There should be a 'Publisher' framework to load plug-ins
The Publisher will call certain functions in each plug-in (say, Put and Get ) to push the data and to pull the data
A simple front end application will use the Publisher framework to load plug-ins from an XML file, and to push/pull data
We may use windows scheduler to schedule the front end application.

Now, as you see, we need to load plug-ins, after reading the information from an xml configuration file. Essentially, we need to separate some kind of functionality from the actual application - i.e., the publishing logic (push/pull data to/from bank) should be decoupled from the actual application (loader/scheduler) through a plug-in model - so that we can add/remove and configure plug-ins at a later stage very easily.

Hence,

When a new bank comes in, we can develop a simple plug-in for data push/pull and deploy it easily.
Modify the configuration file, so that the loader will load it.

Well, this may ring the bell. You suddenly think.. "Separating functionality using a plug-in based model is nothing new - and an existing design pattern should be in existence for this". Well, that is what (actually, some what :-) ) provider pattern is all about. Let us see how we can design our framework, around the provider pattern, to solve the design issue.

If you are not so familiar with design patterns, you are lagging at least one step. Stop here, go and read my articles on 'Applying Design Patterns' - Part 1 and Part 2, Then come back.

4) Designing The Framework

Altogether, the idea about 'provider pattern' is quite simple. There are various ways of implementation - here is a quite simple one.

You have a well defined plug-in interface - public interface IMyInterface , or even an abstract class, in your Framework project
Your plug-in classes will implement this interface or abstract class. The trick is, your plug-in classes that implements your interface will be residing (most probably) in separate library files (say, dll files)
At run time, your framework will read the list of available plug-ins from a configuration file (most probably, an XML file)
You load those plug-in classes using Reflection mechanism (System.Reflection in .NET) and create an object of the class
You can now call the methods inside your class, through your interface

Much like this.

IPublisher is our interface in the Framework, which defines our abstract functions.
- The Init function will be called initially when a publisher is instantiated, to pass the settings to the newly loaded publisher
- Other functions like Get and Put are used for specific purposes - say, to push and pull data to/from the bank.
PublisherFactory is our factory class - which takes two parameters as input (Don't confuse this with Factory pattern or something. Now, just consider it as a class name)
- The assembly name (from where we should load the plug-in class)
- The settings to pass to the publisher when it is loaded.

FTPPublisher and SOAPPublisher are two plug-in classes, which implements the IPublisher interface. KeyValueCollection is a set of name-value pair, to pass some settings (like Host=myhost, Port=30).

5) Implementing The Framework

Now the easiest part - implementing the framework (Unfortunately, most people start thinking from this phase only).

Credits to Sandeep (sandeepkumar@sumerusolutions.com) for implementing the actual code, based on the design

5.a) Creating The IPublisher Interface

As we discussed, the interface definition is pretty simple. The type SettingCollection is synonymous to KeyValue collection we discussed above.

/// <summary>
/// The interface 
for an IPublisher
/// </summary>
public interface IPublisher



    {




/// <summary>
/// To publish data
/// </summary>
void Put();
/// <summary>
/// To retreive data
/// </summary>
void Get();

/// <summary>
/// Function called 
by publisher factor to pass the settings (settings is a NameValue collection)
/// </summary>
/// <param name="settings">
/// </param>
void Init(Sumeru.Publisher.Framework.Data.SettingCollection settings);


}

5.b) Creating The PublisherFactory

The publisher factory has a very simple static function, CreatePublisher.

Publisher factory simply loads the type (i.e, our plug-in class) from the assembly, creates an object of it, and return it. Looks so simple, huh? If the type we loaded is not an IPublisher, an error may occur at the line ipub=(IPublisher) pub because explicit casting may fail. In this way, we make sure that all plug-ins we load should obey the IPublisher interface. It will also call the init function to pass the parameters to the plug-in after loading it.

/// <summary>
/// 
Summary description for PublisherFactory.
/// </summary>
public abstract 
class 
PublisherFactory
{
 
/// <summary>
/// Creates the 
publisher and return a type of IPublisher
/// </summary>
/// <param name="assembly">The path to the assembly</param>
/// <param name="className">The class name to instantiate</param>
/// <param name="settings">The settings collection</param>
/// <returns></returns>
public static IPublisher CreatePublisher(string assemblyName, string 
className,
                     
Sumeru.Publisher.Framework.Data.SettingCollection settings) 

{
try 

{

IPublisher ipub;

Assembly 
asm=Assembly.LoadFrom(assemblyName); 
System.Type 
type=asm.GetType(className);
object pub=Activator.CreateInstance(type); 

ipub=(IPublisher) pub;
ipub.Init(settings);
return ipub;
}
catch (Exception ex) 
{
throw 
ex;
}
} 


}

5.c) Creating The PublisherLoader Class

If you remember, one of our requirements were to load the plug-in information from an XML file. PublisherLoader class is exactly for that. This class is not there in the above UML diagram - pardon me, but still it is a part of the Framework. It will load an XML configuration file, load each plug-in specified, instantiate it by calling the PublisherFactory, and call other functions in the plug-in like Get() and Put(). Before we go to the class definition, here is some more meat regarding how to create your own xml configuration files and how to actually read them.

The XML Configuration File

For eg, consider that we should allow our Framework to load plug-ins by reading a configuration file like this.

<Publishers>
<!-- FTP Publisher -->
<Publisher AssemblyName="Sumeru.Publisher.FTPPublisher.dll" 
                
ClassName="Sumeru.Publisher.FTPPublisher.FTPPublisherPlugin" Name="FTP">
<Setting Key="Host" Value="myhost1"/>
<Setting Key="Port" Value="21"/>
<Setting Key="Password" Value=""/>
<Setting Key="UserName" Value=""/>
<Setting Key="FileToDownload" 
Value=""/>
<Setting Key="DownloadToPath" 
Value=""/>
<Setting Key="FileToUpload" 
Value=""/>
</Publisher> 

<!-- SOAP Publisher -->
<Publisher AssemblyName="Sumeru.Publisher.SOAPPublisher.dll" 

               
ClassName="Sumeru.Publisher.SOAPPublisher.SOAPPublisherPlugin" Name="SOAP">
<Setting Key="URL" Value="http://www.myhost.asmx"/>
<Setting Key="Port" Value="80"/>
</Publisher> 


</Publishers>

Each Publisher has various properties like AssemblyName and ClassName for specifying the plug-in assembly path (dll) and the name of the plug-in class. Also, each publisher has a set of settings, like Host and Port. These settings are passed to the plug-in class by the PublisherFactory class when we instantiate an object of the plug-in, by calling the Init function. How this works? Read on.

How to read a configuration file like above? Fortunately it is very simple in .NET. The steps are,

<ul> <li>Create an XML schema to describe the configuration file format (I.e, to describe the elements and attributes in your configuration file) </li> <li>Create a set of classes from the XML Schema (to serialize and de-serialize an xml document specific to this schema) </li> <li>Use these classes to read a configuration file like above, to an object model. </li> </ul>

Creating The XML Schema

So, first let us create the XML schema.

<?xml version="1.0" encoding="utf-8" ?>
<xs:schema id="Publisher" 
targetNamespace="http://tempuri.org/Publisher.xsd" elementFormDefault="qualified"
xmlns="http://tempuri.org/Publisher.xsd" xmlns:mstns="http://tempuri.org/Publisher.xsd"
xmlns:xs="http://www.w3.org/2001/XMLSchema">


<xs:element name="Publishers">
<xs:complexType>
<xs:sequence>
    <xs:element ref="Publisher" maxOccurs="unbounded" />
</xs:sequence>
    
<xs:attribute name="Version" type="xs:string" />
</xs:complexType>
</xs:element>



<xs:element name="Publisher">
<xs:complexType>
<xs:sequence>
    <xs:element ref="Setting" maxOccurs="unbounded" />
</xs:sequence>
    <xs:attribute name="Name" type="xs:string" />
    <xs:attribute name="AssemblyName" type="xs:string" />
    <xs:attribute name="ClassName" type="xs:string" />
    <xs:attribute name="Version" type="xs:string" />
</xs:complexType>
</xs:element>



<xs:element name="Setting">
<xs:complexType>
<xs:sequence></xs:sequence>
    <xs:attribute name="Key" type="xs:string" />
    <xs:attribute name="Value" type="xs:string" />
</xs:complexType>
</xs:element>



</xs:schema>

If you see the above schema, there are three elements, Publishers, Publisher and Setting. It is easy to understand, still, if you see, the Publishers element can contain more than on Publisher element. A Publisher element has four attributes - Name, AssemblyName, ClassName and Version - and a Publisher element can contain more than one Setting element. A Setting element has two attributes- a Key and a Value. This schema exactly describes the configuration document structure we need. Save the schema to a file, named Publisher.xsd

Creating A Set Of Classes

Now, to create a data structure out of our schema, I've several options. My favorite option is to use XsdObjectGenerator. It will generate a set of classes, which is capable of representing my above schema. I.e., it will create classes to create an object model of the schema, to which I can de-serialize my configuration file.

Let us create the classes from the command line, using xsdobjectgen.

xsdobjectgen Publisher.xsd /n:Sumeru.Publisher.Framework.Data

This will create an object model based on elements in Publisher.xsd (i.e, a set of classes like Publishers, Publisher, Setting etc along with attributes) to describe the elements in Publisher.xsd. The /n switch specifies the namespace for the newly created classes.

And this created a file Sumeru.Publisher.Framework.Data.cs with all the classes - please see the attached source code if required. There'll be a Sumeru.Publisher.Framework.Data.Publishers Class (If you remember, Publishers is the root element according to our schema), and we can use an object of this class to de-serialize an xml document.

Load The Configuration File

This part is pretty simple. We just create an object of Publishers class, and de serialize the xml file to it.

Sumeru.Publisher.Framework.Data.Publishers pubs;
XmlSerializer xsr=new XmlSerializer(typeof(Framework.Data.Publishers));
pubs=(Framework.Data.Publishers) xsr.Deserialize(new StreamReader("c:\config.xml"));

If you remember, Publishers element has a collection of Publisher element, and each Publisher element has few attributes like Class Name and Assembly Name.

We iterate the Publisher elements, to pass the class name and assembly name to the PublisherFactory we wrote earlier. This will dynamically load a publisher, and may return us an IPublisher object - if the assembly name and class name is specified correctly in the XML file. Then, we may call our custom functions

The PublisherLoader Class

The PublisherLoader class does all this work (though it is very simple)

/// <summary>
/// Summary description for PublisherLoader.
/// </summary>
public class PublisherLoader
{

/// <summary>
/// Load all the publishers from the configuration, and create an instance
/// using the factory. Also call our custom functions.
/// </summary>
/// <param name="path"></param>
public void LoadPublisher(string path) 
{
Framework.Data.Publishers pubs;
XmlSerializer xsr=new XmlSerializer(typeof(Framework.Data.Publishers));
pubs=(Framework.Data.Publishers) xsr.Deserialize(new StreamReader(path));
foreach (Framework.Data.Publisher p in pubs) 
{
Framework.IPublisher ipub=Framework.PublisherFactory.CreatePublisher(p.AssemblyName,
                                      p.ClassName,p.SettingCollection);
ipub.Get();
ipub.Put(); 
}
}
}

Here, as you see, we are loading the configuration, iterate each publisher element, and passes the class name and assembly name to the PublisherFactory for creating an IPublisher object. Then we are calling the functions Get() and Put().

If you notice, we are also passing the Setting collection of a publisher element to the PublisherFactory, and the PublisherFactory will in-turn pass this Setting collection by calling the Init(..) function in a Publisher object during the time of instantiation (See the code of PublisherFactory).

Now, we are almost done. Let us write a simple plug-in for our framework now.

6) Developing A Simple Plug-In

To develop a plug in, I've done the following steps.

Created a class library project named Sumeru.Publisher.FTPPublisher
Added a reference to our Framework project
Created a class named FTPPublisherPlugin by implementing the IPublisher interface in our framework.

Here is the FTPPublisherPlugin Class. If you see, it just implements all the functions defined in IPublisher.

You may need to add your own code for putting and getting files to/from ftp server, in place of showing message boxes.


/// <summary>
/// A Simple Plugin Class
/// </summary>
public class FTPPublisherPlugin : Sumeru.Publisher.Framework.IPublisher 
{

private System.Collections.Hashtable ht=new System.Collections.Hashtable();

/// <summary>
/// To publish 
data
/// </summary>
public void Put() 
{
//Put your data
//Note: Implement proper error handling for invalid 
parameters
string host=(string)ht["Host"];
string port=(string)ht["Port"];
System.Windows.Forms.MessageBox.Show("Putting File To " + 
host + "," + port); 
//Write Your own ftp 
code to put file
}
/// <summary>
/// To retreive data
/// </summary>
public void Get() 
{
//Get your data
//Note: Implement proper error handling for invalid 
parameters
string host=(string)ht["Host"];
string port=(string)ht["Port"];
System.Windows.Forms.MessageBox.Show("Getting File from " 
+ host + "," + port); 

//Write Your own 
ftp code to get file
 
}

/// <summary>
/// Init Function called by publisher factory to pass the settings
/// </summary>
/// <param name="settings">
/// </param>
public void Init(Sumeru.Publisher.Framework.Data.SettingCollection settings)
{
//Keep our settings locally 
in a hash table
//Implement proper error handling for duplicate keys
foreach(Framework.Data.Setting s in settings) 
{
ht.Add(s.Key,s.Value);
}

}

}

7) An Application To Test The Framework

Now, let us develop a very simple Loader application, which consumes our Framework. It basically invokes the LoadPublisher function in PublisherLoader class we created earlier. I created a form, with a text box which has the path to the Xml configuration file.

The XML Configuration file has information to load our newly written Plug-in. I'm just loading the same plug-in twice with different settings. Try developing yet another plug-in, and add a new publisher element to load that. Make sure that you are copying the dll to the path of this Loader app - if you are not explicitly specifying the full path for the dll.

<?xml version="1.0" encoding="utf-8" ?>
<Publishers xmlns="http://tempuri.org/Publisher.xsd">
<!-- FTP 
Publisher - A simple ftp transaction to host 1 -->
<Publisher AssemblyName="Sumeru.Publisher.FTPPublisher.dll" 
ClassName="Sumeru.Publisher.FTPPublisher.FTPPublisherPlugin" 
Name="FTP">
<Setting Key="Host" Value="myhost1"/>
<Setting Key="Port" Value="21"/>
<Setting Key="Password" Value=""/>
<Setting Key="UserName" Value=""/>
<Setting Key="FileToDownload" Value=""/>
<Setting Key="DownloadToPath" Value=""/>
<Setting Key="FileToUpload" Value=""/>
</Publisher> 
<!-- FTP 
Publisher - A simple ftp transaction to host 2 -->
<Publisher AssemblyName="Sumeru.Publisher.FTPPublisher.dll" 
ClassName="Sumeru.Publisher.FTPPublisher.FTPPublisherPlugin" 
Name="FTP">
<Setting Key="Host" Value="myhost2"/>
<Setting Key="Port" Value="21"/>
<Setting Key="Password" Value=""/>
<Setting Key="UserName" Value=""/>
<Setting Key="FileToDownload" Value=""/>
<Setting Key="DownloadToPath" Value=""/>
<Setting Key="FileToUpload" Value=""/>
</Publisher> 
 
</Publishers>
  


In the Load button click, I have the code to load the plug-ins in this 
configuration file, like


private void btnLoad_Click(object sender, System.EventArgs e)
{
PublisherLoader pl=new PublisherLoader();
pl.LoadPublisher(this.txtFileToLoad.Text);
}

As you see, we are using the PublisherLoader class, to read the configuration file, and to load the plug-ins as explained earlier. Now, if you click Load button, the plug-ins will be loaded, and you may see the message boxes from the plug-in we developed. (Let us keep it simple ;)

8) Design Variations

Here are some interesting design variations so that you can implement for practice, and for some additional brainstorming.

Yesterday I spend some time with Abhilash - and he was mentioning how he is planning to use a script engine in his provider-like framework. Instead of developing separate classes as plug-ins, there will be a core engine, and it will load script files which can be interpreted by the core engine from a configuration file. That framework is mainly for data mapping between protocols.
Right now, the framework is not passing any data to the plug-ins (other than the settings). Some times (most of the times), this framework may need to pass data to/from plug-in functions (like, data to put and get). If the data passed from Framework to put/get functions are standardized (i.e., a pre-defined schema), everything is fine - but what if the data format is not pre-defined? How will you solve this issue? Post your comments here :)

9) Conclusion

Hope you enjoyed it - I tried to keep it very simple.

Some more information if you are health conscious.

When you work with your comps, normally your entropy will go high (just if you are a smart programmer), and this may decrease the level of water in your body. Make sure that you are drinking lots of water - this will eliminate a lot of stress, strain and depression.

And finally, I'll also recommend an Art Of Living Part I workshop for you (See http://www.artofliving.org/ and have a look at the Courses). It is an interactive workshop of 18 hours spread over 6 days. As it did for me, I hope that it may help you to find the right balance between your work and life - to improve the mental strength and clarity of your mind, and to improve the quality of your life.

Here are some other popular articles I wrote.

You can visit my own website at http://amazedsaint.blogspot.com/ for more articles, projects and source code

Also, you may view my tech-bits blog , download my opersource projects. Also, if you have some time to read some insights, see my intuitions blog

Thanks a lot :)