Prash's Blog

Introduction to NServiceBus – Part 2 July 29, 2014

Filed under: C# — prazjain @ 7:59 pm
Tags: ,

Service Bus Host

 

In the previous post, we have created project to send messages on nservicebus, now we will create a project to host the service bus.

Create a class library project “App.Backend”, in solution “ServiceBus”. Add reference to “App.Contracts” project in this new project.

Right Click on “References” -> “Manage NuGet References”. Search online for “NServiceBus RavenDB” and add it.

Right Click on “References” -> “Manage NuGet References”. Search online for “NServiceBus Host” and add it.

This auto-generates a class “EndpointConfig” class. This class configures this endpoint and assigns it the role of server. A host can have more than one endpoint and an endpoint can have only one queue for it.

Lets add a class that will handle messages sent by our earlier application

Create a class PlaceOrderHandler, which just prints out the order id for now

    public class PlaceOrderHandler : IHandleMessages<PlaceOrder>
    {
        public void Handle(PlaceOrder message)
        {
            Console.WriteLine("Order received : " + message.OrderId);
        }
    }

IHandleMessages interface of NServiceBus tell the bus, this implementation will process the message of type (as specified in generic args).

 

Housekeeping

 

Lets cleanup the queues created earlier, so we see new queues there.

In “Console.Request” client project from previous post -> “App.Config”, update the UnicastBusConfig section as follows:

  <UnicastBusConfig>
    <MessageEndpointMappings>
      <add Messages="App.Contracts" Endpoint="App.Backend" />
    </MessageEndpointMappings>
  </UnicastBusConfig>

 

Now is the time to run this application.

Set App.Backend as startup project and start it in debug mode.

Now navigate to the exe for “Console.Request” project and run it.

Once they are both up, you will see output like this :

Console.Request output

Generated new Order id : 7417a25b-c278-46ab-bb35-758177ca822d

App.Backend output

Order received : 7417a25b-c278-46ab-bb35-758177ca822d

Check the queues created, it will look something like this:

check queues

 

These queues were generated by the host project when it started up, and Audit queue contains the message that was sent and processed above.

 

Advanced Concepts

 

Fault Tolerance

 

Our Console.Request application may send a message and continue doing about its business, but while processing the message in App.Backend database transaction is rolled back due to database failure or application crash, then we may loose the message.

So we want to make sure when our database rolls back, our queue also rolls back. Similarly when our database commits, our queue also commits (as in removes the message). NService Bus provides following options to deal with such cases:
Simple Retrying can solve transient exceptions like deadlock in database

<MsmqTransportConfig MaxRetries="5"/>

 

Second level retries (SLR) like database is down and will come back up in couple of seconds or minutes

<SecondLevelRetriesConfig Enabled="true" TimeIncrease="00:00:05" NumberOfRetries="2"/>

 

Messages that always fail are moved to an error queue

<MessageForwardingInCaseOfFaultConfig ErrorQueue="error"/> <!-- for remote server : error@remotemachine -->

For permanent exceptions like message deserialization, nservice bus will not even re-try and move them to error queue directly.

To see what will happen when you add above settings in your App.Backend project -> app.config, change PlaceOrderhandler.cs class as follows:

    public class PlaceOrderHandler : IHandleMessages<PlaceOrder>
    {
        public void Handle(PlaceOrder message)
        {
            throw new Exception("Exception occurred");
            Console.WriteLine("Order received : " + message.OrderId);
        }
    }

Now run App.Backend in debug mode, and navigate to Console.Request.exe path and run it as well.

You will notice that when you sent Order through Console.Request, App.Backend threw an exception in Handle method above. This method will be call 5 times in quick succession because of MsmqTransportConfig maxRetries configuration. If still not succeeding then Second Level Retries (SLR) kick-in they further enable another 5 retries, if that fails it SLR makes another try after 5 seconds as configured which enabled further 5 retries as configuration in MsmqTransportConfig.

When all tries fail, the message is saved in ErrorQueue=”error” as configured above. (You can navigate to the queues and check the saved message there for its contents).

Now lets say, the problem has been fixed and we want to process the messages which were ignored earlier. (remove the line – throw new Exception(“Exception occurred”); )

For this you will need to install “NServiceBus Tools” nuget package.

 

Move the error messages from error queue back to their source queues, run this executable :

<SolutionDirectory>\ServiceBus\packages\NServiceBus.Tools.4.4.3\ReturnToSourceQueue.exe

 

This will move the message back to the incoming queue for App.Backend application.

Now run the App.Backend application (if it is not already running), and you will see, it picks up the messages from its incoming queue and processes them as usual.

 

Distributed Transactions

 

Let’s ensure Network DTC is enabled. Go to “Component Services” from Windows start menu.

dtc_enabled

Install “RavenDB Client” package in App.Backend project.

Now update PlaceOrderHandler.cs as follows to save some data in RavenDB. Code is self explanatory. (place both classes below in same file, unless you want to create a separate file for Order class with just one property)

public class PlaceOrderHandler : IHandleMessages<PlaceOrder>
    {
        public void Handle(PlaceOrder message)
        {
            // Initialize the data store, point url to where db is listening
            var store = new DocumentStore { Url = "http://localhost:8080", DefaultDatabase = "App.Backend" };
            store.Initialize();
            using (var session = store.OpenSession())
            {
                session.Store(new Order() { OrderId = message.OrderId });
                session.SaveChanges();
            }
            //throw new Exception ("Exception!");
            Console.WriteLine("Order received : " + message.OrderId);
        }
    }

    public class Order
    {
        public Guid OrderId { get; set; }
    }

Now run the App.Backend project, and navigate to Console.Request.exe and run it to send request.

When you see request generated and processed, navigate to http://localhost:8080 and check documents to see the newly inserted Order object.

To see how exception handling and transaction work, uncomment Exception line in PlaceOrderHandler.Handle method and run App.Backend again.

This time you will when handling a new order from Console.Request, App.Backend throws exception and goes through all the retry logic (as configured), but no data is saved!

Even though we had a session.SaveChanges() call, no data was saved in RavenDB, this is because NServiceBus encloses each message handling method inside a transaction. So if a message handler completes successfully, transaction is committed else transaction is rolledback.

(You can now comment out the exception line again before proceeding ahead)

 

Dependency Injection

 

If you look at our PlaceOrderHandler class, you can see we instantiate DocumentStore and Session objects there. These are good candidates for dependency injection.

Update the PlaceOrderHandler code as below:

 public class PlaceOrderHandler : IHandleMessages<PlaceOrder>
    {
        public IDocumentStore Store { get; set; }
        public IDocumentSession Session { get; set; }
 
        public void Handle(PlaceOrder message)
        {
            Session.Store(new Order() { OrderId = message.OrderId });
            Session.SaveChanges();
            Console.WriteLine("Order received : " + message.OrderId);
        }
    }

Now lets add a class RavenBootstrapper.cs in App.Backend project, which will instantiate these dependencies and register them with NServiceBus container. NServiceBus container will then inject them into PlaceOrderHandler properties. Below is the code for RavenBootstrapper.cs:

public class RavenBootstrapper : INeedInitialization
    {
        public void Init()
        {
            Configure.Instance.RavenDBStorage("RavenDB", "App.Backend");
            Configure.Instance.Configurer.ConfigureComponent<IDocumentStore>(
                () =>
                    {
                        // Initialize the data store, point url to where db is listening
                        var store = new DocumentStore { Url = "http://localhost:8080", DefaultDatabase = "App.Backend" };
                        store.Initialize();
                        store.JsonRequestFactory.DisableRequestCompression = true;
                        return store;
                    }
                    , DependencyLifecycle.SingleInstance);

            Configure.Instance.Configurer.ConfigureComponent<IDocumentSession>(() =>
                {
                    return Configure.Instance.Builder.Build<IDocumentStore>().OpenSession();
                }
                , DependencyLifecycle.InstancePerCall);

        }
    }

INeedInitialization interface, as the name suggests, says to NServiceBus that it needs to be initialialized. It is here that we have configured and registered a DocumentStore and DocumentSession.

DocumentStore has dependency lifecycle of SingleInstance because we only need one instance of DataStore for the life of this application, but DocumentSession has dependency lifecycle of InstancePerCall because this needs to be instantiated for every call.

In NServiceBus a single message can have multiple handlers, so a single Order message above can be sent to multiple handlers. In such cases we do not want to have SaveChanges() call in every handler, we can have it once at the end, when all handlers have been invoked for that message.

So now lets update PlaceOrderHandler Handle method as below:

public void Handle(PlaceOrder message)
        {
            Session.Store(new Order() { OrderId = message.OrderId });
            Console.WriteLine("Order received : " + message.OrderId);
        }

To make sure SaveChanges is called on Session, update the RavenBootstrapper.cs file as below:

namespace App.Backend
{
    public class RavenBootstrapper : INeedInitialization
    {
        public void Init()
        {
            Configure.Instance.RavenDBStorage("RavenDB", "App.Backend");
            Configure.Instance.Configurer.ConfigureComponent<IDocumentStore>(
                () =>
                    {
                        // Initialize the data store, point url to where db is listening
                        var store = new DocumentStore { Url = "http://localhost:8080", DefaultDatabase = "App.Backend" };
                        store.Initialize();
                        store.JsonRequestFactory.DisableRequestCompression = true;
                        return store;
                    }
                    , DependencyLifecycle.SingleInstance);

            Configure.Instance.Configurer.ConfigureComponent<IDocumentSession>(() =>
                {
                    return Configure.Instance.Builder.Build<IDocumentStore>().OpenSession();
                }
                , DependencyLifecycle.InstancePerUnitOfWork);

            Configure.Instance.Configurer.ConfigureComponent<RavenUnitOfWork>(DependencyLifecycle.InstancePerUnitOfWork);
        }
    }

    public class RavenUnitOfWork : IManageUnitsOfWork
    {
        public IDocumentSession Session { get; set; }

        public void Begin()
        {
            
        }

        public void End(Exception ex = null)
        {
            if (ex==null) Session.SaveChanges();
        }
    }
}

Over here, we have created a RavenUnitOfWork, which implements IManageUnitsOfWork, this is registered with NServiceBus which manages its lifecycle as InstancePerUnitOfWork (i.e per message processing across multiple handlers).

We have also update the dependency life cycle of DocumentSession to be InstancePerUnitOfWork, because now we want instance to be created / disposed only once after the message has been processed through all its handlers (rather instantiating and disposing than per handler).

 

Lets conclude the post here, in this post we have seen:

  • How to write a backend application using NServiceBus
  • How to deal with exception handling and recover unprocessed messages.
  • How to interact with database.
  • How to use Dependency Injection
Advertisements
 

Introduction to NServiceBus July 28, 2014

Filed under: C# — prazjain @ 4:39 pm
Tags: ,

It is a light weight communication framework used generally in a heterogeneous environment where different applications, all using different technologies, need to communicate with each other.

What is a ServiceBus
When do I need one

 

Getting Binaries

 

  • Download NServiceBus.
    • From the installation options, just select NServiceBus (and unselect the rest, we do not need them for this tutorial). This will also setup MSMQ, Distribute Transaction Coordinator (DTC), RavenDB database, Performance counters, Lucene etc.
    • RavenDB will get installed as windows service, stop the windows service. Install latest version of Raven from official website. Navigate to services and make sure it is now running latest version of RavenDB.
    • Check RavenDB is setup by visiting url : http://localhost:8080/raven/studio.html

RavenDB home page

  • Check queues under Computer Management -> Services and Application -> Message Queuing. (Right click on “My Computer” -> Manage : To reach at Computer Management screen)

Computer Management - Queues

  • Install NuGet Package Manager
    • Go to Visual Studio -> Tools -> Extension and Updates and download as depicted below

Install NuGet Package Manager

Preparation

 

  •  Lets create a console project and call it Console.Request and name the solution ServiceBus. This project will contain code that generates messages.

Console.Request project

  • Now that we have defined our Message, let go back to Console.Request project and use this message type.In Console.Request project, open App.config file and add the below code:
      <configSections>
     <section name="UnicastBusConfig" type="NServiceBus.Config.UnicastBusConfig, NServiceBus.Core"/>
     </configSections>
     
     <UnicastBusConfig>
     <MessageEndpointMappings>
     <add Messages="App.Contracts" Endpoint="backendQueue" />
     </MessageEndpointMappings>
     </UnicastBusConfig>
    

    The above configuration allows any messages defined in namespace App.Contracts to be sent to “backendQueue”. This allows us to change the destination of messages based on their type, without changing the code.

    Until now, we have defined a message type and associated a queue with particular types of messages. To be able to send message we need to write code to send it. Add OrderGenerator class as below to Console.Request project:

    public class OrderGenerator
     {
     public OrderGenerator()
     {
     // Lets configure the bus using Fluent API
     Bus = Configure.With().Log4Net().DefaultBuilder().UseTransport&lt;Msmq&gt;().UnicastBus().SendOnly();
     }
     
     public IBus Bus { get; set; }
     
     public Guid NewOrder()
     {
     var order = new PlaceOrder
     {
     OrderId = Guid.NewGuid(),
     ProductId = Guid.NewGuid(),
     CustomerId = Guid.NewGuid()
     };
     
     Bus.Send(order);
     return order.OrderId;
     }
     }
    

    Next, update your Program.cs file in Console.Request as follows:

    class Program
     {
     static void Main(string[] args)
     {
     OrderGenerator orderGenerator = new OrderGenerator();
     while (true)
     {
     System.Console.WriteLine("Generated new Order id :" + orderGenerator.NewOrder());
     System.Console.ReadLine();
     }
     }
     }
    

    Now lets run the code we have so far. You are on right track if you see message as below:

Queue_not_exists

  • Include NServiceBus libraries in the project
    • Go to Visual Studio -> View -> OtherWindows -> Package Manager Console. Run the command on prompt
      PM> install-package nservicebus
    • Messages are common types that will be needed by all projects / libraries. So we will create messages in a separate library project.
      • Create a new library project “App.Contracts”.
        • Go to Package Manager Console. Run the command
          PM> install-package nservicebus-interfaces

          In this project we do not need the entire nservice bus library package, we just need the interface to define nservice bus messages, so we install just the interfaces.

      • Lets create a sample message as below:
        namespace App.Contracts
        {
            /// &lt;summary&gt;
            /// This is a message type. Any NServiceBus message needs to implement marker interface IMessage.
            /// &lt;/summary&gt;
            public class PlaceOrder : IMessage
            {
                public Guid ProductId { get; set; }
                public Guid OrderId { get; set; }
                public Guid CustomerId { get; set; }
            }
        }
        

 

In NServiceBus, it is the responsibility of the hosting process to create the queue. Until now we have only written the requesting process, so the queue does not exists and requesting process is surely complaining about it.

Lets go ahead and create the queue ourselves. Create a private transactional queue as shown below, as NServiceBus uses private and transactional queues.

create backendQueue

Once the queue is created, run the program again. (Press enter a few times to generate more messages) If it has completed successfully, you can check the queue again (refresh if needed).

 

Output

 

Console will show message like this:

Generated new Order id : 21217e71-1ec0-4c1b-a333-38a9b7d9c6a8

Message queue will look like this:

backendQueue message body

Xml is a bit like this:

<?xml version="1.0"?>
<Messages xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://tempuri.net/App.Contracts">
 <PlaceOrder>
 <ProductId>27a06c53-13d1-42f5-8cbf-f875742f540e</ProductId>
 <OrderId>21217e71-1ec0-4c1b-a333-38a9b7d9c6a8</OrderId>
 <CustomerId>1f83a791-36ff-4f0d-8bf2-bff8134e406e</CustomerId>
 </PlaceOrder>
</Messages>

This is it!

We wrote ‘producer’ application (application generating messages) that can send messages using NServiceBus, to be used by another application.

In the second part to this post, we will see how a ServiceBus Host application is written using NServiceBus. This second application can then be used to process messages we have in the queue.

 

Visual Studio 2013 builds fine but Resharper 7 shows errors February 18, 2014

Filed under: C# — prazjain @ 9:59 am
Tags: , ,

I hit this problem while doing some massive refactoring on my VS 2013 solution. Visual studio showed build successful, but resharper had 400+ errors. This is how i resolved it:

Menu -> Tools -> Options -> Resharper -> Suspend

Delete Resharper cache -> [User]/AppData/Local/JetBrains/v7.1/SolutionCache

Menu -> Tools -> Options -> Resharper -> Resume

 

Bulk Xml Schema Generation and Xml Validation January 17, 2014

Filed under: C# — prazjain @ 3:22 pm
Tags: , ,

I got into a situation where I had 250+ xml files with business information (logic + data), and the code was reliant on it being correctly typed by another external business facing team.
I wrote this quick xml schema generator and validator for helping me in that process.

This is what I did:

1) Generate a xml schema for each of those xml files using this code below

public class MultibookXsdGenerator
{
public void Generate()
{
string xmlFilesDir = @”C:\Source\Grids”;
string outputDir = @”C:\XsdGenerator”;
string copyToDirectory = “xmls”;
string xsdDirectory = “xsds”;

if (!Directory.Exists(Path.Combine(outputDir, copyToDirectory)))
{
Directory.CreateDirectory(Path.Combine(outputDir, copyToDirectory));
}
if (!Directory.Exists(Path.Combine(outputDir, xsdDirectory)))
{
Directory.CreateDirectory(Path.Combine(outputDir, xsdDirectory));
}

string[] xamlFiles = Directory.GetFiles(xamlFilesDir, “*.xaml”);
foreach (var xamlFile in xamlFiles)
{
FileInfo fileInfo = new FileInfo(xamlFile);
FileInfo xmlFileInfo = new FileInfo(Path.Combine(outputDir, copyToDirectory, fileInfo.Name.Replace(“.xaml”, “.xml”)));
File.Copy(xamlFile, xmlFileInfo.FullName);
ProcessStartInfo oInfo = new ProcessStartInfo(Environment.ExpandEnvironmentVariables(“\”C:\\Program Files (x86)\\Microsoft SDKs\\Windows\\v7.0A\\Bin\\x64\\xsd.exe\””), “/o:” + Path.Combine(outputDir, xsdDirectory) + ” ” + xmlFileInfo.FullName);
oInfo.UseShellExecute = false;
oInfo.ErrorDialog = false;
oInfo.CreateNoWindow = true;
oInfo.RedirectStandardOutput = true;
try
{
Process p = Process.Start(oInfo);
StreamReader oReader2 = p.StandardOutput;
string sRes = oReader2.ReadToEnd();
oReader2.Close();
// sRes now contains the output from xsd.exe
//File.Move(xmlFileInfo.FullName.Replace(“.xml”,”.xsd”),Path.Combine(outputDir,xsdDirectory));
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
}
}

2) Start to manually merge those xsd files to one single xsd. This is basically go to each

3) Validate the all the xml files with the main schema using the code below.

public class Validator
{
// Validation Error Count
static int ErrorsCount = 0;
// Validation Error Message
static string ErrorMessage = “”;
public static void ValidationHandler(object sender,
ValidationEventArgs args)
{
ErrorMessage = ErrorMessage + args.Message + “\r\n”;
ErrorsCount++;
}
public void Validate()
{
string dirName = @”C:\XsdGenerator\xmls”;
int fileCount = 0;
foreach (var file in Directory.GetFiles(dirName,”*.xml”))
{
fileCount++;
Console.WriteLine(“File number : ” + fileCount);
try
{
string strXMLDoc = File.ReadAllText(file);
// Declare local objects
XmlDocument xmld = new XmlDocument();
xmld.LoadXml(strXMLDoc);
xmld.Schemas.Add(null, @”./Xsds/Main.xsd”);
xmld.Validate(ValidationEventHandler);
// XML Validation succeeded
Console.WriteLine(“XML validation succeeded.\r\n”);
}
catch (Exception error)
{
// XML Validation failed
Console.WriteLine(“XML validation failed.” + “\r\n” +
“Error Message: ” + error.Message);
}

}

Console.WriteLine(“Console Messages : ” + Validator.ErrorMessage);
Console.WriteLine(“Console Error count : ” + Validator.ErrorsCount);
Console.ReadLine();
}
}

 

Compress and Uncompress in C# using .Net APIs December 3, 2012

Filed under: C# — prazjain @ 4:02 pm
Tags: , ,

Code snippet below for uncompressing a zip file to text file and compressing a text file back to zip file.
It is really minimal and I was able to uncompress 3+ gb text files using this, and as it just deals with stream you do not need to store the contents of the file in memory!
Here you go

        internal void UncompressZipFile(string fullPath, string zipPath)
        {
            //uncompress it
            Log.InfoFormat("Uncompressing XML zip file : Start");
            using (GZipStream gzs = new GZipStream(new FileStream(zipPath, FileMode.Open), CompressionMode.Decompress))
            {
                using (FileStream xmlFile = File.Create(fullPath))
                {
                    gzs.CopyTo(xmlFile);
                }
            }
            Log.InfoFormat("Uncompressing XML zip file : Finish");
        }

        internal void CompressCSVFile(string fullPath, string zipPath)
        {
            //Compress it
            Log.Info("Compressing CSV file : Start");
            using (FileStream csvFile = new FileStream(fullPath, FileMode.Open))
            {
                using (GZipStream gzs = new GZipStream(new FileStream(zipPath, FileMode.Create), CompressionMode.Compress))
                {
                    csvFile.CopyTo(gzs);
                }
            }
            Log.Info("Compressing CSV file : Finish");
        }

 

Restore of database failed : Operating System returned error Access denied March 5, 2012

Filed under: C# — prazjain @ 12:13 pm
Tags: ,

I was restoring a database backup in sql server after a long time, and hit upon this error. Well how difficult can restoring a db backup be in sql server express.
Hmm, not that difficult, here is what happened.

Error Message:

System.Data.SqlClient.SqlError: The operating system returned the error '5(Access is denied.)' while attempting 'RestoreContainer::ValidateTargetForCreation' on 'C:\Program Files\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\DATA\MDS.mdf'. (Microsoft.SqlServer.SmoExtended)

The backup creator had MSSql version 10 installed, so when he took the backup it also stores the original file path (to be able to restore it in same location), but I had version 11, so it could not find the destination directory.

So I changed the output file directory to C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\DATA\, and it was able to restore the database successfully.

 

Entity Framework 4.0 supports POCO May 13, 2010

Filed under: C#,Entity Framework — prazjain @ 8:49 pm
Tags: , ,

POCO – Plain Old CLR Object

In Entity Framework 3.5 an Entity Class had to extend EntityObject class which in turn implemented IEntityWithKey, IEntityWithChangeTracker, IEntityWithRelationships. All of this meant more persistence logic into a domain object, a place where it does not belong. It also made testing of these domain object a bit more complicated.

Entity Framework 4.0 now supports POCO, so simple implementation like below could be used with EF 4.0 (database used with this example is the Microsoft AutoLot database).


public class Customers {

public int CustID {   get;   set;   }

public string FirstName {   get;   set;   }

public string LastName {   get;   set;   }

}

This also makes the code easier to test, and it extracts the persistence logic out of domain objects.