Cleaning up Error logs and Dealing with vague Exceptions like “Object Reference not set to an instance of an Object”

In my experience in most cases when finding an issue with a dot NET app 90% of issues will have a corresponding exception, this exception might not be the root cause but can lead you to the root cause if you follow it, having clean error logs is essential if you want to get to the root cause fast.

The approach I take with getting to the root cause is pretty straight forward and systematic, but I find a lot that developers don’t seem to grasp troubleshoot concepts like “Process of Elimination” very well, which are common more in the Infrastructure and support side of IT, so I wanted to give a small overview for my Devs.

The steps I usually follow are:

  1. Check the line of code from the stack trace
  2. If not enough data, add more and wait for the exception to occur again
  3. If enough data then fix the problem

This is assuming you cant replicate the exception.

If you take this approach you will eventually get to the root cause and fix an exception, the next step after that is apply this to “every” exception and clean up your logs.

I will use an example of a recent issue I fixed.

I recently worked on one of our selenium testing projects, which was having some intermittent issues that we couldn’t trouble shoot and there was nothing in the Error Logs, when this happens my initial reaction is to look for code that is swallowing Exceptions,  and low and behold I find code like the below that I fixed up.

emptycatchexception

I fixed about 5 of these in key areas, now I start getting data into my logs, and I get the age old “NullReferenceException”

nullreferenceexception

Looking at the line of code, i can’t tell much, so we are using Serilog, so its easy just to throw some additional objects into the Log.Error method in order to better understand what was going on at run time. In this example the null reference exception was on the line of code that accesses the driverPool object, so I pass this in like below.

logtheobjectthatiscausingthenullreferenceexception

Now I wait for the exception to occur again and then check logs.

findingtherootcauseofexception

Now i can see the driver pool is a list of null objects. The reason for this will be earlier in the code, but this particular case is filling up the logs so we need to clean it up. So in this situation what i would do is do  null check then log a warning message and not attempt the code that is accessing it.

The reason for this is that this method does a “clean up” to dispose the object, but something else has already disposed it, there is not hard set rule here, you need to judge thee on a case by case basis.

fixingtherootcauseofexception

After waiting again my error logs are a lot more clean now and I can start to see other exception that I couldn’t see before, below you can visibly see the decrease in volume after I fixed this and a few other errors up as well, after 10:15 when my code was deployed i have 1 exception in the next hour, as opposed to hundreds per  minute. My warning logs are a bit more noisy, but that’s ok because I am trying to filter out the noise of the less important errors by using warnings.

errorlogvolume

 

Fluent C# for Unit Testing

I’ve been playing with different methods of this for a while now, ever since i got introduced to it by Shea when I was at Oztix.

Note:  in this example I am using the Shouldly library, which i recommend checking out.

I’ve seen a number of different implementations for Unit Testing and mocking specifically, but then all seem a bit complex, and also these days we run NUnit is massive degrees of parallel, and a lot of implementations use static, which causes issues in this instance.

So in introducing my new team to it I revisited it and came up with what i think is a rather simplified version using Moq, which we are using for mocking at Agoda – I miss NSubstitute 😦

When using fluent within Unit Testing the goal is, in my mind, to get your naming using the GivenWhenThen convention from BDD.  I don’t like creating “builder” classes as they dont selfdocument with good context of what you are doing.

So you test’s C# code should read something like this

GivenAPerson()
.WhenAngry()
.ThenHappinessLevelShouldBeLow();

So if we look at a more detailed example

I am going to create a person class with a happiness level property, and also something else to check, lets say Number of fingers.


public class Person
 {
 public Person()
 {
 //Set Defaults
 HappynessLevel = 5;
 NumberOfFingers = 10;
 }
 public int HappynessLevel{ get; set; }
 public int NumberOfFingers { get; set; }
 public void MakeAngry()
 {
 HappynessLevel = 1;
 }

public void MakeHappy()
 {
 HappynessLevel = 9;
 }
 }

Now I am going to create my test class in my unit test project to assist with Testing and mocking it.


 public class GivenAPerson
 {
 private Person _person;
 public GivenAPerson()
 {
 _person = new Person();
 }

public Person Then()
 {
 return _person;
 }

public GivenAPerson WhenAngry()
 {
 _person.MakeAngry();
 return this;
 }

public GivenAPerson ThenHappinessLevelShouldBeLow()
 {
 _person.HappynessLevel.ShouldBeLessThan(3);
 return this;
 }
 public GivenAPerson ThenNumberOfFingersShouldBeDefault()
 {
 _person.NumberOfFingers.ShouldBe(10);
 return this;
 }
 }

You can see that the methods all return “This” or the class itself, this is one of the core concepts in fluent C#. This allow us to achieve the desired outcome of style we are looking for above.

There is one method that doesn’t though, which is “Then”, I sue this in conjunction with shouldly, and I’ll get into why at the end.

So now lets look at putting it all together in the Test Fixture


 [TestFixture]
 public class TestFixtureExample
 {
 [Test]
 public void MyTest()
 {
 new GivenAPerson()
 .WhenAngry()
 .ThenHappinessLevelShouldBeLow()
 .ThenNumberOfFingersShouldBeDefault();
 }
 }

The first thing you will notice is that i use the word “new”, in the past I have used static to avoid this, but when using massive parallelism in NUnit you want to do this so you avoid static.

You will also note that I have added 2 checks to this test, this is to give an example of how you would use this to scale the framework.

Using this method you should be able to easily reuse your checks, in this example we verify that the persons number of fingers doesn’t change when they are angry, which is very odd, but it’s just to give an example how you can apply multiple Then or When to a single test.

Now what about the “Then()” method. I normally don’t write a method like what is in ThenNumberOfFingersShouldBeDefault(), because it is just a single value. Normally i use ThenBlahBlah() methods for complex checks, if the check is not complex i would write something like the following


 [Test]
 public void MyTest2()
 {
 new GivenAPerson()
 .WhenAngry()
 .ThenHappinessLevelShouldBeLow()
 .Then().NumberOfFingers.ShouldBe(10);
 }

This allows me a quick check for a single value at the end if I need to and avoids me writing methods.

 

 

Dependency Injection Recomendations

Recently started working with a project that has a class called “UnityConfiguration” with 2000 lines of this

container.RegisterType<ISearchProvider, SearchProvider>();

This fast becomes unmanageable, wait, I hear you say, not all Types are registered in the same way! Yes, and you won’t get away with a single line to wire-up your whole IoC container, but you should be able to get it way under 50 lines of code, even in big projects.

I prefer to go a bit Hungarian and file things via folders/namespaces by their types, then use the IoC framework to load in dependencies using this. This is because based on the Type is generally where you find the differences.

For example I put all my Attributes under a namespace called “Attributes”, with sub-folders if there is too many of course, as so on.

Below is an example of a WebApi application i have worked on in the past. This method is called assembly scanning and is in the Autofac doco here

var containerBuilder = new ContainerBuilder();
containerBuilder.RegisterAssemblyModules(
typeof(WebApiApplication).Assembly);

containerBuilder.RegisterAssemblyTypes(typeof(WebApiApplication).Assembly)
.Where(t => t.IsInNamespace("Company.Project.WebAPI.Lib")).AsImplementedInterfaces();
containerBuilder.RegisterAssemblyTypes(typeof(WebApiApplication).Assembly)
.Where(t => t.IsInNamespace("Company.Project.WebAPI.Attributes")).PropertiesAutowired();
containerBuilder.RegisterAssemblyTypes(typeof(WebApiApplication).Assembly)
.Where(t => t.IsInNamespace("Company.Project.WebAPI.Filters")).PropertiesAutowired();

containerBuilder.RegisterApiControllers(Assembly.GetExecutingAssembly()).PropertiesAutowired();
_container = containerBuilder.Build();

You can see form the above code that things like the Attributes and filters require the Properties AutoWired as I use Property injection as opposed to the constructor injection, as these require a parameter-less constructor. So I end up with one line for each sub-folder in my project basically.

So as long as I keep my filing system correct I don’t have to worry about maintaining a giant “Configuration” class for my IoC container.

You can also make use of modules in Autofac by implementing the Module, I recommend using this for libraries external to your project that you want to load in. you can use the RegisterAssemblyModules method in Autofac in a similar way

 

 

 

 

 

Swagger/Swashbuckle and WebAPI Notes

If you aren’t using Swagger/Swashbuckle on your WebAPI project, you may have been living under a rock, if so go out and download it now 🙂

Its a port from a node.js project that rocks! And MS is really getting behind in a big way. If you haven’t heard of it before, imagine WSDL for REST with a snazy Web UI for testing.

Swagger is relatively straight forward to setup with WebAPI, however there were a few gotchas that I ran into that I thought I would blog about.

The first one we ran into is so common MS have a blog post about it. This issue deals with an exception you’ll get logged due to the way swashbuckle auto generates the ID from the method names.

A common example is when you have methods like the following:

GET /api/Company // Returns all companies

GET /api/Company/{id} // Returns company of given ID

In this case the swagger IDs will both be “Company_Get”, and the generation of the swagger json content will work, but if you try to run autorest or swagger-codegen on this they will fail.

The solution is to create a custom attribute to apply to the methods like so


// Attribute
namespace MyCompany.MyProject.Attributes
{
[AttributeUsage(AttributeTargets.Method)]
public sealed class SwaggerOperationAttribute : Attribute
{
public SwaggerOperationAttribute(string operationId)
{
this.OperationId = operationId;
}

public string OperationId { get; private set; }
}
}

//Filter

namespace MyCompany.MyProject.Filters
{
public class SwaggerOperationNameFilter : IOperationFilter
{
public void Apply(Operation operation, SchemaRegistry schemaRegistry, ApiDescription apiDescription)
{
operation.operationId = apiDescription.ActionDescriptor
.GetCustomAttributes&lt;SwaggerOperationAttribute&gt;()
.Select(a =&gt; a.OperationId)
.FirstOrDefault();
}
}
}

//SwaggerConfig.cs file
namespace MyCompany.MyProject
{
public class SwaggerConfig
{
private static string GetXmlCommentsPath()
{
return string.Format(@&quot;{0}\MyCompany.MyProject.XML&quot;,
System.AppDomain.CurrentDomain.BaseDirectory);
}
public static void Register()
{

var thisAssembly = typeof(SwaggerConfig).Assembly;

GlobalConfiguration.Configuration
.EnableSwagger(c =&gt;
{
c.OperationFilter&lt;SwaggerOperationNameFilter&gt;();

c.IncludeXmlComments(GetXmlCommentsPath());

// the above is for comments doco that i will talk about next.

// there will be a LOT of additional code here that I have omitted

}

}

}

}

Then apply like this:


[Attributes.SwaggerOperation(&quot;CompanyGetOne&quot;)]
[Route(&quot;api/Company/{Id}&quot;)]
[HttpGet]
public Company CompanyGet(int id)
{
// code here
}

[Attributes.SwaggerOperation(&quot;CompanyGetAll&quot;)]
[Route(&quot;api/Company&quot;)]
[HttpGet]
public List&lt;Company&gt; CompanyGet()
{
// code here
}

Also mentioned I the MS article is XML code comments, these are awesome for documentation, but make sure you don’t have any potty mouth programmers

This is pretty straight forward, see the setting below

XmlCommentsOutputDocumentationSwaggerSwashbuckle

The issue we had though was packaging them with octopus as it’s an output file that is generated at build time. We use the octopack nuget package to wrap up our web projects, so in order to package build-time output (other than bin folder content) we need to create a nuspec file in the project. Octopack will default to using this instead of the csproj file if it has the same name.

e.g. if you project is called MyCompany.Myproject.csproj, create a nuspec file in this project called MyCompany.MyProject.nuspec.

Once you add a file tag into the nuspec file this will override octopack ebnhaviour of looking up the csproj file for files, but you can override this behavior by using this msbuild switch.

/p:OctoPackEnforceAddingFiles=true

This will make octopack package files from the csproj first, then use what is specified in the files tag in the nuspec file as additional files.

So our files tag just specifies the MyCompany.MyProject.XML file, and we are away and deploying comments as doco!

We used to use sandcastle so most of the main code comment doco marries up between the two.

Autofac DI is a bit odd with the WebAPI controllers, we generally use DI on the constructor params, but WebAPI controllers require a parameter-less constructor. So we need to use Properties for DI. This is pretty straight forward you juat need to call the PropertiesAutowired method when registering them. And as well with the filters and Attributes. In our example below I put my filters in a “Filters” Folder/Namespace, and my Attributes in an “Attributes” Folder/Namespace


// this code goes in your Application_Start

var containerBuilder = new ContainerBuilder();

&amp;nbsp;

containerBuilder.RegisterAssemblyTypes(typeof(WebApiApplication).Assembly)
.Where(t =&gt; t.IsInNamespace(&quot;MyCompany.MyProject.Attributes&quot;)).PropertiesAutowired();
containerBuilder.RegisterAssemblyTypes(typeof(WebApiApplication).Assembly)
.Where(t =&gt; t.IsInNamespace(&quot;MyCompany.MyProject.Filters&quot;)).PropertiesAutowired();

containerBuilder.RegisterApiControllers(Assembly.GetExecutingAssembly()).PropertiesAutowired();

containerBuilder.RegisterWebApiFilterProvider(config);

 

 

Exception Logging and Tracking with Application Insights 4.2

After finally getting the latest version of App Insights Extension installed into Visual Studio, its been a breath of fresh air to use.

Just a note, to get it installed, I had to go to to installed programs, hit modify on VS2015, make sure everything else was updated. Then run the installer 3 times, it failed twice, 3rd time worked.

Now its installed I get a new option under my config file in each of my projects called “search”.

ApplicationInsightsVisualStudioExtension.PNG

This will open the Search window inside a tab in visual studio to allow me to search my application data. The first time you hit it you will need to login and link the project to the correct store though, after that it remembers.

ApplicationInsightsSearchExceptionsVisualStudio

From here you can filter for and find exceptions in your applications and view a whole host of information about them. Including information about the server, client, etc. But my favorite feature is the small blue link at the bottom.

ExceptionInformationApplicaitonInsights

Click on this will take you to the faulting function, it doesn’t take you to the faulting line though (which i think it should) but you can mouse over it to see the line.

LinkToCodeWhereExceptionWasThrown

One of the other nice features, which was also in the web portal. is the +/- 5 minutes search.

5MinuteSearchEitherSideOfException

You can use this to run a search for all telemetry within 5 minutes either side of the exception. In the web portal there is also an option of “all telemetry of this session”, which is missing from the VS interface, I hope they will introduce this soon as well.

But the big advantage to this is if you have setup up App Insights for all of your tracking you will be able to see all of the following for that session or period:

  • Other Exceptions
  • Page Views
  • Debug logging (Trace)
  • Custom Events (If you are tracking thinks like feature usage in JavaScript this is very handy)
  • Raw Requests to the web server
  • Dependencies (SQL calls)

Lets take a look at some of the detail we get on the above for my +/- 5 Minute view

Below is an SQL dependency, this is logging all my queries. So I can see whats called, when, the time the query took to run, from which server, etc. This isn’t any extra code I had to write, App Insights will track all SQL queries that run from your application out of the box, once setup.

AppInsightsSQLDependancy

And dependencies won’t just be SQL, they will also be SOAP and REST requests to external web services.

Http Request monitoring the detail is pretty basic but useful.

HttpRequestMonitoring

And Page views you get some pretty basic info also. Not as good as some systems i have seen, but defiantly enough to help out with diagnosing exceptions.

PageViewAppInsights

I’ve been using this for a few days now and find it so easy to just open my solution in the morning and do a quick check for exceptions, narrow them down and fix them. Still needs another version or two before it has all the same features as the web interface, but defiantly worth a try if you have App Insights setup.

 

 

 

Swagger WebAPI JSON Object formatting standards between C#, TypeScript and Others

When designing objects in C# you use pascal casing for your properties, but in other languages you don’t, and example (other than java) is TypeScript here’s an article from Microsoft about it.

And that’s cool, a lot of the languages have different standards and depending on which one you are in, you write a little different.

The problem is when you try to work on a standard that defines cross platform communication that is case sensitive, an example being Swagger using REST and JSON.

So the issue we had today was a WebAPI project was generating objects like this:


{
"ObjectId": 203
"ObjectName" : "My Object Name"
}

When swaggerated the object comes out correctly with the correct pascal casing, however when using swagger codegen the object is converted to camel case (TypeScript Below)


export interface MyObject {
 objectId: number;
 objectName: string;
}

The final output is a generated client library that can’t read any objects from the API because JavaScript is case sensitive.

After some investigation we found that when the swagger outputs camel casing the C# client generators (Autorest and Swagger codegen) will output C# code that is in camel casing but with properties to do the translating from camel to pascal, like the below example


/// <summary>
/// Gets or Sets TimeZoneName
/// </summary>
[JsonProperty(PropertyName = "timeZoneName")]
public string TimeZoneName { get; set; }

So to avoid pushing shit up hill we decided to roll down it. I found this excellent article on creating a filter for WebAPI to convert all your pascal cased objects to camel case on the fly

So we found that the best practice is:

  • Write Web API C# in Pascal casing
  • Covert using an action filter from pascal to camel case Json objects
  • Creating the client with TyepScript (or other camel language) default option will then work
  • Creating the C# client will add the JsonProperty to translate from camel to pascal and resulting C# client will be pascal cased

I raised a GitHub Issue here with a link into the source code that I found in swagger codegen, however later realized that changing the way we do things will mitigate long term pain.

XML and JSON (De)Serialization Tools

I’ve been working with a lot of external systems lately that have their own objects that they serialize into either JSON or XML, and have found some good to tools to rebuild those objects into C# classes that i can add to my apps.

For XML I’ve found this one (http://xmltocsharp.azurewebsites.net/)

Some Example XML from one of the Apps I’ve been working with (This was a sample on their website)


<reply>
    <contents>...</contents>
    <attachment filename="..." md5="...">
      <!-- base-64 encoded file contents -->
    </attachment>
  </reply>

The tools nicely converts to the below C#


[XmlRoot(ElementName="attachment")]
public class Attachment {
[XmlAttribute(AttributeName="filename")]
public string Filename { get; set; }
[XmlAttribute(AttributeName="md5")]
public string Md5 { get; set; }
}

[XmlRoot(ElementName="reply")]
public class Reply {
[XmlElement(ElementName="contents")]
public string Contents { get; set; }
[XmlElement(ElementName="attachment")]
public Attachment Attachment { get; set; }
}

Then i can use the following code to deserialise their Response Stream from the HTTP response


Reply reply=new Reply ();
var resp = CallA MethodThatReturnsaStreamResposne();
using (var sr = new StreamReader(resp))
{
var xs = new XmlSerializer(reply.GetType());
reply= (Model.Tickets)(xs.Deserialize(sr));
}

Pretty similar with JSON too, using this site (http://json2csharp.com/)

However they don’t support invalid names, so i am going to use one int his example and how to work around it.

{".Name":"Jon Smith"}
public class RootObject
{
    public string __invalid_name__.Name { get; set; }
}

JSON properties support dashes and periods, which are not supported in C#, so in C# there is support for a attribute “jsonproperty” that you can use like the below to fix the issue.

 
public class RootObject
{
    [JsonProperty(".Name")]
    public string Name { get; set; }
}

Then to deserialize I use the Newtonsoft JSON libraries example below

var del = message.GetBody<string>();
var myObject = JsonConvert.DeserializeObject<RootObject>(del); 

Obviously thought you would not leave it named as “RootObject” though 🙂

In the example above i was reading a JSON object from a Azure Service Bus Queue.

Lastly, thought i would add in the class i use for going back to XML. In this case i was working on today I was going to a PHP based system Kayako and it has some strange requirements.

I created a class to inherit my XML objects from so add an extension method.

     public class XmlDataType
    {
        public string ToXmlString()
        {
            StringBuilder sb = new StringBuilder();
            string retval = null;
            
            using (XmlWriter writer = XmlWriter.Create(sb, new XmlWriterSettings() {Encoding = Encoding.UTF8}))//{OmitXmlDeclaration = true}
            {
                var ns = new XmlSerializerNamespaces();
                
                ns.Add(&quot;&quot;, &quot;&quot;);
                new XmlSerializer(this.GetType(),new XmlAttributeOverrides() { }).Serialize(writer, this,ns);
                retval = sb.ToString();
            }
            return retval.Replace(&quot;utf-16&quot;,&quot;UTF-8&quot;);
        }
    }

Kayako only accepts UTF8 not UTF16, adn you need to remove the xmlns properties, which i do with the Namespace declaration above.

The way I’ve changed UTF16 to 8 is a bit dodgy but it works for Kayako.

Now i can just call MyObject.ToXmlString() and get a string for the XML data that i can pass into a HTTP Request.

Noting that some of the Kayako methods don’t like the xml tag at the start, so if you need to remove this I’ve left in a bit of commented out code that you can use in the XML Writer settings initialization.