Metrics are really important for decision making. To often in business I hear people make statements based on assumptions, followed by justifications like “It’s an educated guess”, if you want to get educated you should use metrics to get your facts first.

Microsoft recently published a great article (From Agile to DevOps at Microsoft Developer Division) about how they changed their agile processes, and one of the key things I took out of it was the change away from the product owner as the ultimate source of information.

A tacit assumption of Agile was that the Product Owner
was omniscient and could groom the backlog correctly…

… the Product Owner ranks the Product Backlog Items (PBIs) and these are treated more or less as requirements. They may be written as user stories, and they may be lightweight in form, but they have been decided.

We used to work this way, but we have since developed a more flexible and effective approach.In keeping with DevOps practices, we think of our PBIs as hypotheses. These hypotheses need to be turned into experiments that produce evidence to support or diminish the experiment, and that evidence in turn produces validated learning.

So you can use metrics to validate your hypotheses and make informed decision about the direction of your product.

If you are already in azure its easy and free for most small to mid-sized apps. You can do pretty similar things with google analytics too, but the reason i like the App Insights is it combines server side monitoring and also it works nicely with the Azure dashboard in the new portal, so i can keep everything in once place.

From Visual studio you can simply right click on a project and click “Add Application Insight Telemetry”

This will bring up a wizard that will guide you through the process.

One gotcha i found though was that because my account was linked to multiple subscriptions i had to cancel out of the wizard, then login with the server explorer then go through the wizard again.

It adds a lot of gear to your projects including

• Javascript libraries
• Http Module for tracking requests
• Nuget packages for all the ApplicaitonInsights libraries
• Application insights config file with the Key from your account

After that’s loaded in you’ll also need to add in tracking to various areas, the help pages in the azure interface have all the info for this and come complete with snippets you can easily copy and paste for a dozen languages and platforms (PHP, C#, Phyton, Javascript, etc) but not VB in a lot of instances which surprised me 🙂 you can use this link inside the Azure interface to get at all the goodies you’ll need

Where I recommend adding tracking at a minimum is:

Global.ascx for web apps

public void Application_Error(object sender, EventArgs e)
{
// Code that runs when an unhandled error occurs

// Get the exception object.
Exception exc = Server.GetLastError;

log.Error(exc);
dynamic telemetry = new TelemetryClient();
telemetry.TrackException(exc);
Server.ClearError();

}

Client side tracking copy the snippet from their interface

Add to master page or shared template cshtml for MVC.

 

Also you should load it onto your VMs as well, there is a web platform installer you can run on your windows boxes that will load a service on for collecting the stats.

Warning though that the above needs a IISReset to complete. A good article on the setup is here.

For tracking events I normally create a helper class that wraps all my calls to make things easier.

Also as a lot of my apps these days are heavy JavaScript I’m using the tracking feature in JavaScript a lot more. The snippet they give you will create a appInsights object in the page you can reuse through your app after startup.

Below is an example drawn from code I put into a survey App recently, not exactly the same code but I will use this as an example. This is so we could track when a user answers a question incorrectly (i.e. validation fails), there is a good detailed article on this here

appInsights.trackEvent("AnswerValidationFailed",
{
SurveyName: "My Survey",
SurveyId: 22,
SurveyVersion: 3,
FailedQuestion: 261
});

By tracking this we can answer questions like

• Are there particular questions that users fail to answer more often
• Which surveys have a higher failure rate for data input

Now that you have the ability to ask these questions you can start to form hypotheses. For example i make make a statement like;

“I think Survey X is too complicated for users and needs to be split into multiple smaller surveys”

To validate this I could look at the metrics collected for the above event for the user error rate on survey X compare to other surveys and use this as a basis for my hypotheses. You can sue the metrics explorer to create charts that map this out for you, filtering by you event (example below)

Then after the change is complete I can use the same metrics to measure the impact of my change.

This last point is another mistake I see in business a lot, too often changes is made, then no one reviews success. What Microsoft have done with their “Dev Ops” process embeds into their metrics into the process. So by default you are checking your facts before and after change, which is the right way to go about things imo.