Wednesday, August 22, 2012

Is serialization really that expensive?

While wading through an exotic codebase, I stumbled upon a static class named Convert. This class contained somewhere around 2700 (non-generated) lines of code, where each method manually converted some object to a simple textual representation. These methods were then used to convert requests and reponses to and from a remote third party service before logging them to the database for auditing reasons.
public static class Convert
{
    public static string PaymentRequest(PaymentRequest req)
    {
        var sb = new StringBuilder();

        sb.Append("Reference: " + req.Reference + " - ");
        sb.Append("NumberOfLicenses: " + req.NumberOfLicenses + " - ");
        sb.Append("PricePerLicense: " + req.PricePerLicense + " - ");
        sb.Append("CardNumber: " + req.CardNumber + " - ");
        sb.Append("Address: " + req.Address);

        return sb.ToString();
    }
}
My first thoughts were something along of the lines of "What the.. this is insanely stupid code." This must be a PITA to maintain and be extremely error-prone. Looking at the solution now, it looks simple enough to move that to some infrastructure and have the conversion done by something more generic. Serializing to JSON comes to mind; interpretable by man ánd machine.

Trying not to jump to conclusions, I looked for one of the remaining team members, and asked why they made that decision. "Well", he said, "Those remote service calls are expensive as is; it's a slow connection, we have to encrypt everything going over the wire, and we can't make them asynchronously. We optimized where we could. Including logging."
I asked if they found serialization to be so expensive that it could warrant all the monkey code. He said yes, but that he couldn't vouch for the decision since they never measured.

Later that day, I took five minutes to see how the two really compare. I have this code snippet lying around if I quickly want to profile something.
static void Profile(string description, int iterations, Action func)
{
    // clean up
    GC.Collect();
    GC.WaitForPendingFinalizers();
    GC.Collect();

    // warm up 
    func();

    var watch = Stopwatch.StartNew();
    for (int i = 0; i < iterations; i++)    
        func();    
    watch.Stop();
    
    Console.Write(description);
    Console.WriteLine("Time Elapsed {0} ms", watch.ElapsedMilliseconds);
}
I picked an average sized object graph and ran the benchmark.
var req = new PaymentRequest()
{
    Reference = "ABC123",
    NumberOfLicenses = 3,
    PricePerLicense = 15.99,
    CardNumber = "123456",
    Address = "Sunset Boulevard"
};

Profile("Serializing a request.", 1, () => Newtonsoft.Json.JsonConvert.SerializeObject(req));
Profile("Doing it manually.", 1, () => Convert.PaymentRequest(req));
This yielded following results.
Serializing a request. Time Elapsed 0 ms
Doing it manually. Time Elapsed 0 ms
Neglectable.

Turning up the number of iterations to 100 produces following results.
Serializing a request. Time Elapsed 9 ms
Doing it manually. Time Elapsed 1 ms
This time around, we see a huge relative difference; doing it manually is 9 times as fast. The absolute difference is still neglectable though.

As it turns out, for this specific scenario, with this specific serialization library, the overhead of serialization would be very tolerable. Other serialization libraries might produce less tolerable results though. It's important to measure this stuff; I'm (re)learning almost daily that assuming is a mistake.
Measurement is the first step that leads to control and eventually to improvement. If you can't measure something, you can't understand it. If you can't understand it, you can't control it. If you can't control it, you can't improve it. - H. James Harrington

Monday, August 20, 2012

ASP.NET Web API error detail policy now defaults to the custom errors configuration

While working on an already updated ASP.NET Web API project, I noticed an extra value in the IncludeErrorDetailPolicy enumeration. The IncludeErrorDetailPolicy configuration tells the Web API host when it's allowed to include full error details in responses. Before updating, the RC version of the IncludeErrorDetailPolicy enumeration only had three possible values: LocalOnly, Always and Never. With the released version comes a new value: Default.
// Summary:
// Use the default behavior for the host environment. For ASP.NET hosting, use
// the value from the customErrors element in the Web.config file. For  self-hosting,
// use the value System.Web.Http.IncludeErrorDetailPolicy.LocalOnly.
Default = 0,
So now, with the default policy enabled, the ASP.NET host will look inside the custom errors element in the web.config by default, to determine whether it should include error details or not. This makes for the error detail behavior of Web API to always be consistent with the one of MVC and WebForms in the same application.

Those that have yet to have their first look at ASP.NET Web API will probably take this behavior for granted. But we will never forget.

Sunday, August 19, 2012

My learning resources distilled

I have picked up a few new tools this summer (MongoDB, NancyFx and WebAPI), and it occurred to me that I've built certain habits these last few years in how I make use of all the learning resources out there.
I tried to identify all of them, to then categorize them, to finally order them according to in which phase of my study process I use them.

The written and spoken word

The first thing I look for online is documentation. It might be a coincidence, but all the documentation I found for these three tools was excellent. It seems obvious, now more than ever, that the early adaptation of new tools can be proportional to the quality of their documentation.

When there is no documentation - studying a concept instead of a technology, when the documentation is too dispersed, or when the quality is too low, I might order a book. The problem with technical books though, is that they are often too lengthy. I think the first publisher that cuts down on the number of pages, while guarding quality, will be pleasantly surprised by demand. I really like the "Little book" format Karl Seguin has been experimenting with.

When there is no documentation, and there are no books, I find myself turning to the spoken word. In general, I prefer reading over watching and listening because I can do that at my own pace, skip chapters, or quickly skim back for a look-up.
Although often on the slow side, the videocasts from Pluralsight are not bad. If you don't want to subscribe to one of the paid videocast services, you can also watch recorded conference sessions instead. Before I commuted by train, I also listened to relevant podcasts.

Getting intimate

As I wrote earlier, I am a strong advocate of learning by doing. Find something small, and build it.
When I'm using a new tool, I also tend to regularly peek at its internals to improve my understanding of how the tool works under the covers. It's never a bad idea to get intimate with the tools you rely on.
This becomes extremely easy when the tool you are using is OSS and uses a source control provider that makes it easy to browse or fork the source - read GitHub. If the tool isn't OSS, you can still decompile the sources using one of the free decompilation options out there. If you have a Resharper 6 license, you can even decompile straight from Visual Studio.

Sipping from the fire hose

Once I start building something, I crank open the fire hose - Twitter Search, and take a sip. Although trying to consume all the information on a subject is hardly bearable, and feels as being waterboarded at times, I think it's a necessary evil. Until you have assembled your personal digest, that is.

I'll do my best to keep up with the Twitter Search feed for a while, and try to filter knowledgeable people to follow, blogs to subscribe to, and repositories to watch. Once I have a qualitative list of subscriptions, I'll shut off the fire hose, and limit my consummation to my personal high quality-to-noise digest. This digest will serve as one of the biggest aids in continuously improving my knowledge of said tool.

Free pop quizzes

I always thought that if you really want to become good at something - become an expert, you must have spent time in the trenches, knees deep in shit. Experiences like this force you to broaden your understanding of a technology.
Next to problems you encounter producing things, you can also simulate this experience by following Stackoverflow questions and commit to help solving them. The less ambitious can also wait a while, and just skim over the questions and their answers. The same goes for mailing lists obviously.


I'm very curious to hear about the techniques you use to unravel all the information out there. 

Thursday, August 16, 2012

A really quick look at ASP.NET Web API Help Pages

While skimming over future features of ASP.NET Web API, I came across the ASP.NET Web API Help Page feature. I couldn't find an introduction online, and the Nuget package has only been downloaded 16 times, so I had to have a really quick look. I documented my baby steps below.

So, I had a really simple ASP.NET MVC4 project, with one API controller exposing tweets.
public class TweetsController : ApiController
{   
    /// <summary>
    /// Get a <c>Tweet</c> by its identifier.
    /// </summary>
    /// <param name="id">Id of the tweet</param>
    /// <returns>A <c>Tweet</c></returns>
    public Tweet Get(int id) 
    {
        //...
    }

    /// <summary>
    /// Adds a <c>Tweet</c>
    /// </summary>
    /// <param name="tweet">An instance of <c>Tweet</c></param>.
    public void Post(Tweet tweet) 
    { 
        //...
    }      

    /// <summary>
    /// Deletes a <c>Tweet</c> by its identifier.
    /// </summary>
    /// <param name="id">Id of the <c>Tweet</c></param>
    public void Delete(int id) 
    { 
        //...
    }
}

public class Tweet
{
    public int Id { get; set; }

    public string Message { get; set; }

    public string User { get; set; }
}
To add the Help Pages feature, I added the Nuget package to my MVC project. This adds a new area, containing a bunch of code.


Browsing to the area's default url now, you already get to see something like this.



There are a few things missing though. Most importantly, where is the documentation I provided in the controller?

To expose the existing documentation, enable generation of the XML output.


Then open the HelpPageConfig class, and set the documentation provider - we can use the default implementation included in the package.
config.SetDocumentationProvider(new XmlDocumentationProvider(
    HttpContext.Current.Server.MapPath("~/App_Data/Documentation.xml")));



That looks better. There is one more thing I'd like to get working: samples of the different representations.
This can also be configured in the HelpPageConfig class.
config.SetSampleObjects(new Dictionary<Type, object>
        {
            {typeof(Tweet), new Tweet() 
                { 
                    Id = 1, 
                    Message = "sample_message", 
                    User = "sample user"  
            }},               
            {typeof(string), "sample string"},
            {typeof(IEnumerable<string>), new string[]{"sample 1", "sample 2"}}
        });
This produces a nice sample for consumers of the API.


All of this took me five minutes to get working. There seems to be a lot of room for optimizations and customizations. You can configure a bunch of stuff in a centralized spot, tweak the code - it's not in a separate assembly, and you can adapt the views however you like.