Monday, December 31, 2012

2012 Annual Review

It's that time of the year again where I take some time to look back on the year passed and peek at the year ahead. This is mostly a post I like to write up for myself, forcing reflection.

The biggest personal changes this year were moving out of my parents' place, and my girlfriend's decision to study medicine after graduating as a Master of Arts earlier this year. I had no idea how things would turn out once we lived together, but so far we have been doing great.

My career

September marked one year working for Euricom. I'm still at my first client, and while I definitely regularly have my share of the enterprise blues, I also learned a lot working on several projects: from maintaining legacy to leading green fields. Fairly, I often doubt if traditional enterprise life is it for me though.

Blog

I published 66 posts this year, which seems to be consistent with previous years.

While writing has turned into a habit over the years, I often struggle with deciding what I should write about. Any reason is probably a good one though; documenting, sharing opinions and experiences, or just writing for the sake of writing... But I still feel like I should add value and contribute something worthwhile when I spend time on this blog.

The topics haven't changed that much this year. I still write often on Web technologies (ASP.NET, NancyFx and REST) - something I didn't get to spend much time on at work this year, but I also wrote about architecture, testing and NoSQL. Next to those topics, I also penned some of my experiences gained doing experimental side projects.
In 2013, I intent to get more out of my comfort zone, and to also write down thoughts which are not that strongly held.

Page views have more than doubled (173k) compared to 2011, which I'm humbled by, but I seem to care less about that than a little while ago. It had even been over a month since I logged into Google Analytics.


Community

It very much depends on what I'm doing that day, but I still spend quite some time on Twitter. I often blurt out thoughts on software, and regularly, someone who knows better than me replies, and challenges my looks on software. That's how I get value out of Twitter. That, and animated GIFs of course.

In real life, I went to see a few interesting speakers at local user groups, and gave a talk myself at Web.NET Europe Milan - international speaker, wooh. I want to applaud the organization of Web.NET Europe once more; they did a great job.

In general, I started to value artificial sponsored user groups less. They have their place, and they do add value, but it's not something that I feel I need to be a part of. Having some beers with a few likeminded developers suits me better.

Travelling

Buying an apartment and all of that, we didn't get to make any epic travels this year. I got to visit three different countries though: Italy, Tunisia, Italy again, and France. Two out of four were sponsored by my job; not bad.

We're setting out for a big trip again in 2013. We fell in love with America two years ago, so we've been researching the South-West again - but closer to the Mexican border this time. Other destinations are still open for discussion though.

Projects

I spent a considerate part of 2012 building small things on the side. This didn't result in anything useful, but most importantly, it got me in the habit of working on my own things, and I put together a toolset which enables me to ship things fast (NancyFx, Twitter Bootstrap, MongoDB, AppHarbor, ...).

The first project I did in 2012 was Docary. This was supposed to be something between a timesheet and a diary. This idea quickly died after I couldn't commit to using it myself. Technically I played with early versions of jQuery mobile and EF, and learned why you shouldn't overlayer your MVC application - or any web application for the matter.

For the second project, I and Davy Brion worked on TopDevLinks. This was supposed to be a linkblog implementation on top of ASP.NET MVC, CQS and MongoDB. We both lost interest in the concept eventually.

I made my girlfriend a petite portfolio site built on top of NancyFx, the FlickR API, and some jQuery plug-ins. Right now, it's actually running as a static site on GitHub pages though.

I already blogged about the next project: Kill Long Meetings. This one is still around, but quickly floundered into forgetfulness after its five minutes of Twitter fame.

The last two things I worked on this year were my Antwerp Open Data submissions. I jotted down some technical details here. The Nancy.AspNetSprites.Razor package also sprouted out of that effort (almost 100 downloads!).

Now that I look back at it, I did write quite some code after my hours this year... Next to being in the moment of building and shipping stuff, I genuinely believe it makes me better at my job.

I hope to be able to do as much in 2013, but to also let it be something worthwhile. I'll be actually already shipping something new in a few weeks if all goes well.

Sport/Health

The biggest gift I gave myself health wise this year, was giving up on smoking; I lit my last one over six months ago!

Moving to Antwerp, and not having any area to put my weights, I've joined a gym again, where I'm doing a two times a week compound routine. That should make me stronger again; I've lost a lot of beef since picking up running three years ago.

I haven't given up on running - living close to the city park helps. I just missed the 1000km (or 621 miles) milestone this year, which is kind of lame, but I'm far from complaining.


I don't know what I hope for in this area in 2013. It's starting to dawn on me that, even though those goals I set for myself have pushed me further than I thought I was capable of - I couldn't jog 1km without a break three years ago, they are not that important. It eventually comes down to nourishing your body, and making use of it in every way possible while you still can.

Conclusion

Earlier this week, I strolled down the main street of Brussels and crossed a beggar with one leg and no arms. Next to feeling extremely sorry for that man, I felt truly blessed; I'm healthy, loved, have more than I need, and I get to do what I like for a living. I need to remind myself more of that in 2013 - I can be a whiny bitch.

I plan to keep course in 2013, but as a better man.

How was your year? What do you plan for in 2013?

Now, if you'll excuse me, I'm going to help a hand in preparing the festivities before the girlfriend really gets in a faul mood.

Holy crap, when I sat down to write this, I didn't expect to write a whole book. I'm wondering how many people actually made it down here. 

Sunday, December 23, 2012

2012's most read posts

I look forward to writing this post each year; it's by far the easiest one to write, and it provides me with an occasion to look back on previous years (2009, 2010, 2011). It's entertaining - and shameful too - to see what kept me busy back then, which opinions I held, and which technologies and techniques are still relevant in the meanwhile. Anyways, here it goes.

The most read post this year, with 37.797 page views, is How a web application can download and store over 2GB without you even knowing it. I just ran this experiment again with the latest Chrome build, and apparently the Chrome team hasn't addressed these concerns in the meanwhile; the browser still behaves exactly the same. I'm guessing usage stats might prove that it's not worth the effort.

The second most read post, with 14.371 page views, is Learning: the Hacker Way. I still firmly stand by this one today. Passive learning can only take you so far; it's crucial to supplement passive learning with a high volume of getting your hands dirty.

With 14.085 page views, Commuting: have you done the math? comes in third. Commuting wastes a good part of your life; optimizing your commute can significantly improve the quality of your average day. Readers shared some interesting experiences in the comments on this one.

Having 5.215 page views, the post Why I will always love RSS is the penultimate item in this list. This writing is basically a rant on how, although I somewhat understand the motivation behind public companies fencing their gardens, it is sad to see that even peers seem to advertise the death of RSS, while it's RSS that has enabled me to follow my favorite sources of information without having to rely on others to tell me what to read, and without having to be connected the whole damn time.

With 1000 views less, making it the last item in the list, is the post HTML5 Offline Web applications as an afterthought in ASP.NET MVC. It was this post that landed me a paid article on InfoQ. I should revisit this topic again, I'm curious to see how many websites are already making use of this HTML5 super cache to improve performance, or to truly provide offline access.

Thank you for reading in 2012!

Sunday, December 16, 2012

Released: My Antwerp Open Data submissions

A little while ago the city of Antwerp released their Open Data initiative, and it included a meetup where you could show something you built, build something on the spot, or pitch your ideas. When I first heard of the initiative I had nothing going on the side, and was looking for something tangible I could build to try out a few technologies. I couldn't come up with an original idea, and ended up building two web applications using the Open Data datasets: Culture Mosaic, and Where to pee in Antwerp?
Since I was bedridden due to the flu, and my ideas were not that great, I eventually ended up not going to the meetup, but I figured I would share some stuff I picked up along the way here.

Culture Mosaic

The first web application uses the culture dataset to display a masonry of cultural attraction images.


The images available vary in size, and some turned out to be quite large. If I wanted to display all the images at once, I had to make them smaller; resize them. For this task, I found the ImageResizer library on Nuget. Although you can reference remote images using an ImageResizer extension, I chose to download the images locally, to then resize them and to finally cache them on disk.

I run these tasks in parallel and wait for all of them to finish, using the TPL.
private void TryToDownloadAllImages(Culture culture)
{
    var tasks = new List<Task>();

    foreach (var item in culture.Items)
    {
        var localPath = Path.Combine(_pathConfiguration.ImagesFolderPath, item.ImageName);
        if (!File.Exists(localPath))
            tasks.Add(CreateImageDownloadTask(item.Image, localPath));
    }

    Task.WaitAll(tasks.ToArray());     
}
Downloading a remote file is straightforward using the WebClient class. Right after downloading the image, I use the ImageBuilder singleton to resize the image to a width of 340px.
private Task CreateImageDownloadTask(string imageUrl, string localPath)
{
    return Task.Factory.StartNew(() =>
    {
        using (var client = new WebClient())
        {
            try
            {                        
                client.DownloadFile(imageUrl, localPath);

                ImageBuilder.Current.Build(
                    localPath, 
                    localPath, 
                    new ResizeSettings("maxwidth=340"));
            }            
            catch (ArgumentException) 
            {
                // filename malformatted; swallow
            }
        }
    });                    
}                
Now that I have my resized images, I can render them in my view. This is NancyFx with the Razor view engine by the way.
<div id="loading">
    Loading..
</div>      
<div id="container" style="display:none;">
    @foreach (var item in Model.Items)
    {               
       <a href="@item.Url">
          <img src="/Content/images/@item.ImageName" alt="@item.Name" title="@item.Name" />
       </a>             
    }
</div>
To achieve the masonry effect, I used this jQuery plug-in. The one tricky part here is to only initialize the effect after all the images have really loaded. For this, I used the imagesLoaded plug-in. 
$container.imagesLoaded(function () {
    $loading.hide();
    $container.show();
    $container.masonry({
        itemSelector: 'img'
    });
});
One last thing I wanted, was to center the masonry on screen, but since I didn't want to use a fixed width, I had to dynamically recalculate the optimal width each time the window is resized. 
var setContainerWidth = function (container) {
    var imageWidth = 340;
    var bodyWidth = $("body").width();

    var numberOfImagesNextToEachother = Math.floor(bodyWidth / imageWidth);
    var optimalWidth = (imageWidth * numberOfImagesNextToEachother) + (numberOfImagesNextToEachother * 4);

    container.css("width", optimalWidth + "px");
};

$(window).resize(function () {
    setContainerWidth($container);
});
Since we have the width of the images (340px), we can use trivial math to calculate the optimal amount of width we can consume.

You can find the working bits on AppHarbor. Source is on GitHub.

Where to pee in Antwerp?

The next application is a mobile web application that uses the public toilets dataset to show the nearest toilets based on your location.


I wanted to introduce a constraint client-side of having to work with a somewhat crooked dataset, that's why I just return the dataset using Nancy as is.
public ToiletsModule()
{
    Get["/Toilets"] = p =>
    {
        var response = Response.AsFile(@"Content\json\toilets.json");
        response.ContentType = "application/json";

        return response;
    };
}
At the client, I'm using jQuery Mobile and angular.js. It had been a while since I had done something with jQuery Mobile, but this version felt very solid (compared to the beta versions I played with before). Turns out though, that to make jQuery Mobile and angular.js play nice, you have to include the jQuery Mobile Angular adapter. The size of all these libraries unminified was a performance hog; Cassette to the rescue.

The first thing you do with angular, is defining a module. A module instantiates, wires and bootstraps the application.
var toiletApp = { };        
toiletApp.module = angular.module('toiletModule', []);
To separate concerns, and make things testable, angular lets you define services for your application using a factory. By using a factory you also make your service an injectable dependency for other parts of your application.
I added a toilet service to this module which makes the ajax call, sorts the results based on nearest by, to eventually make them available to the caller by invoking a callback. The $http dependency is one of angular's core services that lets you do ajax requests.
toiletApp.module.factory('toiletService', function ($http, Toilet) {

return {

    getToilets: function (onSucess, onError, coordinates) {

        $http
            .get('toilets')
            .success(function (data) {

                var toilets = [];

                for (var i = 0; i < data.openbaartoilet.length; i++) {
                    var toilet = data.openbaartoilet[i];

                    var ourToilet = new Toilet(
                        toilet.omschrijving,
                        toilet.point_lat,
                        toilet.point_lng,
                        toilet.betalend === "niet betalend" ? false : true,
                        toilet.straat,
                        toilet.huisnummer);

                    toilets.push(ourToilet);
                }

                if (coordinates) {

                    toilets.sort(function (a, b) {
                        var aDistance = a.calculateDistance(coordinates);
                        var bDistance = b.calculateDistance(coordinates);

                        if (aDistance < bDistance)
                            return -1;
                        if (aDistance > bDistance)
                            return 1;
                        return 0;
                    });

                }

                onSucess(toilets);
            })
            .error(function() { onError() });

    }

};
You might have noticed the Toilet dependency; into this class I project the items returned from the ajax call.
toiletApp.module.factory('Toilet', function () {               

var Toilet = function (name, lat, lng, isPaying, street, houseNumber) {
    var lastCalculatedCoord;
    var lastCalculatedDistance;

    this.name = name;
    this.latitude = lat;
    this.longitude = lng;
    this.isPaying = isPaying;
    this.street = street;
    this.houseNumber = houseNumber;
    this.lastCalculatedDistance = function () {
        return lastCalculatedDistance;
    };
    this.calculateDistance = function (coordinates) {
        if (coordinates) {
            if (lastCalculatedCoord == coordinates) {
                return lastCalculatedDistance;
            }

            var currentPosLatLon = new LatLon(coordinates.latitude, coordinates.longitude);
            var destinationPosLatLon = new LatLon(this.latitude, this.longitude);
            var dist = Math.ceil(currentPosLatLon.distanceTo(destinationPosLatLon) * 1000);

            lastCalculatedCoord = coordinates;
            lastCalculatedDistance = dist;

            return lastCalculatedDistance;
        }
        return null;
    };
    this.getMapUrl = function () {
        var coord = this.latitude + "," + this.longitude;

        return "...";
    };
};
The Toilet class also has a method to calculate the distance from the item to any given coordinate. I first wanted to make use of the Google Distance Matrix API to calculate the actual travel distance, but that didn't really work out to well for more than 100 items; you can't batch your requests, and the free version of the API has a threshold on the amount of queries. I turned to calculating the great-circle distance instead, using this library.

Since we want to make use of the client's coordinates, I also defined a geo service, which makes use of the HTML5 GeoLocation API to query for the user's coordinates.
toiletApp.module.factory('geoService', function() {

    return {

        getLocation: function (onSucess, onError, onNotAllowed) {
            if (navigator.geolocation) {

                navigator.geolocation.getCurrentPosition(

                    function (position) {
                        onSucess(position);
                    },

                    function (err) {
                        onError(err);
                    }

                );

            } else {
                onNotAllowed(message);
            }
        }

    };

});
After defining these pieces, I injected them into the ContentController. The controller is responsible for augmenting angular's scope, which is the connection to our view, our model. Properties and methods are made available to the view by augmenting the $scope variable. There is quite some not that interesting tinkering going on in the controller, so I left it out for brevity.
toiletApp.ContentController = function ($scope, geoService, toiletService) {
    $scope.toilets = [];
    ...
}
After setting ng-app and ng-controller, we can make use of angular's directives and filters to render our page. Directives are - as the documentation puts it - a way to teach HTML new tricks. Think of it as HTML helpers. Filters are used to format data.

The view binds each toilet to a new collapsible div. The content of the div shows some basic information, and a button which shows the toilet on a map.
<html ng-app="toiletModule">
    <head>
        ....
    </head>
    <body>
        <div data-role="page" id="home" ng-controller="toiletApp.ContentController"> 
            
            <div data-role="header">
                <h1>Where to pee in Antwerp?</h1>
                <a href="#" data-icon="refresh" ng-click="reload()">Reload</a>                
            </div> 
            
            <div data-role="content">
                
                <div class="content-primary">    
                    <p>{{ message }}</p>
                   
                    <div data-role="collapsible" 
                         ng-repeat="toilet in toilets | limitTo: itemsLimit()">
                        <h3>
                            {{ toilet.name }} 
                            ({{ toilet.lastCalculatedDistance() | distance }}) 
                            {{ toilet.isPaying | isPayingDollarSign }} 
                        </h3>                        
                        <p><strong>Name: </strong>{{ toilet.name }}</p>
                        <p><strong>Distance: </strong>{{ toilet.last..() | distance }}</p>
                        <p><strong>Paying: </strong>{{ toilet.isPaying | isPayingYesNo }}</p>
                        <p><strong>Street: </strong>{{ toilet.street }} </p>
                        <p><strong>Housenumber: </strong>{{ toilet.houseNumber }} </p>
                        <a href="{{ toilet.getMapUrl() }}" data-role="button">
                            Show location on a map
                        </a>
                    </div>                     
                             
                    <div ng-show="hasMoreItemsToShow()">
                        <button ng-click="showMoreItems()">Show more toilets</button>    
                    </div>        
                </div>        

            </div> 
            
            <div data-role="footer">
                <h4>Made possible by Antwerp Open Data</h4>
            </div> 
        </div>
    </body>    
</html>
You might notice how I used filters to format the isPaying and distance property. These are also defined in the module.
toiletApp.module.filter('isPayingDollarSign', function () {
    return function (value) {
        return value ? "$" : "";
    }
});

toiletApp.module.filter('isPayingYesNo', function () {
    return function (value) {
        return value ? "Yes" : "No";
    }
});

toiletApp.module.filter('distance', function () {
    return function (value) {
        return value ? value + "m" : "";
    }
});
You can find the working bits on AppHarbor. Source is on GitHub.

Summary

I can only applaud the Open Data initiative; my hope is that the datasets get more interesting though. Which also makes me wonder if the government is even measuring the more interesting stuff?

All in all, these two applications aren't very innovative. Yet, they did give me the opportunity to experiment with some stuff I can't play with on the day job (yet). I'm always amazed how quickly you can slap together a small application using the available building blocks out there. I'm most content with picking up angular.js; I will be advertising this framework in a professional context from now on.

Wednesday, December 12, 2012

Show More pagination with angular.js

I built my first application with angular.js over these last few weeks (not during business hours), and although I still have lots and lots to discover and learn, I think I somewhat grasp the basics.

In the application I built, I had to implement paging because rendering all the items at once was too slow on mobile devices (on my Windows Phone 7 anyways). The paging variant I decided on was the 'Show More' technique.

Let me walk you - as an introduction to Angular - through a simple application that uses paging for a list of 24 items.

We start by defining a module. In an Angular module, you should instantiate, wire and bootstrap parts of your application.
var module = angular.module('module', []);
We can already bind the module to our empty view.
<div ng-app="module">
</div>
In our module we want to define an itemService which will return an array of 24 items. By using a factory method we make our service an injectable dependency for other parts of our application.
module.factory('itemService', function() {
    return {
        getAll : function() {
            var items = [];
            for (var i = 1; i < 25; i++) {
                items.push('Item ' + i);                       
            }
            return items;
        }
    };              
});
Now that we have a service that can give us items, we want to work towards actually rendering these items. To do that, we'll have to define a controller. An Angular controller lets you augment an instance of Angular's scope, which in its turn serves as your connection to the view. 
ListController = function($scope, itemService) {
    $scope.items = itemService.getAll();    
};
Notice how we use dependency injection to inject an instance of the itemService.

All our items are available on the scope now, but we still have an empty view. Let's fix that.
<div ng-app="module" ng-controller="ListController">
    <ul>
       <li ng-repeat="item in items">
          {{ item }}        
       </li>               
    </ul>
    <button>Show more</button>    
</div>
Items are being rendered making use of Angular's built-in ng-repeat directive. Directives are - as the documentation puts it nicely - a way to teach HTML new tricks. MVC folks may think of it as HTML helpers.

What's left to do in this example, is what we initially set out to do; implement paging. For that, we'll first add some extra variables to the controller, and some new methods on to the scope. These should be self-descriptive.
ListController = function($scope, itemService) {
    var pagesShown = 1;
    var pageSize = 5;
    $scope.items = itemService.getAll();
    $scope.itemsLimit = function() {
        return pageSize * pagesShown;
    };
    $scope.hasMoreItemsToShow = function() {
        return pagesShown < ($scope.items.length / pageSize);
    };
    $scope.showMoreItems = function() {
        pagesShown = pagesShown + 1;         
    };
};​
We can now make use of those new methods on the scope in our view. 
<div ng-app="module" ng-controller="ListController">
    <ul>
       <li ng-repeat="item in items | limitTo: itemsLimit()">
          {{ item }}        
       </li>               
    </ul>
    <button ng-show="hasMoreItemsToShow()" ng-click="showMoreItems()">Show more</button>    
</div>
We made use of directives to add behaviour to the 'Show more' button and used the limitTo filter to limit the number of items rendered. Filters are used to format display data.

Here is the full jsFiddle.

I'm pretty sure you can abstract paging into a reusable component, but I thought this scenario shows a good bunch of basic Angular concepts.

So what are your thoughts on Angular? Pretty slick, right? Or are you already using Angular?

Sunday, December 9, 2012

It's not cake we are baking

I recently watched a talk on Vimeo where Christin Gorman talks about how cookie dough relates to Hibernate; why use the generic, bloated and one-fits-all solution when you can mix together your own yummy cookie dough? We should aspire to be the Gordon Ramsey of software, not the college student who can only cook Ramen noodles. If you haven't watched or listened to her talk, you should; it's only a few minutes long, and she brings it really well. Go ahead, I'll wait.

When I first watched her talk, it rubbed me the wrong way, but I couldn't yet figure out the flaw in her plea. It recently dawned upon me that we are not in the business of baking cakes, but we are in the business of serving hungry people. And those people really don't care which ingredients were used to bake your cake; they only care about how it tastes, how fast it's served, and how much they're paying for it. And although the romantic in me really sympathizes with her view, the realist just can't. Over these few years only, each and every attempt of a hand-rolled ORM I've seen, ended with a horrible aftertaste. There is no way that you spend as much time building a good API from scratch as you do setting up an existing one. Writing and maintaining your own ORM is not a trivial thing to do, and there are already so many flavors* - from grandma's recipe to decadent supreme deluxe - to pick from, that it's hardly ever a justified decision. Read the package and choose the option that satisfies your daily caloric needs, and go from there. You can still sweeten, or swap some ingredients, before serving. And if you really can't resist - which is perfectly healthy, bake all the cake you want, at home. Maybe, one day, you can bring a successful one to work.

* NHibernate, Entity Framework, Dapper, Massive, Simple.Data, LLBLGen Pro, OpenAccess, SubSonic ...

Sunday, December 2, 2012

Some notes on performance tuning with NHibernate

A few weeks back, I spent an intensive day performance tuning parts of a, to me, relatively unfamiliar part of our codebase. Like it often is, the biggest optimizations were to be found in how we work with the database. Now, I don't consider myself to be an NHibernate expert; I read this book and have used it on two projects, but in the end I just do my best to avoid doing stupid things with it. The topics discussed below are mostly common knowledge for long time NHibernate users, but I thought it might be convenient for others to just summarize them, and add references to other, more in-detail, posts.

Under the covers

When you're looking into optimizing, you probably want to have a look at what's really going on. You could do this by turning on NHibernate's log4net debug logging. This might be good enough for some scenarios, but it's not really convenient for when there is lots and lots of stuff happening. Instead, you might want to look into NHibernate Profiler. It's trivial to get started with, yet the feedback it provides is very powerful: next to session statistics and executed queries, you also get alerts which suggest techniques to improve your code. I need to use this tool more often just to get a better grip of the NHibernate internals.

The fastest query is the one that isn't

Going to the database is probably one of the slowest things your application is going to do. If you can avoid it, do so.

If you're using a stateful session, NHibernate will track your entities, and store them in its first level cache. When you get these items by id later on, you avoid going to the database. So instead of querying the database for the same entity multiple times in the same session, do it once, and get the entity by its id on subsequent calls.

When you're having a hard time keeping those ids around, consider introducing a light-weight datastructure, such as a dictionary, which can help you build a small look-up cache. This could also be a sign that you might want to reconsider your identity strategy though.

Working with batches

An ORM really isn't the best tool to do bulk inserts or updates; look at SqlBulkCopy instead.

If your batches are still relatively small, and you opt to stay with NHibernate anyways, there are two things you can do which will improve performance tremendously: use a stateless session and configure batching.

Switching to a stateless session is simple enough. Do take into account that some features won't work anymore: lazy loading, caching, cascading and implicit updates. Setting up batching is also just a matter of configuration. The most important thing to remember is to use an appropriate identity generator; batching will only work when the application is responsible for generating the ids. I'm using a HiLo generator, but GUIDs or assigned ids will work too.