Automated JavaScript tasks with Grunt

I am currently working on a JavaScript only project and was looking for a way to automate some of the JavaScript processes. I was primarily looking for a way to get my tests automated, more or less the same as when you check-in code, your tests get automatically run.

For this I chose grunt, which can easily execute different tasks, not only can it automate your test runs, but it can also automate other JavaScript tasks, like minifying your files, running them through jslint, ... You can find a complete list on the grunt site.

The project I am working on is a Windows 8 application (not XAML this time, but JavaScript, HTML5, ...). So I am working in a .Net environment with a solution and a couple of projects.

Grunt Basics

To get started with grunt, first thing you will need to do is install node. All tasks you will be running in grunt will need to be installed using nodes packaging system npm (you can compare npm to the gem install process in ruby or nuget in .Net). After installing node, you can install grunt using npm. Once you've done this, you can start creating a grunt file in your web projects. A grunt file contains different tasks that need to be run, like uglify, jslint, minify, ...). You can find all info on getting started here.

Running jasmine tests with grunt

Now, the thing I wanted to be able to do, was run my jasmine tests though grunt. For this I have some simple jasmine tests set up. I could already run these through the jasmine browser runner. I also tried out a setup that ran my jasmine tests browserless through phantomjs and with the help of the build in Resharper runner, which works great. 

For running these same tests with the help of grunt I needed to install a couple of extra grunt tasks through npm: contrib-jasmine and contrib-connect. The first one, obviously is needed to run your jasmine tests. The second one, connect, can be used to set up a browserless environment for grunt, so it won't start up a browser session with every run of your grunt file. Connect can also be replaced by node itself.

The grunt file itself contains tasks for connect and jasmine:

module.exports = function (grunt) {

    // Project configuration.
    grunt.initConfig({
        pkg: grunt.file.readJSON('package.json'),
        jasmine: {
            src: '<%= pkg.name %>.Web/*.js',
            options: {
                vendor: ['<%= pkg.name %>.Web/Scripts/*.js', '<%= pkg.name %>.Web/lib/*.js'],
                host: 'http://127.0.0.1:<%= connect.test.port %>/',
                specs: '<%= pkg.name %>.Web/specs/*.js'
            }
        },
        connect: {
            test: {
                port: 8000,
                base: '.'
            }
        }
    });

    grunt.loadNpmTasks('grunt-contrib-jasmine');   
    grunt.loadNpmTasks('grunt-contrib-connect');   

    // Default task(s).
    grunt.registerTask('default', ['connect', 'jasmine']);
       
};

You can see that in the jasmine task, I use the port number of my connect task. In my default task, at the bottom of the file, I first fire up the connect port and once that one is running, I let grunt run my jasmine tests.

Once I now issue the grunt command at the command line,I can see it running my tests.

One step further: automated build

Running my jasmine tests locally this way, is cool by itself, but it would be even nicer if I could integrate this in some sort of automated build process. Ie. check-in my code, trigger the grunt task runner, which will run my tests, run jslint, run uglify, ... Basically, get a finished product at the end of the pipeline. 

The project I am working on right now, I got it using tfsservice, since I wanted to find out what its' pro's and cons are. This means, for automating my build, I had to rely on msbuild to do the trick for me. Now, tfsservice has got iisnode installed on it, so it should be possible to have it run grunt tasks as well. 

To get this working, I altered some things in my grunt setup. First of all, I reinstalled grunt and all the packages it uses, so that they got saved locally into my project. This means reissuing the npm install command for grunt, contrib-jasmine, ... but WITH the --save-dev option. This will create a packages folder inside your project with all necessary files for each plugin saved locally inside your project. Once you commit your project to your source control system, all packages will also be present locally on the build server and don't need to be installed globally on your build server (something you just cannot do on tfsservice, being, that you're not sure on which build server you will be running next, which means reinstalling all packages with every run, which is just time consuming. You just don't want to do that.). 

I additionally installed the grunt-cli package locally (so, again, with the --save-dev option). Grunt-cli is the grunt command line interface. It gives you the grunt command locally (read: on your build server). 

Once you have done this, you can alter your csproj file of the project you want to use grunt in (remember: I am working in a .Net context here). For this, I added an additional target at the end of my csproj file: 

  <Target Name="RunGrunt">
    <Exec ContinueOnError="false" WorkingDirectory="$(MSBuildThisFileDirectory)\.." Command="./node_modules/.bin/grunt --no-color" />
  </Target>

This target uses my locally installed version of grunt. The --no-color option I added to get the grunt output nicely formatted. If you don't add this option, your output will look pretty messy.

You will also need to tell your build process to also run grunt after your build. So also add:

<Project ToolsVersion="4.0" DefaultTargets="Build;RunGrunt" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

After these alterations, you will see your grunt output in the output window of Visual Studio after building your project. This should also make it possible to automatically run grunt on my tfsservice.

For this I made a build definition for my tfsservice. After checking in my code, the grunt task gets run as well, only, for now, it exits with errors (return fs.existsSync(filepath);
              ^
  TypeError: Object #

Async VI: JavaScript Promises

The previous posts (Async basics, Exception handling, Cancellation, Progress reporting, void methods) all talked about async from a C# point of view. But let's not forget there is another very important language coming to WinRT and that's JavaScript in combination with HTML5. The cool thing is you can use more or less the same async capabilities here that you find in C#. Big difference is the syntax you will be using. Since the async syntax will be closer to what you are used to in JavaScript.

The UI in the JavaScript BlitzHiker application looks pretty much the same as its C# counterpart. It also has a page to publish a hikerequest, which does three big things, initialize the Bing map we use, find my own location and publish my hikerequest.

function initialize() {
    initializeBing();
    findMe();
    publish();
}

The findMe function has the first bit of asynchrony.

function findMe() {
    if (myPosition) {
        setMePushpin(myPosition)
    }
    else {
        if (!loc) {
            loc = Windows.Devices.Geolocation.Geolocator();
        }
        if (loc) {
            loc.getGeopositionAsync()
            .then(setMePushpin, errorHandler);
        }
    }
}

The call to GetGeopositionAsync is an asynchronous call. We can subscribe a callback to this method by using the then function. So no special keywords like await and async here, but more JavaScript style callbacks. You also see me using an error handler. In the JavaScript case, it is not with try catch that you handle exceptions, but with an extra error handler function you can send along.

I also made use in this function of the loc variable, that I defined globally. My original version of this method looked like this:

function findMe() {
    if (myPosition) {
        setMePushpin(myPosition)
    }
    else {
        Windows.Devices.Geolocation.Geolocator()
            .getGeopositionAsync()
            .then(setMePushpin, errorHandler);
    }
}

But funny enough only one times out of 10 the pushpin for my position was set correctly. This had me puzzled. Apparently, my local Geolocator had already gone out of scope before the callback came in. That is why I altered this to a global variable.

Another place where I used asynchronous callbacks, is in the publish method, which publishes a hikerequest, gets all the drivers and shows those on the Bing map.

function publish() {
    var caller = new BlitzHikerLib.RestCaller();
    caller.publishHikeRequestAsync()
    .then(function () {
        caller.getDriversAsync()
        .then(function (driverIds) {
            if (driverIds == null || driverIds.length == 0)
                return;
            for (var i = 0 ; i < driverIds.length && i < 200 ; i++) {
                var driverId = driverIds[i];
                caller.getUserAsync(driverId)
                .then(showDriver);
            }
        });
    });
}

You see me use then three times, each time sending a callback function along. Now, in my humble opinion, this syntax doesn't make this code the most readable code in the world. You need to look at it a couple of times to see what's going on. Now, you can actually improve on this by use of a little trick.

function publish() {
    var caller = new BlitzHikerLib.RestCaller();
    cancellable =
        caller.publishHikeRequestAsync()
        .then(function () {
            return caller.getDriversAsync()
        })
        .then(function (driverIds) {
            if (driverIds == null || driverIds.length == 0)
                return;
            for (var i = 0 ; i < driverIds.length && i < 200 ; i++) {
                var driverId = driverIds[i];
                caller.getUserAsync(driverId)
                .then(showDriver);
            }
        })
        .then(null, errorHandler);
}

You can see that for all but one, all then statements are nicely aligned. This can be done since we return the actual asynchronous call from the callback.

The reason I don't use this trick with GetUserAsync, is that, if I were to do that, I would jump out of the for loop that's processing each driver and only the first driver would be shown on the map.

The last then statement is special as well, since it defines a global error handler. Whatever error occurs in whatever callback, this error handler will catch it. Another useful trick.

I also extended this method a bit and actually caught the asynchronous calls in a variable, I called cancellable. Cancellation in JavaScript as well is done a bit differently. Once you define a global variable like this, you can call cancel on it.

function cancelHike() {
    if (cancellable != null) {
        cancellable.cancel();
        var errorDiv = document.getElementById("errorText");
        errorDiv.innerText = "operation cancelled";
    }
}

With this you have more or less the exact same application as I showed in the C# examples. But I do have to say I did need to alter some things in the RestCaller to get all this working for JavaScript. That is something I'll show you in the next (and last) blog post.

Adding navigation to WinRT JavaScript

In the previous post we looked at the anatomy of a basic WinRT JavaScript navigation application. In this post we will add 2 additional pages and navigate between them. I will use the same fragment navigation as used in default.html and homePage.html.

First thing I'll do - just to spare me some typing work - is make two copies of homePage.html and call these page2.html and page3.html. Next, I'll alter the default content of homePage.html (there where it says 'content goes here') and replace it with two link tags. I'll add the same content to page2 and 3 as well, so as to easily navigate between all three pages.

<section role="main" aria-label="Main content">
    <p>
        <div>Navigate to: &lt;/div>
        <div>
            <a href="page2.html">Page 2</a>
            <a href="page3.html">Page 3</a>
        </div>
    </p>
</section>

When you try this out, you can actually navigate to page2 and 3. That wasn't too hard to do, now was it. Only downside with this app is, that the back button is not functioning (it stays disabled both on page2 and page3). The appbar as well is not popping up when you click your right mouse button. Also, when you look at it, the contents of pages 2 and 3 are not shown right, they are shown too far to the left. Obviously we need to add some extra code to fix all this.

So I started to play around with this little application, to see what could be done to get it functioning correctly. First thing I did, was add some breakpoints to the default.js file's navigated function. This showed me that, when clicking the page2 or page3 link, the navigated event was not being hit. Apparently, using a simple a tag with an href attribute doesn't trigger navigation. This had to be done some other way then.

For this I added some extra code to the homePage.js file.

function fragmentLoad(elements, options) {
    WinJS.UI.processAll(elements)
        .then(function () {
            // TODO: Initialize the fragment here.
            var page2Nav = document.getElementById('page2Nav');
            if (page2Nav) {
                page2Nav.addEventListener('click', function () {
                    WinJS.Navigation.navigate('/html/page2.html');
                }, false);
            }
        });
}

The fragmentLoad function is called when the homePage gets loaded. When this happens, we get hold of the page2Nav element (I added an id to the link element). If we find this element, we add an eventlistener for its click event. When clicked, we should navigate to page2.html. A similar piece of code can be added to navigate to page3.

After adding this, you will notice that after you click the page2 link, the navigated event in default.js is being hit. That's what we wanted. But wait, as it gets hit, the application doesn't find our back button, its value stays null. That's not right, it should be able to find the back button.

The thing that's wrong here is the fact I added an href to my a tags. If you remove this and let WinJS just handle all of the navigation, you will see the back button is being found. Now, if you land on page2, the back button can be used to navigate back to the homePage. Even the appbar is functioning as expected.

As cool as this is (hey, I can navigate), the downside of using these techniques is that you get an awfully big buy-in into the WinRT way of building apps. When you choose to develop your applications with HTML5 and JavaScript, I think one of the reasons for your choice is portability, the fact that you can take this same code and run it on a different machine with minimal effort. That is why you should think twice when using WinRT functionality. Or, you'd better start writing wrappers for platform specific libraries. But than still ... Hey, I can navigate.

Navigating with WinRT JavaScript

I am currently playing around with WinRT JavaScript applications. Starting out with a navigation application written from scratch, just to find out how it all fits together. The MSDN library is actually quite scarce when it comes to resources on WinRT. They give you the overall picture of how Metro style apps work, but, for JavaScript applications, the actual information is a bit meager. So I thought it would be nice to dissect a Metro style navigation app and see what code you need to get it up and running. Also beware that the information I will be giving here is based on the first developer preview of Windows 8 and Visual Studio 2011 Express Edition. Things are still very likely to change in the next couple of months.

What I did, was create two Metro style JavaScript applications, one based on a blank project template and one based on the navigation application project template. Let's first see what the navigation application gives us.

When you look at the project of the navigation application, you will see it gives you a couple of predefined html, JavaScript and css files. There are two html files present, a default.html and a homepage.html file. First let's take a look at the homepage.html file. In the head section it includes the JavaScript files you find in the solution, so nothing new or fancy there. In the body there is a div defined with content.

<!-- The content that will be loaded and displayed. -->
<div class="fragment homePage">
    <header role="banner" aria-label="Header content">
        <button disabled class="win-backbutton" aria-label="Back">&lt;/button>
        <div class="titleArea">
            <h1 class="pageTitle win-title">Welcome to NavApp!&lt;/h1>
        </div>
    </header>
    <section role="main" aria-label="Main content">
        <p>Content goes here.&lt;/p>
    </section>
</div>

The main div has a couple of classes defined on it, fragment and homePage. The fragment class we will come back to later on, when we take a look at the JavaScript files of the project. Just remember that it's there on the main div of the homepage.

Within the main div there is a header tag defined. This shows that WinRT JavaScript applications make use of new HTML5 features. The header tag has a role attribute, with a value of banner. There is also a role defined on the section tag a bit further down the homepage, with a value of main. These role attributes are primarily used as selectors in the css and JavaScript files. There is no real functionality associated with them.

The last peculiar attribute values can be found on the button and the h1 tags. Their classes are set to win-backbutton and to win-title respectively. With WinRT JavaScript applications you will find there are a lot of these win-something attribute values. You would think these give you out of the box functionality. Well, they don't. You will find that all magic is really just JavaScript doing it's thing. Those win-something attributes again, are just selectors for the css and JavaScript files. When you run the appication, you will see that the win-backbutton is actually an arrow with a circle around it. That's just plain css:

.win-backbutton::before
{
    font-family: "Segoe UI Symbol";
    content: "\E0D5";
    vertical-align: 50%;
}

So, let's take a look at the default.html page. The header again is just JavaScript and css file includes. The body is more interesting.

<body data-homePage="/html/homePage.html">
    <div id="contentHost">&lt;/div>
    <div id="appbar" data-win-control="WinJS.UI.AppBar" aria-label="Command Bar" data-win-options="{position:'bottom', transient:true, autoHide:0, lightDismiss:false}">
        <div class="win-left">
            <button id="home" class="win-command">
                <span class="win-commandicon win-large">&#xE10F;&lt;/span>&lt;span class="win-label">Home&lt;/span>
            </button>
        </div>
    </div>
</body>

There is a data-homePage attribute on the body tag and it is set to our homePage.htm file. You will also find a contentHost and appbar div. The contentHost makes you suspect that the default.html page is actually the start page of your application and that other html pages will actually be loaded as fragment into it. That is indeed the case here. But then, why, as you run the application, don't I see the appbar that is defined at the bottom. You do get to see all the content of the homePage, but the appbar is nowhere in site. The reason is that, to see the appbar, you need to swipe up, or, if that doesn't work on your device, perform a right mouse click. There's the appbar for you.

So, where does all the magic happen? In the JavaScript files of course. Let's first take a look at the default.js file. There are a couple of functions defined here. Let's not start at the top, but at the bottom.

WinJS.Navigation.addEventListener('navigated', navigated);
WinJS.Application.start();

An eventlistener for the navigated event gets added and the application is started. That's simple, right. Right above these two statements we find the handler for activating the mainwindow.

WinJS.Application.onmainwindowactivated = function (e) {
    if (e.detail.kind === Windows.ApplicationModel.Activation.ActivationKind.launch) {
        homePage = document.body.getAttribute('data-homePage');

        document.body.addEventListener('keyup', function (e) {
            if (e.altKey) {
                if (e.keyCode === WinJS.Utilities.Key.leftArrow) {
                    WinJS.Navigation.back();
                }
                else if (e.keyCode === WinJS.Utilities.Key.rightArrow) {
                    WinJS.Navigation.forward();
                }
            }
        }, false);

        WinJS.UI.process(document.getElementById('appbar'))
            .then(function () {
                document.getElementById('home').addEventListener('click', navigateHome, false);
            });

        WinJS.Navigation.navigate(homePage);
    }
}

This code says as much as, when we launch the application, do the following things:

• Our homepage is set to the attribute value of the data-homePage attribute of the body element.
• If a user presses the alt key and the left or right button, navigate forward and backward. 
• Find the appbar and attach a click handler to the home element. The navigateHome function is defined at the top of the JavaScript file and contains pretty straightforward code.
• And last, but not least, navigate to the homePage.

Right abobe this function, you will find another function definition. 

function navigated(e) {
    WinJS.UI.Fragments.clone(e.detail.location, e.detail.state)
        .then(function (frag) {
            var host = document.getElementById('contentHost');
            host.innerHTML = '';
            host.appendChild(frag);
            document.body.focus();

            var backButton = document.querySelector('header[role=banner] .win-backbutton');
            if (backButton) {
                backButton.addEventListener('click', function () {
                    WinJS.Navigation.back();
                }, false);
                if (WinJS.Navigation.canGoBack) {
                    backButton.removeAttribute('disabled');
                }
                else {
                    backButton.setAttribute('disabled', 'true');
                }
            }
            WinJS.Application.queueEvent({ type: 'fragmentappended', location: e.detail.location, fragment: host, state: e.detail.state });
        });
}

This tells us what happens when we navigate (remember, the previous function actually navigated to the homePage, so this gets called at the start of the application).

• It clones the current fragment state (I am guessing this remembers where we are coming from and in what state the page is at the moment).
• And, hey, next bit is some asynchrony. After the cloning is done, a function gets executed for the fragment.
• It finds the contentHost on the default.html page and appends the fragment to it. There you have masterview (default.html)- detailview (homePage.html) behavior.
• The backbutton also gets some extra functionality. Based on the fact whether we can go back or not, the button gets disabled or enabled and the click handlers get attached.
• Last bit is queuing the fact that the fragment got appended, so the WinRT runtime can do its thing.

On the one hand, that's all simple JavaScript code. On the other hand, if you are used to working with the regular .NET languages, this seems like quite some code you need to write for some basic functionality. 

Let's find out, in a next post, what we need to do to navigate to a new page in our application. 

Doing the dynamic thing (Part II)

After the basic intro given in the first part of this series, it is time for something more advanced. It won’t be rocket science, yet, but you will get more insight into what dynamic coding can do for you.
As mentioned in the previous post, dynamic programming can help you a lot when working with COM objects. I will give an example of this for JavaScript objects in a small Silverlight application. In the next post I will extend this simple example and I will explain interaction with Excel. But first, let’s talk about JavaScript.
Interacting with JavaScript might be something you need in real life. For instance if you have a Silverlight app that is hosted on a page that relies on JQuery a lot. JQuery can provide you with objects your SilverLight application does not know about. How can you interact with these objects if your managed code doesn’t know their type? The answer is to utilize dynamic.
To illustrate this, suppose we have a service running that can give us data about persons. A Person being just a DTO class with a FirstName and LastName property. Plain and simple.

[ServiceContract]
public interface IDataService
{
    [OperationContract]
    List<Person> GetData();
}
 
[DataContract]
public class Person
{
    [DataMember]
    public string FirstName { get; set; }
    [DataMember]
    public string LastName { get; set; }
}

I also have a HTML page that includes a SuperPerson object. A SuperPerson being the same as a Person, but with one extra property, 'somethingextra'.

function SuperPerson(firstname, lastname, somethingextra)
{
    this.firstname = firstname;
    this.lastname = lastname;
    this.somethingextra = somethingextra; 

}

There is also a JavaScript function that can create one of these SuperPerson objects with the extra data.

function GetSuperPerson(firstname, lastname, somethingextra) {
    var person = new SuperPerson(firstname, lastname, somethingextra);
    return person; 

}

To illustrate how to work with such a JavaScript object, I’ve made a small Silverlight application that receives a list of person objects from the WCF service described above. The Silverlight application will display this list of persons in a DataGrid control. 

public MainPage()

{
    InitializeComponent();
    DataService.DataServiceClient client = new DataService.DataServiceClient();
    client.GetDataCompleted += new EventHandler<DataService.GetDataCompletedEventArgs>(client_GetDataCompleted);
    client.GetDataAsync();
}
 
void client_GetDataCompleted(object sender, DataService.GetDataCompletedEventArgs e)
{
    dataGrid1.ItemsSource = e.Result;
}

This is the basic set up I will use in this example. The Silverlight application will be hosted on an HTML page that has the above mentioned JavaScript code present. 
In this example, to keep it simple, I will build a HTML table. It will be the Silverlight code that builds this table, not the JavaScript code. For this, first thing will be to get hold of a specific div tag on the HTML page in which we will build our table. Silverlight lets you interact with the hosting page through the HtmlPage class. This class gives you a Document property that behaves pretty much the same as the document object in JavaScript, which makes it very easy to use.

HtmlElement theDiv = HtmlPage.Document.GetElementById("dynamicDiv");
HtmlElement table = HtmlPage.Document.CreateElement("table");

Next, we will iterate over all Person objects in the DataGrid.

foreach (DataService.Person p in dataGrid1.ItemsSource)
{

For each of these Person objects we will execute the GetSuperPerson function, present in the JavaScript of the hosting page. Since the Silverlight application has no knowledge of the SuperPerson type, we will use the capabilities of .NET 4.0.

    dynamic superperson = HtmlPage.Window.Invoke("GetSuperPerson", p.FirstName, p.LastName, System.DateTime.Now.Millisecond);

With the dynamic keyword in place our compiler does not complain about the SuperPerson type it does not know about. And since we know a SuperPerson has a firstname, lastname and somethingextra, we can interact with it though these operations with no problem.

    HtmlElement row = HtmlPage.Document.CreateElement("tr");
    dynamic td = HtmlPage.Document.CreateElement("td");
    td.innerHTML = superperson.firstname;
    row.AppendChild(td);
    td = HtmlPage.Document.CreateElement("td");
    td.innerHTML = superperson.lastname;
    row.AppendChild(td);
    td = HtmlPage.Document.CreateElement("td");
    td.innerHTML = superperson.somethingextra;
    row.AppendChild(td);
    table.AppendChild(row);
}
 
theDiv.AppendChild(table);

Two things are notable about the code above. First of all, I use firstname and lastname, not FirstName and LastName (mind the casing). This is because we are interacting with the JavaScript object, not with the original Person object. I purposely used other casing in the JavaScript code to make my point on this clear.
A second thing you may have noticed is a second use of the dynamic keyword. When I create a new td element, I do not use the HtmlElement type (which is valid for a td tag), but instead I use the dynamic keyword. I do this because I know for sure it is possible to call the innerHtml operation on a td tag. The HtmlElement, however, does not have this operation present (you can check this in your intellisense). There is also no specific type for a td element that does have this operation present. With dynamic there is no problem at all to call innerHtml, not at compile time and not at run time.
There you have it, two good reasons to use dynamic when you're interacting with JavaScript. You can get around unknown objects in managed code and you can call operations managed code has no clue of.
Be ready for the next two parts, first on dynamic and Excel and next about ExpandoObjects (this is where the fun begins!).