Grails: Applying build information to your builds

Occasionally, when I buy some food I check the label to see how unhealthy it is in an effort to remind myself to eat better. I probably should do this more often but that's another story.

With software, I take a more strict approach. I like to know exactly what version of what I am using and if it pertains to a build coming from a project I am working on, I like to know even more:

• the branch it came from 
• any code or config changes 
• the time the build was done 
• the person who did the build
• the commit revision it corresponds to

The advantages of this are obvious but worth re-stating.

• While you are in development, if you have multiple deployments and see something unusual you will immediately want to compare them. 
For example say you have:
• build #349 on box 1
• build #352 on box 2 (which includes development changes from a feature branch Y)
You notice some strange behaviour for an error message on box 2 but don't on box 1. You can immediately check which box has what version, then check the changes and then try to rationalise the difference.
• You can be sure your latest code check-in is deployed
• All this build information should be in official release notes, so if you can automate it as part of your development process you have saved yourself having to automate it as part of your release
• In enterprise architectures where you have multiple components developed from different teams integrating with each other, all components should make it easy to get this information. This helps to manage the stack and for everyone to be able to immediately check if new versions of any components have been deployed or to roll back to stable versions if problems with  any upgrades of components happen.

So away with the opinions and now with some examples. The project that I am currently working uses a Grails based architecture and Atlassian's Bamboo for builds and deploys. I wanted to get this information I described into every build - and yes that includes every development build.  I will now outline the steps.

Step 1

Write an event handler to execute when the war is being created. This should read a bunch of system properties (which you'll set in bamboo) and put them in your application.properties of your war file. Example:

eventCreateWarStart = { warName, stagingDir ->
    def buildNumber = System.getProperty( "build.number", "CUSTOM" ) 
    def buildTimeStamp= System.getProperty("build.timeStamp", "") 
    def buildUserName= System.getProperty("build.userName", "")
    def repositoryRepositoryUrl= System.getProperty("repository.repositoryUrl",  "")
    def repositoryRevisionNumber= System.getProperty("repository.revision.number", "")
    def repositoryBranch= System.getProperty("repository.branch", "")
 
    ant.propertyfile(file: "${stagingDir}/WEB-INF/classes/application.properties") {
        entry(key:"build.number", value:buildNumber) 
        entry(key:"build.timestamp", value: buildTimeStamp)
        entry(key:"build.userName", value: buildUserName)
        entry(key:"repository.revision.number", value: repositoryRevisionNumber)
        entry(key:"repository.branch", value: repositoryBranch) 
    }

}

Step 2

Update your bamboo build to set these system properties when the war is being built.  To do this, update the build war task to something like:

war -Dbuild.number=${bamboo.buildNumber} 
    -Drepository.branch=${bamboo.repository.git.branch} 
    -Dbuild.userName=${bamboo.ManualBuildTriggerReason.userName} 
    -Dbuild.timeStamp=${bamboo.buildTimeStamp} 
    -Drepository.revision.number=${bamboo.repository.revision.number}

This will mean when bamboo builds you war all the above properties will be put in your application.properties file.

Step 3

Now all you need to do is make it easy for a Grails GSP to read these properties and display them.  This means that when deployed all anyone will have to do is hit a special URL and then they'll get all the build info for the deployed WAR.

...
... Some CSS and JavaScript


Environment: ${grails.util.Environment.current.name}



Build Info:


Name
Value

UserName
#${g.meta([name: 'build.userName'])}
Built by
 #${g.meta([name: 'build.number'])}
Date
#${g.meta([name: 'build.date'])}
Branch
#${g.meta([name: 'repository.branch'])}
Revision number
#${g.meta([name: 'repository.revision.number'])}

And in the spirit of food analogies... Here's one I made earlier and how it looks

So there you go, software taken more seriously than food. I'm off to get a Big Mac!

SQL Server tips

Recently, I was working on a project which involved SQL Server. I made a note of some useful commands that help me find useful things out. Here they are:

Check column types for a specific table:

SELECT * FROM INFORMATION_SCHEMA.COLUMNS where TABLE_NAME='TOSENV'

Check column constraints

SELECT * FROM CONSTRAINT_COLUMN_USAGE

Check length of text in a varchar column

SELECT LEN(MYCOLUMN) FROM MYTABLE;

List all tables that have a column named EntityID

SELECT TABLE_SCHEMA, TABLE_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE COLUMN_NAME = 'EntityID'

Check all constrains...

SELECT * FROM INFORMATION_SCHEMA.CONSTRAINT_COLUMN_USAGE

Find blocked processes

SELECT * FROM  SYS.SYSPROCESSES WHERE BLOCKED > 0

To see long running open transactions

DBCC OPENTRAN;

Check the lock timeout

SELECT @@LOCK_TIMEOUT

Check current session

SSP_WHO2

Find creation date of databases

SELECT NAME, CRDATE FROM MASTER..SYSDATABASES

Get the database id for a database.

SELECT DB_ID('PLANES')

Get the median value for a column named responsetime in a table allrequests

SELECT( (SELECT MAX(responsetime) FROM (SELECT TOP 50 PERCENT responsetime FROM allrequests ORDER BY responsetime) AS BottomHalf) + (SELECT MIN(responsetime) FROM (SELECT TOP 50 PERCENT responsetime FROM allrequests ORDER BY responsetime DESC) AS TopHalf)) / 2 AS Median

Most CPU intensive queries

SELECT HIGHEST_CPU_QUERIES.PLAN_HANDLE, HIGHEST_CPU_QUERIES.TOTAL_WORKER_TIME, Q.DBID, Q.OBJECTID, Q.NUMBER, Q.ENCRYPTED, Q.[TEXT]FROM (SELECT TOP 50 QS.PLAN_HANDLE, QS.TOTAL_WORKER_TIME FROM SYS.DM_EXEC_QUERY_STATS QS ORDER BY QS.TOTAL_WORKER_TIME DESC) AS HIGHEST_CPU_QUERIES CROSS APPLY SYS.DM_EXEC_SQL_TEXT(PLAN_HANDLE) AS QORDER BY HIGHEST_CPU_QUERIES.TOTAL_WORKER_TIME DESC INDEXES AND KEYS

Get me all the tables that have a FK to the table named Person

SELECT OBJECT_NAME ([PARENT_OBJECT_ID]) AS 'REFERENCING TABLE', * FROM SYS.FOREIGN_KEY_COLUMNSWHERE [REFERENCED_OBJECT_ID] = OBJECT_ID ('PERSON'));

Check all objects that have changed in last ten days

SELECT NAME AS OBJECT_NAME ,SCHEMA_NAME(SCHEMA_ID) AS SCHEMA_NAME ,TYPE_DESC ,CREATE_DATE ,MODIFY_DATEFROM SYS.OBJECTS WHERE MODIFY_DATE > GETDATE() - 10 ORDER BY MODIFY_DATE;
Disable indexes

ALTER INDEX [IX_NAME] ON TABLE DISABLE

Enable indexes

ALTER INDEX [IX_NAME] ON TABLE REBUILD USERS CHECK CURRENT USER

Check current user

SELECT SYSTEM_USER;

Check database owner

SELECT NAME, SUSER_SNAME(OWNER_SID) FROM SYS.DATABASES STATS

Check last time Stats was run

SELECT NAME AS STATS_NAME, STATS_DATE(OBJECT_ID, STATS_ID) AS STATISTICS_UPDATE_DATE FROM SYS.STATSWHERE OBJECT_ID = OBJECT_ID('DBO.TASCONTACT');

Run stats for a table named Person in Sales

UPDATE STATISTICS SALES.PERSON

Check the size of various tables

SELECT OBJECT_NAME(P.OBJECT_ID) AS "TABLE", P.INDEX_ID, F.NAME, SUM(TOTAL_PAGES)/128 AS "SIZE IN MB", CONVERT(VARCHAR(10), GETDATE(), 101), COUNT(*) AS PARTITIONSFROM SYS.PARTITIONS P JOIN SYS.ALLOCATION_UNITS A ON P.PARTITION_ID = A.CONTAINER_ID JOIN SYS.FILEGROUPS F ON A.DATA_SPACE_ID = F.DATA_SPACE_IDGROUP BY P.OBJECT_ID, P.INDEX_ID, F.NAMEORDER BY SUM(TOTAL_PAGES)/128 DESC

Find the creation time for all tables.

SELECT NAME AS TABLENAME, CREATE_DATE AS CREATEDDATE FROM SYS.TABLES VERSION

Check version

SELECT @@VERSION;

or for more info

SELECT SERVERPROPERTY('EDITION') AS SQL_Server_Edtition, SERVERPROPERTY('PRODUCTLEVEL') AS SQL_Product_Level, SERVERPROPERTY('PRODUCTVERSION') AS SQL_Version;

JavaScript tip: Log those important APIs and get some code coverage

Any web architecture using JavaScript will always have plenty of AJAX requests to the server. There are many, many, many different ways AJAX calls can be made. Suppose you're thinking good software engineering and you want to avoid coupling your client code to the AJAX mechanism you are using? You could use a wrapper / bridge / proxy type object which contains all the APIs to get information from the Server and encapsulates your AJAX mechanism. Following the Crockford Module pattern this could pan out something like:

dublintech.myproject.apiBridge = function() {
    function createEntity(data) {
        ...
        // actual AJAX call
        ...
    }
 
    function readEntity(id) {
        ...
        // actual AJAX call
        ...
    }
 
    that = {}
    ...
    that.createEntity = createEntity;
    that.readEntity = readEntity;
    return that;
};

This could be invoked simply as:

apiBridge.createEntity(data);

What are the pro's of using the wrapper approach? Well essentially the advantages are all derived from the fact there is a nice separation of concerns: the transport mechanism (doesn't even have to be ajax) is separated from the actual data requests coming from the client. This means:

1. If you wish to change Ajax mechanism (i.e. move from one Ajax library to another), it's not a big deal. The impact is limited.
2. It's easier to stub out and test.
3. It's easier to achieve code reuse. Suppose you want to have consistent error handling for exceptions from Ajax requests, it's much easier to do this if all Ajax request are following the same pattern.

Tell me something more interesting.

Well we all know that you can see the AJAX requests in firebug. But say you wanted another way to log what requests had been invoked on your wrapper. In Java, you could write an Aspect and set it on every method that made an Ajax request. There is no direct equivalent to aspects in JavaScripts. But we can do something similar. Firstly, the function to do the pre and post logging:

function wrapPrePostLogic(fn) {
    return function withLogic() {
 console.log("before" + fn.name);
 var res = fn.apply(this, arguments);
 console.log("after" + fn.name);
    }
}

Ok so wrapPrePostLogic takes a function, and returns a new function which wraps around the existing function and puts logging before and after the invocation of that function. Hold on sec - IE is awkward. It doesn't support getting the function.name variable. So let's write a helper method to get function name for any browse and use that instead.

function wrapPrePostLogic(fn) {
    return function withLogic() {
        getFnName("before " + getFnName(fn));
        var res = fn.apply(this, arguments);
        getFnName("after " + getFnName(fn));
    }
}

function getFnName(fn) {
    var toReturn =  (fn.name ? fn.name : (fn.toString().match(/function (.+?)\(/)||[,''])[1]);
    return toReturn;
}

So now what we need to ensure wrapPrePostLogic() is called for every method. We could do:

    that = {}
    ...
    that.createEntity = wrapPrePostLogic(createEntity);
    that.readEntity = wrapPrePostLogic(readEntity);
    ...
    return that;

But I am smelling code bloat and human error. What would be nicer is to iterate over all functions returned by that and wrap them appropriately. To do that we could do...

    that = {}
    ...
    for (var prop in that) {
        if (typeof that[prop] === "function") {
            that[prop] = wrapPrePostLogic(that[prop]);
        }
    }

    return that;

Not bad. Anything else?

Ok, so say you are you using a UI test framework and wanted to test code coverage of all methods that send / receive data to / from the server. Instead of getting your wrapper function to log to the console you could update a hidden div and then make your test framework check this hidden div.

function updateHiddenDiv(methodName) {
    if (jQuery("#js_methods_invoked").length === 0) {
        var hiddenDiv = jQuery('

');
        jQuery('body').append(hiddenDiv);
    }
    var myHiddenDiv = jQuery("#js_methods_invoked"); // save the div in a var
    if ((myHiddenDiv.text() === "") || (myHiddenDiv.text().indexOf(methodName) === -1)){
        myHiddenDiv.append(methodName + ",");
    }  // read from the same DIV
}

Now we can just change our wrapper function to:

function wrapPrePostLogic(fn) {
    return function withLogic() {
        var res = fn.apply(this, arguments);
        updateHiddenDiv(fn.name);
        return res;
    };
}

In addition, you can find what JavaScript methods have been invoked by opening a JavaScript console and wacking in:

jQuery(#hiddenDiv).

This will return what methods have been invoked. Until the next time, take care of yourselves!

How could Scala do a merge sort?

Merge sort is a classical "divide and conquer" sorting algorithm. You should have to never write one because you'd be silly to do that when a standard library class already will already do it for you. But, it is useful to demonstrate a few characteristics of programming techniques in Scala. Firstly a quick recap on the merge sort. It is a divide and conquer algorithm. A list of elements is split up into smaller and smaller lists. When a list has one element it is considered sorted. It is then merged with the list beside it. When there are no more lists to merged the original data set is considered sorted. Now let's take a look how to do that using an imperative approach in Java.

public void sort(int[] values) {
   int[] numbers = values;
   int[] auxillaryNumbers = new int[values.length];
   mergesort(numbers, auxillaryNumbers, 0, values.length - 1);
}

private void mergesort(int [] numbers, int [] auxillaryNumbers, int low, int high) {
    // Check if low is smaller then high, if not then the array is sorted
    if (low < high) {
        // Get the index of the element which is in the middle
        int middle = low + (high - low) / 2;
       
        // Sort the left side of the array
        mergesort(numbers, auxillaryNumbers, low, middle);
        // Sort the right side of the array
        mergesort(numbers, auxillaryNumbers, middle + 1, high);
      
        // Combine them both
        // Alex: the first time we hit this when there is min difference between high and low.
        merge(numbers, auxillaryNumbers, low, middle, high);
    }
}

/**
 * Merges a[low .. middle] with a[middle..high].
 * This method assumes a[low .. middle] and a[middle..high] are sorted. It returns
 * a[low..high] as an sorted array. 
 */
private void merge(int [] a, int[] aux, int low, int middle, int high) {
    // Copy both parts into the aux array
    for (int k = low; k <= high; k++) {
        aux[k] = a[k];
    }

    int i = low, j = middle + 1;
    for (int k = low; k <= high; k++) {
        if (i > middle)                      a[k] = aux[j++];
        else if (j > high)                   a[k] = aux[i++];
        else if (aux[j] < aux[i])            a[k] = aux[j++];
        else                                 a[k] = aux[i++];
    }
}
 
public static void main(String args[]){
     ...
     ms.sort(new int[] {5, 3, 1, 17, 2, 8, 19, 11});
     ...
}

}

Discussion...

1. An auxillary array is used to achieve the sort. Elements to be sorted are copied into it and then once sorted copied back. It is important this array is only created once otherwise there can be a performance hit from extensive array created. The merge method does not have to create an auxiliary array however since it changes an object it means the merge method has side effects.
2. Merge sort big(O) performance is N log N.

Now let's have a go at a Scala solution.

  def mergeSort(xs: List[Int]): List[Int] = {
    val n = xs.length / 2
    if (n == 0) xs
    else {
      def merge(xs: List[Int], ys: List[Int]): List[Int] =
          (xs, ys) match {
          case(Nil, ys) => ys
          case(xs, Nil) => xs
          case(x :: xs1, y :: ys1) =>
            if (x < y) x::merge(xs1, ys)
            else y :: merge(xs, ys1)
      }
      val (left, right) = xs splitAt(n)
      merge(mergeSort(left), mergeSort(right))
    }
  }  

Key discussion points:

1. It is the same divide and conquer idea.
2. The splitAt function is used to divide up the data up each time into a tuple. For every recursion this will new a new tuple.
3. The local function merge is then used to perform the merging. Local functions are a useful feature as they help promote encapsulation and prevent code bloat.
4. Neiher the mergeSort() or merge() functions have any side effects. They don't change any object. They create (and throw away) objects.
5. Because the data is not been passed across iterations of the merging, there is no need to pass beginning and ending pointers which can get very buggy.
6. This merge recursion uses pattern matching to great effect here. Not only is there matching for data lists but when a match happens the data lists are assigned to variables:
• x meaning the top element in the left list
• xs1 the rest of the left list
• y meaning the top element in the right list
• ys1 meaning the rest of the data in the right list

This makes it very easy to compare the top elements and to pass around the rest of the date to compare. Would the iterative approach be possible in Java? Of course. But it would be much more complex. You don't have any pattern matching and you don't get a nudge to declare objects as immutable as Scala does with making you make something val or var. In Java, it would always be easier to read the code for this problem if it was done in an imperative style where objects are being changed across iterations of a loop. But Scala a functional recursive approach can be quite neat. So here we see an example of how Scala makes it easier to achieve good, clean, concise recursion and a make a functional approach much more possible.