RTC Build Traceability with UrbanCode Deploy

There have been great developments in the past 12 months or so to improve the traceability between builds managed by RTC and UrbanCode Deploy.

  1. RTC build definitions now have built-in support for post-build deployment steps. If you use the Jazz Build Engine as your build engine, you can specify connection information and component information in the build definition itself to allow the build process to push a new version of the build result into a new component version. This is great and makes it simple and easy to accomplish getting your builds to the deployment engine. See Freddy’s blog entry from about a year ago on this very topic for more details. There are of course some limitations to this feature. It assumes you want to push your build result into a single UrbanCode Deploy component. This is fairly unrealistic in that most applications of even modest complexity will most likely have more than one component in their application. If you build each component individually, then you are all set. However, if you have a single build file for the entire application (or for more than one component), this feature won’t cut it. To solve this, you can always use the old tried and true method of adding ANT tasks to your build.xml to create a new component version and push the contents into that version.

    From a forward traceability perspective, RTC build results records also have links that you can add. A common practice is to create a link to the UrbanCode Deploy component that gets created.
  2. UrbanCode Deploy also has a feature that allows you to create links associated with component versions. These links can be URLs to anything you want, but the best use of this feature is to provide a link back to the build entries that produced the component version. In the case of RTC, a simple link to the build result record gives you that backward traceability. If you use another build engine like Jenkins, for example, you can also create another link that points back to the Jenkins job that produced the build.11-20-2014 12-03-08 PM
    And there are REST and command line API calls that allow you go GET and PUT links. This makes the creation of the link something you can do as part of the build step of pushing a new component version to UrbanCode Deploy.

So both RTC and UrbanCode Deploy have the ability to create and maintain links to and from build result records and component versions. But let’s examine a bit closer how these links can be used to enhance the visibility of code as it walks down the release pipeline.

But first a quick aside. RTC has the concept of baselines and UrbanCode Deploy has the concept of snapshots. At some point in time someone needs to lay out some strategies on how these two concepts can be used together. The concept of a baseline in RTC collects together all of the code versions of all the files in all the selected components (RTC component here) into a single entity. This make sense to do at build time. At the same time, UrbanCode Deploy has the concept of a snapshot that essentially does the same thing but at a build output level. There may be additional components in a snapshot that don’t have a corresponding build equivalent, but marrying the two concepts is something I hope the Rational development teams consider. Being able to build all components of an app, take an RTC snapshot of the code that corresponds to that build, push all the components into UrbanCode deploy as new component versions, and create a snapshot of the application with the new component versions seems like a logical thing to do. I am sure there are many scenarios where this simple case breaks down, but the concept is solid. Adding links to more element types in UrbanCode deploy will help make this a reality.

OK, now back to our story. We have the bi-directional traceability between RTC build results and component versions via links on either side. But as the components get deployed to new environments as they move down the release pipeline, there is no indication of that movement in the RTC build record. How can we make this happen? Let’s build a plug-in.

The concept of the plugin is that each time a component version gets deployed to a new environment, let’s put some type of indication in the build result record to indicate that it has been deployed to environment X. This way anyone observing the build result record will know how far into the release pipeline, if at all, the build output has made it. The easiest way to do this is to use the tag field of the build result record. Tags can be anything so let’s add some tags to indicate deployment to environments.

In order to do this programmatically, we need to take advantage of the RTC Java API. For any release of RTC, you can get a zip file of the Java API jars that can be used to write java programs that manipulate RTC elements. There were a few jazz.net articles here and here that I used to help come up with the code I wrote. I also made the plug-in step that adds the tag a two step java plug-in (the plug-in calls a java program that then collects properties and then spawns another java process to do the work). This was for a specific reason. The code that does the work needs the RTC Java API jar files in its classpath. For the latest release of RTC, there are about 100 jar files that make up the classpath list. I didn’t want to include that entire set in the plug-in. Therefore, I ask the user to provide a path to the directory that holds these jar files. I then spawn a second java process that includes this directory in its classpath. This also keeps the plug-in RTC version-independent as long as the Java API for builds doesn’t change that much and it puts the onus on the user to get those jars.

So I created a plug-in step that adds a tag to the list of tags of an RTC build result record. The information needed for this step is relatively small: the RTC URL, a username and password, and the UUID of the build result record, and the tag to add. The java for this took a bit of time but again using existing examples made things much easier.

The other step in the plug-in that I created was getting the RTC build result ID from the component version link. The URL to an RTC build result looks something like this:

hostname:9443/ccm/web/projects/JKE%20Banking%20(Change%20Management)
#action=com.ibm.team.build.viewDefinition&id=_PYarAGq4EeS_krMxQ9smpw

The id fragment at the end is the UUID of the build result record. So the plugin step needs to get the link from the component version, then parse that link to get the build record UUID, and then create an output property to hold that value so that the previously mentioned java step can use it. There is a REST API call that can be used to get the link from a component version. Groovy can then be used easily to parse the full URL and retrieve the id.

So here is a working example. I envision that at the end of a component deployment process you would add these two steps to update the build result record.
11-20-2014 2-10-24 PM

Here is the details for the first step, GetBuildId. It takes the parameters mentioned above. Using component name and version name properties makes this step dynamic. The link name comes from the process of creating the link through an ANT task when building the component. And the output property is simply whatever you want to call it.

11-20-2014 2-12-32 PM

Here are the properties for the second step. The RTC parameters are self explanatory. The build result record UUID is the property that is set by the previous step. And the Java API location is where the user has unzipped the RTC Java API files on the agent server. The tag value also uses the current environment name property so that this step works for any environment.

11-20-2014 2-15-43 PM

Both of these steps work in my environment without a hitch. If you are interested in trying it out, you can get the plug-in source code here in this IBM DevOps Services project. Let me know how it goes.

New UrbanCode Deploy Plug-in Capabilities – Auto-Discovery

A few new plug-in capabilities were added in UrbanCode Deploy 6.0.1 that have gone relatively unnoticed.  I would like to take the opportunity to highlight them in a few posts.

First, the ability for an agent to auto-discover things is a new plug-in capability. Let’s explore how it works. The purpose of this feature is to get a jump start on configuring your resources and agent/machine properties by having an agent proactively look for the existence of things on the machine it is running on. For example, if you are a WebSphere shop you most likely make extensive use of the WebSphere plug-in.  And part of that process involves capturing the location of WebSphere on a given server. The WebSphere plug-in has that capability today.

If you look at the WebSphere plug-in, you will see two unique steps in the plug-in. One is called WebSphere Discovery. You will notice if  you open the plugin.xml file that the step has a new element in its definition:

<server:type>AUTO_DISCOVERY</server:type>

This type of step causes special behavior to occur when a new agent is added to a resource in the resource tree.  Every auto-discovery step in every plug-in gets run by the agent when this occurs.  For the WebSphere plug-in, this step looks for the existence of WebSphere by searching for standard WebSphere installation locations.  If it finds it, it creates a sub-resource to represent the WebSphere cell and sets a role property on that resource defining the path to WebSphere found on that machine.

Like I said earlier, auto-discovery steps are automatically run when a new agent resource is defined in the resource tree.  Auto-configure steps are manually executed on specific resources that have auto-configure roles applied to them.

At a minimum this save you some typing.  But imaging if you have hundreds or thousands of WebSphere servers. This helps to insure that you always have the right WebSphere information for that server.

But there is more you could automatically learn about a WebSphere box.  There are WebSphere nodes, server, etc. that could also be discovered.  Or maybe you want to create some things on WebSphere once you know WebSphere is there.  The other feature is called auto-configure.

Again, there is an auto-configure step in the WebSphere plug-in.

<server:role>WebSphereCell</server:role>
<server:type>AUTO_CONFIGURE</server:type>

The additional step element indicates why type of resource role this auto-configure step can be run on.  In the WebSphere case, the previous auto-discovery step identified WebSphere and created a sub-resource identifying the WebSphere cell.  You can now run auto-configure on this WebSphere cell resource.  However, before you do, you must provide some additional information to the WebSphereCell resource, namely the WebSphere username and password.  The auto-configure step then goes out and discovers the full WebSphere architecture and creates the necessary sub-resources to capture that architecture.

The last thing that is present in the plugin.xml file are the properties associated with any of the resource roles that your auto-discover or auto-configure steps will create.  These property groups define the properties that will automatically get applied to resource roles when they are created.

In my next post we will talk about the ability to include processes and templates with your plug-in.

Dynamically creating a property

There are times in a deployment process that you may want to create a property dynamically and use it in a later process step.  Fortunately, Urbancode Deploy provides a mechanism to do this.

Let’s use the following scenario.  As part of your process, you want to create a tar file backup of a directory structure.  You wish to create a tar filename that includes a date stamp.  You therefore have to get the current date and time and format it to your liking. Here is how to do it.

First, create a Groovy process step.  There is a Groovy plugin that allows you to create Groovy scripts and have them executed as a process step.   We will use a simple script like below to grab the date and time and format it.

import java.text.SimpleDateFormat
def today = new Date()
def formattedDate = new SimpleDateFormat("ddMMyyyy, Ka").format(today)
outProps.put("date",formattedDate)

The Groovy step will look like this:

datetime

Notice the name of the Groovy step as we will use it later.

Also, notice the last line of the Groovy script. This exports the formattedDate as a property.  We can now use that property in subsequent steps.

To reference this property, use the following syntax:

${p:set-date/date}

“set-date” is the name of the step that set the property and “date” is the property name.  You can now use this variable to create your tar file, maybe like this:

tar cvf /tmp/${p:set-date/date}.tar .


					

A New Plugin – Send an Email in a Process Step

I was asked by a customer if Urbancode Deploy could send a generic email as part of the deploy process.  Urbancode Deploy has notification capabilities but it relies on the notification schemes that are built in.  You can’t simply add a step in a process to send an email.

So I set out to write a plugin to accomplish this.  Whenever I tackle a new “programming” exercise, I always resort to two principles.  1)  Google to find what others have already done.  And 2)  reuse anything you already have at your disposal.

Urbancode Deploy already takes advantage of an SMTP server to be able to send emails, and you specify the SMTP settings on the System Settings page.

SMTP

So the first problem to solve is how to get at these values.  Here is where reuse comes in. There is an existing Urbancode plugin that accomplishes this.  The Udeploy-System plugin has a task that gets all of the System Configuration information, which includes the SMTP server values.  So the first step is to tear apart this plugin and learn how it gets the information.  It turns out there is a Rest Client jar file that implements a call to get the Urbancode Deploy system configuration.  So I copied most of the Udeploy-System plugin contents into my plugin so I could reuse it.

The second problem then is how to send an email.  A quick Google search for “Groovy script to send email” produced numerous results that I could easily steal re-use.  And to actually carry out the sending of the email, I had to go and grab JavaMail mail.jar and its corresponding activation.jar and put them in the lib directory of the plugin.

The resulting plugin works great.  There are three parameters that the step takes, the recipient list, the email subject, and the email body.

region

Here is the resulting Groovy script that I came up with.  Note that this version does not deal with authentication as most SMTP servers don’t require it, but I am sure it is an easy addition.

groovy

send_smtp_email.groovy.doc

The above link is the code content in a Word doc.

Give it a try.