Quick Post – UrbanCode Deploy REST API and the pesky SSL certificate

Just a quick post for those that are playing with the UrbanCode Deploy REST API.  If you are building a Java solution that utilizes the UrbanCode Deploy REST API, you no doubt have run into the SSL certificate issue.   The default installation for UrbanCode Deploy utilizes a secure socket layer protocol and therefore you have to deal with SSL and its certificate.  Most browsers deal with this and present a warning allowing you to pass at your own risk.

cert

We are all used to this Chrome warning.  You would be surprised at how many internal RTC and UCD servers within IBM don’t bother installing a real certificate 🙂

However, when you are writing Java code that is accessing the REST API, the Java JVM is not as forgiving.  You have to deal with it.  Well, after going to the source of all worldly knowledge, Google, I found an easy solution.  I use Rational Software Architect as my Eclipse IDE, but obviously this will work with any Eclipse-based IDE (any JVM-based IDE for that matter).  Go and get this file.  Create a Java project and add this Java file to it.

In my version, I commented out the section that determines the keystore location (File file = new File…) with a single line that points to my jre that is part of RSA.

File file = new File(“C:/Program Files (x86)/IBM/SDP/jdk/jre/lib/security/cacerts”);

You then simply run the Java program passing the URL (https://<hostname&gt;:8443) of your UrbanCode Deploy server.  Apparently the code attempts to connect and fails, but as part of the SSL handshake process it grabs the server certificate and puts it into the keystore for your JRE.  Not sure exactly how it works but it worked for me so I didn’t dig deeper.

The next time you attempt to run some Java code that access an UrbanCode Deploy REST API from your IDE, you should not have anymore SSL certificate issues getting in your way.   Of course you still have to deal with the certificate “for real” when it comes to production time, but this should get you through the debugging stage.

Advertisements

Everything is a Resource – Resource Templates

UrbanCode Deploy 6.0 introduced the concept of a resource tree.  It takes some getting used to, but overall it gives a nice OPs-centric view of the landscape of things.   But buried in the resource topic is the little known yet powerful concept of resource templates.  Let’s walk through the process of creating and using one.

Note:  My examples below use UrbanCode Deploy 6.0.  I am hoping things look a bit better in 6.0.1 from a user interface standpoint.

First, let’s create a new template.  On the Resources main tab, click on the Resource Templates sub-tab.  Click the Create New Sample link.  You will also notice that you can create a new resource template by connecting to a Cloud provider.  This I believe was the original reason for this feature.  Cloud patterns essentially define resource templates.  So by connecting to a Cloud provider, UrbanCode Deploy creates a resource template from the Cloud pattern.  Luckily for us, they also generalized that feature and let us create resource templates from scratch.

resource template

In this case, we are going to create a new 3 tier topology resource template that can be used to deploy a 3 tiered application (ok its just made up but good for an example).  Once I click Save, I get to define my template.  Using the Action menu on the base resource that gets created, we can create a series of sub-resources to represent the tiers.

rt2

And finally, we can add Agent Prototypes to each sub-resource as placeholders for real agents.

Note:  You will notice that you can add a component to an agent prototype.  Why in the world would you want to do that?  In the rare case where you may have some generic component that should be applied to every instance of this template, you can define it here.

rt3

We now have our completed template.  This template is now available as a basis for an application environment.  But first we need to create an Application Blueprint, which inserts this application template into a location in an application’s resource tree.  Moving over to the Application Main tab, selecting our application (JPetStore) and finally the Blueprints sub-tab, we can create our new blueprint.  During the creation process, we select the resource template that we want to use, which is the one we just created.

Once the blueprint is created, we can again use the Action menu for each agent prototype and assign the component from our application to the agent prototype.  This process is now mapping our application to an existing resource template, as shown below.

rt5

Now that we have our application mapped to the resource template in the blueprint, we can create a new application environment from the blueprint.  Back to the JPetStore application environments page, we can create a new environment.

rt6

We give the environment a name, chose the blueprint we just created, and select the base resource where we want to insert this resource template.  This base resource is key and depends on how you have organized your resource tree.  If you organized things well, you can insert this new resource along side the other resource nodes that define other environments for this application.

When you click save, you get an error (that is horribly named) but it helps you to know that you have a step to perform yet.  We have to assign real agents to where we had agent prototypes.  Click on the newly created environment and we see its resource tree.  We can assign a real agent to each node in our tree using the Action menu.

rt7

We now have a new environment with real agents assigned to our components, ready for deploy.  Well, almost.  Can you think of what might yet have to be defined?  How about environment properties?  Those will need to be defined if needed.  But once you do that, deploy away.

Taking Continuous Delivery to the Max

Now that we have some large UrbanCode customers under our belt, we can now look at some of the metrics involved in deploying a continuous delivery solution like IBM UrbanCode Deploy.  There are definitely some hard measurements that can be taken. You can easily look at a simple metric like the time it takes to perform deployments.  Time savings is the easy valuable result the comes from automation.  Don’t forget to take into account the amount of time it takes to create the automation, but once it is in place, the more times it is utilized the bigger return on that investment.  Over the landscape of an enterprise and a duration of a year or two, your investment in automation in a continuous delivery solution can pay for itself.

But let’s be honest, automation has been around for years and no sys-admin is on the job for more than a day without building a script to automate something.  Automation has always been a valuable component to deployments.  Using a continuous delivery solution helps to capture that automation into reusable chunks so that it can be extrapolated across the enterprise.  But I will say that I have run across some organizations that have been pretty good at this before “continuous delivery” was first uttered.

tent-center-poleSo what are some other long poles in the deployment tent?  I once consulted at a customer that had a testing data center that so large that you literally couldn’t see the opposite wall.  There was more hardware in that room (and consequently more needed power and cooling) than I had ever seen. Despite this, it was a 6 month wait to get an available test environment.  You would think that with that much computing power under one roof that you would have immediately available systems.  However, at any given time more than half of the systems in that room were in “transition” from one testing environment to another.  The process of provisioning an environment for a specific application (at that time) took a lot of manual labor and your request was put on a queue that took time to get to.

So to me to get the biggest bang for your buck in continuous delivery is combining deployment automation with system provisioning.  And taking it even a step further, provisioning a physical system is one thing, but Cloud solutions bring even more to the value proposition by removing the need for physical deployment targets.

Improvements to IBM UrbanCode 6.x have been made to help bring integration with provisioning and Cloud as a standard capabilility.  I will spend more time in a future post or two on this, but here is the high level process.

1.  Prepare the Cloud – a deployment pattern is created in the cloud catalog.  This pattern specifies the process of creating an infrastructure for an application.  The pattern codifies enterprise standards and insures consistent infrastructure.  Part of the pattern should be installation of the IBM UrbanCode Deploy agent.  When the nodes of the pattern are booted up they will communicate with the UrbanCode Deploy server.
2.  Import the Cloud pattern into UrbanCode Deploy – this will create a new Resource Template that has an Agent Prototype for each node in the pattern.  Properties that need  to be specified for the pattern are captured as UrbanCode Deploy properties.
3.  Create a new Application Blueprint that specifies the Resource Template created above.  The blueprint binds application information (components) to the Agent Prototypes in the template.
4.  Now create a new Application Environment based on the Application Blueprint.  You specify your Cloud connection properties as well as any properties needed by the Cloud pattern.

The result of all of this is a newly provisioned Cloud environment with your application deployed to it.  Nice.

In a future post or two, I will go into some of the specifics of this solution.  But needless to say, the value proposition of this solution is the promised land of continuous delivery.

Getting the Most out of Your UrbanCode Deploy Implementation

I think it is fair to say that almost every customer that purchases or even investigates the use of UrbanCode Deploy, or any continuous deployment solution, has already spent a ton of time building deployment scripts, build processes, and “best practices” of their own.   And typically these things do the job, albeit most of the time not the most reusable and transparent solution.  So their first approach to implementing a continuous deployment solution is to wrap their existing scripts or processes.  After all, they work and do the job, why not reuse them?  Beside, there are some other features, like the UrbanCode Deploy inventory, that are valuable in and of themselves.   This is considered the low hanging fruit implementation of something like UrbanCode Deploy.  Simply wrap what you have and you are off and running.

md-deploy

But if you peek just a bit into the future, this is not the best approach.  The following are some common shortcuts that people take instead of implementing with future benefits in mind.

  • Simply executing shell scripts to perform deployments – this is the quick and dirty solution, but misses out on a number of valuable capabilities.  First, you should take advantage of as many existing plugin steps as possible.  Plugins offer canned and consistent execution of a single function.  Reusing a step that you know works insure that you don’t have random errors across different deployment processes.  I would even argue that you should be on the lookout for abstracting process steps into custom plugins.  If you think there might be some reuse of a step that you have scripted, find a way to wrap it in a plugin step.
    Also, upgrading a plugin upgrades the use of those plugin steps everywhere in your UrbanCode Environment.  If you find a better way to execute a step, upgrade the plugin and all deployment processes that use that step get the new and improved way.
  • Not taking advantage of templates – It is understandable that you would want to build a deployment process and test it first before you would consider it the default way.  However, you should quickly abstract processes into templates to insure best practices are followed.  If you have developed a tried and true way to deploy apps to Tomcat, for example, then you should provide a way for each and every Tomcat application to take advantage of your success.  Creating component templates that encapsulate standard processes is something that should be put at the forefront of an implementation effort.
  • Not taking advantage of UrbanCode Deploy code station – Code station is UrbanCode Deploy’s internal component content storage mechanism.  Many customers already have some type of build target repository (i.e. Nexus) and don’t see the need to replicate this storage also in UrbanCode Deploy.  But code station is more than simply a repository.  It also hashes each and every file that makes up a component.  This provides an easy mechanism to know if files have changed when a new version of a component is provided.  A key value of UrbanCode Deploy is that it only updates the files that have changed, and you don’t need to do anything to enact this feature.  It accomplishes this by comparing hashes.  If the file has not changed and the inventory indicates it already exists, then there is no reason to push it again.  Of course you can override this and push it anyway, but why would you?
  • Providing only changed elements to UrbanCode Deploy – many customers have been bitten in the past by overwriting code that works with code that doesn’t.  Almost everyone has a story that involves many hours of hunting for the cause of production problem only to discover that something that worked got overwritten by something that was broken and that element being overwritten wasn’t even supposed to be affected by the changes.  The natural knee-jerk reaction to this is that only things that you believe have changed get deployed.  The mentality goes way back to mainframe days.  Providing complete builds to UrbanCode Deploy allows you to take advantage of the code station feature as described above, and it allows you to always be able to deploy any version since UrbanCode always knows how to get from full component version to full component version.

Most implementation of a continuous deployment solution start out with a quick win and hopefully everyone can be back on to their “regular job.”  But by spending a small amount of time to do things right from the beginning, there is a lot of value to be had in the future.

Dynamically creating a property

There are times in a deployment process that you may want to create a property dynamically and use it in a later process step.  Fortunately, Urbancode Deploy provides a mechanism to do this.

Let’s use the following scenario.  As part of your process, you want to create a tar file backup of a directory structure.  You wish to create a tar filename that includes a date stamp.  You therefore have to get the current date and time and format it to your liking. Here is how to do it.

First, create a Groovy process step.  There is a Groovy plugin that allows you to create Groovy scripts and have them executed as a process step.   We will use a simple script like below to grab the date and time and format it.

import java.text.SimpleDateFormat
def today = new Date()
def formattedDate = new SimpleDateFormat("ddMMyyyy, Ka").format(today)
outProps.put("date",formattedDate)

The Groovy step will look like this:

datetime

Notice the name of the Groovy step as we will use it later.

Also, notice the last line of the Groovy script. This exports the formattedDate as a property.  We can now use that property in subsequent steps.

To reference this property, use the following syntax:

${p:set-date/date}

“set-date” is the name of the step that set the property and “date” is the property name.  You can now use this variable to create your tar file, maybe like this:

tar cvf /tmp/${p:set-date/date}.tar .


					

A New Plugin – Send an Email in a Process Step

I was asked by a customer if Urbancode Deploy could send a generic email as part of the deploy process.  Urbancode Deploy has notification capabilities but it relies on the notification schemes that are built in.  You can’t simply add a step in a process to send an email.

So I set out to write a plugin to accomplish this.  Whenever I tackle a new “programming” exercise, I always resort to two principles.  1)  Google to find what others have already done.  And 2)  reuse anything you already have at your disposal.

Urbancode Deploy already takes advantage of an SMTP server to be able to send emails, and you specify the SMTP settings on the System Settings page.

SMTP

So the first problem to solve is how to get at these values.  Here is where reuse comes in. There is an existing Urbancode plugin that accomplishes this.  The Udeploy-System plugin has a task that gets all of the System Configuration information, which includes the SMTP server values.  So the first step is to tear apart this plugin and learn how it gets the information.  It turns out there is a Rest Client jar file that implements a call to get the Urbancode Deploy system configuration.  So I copied most of the Udeploy-System plugin contents into my plugin so I could reuse it.

The second problem then is how to send an email.  A quick Google search for “Groovy script to send email” produced numerous results that I could easily steal re-use.  And to actually carry out the sending of the email, I had to go and grab JavaMail mail.jar and its corresponding activation.jar and put them in the lib directory of the plugin.

The resulting plugin works great.  There are three parameters that the step takes, the recipient list, the email subject, and the email body.

region

Here is the resulting Groovy script that I came up with.  Note that this version does not deal with authentication as most SMTP servers don’t require it, but I am sure it is an easy addition.

groovy

send_smtp_email.groovy.doc

The above link is the code content in a Word doc.

Give it a try.