UCD, UCD+P, ICO, and PureApp

As I mentioned on my last post, lots of changes happening. I have continued to change my role and have now landed as a cloud advisor. I am excited but enough about me.

What my new role has afforded me is to continue to explore and understand the various IBM cloud software solutions out there. It is an interesting landscape and is changing faster than ever. This post delves into IBM cloud provisioning and deployment solutions but leaves the on-premises/off-premises concept OFF the table. For the most part this discussion is concerned about the ability to automatically stand up an application environment regardless of its physical proximity to developer. So before I even dive into specifics, we can talk some general capabilities. The following list is not exhaustive and there are many more capabilities that the products I talk about are capable of. These are just the ones that I am interested in for this post. 

For the purpose of this post, the definition of the deployment stack, shown here, is the total of all the layers of technology needed to be created in order for a full application to execute.

stack2

Provisioning – As I said before, in this post I am interested only in the ability to automatically stand up new environments. This concept also assumes some type of pattern or infrastructure-as-code concept to enable the automation.

Deployment – For the sake of this post, deployment refers to the automated deployment of an application including its specific application configuration applied to a provisioned environment.

GO LIVE tasks – Those tasks that must occur for an application to GO LIVE that are above and beyond simply provisioning an environment and deploying the application. Tasks such as appropriate approvals, insure the app is properly monitored, backups are in place, property security and end point compliance policies, notifications are setup when things go wrong, etc. These important tasks are part of every operations teams set of responsibilities and have a large impact on production deployments.

Pattern – The ability to capture some part or all of the stack definition in a reusable artifact.  There are two pattern capabilities that we will talk about, vSYS (virtual system) patterns and HEAT.

Let’s now take a look at the IBM tools currently in this space. Big disclaimer here. There are many many additional solutions from the IBM portfolio many of which are highly customized and include a services component. These types of solutions are highly desirable in a hybrid cloud scenario where you need brokerage services to not only serve the line of business but also the ability to manage the provisioned environments across your hybrid landscape and the ability to manage cost and charge back across that same landscape. There are outsourced solutions from IBM that target unique cloud platforms.  For the purposes of our conversation here today, we assume we have a IaaS cloud solution that we want to take advantage of. In lieu of a big description of each product, I will simply list the capabilities that it provides (there are many more but these are the ones of interest for this post).

IBM Cloud Orchestrator – provisioning, patterns, deployment, go live tasks, BPM

UrbanCode Deploy – deployment

UrbanCode Deploy with Patterns – provisioning, patterns, deployment

PureApplication – provisioning, patterns, deployment

So that doesn’t help. Lots of overlap and obviously lots of details under the covers. Each product does a thing or two very well, so let’s look at the list again and I will expand on each capability highlighting its strength(s).

IBM Cloud Orchestrator

provisioning – ICO currently can provision nodes based on both types of patterns.  ICO can provision to many virtual environment both OpenStack based and not.
patterns – ICO currently supports 2 pattern technologies, HEAT and vSYS (virtual system). The vSYS patterns are the legacy pattern type. HEAT patterns are based on OpenStack and therefore require an OpenStack implementation. ICO has full editing capabilities for vSYS patterns, however ICO does not provide an editor for HEAT patterns.
deployment – while ICO doesn’t have a separate deployment capability, you are able to build into your vSYS patterns application components and Chef scripts that can ultimately deploy applications as part of the provisioning process. However this capability is not very scalable and is precisely why deployment tools like UrbanCode Deploy were created. HEAT patterns as defined by OpenStack definition do not contain deployment-specific capabilities (more details below).
GO LIVE tasks – ICO has a large list of pre-configured integrations to common operations tools to manage your go-live tasks.
BPM – ICO has a robust BPM engine allowing you to craft an detailed process that can be initiated through a self-serve portal. This allows you to string together your provisioning, deployment, and GO LIVE tasks into a single user-driven process.

UrbanCode Deploy

deployment – UCD’s strength is application deployment including environment inventory and dashboarding.

UrbanCode Deploy with Patterns

deployment – UCD+P includes UrbanCode Deploy and relies on it to deploy application components.
patterns – UCD+P is a full HEAT pattern visual/syntax editor. UCD+P also has encorporated HEAT engine extensions that allow the heat pattern and the engine to not only provision nodes but also execute Chef recipes from a Chef server and also deploy applications using UrbanCode Deploy. The resulting HEAT pattern is truly a full stack representation as represented by the picture above in a single pattern artifact.

PureApplication

provisioning – PureApplication software has the ability to provision nodes (I will leave it at that without going into all the different flavors. For the purpose of this discussion we are interested in PureApplication software that manages PureApplication systems.)
patterns – PureApp has numerous vSYS patterns that help to automate the installation and configuration of IBM middleware. The provisioning engine is robust and can orchestrate the configuration of multiple nodes that would make up an IBM middleware topology.
deployment – in the same sense as ICO, you can add application deployment information into your patterns but the same limitations apply.

So if we cherry pick the best capabilities out of each tool, we would grab the go-live tasks and BPM from ICO, the app deployment from UCD, the HEAT pattern editing and HEAT engine extensions from UCD+P, and the IBM middleware patterns from PureApp. There, we are done. Maybe at some point in the future this will be a single PID that you can buy. But until then, is it possible to string these together in some usable way?

Again, a big disclaimer here. I am not an expert here on this entire stack but hope to drive conversation.

OK, to begin let’s take advantage of the BPM capabilities of ICO and drive everything from here. The BPM capabilities allow us to construct a process that executes the provisioning tasks and the go-live tasks with the appropriate logic. You can envision a self-serve portal with a web page that asks for specific information for a full stack deployment. Things like the app, the type of topology to deploy the app on top of, the name of the new environment, etc. ICO would need the appropriate pattern to then use to provision. Here is where ICO can “grab” the HEAT pattern from UCD+P via an integration. It will then execute the provisioning via the OpenStack HEAT engine. This HEAT engine must have the UCD+P HEAT engine extensions applied to it. Since these extensions contain application deployment capabilities, the provisioning process will also utilize UrbanCode Deploy to deploy the application components to the appropriate provisioned nodes based on the HEAT pattern. The BPM process can also call the appropriate operations products to execute the go-live tasks either before or after the provisioning step in the process. Whew!!

So what is missing? It would be great to take advantage of the PureApp IBM middleware patterns which are bar none the best and most robust installation/configuration patterns available. A general solution here would be to include the appropriate Chef recipes as part of your HEAT pattern to get the middleware installed and configured and for non-IBM middleware solutions this is your best bet. But there is a lot of orchestration involved in setting up WebSphere clusters, for example, that are not easily accomplished using Chef or Puppet. PureApp has this capability in spades. The best practice in using PureApp today is to use UrbanCode Deploy to deploy the app to a PureApp provisioned environment as the patterns in PureApp are not HEAT-based. A lot has been invested in the PureApp patterns and what the future holds as far as other pattern technologies here is uncertain today. It is important to know that PureApp is the preferred solution when it comes to provisioning systems that will utilize IBM middleware.

This is the story so far and I am sure I have holes in my description above. I welcome any feedback and experiences.

UrbanCode Deploy with Patterns and its Integration with UrbanCode Deploy

Well it has been quite a while since I have posted and lots has happened in the past 3 months. The re-organization of IBM has caused some internal shakeup but I have landed in the Cloud group which still is the home of the IBM DevOps solution. So I get to keep my current trajectory. With that though I have tried to expand my horizons and learn some new things. I have started to look at UrbanCode Deploy with Patterns more closely. It will play a key role in the IBM Cloud software offerings and will be the center point of HEAT document editing.

To back up a bit, HEAT is the OpenStack project focused on orchestration. The mission of the OpenStack Orchestration program is to create a human- and machine-accessible service for managing the entire lifecycle of infrastructure and applications within OpenStack clouds. In other words, HEAT provides the ability to define full stack application definition in a human readable artifact. The HEAT engine then reads the artifact and farms out the necessary tasks to other parts of the Cloud infrastructure to fulfill the parts as defined in the artifact.

UrbanCode Deploy with Patterns (UCD+P) provides a rich editor that allows you to craft HEAT documents in a visual way. It also defines some custom extensions to HEAT that allow some additional capabilities that are necessary to provide full stack functionality. Let’s examine in more detail.

UCD+P provides extensions that allow you to assign components from UrbanCode Deploy (UCD) to nodes in the blueprint. The component drawer of the resource pallet provides the list of components available to the user that can be applied to nodes.

comp_drawer

The entries in this drawer come directly from the UCD integration. The first time you drag a component onto the blueprint canvas you are presented with a dialog. This dialog is to confirm the application that you want to associate to the blueprint. In UCD you will end up with a new environment of the application you choose once the blueprint is provisioned. Remember that components can be included in more than one application and therefore this dialog allows you to choose which one.

app_dialog

 

 

 

Here is a picture of a node after a few components have been assigned to a node.

blueprint

Now let’s switch over to the blueprint source view and take a look at the UCD information that is included. First we see the blueprint parameters. Notice the UCD values that are necessary when provisioning the blueprint.params

 

Next we can look at the component information. We see two different resources defined. The war file deploy component defines the type of server it is being installed on. The version property is important. UCD+P defaults to the LATEST component version. This is fairly unrealistic. There are most likely a limited amount of scenarios where the LATEST version of all components are installed. More likely would be a snapshot. Unfortunately as of the writing of this post there is no capability to specify a snapshot name. This would be easy to add to the “choose an application” dialog since snapshots are associated with applications. This will be my first UCD+P enhancement request. I encourage anyone reading this to also chime in on this RFE.

A few other details to note are found in the component configuration resource. Here we see the name of the component and the component process to use. Note that the process name defaults to “deploy.” I believe it assumes you will have a process named “deploy.” You can change it to something else if you want to use another defined process. This again would be easy to add to the user interface when dragging it onto the blueprint canvas. But this one isn’t as critical as users typically define a default deployment process named “deploy.” Also note the input property that is required (JKE_DB_HOST). This is defined in UCD as a component environment property and is picked up via the UCD integration. It is not clear if, for example, I have a process property that is required or some other property whether those will be picked up as well. Note that this property can be associated with a UCD+P property that is then gathered at provision time.

comp

 

The UCD+P integration to UCD is very powerful and these HEAT extensions allow us to define both storage, compute, and networking information along with UCD component info in the same blueprint making this a true full stack artifact.

Now I am sure you can image some things that are missing in this picture. I will give you some hints. First, it looks to me in the example that I have shown that the middleware installation (and non-application configuration) is being performed by UCD. I am not sure this is the best place to perform this type of task and maybe there is a better way to do it. Also, it won’t take long in talking with an operations team some additional activities that are necessary in a production “Go Live” scenario. Stay tuned to some additional posts on things coming down the pike to fill in some of the gaps in the full stack picture.