UrbanCode Deploy and SmartCloud Orchestrator (extended addition)

It has been awhile since I posted on the UrbanCode Deploy and SmartCloud Orchestrator integration to provide a self-service portal for one-click environment provisioning and application deployment. Part 3 finished with the idea that a simple generic process can be called to drive this entire effort. You can use the Self Service Portal of SmartCloud Orchestrator to provide the user interface for all of this.

I also brought up in Part 3 that there are some details that need to be worked out in order to make it a reality. At the time I did not have access to a system to work through those details. Since then I was able to meet with a team that has made it happen. I want to share some of those details for those that are interested.

genproc2If we look back at our generic process that was proposed in Part 3, there were three steps. Step 1 creates the environment from the blueprint. A few issues exist with the default “create environment” step that already exists. First, you may have more than 1 cloud connection. There needs to be a way to specify a cloud connection in this step. By all means you can specify a default connection but there needs to be a way to distinguish between cloud connections. The easiest way to do this is to find the UUID of the cloud connection and use that when specifying the connection.

The other area that is not covered are the node properties. Each node in your pattern may require property values. This is totally dependent on the pattern designer and how much flexibility is provided by the pattern. There are those that might argue that you should limit the variability of patterns but that requires unique patterns for each variable combination. Either way, there is most likely a need to specify node properties.

The easiest way to do this is to use a JSON formatted string to specify the node properties. There is a CLI entry called “getBlueprintNodePropertiesTemplate” that will return a JSON template that can be used to specify the required node properties for a given blueprint. Use this template as the basis for passing node parameters to the provisioning process.

To make all this happen, there is a REST API PUT call that provisions environments. Its URL looks like this: “/cli/environment/provisionEnvironment”. It takes a JSON input that specifies the application name, the blueprint name, the new environment name, the connection, and the node properties, among other things. It makes sense to me to create a new plug-in step that looks similar to the existing “create environment” step but adds 2 additional properties, the cloud connection and the node properties (in JSON format). You may need to get creative in how you populate the node properties JSON string. Since you can potentially have different node properties for different blueprints, you may need to work with your pattern designer to make things consistent. This is again where good cooperation between deployment and provisioning SMEs makes sense. If you want to expose any of these variables to the end user, you will have to make that part of the Self Service portal on the SCO end and pass those choices into the generic process as process properties.

The next step in our generic process is to Wait for Resources. While this step is easy is principle, it breaks down big time when we get to reality. Each environment can have more than 1 node. Even if you use the agent prototype name pattern you still will have trouble determining the agent names. The existing “wait for resources” plug-in step requires you to type in the resources to wait for. This does not lend itself to a dynamic and generic provisioning process.

The best approach here is to write a custom plug-in step to wait for the resources of the specific new environment that you are provisioning. And you will probably have to extend the REST client to add some additional methods. The first step is to get a list of all the resources that you need to wait for. You can use the “/rest/resource/resource” REST API GET method to get a JSON of all resources. You will have to parse this using the environment name (and potentially the base resource) to get all of the resources that are part of the environment. Once you get that list, you can use the “cli/resource/info?resource=” REST API GET call to retrieve a JSON of the resource status. if the “status” field shows “ONLINE”, then your resource is up and running. Another property you may want to create for this plug-in step is a wait timeout value. Waiting forever doesn’t make much sense and you would like to know if things go wrong. Building in timeout logic will insure you get some notification back whether things to well or not.

The final step is the Run Application Process step. We should be able to use the default one here.

I am hoping to post all the code at some point for this solution, but until I get approval you will have to be happy with what is here. I hope this provides some additional ammo for making that self service portal a reality.

UrbanCode Deploy and SmartCloud Orchestrator (Part 2)

Back to our story. In Part 1, we connected to our cloud provider (SCO) and created a resource template from a virtual system pattern. We then organized our resource template and created some resource properties that will no doubt become useful in component processes that will deploy to this pattern.

The next step is to now switch perspectives and put our application hat on. Applications are deployed to environments, and UrbanCode Deploy allows us to create environments from resource templates. But first we have to map our application to the resource template. This is done via an application blueprint. This is a one time exercise for any given application/resource template combination. Many environments can be created from a single application blueprint.

The blueprint creation process from a resource template is very easy. On the blueprints tab of an application, you simply click the Create New Blueprint entry. You simply give the blueprint a name, description, and chose the resource template you want to map to.  The blueprint is created.

You now need to map your application to the blueprint. This involves simply assigning your application components to the agent prototypes. In our example, we have a 3-tiered pattern and we have to map each of our 3 components to the 3 tiers. After mapping, the blueprint looks like this. We have a simple 3-tiered application that will map easily to this pattern.

blueprint

The database component maps to the database tier node. The web service component maps to the application server tier node. And the web component maps to the web tier node. This makes sense. But what if we had different sized virtual system patterns and corresponding resource templates.

For example, what if we had a small single node pattern. Our blueprint mapping would look like this. We have a 3-node app server tier, so we simply map the web services component to all 3 of the nodes in that tier.

big

Or maybe at the other end of the spectrum, we have a small pattern with only a single node. In this case we could create a blueprint that would map all 3 components to the same node.small

Mapping our application to a resource template using a blueprint now gives us the final piece we need to be able to create a new application environment. newenv

The process of creating an environment from a blueprint will cause a new virtual system to get created from the virtual system pattern. You may have to select location in your cloud provider for the nodes to get created. There also may be some properties that the pattern requires for each node (if you have done any SCO work, you know there there are typically many properties values that are needed for the various script packages that you include in a node pattern).

 

 

 

cloudprops2

Once you click save, UrbanCode Deploy kicks off the provisioning of the new nodes that make up the new environment in SCO.  We can now go and check the resources and see what agents we are waiting for.

waiting4resources

As you can see, we have 3 agents that are waiting to come online. The agent names are important. You can see that they follow a pattern:  <application>_<environment>_<agent prototype>.

This is valuable information to have as you will most likely have to get agent properties as part of your deployment process. You can use the Get Agent Property component process step to get an agent’s IP address, for example. When specifying the agent, you would use something like the pattern:

${p:application.name}_${p:environment.name}_${p:resource/ws.node}

Remember in part 1 when I mentioned that I like to capture the agent prototype names as properties in the top level folder of the resource template.  This is where they come in handy.

Once the agents come online (if you put the agent script package at the end of each node in the pattern), then you can be sure that the environment is ready for deployment. You can now execute your normal application deployment process to your newly created environment.

Pretty simple and pretty efficient. In part 3, we will explore how you can create a developer self serve portal where developers can request environments with no interaction with UrbanCode Deploy.