Mapping applications to cloud domains

Among both users and industry professionals, there is no shortage of discussion about mapping application types to the different cloud domains (public, private, hybrid, etc.). In my experience, quite a bit of this discussion centers on breaking down the characteristics and traits of the application (what kind of data does it deal with, where are the external components it connects to, what are the throughput demands, etc.), and mapping those to a distinct cloud domain.

I believe that this way of mapping applications to cloud domains overlooks a simple fact: applications have lifecycles. Most of the time organizations map application characteristics to a cloud domain, they do so with the application’s production requirements in mind.  This overlooks the development, testing, staging, and probably a host of other phases in the application’s journey toward production readiness, and in some cases, it may be best to use different types of cloud environments along this journey.

Consider the case that an IT organization has decided to begin leveraging cloud computing techniques both within their own data center and from the public cloud. For a particular application (maybe an accounts management application), they decide it is best suited for the private cloud due to concerns over the sensitive nature of the data with which it interacts (I’m not endorsing private over public or vice versa, just posing a scenario and decision-making process that is not all that uncommon).

In all likelihood, the decision to use a private cloud for the organization’s account management application is indicative of the application’s production-level requirements. However, does this necessarily mean the application cannot benefit from a public cloud environment as it moves along to production? Nope.

In a fair number of cases where applications deal with sensitive data, they really only do so in their production environments. During the development and testing process, the application interacts with mock data. The data has the same structure, but the contents are not confidential. If the sensitive nature of the data used by the application was the chief reason  for choosing the private cloud, acknowledging the fact that such data is really only sensitive in a production environments opens the door to the possibility of using a public cloud during the development and test process.

Simply put, one should consider the characteristics of an application and put those characteristics in the context of each phase of the application’s lifecycle in order to understand how the application can make use of the cloud. Throughout this process, there are important factors to keep in mind. If you do in fact plan to leverage multiple domains in the delivery of an application, how easy will it be to move the application from one domain to the next? Will your application use the same platforms in the different domains? Can you automate the migration of the application from one lifecycle phase to the next? Can you capture an environment and easily reproduce it in a different cloud domain?

This is a small sample of the kinds of questions one should consider if they plan to leverage multiple cloud domains for a single application. While leveraging multiple domains during the lifecycle of an application may seem to make things more complicated, keep in mind that it can bring substantial benefits. It may mean decreasing the cost of developing and delivering applications, decreasing the time to market, or even enhancing support capabilities of an application already in production. Simply put, do not fall into the trap of making a one-to-one mapping between an application and its cloud domain. Consider the application’s lifecycle and construct a plan for using the cloud.

Building cloud-based application platforms

It seems that cloud computing conversations are slowly beginning to march up the stack. Much of the initial focus in the space has been on cloud-based infrastructures such as servers, storage, networks, etc. To some degree, cloud-based approaches to this layer of IT are becoming more of a norm and less of an exception. As this happens, the industry begins to look at the next layer of the IT stack in the context of cloud computing: application platforms.

Cloud-based application platforms are intriguing in that they promise to shift the focus of cloud computing to what is important, the application. In its ideal form, users describe their applications, the application’s dependencies, service level agreements, and then they hit the easy deploy button. The cloud platform renders the underlying infrastructure and provides a robust, dynamic runtime for the application.

If charged with building and delivering a cloud application platform, my end goal would be to render both the infrastructure and application platform software as a black box to end users. That is much, much easier said than done.

Recently I chatted with a colleague about this very same topic, and I realized just how much would go into delivering such a solution to end-users. I left that conversation with several different requirements, but to give you an idea I’ll share three with you here:

1) Robust stable of platform software: It is no secret that enterprises run their applications on a diverse set of platform/middleware software. This includes simple web servers, J2EE servers, databases, mediation containers, rules engines, data grids, and much more. To be viable in an enterprise setting, cloud platforms will need to deliver a wide array of application platform software.

2) Operational and runtime management services for applications: In order to absolve the user from having to tinker with the application infrastructure, the cloud platform will obviously need to understand how to deploy applications to the platform software. Beyond the operational aspects of just getting the application out there and running though, users will expect the cloud platform to supply runtime management services as well. This includes elasticity, fault tolerance, high availability, etc.

3) Extensible foundation for runtime service delivery: When I say runtime services, I am talking about hosted, managed function that applications running on the platform can leverage. These services supply functionality like messaging, billing services, search capability, authentication management, and a myriad of other things. In my opinion, as the number of players in the cloud-based application platform grows, these runtime services will be a key differentiator. Providers that offer a vast array of quality services are likely to attract more users to their platform. It is unrealistic to expect that a given provider has both the resource and expertise to deliver the complete set of services that would attract users, so the foundation for these runtime services must be well-documented and extensible so as to be attractive to the provider’s partners.

Clearly, there is quite a bit to providing a solid application platform in the cloud. The requirements above do not even begin to scrape the surface, and just those are a pretty tall order for any potential provider. I am definitely interested to hear what you think will be requirements for cloud application platforms, so send me your recommendations on Twitter @damrhein.

Setting apart WebSphere CloudBurst

From a constrained viewpoint, the WebSphere CloudBurst Appliance serves as a virtualization management solution for WebSphere application environments. In that light, I cannot tell you how many times customers ask me to delineate WebSphere CloudBurst from the other virtualization management solutions that are either out on the market or currently used in their business. I love to hear this request for two reasons:

1)      It signals that many enterprises already practice or are thinking about virtualization in the application environment space.

2)      I have a good answer!

To understand the main delineation between WebSphere CloudBurst and other virtualization management solutions, it is probably helpful to understand what the appliance does! Put simply and shortly, the appliance enables you to create, deploy, and manage virtualized WebSphere application environments. In particular, the deployment phase of that lifecycle equips users with the capability to turn a selected application environment (represented by a WebSphere CloudBurst pattern) into a collection of one or more virtual machines using special virtual images shipped on the appliance.

Once users hear this, they usually ask, “Why do I need WebSphere CloudBurst to do this? Couldn’t I achieve the same results using my own virtual image and management solution?” The same type of question applies to almost any kind of vendor solution. Of course you could potentially achieve something close to the same result on your own. I mean, not long ago most organizations were essentially writing their own ESB applications. This means then that asking why/if you need WebSphere CloudBurst is the wrong question. Instead, you should ask, “Why would I want WebSphere CloudBurst?”

The answer to that is decreased time to value and cost of ownership for your virtualized WebSphere environments. Consider that you are going down the road of creating virtual images for your WebSphere application environments, and you plan to provision those images from a general-purpose management solution. This means that you will have to provide quite a bit of scripting that tells the management solution how to interact with the software inside the virtual machines because to the management system, the virtual machine is a black box.

With WebSphere CloudBurst the virtual machine is anything but a black box. The appliance comes with the knowledge necessary to administrate and interact with the WebSphere software running inside the virtual machine. If you are familiar with WebSphere software, this means the appliance can do things like update hostnames for application nodes (important when reusing virtual images), federate nodes into cells, create application server clusters, tune the JVM for performance optimization, apply fixes, apply upgrades, and more all without the user needing to supply one line of custom scripting!

The fact that WebSphere CloudBurst comes with this out-of-the-box capability means you can avoid the significant investment in creating and testing these types of scripts in the first place. In addition, your cost of ownership with respect to this virtualized approach to WebSphere environments is significantly lower because IBM provides the support for these automated administration actions. You do not have to dedicate resource to supporting and updating administration scripts over time.

In addition to this out-of-the-box WebSphere intelligence, I cannot stress the value of WebSphere CloudBurst patterns enough. Patterns are the unit of deployment in WebSphere CloudBurst and they represent your complete application environment. This includes the topology of the environments (the kinds of nodes and the relationship between them) as well as your custom configuration (applications, application resources, tuning, etc.). This patterns approach allows you to codify your infrastructure, save it, and deploy it repeatedly with consistent results.

Patterns also provide a very effective abstraction from the underlying infrastructure. Regardless of the infrastructure to which you will deploy your pattern, you build and edit it the exact same way. This means, for instance, that you do not have to be an expert on the IBM pSeries platform in order to deploy a WebSphere environment there. Simply build your pattern, select the underlying image that will serve your pattern (which implies the platform), and point at the target infrastructure. WebSphere CloudBurst and its virtual images encapsulate platform specific knowledge that will transform your pattern into a running, integrated, and optimized virtualized WebSphere environment.

If you are a WebSphere user and you have not yet heard about the appliance, check out this brief introductory article. If that article does not answer your questions, you can reach me on Twitter @damrhein.

The IBM X-Force 2009 report

When it comes to my technical expertise in IT security, I’m generally familiar enough to know I should not pretend to be an expert. However, that has not kept me from getting a lot of valuable insight at the RSA conference this week. RSA has provided me the opportunity to hear a lot about the security threats out there, a little about how they work (including a really cool Conficker presentation by IBM’s Tom Cross), and a lot about what the industry is doing to combat these threats.

The most enlightening bit of information for me came when I listened in to a short presentation delivered by the IBM X-Force team. IBM X-Force is a team of thousands of security researchers and experts who are constantly researching and evaluating vulnerabilities in IT systems. They turn this research into countermeasures in IBM products, and they use it to educate the public at-large about dangerous vulnerabilities in their IT systems.

This team puts out reports at various points in the year that detail a wide range of vulnerabilities and their pervasiveness throughout the IT world. At RSA they provided an overview of their 2009 year-end assessment (which you can download here), and it contained some interesting insights into the state of IT security.

For the purpose of the report, the X-Force team describes vulnerability as “Any computer-related vulnerability, exposure, or configuration that may result in a weakening or breakdown of the confidentiality, integrity, or accessibility of the computing system.” The X-Force team saw a significant decline in the number of disclosed vulnerabilities they analyzed. In all, 6,601 new vulnerabilities were scrutinized as opposed to well over 7,000 in 2008. Tuesday continued to be the day with the most vulnerability disclosures, the weekends were the slowest, while the slowest and busiest months came in November and December respectively.

The report contains information about various types of vulnerabilities, but the piece focused on web application vulnerabilities is particularly interesting. Overall, the number of disclosed vulnerabilities in web applications grew in 2009. The first half of the year saw exponential growth (standard fare over the last 5+ years), while data from the second half may suggest a slight slow down in the increase. The report cites the most common web application vulnerabilities as cross-site scripting, file include holes, and SQL injection.

It’s doubtful that anyone is surprised to learn those three culprits make up the largest percentage of web application vulnerabilities, but what is a bit surprising in regard to those vulnerabilities is that from 2008 to 2009 only cross-site scripting increased its share of the pie. In 2008 cross-site scripting accounted for a little under 30% of disclosed web application vulnerabilities. In 2009, this percentage was closer to 40%. This is quite an increase especially when you consider that this is arguably one of the most widely-known, well-understood types of vulnerabilities for applications.

I’m not usually one to write much about security, but the web application vulnerability results have clear implications for cloud computing. Cloud computing did not start the notion of delivering applications via the internet, nor can one consider any web application to be a cloud-based application. However, there is little debating that cloud computing is leading to the production and use of even more web-based applications. On the brink of what should be a rapid and considerable increase in the number of web-based solutions, these vulnerabilities only increase in importance.

If you are in the business of either delivering or consuming web applications, it is critically important to understand the vulnerabilities that exist, especially those most common. For consumers of web applications (read all of us), not only is it important to understand the vulnerabilities of those web applications, but it is also helpful to understand the threats that exist for clients. The report also has this information.  Ignoring the threats and vulnerabilities will not make them go away, so get proactive and check out the IBM X-Force report for 2009.

Security isn’t the biggest cloud challenge

Frankly I’ve grown weary of the debates over the security of cloud computing. It’s not that I don’t appreciate that there are technical hurdles in front of us, but we have reached a point that a security vulnerability in a single offering, whether that offering is in the public or private cloud, results in loads of silly commentary that links the particular problem to the overall state of cloud security. I’m not sure if those involved in this commentary have a vested interest in driving this kind of dialogue, or if it happens because it is easy to write about, but in either case a majority of the discussions around cloud security have degraded to pure absurdity.

By and large I’ve learned to ignore most of this commentary as useless rambling. However, the problem with all this talk is not that it is simply aggravating, but it distracts from what I think is a more serious challenge in the cloud computing space, one that I believe slows adoption more than concerns over security. That’s right. While I’m positive many heartily disagree, I don’t think security is the biggest challenge to the adoption of cloud computing. In fact, in my opinion the biggest challenge is not even a technical issue.

I can’t even begin to remember how many times I’ve been in meetings where a cloud solution was being pitched and all of a sudden the conversation becomes dominated by audience members. This dialogue isn’t usually conversation between the audience and the presenter either. Rather it is a back and forth between company reps in the audience that usually includes questions like the following:

- “What is the networking team going to think about this?”

- “Will the operating system group approve of this approach?”

- “Can our data center administrators work with this new solution?”

Naturally the questions vary depending on the solution being presented, but these types of questions, which in my experience are as frequent as the technical questions, lead me to believe that the operational culture of today’s enterprise poses the single biggest challenge toward increasing adoption of cloud computing solutions.

In large part due to the types of technologies built over the last several years, organizations have been “encouraged” to solve problems by building up silos around particular activities. Teams are tasked to complete a piece of the overall solution (i.e. networking, operating system, application, operations, etc.), and each team has their own tools and processes to help them achieve their goals.

While there are arguments to be made both for and against this segmented approach to IT, one thing that cannot be argued is that this approach is at odds with many cloud computing solutions that are more holistic in their approach. As a simple example, think about a typical provisioning process for an application stack:

1)      User submits request for application platform

2)      Infrastructure team assigns physical machines and networking resources

3)      Operating system team installs and configures operating system

4)      Middleware team installs and configures application platform

5)      Applications team installs and configures application

6)      User gains control of the environment

Compare this to the approach taken to provision the same application stack with a PaaS solution whereby the requesting user simply selects their required environment and provisions it into the cloud. This single-step, self-service approach is certainly more streamlined and perhaps sustainable, but it implies quite a bit of a change in the organization.

In this case, the use of the new PaaS solution doesn’t mean the organization is throwing out their normal requirements for setting up application environments. However, it does mean that the operational culture they have built up around this process changes. No longer are the teams working in silos with their own tools, but instead they all work within the domain of the PaaS solution and its toolset to enable the one-step, self-service provisioning process exposed to the end user. This change can be quite significant and require working relationships and dependencies that were previously unnecessary in the organization. While that may sound simple enough, changing the way teams interact within an enterprise often entails tons of analysis and discussion, and sometimes proves to be all but impossible.

We must filter the noise from things like cloud security commentary and get out in front of the cultural obstacles that stand in the way of cloud computing adoption. Overcoming these hurdles means the inclusion of certain capabilities in products like change management control, process approval, and the careful division of user permissions and access. It also means that vendors need to be proactive and engage the industry and challenge enterprises to start thinking about what adopting cloud computing within an organization really means. Beyond the technical challenges, emphasis needs to be put on the importance of understanding the impact of cloud computing on operational culture. After all, without this crucial understanding the adoption process will either suffer from a turtle-like pace or altogether fail, and obviously neither outcome is a good one for the cloud industry.

IBM and the Air Force sign cloud deal

Today my employer, IBM, announced that it was awarded a 10 month long contract with the United States Air Force. According to the press release, the contract calls for IBM to “design and demonstrate a secure cloud computing infrastructure capable of supporting defense and intelligence networks.” Lieutenant General William Lord, Chief Information Officer and Chief, Warfighting Integration for the Air Force, goes further by stating the purpose of the project will be to “develop an architecture that could lead to improved performance within the Air Force environment to improve all operational, analytical and security capabilities”

As near as I can tell from the press release there are two main requirements of the project. The first requirement is that IBM employs cloud computing techniques to enable the Air Force to perform what the press release calls “stream computing.”  Simply speaking, this is about producing actionable intelligence by inspecting massive amounts of continually streaming data in real time. There are probably few use cases where stream computing capabilities are more mission-critical. The Air Force will be leveraging the capability to detect cyber threats, network failures, application failures, etc., and prevent those dangers from becoming a service disruption that could inhibit their operational capability and possibly threaten our security.

Unlike the first requirement, the second requirement is a goal for a majority of cloud projects. The Air Force is looking for the solution to deliver autonomic computing capabilities. Resources in the cloud must continually be tuned for the highest level of optimization, and the tuning should happen with no need for human intervention. While the requirement is standard fare, I would bet the implementation will be anything but ho-hum. The kind of optimization that is being talked about here is balancing world-class performance with little tolerance for disruption, with the guiding principle of constraint in the consumption of resources. Only the resources necessary for the job should be used, no more. That may be easy in environments with lenient performance goals, but the environment proposed here will be anything but that.

It’s too early to tell exactly what will come of this joint project between IBM and the Air Force, but it warrants a close eye over the coming months. Not only will the implementation details, at least what gets publicized, likely be pretty interesting, one can only assume that the project will lead to reusable discoveries and best practices in at least the areas of cloud optimization and security. If the resulting architecture and techniques hold up to the rigors and requirements put forth by the mission-critical, super security sensitive needs of the Air Force, it’s probably hardened enough for nearly any use case.

What does elastic really mean?

In terms of cloud computing in application environments, elasticity is perhaps one of the more alluring and potentially beneficial aspects of this new delivery model. I’m sure that to many of those responsible for the operational and administrative aspects of these application environments, the idea that applications and associated infrastructure grows and shrinks based purely on demand, without human intervention mind you, sounds close to a utopia. While I would never dispute that such capability can make life easier and your environments much more responsive and efficient, it’s important to define what elasticity means for you before you embark down this path. In this way, you can balance your expectations against any proposed solutions.

For me, the idea of elastic application environments starts with the implication that there is some sort of policy or service level agreement (SLA) that determines when to grow and when to shrink. However, just having the capability to govern your runtime with SLAs isn’t enough. The SLAs should be applicable to performance metrics directly related to your applications. For example, it may be nice to be able to make operational decisions in your application environment based on the CPU usage of the physical machines supporting that environment, however it is much nicer to make those same decisions based on the average response time for requests sent to your application instances or perhaps the average time a particular message waits in your application’s queue. When you have the ability to define SLAs based on these kinds of application performance metrics you can remove a lot of ambiguity that otherwise could creep in when making expansion/contraction decisions.

What’s obvious is that there’s no reason to have SLAs that cannot be enforced. When I think about SLA enforcement there are a couple of things that come to mind. The first is that the party responsible for enforcement should be configurable. In many cases you may want your application environment to grow and shrink based on the system’s autonomic enforcement of SLAs, but I doubt this will always be the case. For example, if you are running in a pay-for-use public cloud environment, you may, in an attempt to keep costs under control, want to insert a manual approval process before the system grows. As another example, you may insert manual approval processes for contracting application environments in a production setting where demand fluctuates wildly. In any case, the ability to configure who is responsible for SLA enforcement is useful.

The second thing that comes to mind with respect to SLA enforcement is that you should be able to prioritize such enforcement. The ability to prioritize SLA enforcement means that you can ensure that conditions in some applications warrant a faster response than in other applications. This is just an acknowledgement that not all applications are created equally. Obviously if a user-facing, revenue-generating application starts to violate its SLA you want that condition addressed before you address any SLA slippage in an internal environment.

Besides the ability to define and enforce SLAs, there are certainly other capabilities that contribute to the robustness of a truly elastic application environment. One area that warrants attention is the degree to which application health monitoring and maintenance can be automated. For instance, when an application begins to leak memory and response times slow to the point that SLAs are violated, it may be more efficient to automatically address the leak by say restarting the application as opposed to adding more application instances.

These are just a few of what I’m sure are many facets that contribute to elasticity in an application environment. They happen to be the ones that bubble to the top for me, but I have no doubt there are others that may be more important for you. If you have your own ideas for elastic application environments I’d like to hear them. Drop me a line on Twitter @WebSphereClouds.

Is WebSphere CloudBurst right for you?

Trying to pick apart a particular IT solution to figure out if it is right for you and your business can be a daunting task. Trying to do that for solutions in emerging spaces, like cloud computing, can seem even more difficult to the point of looking almost impossible. Case in point, at most of my WebSphere CloudBurst sessions someone (usually multiple people) always asks how they can determine if this particular product is right for them. It’s a good and fair question, and one that I usually answer by asking a few questions of my own.

To best understand if WebSphere CloudBurst is potentially a fit for you and your organization, it’s best if you ask yourself some of the questions below:

1) Do you feel as if you spend too much time and resource installing the software that makes up your application infrastructure stack?

Let’s face it, if your goal is to develop and deliver applications to end users, chances are you want to spend as little time as possible installing and configuring the software that makes up your application infrastructure. It’s likely you would rather invest more time and resource toward the research and development of applications.

The Hypervisor Edition virtual images that are used by WebSphere CloudBurst contain a pre-installed, pre-configured application infrastructure stack. The images include an operating system, web server tier, application middleware tier, and any other customizations you choose to include. Your valuable time and resource can be diverted away from tedious installations and back toward your applications.

2) Do you spend a lot of time and resource configuring your middleware application environments?

WebSphere CloudBurst makes it very easy to quickly build meaningful WebSphere application environments. You simply drag and drop components on a canvas to build your application infrastructure topology in the form of patterns. The appliance understands and automatically configures the relationship between the components in your pattern (i.e. it knows how to cluster application server nodes, configure web servers, etc.), so you don’t have to spend your time doing these tasks. The appliance also automatically tunes the application environment during deployment to ensure that you start off on a solid performance footing.

In addition to what the appliance provides, you can include and save your own customizations in these reusable patterns. This means you can configure it once, save it in a pattern, and subsequently utilize that application environment as many times as necessary.

3) Are you plagued by bugs introduced due to inconsistent configuration as you promote your application environments from one context (development, test, staging, production, etc.) to the next?

If you are currently in or have ever spent time in an application development organization, you are probably acutely aware of the pains associated with the promotion of applications. Each time an application and its associated infrastructure is moved from one environment to the next, the possibility of a bug-causing change, ever so slight it may be, is real. These error-producing changes, especially those to configuration, can be extremely hard to track down and end up slowing down the project and ultimately costing you serious money.

Using the patterns-based approach in WebSphere CloudBurst drastically reduces the possibility of what is typically referred to as configuration drift for your application and application infrastructure as it moves along the promotion chain. The reason for this is that the configuration for your application and its infrastructure is preserved on the appliance in the form of a WebSphere CloudBurst pattern that can be locked down. These patterns can be deployed as many times and to as many different settings as necessary, and each time you are sure to get the same result. This isn’t to say there is no flexibility with the patterns-based approach. The patterns can be parameterized to allow for deploy-time configuration for those things that may be different in each setting (i.e. database names, hostnames for external connections, etc.).

4) Is the lead time for setting up your middleware application environments inhibiting your capacity to be agile and responsive to changing requirements?

This one is quite simple and objective. Do you measure the amount of time it takes to get your application environments up and running in terms of days or weeks?

If so, WebSphere CloudBurst can improve upon that and turn your measurement of this lead time into minutes. Using the appliance, you can go from nothing to a fully running, clustered WebSphere Application Server cell in under 20 minutes. Similarly, you can have a fully installed and configured WebSphere Portal environment running in less than 15 minutes. Need a running DB2 environment? Try 3 minutes! No matter the environment you are provisioning with WebSphere CloudBurst, you can expect deployment times that enable agility and flexibility for your business.

5) Would you benefit from being able to schedule and automate the application of fixes and upgrades to your middleware application environments?

Do you dread the thought of applying fixes, upgrades, or other maintenance to your application environments? Perhaps it is a manually driven process that takes too long and is conducted at odd hours to avoid disruptive service outages. WebSphere CloudBurst allows you to apply both fixes and upgrades, and in addition you can use built-in scheduling to automate these actions. Each time the appliance applies a fix or upgrade, it begins the process by taking a snapshot of your entire environment. This provides a last-known good state that you can rollback to at any point with the click of a button.

These aren’t the only issues to consider if you are thinking about the WebSphere CloudBurst Appliance solution, but they are at least a good starting point. If you answered yes to any or most of the questions above, you may want to take a look under the hood via a nice collection of technical articles and demonstrations. Also, feel free to reach out to me on Twitter @WebSphereClouds.

Reporting for the cloud

Normally when you read or hear someone talk about application environments running on cloud platforms a lot of focus is put on provisioning and elasticity. Mainly the claims are that you should be able to very quickly provision full application environments on the cloud platform and that those environments should grow, and shrink, based on the demands on the system.

I certainly have no argument that those capabilities are important functionality for a cloud platform, but based on some recent conversations with several different users I’m beginning to think we aren’t talking enough about another important feature for these solutions. That feature is the degree to which the application environments running on cloud platforms can be audited.

Auditing of the application environments running in a cloud has many facets, but let’s just consider it as it relates to one of the basic tenets of cloud computing: utility-based pricing. If you are thinking of either using or constructing a cloud application platform within your enterprise, chances are utility-based pricing is important to you. You either expect to be charged for only the resources you use, or you want to allocate costs back to teams or users based on usage of your platform. Your usage charges may be for the number of hours that an environment runs in the cloud (the Amazon EC2 pricing model), or it may mean something even finer-grained such as paying for the utilization of compute resources like CPU, memory, and storage. Any way you account for usage, the implication is that the cloud platform, or associated tooling, provides or enables you in some way to extract and filter the relevant data about the environments running on said platform in order to produce meaningful reports.

The best-case scenario is that the cloud platform directly provides you with your reports. For instance, if you decided to charge for use of your cloud platform solely based on usage hours, then the cloud platform solution might provide you with a report that showed the breakdown of usage hours for each user or team over a specified period of time. In this case, you don’t have to do anything to extract, filter, or otherwise massage the data. You tell the solution you want a usage report, and it gives you just what you need.

Obviously, this turnkey reporting capability is not likely to be there for everything you need because there is no way for a solution provider to anticipate and provide every meaningful report for every user right out of the box. What do you do when you encounter a solution that you otherwise like, but doesn’t necessarily give you the reporting you need right out of the box? Simple. You ensure that the cloud platform solution provides a mechanism by which you can extract audit data to support your own custom reporting.

The solution may enable you to extract data by providing a robust CLI, REST interface, web services interface, or query building capability. Regardless of the data extraction interface, if custom reporting is important to you, you’ll want to ensure that not only can you extract the data, but that you can also filter and shape the data to fit your needs. If you can’t filter the data, you’re left with little more than a pile of meaningless bytes and bits. Just as importantly, if it looks too hard to filter the data, you’ll want to consider whether it’s worth the investment to write the necessary data mining logic.

The bottom line is that when you look at a cloud platform solution, be sure to look past that solution’s approaches to provisioning and elasticity. Take a hard look at how it helps you to audit the application environments it will support. It’s better to understand and accept those capabilities upfront instead of being disappointed once you put the solution in place.

Declarative programming for the cloud

Earlier today I read an interview with fellow IBMer Andrew Spyker that touched on the use of declarative programming models in the new WebSphere XML Feature Pack. I think most of you are probably comfortable with what a declarative programming model is, especially in comparison to imperative programming models, but I really like Andrew’s explanation in the interview:

… declarative programming asks the user what they want to do. This is as opposed to imperative programming (ex: Java code working with the DOM or JAXB APIs) which asks the user how they want to do what they want to do.

Andrew goes on to make the point that not only is code written in the declarative style easier to adapt and maintain over time, it also lends itself to better optimizations. The reason for this is pretty obvious based on his definition above. Code written in the imperative style explicitly tells the computer how to do what the user wants to do. While some optimizations to this code can be, and often times are, inserted by a compiler, it doesn’t lend itself to the same optimizations that code written in a declarative style does. When coders simply tell a system what they want to do, as is the case in the declarative style, the system can decide the most efficient way to carry out that action.

While not the focus of the interview, Andrew does mention that declarative programming models may be particularly useful in the realm of cloud computing. I believe that line of thought has quite a bit of merit. As far as application environments go, most of the cloud computing movement to date has been centered on driving efficiencies through the use of virtualizing complete application stacks. This provides us with the necessary encapsulation to be able to very easily and quickly bring complete stacks online, all the while allowing us to run more of those stacks, with the same set of compute resources, when demand dictates.

There is no doubt that this type of virtualization is a good thing and when leveraged correctly can bring significant improvement with respect to the responsiveness of our applications. However, we can’t stop by putting a box around our application stacks. If we truly want to get the most out of the dynamic environments offered by cloud computing platforms, we have to look inside that stack as well to drive efficiencies into the code that makes up our applications.

What’s the best way to do that? Well, the fact is that right now the community of developers who are well prepared to write code that harnesses the power of massive, highly scalable cloud platforms isn’t exactly bursting at the seams. To address that many initiatives are being launched that seek to make these type of programming skills part of core computer science curriculums in colleges and universities across the world. While I do not at all dispute the fact that those initiatives are necessary and valuable, I also can’t help but wonder if that’s the best and only way.

What if we focused more on the use of declarative programming models as the style of choice when writing applications deployed on cloud environments? In this way developers write applications that tell the cloud platform what they want to do and it’s up to that platform to decide exactly how to do it. This approach means that developers do not need deep knowledge on how to best harness the often times massive and complex underlying infrastructure of a cloud platform to meet their needs. Instead the cloud platform makes decisions, being presumably much more equipped to do so, about how to most efficiently take the action the user is requesting.

To me this approach is a win-win for developers and cloud platform providers. Obviously developers can get back to focusing on the business logic they are trying to provide, and cloud platform providers have more control over exactly how its resources are leveraged which can only help in terms of meeting user demands and SLAs. I acknowledge that declarative programming techniques will also be a new set of skills for some developers, but to me this seems more easily achieved than learning how to best use complex cloud environments within application code. I say we put the burden for leveraging massive compute resources squarely on the shoulders of the cloud platforms.