Cloud Standardization

motivation

The cloud is omnipresent. On every conference, in every journal and even while the newspaper one wants to stumble upon this buzzword. Whatever the cause or the effect of the cloud, [1, 2, 3] (or even a cloud itself [4, 5, 6]). However, customers are still very cautious in their own private cloud or migrating their application into the public cloud. One of the most important things about privacy and security is the implicit vendor lock-in.

Vendor lock-in

Currently cloud providers focus on adding new services, adding unique features and integrating / coupling their offerings. Since there is no universal agreement on what constitutes a cloud provider. Most offerings target a certain group of customers. Once a customer is won, it is hard for him to change to another provider. The customer is already using it to migrate his application to a new cloud provider. Amazon for example published 160 new features or services in 2012 underpinning its position as one of the market leaders.

Hybrid cloud

Another drawback of diverging cloud technologies is the growing effort needed to combine cloud offerings. this hybrid cloud approach enables a customer to keep his or her sensitive data in his or her private cloud. Not only has the application to work with two different APIs to manage the private and public cloud.

What the Customer Wants

From the customer’s point of view it is just as varied as it is different in cost, functionality and quality. To get the best, fastest and cheapest solution possible, it is unacceptable to only the offerings of a single provider. However, this is where the road leads today. The additional complexity of finding and integrating the right services weighs much less on the development budget when compared to established ready-to-use cloud environments.

This can only be changed with interoperability and portability of cloud services backed by an industry standard. The following paragraphs are trying to make a difference with the different clouds.

Tools

This section is not intended to be used in depth usage analysis of the tools mentioned nor does it compare competitors. Solving a specific problem.

OpenStack

OpenStack is a software to build your own (private) cloud. IBM, HP, Dell, SUSE and CERN to name only a few of them. Once deployed it offers the capabilities to manage virtualized computing, storage and network resources. It has a dashboard so no administrator or user needs to understand the details of a hypervisor or a network management system. IBM, CALXEDA and SUSE are already offering products / services powered by OpenStack.

Launching an instance on OpenStack (Source www.openstack.org)

(Yes, there are Eucalyptus, OpenNebula, CloudStack and others that are just as popular right now.)

Apache Libcloud

Apache Libcloud is a Python module that gives administrators and programmers a uniform interface to access different IaaS providers. There is a long list of supported providers and features. See following code snippet for an example of listing all three different providers (Amazon EC2, Slicehost and Rackspace).

The big advantage is that one of the differences between the providers. This makes it a lot easier and eliminates the need for an expert. Libcloud implements and releases API changes.

Delta Cloud

Deltacloud is similar to Libcloud, but is implemented in Ruby and not in Python. There is also a vast amount of supported ‘Clouds’. The major difference is that DeltaCloud also supports a CIMI mode (see Cloud Infrastructure Management Interface in the next chapter standards).

standards

The relevant paragraphs outline the scope of the standards but, by far, do not resemble a full analysis.

Open Virtualization Format (OVF)

The specification of the Open Virtualization Format (OVF) is maintained by the Distribute Management Task Force (DMTF). According to the spec it

“This is an open, secure, portable, efficient and extensible format for the packaging and distribution of software in virtual machines.”

An OVF descriptor not only can one VM but multiple VMs, called systems of machines. According to the OVF package specification the following artifacts:

one OVF descriptor with extension .ovf
zero or one OVF manifest with extension .mf
zero or one OVF certificate with extension .cert
zero or more disk image files
zero or more additional resource files, search as ISO images

Furthermore, it provides a descriptive file defining a virtual machine amount of memory and cpu as well as the network configuration. The manifest file contains SHA-1 digests of all files to ensure the integrity of multiple transfers. To ensure authenticity the manifest file can be signed and the digest stored in the certificate file. All artifacts can be compressed into an appliance (.ova as file ending) for easier distribution (as done here for Zimbra’s Collaboration Server). All major virtualization products support import and export of OVF files, so chances are, this standard will be around for a while..

Cloud Data Management Interface (CDMI)

Storage Networking Industry Association (SNIA) put some effort into specifying a Cloud Data Management Interface (CDMI). It “[…] defines the functional interface that applications want to create, retrieve, update and delete data elements from the cloud” (source). SNIA chose REST over HTTP for implementing the standard.

The standard defines a Container and a data object as the major artifacts. Storage space is exposed as a container. Containers can be created with a parent child relationship to recursively group data objects, although this recursion is an optional feature. Following operations can be carried out on the afore-mentioned artifacts:

discovering the capabilities of a cloud storage provider
creating a new container
creating a new data object
listing the contents of a container
reading the contents of a data object
reading only the value of a data object
deleting a data object

The next code listing, taken from the specification, shows an example request for creating a container. One can see that the container should be reachable through the iSCSI as well as through the NFS.

More detailed information on the given excerpt and examples as well as additional information on cues, domains, logging, snapshots and the concept of capabilities can be found in the specification.

Cloud Infrastructure Management Interface (CIMI)

The Cloud Infrastructure Management Interface is another standard defined by the DMTF. It includes a model for cloud infrastructure resources as well as a standardized REST over HTTP interface to manage said resources. Data is either transferred to XML or to JSON notation. Currently the model covers systems (cluster of machines), machines, volumes, network and monitoring. CIMI is compatible with the OVF format which was earlier and thus comes from the DMTF. OVF appliances can be imported (and exported) resulting in the creation of or are used by the CIMI resources.

Each CIMI implementation needs to provide a Cloud Entry Point to publish the supported resources and their corresponding URIs for interaction. A CIMI client can discover all resources starting from the Cloud Entry Point, regardless of the implementation of the serving component.

The following example is from the specification and shows an update operation on a machine. A new Surname is set and the description is being removed (because no value for description is provided).

The simplest version of a CIMI client is a HTTP client like curl.

This example is taken from the Deltacloud documentation and retrieves a list of existing machines. More examples can be found here.

Open Cloud Computing Interface (OCCI)

Much like CIMI the Open Cloud Computing Interface provides a REST over HTTP based way to create, update and destroy cloud infrastructure resources. It is on computer, network and storage only. Even the example of updating an instance attributes looks very similar (found in the specification):

OpenNebula for example already has built-in support for OCCI. From a quick look into the specifications of OCCI and CIMI I would say both are powerful IaaS standards.

Cloud Application Management for Platforms (CAMP)

CAMP attempts to standardize the interaction with PaaS providers. The Organization for the Advancement of Structured Information Standards (OASIS) based project also uses REST for communication. The resources managed by the system are divided into platform, platform components, applications components and assemblies. A platform describes a whole cloud offering. A platform component is a distinguished service offered by the platform. Application components are configured parts of an application. All application components (and resources) together form an assembly. Besides these resources CAMP also covers the whole lifecycle of an application as shown in the figure below.

An application’s lifecycle (source http://docs.oasis-open.org/camp/camp-spec/v1.1/camp-spec-v1.1.pdf)

Similar to OVF, CAMP defines a Platform Deployment Package (PDP) to hand over an application’s data in a structured manner. This package contains the following items:

one Deployment Plan file with extension .dp
zero or one manifest with the extension .mf
zero or one certificate with the extension .cert
zero or more language specific bundles
zero or more additional resource files

Again, the PDP and can be signed through the certificate file. The deployment plan contains all the information needed to set up the environment needed by the application. It provides templates and configurations for the platform components and defines the needed dependencies.

Currently the standard only defines a database platform component (mainly because Oracle is the driving force?).

Open Services for Lifecycle Collaboration (OSLC)

Open Services for Lifecycle Collaboration (OSLC) is a framework of defined interfaces to integrate Application Lifecycle Management (ALM) and Product Lifecycle Management (PLM) tools. The specification of these interfaces is divided into different working groups that focus on a specific topic (automation for example). Through these interfaces the different tools can interact, share data and create a workflow that is spanning an application’s lifecycle with a common view for the user.

This is great news for users. No need to worry about integrating different products or finding workarounds for product incompatibilities. So there is a huge opportunity for software companies. It is supposed to integrate with all its components. If you are interested, here is a short introduction to OSLC.

By downloading the video you accept the privacy policy of YouTube.
Learn more

Always unlock YouTube

There is already an SDK to help you getting started with an OSLC implementation, too.

Last but not least, the list of supporting organizations is very impressive.

Topology and Orchestration Specification for Cloud Services (TOSCA)

Topology and Orchestration Specification for Cloud Services (TOSCA) is a standard framework for describing service templates. The primary target is to enable developers or architects to describe the topology of a service in an interchangeable format. Which software components does the service need and how do they interact? The strength of the approach lies in its generality. Fundamental artifacts are NodeTypes and Relationship Types. Both can be enriched with Properties and expose interface for management functionalities. A Service Template assembles Node- and RelationshipTypes to a structured description of the cloud application or cloud service.

A NodeType can therefore be a recursive composite of a Service Template wrapped in a Boundary Definition. This allows for the abstraction of complex applications. Node- and RelationshipTypes have to be backed by so called Implemetation artifacts which contain the actual work in the form of scripts or program code. A NodeType for Apache Tomcat Server for example could install, Uninstall, Start and Stop. A NodeType only defines the existence of those operations. Therefore it is possible to have more than one implementation for a specific NodeType. TOSCA does not regulate which programming language has been used for Implementation Artifacts. Rather it defines different runtime environments that a TOSCA container can provide (eg Python, Java, Shell). So it’s the developers’ decision. “NodeTypes or if it defines a NodeType as TOSCA Container should use any matching implementation. This could be a valid scenario if one only needs a JEE application server.

More complex installation processes, which are not derivable from the topology, can be described with plan. All artifacts combined (definitions, resources, scripts, …) form the Cloud Service Archive (short CSAR). In the future a CSAR can be seen as WAR or EAR of the cloud to be exchanged between different cloud providers for deployment.

Overview of TOSCA artifacts version 1.0 (http://docs.oasis-open.org/tosca/TOSCA/v1.0/cs01/TOSCA-v1.0-cs01.html)

Most important for the understanding of TOSCA is that it does not standardize a product or a certain topology for cloud applications but a language for describing (cloud) service topologies. With OpenTosca an open implementation of TOSCA was started by the University of Stuttgart. Most of the supporting product companies have already announced products for this standard. Usually these are not brand new products but additions to existing automation suites.

Chances are good that a whole new eco system will emerge around this standard, if it prevails. Like apps in an apps store, implementation artifacts could be made available and enabled. I’m looking forward to seeing this happen!

Final Overview

Here is a brief overview of the individual topics presented earlier and how they fit together.

Cloud standards sorted by topic

As one can see, most implementations can be used simultaneously to pave your way to the cloud. TOSCA and OSLC are very high-level approaches to the cloud. Both can be combined as a TOSCA implementation thus supporting the corresponding OSLC interface (of automation for example). Since TOSCA does not include any pre-defined implementation, it does need to be based on 3rd party artifacts. Either way someone needs to fill implementation artifacts with operational logic. This effort may be minimized by leveraging OCCI or CIMI for infrastructure resources and CAMP for platform configurations.

APIs like libcloud or deltacloud really help to lower the barrier for portable cloud deployments. OCCI or CIMI one day.