Data Centers - Virtualization

Looking at projected increases in energy prices and real estate over the next 5 to 10 years, the data Virtualization Diagramcentre has come under huge scrutiny from the business its supports and is being challenged to deliver more operational savings, through lower power consumption, and to achieve increased productivity.

The notion of Virtualization offers the Data Center Manager the promise of achieving both those goals, whilst at the same time simplifying the network

Prior to virtualization the majority of business applications were hosted on their own specific servers.   The consequence was that servers were being powered and cooled 24/7 to maintain storage and processors that were typically only 15% utilized.

With virtualization, multiple business applications can be run on a single server - with each one running in its own operating space or "virtual" server. In some ways it is similar to running two different Operating Systems (Like Windows & MAC OS) on a PC or laptop by loading them on to different disk partitions - but instead of running one or the other - you can run both simultaneously. 

Consequently, server utilization rates increase and operational savings can be achieved. An example of virtualization would be taking 6 physical servers running at 15% utilization and replacing them with 1 physical server to run the 6 servers virtually and achieving 90%.

In 2009, IDC noted that:

"The IT industry is witnessing a dramatic shift to converged architectures. It is clear that the current separate deployments of servers, storage, and network will not meet the requirements of the new data center.  A new, more flexible I/O architecture will be needed to reduce complexity and meet IT requirements. Server, storage, and network I/O virtualization will be the overriding premise of the future datacenter build-out."

In its 2011 Worldwide Data Center Forecast, IDC noted the synergy among virtualization, converged infrastructures, and the Unified Fabric:

So, what does the IDC perspective tell us?

Yes, you consolidate space and energy but virtualization also adds another layer of administration tools - troubleshooting on a virtual server is done via virtual server management software rather than direct server access (as you would in non-virtualized). This can make administration & troubleshooting more "distant" and less "hands-on".

A "Converged Infrastructure" begins to look and feel more like a "Cloud" infrastructure within the data center. i.e. A cloud within a cloud. But with significant organizational benefits specifically in terms of Efficiency, Availability and Agility.

The panels on the right give IDC's perspective & definitions and potential outcomes and consequences of Virtualization, Unified Fabric and Converged Infrastructures.

Since Ethernet has become the primary networking technology there has been an understandable desire by the active community to gain acceptance of Ethernet at every point in an infrastructure possible.  However, until recently within the data centre the technologies supporting transport have not been exclusively Ethernet but have also included FibreChannel, SCSI, and Infiniband.

This presented a problem as Data Centre Managers wanted to consolidate more workload on to fewer servers (ie. Virtualize), when those servers may not have had the correct port adaptor to communicate to the outside world.

The solution to this problem was the development of a concept called the Unified Fabric.  In this concept the base processing can still take place on a server as normal in the native technology (e.g. FibreChannel) but the data that is being transmitted is inserted into Ethernet packets prior to sending and then decoded back to the native language at the receiving server.


Network support of virtualized networks will continue to drive data center network investments. One of the greatest challenges for data center network managers is architecting a new network that supports and furthers application availability on virtualized x86 systems. Server virtualization has created a problem for network administrators because virtual machines tend to be managed through server management platforms and tools. This creates a blind spot for network administration teams, making it difficult to effectively troubleshoot and manage problems related to virtual machines. This lack of visibility makes it difficult for network administrators to understand traffic patterns and create policies around virtual machines. Additionally, in order to move to the "next level" of virtualized IT, the network must be an active participant in virtual machine policies and service levels.

Source: IDC, Worldwide Datacenter Network 2010-2015 Forecast and Analysis, Doc #226224, December 2010

This has multiple advantages to the operator of a Data Centre:

  1. Different applications that were once totally disparate can now be hosted on one server (virtualized) with a single common interface, Ethernet.
  2. Data that is encapsulated within Ethernet packets, can be sent anywhere within a network, or to another network that can support it.  This is the perfect scenario for a world that is increasingly looking toward ‘Cloud' based networks to support demand.
  3. The physical infrastructure moves from a number of application specific channels with different connectivity, to a vastly reduced one with common connectivity (either RJ45, LC or SFP+)

It's Ethernet that enables virtualization and for infrastructures to converge into a single "unified fabric" - so called because the closely connected networks nodes resemble a "weave" or "fabric" when viewed from a distance.

Whilst not a new term, fabric or Unified Fabric has become more commonplace with Cisco's promotion of its "unified data center fabrics". The main advantages of these "fabrics" are that they enable vast concurrent processing plus they have very large, tightly-coupled address space. This makes it possible to solve massive computing problems (i.e. delivery of cloud computing services) and they are both scalable and able to be dynamically reconfigured.

With virtualization mainstream in data centers, and with virtual machines moving dynamically across physical devices in the data center, each of the I/O connections to virtualized server or storage devices needs to handle more traffic, in part for data payload, and in part for communications with the other physical devices in the data center around it, part of the same resource pool.  This is driving higher speeds in intra-rack and inter-rack cabling connections.

The common or Unified Fabric therefore needs to be of sufficient capacity to handle this aggregated bandwidth. All data center Unified Fabric is therefore 10GbE.

Unified Fabric Diagram
Unified Fabric Diagram

Because Ethernet is the foundation of the Unified Fabric, Ethernet switches are growing at the expense of fiber channel switches.

According to IDC, Layer 2/3 Ethernet, Layer 4-7 Ethernet switches and InfiniBand switches are seeing or predicted to see strong growth figures whilst fiber channel switches are declining at -2.3% CAGR over the similar time frames.

Top-of-rack architecture (the placing of switches in each rack so that server connectivity can be aggregated and interconnected within a rack) has seen strong adoption over the past several years, with the larger data centers being the early adopters of this architecture.

As part of this architectural change the industry has seen the introduction of a new cable type, Direct Attach Cable assemblies (DAC).  

CommScope has developed these cables to support the roll out of virtualization within the data centre

One of the biggest practical challenges of virtualization for IT departments is keeping track of virtual machines that move dynamically across the physical host devices in a virtualized environment. 

The chart below (Source IDC: 2011) nicely illustrates the projected gap between the physical servers and the logical (virtual) machines running on their servers. 

There is a great need for tools to help IT organizations handle the "Virtualization Management Gap."

Unified Networks
The evolution in application architectures is also changing the traffic patterns in the data center because there is an increasing reliance on the servers, either physical or virtual, to act in a single fabric instead of silos of resources. So instead of traffic flowing from a "single server to the end user" model, you increasingly have traffic that flows between multiple servers and data stores before flowing out to the end user. While organizations can benefit from server consolidation, the bandwidth requirements are higher for each physical server because there is more communication between servers for every application. And possible movement of the virtual machines around the data center makes network traffic patterns more difficult to predict. In the new virtualization data center, a 10GbE fabric is the foundation.

For the network, a migration to converged infrastructure means that the network must converge onto Ethernet with a unified network fabric. A unified fabric provides the ability to set up, move, and change both physical and virtual servers faster to more easily respond to changing business needs and is designed to provide greater scalability for the entire data center. Additionally, it provides benefits in terms of I/O and server virtualization, bringing greater scalability and intelligence into the network and extending benefits to servers, storage and networking.

Source: IDC, Worldwide Datacenter Network 2010-2015 Forecast and Analysis, Doc #226224, December 2010

Virtualization Gap Chart

It is important to note that by communicating with leading software hypervisors (that tell virtual machines where to go), having live real time data representing the physical layer, can help IT organizations visualize the physical connectivity to physical servers and the virtual machines they host. 

This is a very important capability in data centers and also is becoming an increasingly important capability in the LAN, because many IT Managers do not believe that they have the in house skills to effectively manage the migration to a virtualized world. 

This problem will only accelerate as desktop virtualization takes hold in the LAN.


Converged Infrastructures

As organizations move to create a dynamic data center enabled virtualization, they are moving to architectures where server, storage, and network assets are in tighter alignment into converged infrastructures. IDC defines a converged infrastructure as one in which the server, storage, and network infrastructure resources are treated as pools to be assigned as needed to business services. According to research conducted by IDC, the top benefits organizations achieve by implementing a converged infrastructure are cost savings, simplified management, better availability, increased flexibility and higher utilization.

Source: IDC, Worldwide Datacenter Network 2010-2015 Forecast and Analysis, Doc #226224, December 2010

  Virtualization 10 Point Summary
  1. Virtualization is here to stay, and is moving from the data center to the enterprise LAN.
  2. Virtualization is driving servers, switches, and storage to exist on a single Ethernet cabling fabric in the data center.
    a) The early adopters have used pluggable cable assemblies for the device to top-of-rack switch links.
    b) CommScope is a leading innovator of these new pluggable cable assemblies.
  3. Virtualization results in greater infrastructure bandwidth requirements to support the aggregated I/O data streams of the virtualized physical server.
  4. The data center Unified Fabric requires10GbE connectivity as a minimum.
  5. Virtualization has created significant IT management challenges from the sheer number of virtual machines, moving dynamically from device to device in real time.
  6. Having physical layer data that is 100% accurate is imperative to enable successful virtualization.
  7. In general, virtualization enables data centers to consolidate operations, reduce energy consumption and simplifies networks.
  8. Virtualization helps organizations achieve greater R.o.I. for IT investments from increased network resilience and higher device utilization compared to non-virtualized infrastructures
  9. The Unified Fabric architectural change brings new opportunities for rack-to-rack and row-to-row connectivity innovation which CommScope is embracing.
    a) The Unified Fabric is still in the early adopter phase, and is promoted by many as the infrastructure of choice for effective virtualization.
  10. There is a mix of end-of-row, middle-of-row and top-of-rack architectures in today's data centers, and CommScope has comprehensive solutions for both.