An ongoing discussion about SAP infrastructure

IBM PureSystems for SAP

On April 11, 2012, IBM announced a new family of converged architecture systems called PureSystems.  IBM does not use the term “converged architecture” in the materials released with this announcement, preferring the term “Expert Integrated Systems” due to the fact that it goes well beyond the traditional definition of converged infrastructure.   Other companies have offered converged infrastructure in one form or another for a few years now.  HP introduced this concept several years ago, but in my countless meetings with customers, I have yet to hear a single customer mention it.  They talk about HP blades frequently, but nothing about the converged solution.  Cisco UCS, on the other hand, is much more often mentioned in this context.  While Oracle might try to suggest that they offer a set of converged infrastructure solutions, I believe that would be a stretch as each of the Exa offerings stand on their own, each with their own management, storage and network framework.  The Exa solutions might better be described as special purpose appliances with a converged hardware/software stack.  Dell’s converged solution is basically a management layer on top of their existing systems.  This would be like IBM trying to suggest that IBM Systems Director is a converged infrastructure, which has never been the case as this would imply that IBM is ignorant.


IBM learned from the missteps and mistakes of our competitors and designed a solution that takes a leadership position.  Let’s take a short journey through this new set of offerings during which I will attempt to illustrate how it is superior to competitive offerings.  A more comprehensive look at PureSystems can be found at:


Converged infrastructures generally include servers, storage, networking, virtualization and management.  Efficient utilization of resources is at the cornerstone of the value proposition in that businesses can deliver significantly more application environments with fewer personnel, lower hardware costs, greater datacenter density and lower environmental costs.  Whether and how well companies deliver on these promises is where the story gets interesting.


Issues #1: open vs. closed.  Some, such as Oracle’s Exa systems, are so closed that existing storage, servers, virtualization or software that you may have from a company other than Oracle, with rare exceptions, cannot be part of an Exa system.  Others suggest openness, but rapidly become more closed as you take a closer look.  Cisco UCS is open as long as you only want x86 systems, networking and SAN switches only from Cisco and virtualization only from VMware or Microsoft HyperV.  VCE takes UCS further and limits choices by including only EMC V-Max or Clarion CS4-480 and VMware.  By comparison, PureSystems are built on openness starting with the choice of nodes, x86 and Power, and OSs, Microsoft Windows, RedHat and SUSE Linux, AIX and IBM i.  Supported virtualization offerings include VMware, HyperV, KVM and PowerVM.  Storage can be almost anything that is supported by the IBM V7000 which includes most EMC, HDS, HP, Netapp and, of course, all IBM storage subsystems.  Networking is built into each PureSystems chassis but supports network adapters from Broadcom, Emulex and Mellanox and Fibre Channel adapters from Qlogic, Emulex and Brocade plus both QDR and FDR Infiniband adapters.  Top of rack (TOR) switching can be provided by just about any network technology of your choosing.  Management of the nodes, networking, storage and chassis is provided by IBM, but is designed to be compatible with IBM Systems Director, Tivoli and a variety of upstream managers.


Issue #2: management interface.  IBM spent a great many person years developing a consistent, intuitive and integrated management environment for PureSystems.  Among a wide variety of cross system management features, this new interface provides a global search feature allow an administrator to quickly identify where a virtual resource is located in the physical world.  Ask any administrator and you will find this is a lot more difficult than it sounds.    Cisco does a great job of demonstrating UCS based on an impressive level of prep work.  They show how easily images can be cloned and provisioned and this is indeed a significant accomplishment.  The problem is that a significant amount of prep work is required.  Likewise, when changes occur in the underlying environment, e.g. a new storage subsystem is attached or expanded, or a new network environment is added, a different set of management tools must be utilized, each with their own interface and some less intuitive than others. VCE offers a more consistent and intuitive interface, but at the cost of a very rigid set of components and software.  For instance, “Vblocks”, the term for VCE systems, must be implemented in large chunks, not granularly based on customer demands, must be “approved” for SW or firmware updates by VCE, even emergency fixes for known security issues, and do not allow any sort of outside components at all.


Issue #3: the network is the computer.  This is a bit tongue in cheek as that was the slogan of the old Sun company, anyone remember them?  Cisco’s architecture seems to be an echo of this old and outdated concept.  PureSystems, as noted above, provides an integrated network but allows a wide variety of adapters and upstream devices.  By choice, customers can directly integrate multiple chassis together without the need for a top of rack switch until and whenever they want to communicate with external networks.  For instance, should an SAP application server have to use a TOR switch to talk to a SAP DB server?  Should a DB2 PureScale cluster have to use a TOR switch to talk among its nodes and to a central lock manager?  Should an Oracle RAC cluster have to incur additional latency when communicating with its distributed lock manager?  IBM believes the answer to all of the above is that it is up to the customer.  If you want to use a TOR switch, you should, but that should be your choice, not a mandate.  After all, IBM’s goal is to provide an excellent computing infrastructure, not sell switches.  By comparison, Cisco’s architecture is dependent on very expensive 6100 and similar interconnects.  In fact, Cisco even suggests that customers utilize VM-FEX technology as they claim that it greatly simplifies network management.  What some customers may not realize is that to utilize this technology, you must disable the virtual switch used by VMware.  This switch allows different VMs on a single system to communicate at near memory speeds.  Using VM-FEX, this switch is disabled and communication, even between adjacent VMs, must communicate via TOR switches and instead of interconnect latencies measured in 100s of nanoseconds, those latencies can take several orders of magnitude greater time.


For SAP, it is reasonable to ask whether a converged infrastructure solution is required.  Clearly, the answer is no, as customers have been successfully implementing SAP on everything from single, massive, 2-tier virtualized servers to large arrays of 3-tier, small, non-virtualized systems and everything in between for many years now.  There is nothing on the SAP roadmap that specifies or certifies such technology.  But, is there value in such a solution.  The answer, obviously, is yes.


While consolidation of many different SAP instances on large, 2-tier virtualized systems offers tremendous value to customers, there are a variety of reasons why customers chose to utilize a landscape with multiple, smaller servers.  Cost of acquisition is usually the biggest factor and is almost always less when small servers are utilized.  The desire to not have all eggs in one basket is another.  Some customers prefer to keep production and non-production separate.  Yet others are uncomfortable with the use of virtualization for some systems, e.g. Oracle database systems under VMware.  This is not intended to be a comprehensive list as there may be many other factors that influence the use of such an architecture.


If multiple systems are utilized, it is very easy to get into a situation in which their utilization is low, the number of systems is multiplying like rabbits, the cost of management is high, flexibility is low, the space required is ever increasing and power/cooling is a growing concern.  In this situation, a suitably flexible converged infrastructure solution may be the optimal solution to these problems.


PureSystems may be the best solution for many SAP customers.  For existing Power Systems customers, it allows for a very smooth and completely binary compatible path to move into a converged architecture.  Both 2 socket and 4 socket Power nodes are available, the p260 and p460.  Pure Systems feature an improved airflow design and a higher power capacity than IBM’s BladeCenter, which therefore allows for nodes that can be outfitted with the latest processors running at their nominal frequency and with full memory complements.  As a result, these new nodes feature performance that is very close to the standalone Power 740 and 750 systems respectively.  With a very fast 10Gb/sec Ethernet backbone, these new nodes are ideal for virtualized application and non-production servers.  The p460 offers additional redundancy and support of dual VIO servers which makes it an excellent platform for all types of servers including database.   One, or many, of these nodes can be used for part of an SAP landscape featuring existing rack mount servers and BladeCenter blades.  Live Partition Mobility is supported between any of these nodes assuming compatible management devices, e.g. HMC, SDMC or PureFlex Manager.


Entire SAP landscapes can be hosted completely within one or more Pure Systems chassis.  Not only would such a configuration result in the most space efficient layout, but it would provide for optimized management and the fastest, lowest latency possible connections between app and DB servers and between various SAP components.


Some SAP customers feel that a hybrid approach, e.g. using Power Systems for database and x86 systems for application servers, is the right choice for them.  Once again, PureSystems delivers.  Power and x86 nodes may coexist in the same chassis, using the same management environment, the same V7000 virtualized data storage and, of course, the same network environment.  Clearly, the OSs, virtualization stacks and system characteristics are dependent on the underlying type of node, but regardless, they are all designed to work seamlessly together.


Yet other customers may prefer a 100% x86 landscape.  This is also completely supported and offers similar benefits as the 100% Power environment described above, with the inherent advantages or disadvantages of each respective platform characteristics, which has been discussed at some length in my other blog postings.


There are many good blogs that have discussed PureSystems.  Here are but a few that you may wish to check out:



May 8, 2012 - Posted by | Uncategorized | , , , , , , , , , , , , , , , ,


  1. […] the original article here: IBM PureSystems for SAP « SAPonPower Share this post to your […]

    Pingback by IBM PureSystems for SAP « SAPonPower | IBM Desktop Reviews | May 8, 2012 | Reply

  2. Who in their right mind would put all their SAP or Oracle RAC VMs on the same esx blade to avoid less than a microsecond of latency on inter-vm packets?

    In a large infrastructure the vms would be spread across blades and chassis for load balancing and you need them spread across blades for availability reasons!

    Is it open? Can I get a Sun SPARC blade for Pure? An itanium blade? a pa-RISC blade? On release will any blade other than the two high end intel blades be shipping? Is the blade design being licensed for others? In other words you have architecture two options – intel or IBM propriatory Power. If you don’t need a Power system, then Pure isn’t any more open than any others (switching excluded).

    One of UCS’s great strengths is the management of the chassis switching within its top of rack switching. Being able to deploy a chassis without even having to think about presenting 10s or 100s of VLANs is compelling..

    It is a nice idea to allow V7000 subsystems to be integrated within the chassis, leaving 10 blade slots, with only a single tray of disks. With a 10 to 1 consolidation ratio that gives an average of about 1/4 of a disk per VM in average read I/O bandwidth or 1/8th of a spindle of capacity per vm (if you use Raid 1).

    If you will end up adding external disk trays, why not do this from the start and just add extra shelves as you storage needs grow?

    I feel that Pure has it’s niche where it is perfect, but it doesn’t seem to be as revolutionary as IBM would like me to think.

    Comment by Michael Field | May 14, 2012 | Reply

    • Michael raises some interesting points. At the same time, he has misinterpreted my remarks. I said “Should an Oracle RAC cluster have to incur additional latency when communicating with its distributed lock manager?” I did not suggest that RAC nodes be placed within virtual machines within a single blade. That, I agree, would be rather misguided. RAC nodes are often placed in different blades in a blade enclosure and latency is absolutely critical meaning that a switch infrastructure that takes a millisecond or two will result in a faster RAC than one that uses only a top of rack switch with latencies twice to four times as large. Also, to suggest that partitions will not benefit from communications with one another at near memory speeds but instead should run at latencies that are roughly 1,000 times slower is equally misguided. Application servers are often placed in separate partitions from DB servers and can certainly benefit from this sort of high speed communications. The same goes for PI and its communications among different SAP components.

      Is it open? Is any system completely open? VMware is most certainly not open as it is provided by a single company. Not a knock there, just a statement of fact, the statement would apply to AIX or HPUX. But it does follow certain standards which are published and for which there are a great many third parties involved in their ecosystem. There are different degrees of openness. A system, such as PureSystems which includes various hypervisors, various chip architectures, various OSs and various network technologies is certainly more open than one that has only selected x86 hypervisors, one chip architecture, only supported x86 OSs and only one supported network technology.

      Regarding SPARC, Itanium and PA-RISC blades, I did not realize there was such a great demand for those types of blades. Perhaps Cisco, if they think there is such great market demand for these will “open” their architecture to encompass them. Or maybe not as Cisco is not, by Michael’s definition, open. And to suggest that IBM support, “on release” all available possible combinations of Intel technology is simply not even a logical argument.

      Yes, one of UCS’s strengths but also one of its significant weaknesses is the reliance on the top of rack switching. Every system involves tradeoffs, and in the case of UCS, that tradeoff is less complex management with a tradeoff of a very expensive and high latency switch environment. Fair enough.

      Wow, on the comment about the V7000. I would suggest that reading even an executive summary on IBM’s web site might be slightly more informative than whatever inaccurate source Michael uses. The internal disks in a V7000 represent only one layer, and often the fastest layer, e.g. SDDs, of a multi-tier hierarchy of disk drawers and subsystems which can include IBM, EMC, Hitachi, HP, Oracle/Sun, NetApp, NEC and about a dozen or so other disk subsystems while presenting a single cohesive view to the supported OSs both within PureSystems and without. Currently, the V7000 is a separate device, which can be managed by PureSystems but does not take up any slots in a chassis. In the future, customers will have the option, but not the requirement, of adding an internal V7000.

      I don’t expect anyone to think this is revolutionary, but, in my opinion, that this simply converged infrastructure done right.

      Comment by Alfred Freudenberger | May 14, 2012 | Reply

      • HI again -thanks for posting up the comment – should not drink so much coffee in the afternoons!

        I am sorry to disappoint you, but don’t work for Cisco, I work for an Independent system integrator in the corner of the Asia Pacific. I work on blade environments from all the usual vendors (except Dell), and worked with pretty much all the usual storage subsystems you can think of (including IBM’s SVC, XIV and V7000 products). I personally support infrastructure for SAP, Oracle eBusiness Suite, BAAN and other packages.

        [Note] In your reply you talk of latency a a millisecond or two vs one that takes 2x or 4x as long, but in the original post you talk of latencies in “100s of nanoseconds” – that is around 10,000x different – light can travel 186 miles in a millisecond! Obviously that is going to hurt RAC performance, but it isn’t the level of latency being talked about here. You must have been meaning microseconds…. [/note]
        But back to “a few 100ns”.

        This “100ns” unit of time is far smaller than the 10 or 20 microseconds that it takes a CPU to perform a context switch from user space to kernel space ( – which also indicates that ESX increases the cost of a context switch to something like 30 to 30 microseconds).

        In fact 100ns is the time it takes to send a approx 100 of bytes at 10Gb/s, or maybe 10 bytes at Gigabit speeds. A few 100ns is a good approximation for the time it would take for the TCP/IP header to pass through one or two cut-through switches, so I believe that grounded in reality.

        I’ve just had a look at, and the end-to-end latency (including a switch) for GigE is 29us – 100us, for 10GigE it is around 12.5us (and for Infiniband it is around 1.7us).. I strongly doubt that anybody out there seen a great jump in performance when a 1Gb/s cluster interconnect configuration was upgraded to 10Gb/s, even though performing such an upgrade could remove something like 16,500ns of latency.

        You say “to suggest that partitions will not benefit from communications with one another at near memory speeds but instead should run at latencies that are roughly 1,000,000 times slower” – where did your get that 1,000,000 slower figure? If your take only the CAS latency on DDR3-2133 memory is up to 13ns (see A million times slower than 13ns is 13 milliseconds. (or about 1/76 of a second). Sorry, numbers don’t stack up. I could believe the number if you used “1000” (as 13us is a sensible figure for latency on a Gigabit network – see Fig 2 in

        But even if this in-memory switching does give an impressive a 99.9% reduction in latency the obvious pitfall required to get of this source and target VMs must be on the same physical node – more than likely defeating the environment’s availability requirements.

        However, given that Infiniband shaves over 95% off off network latency over GigE, and over (80% off of 10GigE), then why isn’t this the ‘must have’ for all cluster interconnects? The only answer I can give is this “cutting latency isn’t that big a performance boost that the average enterprise cares enough to spend real money on it”.

        I was just “poking a stick”, but I must admit that there isn’t a great demand for RISC blades of any sort in most business environments – but for you to say “we can put an IBM Power Blade, therefore we are much more open” is drawing an very long bow, especially give that despite the picture you seem to paint Cisco UCS it is not limited to just ESX – as well as ESX we have a few HyperV and Oracle VM hypervisors running on some UCS kit somewhere,

        And I feel that my point of storage is still valid. An internal v7000 is not likely to meet all the capacity and performance needs of a fully populated chassis, so in a lot of cases (esp in the Enterprise environment) it will need additional external disk shelves (either behind the v7000, or stand alone). This negates the idea of it being a nice self contained all in one solution. It is a neat feature, but more a niche feature than a compelling one.

        But in the end, number talk – are you able to supply any benchmark figures? Can you give *ME* any sort of idea of what real world performance difference it makes? I want to believe, but I read lots of words everywhere that just don’t make sense!

        Look forward to your reply.

        Comment by Michael Field | May 14, 2012

      • Rather than a deep technical discussion of the latencies of different types of communication, let me boil this down to some simple facts as I don’t have the technical background or desire to debate the numbers that you provided. My points are regarding business value.

        VM to VM communication using a TOR switch is orders of magnitude slower than within a system regardless of context switches.

        Blade to blade communication with a 1 or 2 microsecond latency is better than 4 microseconds.

        Most computer experts agree that faster is better. Many, but not all, applications benefit from faster communications. I am sure, if I did not an exhaustive search of benchmarks, which I am not going to do, I could find examples where internal communications resulted in faster results than external, but this would be pointless.

        You are correct, I did use 1,000,000 when perhaps a smaller number would have been more accurate, so I edited my first reply to you to reflect this, but the point is simply that memory speeds are much faster than Ethernet or Infiniband.

        As to the V7000, I did not make the point regarding placing everything within the enclosure, so your comments are not relevant to the points that I made. The consolidated management, as provided by the Flex Systems manager with the V7000, of many different types of storage with a unified and seamless view to many different systems and partitions is of tremendous business value. That is the point that I am trying to get across. This is also something that, for which, there is no equivalent in the UCS model even when FlexPod or Vblock is considered.

        Comment by Alfred Freudenberger | May 15, 2012

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: