SAPonPower

An ongoing discussion about SAP infrastructure

Oracle publishes another SAP benchmark result with limited value

About a month ago, I posted a review of Oracle’s SAP ATO benchmark result and pointed out that ATO is so obscure and has so few results, that other than marketing value, their result was completely irrelevant.  About two weeks later, they published a result on another rarely used SAP benchmark, Sales and Distribution-Parallel.  While there have been a few more publishes on this variety of SD than with the ATO benchmark, the number of publishes prior to this one over the past two years could be counted on one hand, all by Oracle/Sun.

This benchmark is not completely irrelevant, just without context and competitors, it says very little about the performance of systems for SAP.  As the name implies, it requires the use of a parallel database.  While some customers have implemented SAP with a parallel database like Oracle RAC, these customers represent a very small minority, reportedly less than 1% of all SAP customers.  The reason has been discussed in my post on Exadata for SAP, so I will only summarize it here.  SAP is RAC enabled, not RAC aware meaning that tuning and scalability can be a real challenge and not for the faint of heart.  While a good solution for very high availability, the benefit depends on how often you think that you will avoid a 20 minute or so outage.   Some non-IBM customers predict their DB server will only fail every 2 years meaning RAC may help avoid 10 minutes of downtime per year for those customers.  Obviously, if the predicted failure rate is higher or the time for recovery is longer, the benefits of RAC can increase proportionately and if failures occur less often, then the value decreases.

But that is not why this benchmark is of little value.  In reality, the SD benchmark is approximately 1/16 DB workload, the rest being app servers and CI.  To put that in context, for this benchmark, at 1/16 of the total, the DB workload would be approximately 46,265 SAPS.  A 16-core Power 730 has a single system result higher than that as do just about all Westmere-EX systems.  In other words, for scalability purposes, this workload simply does not justify the need for a parallel database.  In addition, the SD benchmark requires that the app servers run on the same OS as the DB server, but since this is SD-Parallel, app servers must run on each node in the cluster.  This turns out to be perfect for benchmark optimization.   Each group of users assigned to an app server is uniquely associated with the DB node on the same server.  The data that they utilize is also loaded into the local memory of that same server and virtually no cross–talk, i.e. remote memory accesses, occurs.  These types of clustered results inevitably show near-linear scalability.  As most people know, near-linear scalability is not really possible within an SMP much less across a cluster.  This means that high apparent scalability in this benchmark is mostly a work of fiction.

Before I am accused of hypocrisy, I should mention that IBM also published results on the SD-parallel benchmark back in early 2008.  Back then, the largest single image SD result achieved 30,000 users @ 152,530 SAPS by HP on the 128 core Superdome of that era.  While a large number, there were customers that already had larger SAP ERP instances than this.  So, when IBM proved that it could achieve a higher number, 37,040 users @ 187,450 SAPS with a 5-node cluster with a total of only 80 cores, this was an interesting proof point especially since we also published a 64-core single image result of 35,400 users @ 177,950 SAPS using the Power 595 within a few days.  In other words, IBM did not try to prove that the only way to achieve high results was using a cluster, but that a cluster could produce comparable results with a few more cores.  In other words, the published result was not a substitute for providing real, substantial results, but in addition to those as a proof of support of Oracle and Oracle RAC.   The last time that Oracle or Sun provided a single image SD result was way back in December, 2009, almost ancient history in the computer world.

This new result, 134,080 users @ 740,250 SAPS on a cluster of 6 Sun Fire x4800 systems, each with 80 Intel Xeon cores is a very high result, but only surpasses the previous high water result on any version of the SD benchmark by 6% while requiring 87.5% more cores.  We can debate whether any customer would be willing to run a 6-node RAC cluster for OLTP.  We can also debate how many customers … in the entire world, have single instance requirements anywhere close to this level.  A typical SAP customer might have 1,000 to 5,000 named users but far fewer concurrent users.  This benchmark does nothing to help inform those customers about the performance they could expect using Oracle systems.

So, this new parallel result neither demonstrates true parallel scalability nor single system scalability or even relevance for small to even very large SAP workloads.  In other words, what value does it provide to the evaluators of technology?   Nothing!  What value does it provide to Oracle?  Plenty!  Not only do they get to beat their chest about another “leadership” result, but they get to imply that customers can actually achieve these sorts of results with this and various other untested and unproven configurations.  More importantly, if customers were actually to buy into RAC as being the right answer for scalability, Oracle would get to harvest untold millions of dollars in license and maintenance revenues.  This configuration included 480 cores meaning customers not utilizing an OEM license through SAP, would have to pay, 480 x .5 (core license factor) x ($47,500 (Oracle license cost) + $22,500 (Oracle RAC license cost)) = $16.8M @ list for the Oracle licenses and another $18.5M for 5 years of maintenance and this is assuming no Oracle tools such as Partitioning, Advanced Compression, Diagnostic Pack, Tuning Pack, Change Management Pack or Active Data Guard.

For comparison, the largest single image system result for IBM Power 795, mentioned above, achieved  just 6% few users with DB2 on a 256 core system.  A 256-core license of DB2 would cost a customer, 256 x (120 PVU) x ($405/PVU) = $12.4M @ list for the DB2 licenses and another $10.9M for 4 years of maintenance (first year of maintenance is included as warranty as opposed to Oracle which charges for the first year of maintenance.)  So, the DB2 license would not be inexpensive, total of $23.3M over 5 years, but that is quite a bit better than the $35.5M for the Oracle licenses mentioned above.

Full results are available at: http://www.sap.com/solutions/benchmark/index.epx

Advertisement

October 4, 2011 Posted by | Uncategorized | , , , , , , | Leave a comment

Oracle Exadata for SAP

On June 10, 2011, Oracle announced that SAP applications had been certified for use with their Exadata Database Machine. I was intrigued as to what this actually meant, what was included, what requirement was this intended to address and what limitations might be imposed by such a system. First the meaning: Did this mean that you could actually run SAP applications on an Exadata system? Absolutely not! Exadata is a database machine. It runs Oracle RAC. Exadata has been on the market for almost 2 years. Oracle 11G RAC has been certified to run SAP databases for well over a year now. Now, there is a formal statement of support for running SAP databases on Exadata. So, the obvious question, at least to me, is why did it take so long? What is fundamentally different about Oracle RAC on Exadata vs. Oracle RAC on any x86 cluster from an SAP perspective? To the best of my knowledge, SAP sees only a RAC cluster, not an Exadata system. I offer no conclusion, just an observation that this “certification” seems to have taken an awfully long time.

What was included? As mentioned before, you can’t run SAP applications on Exadata which means that you must purchase other systems for application servers. Thankfully, you can run the CI on Exadata and can use Oracle Clusterware to protect it. In the FAQ and white papers published by Oracle, there is no mention of OracleVM or any other type of virtualization. While you can run multiple databases on a single Exadata system, all would have to reside in the same set of OS images.  This could involve multiple Oracle instances, whether RAC or not, under one OS, multiple databases under one Oracle instance, or even running different database instances on different nodes, for example.   Many customers chose to have one OS image per database instance to give them the flexibility of upgrading one instance at a time. Apparently, that is not an option when a customer chooses to use Exadata, so if a customer has this requirement, they may need to purchase additional Exadata systems. So, it might seem natural to assume that all of the software required or recommended to support this environment would be included in the SAP OEM edition of Oracle, but that would be wrong. Since Exadata is based on Oracle RAC, the RAC license must be obtained either through an additional cost for the OEM license from SAP or directly through Oracle. Active DataGuard and Real Application Testing, optional components but considered by many to be important when RAC is utilized, are also not included and must be purchased separately. Lastly, Oracle Exadata Storage Server must be purchased separately.

So, what problem is this intended to solve? Scalability? IBM’s System X, as well as several other x86 vendors, have published SAP 2-tier benchmarks in excess of 100,000 SAPS not to mention over 680,000 for IBM’s Power Systems. Using the typical 1:4 ratio of database to application server SAPS, this means that you could support an SAP requirement of at least 500,000 SAPS with a single high end x86 database server. Perhaps 1% or less of all SAP implementations need more capacity than this, so this can’t be the requirement for Exadata. How about high availability? Oracle RAC was already available on a variety of systems, so this is not unique to Exadata. A primary requirement of HA is to physically separate systems but Exadata places all of the nodes in a single rack, unless you get up to really huge multi-rack configurations, so using Exadata would go contrary to HA best practices.
Let us not forget about limitations. Already mentioned is the lack of virtualization. This can be a major issue for customers with more than one database. But what about non-production? For a customer that requires a database server for each of development, test and QA, not to mention pre-prod, post-prod or any of a variety of other purposes, this could drive the need for multiple other Exadata systems and each would have a radically more capacity than a customer could reasonably be expected to utilize. What is a customer has an existing storage standard? Exadata supports only its own storage, so must be managed separately. Many customers utilize a static and/or space efficient image of the database for backup purposes but that requires a storage system that supports such a capability not to mention the ability to mount that image on a separate server, both of which are not possible with Exadata.  A workaround might involve the use of Active Data Guard to create a synchronous copy on another system which can be utilized for backup purposes, but not only does this come with additional cost, but is more complex, is not space efficient and might require additional systems capacity.  And then there are the known limitations of RAC for SAP.  While SAP is RAC enabled, it is not RAC aware. In other words, each SAP application server must be uniquely associated with a single RAC node and all traffic directed to that node. If data is located in the memory of another RAC node, that data must be moved, through cache fusion, to the requesting node, at a speed of over 100,000 times slower than the speed of moving data through main memory. This is but one of the many tuning issues related to RAC but not intended as a knock to RAC, just a reality check. For customers that require the highest possible Oracle database availability, RAC is the best choice, but it comes at the cost of tuning and other limitations.

I am sure that I must be missing something, but I can’t figure out why any customer would need an Exadata system for SAP.

July 27, 2011 Posted by | Uncategorized | , , , , , , , , , , , , , , , , , , , , | Leave a comment