We know, it is not evident to think of SAP as a platform built for agility, historically that has never been the case. But with SAP HANA the Business Intelligence game has changed. Dramatically.
For those who are not that familiar with SAP HANA as a database platform, allow us to put the birth of SAP HANA in a historic perspective. Some say that HANA stands for Hasso Platner’s Amazing New Architecture. Others say it is the name of the bay where Hasso was sailing when he got the inspiring idea that led to this new platform. What is really interesting is how HANA came about and how it solves one of the most challenging problems in database platform history.
The story with databases
Performance issues have always been at the core of database design. The bottleneck in database performance have always been the hard disks, slow to read data from, slow to write data to. Structuring the data in tables with the size of almost infinite excels – thousands of columns wide and millions of record lines deep – makes that performance has always been critical. Especially when thousands of processes and people are reading and writing to the database simultaneously and in real-time. To keep ERP and other enterprise systems performant has always been a daunting task, resulting in an ever-growing data model and a myriad of tables, especially for the smart way ERP originated at SAP.
Simply put, an ERP system like SAP captures all data in a business process along a chain of documents and uses the relationship between these documents to organise all data entered in a database.
Quote->Order->Purchase Order->Delivery->Invoice to give a simple example.
Every business activity has its own document that collects the data that you need for that step in the process, the very same data can then be constantly reused, calculated and updated in follow-on processes and documents, without having to re-enter the data at every next stage in the business process.
To keep such a system performing, a lot of extra table structures and tables are needed to aggregate data and calculate values in between the numerous transactions within modules and between applications.
Coming back to the initial problem of reading and writing, for decades, we have only been able to build a database that was good at one of these actions, but never both. Hence an OLTP-database for your ERP system and an OLAP-database for your Business Intelligence and reporting systems.
True SAP experts will thus combine a thorough knowledge of business processes and “their” SAP modules with a deep understanding of the SAP database structures underneath. On the analytical BI-side, this explains why SAPs Business Warehouse (BW) is as complex a database as the ERP OLTP database, as almost only SAP is fully capable of extracting and transferring data from the operations database(s) into the analytical database SAP BW runs on. Experts that combine ERP and BI knowledge are the most sought after, evidently.
Embracing technology
About a decade ago, one of the founding fathers of SAP had an idea.
Database hosting in data centres became mature in virtualising database servers and running complete databases in live memory, like the DRAM memory modules in your laptop or PC, offering promising speed improvements for database performance. Hasso Platner predicted that quite soon, live memory modules would become cheaper and would be a commercially viable alternative for the hard disks that troubled the industry since the seventies.
Next to the classic databases used in the ERP world, new database types had emerged to handle sheer impossible volumes of data, essential to run platforms like Facebook, Twitter and Amazon or data lakes to handle colossal amounts of data streaming in from sources on the Internet of Things.
Hence the HANA mission: High time to invest in finding a radical new way to combine these technological revolutions into a durable platform to host all things SAP on, for the future decades to come. And with HANA, that is exactly what Hasso’s dream team came up with.
HANA effectively combines the best of these new worlds into one platform.
SAP’s Henry Cook explains this journey extensively in this video: “HANA, the why”. For more details on the how, I recommend Werner Daehn’s article “The secret of SAP HANA”, from which I copy the image below, to explain what makes HANA a unique combination.
- Running the database in-memory allows for incredible performance to read and write to and does away with the hard disk performance issues of the past
- Columnar storage is the new technology to no longer organise your data in classic, excel-like row-store structures – these take ages to plough through in case of millions of lines and need index tables to keep search queries performant. Instead, data are stored in columns, allowing incredibly fast querying and looking up of your data.
- Compression allows to keep the data footprint in the expensive live memory affordable
- Insert-Only is the optimal way to also be able to write data fast into your columnar store database without losing read performance. You keep a history of the changes when inserting new data, that in turn gets merged periodically to effectively update and (re)sort your data per column. Possibly the smartest idea in the industry to date.
Allow all this to sink in a bit and you will understand that – for the first time in history – SAP no longer needs an OLAP database. No separate datastore to serve the business side’s hunger for real-time analytics of operational data to steer and navigate their business on.
HANA can write all its transactional data into the database in real time, from almost any number of sources (ERP, CRM, IoT, …) It also does away with all complexity of its former table structure, as SAP’s Chris Hallenbeck and Erich Schneider convincingly explain on a white board in this video.
Running ERP, CRM and all other SAP modules as before, the operational database gets exponentially smaller, exponentially faster and the structure of the data becomes exponentially more transparent to analyse.
The greatest news from an analytical point of view is that the data can be queried directly from the operational database. We no longer need a parallel database to copy operational data to at regular intervals into tables structures that are optimised for analytic purposes. No snapshots anymore, for the first time in history, SAP operational data can be analysed real time.
The mind-shift in modelling
Evidently, the data structure of an SAP ERP system is not optimally organised for advanced analytics. The beauty of SAP HANA is that we can create optimized data structures for analytic purposes in a virtual way. Just like we used to do on the separate Business Warehouse data warehouses, but without having to copy the data over into an underlying database and make calculations in added tables, the so-called staging. Instead, we create a logical data structure in virtual tables to show the information and perform the calculations in views that the business needs and read the data that we need in real time from the applications database, combining data from any system, ERP, CRM, … both SAP and non-SAP sources.
Before HANA, we needed to build and organize physical datastores to build our analytics on and fill them with imported data at regular intervals and stage the data for optimal reporting, in order to realize the business requirements. This staging complexity entailed an inevitable waterfall project delivery method, taking months to realize before we could present our results to the business side, evaluate and rework if needed. How many projects failed to produce the business demand in time?
With HANA, we are now able to satisfy business demands in a very flexible way. For the first time, agile project management concepts in data analysis projects are possible.
Even the combination of Design Thinking, Leam Startup and Agile, as Gartner described in 2016, has become a reality in Business Intelligence, allowing analytics to steer business operations in real time.
The result-driven Business Intelligence diehards that we are, ONE Consultants embraced this opportunity from day one and forced a mind-shift with its customers. “Forgetting” decades of SAP BW waterfall methodology, we find a balance between the expensive pyramids -staging all data and replicate data the old BW way – and going virtual all the way.
We address every business requirement with a fresh mindset, using HANA’s native capabilities to answer business intelligence challenges real-time with agile BI projects in a scrum methodology.
Contact us if you want us to help you become agile at HANA!
[[form id=’7262′]]