An A-to-Z Guide to Implementing SAP HANA: Planning, Scoping, Staffing, Budgeting, and Execution

An A-to-Z Guide to Implementing SAP HANA: Planning, Scoping, Staffing, Budgeting, and Execution
Dr. Berg Comerit

In Part 2 of The Session Examine modeling options in SAP HANA Studio and see how views and tables can be created Step through a demo on how to use SAP Data Services for loading data, building views, and accessing them from SAP BusinessObjects Explorer Learn the various steps and options for migrating BW to HANA or building your own EDW on HANA Take away a solid understanding of how HANA can be implemented in your organization

What We’ll Cover Background The standalone HANA project: Creating views and tables SAP NetWeaver BW 7.4 on HANA – New features Preparing for the BW upgrade and HANA migration Cleaning up existing BW system and reducing HANA size BW transformation and HANA optimization Demo of non-BW HANA development The Direct Migration Option (DMO) for BW Project execution examples Wrap-up

Background SAP HANA is fundamentally a database and not “magic” However, it provides major benefits in terms of: Landscape simplification Speed of queries and data access Simplification of environment management Long-term lower Total Cost of Ownership (really!) An enabler of the next generation of ERP and data warehouses for real-time access, model simplification, agility, and support for Big Data volumes HANA projects should think big and act small. After the installation project is ended, what do you plan on deploying in terms of new capabilities?

Inside SAP HANA — In-Memory Computing Engine(ICME) Persistence Layer Disk Storage Data Volumes Page Mgmt. Session Manager Load Controller Replication Server Metadata Manager Relational Engine -Row Store -Column Store Logger Log Volumes MDX SQL Parser Authorization Manager SQL Script Transaction Manager Calculation Engine Inside the Computing Engine of SAP HANA, we have many different components that manage the access and storage of the data. This includes MDX and SQL access, as well as Load Controller (LC) and the Replication Server. BusinessObjects Data Services

Tables Inside HANA Viewed Using HANA Studio We can view table definitions and data inside HANA Studio. This includes SAP BW tables, ERP tables, and tables we created ourselves.

Creating an Attribute View An attribute view is normally used to expose master data to the front-end tools, such as those found in the SAP BusinessObjects tool suite They can also be accessed by any ODBC/JDBC compliant third-party tool A view is a logical way to simplify the complexity of a database for users who need access

Adding Table Joins to More Customer Information In this example we are adding more information, such as country and language, to the view. This data was stored in other tables, but we want to make it easy for our users to access without database knowledge.

Applying a Filter to an Attribute View Sometimes we don’t want to give users access to all the information, and we can “hard code” a filter in our view In this example, we are limiting the view only to valid customers

Deciding What Users Should See and Assigning Key Attribute Sometimes, we don’t want to give access to everything available in the tables. We can then decide what fields are available to the users (these are flagged with orange balls)

Validate and Activate a View HANA also provides a validation of each view to make sure it is correctly defined and meets all standards before it is activated Once activated, you can expose it to your target users via standard HANA security

Creating an Analytical View An analytical view is a way to join master data and transactional data together for simple access

Data Foundations and Logical Joins The resulting models in the view can be further refined by controlling what is exposed to the end users We call the transaction-oriented data the “data foundation” and can link other attribute views to this

Adding a Calculated Field to an Analytical View We can also create our own calculations in the view For example, we are now creating a new field called “net revenue” This we have defined as “revenue” minimum “discount” Complex math functions can be called and custom coding can also be added

Multi-Currency Handling If we have loaded other currencies to the system, we can also add these capabilities to the view I.e., in the advanced section of the calculated column, we can flag the field as using a source and a target currency and let the system take care of the translation

The Resulting Analytical View Views are found in the navigator section of the modeler inside HANA Studio. You can preview the results and also view metadata about the view and its definition.

Deploying Analytical View in BO Explorer The SAP BusinessObjects tools can easily access the views inside HANA Here we are building an Information Space in SAP BusinessObjects Explorer that is accessing the view we just created

Accessing the View Through an Information Space All the fields we exposed and created in our analytical view are now displayed here for easy access by the SAP BusinessObjects Explorer tool We don’t have to do any more development work!

The Result in SAP BusinessObjects Explorer Users can now navigate the HANA views at sub-second speed without having to develop anything further More complex analysis can also be added via other SAP BusinessObjects tools

HANA Development Demo Step-by-Step In this demo, we will look at our example in more detail and also see how to load data to our HANA system using SAP Data Services

SAP HANA — Some of the Front-End Connections All SAP BusinessObjects tools can be connected to SAP HANA. You can even access HANA using Microsoft Query and other tools.

New Business Content Optimized for BW on HANA SAP has developed specific, new content for BW on HANA This takes advantage of HANA optimized transformations, consolidated InfoObjects, more line item details, and in some cases leverages the BW models and the HANA models in combined scenarios For those “green fielding” a HANA implementation, or planning to develop analytics in these areas, serious consideration should be spent on leveraging this new content

SAP HANA and BW 7.4 BW 7.4 on HANA introduces the Smart Data Access that allows us to create: Virtual table (real-time) links to a source system CompositeProviders to link BW DSOs and “native” HANA views and tables Create a flexible logical EDW that can be remodeled without moving data We also get a shared modeling tool based on Eclipse Using the new Operational Data Provisioning (ODP) we can also remove the need for PSAs, thereby simplifying data architectures and reduce data latency

SAP HANA and BW 7.4 — Open ODS View For BW 7.4, we get a new object called an Open ODS View This new object allows us to include external data models in BW and to query BW and non-BW data in a single BEx query In the past, BW has been criticized for the complexity to integrate non-BW data in an EDW. The new Open ODS view and the new CompositeProvider solves most of these issues.

Cool New Feature: Automatically generate HANA Models from BW You can now automatically create new HANA models based on InfoCubes and DSOs in BW The generated HANA views access the data in the InfoProviders All HANA View capabilities are available for the BW data BW Security in leveraged, and views can be accessed by tools such as SAP BusinessObjects Explorer, SAP Lumira, and other SAP BusinessObjects tools In addition to the BW model, import in the HANA modeler, this functionality leverages the existing metadata in BW to create new views

Some of the features in BW 7.4 are HANA specific Others are available for traditional RDBMs This is an overview of what is available for HANA and non-HANA BWs

For more on this and an in-depth view of the new BW 7.4 features, join me at:“Deep dive into leveraging new features and functions of SAP NetWeaver® BW 7.4” Tuesday, March 25, 2014, 4:45 pm - 6:00 pm

The SAP_BW_HOUSEKEEPING Task List If you are on 7.0 SP32 of higher, you can generate an SAP BW Housekeeping task list and get automated help in cleaning the system weeks before upgrading it Checks BW metadata with DDIC Delete RSTT traces Delete BW statistical data Delete Aggregate data via deactivation Ensure partitioned tables are correctly indexed for PSA Ensure request consistencies in the PSA Re-assign requests written into the incorrect PSA partition Verify DataSource segments assignment to PSA Deletes the entries no longer required in table RSIXW Clear all OLAP Cache parameters Repair InfoCube fact table indices at Data Dictionary level Reorganize and delete bookmark IDs & view IDs You first have to install the program from SAP Note 1829728 before you can generate the SAP_BW_HOUSEKEEPING task list using tcodeSTC01

The SAP_BW_BEFORE_UPGRADE Task List If you are on 7.0 SP31 or higher, you can generate a before the upgrade tasks list and get help in preparing the system for the upgrade The more of these tasks you complete, the faster the upgrade may proceed since you are reducing size and complexity, while assuring that your system is ready for the 7.4 upgrade You first have to install the program from SAP Note 1734333 before you can generate the SAP_BW_BEFORE_UPGRADE task list using tcodeSTC01

Pre-Steps — Cleaning up Your BW System You can save significant amounts of work by doing a cleanup effort before you start your HANA migration or BW upgrade project For example, an international company had a BW system with over 108 TB, with only 36 TB in the production box and the remaining data on their Near-Line Storage (NLS) solution This cleaned BW system saved them potentially millions of dollars in hardware and HANA licensing costs It is not unusual to reduce a BW system size by 20-30% during a clean up effort

12 Pre-Steps — Cleaning up Your BW System Clean the Persistent Staging Area (PSA) for data already loaded to DSOs. Delete the Aggregates (summary tables). They will not be needed again. Compress the E and F tables in all InfoCubes. This will make InfoCubes much smaller. Remove data from the statistical cubes (they start with the technical name of 0CTC_xxx). These contain performance information for the BW system running on the relational database. You can do this using the transaction RSDDSTAT or the program RSDDSTAT_DATA_DELETE to help you. Look at the log files, bookmarks, and unused BEx queries and templates (transaction RSZDELETE). Remove as much as possible of the DTP temporary storage, DTP error logs, and temporary database objects. Help and programs to do this are found in SAP Notes 1139396 and 1106393.

12 Pre-Steps — Cleaning up Your BW System (cont.) For write-optimized DSOs that push data to reportable DSOs (LSA approach), remove data in the write- optimized DSOs. It is already available in higher level objects. Migrate old data to Near-Line Storage (NLS) on a smallserver. This will still provide access to the data for the few users who infrequently need to see this old data. You will also be able to query it when BW is on HANA, but it does not need to be in-memory. Remove data in unused DSOs, InfoCubes, and files used for staging in the BW system. This includes possible reorganization of master data text and attributes using process type in RSPC.

12 Pre-Steps — Cleaning up Your BW System (cont.) You may also want to clean up background information stored in the table RSBATCHDATA. This table can get very big if not managed. You should also consider archiving any IDocsand clean the tRFCqueues. All of this will reduce the size of the HANA system and help you fit the system tables on the master node. In SAP Note 706478, SAP provides some ideas on how to keep the Basis tables from growing too fast in the future; if you are on Service Pack 23 on BW 7.0 or higher, you can also delete unwanted master data directly (see SAP Note: 1370848). Finally, you can use the program RSDDCVER_DIM_UNUSED to delete any unused dimension entries in your InfoCubes to reduce the overall system size.

A Tool to Help to Migrate and Clean Up SAP has created a cockpit to: Clean up the SAP BW system Reduce system size Conduct pre-checks (readiness checks) Size the system Find sub-optimal code (i.e., transformations) Look at table distributions and loads There are over 235 tests in this tool as of version 3.0 in March 2014 These tools are thanks to SAP’s Marc Bernard and his team at SAP Labs Canada

Tips to Make the Database Smaller Use write-optimized DSOs as first level data stores. These caneasily be off-loaded out of main memory in HANA and save you money. Keep your Persistent Staging Tables (PSA) clean. BTW: The PSA is often not needed at all in BW 7.4. If you are on BW 7.3 Service Pack 8 and HANA with at least Service Pack 5, the write-optimized DSOs and PSAs are flagged as “early unload” from the HANA memory. This will help you keep the system smaller and require less memory. You can also flag other InfoCubes, DSOs, tables, and partitions as “not active”. If you do so, they will only be loaded into memory when actually required. The sizing program in SAP Note 1736976 takes these size savings settings into account when sizing your HANA system

Clean up of SAP BW Before Migration — Tool Support Areas of database size reduction Delete application logs (and/or archive) Delete job logs Delete OLAP and planning statistics Delete data from the change log Delete requests from the PSA Delete master data and texts for a characteristic Delete data selective DTP temporary storage reduction Archive IDocs Archive request administration data The Technical InfoCube (0TCT_C25) contains information on large tables that can be candidates for further reduction For more details on how to get sizes down and what each of these areas mean, see the Intro to HANA book, Chapter 5

BW Transformation Finder Some custom data transformations in SAP BW can be sub-optimally written and have negative impact on HANA performance The BW Migration Cockpit provides a tool to help you identify them This part of the tool can be run on SAP BW 3.5 or higher. See SAP Note 1908367 for more details.

Code Inspector and Analyzer for ABAP in BW After you have found the codes you searched for, you can analyze the code to see if improvements can be made to make it even faster in HANA You get a lot of additional information in the log (scroll to the end for some great ideas of how to optimize the code) This tool also runs on all BW releases 3.5 or higher (see SAP Note 1847431)

System Migration Optimization and SUM The Software Update Manager (SUM) was enhanced and now also contains an option called the “Database Migration Option (DMO)” The DMO allows you to combine the BW upgrade, Unicode conversion, and other tasks that are often required, as well as the actual HANA migration in one interface instead of two “projects” You can use the DMO if your are on minimum SAP BW version 7.0 and have applied Service Pack 17 (see SAP Note 1799545)

BW 7.4 DSOs and Re-Conversion of DSOs BW optimized DSOs were reccomended in 2013. HANA optimized DSOs were available for BW 7.3. As of February 2014, SAP reccomends that you do not to do any optimization of DSOs and actually convert back to the old DSOs for SAP (Note: 1849498)

Converting InfoCubes and/or Data Flows While not required, InfoCubes can be optimized further for HANA performance This basically means “flattening” the data structures and removing the dimensions in BW from the physical layer (they still look as if they exists) Many refer to this optional step as a “functional migration” and do this after the HANA migration has been completed, often as a separate initiative (see SAP Note 1849497) PS! Optimization of DSOs are no longer reccomended

Converting InfoProviders and/or Data Flows To help you, the SAP Migration Cockpit also allows you to migrate your data flows from 3.x to Data Transfer Processes (DTPs) as used in versions 7.0 and higher If you convert the data flows you get better automated data package DTP optimization, which loads data faster into HANA. You can also simulate the data flow before you do the real conversion. When doing so, data is loaded for both versions (3.x and 7.x) of the dataflows and the results are stored in cluster tables. The data is then compared to verify that the dataflow after migration calculates the same data as it did before migration Since the differences are displayed separately, you can analyze the results and changes in details

The Application Specific Upgrade (ASU) Toolbox Once you have completed the SAP_BW_HOUSEKEEPING and the SAP_BW_BEFORE_UPGRADE task lists, you can start the ASU schedule manager and prepare the environment While there are some overlapping tasks in these task lists, you are required to complete all tasks listed as “obligatory” You start the upgrade by executing the transaction /ASU/START (see SAP Note 1000009) The Direct Migration Option (DMO) for upgrading BW and moving to HANA at the same time also uses these task lists and the ASU Toolbox. DMO is an option in the SUM tool.

Direct Migration Option (DMO) During the upgrade from BW 3.x to version 7.0 many companies decided not to complete Unicode conversions, security conversion, and other recommended steps Since these are now required for the BW 7.4 and the subsequent migration to HANA, some companies are planning first to do the upgrade and then do a migration project. That is a mistake! With the new Direct Migration Option tool you can accomplish both the 7.4 upgrade and the HANA migration in one step. DMO is a key option in the Software Update Manager (SUM), for those with older, out-of-date, BW systems that want to migrate to HANA.

Creating a DMO Migration Run-Book The best way to approach this is to start with the sandbox system and create a runbook with step-by-step lists on how each problem and software task are created. It is not unusual to have a 90-100 page word document with screenshots and documentation at the end of this first migration. The “runbook” is the key to success. You should build on this when you migrate to the Development and then the QA and the Production systems DMO started supporting Unicode conversions at the end of 2013 and is now in “controlled availability,” but with SAP approval you can use it now Speed is not important in the first sandbox migration. The creation of a repeatable process is far more important.

A 108 Step Example of a DMO Migration Runbook There are many repertory task you must do before the actual migration In steps 1,2, and 3 we are reviewing the latest notes and setting up user and system access In step 4.2 you must request a migration key

A 108 Step Example of a DMO Migration Runbook (cont.) For those doing a Unicode conversion, there are many additional steps. We first need to check what is already in-place. Thankfully, SAP provides programs to help you with these to check the config. You can get this report by running the report UCCHECK and seeing the installed languages in the source system by using the transaction SMLT

A 108 Step Example of a DMO Migration Runbook (cont.) The next major step is to extract the files needed for the migration. Here you will need the migration keys you obtained in step 4.2.

A 108 Step Example of a DMO Migration Runbook (cont.) If you are working with a BW system that is not heavily used, or one that has lots of processing capacity, you can minimize the downtime by using a “shadow system” during the upgrade If you use a shadow system (option 2/3), the system will be copied (not the data) and many of the upgrade tasks will happen on this “shadow’ system” while the real system is still running Only in the later stages is the system unavailable to the users while the configuration and data are moved to SAP HANA This minimizes the downtime of the system

A 108 Step Example of a DMO Migration Runbook (cont.) Now we have to tell the DMO what system we are coming from and what system we want to migrate to

A 108 Step Example of a DMO Migration Runbook (cont.) It is now time to check inside SAP HANA Studio that the BW schema has been created by DMO. You find this under “users” in the HANA navigator. We also have to decide what support packages we want to include in the upgrade. Normally we pick the latest and ignore the equivalent SPs.

A 108 Step Example of a DMO Migration Runbook (cont.) In the check phase, you may get “soft warnings” or items that have to be fixed before progressing. Pay close attention to both and make sure you understand what each mean.

A 108 Step Example of a DMO Migration Runbook (cont.) If you did not complete the in-depth cleanup tasks we recommended earlier, you can now do some basic system cleanup as well Most of the cleanup tasks are better performed prior to starting this process

A 108 Step Example of a DMO Migration Runbook (cont.) Since the “shadow system” is created, no more changes to the configuration or settings in the existing BW system can occur after this stage NOTE: Users can still access the system

A 108 Step Example of a DMO Migration Runbook (cont.) If we want to make changes to the shadow instance, we can do that by logging on as user DDIC and changing the system setting using the transaction code SE06. We can now make changes directly using the transaction code SPDD

A 108 Step Example of a DMO Migration Runbook (cont.) After the tables have been created in the HANA system, you can reorganize them This allows you to load balance the HANA system even further before completing the migration of the data from BW to HANA This is not a required step for most systems

A 108 Step Example of a DMO Migration Runbook (cont.) We now have to lock down the system and stop all jobs and access

A 108 Step Example of a DMO Migration Runbook (cont.) Before we proceed any further and start migrating the data, we should complete a full backup of the system

A 108 Step Example of a DMO Migration Runbook (cont.) We now are in the lockdown and downtime phase. The instance is running as a “remote host” and we will start moving data to the new system.

A 108 Step Example of a DMO Migration Runbook (cont.) Before we provide users and developers access to the new system and start testing, you should run another backup of the system so that valuable time can be saved if you have to revert back to a pervious system

A 108 Step Example of a DMO Migration Runbook (cont.) At this stage the users can access the new system and start the testing of the migration. We normally have both technical and functional testers involved in this phase.

A 108 Step Example of a DMO Migration Runbook (cont.) We are now ready to access the HANA system and all post-processing tasks have been completed The next step is to start the development migration using the runbook

After Migration — The Delta Merge Monitor The Delta Merge Monitor can help developers and administrators in load optimization and processing of queries in HANA Since the tool can be very resource intensive, it is recommended to be used when the system has minimal usage (i.e., weekends and nights) To use the tool, you will have to be on HANA 1.0 Service Pack 5 and BW version 7.3 Service Pack 6 or higher. More details are available in SAP Note 1663501.

After Migration — Optimize Database with Secondary Indexes Normally SAP BW with HANA doesn’t use secondary indexes. But this can be added for improving lookup performance in cases such as transformations. It is important to be aware that secondary indexes, if used extensively, can require more memory and can also slow down data loads. Therefore, you may consider adding “HANA hints” in the ABAP code first (see SAP Note 1662726) or deleting the filter in the SQL statement and instead executing it in the ABAP engine after HANA returned non-filtered data (see SAP Note 1740373) While HANA is much faster than a traditional database, it also can be performance tuned even faster (this is not normally required as part of typical HANA migration)

After the Migration — Table Consistency Checks Once migrated to HANA, you can check your tables, indexes, and partition distribution and setup to assure optimal operations This tool requires that your BW on HANA system is at least a BW 7.3 Service Pack 6 You can access it by running RSDU_TABLE_CONSISTENCY To get more information on this tool, check SAP Note 1695778

Automated Updates and More Information The BW Migration cokpit also provides information on new updates, documentation and also additional information links for SAP HANA. It is a key tool for all who do a BW to HANA migration.

An Example of a Mid-Sized (10 TB) HANA DMO Migration Plan This

An Example of a Very Large (40 TB) HANA Migration Plan This very large 110+ TB system with NLS and substantial complexity was moved in less than 5 days in the final test migration

Where to Find More Information www.sap-press.com/products/SAP-HANA%3A-An-Introduction-(2nd-Edition).html Bjarne Berg and Penny Silvia, SAP HANA: An introduction, SAP Press; 2nd edition (May 1, 2013) http://www.saphana.com/welcome SAP’s main page for all SAP HANA related information http://www.saphana.com/community/try Powered by HANA demos http://scn.sap.com/community/netweaver-bw-hana SAP NetWeaver BW Powered by SAP HANA Community Dr Berg: I wasn’t able to see the BW powered by HANA demo at this site. Can you double check? Berg: Fixed

7 Key Points to Take Home There are programs to do pre-readiness checks for an ERP and BW system for migration to HANA A BW Migration Cockpit and a new DMO tool are now available to assist in the tasks While one is more common, there are actually four possible approaches to the BW on HANA migration project SAP is providing guidance and support — more than ever — to make customers successful, and HANA is now very mature Standalone HANA projects are also becoming very common, even with companies that also have BW Most HANA projects can be done in a matter of weeks, only extremely large systems may require 4-7 months Experienced resources with hands-on knowledge is in high demand

Your Turn! How to contact me: Dr. Berg bberg@comerit.com Please remember to complete your session evaluation

Disclaimer SAP, R/3, mySAP, mySAP.com, SAP NetWeaver®, Duet™®, PartnerEdge, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are the trademarks of their respective companies. Wellesley Information Services is neither owned nor controlled by SAP.

An A-to-Z Guide to Implementing SAP HANA: Planning, Scoping, Staffing, Budgeting, and Execution