350 likes | 518 Views
PG&E Corporation. Ishaq Mohammed/Anil Saka Online and Middleware Application Operations. Agenda. Overview Challenges Solution Middleware Provisioning Use Cases PG&E’s Provisioning Portal Benefits Gained Future Plans Summary Q/A. Agenda. Overview Challenges Solution
E N D
PG&E Corporation Ishaq Mohammed/Anil Saka Online and Middleware Application Operations
Agenda • Overview • Challenges • Solution • Middleware Provisioning Use Cases • PG&E’s Provisioning Portal • Benefits Gained • Future Plans • Summary • Q/A
Agenda • Overview • Challenges • Solution • Middleware Provisioning Use Cases • PG&E’s Provisioning Portal • Benefits Gained • Future Plans • Summary • Q/A
Company Profile • Headquarters in San Francisco, California • Provides energy to approximately 15 million people • 70,000 square mile service territory • Total Assets: $50 billion • 2012 Net Income: $816 million • 2012 Capital Expenditures: ~$4.5 billion PG&E SERVICE AREAIN CALIFORNIA
IT Overview - Infrastructure • 280+ WebLogic Domains (versions 9.2 through 12.1.1) • SOA Suite & Oracle Service Bus (OSB) • WebLogic Portal & Integration • 970+ WebLogic Servers • 205 Databases (versions 10g through 11.2.0.3) • 500+ host servers running Linux and AIX • Microsoft SQL Server, Apache HTTP Server 2.2 • 2,000 IT employees - 15 involved in middleware management
IT Overview – MyEnergy Business Application • MyEnergy application most critical from business perspective • Account management (e.g. customers can monitor /manage energy use, pay bills online) • Promotion of energy-savings programs • Scheduled outages of PG&E services • 2.2 million active users • 6,000 users logged in hourly during peak times
Agenda • Overview • Challenges • Solution • Middleware Provisioning Use Cases • PG&E’s Provisioning Portal • Benefits Gained • Future Plans • Summary • Q/A
Challenges • High cost in manually building 30+ middleware environments each year (~$2.5 million) • 20% increase in WebLogic landscape over next 12 months • What worked in pre-production sometimes failed to work in production
Agenda • Overview • Challenges • Solution • Middleware Provisioning Use Cases • PG&E’s Provisioning Portal • Benefits Gained • Future Plans • Summary • Q/A
Solution: Oracle Enterprise Manager to Provision Middleware Environments • Modernizes the build process • Guarantees speed to market delivery • Eliminates errors • Reduces labor cost and time spent • Provides out-of-box monitoring
Oracle Enterprise Manager Cloud Control 12c Deployment • Implemented four environments for Cloud Control 12c – dev, test, stage and production • Production environment for Cloud Control 12c: • Management Service 12.1.0.3 Release 3on Redhat Linux with Active-Standby high availability configured • Management Repository in two node RAC DB 11.2.0.3 on Redhat Linux • Management Agents 12.1.0.3 Release 3 on Redhat Linux and AIX • Fusion Middleware Plug-in 12.1.0.5
Agenda • Overview • Challenges • Solution • Middleware Provisioning Use Cases • PG&E’s Provisioning Portal • Benefits Gained • Future Plans • Summary • Q/A
Middleware Use Cases WebLogic Server 12c (WLS) version 12.1.1 Oracle Service Bus 11g (OSB) version 11.1.1.6 WebLogic Portal 10g (WLP) version 10.3.4 Small, Medium Or Large Deployment
Prerequisites • Install and start Oracle Management Agent on new server • Install required packages on the OS, create OS user and group(webLogic) • Enable passwordless (ssh keys) connection between Oracle Management Service and new VM • Update the /etc/sudoerswith: weblogicALL=(root)/u01/app/Oracle/product/agent12c/sbin/nmosudo
Middleware Provisioning • Discover reference Oracle Home/domain • Create provisioning profile in software library • Create directives in software library • Edit predefined deployment procedure • Test deployment procedure from console to capture input parameters • Create wrapper script which calls emcli “submit_procedure” • Enable end users to make requests using PG&E’s Provisioning Portal
Directives • Provisioning Profiles Capture “Standard” Files and Artifacts of WebLogic Domain. • Additional Files Specific to Line of Business must be moved to Target Host(s) with File Transfer Step or Directive. • With Directives you upload required Files to Software Library along with script to move and/or Manipulate Files, called Main file • There are other uses for Directives. However, this is how we primarily used them • Be sure to check “Use Most Recent Version” check box, else your procedure keeps calling older version
Customized Deployment Procedure Oracle-provided deployment procedure named "Provision Middleware" customized to accommodate PG&E requirements • Custom Directives (Described Earlier), or • Shell Script that Execute Shell Commands and/or call WLST • We Generally Added One Directive and 3 to 4 Shell Scripts to Customized Procedures • Shell Script Cannot Exceed 4096 Characters. • Can Use Runtime Variables in Scripts, e.g. ${target.DOMAIN_HOME_DEST_ADMIN_HOST} • Steps Can Have Conditions (e.g. execute only on remote servers)
Options to Provision • Provision using EMCLI: • Capture once and execute many Provision using console: • Several clicks and repetitive process
EMCLI Wrapper Script • Generated Properties Files Made Generic by Substituting Actual Values with Specific Keywords: <Hostname> -> MyHostName <Domain Name> -> MyDomainName <DBSid> -> MyDBSid • Wrapper Scripts Creates Copy of Properties Files and Substitutes New Values Passed on Command Line • Removes Copy When Completed • Provision New WebLogic Portal Domain Example: sh./emcliWrapperDB.sh "DB_DATA" "S" “database-host“ "Linux" "WLPDBS" sh./emcliWrapperWL.sh WLP S EIWLP wlp-mw-host database-host WLPDBS
Agenda • Overview • Challenges • Solution • Middleware Provisioning Use Cases • PG&E’s Provisioning Portal • Benefits Gained • Future Plans • Summary • Q/A
PG&E Provisioning Portal • Operating Systems • SAP Environments • Oracle Databases • Oracle Fusion Middleware Environments • Capable of Provisioning:
PG&E Provisioning Portal Processes • PG&E has built a portal for request intake and complete orchestration of the provisioning process • Provides central interface to end users • Makes REST calls to Pureflex System for OS provisioning • Installs Management Agent • Invokes EMCLI scripts to provision middleware environments
PG&E Provisioning Portal Features • Authentication against PG&E Active Directory / SSO • Ability to notify Requestor/Approver via email • System provisioning clients pay for the resources • System provisioning clients are not responsible for the maintenance of any part of the shared infrastructure • Ability to monitor performance of their workloads to ensure SLAs are met • Provides at least three deployment options for each artifact in terms of sizing: small, medium, large
One Click! • Provision Operating System • Install Middleware • Configure Domain • Discover Domain to obtain out-of-box monitoring
Agenda • Overview • Challenges • Solution • Middleware Provisioning Use Cases • PG&E’s Provisioning Portal • Benefits Gained • Future Plans • Summary • Q/A
Achieved Results and Improved Business • Standardized OS, middleware and database images • Reduced overall provisioning time by 75% • Saved ~$1 million since inception
Agenda • Overview • Challenges • Solution • Middleware Provisioning Use Cases • PG&E’s Provisioning Portal • Benefits Gained • Future Plans • Summary • Q/A
Future Plans • Extend provisioning to more use cases: • Oracle SOA Suite • Oracle Business Process Management Suite • Web Servers
Agenda • Overview • Challenges • Solution • Middleware Provisioning Use Cases • PG&E’s Provisioning Portal • Benefits Gained • Future Plans • Summary • Q/A
Summary • PG&E uses Oracle Enterprise Manager Cloud Control 12c deployment procedures with custom steps and scripts to provision all Oracle WebLogic Server, Portal and Service Bus environments • PG&E has built an end-user portal to provision middleware infrastructure with a single click • Provisioning brings PG&E a 75% reduction in time to market • Many lines of businesses are enjoying the benefits of lower costs, faster delivery, and standardized deployments
Agenda • Overview • Challenges • Solution • Middleware Provisioning Use Cases • PG&E’s Provisioning Portal • Benefits Gained • Future Plans • Summary • Q/A