240 likes | 394 Views
Exadata Management for DBAs. Arup Nanda Longtime DBA and Now DMA. What it this About?. How Exadata is different? Who manages it? Oracle DBAs System Admins Composite Team Who does what? What you need to know about managing it What must you learn about potential issues. What is Exadata.
E N D
Exadata Management for DBAs Arup Nanda Longtime DBA and Now DMA
What it this About? • How Exadata is different? • Who manages it? • Oracle DBAs • System Admins • Composite Team • Who does what? • What you need to know about managing it • What must you learn about potential issues
What is Exadata • Looks like an appliance • But is not an appliance. Why? • It contains additional software to make it a better database machine • The components are individually adminstered • That’s why Oracle calls it a Database Machine (DBM) • DMA – Database Machine Administrator
The Magic #1 CPU iDB Memory Get NAME … STATUS = ‘ANGRY’ Network I/O Controller Disk
Magic #2 Storage Cell Server • Cells are Sun Blades • Run Oracle Enterprise Linux • Software called Exadata Storage Server (ESS) which understands iDB iDB Cell Server ESS Disk1 Disk2 Disk3
Magic #3 Storage Indexes Cell Server Storage Indexes store in memory of the Cell Server the areas on the disk and the MIN/MAX value of the column and whether NULL exists. They eliminate disk I/O. ESS Disk1 Disk2 Disk3 MIN = 3 MAX = 5 MIN = 4 MAX = 5 MIN = 1 MAX = 2 MIN = 3 MAX = 5 SELECT … FROM TABLE WHERE COL1 = 1 1 2 3 4 Storage Index Disk4
Put Together: One Full Rack RAC Cluster Database Node 1 Database Node 8 Database Node 7 Clients connect to the database nodes. InfiniBand Switch Network Switch Cell 1 Cell 14 Cell 1 Cell 1 Cell 1 Disk1 Disk 12 Disk1 Disk1 Flash
Disk Layout • Disks (hard and flash) are connected to the cells. • The disks are partitioned at the cell • Some partitions are presented as filesystems • The rest are used for ASM diskgroups • All these disks/partitions are presented to the compute nodes Compute Nodes Storage Cell Disk1 Disk 12 Disk1 Disk1 Flash
Disk Presentation Node ASM Diskgroup ASM Disk filesystem Griddisk Disk Celldisk filesystem LUN LUN Cell Partition 1 Partition 2 Physical Disk
Command Components Linux Commands – vmstat, mpstat, fdisk, etc. Compute Nodes ASM Commands – SQL*Plus, ASMCMD, ASMCA Database Commands – startup, alter database, etc. Disk1 Clusterware Commands – CRSCTL, SRVCTL, etc. Storage Cell Linux Commands – vmstat, mpstat, fdisk, etc. CellCLI – command line tool to manage the Cell Disk1 Disk 12 Disk1 5-part Linux Commands article series http://bit.ly/k4mKQS 4-part Exadata Command Reference article series http://bit.ly/lljFl0 Disk1 Flash
Disk Failures Datafile block1 Cell 1 Cell 2 block1 block1
Server Management • Sun Blades and Oracle Enterprise Linux • Normal Sysadmin Work • Shutdown, fdisk, etc. • ILOM – Integrated Lights Out Management • KVM allows physical access • But you can use ILOM for virtual console • Needs Pure Linux Skills
Network Management • Two types of network • Ethernet • Infiniband • Tools • ibstatus • iblinkinfo • verify-topology
Special Oracle Provided Tools • All tools are found at /opt/oracle.SupportTools • CheckHWnFWProfile • to check the HW profile • Directory ibdiagtools /opt/MegaRAID/MegaCli/MegaCli64 # ipmitool -H prolcel01-ilom -U root chassis power on # imageinfo # imagehistory
Database and Cluster Management • Cluster • crsctl, srvctl • ASM • asmcmd, SQL*Plus • Database • srvctl, SQL*Plus
Storage Management • Two ways to manage the storage • Enterprise Manager • CellCLI
Checking Storage Index Use select decode(name, 'cell physical IO bytes saved by storage index', 'SI Savings', 'cell physical IO interconnect bytes returned by smart scan', 'Smart Scan' ) as stat_name, value/1024/1024 as stat_value from v$mystat s, v$statname n where s.statistic# = n.statistic# and n.name in ( 'cell physical IO bytes saved by storage index', 'cell physical IO interconnect bytes returned by smart scan’)
Smart Scan Savings • Output STAT_NAME STAT_VALUE ---------- ---------- SI Savings 0.000 Smart Scan 0.000 • Smart Scan did not yield any savings • Why not? • Disable Smart Scans, if needed • cell_offload_processing = true; • _kcfis_storageidx_disabled = true;
Why Not? • Pre-requisite for Smart Scan • Direct Path • Full Table or Full Index Scan • > 0 Predicates • Simple Comparison Operators • Other Reasons • Cell is not offload capable • The diskgroup attribute cell.smart_scan_capable set to FALSE; • Not on clustered tables, IOTs, etc.
One Cluster? 1 2 3 4 5 6 7 8 One Cluster QA1 QA2 QA3 Prod1 Prod2 Prod3 Dev1 Int1 QA1 QA2 Prod4 Prod1 Prod2 Prod3 Dev1 Int1 QA1 QA2 QA3 Prod1 Prod2 Prod3 Dev1 Int1 Dev2
Many Clusters? 1 2 3 4 5 6 7 8 QA Cluster Prod Cluster Dev Int QA1 QA2 QA3 Prod1 Prod2 Prod3 Dev1 Int1 1 2 3 4 5 6 7 8 QA Cluster Prod Cluster Dev Int
Resources • My Papers • 5-part Linux Commands article series http://bit.ly/k4mKQS • 4-part Exadata Command Reference article series http://bit.ly/lljFl0 • OTN Page on Exadata • http://www.oracle.com/technetwork/database/exadata/index.html • Tutorials • http://www.oracle.com/technetwork/tutorials/index.html • OTN Exadata Forum • https://forums.oracle.com/forums/forum.jspa?forumID=829