190 likes | 307 Views
Tektronix EGBU Design Win. Randall Rhea March, 2007. Tektronix Deal Profile. Leading manufacturer of test equipment Tektronix acquired Inet in 2004 Inet provided telecommunications test equipment Oracle 10g enterprise deal up for renewal
E N D
Tektronix EGBU Design Win Randall Rhea March, 2007
Tektronix Deal Profile • Leading manufacturer of test equipment • Tektronix acquired Inet in 2004 • Inet provided telecommunications test equipment • Oracle 10g enterprise deal up for renewal • Wanted to upgrade enterprise systems and network equipment to RAC for high availability • Oracle not currently used in some embedded systems • Ken Wajerski established close relationships with the commercial Oracle sales team • Ed Peek – Sales Director • Ryan Feaver – Account Manager • Deal closed in Q3 • Approx. $1,000,000 of the deal credited to embedded/EGBU • Deal included RAC upgrade and Berkeley DB for embedded systems
Tektronix Network Analysis Products • Problem: Wireless carriers cannot determine cause of network problems • Why are calls getting dropped? • Tektronix Geoprobe • Analyzes all packets in a network • Real-time alarms and troubleshooting • Home-grown database (no Oracle products) • Embedded platform, 2U hardware on Linux • Tektronix Orion and Beamer • Allows carriers to focus on its most profitable customers (such as corporate customers) • Goal is to improve quality-of-service to retain big customers • Larger platform that uses Oracle 10g, wanted RAC upgrade
Tektronix Geoprobe Requirements • Existing system does not scale to the demands of larger customers • Existing single-node, in-memory, home-grown database limited to 3 GB of data • 50,000 database operations per second • Tektronix developers tested Oracle, MySQL, and other RDBMS products: not fast enough, not even close • SQL is not necessary • Data is accessed via a key (no ad-hoc queries) • Database size up to 20 GB • Current hardware is Xeon (32-bit), 4 CPU, 4 GB RAM, Linux • Do NOT want to be limited to an in-memory database • TimesTen was ruled out for this reason
Challenges • 64-bit hardware was not available • Geoprobe must use the 32-bit 3GB hardware used in Tektronix’ other products • If disk I/O is required, the queries won’t be fast enough • 15,000 RPM disks can perform about 150 random seeks per second • 20 GB database exceeds RAM on one system (3 GB) • No out-of-the-box solution available • RDBMS like Oracle RAC are not fast enough, even if fully cached • Tektronix considered enhancing their home grown system to provide data distribution and clustering
Sales Approach • Determine appropriate product • Berkeley DB was the only possible option • Determine if Berkeley DB could meet 50,000 TPS requirement • Obtained benchmark from Steve Folkman (The TimesTen TPTBM benchmark re-written for Berkeley DB) • Modified benchmark to better simulate customer’s transaction • Benchmark run on lab systems from TimesTen R&D • If the performance target could be met, propose an architecture to the customer • A combination of Berkeley DB and home-grown code • FUD for keeping existing home-grown database • Particularly with HA (high availability)
Benchmark Results • Performance of 49,000 to 55,000 operations per second on 100% read test • Database was 85% to 100% cached in memory • Consistent performance as database increased in size • Write performance of 40,000 operations per second on a single disk; 75,000 operations per second on a 4-disk array • Conclusion: Meets customer requirement, provided that most of the database is cached • Disk I/O is the bottleneck if the database is not cached • If data was not cached, response time slows down as database size increases due to inherent latency in disk operations
Current Tektronix System • Limitations: • Database size limited to memory on one machine (3 GB, cannot be upgraded on 32-bit machines) • Tektronix must maintain home-grown database software • Tektronix developers should spend time on business logic, not database software Applications Home-Grown TCP/IP interface TCP/IP & Ethernet Home-grown in-memory DB
Architecture Proposal:Distributed/Clustered Cached Database Applications Home-Grown TCP/IP interface In this example, there are four 8 GB nodes, so the capacity is 3x8 = 24 GB Cached Database Cached Database Cached Database Cached Database Cached Database Cached Database Cached Database Cached Database Interconnect (Ethernet)
Challenges in a Clustered System • If a node fails in the cluster, the entire cluster is down • High availability is required • This is usually done with a standby node and replication • This was a key FUD point to discourage customer from keeping home-grown system • Cluster must be managed • This is not easy • Many nodes must be managed – both individually, and how they are working together • It becomes more difficult as you add nodes • Current home-grown system lacks: • HA of any kind • Monitoring capabilities • Caching • Disk persistence • Concurrency
Proposal • A combination of home-grown code and Berkeley DB • Berkeley DB would: • Replace home-grown data manager • Handle basic database I/O, caching, persistence, recovery • Locking, concurrency • Backup, recovery • Home-grown code would: • Distribute the data to multiple nodes by a simple algorithm (a mod function or hash function) • Manage the cluster • Vs. a complete home-grown solution, this reduces the development effort by about 50%
How Berkeley DB Reduces Costs • Berkeley DB is not a distributed database … • However, Berkeley DB can make a distributed database project a lot easier, faster, less expensive, and less risky • With Berkeley DB, the customer would not need to build or maintain low-level database code • Berkeley DB is very flexible: • All in memory • All on disk • On disked and cached in memory
How Berkeley DB Reduces Costs (2) • Berkeley DB has several high availability (HA) options: • Replication allows for a hot standby node: if a node fails, the standby immediately takes over with no downtime • Warm failover allows for quick recovery of a down node’s data on another node – after a short downtime (several seconds), the system is back up and fully operational • Backup and recovery: All database on all nodes can be backed up, in case the entire cluster needs to be recovered • A replica “global” database on disk can provide a backup copy of all data in case a down node needs to be restored, or the entire cluster needs to be re-initialized • HA is very difficult to write and maintain yourself
No high availability?Get ready for a nightmare … • The system is down, you are called to a customer site to fix the system • You need to restore the data on a node, but there is no useable backup • Data is distributed across multiple nodes • There is no single backup with all of the data • The “backup” is several, separate files that are old and don’t contain a consist copy of the database • With no logging, thousands of transactions are lost • Answer: keep a disk-based copy of all the data in Berkeley DB • This acts as a global checkpoint file • If a node needs to be recovered, or the entire cluster restored, you can use the global database to recover it • Each database write would go to an in-memory node and the global replica node
Future Use of Berkeley DB in GeoProbe Geoprobe Applications Home-Grown TCP/IP interface There can be both a full disk-based copy of BDB and in-memory copies. The replica disk database can contain ALL data, and can be used to restore down nodes or re-distribute data when new nodes are added. Home-Grown Cluster Management Restore data to down node Spare Node On disk with all data In Memory In Memory In Memory Interconnect (Ethernet)
Lessons Learned • Sometimes, the customer’s requirements are unique, and no off-the-shelf product can solve the problem • Be creative in proposing a solution that uses an Oracle product • Try to piggyback on top of an Oracle enterprise deal • Berkeley DB alone may not provide enough value to the customer to win a large deal • Establish close relationships with the other Oracle sales teams • The internal sales job is just as important as customer sales job • Make sure other Oracle sales teams are aware that the EGBU can help them • Ask sales managers for a few minutes during staff meetings to present the EGBU story