1 / 22

Andy Pavlo November 14, 2014

H-Store Introduction. Andy Pavlo November 14, 2014. Terminology. Partition: Logical subset of the database. Site: A JVM instance that contains one or more partitions. Host: A single node in the cluster that contains one or more sites. Terminology.

cleopatra-tocci
Download Presentation

Andy Pavlo November 14, 2014

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. H-Store Introduction Andy PavloNovember 14, 2014

  2. Terminology • Partition:Logical subset of the database. • Site:A JVM instance that contains one or more partitions. • Host:A single node in the cluster that contains one or more sites.

  3. Terminology • Catalog: Internal information about the current database. • Client: Application that issues transaction requests at sites.

  4. Environment Setup • Linux / Mac OS X • Only runs on 64-bit platforms. • Dependencies: • gcc/g++ (≥4.3) • java (≥1.6) • ant (≥1.7) http://bit.ly/yCJNGQ

  5. Environment Setup • Network filesystem. • Passwordless SSH login. $ ssh-keygen -t dsa <Use Empty Password> $ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys http://bit.ly/yCJNGQ

  6. Introduction • All operations executed through ant. • H-Store has built-in test applications. • Application + Cluster information gets compiled into “project jar files”. $ ant hstore-prepare –Dproject=tpcc http://bit.ly/yLKvU2

  7. Building H-Store • Use build to compile all of the system: • Can also compile a subset of the system: $ ant clean-allbuild $ ant clean-java build-java $ ant clean-cppbuild-cpp http://bit.ly/zF28BA

  8. Cluster Configuration • Cluster configuration defined in either file or from command-line: • <HostName>:<SiteId>:<PartitionId> $ ant hstore-prepare –Dproject=tpcc–Dhosts=hosts.txt $ ant hstore-prepare –Dproject=tpcc \ -Dhosts=“host:0:0-1;host:1:2-3” http://bit.ly/yXJsMS

  9. Catalog Information • Use catalog-info to view cluster configuration. • Use catalog-viewer for graphical catalog browser. $ ant catalog-info –Dproject=tpcc $ ant catalog-viewer –Dproject=tpcc http://bit.ly/ycNNbs

  10. Executing H-Store • BenchmarkController will automatically deploy cluster, execute benchmark, and then shutdown: • Five built-in benchmarks • TPC-C and TM1 are most stable. $ ant hstore-benchmark –Dproject=tpcc http://bit.ly/w8fHL7

  11. Executing H-Store • Can also execute a single procedure: $ ant hstore-benchmark –Dproject=tpcc \ -Dnoexecute=true \ -Dnoshutdown=true $ ant hstore-invoke –Dproject=tpcc \ -Dproc=ProcedureName –Dparam0=123 http://bit.ly/w8fHL7

  12. Configuration Files • Define parameters in hstore.conf • See website for full listing of available options • Can override parameters at start-up $ ant hstore-benchmark –Dproject=tpcc \ -Dsite.memory=4096 \ -Dclient.txnrate=1000 http://bit.ly/xPH1uU

  13. Log Files • Each site’s log files are written to separate files in obj/logs/sites • Can control log file verbosity by editing log4j.properties

  14. Unit Tests • Continuous integration testing. • Execute Java-only tests: • Execute C++-only tests $ ant junit $ ant eecheck

  15. Source Code Hierarchy • src: System source code directories. • tests: Unit tests source code. • properties: Configuration files. • third_party:Additional libraries/jars.

  16. Source Code Hierarchy • src: System source code directories. • catgen:System catalog schema. • ee:Execution Engine (C++). • frontend: Database Frontend (Java). • protorpc:Network RPC schema. • hsqldb: HSQLDB Wrapper.

  17. System Overview • HStoreSite: • Manages multiple PartitionExecutors. • PartitionExecutor: • Executes Java stored procedures. • Processes query requests. • HStoreCoordinator: • Communicates with remote HStoreSites.

  18. Procedure Listener Procedure Invocation HStoreCoordinator Partition Executor Partition Executor Partition Executor ↓JNI ↑ ↓JNI ↑ ↓JNI ↑ C++ Execution Engine C++ Execution Engine C++ Execution Engine … … Partition Data Partition Data Partition Data

  19. Txn #1001 Execute query at remote site HStoreCoordinator HStoreCoordinator HStoreService (ProtoRPC) Transaction Init Handlers Transaction Work Local Callback Invoke Callback Remote Callback Invoke Callback Dispatcher Execute Work Network Network Transaction Prepare Transaction Finish Send Data

  20. What’s Next? • Try running H-Store yourself. • Let me know if you need more space on department filesystem. • Setup H-Store in Eclipse. • http://bit.ly/xpyLir

  21. What’s Next? • Create a Github account. • Please use a profile picture so that it easier to know who you are. • Fork the H-Store project on Github • If in two-person group, create one fork and make other team member a “collaborator.”

  22. For Next Class (in Two Weeks) • Project Proposal • How you are going to implement your project? • System components and source code files that you think you will need to change/add. • How are you going to test your project? • Interesting issues or unanswered. • Missing or broken features in H-Store.

More Related