1 / 8

SPLITD

SPLITD. Tom Madden NCBI. Problems. Biological databases growing faster than computer memory. Fastest CPU’s get put into 1 or 2 CPU machines first. Most OS’s work better with fewer CPU’s in the box. Splitd solution. Partition search so that it is spread over multiple machines.

dunningk
Download Presentation

SPLITD

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SPLITD Tom Madden NCBI

  2. Problems • Biological databases growing faster than computer memory. • Fastest CPU’s get put into 1 or 2 CPU machines first. • Most OS’s work better with fewer CPU’s in the box

  3. Splitd solution • Partition search so that it is spread over multiple machines. • Break up database “virtually” so the number of chunks can be adjusted on the fly depending upon load, query, etc. • HSP’s (start/stop/score) of alignments are calculated by backends. • Tracebacks are calculated after merging.

  4. Also… • Use MSSQL to store queries and results rather than a home-grown system. • Concatenate queries from different users ala megablast to minimize time spent scanning the database.

  5. FR FR BD BD BD PD PD PD SD = SplitDaemon BD = blastsrvd+blastsrv4+nabrd PD = PartsDaemon MD =MergeDaemon FR = Formatter MD Browser SD MD Blast.cgi MSSQL12 MSSQL20

  6. FR FR BD BD BD PD PD PD SD = SplitDaemon BD = blastsrvd+blastsrv4+nabrd PD = PartsDaemon MD =MergeDaemon FR = Formatter MD Browser SD MD Blast.cgi MSSQL12 MSSQL20

  7. FR FR BD BD BD PD PD PD SD = SplitDaemon BD = blastsrvd+blastsrv4+nabrd PD = PartsDaemon MD =MergeDaemon FR = Formatter MD Browser SD MD Blast.cgi MSSQL12 MSSQL20

  8. FR FR BD BD BD PD PD PD SD = SplitDaemon BD = blastsrvd+blastsrv4+nabrd PD = PartsDaemon MD =MergeDaemon FR = Formatter MD Browser SD MD hsp Blast.cgi traceback MSSQL12 MSSQL20

More Related