190 likes | 279 Views
VOMegaPlot. Efficient Plotting of Large VOTable Datasets. VOPlot. VOPlot is a tool for visualizing astronomical data that is available in the VOTable format. VOPlot reads the xml file in order to load entire data into memory and then processes it to draw various types of plots.
E N D
VOMegaPlot Efficient Plotting of Large VOTable Datasets
VOPlot • VOPlot is a tool for visualizing astronomical data that is available in the VOTable format. • VOPlot reads the xml file in order to load entire data into memory and then processes it to draw various types of plots. • This approach of loading the entire data into memory cannot be used for VOTable files that are very large.
Approach for VOMegaPlot • VOMegaPlot preprocesses the xml file to create intermediate files which are subsequently used for plotting. • Entire data is divided into fixed sized blocks and individual blocks are loaded into memory thus reducing the memory requirement. • The number of intermediate files created is equal to the number of columns present in the xml file.
Pre-processing operationCreation of array blocks Col 1 Col 2 ……. Col m 1 2 . . n Original xml file with m columns and n rows File m File 1 File 2 Block 1 Block 1 Block 2 Block 2 Block 1 Block 2 Intermediate files on disk ……. Block k Block k Block k
Algorithm for drawing a scatter plot • Input the columns to be plotted, say A vs. B • Load a set of corresponding blocks for both columns, A and B. • Take corresponding data elements from both the blocks and plot them. • After plotting all the points, discard the blocks. • If there exist more blocks of data repeat step 2, else stop.
Advantages • The complexity for plotting is to O(2n) where n is the no. of rows. This complexity is independent of the no. of columns in the xml file. • If the user has to plot only a subset of data (as in case of zoom operation) then there exists another set of files which can be used for this purpose.
Dealing with subset of data • Data for every column is stored in an indexed fashion. • This helps in accessing the subset of data without having to go through the entire set of data. • As a result, operations like zoom become much faster.
Pre-processing operationCreation of tree blocks Col 1 Col 2 ……. Col m 1 2 . . Original xml file with m columns and n rows n Indexed File for col 2 Indexed File for col m 0.1–0.3 0-10 2-4 Indexed File for col 1 0.3–0.6 10-20 4-6 Intermediate files with indexed data . . . . . . . . . …….
Pre-processing operationCreation of tree blocks (contd) … 0-20 20-40 40-60 20-30 30-40 30-35 35-40 Indexed file for a column
Features of VOMegaPlot • Scatter Plot with zoom, reverse axis and logged axis • Projection Plot • Density Plot • Histogram
Future Enhancements • Support for reading data stored in binary format • Block level compression while creating intermediate files • Client Server version
References • VOTable http://www.ivoa.net/Documents/latest/VOT.html • VOPlot http://vo.iucaa.ernet.in/~voi/voplot.htm • VOMegaPlot http://vo.iucaa.ernet.in/~voi/vomegaplot.htm • IUCAA http://www.iucaa.ernet.in • Persistent Systems Pvt. Ltd. http://www.persistentsys.com
Sample VOTable <TABLE> <FIELD name=“RAJ2000” datatype =“double” ></FIELD> <FIELD name = “DEC2000” datatype=“double” ></FIELD> <DATA> <TABLEDATA> <TR> <TD>12.4524</TD> <TD>34.2331</TD> </TR> <TR> <TD>25.1321</TD> <TD>47.9055</TD> </TR> <TR> <TD>18.0723</TD> <TD>33.5802</TD> </TR> </TABLEDATA></DATA> </TABLE> Back