Jargon of the SRB

1. Jargon of the SRB Java Programming in the Data Grid. Lucas Gilbert

2. Introduction • JARGON is a pure java API for interacting with a data grid. • The API provides functionality matching the SRB C client API. Including file I/O for local and SRB file systems, querying and modifying metadata.

3. The Transparent Grid • The API’s structure, factory methods and programming model, unify diverse file systems into a single simple interface. • File handling exactly matches Sun’s java.io API. • Factory methods even hide the type of file system, e.g. if it is local or remote. • The API has been developed to allow for the easy inclusion of other grid file systems.

4. Class Structure • java.lang.Object   |   +-edu.sdsc.grid.io.General      |      +-edu.sdsc.grid.io.Local      |      +-edu.sdsc.grid.io.Remote         |         +-edu.sdsc.grid.io.srb.SRB         |         +-...(other filesystems)

5. Connecting to a File System • Create an account object with your user info • SRBAccount account = new SRBAccount( userInfo ); • Then connect to the SRB filesystem. • SRBFileSystem fileSystem = new SRBFileSystem( account );

6. General, Local, and SRB File • The filesystem object is used to instantiate new file objects • SRBFile file = new SRBFile( fileSystem, “/home/testuser.sdsc” ); • LocalFile localfile = new LocalFile( localParent, “test.txt” );

7. The FileFactory Class • The FileFactory class helps abstract away from a particular filesystem • GeneralFileSystem fileSystem = FileFactory.newFile( uri ); • GeneralFile file = FileFactory.newFile( fileSystem, “Test.txt” ); • GeneralFile file = FileFactory.newFile( uri );

8. Common File Operations • From the GeneralFile object, all the methods common to the java.io.File class are available, such as: • file.mkdir(); • boolean read = file.canRead(); • String[] list = file.list(); • Of course, SRB specific functionality requires an SRB declaration or casting the General object. • ((SRBFile) file).replicate( );

9. Reading and Writing a File • The GeneralFile.copyTo( GeneralFile ) can be used to copy files to/from any supported file systems. • Up/downloading using the SRB and a local system: • Large files will automatically use the SRB’s parallel file transfer protocol. • Directories will use the SRB’s bulk load protocol.

10. Reading and Writing a File • GeneralRandomAccessFile supports random access (read, write, seek) matching java.io.RandomAccessFile. • GeneralRandomAccessFile raf = FileFactory.newRandomAccessFile( file, "rw" ); • raf.write( new String( "This is a test.\n" )); • File stream classes are also available.

11. Query the file system. • Queries are simplified from SQL. • MetaDataCondition where = MetaDataSet.newCondition( SRBMetaDataSet.OFFSET, MetaDataCondition.LESS_THAN, 999); • MetaDataSelect select = MetaDataSet.newSelection( GeneralMetaData.FILE_NAME );

12. Query the file system. • An array of the conditions and selections forms the query. • MetaDataRecordList[] rl = fileSystem.query( conditions, selects ); • MetaDataRecordList[] rl = fileSystem.query( conditions, selects, numberOfResults, orderBy, nonDistinct );

13. Retrieve the results • Results from the query are return as a MetaDataRecordList[] rl • rl[0].getRecordCount(); • rl[0].getIntValue(0); • rl[0].getValue( GeneralMetaData.FILE_NAME );

14. Get further results from the query. • By default a query will only return the first 300 values. • MetaDataRecordList methods are used to retrieve further results from the query. • MetaDataRecordList[] getMoreResults() • boolean isQueryComplete()

15. Change the metadata of a file. • Change the metadata with a new or existing MetaDataRecordList and send it to the file system. • recordList.addRecord( SRBMetaDataSet.getField( SRBMetaDataSet.FILE_COMMENTS ), "new comments" ); • file.modifyMetaData( recordList );

16. Query the definable metadata. • Simliar to querying other metadata, but the MetaDataTable class is used instead of a scalar value. • For example, finding files with user definable metadata matching: aaa = 123 AND bbb <= 456

17. Query the definable metadata. • String[][] values = {{“aaa", "123"}, {“bbb", "456"}}; • int[] op = { MetaDataCondition.EQUAL, MetaDataCondition.LESS_THAN }; • MetaDataTable metaDataTable = new MetaDataTable( op, values );

18. Set the definable metadata. • Setting the definable metadata is the same as other metadata. • recordList.addRecord( SRBMetaDataSet.getField( SRBMetaDataSet. DEFINABLE_METADATA_FOR_FILES ), metaDataTable ); • file.modifyMetaData( recordList );

19. Online Examples • http://www.sdsc.edu/srb/jargon/ • Test.java • MetaDataTest.java • FactoryTest.java • CopyTest.java • Scommand-like examples • And more…

20. Jargon as Scommands For those familiar with the Scommands, see Test.java for a non-exhaustive list of Scommands and their Jargon equivalent.

21. Jargon Gui Components • SRB and local files viewable in a single tree • File metadata viewable and modifiable

22. SRB Content Manager Written by Daniel Moore

23. Where to Get More Info • http://www.sdsc.edu/srb/jargon • iktome@sdsc.edu • srb-chat@sdsc.edu