1 / 8

Elements of Revised Plan

Elements of Revised Plan. 25 April 2005. Constraints and Observations. Want HISTORY queries to work ASAP (LCG) Returning 10s of thousands to millions of tuples Archiver can run out of memory and crash! Want to see first tuples returning quickly

almira
Download Presentation

Elements of Revised Plan

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Elements of Revised Plan 25 April 2005

  2. Constraints and Observations • Want HISTORY queries to work ASAP (LCG) • Returning 10s of thousands to millions of tuples • Archiver can run out of memory and crash! • Want to see first tuples returning quickly • Need to always provide evolutionary path (LCG) • LCG making extensive use of R-GMA • Current gLite version does not offer enough advantages over LCG2_4_0 to be worth effort of switching • For web service release we help/advise porting apps • The main concern was the handling of very large history queries • LCG wants a fix ASAP • We need to make gLite version useful and include SD (Frederic) • LCG not obliged to utilise deployment modules Steve Fisher - Revised Plan

  3. Release 1.2 • No more work on LCG branch • Change gLite release 1.2 to allow deployment by LCG • Ignore compatibility with releases 1 and 1.1 • Assume that UI will use LCG2_4_0 or gLite as equivalent • Make authentication optional • Not mixed – just optional in deployment • Also good for non-LCG and we are ready if they change their mind after some security scare • Make service publication sensitive to the schema contents • Then LCG can change schema after deployment • Restore smalll schema changes – e.g. userTable • Decide upon isExecuting() / pop() / count() semantics • Explain how to loop Steve Fisher - Revised Plan

  4. Release 1.3 • Preparation for release 1.3 • Upgrade MySQL – and warn people in advance • Is WS chunking design good here or de we need another design? • Producers and consumers able to deal with chunking • Must also work with 1.2 release and/or with LCG2_4_0 by agreement with LCG • Also need to progress SD API before we get into problems with that! Steve Fisher - Revised Plan

  5. Closer involvement with LCG • Need a contact person • Will have weekly phone meetings with LCG • They should approve our plan • We need to understand if they will release these as bug fixes • Will the gLite label get in the way? • Convince LCG that it is an LCG bug-fix • Convince Erwin and Frederic that it is gLite • Do we still need RH7.3 clients? Steve Fisher - Revised Plan

  6. Chunking and count() • We have for count(): • Returns the number of tuples that are available to pop. While a query is still executing, the number of tuples available to a pop call may be greater than the number returned by a previous call to count, as tuples can be added in the intervening time. Once isExecuting returns false, the value of count can no longer increase, but will just decrease as tuples are popped. • This is evidently a problem as this implies that when isExecuting() is false then count() is the true count of tuples for that query Steve Fisher - Revised Plan

  7. popAll() • This says: • Retrieves all data from the consumer that it has received from producers. To guarantee that this returns the full response to a query, it should be called after the query has finished executing (when isExecuting is false). • This shows the same problem • We could change isExecuting to mean that all tuples are ready to pop – but this fails if someone waits for isExecuting to be false without popping tuples if they won’t all fit in the consumer buffer. • Such a loop should anyway be discouraged! Steve Fisher - Revised Plan

  8. Solutions • isExecuting() is of not much interest except that it shows the behaviour of the underlying query – we might usefully get rid of it! • Do we add inProgress() to indicate that there are still tuples to come • Or do we just change the meaning of isExecuting() • Currently we just have to tell a few people in LCG Steve Fisher - Revised Plan

More Related