1 / 56

What's New in Hydstra

What's New in Hydstra. Peter Heweston - Aug 2009. Strategic Drivers. .Net 64 bit Windows Kisters TSM Water Act 2007 Modelling and Data Use System Enhancements. .Net. .Net. Microsoft is pushing to .Net Language independence VB, C++, C#, J#, Delphi

nuala
Download Presentation

What's New in Hydstra

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. What's New in Hydstra Peter Heweston - Aug 2009

  2. Strategic Drivers • .Net • 64 bit Windows • Kisters TSM • Water Act 2007 • Modelling and Data Use • System Enhancements

  3. .Net

  4. .Net • Microsoft is pushing to .Net • Language independence • VB, C++, C#, J#, Delphi • Rich base class library (I/O, graphics, database, XML etc) • Security • Web orientation • Interactive web sites using ASP.NET, Silverlight, etc • Web services • Simplified deployment • No 'DLL hell' • Different versions can run side by side • Optimisation for different CPUs is possible

  5. Delphi and .Net • Delphi 2007 provided dual mode compilation - Win32 and .Net • Hydstra 9.6 and Hydstra 10 are compiled with Delphi 2007 • Microsoft hired the lead Delphi designer • Anders Hejlsberg (Turbo Pascal, Delphi etc) • C# conceptually very similar to Delphi • We had virtually completed a .Net port of Hydstra • Using conditional compilation to preserve single source code

  6. Delphi and .Net • May 2008 Delphi sold to Embarcadero Technologies • Embarcadero announced two products • Delphi 2009 - continues the Delphi path for Win32 • Delphi Prism - new .Net compiler • Problem - no VCL for .NET in Prism at this stage • We would need to redevelop all GUI objects - a major task • Hydstra will remain Win32 for a while longer, until the fog clears • Win32 development continues apace

  7. TSM Integration • For a while we were stuck in our TSM integration because of compiler limitations • Recently Embarcadero announced unmanaged exports facility • This allows a .Net DLL to expose itself to the Win32 world as a native Win32 DLL • Allows for object links between Hydstra and TSM • Integration effort can re-commence

  8. 64 Bit Windows

  9. 64 Bit Windows • Many new servers come loaded with 64 bit Windows • All device drivers need to be rewritten for 64 bit • 32 bit apps run under 64 bit just fine • 16 bit apps do NOT • Hydstra 9 does NOT run under 64 bit windows • Hydstra 10 runs fine, as a 32 bit app • 64 bit Delphi coming in 2010 (maybe)

  10. V10 32-bit Tools and Utilities • All 16 bit command line tools and utilities have been upgraded / replaced and are now 32-bit • All tools deal correctly with long file names • Hydstra V10 runs under 64-bit Windows • Users should not notice any difference unless they are writing advanced batch jobs • Scripts in INIPATH may need attention

  11. Long File Names and Spaces • Hydstra V10 should uniformly deal with long file names everywhere • Hydstra runs under a long directory root but • No Spaces • Like C:\Program Files or C:\Documents and Settings • Every job would need to quote every parameter and they don’t

  12. Finding 16 Bit References • HYFIND.HSC is delivered in Hydstra V10 • Available for 9.6 – ask us • Download from http://hydstra.kisters.com.au/index.htm • Searches INIPATH for obsolete tools • Help file documents tool changes • Help file suggests workarounds * Patchable to V9.6

  13. TSM

  14. TSM Integration • Time Series Manager is the Kisters server that will eventually underlie all Kisters products • TimeStudio, Wiski, Hydstra, AquisNet, Belvis, … • Much work in Hydstra is being done to align concepts with TSM • Basic data tree • Datasources • Variables • etc

  15. All Database Tables in DBF • Having tables scattered across multiple folders makes it difficult to manage progress towards modern server-based databases and .Net • All tables now live in DBFPATH • UPGRADE copies them there for you • All users get all tables - simplifies UPGRADE • Scripts and other commands that access obsolete paths will display warnings, and will very likely require attention • Obsolete paths are: • SECTPATH, RATEPATH, GAUGPATH, QDBPATH, RESPATH, GDBPATH, LDBFPATH, OPSPATH, OSYSPATH • HYFIND option [V10_BADPATHS] will find them

  16. Alphanumeric Variables – Phase One • V10 sees the introduction of phase one of alphanumeric variables. • Don't need to do anything immediately • Alphanumeric variables fit better with TSM concepts • Provide larger variable space for biologicals • During the life of V10 users will need to decide about alphanumeric names, subvariables • Separate presentation

  17. Support for Corporate Databases • Hydstra V10 should run over mainstream databases • Oracle • PostgreSQL • SQL Server • Not sure about all script tools • Time series remain in TS files • Perl data sources only work on TS files. • Will cost more $$$ to migrate • Need friendly user to help us finalise testing

  18. V10 System Enhancements

  19. Look and Feel • All forms and dialogs respect Windows Vista themes:

  20. HYXPLORE Enhancements • HYXPLORE has several enhancements • Run HSC jobs by name from Run box (.HSC optional) • Run HYMANAGE jobs by name from Run box • History of Run box preserved over runs • List locked TS files • Test path permissions • Open HYDLOG.TXT • Prevent accidental saving (and loss) of menus • Doesn't try to register .HSC • XDOCs - external supplied documentation • user extensible to access other documentation • not site related

  21. HYMANAGE Enhancements • HYMANAGE has many usability enhancements • Drag column borders in browse mode • Consistent colours in form and browse mode • Preserves filters on multiple tables within a manage • Quickfind on primary key or sorted column • Replace field value for filtered records (constants only) • Enhanced Statistics summarises text fields • Classify by unique values shows count of all values • Load saved queries from command line • HYMANAGE RATINGS /QUERY=RATEPER.FLOWRATINGS

  22. HYMANAGE menus now more standard • The menu bar in HYMANAGE now conforms more closely to Hydstra and Windows conventions • Reports are now created via File | Print • Hyconfig and Hydebug under Tools • Fonts under View

  23. Live Code Update • This feature has been available since 9.6 • A TS=3 user can add a new code from any coded field • Add a 'wrong value' and tab out of field • Dialog appears to help you add a new code

  24. New CSH Format Supported • CSH=CSV with Headers • Exported by HYMANAGE and HYDBUTIL • Imported by HYMANAGE and HYDBUTIL • Easier than XML and still allows data to possibly cross Hydstra versions • Example

  25. SVRIMP Enhancements • SVRIMP has been reworked to significantly improve reliability and performance • Marshals incoming files by site to reduce Hydstra file access • Processes 10-20 files/sec sustained • Preserve TS file edits (usually) • Much improved backlog processing • Restructured INI tab • Suggest new settings • Folder Scan Limit=5000 • Batch Processing Limit=100 • Shut down with SVRIMP /K for backups and replications

  26. SVRIMP Enhancements • Version 2 CSV format supported • #V2, site, datasource, variable, time, value, quality, datatrans, maxgap, DECUM, comment • Send files to different datasources • V1 format still supported

  27. Ordered Site Lists • Some site list expressions preserve order and dupes • GROUP, FILE, PERLLIST,CONCAT • Allows for run of river lists etc • Use SORT if you don’t care • SORT(FILE(c:\temp\sites.csv)) • Warning some existing lists may now allow duplicates • Wrap in SORT() to fix

  28. HYSTNS from Clipboard • New Perl script HYSTNS.COPY.PL copies HYSTNS list from clipboard • Almost any delimiter is recognised • for example copy the following text • hydsys01, 410776 / / 201235a##310996b456722301876,hydsys01 • Then run HYSTNS SORT(PERLLIST(HYSTNS.COPY.PL))

  29. Text Editing of GROUPS • You can now edit GROUPS lists using a text editor • Support copy and paste • More convenient than adding them row-at-a-time

  30. VARIABLE • VARIABLE table updated to prepare for Alphanumeric variables in V11 • Proposed Alpha ID field preparing for V11 • DOMAIN field to classify variables • Parameter Screens have new variabletree object to look up variables • By Domain • by Report Type • by Units • by word (with quicksearch) • Report Decimals

  31. Web style plots from HYPLOT • HYPLOT is now able to produce “bare” plots for use in web pages. • Allows web designers to create titles and labels using HTML • May do other ‘widget’ style plots as required

  32. HYDATSUM – Show Data Holdings • HYDATSUM has been enhanced to show WQ and GW holdings

  33. HYCLIPIN – Import from Clipboard • Import time series or database data from Excel • Export database data to Excel • Allows Perl processing step • Easy to build standard data entry sheets for time-series or tables • Useful for Exceldata manipulation of tables

  34. HYAUDIT Enhancements • Post-processing script HYAUDIT.XMLSPLIT.PL XML splits XML into single files • HYAUDIT.SUMREP.HSC gathers selected files together into a colour-coded intelligent HTML report • Click on errors in red to see full text • Open offending site in WB, HYMANAGE • Rerun HYAUDIT for that site whenerror has been fixed

  35. HYAUDIT Helpers • HYAUDIT.STNINILIM.HSC computes limits from data and loads them into STNINI • Should only be run on clean data • HYAUDIT.STNINISET.HSC loads constants into STNINI • Ready for use in HYAUDIT

  36. HYREP CHECK • Checks TS files for problems • Zero length files • Corrupt TS file (invalid flag byte) • Unregistered variable          • Unregistered quality code • Invalid datatrans code • Index points to a block which does not exist • Overlaps in index • Also leaves a CSV file in TEMPPATH HYREP.CSV • Useful in highly automated systems

  37. HYWOTSUP • Scans a variety of log files and reports errors • Configure where to look for log files • svrimp=&hyd-logpath.svrimp\ • hydlog=&hyd-dbfpath.hydlog\ • path01=&hyd-temppath.hydlog.txt • Run it daily to find SVRRUN problems

  38. HYDBSQL • Perform SQL queries on (Foxpro) tables • Supports quite complex one-line SQL statements • SELECT STN,MEAS_DATE,M_GH,FLOW FROM GAUGINGS WHERE FLOW=(SELECT MAX(FLOW) FROM GAUGINGS GROUP BY STN) GROUP BY STN • Example

  39. Changes to the Format of Binary TS Files • We have made some changes to the format of binary time series files • V10 TS files not readable by V9 • V9 files are readable by V10 • Related agencies better upgrade together • Values are now stored and read back rounded exactly to your nominated precision • 0.125m, stored to 0.001m precision, will come back as 0.125m, not .012496431 • Alphanumeric variables will be fully released in V11 • We are defining the binary tags in advance, as part of V10

  40. New Lossless Compression V9 V10

  41. Storage space consequences • Reduced the space required by Hydstra to store typical time series data. Expected savings: • Typical water level data (with point-removal compression) 5-15% reduction • Typical water level data (no compression)25-35% reduction • Typical rainfall data 20-25% reduction • DIPNR archive:15% reduction • Lots of small blocks:size may increase

  42. HYSECPIC • Enhanced to allow coloured ground:

  43. Daily River Report • As a project for DWE we implemented DWRRR, the Daily River Report • Reports by catchment in stream order for the last 7 days • Configurable via INI file • Rebuilt ever 30 mins in the morning • FTP output to web, email process report to manager • Example

  44. Hydstra/WEB • We have released Hydstra/WEB and have a couple of clients up and running: • One in Australia • One in Italy

  45. TSADMIN manage • TSADMIN manage centralises all TS related tables • Used by a TS=2 or TS=3 data manager • Variables • Qualities • Groups • Generalised Logger configuration • Digitiser related tables • ARCHSIZE • etc

  46. GWREP • GWREP produces a nice HTML report from GW tables • Example

  47. HYPERUPD • Writes actual datasource for TS files • Keep separate PERIOD entries for A and T files • Update for composite datasources much faster • Example

  48. TS File Locking • Time-series files are now logically locked while they are being written • Locks last over multiple operations, preventing interruption • Should make SRVRUN triggers more robust • HYXPLORE option shows locked TS files • Lock files exist in TS directory alongside TS files • C:\hydstra\systems\g\HYD\DAT\HYD\HYDSYS01.A • C:\hydstra\systems\g\HYD\DAT\HYD\HYDSYS01.AX • C:\hydstra\systems\g\HYD\DAT\HYD\HYDSYS01.A.LOCK(READ) • C:\hydstra\systems\g\HYD\DAT\HYD\HYDSYS02.A • C:\hydstra\systems\g\HYD\DAT\HYD\HYDSYS02.AX • C:\hydstra\systems\g\HYD\DAT\HYD\HYDSYS02.A.LOCK(READ)

  49. HYCSV.DETAILS.PL • Postprocessor on HYCSV • Documents variables and qualities output • Publishing HYCSV data on the web • Sending HYCSV data to other users • Example

More Related