How to Present your Data on Internet? A guide for beginners

1. How to Present your Data on Internet?A guide for beginners Martin Molhanec, M.Sc., Ph.D.

2. Abstract • How to get data. • How to save data. • How to manipulate with data. • How to present data on Internet.

3. How to get data • Directly MeasuringApparatus User PROBLEM: How to communicate with measuring apparatus? Usually low level communication!

4. How to get data • Indirectly Two steps: 1) Save data with the use of supplier program. 2) Present data saved in file or database. User MeasuringApparatus FileorDatabase

5. How to get data • Directly • Indirectly Two ways! MeasuringApparatus User FileorDatabase

6. How to get data directly • We must wrote a program! • We must understand the API! • Program language? • Visual Basic – Microsoft • C, C++ - Microsoft, Borland, etc. • Pascal (Delphi) – Borland • Assembler – Microsoft, GNU MeasuringApparatus User API Application Program Interface

7. How to get data directly • We must wrote a program • Program language • Visual Basic – Microsoft • C, C++ - Microsoft, Borland, etc. • Pascal (Delphi) – Borland • Assembler – Microsoft, GNU MeasuringApparatus User Not easy task! Very good programming practice! API Application Program Interface

8. Structure of program • Initialization • HW configuration and setup

9. Structure of program • Initialization • HW configuration and setup • While not end • Get data from measuring apparatus • Put data to database (or write to file) • End while

10. Structure of program • Initialization • HW configuration and setup • While not end • Get data from measuring apparatus • Put data to database (or write to file) • End while • Finishing • Closing of files, disconnection from database

11. Problems • Low level API • Assembler • Complicated communication • Handshake • Unknown programming language • You must to learn • Bad documentation • As usual • Close API • No information

12. Problems • Low level API • Assembler • Complicated communication • Handshake • Unknown programming language • You must to learn • Bad documentation • As usual • Close API • No information Not possible to advice! Too many ways!

13. How to get data indirectly • 1. phase: • put data to database (or to file) • Suppliers program • 2. phase: • show data on Internet • Our program MeasuringApparatus User Format of data FileorDatabase We must know!

14. Format of data • CSV • Comma Separated Values • Easy to understand • Simple • XML • eXtensible Markup Language • Complicated • Contemporary standard • Many tools to work with it

15. The CSV File Format • Each record is one line   ...butA record separator may consist of a line feed (ASCII/LF=0x0A), or a carriage return and line feed pair (ASCII/CRLF=0x0D 0x0A)....but: fields may contain embedded line-breaks so a record may span more than one line. • Fields are separated with commas.ExampleJohn,Doe,120 any st.,"Anytown, WW",08123 • Leading and trailing space-characters adjacent to comma field separators are ignored. So  John  ,   Doe  ,resolves to "John" and "Doe" Space characters can be spaces, or tabs.

16. The CSV File Format • Fields with embedded commas must be delimited with double-quote characters. In the above example. "Anytown, WW" had to be delimited in double quotes because it had an embedded comma. • Fields that contain double quote charactersmust be surrounded by double-quotes,and the embedded double-quotes must each be represented by a pair of consecutive double quotes. So, John "Da Man" Doewould convert to "John ""Da Man"" Doe"

17. The CSV File Format • A field that contains embedded line-breaks must be surounded by double-quotesSo:  Field 1: Conference room 1    Field 2:    John,    Please bring the M. Mathers file for review      -J.L.  Field 3: 10/18/2002  ... would convert to:   " Conference room 1" ,"John,    Please bring the M. Mathers file for review    -J.L.", 10/18/2002 • Note that this is a single CSV record, even though it takes up more than one line in the CSV file. This works because the line breaks are embedded inside the double quotes of the field.

18. The CSV File Format • Fields with leading or trailing spaces must be delimited with double-quote characters.So to preserve the leading and trailing spaces around the last name above: John ,"   Doe   ",... • Fields may always be delimited with double quotes.The delimiters will always be discarded. • The first record in a CSV file may be a header record containing column (field) namesThere is no mechanism for automatically discerning if the first record is a header row, so in the general case, this will have to be provided by an outside process (such as prompting the user). The header row is encoded just like any other CSV record in accordance with the rules above. A header row for the multi-line example above, might be:Location, Notes, "Start Date", ...

19. The CSV File Format • Example Data: Here is a small set of records that demonstrate some of the constructs discussed above. John,Doe,120 jefferson st.,Riverside, NJ, 08075 Jack,McGinnis,220 hobo Av.,Phila, PA,09119 "John ""Da Man""",Repici,120 Jefferson St.,Riverside, NJ,08075 Stephen,Tyler,"7452 Terrace ""At the Plaza"" road",SomeTown,SD, 91234 ,Blankman,,SomeTown, SD, 00298 "Joan ""the bone"", Anne",Jet,"9th, at Terrace plc",Desert City,CO,00123

20. Example 1. • How to read CSV file format in Visual Basic. Then open your CSV file for input as you would do for any other txtfile. Read a line and store it in a string, like OneLine. Then do this:data = Split(OneLine,",")The Split commands takes the string, cuts it ups according to what delimiter you chose (in this case comma) and stores the pieces in the variable as an array. So now data has become an array and contains following:OneLine contains "12/04/06,John Doe,36,176"data(0) will contain 12/04/06 data(1) will contain John Doedata(2) will contain 36 etc.

21. Example 1. • How to read CSV file format in Visual Basic. Dim delim as String = ","  'define a delimiterDim record as String     ' record storageDim recsplit() as string    ' splited record storage 'open the file.... 'while not end of the file    record = stringReader.ReadLine() 'read a record     recsplit() = Split(record, delim) 'split the record     'make your calculations.....    'each position of recsplit can be accesed normally ‘While end

22. Example 1. • How to read CSV file format in Visual Basic. Using MyReader As New Microsoft.VisualBasic.FileIO.TextFieldParser("tm.CSV") MyReader.TextFieldType = FileIO.FieldType.Delimited MyReader.SetDelimiters(",") Dim currentRow As String() While Not MyReader.EndOfData currentRow = MyReader.ReadFields() End While End Using Will be explained in more details!

23. Example 1. • How to read CSV file format in Visual Basic. Create a new TextFieldParser. The following code creates the TextFieldParser named MyReader and opens the file test.txt. Using MyReader As New _ Microsoft.VisualBasic.FileIO.TextFieldParser _ ("C:\TestFolder\test.txt")

24. Example 1. • How to read CSV file format in Visual Basic. Define the TextField type and delimiter. The following code defines the TextFieldType property as Delimited and the delimiter as ",". MyReader.TextFieldType = FileIO.FieldType.Delimited MyReader.SetDelimiters(",")

25. Example 1. • How to read CSV file format in Visual Basic. Loop through the fields in the file. If any lines are corrupt, report an error and continue parsing. The following code loops through the file, displaying each field in turn and reporting any fields that are formatted incorrectly. Dim currentRow As String() While Not MyReader.EndOfData Try currentRow = MyReader.ReadFields() ‘!!! Here is CVS reading !!! Dim currentField As String For Each currentField In currentRow MsgBox(currentField) Next Catch ex As Microsoft.VisualBasic.FileIO.MalformedLineException MsgBox("Line " & ex.Message & "is not valid and will be skipped.") End Try

26. Example 1. • How to read CSV file format in Visual Basic. Close the While and Using blocks with End While and End Using. End While End Using

27. Example 2. • How to read CSV file format in PHP. <?php // reads a csv file and returns a two-dimensional array of lines/fields function read_csv($file,$delimiter) {   $data_array = file($file); // read whole file to arrayfor ( $i = 0; $i < count($data_array); $i++ ) // process rows  {    $parts_array[$i] = explode($delimiter,$data_array[$i]);   } return $parts_array; } ?>

28. Example 2. • How to read CSV file format in PHP. // this willl display all records in the csv file $data = read_csv('read_csv.txt','|'); for ( $i = 0; $i < count($data); $i++ ) {   for ( $u = 0; $u < count($data[$i]); $u++ )   { echo $data[$i][$u].' '; if($data[$i][$u] == end($data[$i])) {     echo '<br>';   }   } } echo '<p>'; 

29. The XML file format • Features of XML • XML provides a text-based means to describe and apply a tree-based structure to information. • At its base level, all information manifests as text, interspersed with markup that indicates the information's separation into a hierarchy of character data, container-like elements, and attributes of those elements. • In this respect, it is similar to the LISP programming language's S-expressions, which describe tree structures wherein each node may have its own property list.

30. The XML - Quick syntax tour • The basic syntax for one element in XML is • <name attribute="value">content</name> TAG START ATTRIBUTEVALUE TAG END ATTRIBUTE CONTENT

31. The XML - Quick syntax tour • Here is an example of a simple recipe expressed using XML: • <?xml version="1.0" encoding="UTF-8"?> • <recipe name="bread" prep_time="5 mins" cook_time="3 hours"> • <title>Basic bread</title> • <ingredient amount="3" unit="cups">Flour</ingredient> • <ingredient amount="0.25" unit="ounce">Yeast</ingredient> <ingredient amount="1.5" unit="cups“ state="warm">Water</ingredient> • <ingredient amount="1" unit="teaspoon">Salt</ingredient> • <instructions> • <step>Mix all ingredients together, and knead thoroughly.</step> • <step>Cover with a cloth, and leave for one hour in warm room.</step> • <step>Knead again, place in a tin, and then bake in the oven.</step> • </instructions> • </recipe> Too complex example?

32. The XML - Quick syntax tour • Here is an example of a simple recipe expressed using XML: • <?xml version="1.0" encoding="UTF-8"?> • <recipe name="bread" prep_time="5 mins" cook_time="3 hours"> • <title>Basic bread</title> • <ingredient amount="3" unit="cups">Flour</ingredient> • <ingredient amount="0.25" unit="ounce">Yeast</ingredient> <ingredient amount="1.5" unit="cups“state="warm">Water</ingredient> • <ingredient amount="1" unit="teaspoon">Salt</ingredient> • <instructions> • <step>Mix all ingredients together, and knead thoroughly.</step> • <step>Cover with a cloth, and leave for one hour in warm room.</step> • <step>Knead again, place in a tin, and then bake in the oven.</step> • </instructions> • </recipe> The first line is the XML declaration: it is an optional line stating what version of XML is in use (normally version 1.0), and may also contain information about character encoding and external dependencies.

33. The XML - Quick syntax tour • Here is an example of a simple recipe expressed using XML: • <?xml version="1.0" encoding="UTF-8"?> • <recipe name="bread" prep_time="5 mins" cook_time="3 hours"> • <title>Basic bread</title> • <ingredient amount="3" unit="cups">Flour</ingredient> • <ingredient amount="0.25" unit="ounce">Yeast</ingredient> <ingredient amount="1.5" unit="cups“state="warm">Water</ingredient> • <ingredient amount="1" unit="teaspoon">Salt</ingredient> • <instructions> • <step>Mix all ingredients together, and knead thoroughly.</step> • <step>Cover with a cloth, and leave for one hour in warm room.</step> • <step>Knead again, place in a tin, and then bake in the oven.</step> • </instructions> • </recipe> The remainder of this document consists of nested elements, some of which have attributes and content. An element typically consists of two tags, a start tag and an end tag, possibly surrounding text and other elements.

34. The XML - Quick syntax tour • Here is an example of a simple recipe expressed using XML: • <title>Basic bread</title> • <ingredient amount="3" unit="cups">Flour</ingredient> The remainder of this document consists of nested elements, some of which have attributes and content. An element typically consists of two tags, a start tag and an end tag, possibly surrounding text and other elements.

35. The XML - Quick syntax tour • Here is an example of a simple recipe expressed using XML: • <title>Basic bread</title> • <ingredient amount="3" unit="cups">Flour</ingredient> The remainder of this document consists of nested elements, some of which have attributes and content. An element typically consists of two tags, a start tag and an end tag, possibly surrounding text and other elements.

36. The XML - Quick syntax tour • Here is an example of a simple recipe expressed using XML: • <title>Basic bread</title> • <ingredient amount="3" unit="cups">Flour</ingredient> The remainder of this document consists of nested elements, some of which have attributes and content. An element typically consists of two tags, a start tag and an end tag, possibly surrounding text and other elements.

37. The XML - Quick syntax tour • Here is an example of a simple recipe expressed using XML: • <title>Basic bread</title> • <ingredient amount="3" unit="cups">Flour</ingredient> The remainder of this document consists of nested elements, some of which have attributes and content. An element typically consists of two tags, a start tag and an end tag, possibly surrounding text and other elements.

38. The XML - Quick syntax tour • Here is an example of a simple recipe expressed using XML: • <instructions> • <step>Mix all ingredients together, and knead thoroughly.</step> • <step>Cover with a cloth, and leave for one hour in warm room.</step> • <step>Knead again, place in a tin, and then bake in the oven.</step> • </instructions> The remainder of this document consists of nested elements, some of which have attributes and content. An element typically consists of two tags, a start tag and an end tag, possibly surrounding text and other elements. Outer element Inner element

39. The XML - Quick syntax tour • XML provides special syntax for representing an element with empty content. Instead of writing a start tag followed immediately by an end tag, a document may contain the empty element tag where a slash follows the element name. The following two examples are functionally equivalent: • <foo></foo>  <foo />

40. The XML - Quick syntax tour • Correctness in an XML document • For an XML document to be correct, it must be: • Well-formed. A well-formed document conforms to all of XML's syntax rules. For example, if a non-empty element has an opening tag with no closing tag, it is not well-formed. A document that is not well-formed is not considered to be XML; a parser is required to refuse to process it. • Valid. A valid document has data that conforms to a particular set of user-defined content rules, or XML schemas, that describe correct data values and locations. For example, if an element in a document is required to contain text that can be interpreted as being an integer numeric value, and it instead has the text "hello", is empty, or has other elements in its content, then the document is not valid.

41. How to read the XML file format • Processing XML files • SAX and DOM are object oriented programming APIs widely used to process XML data. The first XML parsers exposed the contents of XML documents to applications as SAX events or DOM objects. • SAX is a lexical, event-driven interface in which a document is read serially and its contents are reported as "callbacks" to various methods on a handler object of the user's design. SAX is fast and efficient to implement, but difficult to use for extracting information at random from the XML, since it tends to burden the application author with keeping track of what part of the document is being processed. It is better suited to situations in which certain types of information are always handled the same way, no matter where they occur in the document. • DOM is an interface-oriented API that allows for navigation of the entire document as if it were a tree of "Node" objects representing the document's contents. A DOM document can be created by a parser, or can be generated manually by users (with limitations). Data types in DOM Nodes are abstract; implementations provide their own programming language-specific bindings. DOM implementations tend to be memory intensive, as they generally require the entire document to be loaded into memory and constructed as a tree of objects before access is allowed.

42. Where to save data • In text file • CSV • XML Easy way, but not good for next processing!

43. Where to save data • In text file • CSV • XML • In database • Relational database • Object oriented database Better way for next processing!

44. Which database? • Commercional • MS SQL Server • MS Access • Oracle • … • Free • MySQL • Postgress • …

45. Example 3. • We show, how to: • Managing MS Access. • Configuring ODBC. • Programming in Visual Basic.

46. Example 3. <% ' Old ASP syntax. Dim MyConn Set MyConn = Server.CreateObject("ADODB.Connection") ' New ASP.NET syntax. Dim MyConn MyConn = Server.CreateObject("ADODB.Connection") %>

47. Example 3. <% Dim RS As RecordSet ' Old ASP syntax (retrieving recordset column value).Set MyConn = Server.CreateObject("ADODB.Connection")MyConn.Open("TestDB")Set RS = MyConn.Execute("Select * from Products")Response.Write RS("Name") ' New ASP.NET syntax (retrieving recordset column value).MyConn = Server.CreateObject("ADODB.Connection") MyConn.Open("TestDB")RS = MyConn.Execute("Select * from Products")Response.Write RS("Name").Value %>

48. Example 4. • We show, how to: • Managing MySQL. • Configuring PHP. • Programming in PHP.

49. Example 4. <?php// Connecting, selecting database$link = mysql_connect('mysql_host', 'mysql_user', 'mysql_password')   or die('Could not connect: ' . mysql_error());echo 'Connected successfully';mysql_select_db('my_database') or die('Could not select database');

50. How to manipulate with data? • SQL • Structured Query Language. • Standard language for • Defining • Manipulation • Selecting • data in relational databases.