Free Pascal compiler internationalisation

1. Free Pascal compiler internationalisation Rimgaudas Laucius Institute of Mathematics and Informatics, Vilnius University Lithuania

2. Introduction • Institute of Mathematics and Informatics, Informatics Methodology Department • Software localisation • Teaching of informatics and programming • E-learning and standards • Informatics terminology • Vilnius University • Localisation course

3. Localisation in Lithuania • One of the four priorities emphasised in the strategic project for the development of the information society in Lithuania is: “to uphold the inheritance of Lithuanian language and culture implementing the information technologies and telecommunications”

4. Open Source in Lithuania • Research which was carried out in 2004, “Open Source in Education” revealed that open source software integration into education has a large positive economical and also pedagogical effect • Education requires high quality and fully localised software • Open source software is more flexible in terms of localisation

5. Free Pascal compiler • Excellent, open source compiler • Works under all widely used operating systems Windows, Linux and others • Widely used. Has been used in International, Baltic and national Lithuanian Olympiads in informatics for a few years already • Replacement for obsolete Turbo Pascal system in Lithuanian schools

6. FPS

7. Compilers’ internationalisation • Internationalisation is part of the software development process, so the internationalisation of development tools is very important • Most contemporary software development tools are not internationalised enough • Though this research is done on Free Pascal compiler, most of represented issues are common to most of compilers

8. Programming language standards • Internationalisation relates with programming language standards • Pascal programming language standards • Standards of other languages

9. Examples of internationalised compilers • There are not many of these examples • One of the most well known internationalised programming system is LOGO • Vector Pascal

10. Structure of Free Pascal • Free Pascal is system made up of the compiler program itself and run-time library (RTL) • Compiler and RTL interaction: • Sometimes to change compiler one needs to change the RTL

11. Support of multilingual source code • This is the first stage of compiler internationalisation • There are many scripts which require more than the 8-bit character set

12. UTF-8 implementation • Unicode ~ UTF-8 • Some utilities used by compilers do not support pure Unicode (Unicode chars may be treated as pairs of 8-bit chars; example U+0900 ~ 09 00, (tab and end of string)) • Allows step by step implementation of lexical extensions

13. Lexical extensions • Strings • Identifiers • Directives • Reserved words • Operators • Numbers

14. Strings • WideString implementation issues • Compatibility with other systems • Ambiguity • Conversions between Unicode and other character sets

15. Ambiguity example procedure go(const s: WideString); begin ... end; procedure go(const s: String); begin ... end; beginGo('Hi'); end. Which overloaded procedures have to be called?

16. Unicode support layer • Unicode support layer wraps OS APIs’ in an OS independent way. • Under Win9x implements Microsoft Layer for Unicode (MSLU)

17. Identifiers • Identifiers have to reflect clear meaning of object, be easy to comprehend and memorize. Best way to support these features is to allow use of identifiers written in vernacular language • Unicode Standard Annex #31: Identifier and Pattern Syntax

18. Directives • Names • Parameters • Logical (ON, OFF) • Strings ({$warning Possible malfunctioning}) • File names ({$includepath ..\inc})

19. Reserved words • Unification myth • Compared 13 similar programming languages (Algol, Pascal, Modula, Ada, C, Java,…) • Only ~3% of reserved words are same • 56% met only in particular language • Possible unambiguous translation

20. Example of localised reserved words

21. Operators • Unicode has all mathematical symbols needed to express mathematical operations • Example:

22. Numbers • There are various scripts to express decimal numbers. • Example:

23. Decimal separator • JAV, GB ‘.’ • Most European countries ‘,’ • Localisation of delimiter may cause ambiguity. Solution needs to extend syntax of numbers. 25,88 – real number 25, 88 – two numbers

24. Punctuation • Spaces: general U+0020, nonbreaking U+00A0, ideographic U+3000, etc • Quotes: “English”, "Lithuanian“, • Etc

25. Bi-directional text • Bi-directional text is an issue of text representation, not the compiler

26. Unicode file names support • Handling of files requires OS API, so it have to be done via RTL’s Unicode support layer • Compilers have to use MSLU under Win9x

27. Input/Output • File input/output requires additional support for Unicode encoding • Windows console does not support Unicode • It can be replaced but is it the best solution?

28. Localisation framework • Strings and other resources have to be externalised for easy localisation • Localisation kits have to be prepared

29. Questions? • Thank you • Contact E-mail: