Python template engines and implementation in Indico

1. Python template engines and implementation in Indico Marius Damarackas (Vilnius University)

2. About me • Software engineering student • Doing 3 months long internship in CERN • Improving Indico template system

3. Outline • History of Indico templates • Why change the current system? • Main tasks • Research about available template systems • Integration of the new engine • Replacing XSLT with normal templates • Remaining work

4. History of Indico templates • First generation >>>'<div width="100%%">%(param)s</div>'%{'param':'Value'}'<div width="100%>Value</div>'

5. History of Indico templates • Second generation <% import some.module %> <% foobarList = some.module.createList()%> <table> <%forfooin foobarList: %> <tr> <%iffoo== bar: %> <td><%=foo%></td> <%end%> <%else: %> <td>foobar</td> <%end%> </tr> <%end%> </table>

6. Why change the current system? • Maintenance costs • Slow when including other templates • No cache • No HTML escaping • Strange usage of the colon symbols: <%latestNews=newsList[:5]%> <% end %>

7. 1. Research • Some of the candidates • Genshi • Mako • Cheetah • Jinja2

8. 1. Research • Genshi • Claims to be the fastest XML template engine • Used by Trac • XML and Text versions • Features like caching, XSS protections • Slow • Very difficult to translate Indico templates

9. 1. Research • Mako • Used by python.org and reddit.com • Very fast: compiles to Python modules • Allows to disable Unicode (a plus for Indico) • Template inheritance mechanism • Syntax could be better, more unified

10. 1. Research • Jinja2 • Quite fast • Template inheritance • Strong separation of logic and presentation • Difficult to adapt to Indico • Cheetah • Manual compilation to Python modules • Mako does it automatically • Personally I did not like the syntax

11. 2. Integration • Replace old engine code with TemplateLookup • Problem: more than 800 .tpl files to translate

12. 2. Integration • Using a script to convert files • Mainly based on regexps • Some things needed to be changed by hand

13. 2. Testing integration • Basic idea to test the integration: • use *both* old Indico and new Mako templates • Generate outputs and do diff • If there are significant differences, save them • Analyze differences and fix the script

14. 2. Testing integration • Coverage: 450+ out of 800 templates • 56%, but higher if we count only active files • Allowed to catch some bugs in the script

15. 3. Replacing XSLT

16. 3. Replacing XSLT • Generates XML, then applies transformations • Slow • Different from other parts of the system • Lots of code duplication • Difficult to maintain

17. 3. Replacing XSLT • Maintenance problems … if (chatInfoState]]></xsl:text> <xsl:value-of select="./id" disable-output-escaping="yes"/> <xsl:text disable-output-escaping="yes"><![CDATA[) { IndicoUI.Effect.slide('collaborationInfoLine]]></xsl:text> <xsl:value-of select="./id" disable-output-escaping="yes"/> <xsl:text disable-output-escaping="yes"><![CDATA[', height]]></xsl:text> <xsl:value-of select="./id" disable-output-escaping="yes"/> <xsl:text disable-output-escaping="yes"><![CDATA[); $E('CRMoreInfo]]></xsl:text> <xsl:value-of select="./id" disable-output-escaping="yes"/> <xsl:text disable-output-escaping="yes"><![CDATA[').set('More info'); $E('CRMoreInfo]]></xsl:text> <xsl:value-of select="./id" disable-output-escaping="yes"/> <xsl:text disable-output-escaping="yes"> <![CDATA[').dom.className = "CRDisplayMoreInfo"; …

18. 3. Replacing XSLT • Same code in human language … if (chatInfoState${id}) { IndicoUI.Effect.slide('collaborationInfoLine${id}', height${id}); $E('CRMoreInfo${id}').set('More info'); $E('CRMoreInfo${id}').dom.className = "CRDisplayMoreInfo"; …

19. 3. Replacing XSLT • Performance: • Changing the way parameters are passed • No XML • Maintenance: • Most of the styles are similar • Using Mako template inheritance

20. Remaining work • Think of a smart way to restructure event styles • Fix the i18n script to support the new syntax

21. Thank you