Python Eggs

1. Python Eggs

2. Overview • Part I: Installers • What are Eggs? • What Eggs can do for me? • Package installation with Eggs • Part II: Developers • Component overview • Package compatibility • Developing with Eggs • Part III: Demo

3. JARs What are Python Eggs? Eggs = + CPAN + Backwards Compatibility

4. package_1 package_2 code data EGG-INFO _____________.egg Eggs ≈ JARs • .egg zip archive • Drop into sys.path • pkg_resources API • What about • Dependencies? • Scripts? • C Extensions?

5. Eggs ≈ CPAN $ easy_install MyPackage>=2.1 ... find MyPackage v2.1 ... ... download MyPackage v2.1 ... ... download MyPackage dependencies ... ... build all ... ... install all ... $ python >>> import MyPackage >>>

6. Python Cheeseshop British Atmospheric Data Centre

7. easy_install examples $ easy_install MyPackage $ easy_install MyPackage==3.2 $ easy_install -f http://example.com MyPackage $ easy_install --upgrade MyPackage $ easy_install –d temp_eggs MyPackage-prealpha $ easy_install $SROOT/MyPackage/trunk#egg=MyPackage-dev $ easy_install -m MyPackage==3.2a1

8. Installing setuptoolsrecipe 1: globally • Ensure distutils is available • Ensure you can write python installation $ wget http://peak.telecommunity.com/dist/ez_setup.py $ python ez_setup.py

9. Installing setuptoolsrecipe 2: using PYTHONPATH • Ensure distutils is available • In ~/.profile export PYTHONPATH=$HOME/lib/python export PATH=$HOME/bin:$PATH • In ~/.pydistutilsrc [install] install_lib = ~/lib/python install_scripts = ~/bin $ wget http://peak.telecommunity.com/dist/ez_setup.py $ python ez_setup.py

10. Key Advantages • Self contained • Try new software easily • install / test / throw away • Track development • select version • download from SVN • Supports plug-in architectures

11. Part IIDevelopers

12. Egg components <<package>> pkg_resources <<package>> distutils dependencies resources metadata namespaces <<package>> setuptools <<script>> easy_install find download install upgrade build eggs <<egg>> setuptools <<script>> ez_setup.py

13. Distutils $ python setup.py install $ python setup.py install –prefix=$HOME $ python setup.py build $ sudo python setup.py install $ python setup.py build –build-opt install

14. Distutils • Problems • Insufficient documentation • Easier to shortcut than use properly • Consequences • Inconsistent setup.py interface • Monolithic frameworks

15. CDAT

16. cdat-lite / vcs-lite $ URL=http://home.badc.rl.ac.uk/spascoe/ndg_eggs $ easy_install –f $URL vcs-lite

17. Matplotlib

18. Matplotlib  • easy_install numpy • easy_install matplotlib • easy_install basemap • easy_install –f $SVN basemap   

19. M2Crypto $ easy_install M2Crypto • No way to use the –openssl option! • setup.py's key mistakes • Parses the –openssl argument outside distutils. • Sets include_dirs even if using default location.

20. It will probably work if you • Stick to the distutils framework! • including • argument parsing • install locations • use __file__ or __name__ to find resources

21. import ez_setup ez_setup.use_setuptools() from setuptools import setup, find_packages setup( name = 'MyPackage', version = '1.0', ... packages = find_packages('lib', exclude=["test_*"]), package_dir = {'' : 'lib'}, ... ) Setuptools: preliminaries

22. setup( ... install_requires = [ 'MyDep1>=0.3', 'MyDep2==1.2rc2', 'MyDep3==dev' ], dependency_links = [ 'http://www.example.com/mydep2', 'http://www.example.com/svn/mydep3/trunk#egg=MyDep3-dev' ], ... ) Setuptools: dependencies

23. setup( ... zip_safe = False, include_package_data = True, exclude_package_data = {'': ['README.txt']}, ... ) Setuptools: resources

24. setup( ... entry_points = { 'console_scripts': [ 'foo = my_package.some_module:main_func', 'bar = other_module:some_func', ], 'gui_scripts': [ 'baz = my_package_gui.start_func', ]}, ... ) Setuptools: scripts

25. Setuptools: and more • namespace packages • entry points • extras • SVN hooks • unit tests

26. pkg_resources API #udunits_name=sys.prefix+'/lib/python'+version+'/site-packages/unidata/udunits.dat' import pkg_resources udunits_name = pkg_resources.resource_filename('unidata', 'udunits.dat') import pkg_resources pkg_resources.require('ZSI==2.0rc1')

27. Using Eggs in NDG • Stripping out CDAT components • cdat-lite, vcs-lite • Splitting the ndg package into multiple eggs • namespace packages • Managing multiple versions • ZSI • Shared code in ndg_util.egg • Component bundles

28. Part III: Demo A WSGI compliant OpeNDAP / WMS server in exactly 0 lines of python

29. Demo script #!/bin/sh # Log commands to stdout set -x # Setup staging area rm -fr eggs myserver demo.log mkdir eggs export PYTHONPATH=$PWD/eggs export PATH=$PYTHONPATH:$PATH DAPSVN=http://pydap.googlecode.com/svn/trunk/ # Bootstrap setuptools python ez_setup.py -d eggs >>demo.log 2>&1 # Install pydap prerequisites easy_install -d eggs -f http://www.pythonware.com/products/pil \ numpy Imaging >>demo.log 2>&1 # Install pydap and plugins easy_install -d eggs \ dap[server] dap.plugins.netcdf dap.responses.wms \ $DAPSVN/responses/html >>demo.log 2>&1 # Create the server project paster create -t dap_server myserver project=My-Server >>demo.log 2>&1 # Move data to server directory cp *.nc myserver/data >>demo.log 2>&1 # Start the server paster serve myserver/server.ini British Atmospheric Data Centre

30. Demo paste.deploy configuration [server:main] use = egg:Paste#http # Change to 0.0.0.0 to make public host = 127.0.0.1 port = 8080 [filter-app:main] use = egg:Paste#httpexceptions next = cascade [composit:cascade] use = egg:Paste#cascade app1 = static app2 = pydap catch = 404 [app:static] use = egg:Paste#static document_root = %(here)s/data [app:pydap] use = egg:dap name = My-Server root = %(here)s/data verbose = 0 template = %(here)s/template British Atmospheric Data Centre

32. The End

33. References • http://peak.telecommunity.com/DevCenter/PythonEggs • http://peak.telecommunity.com/DevCenter/EasyInstall • http://peak.telecommunity.com/DevCenter/setuptools • http://peak/telecommunity.com/DevCenter/PkgResources • http://mail.python.org/mailman/listinfo/distutils-sig/