The CCP4 Package.

1. The CCP4 Package. General Overview.. presentation by Alun Ashton, awa@dl.ac.uk

2. On What Systems Will It Work? • SGi: Irix (o32, n32 and 64bit) • Linux Systems (Redhat, SuSE…..) • Compaq OSF1 etc • Sun (Sun OS) • HP (All apart from Xwindows) • IBM (AIX) • Mac OSX • Microsoft Windows • + presentation by Alun Ashton, awa@dl.ac.uk

3. Where It Can Be Downloaded • CCP4 ftp site: • ftp://ftp.ccp4.ac.uk ftp ftp.ccp4.ac.uk user name: ftp pw your@email.address.please cd pub/ccp4/current/ binary • CCP4 mirror sites: • Japan ftp://ftp.protein.osaka-u.ac.jp/mirror/ccp4 • USA ftp://ftp.sdsc.edu/pub/sdsc/xtal/CCP4 presentation by Alun Ashton, awa@dl.ac.uk

4. How BIG Is the Download? • unpacked size =160Mb • tar.gz size = 36 Mb • tar.Z size = 56 Mb • Binary size = 25 – 107 Mb presentation by Alun Ashton, awa@dl.ac.uk

5. Installation • Binaries or source code? • Unpacking • Directory structure • ccp4.setup • Configure • Source Code Make • Binary Make • Windows • Installing on multiple systems presentation by Alun Ashton, awa@dl.ac.uk

6. Installation II • You do not need root privileges on UNIX systems • Just enough space! • You DO need Admin privileges on Windows NT/2K • This is used to set up environment variables etc. • What else do you need to install CCP4? • Source code installation: • Compilers C and FORTRAN. • Make (UNIX) • Gzip/Uncompress/tar (UNIX) • ccp4i requires tcl/tk/BLT • AstexViewer requires a Java VM. presentation by Alun Ashton, awa@dl.ac.uk

7. Binaries or Source Code? presentation by Alun Ashton, awa@dl.ac.uk

8. Binaries or Source II? (Currently supported: irix, linux, OSF1, SunOS, Windows) Binaries or source? Fast Net? no yes Developer? yes no yes Supported Binaries? Reliable Compiler? no no yes Source Code and Compile Binary + Source code presentation by Alun Ashton, awa@dl.ac.uk

9. Unpacking • Download the file into an area with enough space to download and unpack it. • To uncompress a .gz file use gunzip • To uncompress a .Z file use uncompress • A tar file is a collation of many files: To extract the files from a tar file use: • tar –xvf file.tar presentation by Alun Ashton, awa@dl.ac.uk

10. Directory Structure ccp4_version.tar xtal/ccp4_version/ bin doc, examples, help, html, man, manual ccp4i etc include lib src unsupported x-windows CCP4 interface, Interface documentation, roadmaps CCP4 utility scripts ccp4.setup, environ.def and default.def Library src and data Program Source Code Older programs that are being phased out Programs with graphical user interface that require x-windows presentation by Alun Ashton, awa@dl.ac.uk

11. ccp4.setup (UNIX variants only) • All CCP4 programs require ccp4 environment variables set-up to enable them to run. • ccp4.setup files found in include sub directory • ccp4.setup-dist and ccp4.setup-bash • It is essential that the following environment variables are correctly assigned: • CCP4_MASTER (where the ccp4_vers.tar was unpacked) • CCP4_SCR (CCP4 scratch directory), • CCP4I_TCLTK (where ccp4i finds bltwish) • Most other environment variables use these variables. • source ccp4.setup before configure and running CCP4 programs • Automatically source ccp4.setup in login files (.cshrc or .login) presentation by Alun Ashton, awa@dl.ac.uk

12. ccp4.setup II Environment variables There is no need to learn these (and there are more) unless things go wrong! presentation by Alun Ashton, awa@dl.ac.uk

13. configure (UNIX variants only, when making from source code) • Configuring allows system specific flags to be assigned • Configure gives you flexibility, and a quick and easy way to change all the important make files • Configure settings can be overridden by environment variables • configure + Makefile.in  Makefile + config.status • Once the system has been configured you can either make the source code or install the binary files. • A log of the configure settings are in config.status . • More than one configure is run by $CCP4/configure presentation by Alun Ashton, awa@dl.ac.uk

14. configure (II) configure [--flags] [operating system] presentation by Alun Ashton, awa@dl.ac.uk

15. Source Code Make (installing from source code only) • Once you’ve downloaded the package, unpacked, set the environment variables and configured the package you are now ready to make the suite! • In $CCP4 type make. • You might want to keep a log with make > make.log • Make will firstly make the library components before the programs. • Once complete you will need to install with • make install. presentation by Alun Ashton, awa@dl.ac.uk

16. Source Code Make II. • The make utility uses the settings in the Makefile which in turn was created from the Makefile.in by the configure script. • $CLIBS/fsplit – a small program used to break large library.f files into smaller files – this was needed on older systems. • Make in the x-windows directory. • The make in the x-windows directory will ignore errors in compiling programs and will carry on regardless. • Once complete you will need to install with make install. • This will install the programs into $CBIN • Moves the library files into $CCP4_LIB / $CLIB • Make also has numerous other options. presentation by Alun Ashton, awa@dl.ac.uk

17. Source Code Make III make [option] presentation by Alun Ashton, awa@dl.ac.uk

18. Source Code Make IV (reference only) presentation by Alun Ashton, awa@dl.ac.uk

19. Source Code Make V - Compilers • Under the majority of operating systems the native compilers should be used. • The GNU compilers gcc and g77 can used on many systems • Redhat 7.0 and 7.1 users should upgrade their gcc/g77 • Use gcc 3.0.<4 • Linux users can also use a variety of other compilers. • See recent ccp4bb conversations • (e.g Portland Group and Intel compilers). • Irix 6.5 users should not use f77 version 7.2.* • We currently use 7.3.1.2m presentation by Alun Ashton, awa@dl.ac.uk

20. Binary Code Make (UNIX variants only) • You should have: • $CCP4/BINARY.readme • $CCP4/BINARY.setup • $CCP4/bin/ • You need to: • Get normal distribution and unpack it • Edit and source ccp4.setup • Run the script BINARY.setup • See also http://www.ccp4.ac.uk/dist/INSTALL.html • BINARY.about file in ‘binary’.tar.gz presentation by Alun Ashton, awa@dl.ac.uk

21. Windows • Tested on Windows 95/98/Me/NT(4.0)/2000 • You will need Admin privileges • Windows NT and 2000 • Use .exe auto install • Windows 95/98/Me variants • Use compressed .zip file • You will also need Tcl/tk and BLT for ccp4i (install before ccp4) • http://www.ccp4.ac.uk/dev/ccp4onnt.html • Cygwin is possible. presentation by Alun Ashton, awa@dl.ac.uk

22. Installing on Multiple Systems • Multiple systems means e.g. Linux and Irix with shared/mounted disks • Install completely independent source tree • Use duptree (found in $CCP4) • duptree must be run before any files are altered in $CCP4 • duptree creates a second identical source tree using links • Use --srcdir • Why? • Size matters. • You only need to bug fix one src. presentation by Alun Ashton, awa@dl.ac.uk

23. Bug Fixing and Patching. http://www.ccp4.ac.uk/problems.html presentation by Alun Ashton, awa@dl.ac.uk

24. More Information • CCP4 manual. • CCP4 website. • http://www.ccp4.ac.uk/dist/INSTALL.html • CCP4 problems pages. • ccp4@ccp4.ac.uk presentation by Alun Ashton, awa@dl.ac.uk

25. Now What Have You Got? • The programs • How they communicate • CCP4 Data formats • How to use the programs • The main programs • What is new in 4.2 presentation by Alun Ashton, awa@dl.ac.uk

26. The Programs • Most programs written in standard(ish) fortran77 • Over 150 programs mostly runnable from command line • Full listings on: • http://www.ccp4.ac.uk/dist/html/INDEX.html • http://www.ccp4.ac.uk/dist/html/FUNCTION.html presentation by Alun Ashton, awa@dl.ac.uk

27. How They Communicate? Standard data formats that contain information and informative log outputs. The data formats are publicly known, documented and useable by anyone. Nobody is forced to use our data formats as we supply numerous conversion tools. Some CCP4 programs already output key information in XML files. presentation by Alun Ashton, awa@dl.ac.uk

28. CCP4 Data Formats. • Formats used are: • .mtz – reflection data • .map .msk – map and mask formats • .pdb – atom positions • .cif – harvest files, refmac5 restraints • Binary formats were used for: • Size • Security • MTZ/map files are binary but portable as they contain a machine stamp • More information in the FILE FORMATS section on • http://www.ccp4.ac.uk/dist/html/INDEX.html presentation by Alun Ashton, awa@dl.ac.uk

29. CCP4 Data Formats II – MTZ Reflection Files • Table of data • columns - I, F, sigma, phase, FOM, etc. etc. • rows - reflections • Columns grouped as datasets with • project name GerE • dataset name se_inflection • cell dimensions 108.7 61.6 71.6 90.0 97.1 90.0 • wavelength 0.981 • In ccp4i, see: Reflection Data Utilities -> Edit MTZ Project&Dataset presentation by Alun Ashton, awa@dl.ac.uk

30. CCP4 Data Formats III – MTZ Reflection Files II MTZ file Title/history Spacegroup The future .... Crystal 1 Crystal 2 Crystal name Project name Cell dimensions Crystal name Project name Cell dimensions Heirarchical view of reflection data. Dataset 1.1 Dataset 1.2 Dataset name Wavelength Dataset name Wavelength Column Column

31. CCP4 Data Formats IV – MTZ Reflection Files III • Header information TITLE make ccp4itut mtz inconspicuous Sfs from refined RNA_SANAT 8-7. NCOL 8 18077 0 CELL 64.8970 78.3230 38.7920 90.0000 90.0000 90.0000 SORT 1 2 3 0 0 SYMINF 4 4 P 19 P212121 PG222 COLUMN K H 0.0000 42.0000 COLUMN L H 0.0000 21.0000 COLUMN FP F 0.0000 1219.0000 • Keyword – value pairs • TITLE – title • NCOL – no. of data columns • CELL – global cell parameters • SYMINF – symmetry info • COL – column information RESO – resolution range NDIF – no. datasets DATASET – dataset info PROJECT – harvesting info BATCH – batch images presentation by Alun Ashton, awa@dl.ac.uk

32. CCP4 Data Formats V – MTZ Reflection Files IV • Viewing and manipulation • mtzdump – view ascii dump of mtz file • mtz2various – convert mtz file to MULTAN, SHELX, TNT, X-PLOR/CN* formats • mtzutils, cad, sftools – edit mtz files • scaleit, fhscal – column scaling • http://www.ccp4.ac.uk/dist/html/FUNCTION.html presentation by Alun Ashton, awa@dl.ac.uk

33. CCP4 Data Formats VI – MAP Files - Densities Fourier map mtz phases • fft –switch between electron density and reflection data • mapdump – give information on map • View map with – O, PyMol, Mapslicer, AstexViewer • Format – header + symops + data • Binary - but machine independent. presentation by Alun Ashton, awa@dl.ac.uk

34. CCP4 Data Formats VII – Atom Positions • PDB files: • Based on pdb version 2.1 draft • mmCIF • CCP4 uses a subset of the mmCIF dictionary • Used in; • Restraints library of Refmac5 and libcheck • mmdb library • Harvesting files presentation by Alun Ashton, awa@dl.ac.uk

35. How To Run The Programs • ccp4i – the ccp4 interface • Command line options: • program [<logical name> <file name>] • e.g. fft hklin native-Fs.mtz mapout 2Fo—Fc.map • Keyworded input: • Keyword <options> • E.g. CELL a b c [  ] TITLE this job will work this time presentation by Alun Ashton, awa@dl.ac.uk

36. How To Run The Programs II Common Keywords presentation by Alun Ashton, awa@dl.ac.uk

37. The Main Programs • What do you get? • >150 programs encompassing the majority of protein crystallography – including: • Mosflm, Scala, Truncate, Mlphare, Rantan, Revise, Acorn, Oasis, SC, Amore, Molrep, Beast, DM, Fffear, Refmac5, Procheck, Sfcheck • See http://www.ccp4.ac.uk/dist/html/INDEX.html or http://www.ccp4.ac.uk/dist/html/FUNCTION.html presentation by Alun Ashton, awa@dl.ac.uk

38. What Is New in 4.2 • ACORN ab initio procedure for the determination of protein structure at atomic resolution (Yao Jia-Xing) • BEAST Brute-force molecular replacement with Ensemble Average Statistics, Maximum likelihood-based molecular replacement (Randy Read) • PROFESSS determination of NCS operators from heavy atoms (Kevin Cowtan) • ROTAMER list amino acids whose side chain torsion angles deviate from Richardson's Penultimate Rotamer Library (Dirk Kostrewa) • ASTEXVIEWER Java application for viewing proteins, ligands and electron density maps (Mike Hartshorn) • Announcement on: http://www.ccp4.ac.uk/news/msg00098.html presentation by Alun Ashton, awa@dl.ac.uk

39. CCP4i Introduction An Introduction to CCP4i The CCP4 Graphical User Interface Author: Peter Briggs CCP4 presentation by Alun Ashton, awa@dl.ac.uk

40. Why Use CCP4i? • Offers user friendly interfaces to the programs • Tools for file viewing & “project management” • Integrated help system How to get started • (Unix/Linux) Type ccp4i at the command prompt • (Windows/NT) Select ccp4i via the Start menu presentation by Alun Ashton, awa@dl.ac.uk

41. Main CCP4i Window On-line help Modules Tasks Job Database Tools & Utilities presentation by Alun Ashton, awa@dl.ac.uk

42. Modules • Access the module list by clicking on the gold bar displaying the current module presentation by Alun Ashton, awa@dl.ac.uk

43. Modules II • Tasks used in a particular part of the structure determination process • Utility tasks for manipulating different types of data • Alphabetical list of programs/tasks presentation by Alun Ashton, awa@dl.ac.uk

44. Tasks • Click on the appropriate button in the list to start a particular task • This starts the interface for that task presentation by Alun Ashton, awa@dl.ac.uk

45. Example Task Interface Protocol folder Make the key decisions File folder Set input and output file names Open folders Parameters that should be checked by the user before running Highlights indicate compulsory input WORK FROM THE TOP DOWN Closed folders Advanced/infrequently used options Always add a title to distinguish different runs of the same task Save/restore parameters Run task Defaults - “If it’s not visible then it’s not important” presentation by Alun Ashton, awa@dl.ac.uk

46. Running a Task Watch the progress of the job in the Job Database window: presentation by Alun Ashton, awa@dl.ac.uk

47. View Output From Job • View graphs in logfile using loggraph • - also use loggraph <filename> at the command prompt • View logfile in Netscape (if it contains HTML tags) and/or text browser • View input and output files (.mtz, .pdb, CCP4 maps) using appropriate viewer • - also use the View Any File option) • - or ccp4i -v <filename> at the prompt presentation by Alun Ashton, awa@dl.ac.uk

48. Project Management Using CCP4i • Why bother? • Benefits: • - remind you what you did six months ago • - keep track of multiple projects and associated data • - facilitate “back-tracking” • - make it easier to deposit your results/write your paper presentation by Alun Ashton, awa@dl.ac.uk

49. Project Directories One word alias ... … for project directory containing data files “All data files relating to one crystallographic project should be in a single project directory” presentation by Alun Ashton, awa@dl.ac.uk

50. Job Database • Each project directory has an associated job database accessed through the central panel in the main window: • This displays: • - which tasks were run, and when • - their status (RUNNING, FINISHED, FAILED etc) • - the title entered by the user • The database also keeps a record of: • - the parameters used to run the task • - the input, output and log files associated with the task presentation by Alun Ashton, awa@dl.ac.uk