Welcome to the UCLA – IGERT 2005 Summer Bioinformatics Workshop

1. Welcome to the UCLA – IGERT 2005 Summer Bioinformatics Workshop IGERT – Sponsored Bioinformatics Workshop Series Michael Janis and Max Kopelevich, Ph.D. Dept. of Chemistry & Biochemistry, UCLA

2. Who this workshop is for Biologists: Integrating bioinformatics methods into their existing projects Bioinformatics students: Preparing for and augmenting existing bioinformatics coursework Computational researchers: Identifying and understanding common bioinformatics language classes and data structures

3. Why this workshop? • Current graduate curriculum: • Assumes substantial knowledge base upon entry into bioinformatics program • Researchers: • No general overview of computational research environments • some aimed at tools (suites), others deep in theory base

4. Why learn bioinformatics? • The 1 gene, 1 protein research model ignores the deluge of inferential biological data at our disposal • This information must be combined for any thorough study of biological phenomena • Assembly of qualitative (sequence, etc.) and quantitative (measured, inferential) data is beyond the capacity of a human to interpret in a systematic fashion • Computational methods must be addressed both to interpret and to share research data in a meaningful way – Yet Another Tool in a biologist’s arsenal

5. What is Bioinformatics? • A tool in a molecular biologist’s arsenal such as PCR, gels, or microarrays; or: • A support machine for handling and storing large data sets; or: • A discipline in its’ own right, enabled by a critical mass of biological data and mirroring the evolution of a number of scientific sub-disciplines

6. An argument for learning: as a tool for molecular biologists? • 1 gene, 1 protein becomes an interactive series of pathways and protein interactions • A pathway allows novel interaction discovery and ontological groupings of data • Moving from identification of a gene that has biological function to elucidation of a pathway from a given biological function (just for example)

7. An argument for learning: as support for data management? • Completion of the human genome working draft in June 2000 resulted in an absolute requirement for a logical, hierarchical way to store, modify, and retrieve data • Cross-referencing information is necessary to incorporate disparate and sometimes conflicting information • Combination of data and necessary algorithm development to find signal among the noise of the genome

8. An argument for learning: as an entirely new discipline? • Evolutionary biology through homology • (COG for example) • Gene expression and structure prediction • (alternative splicing for example) • Protein and ncRNA modeling • Genome mapping • (statistical genetics and disease correlations) • Pathway prediction • Towards an end: Systems Biology

9. What has bioinformatics allowed us to do? • Evolutionary Biology • Clusters of Orthologous Genes (COG database) http://www.ncbi.nlm.nih.gov/COG/ • Structure and alignments • PFAM/RFAM http://www.sanger.ac.uk/Software/Rfam/ • Expression inferences • clustering to find interactions (for example through SMD: http://genome-www5.stanford.edu/ • Genome mapping • (for example the UCSC Genome Browser: http://genome.ucsc.edu/cgi-bin/hgGateway

10. What do we need to create our own genomic inferences? • That’s the problem… The data is encoded… • AGCTAGCGACTAGCGATTATA… • Now predict what this sequence codes for, how it is transcriptionally regulated, in what pathway does it function … • The human genome is 3 billion bases. • Clearly we need a new set of tools • Problem: no human being could ever look at all this data, so the patterns must be discovered computationally

11. The magic wand analogy • The sum of all human knowledge can be encrypted and stored by marking a single scratch on a metal rod. It all depends upon RESOLUTION and LANGUAGE. • We can apply this analogy to our analytical process: • High RESOLUTION is required to decode the genome • A robust LANGUAGE is required to achieve that RESOLUTION • Computational biology seeks to describe that LANGUAGE

12. How do discover meaningful biological patterns? • Need to be able to pick “needles from haystacks” • Need to be able to assemble needles • Need to orient and classify needles • And so on… • This requires a strong, flexible environment • The Shell, PERL, R, PostGreSQL/MySQL – integration, data exploration • Statistical development environments – making inferences from the data • Databases and resources – combining the data

13. How our workshop will help you (and how it won’t?) • We cover common, portable, powerful resources – but require suitable activation energy! • Integration for a project model is key • Thus, easily understood but limited GUI based suites are not covered • Likewise theory of algorithm development is covered in other courses and will be ignored (for the time being!)

14. How this workshop is structured Two lectures a week for six weeks (fast!) Lots of reading material (need to keep pace) Hands – on practical experience with biological data and programming tasks Peer interaction

15. What you’ll need for this workshop Access to either the online version of the text through O’Reilly Media (linked on the website, $20 for two months) or through purchase of the print bound version (approx $130, we’ll put in an order during the second week) Access to a computer. Any computer will do. Motivation to learn, and to ask questions

16. What is the text for this workshop? O’Reilly Media SafariU agglomerative text of 10 textbooks from O’Reilly, New Riders, and Addison-Wesley publishers Chosen for topics applicable to modern computational biology Since many texts overlap, no need to own all – cost reduction ($130 vs. over $500)

17. Class information mjanis@chem.ucla.edu Office: 4054 Young Hall http://www.bol.ucla.edu/~mjanis/biohackers2005.html http://lists.ucla.edu/cgibin/mailman/listinfo/bioinformatics IRC: #uclaBioinformatics, freenode server (mako) T / R 2-4 pm 1054 Young Hall, Aug 11- Sept. 16 2005

18. Class information BRIEF SUBJECT MATTER OUTLINE (SUBJECT TO CHANGE!)08/11 Unix: Introduction to Unix; Bioknoppix08/16 Unix basics, editors, remote logins08/18 Shell programming08/23 Regular expressions; biological data formats08/25 Perl: data and control structures08/30 Modular programming09/01 Object oriented perl09/06 Biological classes; Bioperl09/08 NCBI tools; BLAST09/13 Database design; perl DBI09/15 Net programming; XML; CGI09/16 R; Bioconductor

19. We used to call it “biohackers”… What is a hacker? One with technical adpetness who delights in solving problems and overcoming limits… Hackers can be found in all walks of life – music, art, science Hackers create things – the internet, Unix, NCBI, the Human Genome project Not what the media would call a “hacker”…

20. The hacker attitude* Hackers believe the world is full of fascinating problems waiting to be solved. Nobody should ever have to solve the same problem twice. Boredom and drudgery are evil. Freedom is good Attitude is no substitute for competence * From “The Cathedral and the Bazaar”

21. Why would I want to become a “hacker” or a “biohacker” or whatever? Spend less time brutally munging your data. Spend more time gaining biological insight. Publish faster. Begin to assemble complex and disparate data in ways not achievable by humans. Think systematically about your data. Think systemically about your research. Smugly travel easily through electronic medium. Automate your life Be cool. Get chicks! (or whomever you want)…

22. It worked for “neo”…

23. In reality, it worked for someone else too …

24. How does one become a hacker? Learn to program. The more languages, the better. (Don’t think in terms of languages… Think in terms of PROGRAMMING) Get an extensible, modifiable OS and learn to use it Learn how to access and use information Practice, practice, practice. This workshop can only show you the door. You have to walk through it.

25. So what is biohacking? Inferential analysis of disparate biological data Discovering and creating new ways of looking at biological phenomenon Creation of novel tools Decoding the nature of life (BHAG)

26. So show me the tool to find the gene… I’ll be in the lab… What if your research looks at biology from a novel angle? What if there are no tools? Where to start? We need a modular framework. Something that will scale nicely so that we can incorporate more data later. Some set of tools that integrate well, each doing one job and doing it well. We need COMBINATORIC TOOLSETS!

27. A tale of two philosophies Windows works extremely well at solving one problem at a time, if it falls within defined parameters… It does not scale well. This monolithic approach is good for small scale biology but not for complex biological analysis

28. We need a construct where we can call upon modular tools to assemble our own analysis… I need bioinformatics tools… Lots of bioinformatics tools…

29. The construct – UNIX (the other philosophy) UNIX is an operating system that provides powerful modular utilities that function together to create computer environments that can solve a wide variety of tasks. Unix Operating System created 1960s provides a multi-user, multitasking system It’s a server by design Networks extremely well due to ports – X11 is an example command line based Bioinformatics relies heavily upon UNIX – and a fundamentally different philosophy: modular by its’ very nature, UNIX is scaleable, robust, and … FREE! But more than anything else, Unix has THE SHELL

30. The Unix Shell • The shell is the unix user environment • The shell is started automatically when you log in • Everything is controlled from a shell • The shell interprets commands • It’s also a programming language itself! • http://www.looking-glass.org/shell.html • http://www.strath.ac.uk/CC/Courses/IntroToUnix/node46.html • More than one shell can be open at a time

31. Commands to the system • Specific commands used to perform functions in the shell • Ex: changing directories, copying files, mv files, sorting, cutting, pasting, etc… • Many of these can be done in XL, but… • What if you need a subsequent command? ChiSquare analysis for example? • What if you need to do this to 10 files? 100? 10000? • Each command is itself a program and take command line arguments • The Shell is the “glue” that allows us to combinatorically daisy-chain our tools together to do a job • The Shell allows us to batch these commands into a pipeline or framework for analysis

32. Command Or Program STDIN STDERR STDOUT The Shell controls input / output

33. Command Or Program STDIN STDERR STDOUT Input / Output II • Keyboard • -File • -Previous STDOUT -Terminal/ monitor -File -Next STDIN

34. Input / Output III Command 1 Command 2 Pipe

35. Large Pipe Command 1 Command 2 Pipe Command 3 Pipe Command 4 Pipe Can daisychain commands from every shell program

36. UNIX then… Linux? BSD? OS X? (over 360 distributions according to linux.org)

37. How can I get *nix to use? • Linux is open source • Available for a variety of platforms • BSD is likewise open source • Forms the basis of Mac OS X, and many MSWin utilities • There are live distributions as well • Example: Bioknoppix - http://bioknoppix.hpcf.upr.edu/ • Bioknoppix is a customized distribution of Knoppix Linux Live CD. With this distribution you just boot from the CD and you have a fully functional Linux OS distribution with open source bioinformatics (sic) applications. • Beside using some RAM, Bioknoppix doesn't touch the host computer.

38. What about administrating the system? I heard it’s hard!

39. Package managers • Most modern distributions have a package manager • Handles installation • Handles dependencies • Handles conflicts • Packages are written in such a way as to ensure that they do not interfere with other parts of the OS install • With BioKnoppix, you won’t really need one • With Debian, there is aptitude, and synaptic • There are a wealth of bioinformatics tools packaged for use with package managers • We’ll be exploring these as we go along…

40. For our Apple brethren…

41. There is fink • Mac OS X forms a natural basis for a end user bioinformatics analysis station • It’s UNIX underneath! • The GUI is advanced • It’s easy to administrate • http://fink.sourceforge.net/ • Finks installs into its own protected directory /sw • Easy to remove if things go south…

42. What is fink? • Fink is a distribution of Unix Open Source software for Mac OS X and Darwin. • It brings a wide range of free command-line and graphical software developed for Linux and similar operating systems to your Mac. • Installation is accomplished via a tarball. • After that, it’s simple command line- interaction (based upon the debian model, which I’ll show in class) • http://fink.sourceforge.net/doc/users-guide/install.php?phpLang=en

43. For our MS brethren… Welcome to Earth A subsidiary of Microsoft

44. There is cygwin • Shell utilities written as windows executables • Creates its own protected / filesystem • Problems with networking • No real package manager beyond the shell utils • Can be problematic to install packages with deep *nix library dependencies • Created / maintained by RedHat developers • http://www.cygwin.com/

45. But mainly have remote access to a real *nix machine

46. Configure Putty

47. SSH Option

48. We’ll be using… • BioKnoppix Beta 0.2.1 (but the Knoppix part is 3.3…) • This distribution is included in your folder • Debian stable 3.1 (“Sarge”) install – which I’ll do in class – will be our main server. We’ll call it “subi” (subi.chem.ucla.edu) for SUmmer BIonformatics. • The text has details about the use of knoppix • Let’s take a look …

49. The shell, files, directories, commands, and editors

50. File System visualize the Unix file system as an upside down tree. At the very top of the tree is the root directory, named "/".