CS 98/198: Intro. to Web 2.0 Development Using Ruby on Rails

CS 98/198:Intro. to Web 2.0 Development Using Ruby on Rails Armando Fox, Will Sobel, Dave Patterson

RESERVED: MW 4–5:30 271 Soda is RESERVED for CS 98-10/198-10, Intro. to Web 2.0 Development Using Ruby on Rails, MONDAYS & WEDNESDAYS 4–5:30 PM. Space permitting, students not enrolled in the class may use the space during those times if their activities don’t interfere with running the class. Thanks. Questions: fox@cs.berkeley.edu

Today • About the course • administrivia, prereqs, course format, projects • about the RAD Lab • course projects & Entrepreneurship tie-in • Movies • zero to blogging in 58 lines of code • Rails “TV ads” • Background concepts review • Web app anatomy & brief history of Web programming • TCP/IP, HTTP, HTML, Web servers

Why this course? • Important trends in “Web 2.0” style applications • Reinventing client-server: the datacenter is the computer • New models of software development & deployment • Rich, sophisticated UI’s • Modular apps (mashups, etc.) • Attract “end users” to RAD Lab platform • Student-conceived apps generate traffic • We provide “hosting service” • Exercise RAD Lab infrastructure Opportunity: combine sound pedagogy with “real world” skills & gratification of crafting a useful artifact early in your CS career.

Web 1.0 • Static and some dynamic content • lots of ad-hoc scripting languages, cgi-bin • “homegrown” sites not yet standardized on a stack (like LAMP); eg, variants of Berkeley DB or filesystem for storage • Big companies: $$$ relational DB (eg Oracle) • Primitive web page GUIs • 1 interaction == 1 roundtrip to server for page view • “Services” available but have to be a company to get them (small sites use libraries instead) • Doubleclick (ads) • Inktomi et al. (web search on your site) • Visa/MC (accept credit cards)

Software as a Service (SaaS) • Run your own database => Salesforce • Email Word files around => Google Docs • Shrinkwrap mapping software => Google Maps, Google Earth • Buy a hard disk => Amazon S3 • Buy servers => Amazon EC2 • Desktop GUI app => AJAX (Javascript-based) GUI in browser

Web 2.0 Trends • Company => Individual can get services • DoubleClick => Google AdSense; Visa => Paypal • Sophisticated UI: AJAX decouples interactions/page updates from server roundtrips • Effect 1: “desktop-like” interactive Web apps • Effect 2: Desktop => cell phone (Browser w/Javascript == universal platform) • “Mass customize” to 1 => Wisdom of crowds • global search, etc are more compelling than local because learns from everybody • Libraries => Services, thanks to arch. standards • REST/CRUD for state management • Javascript/DOM for advanced browser interactions

Effect: Complexity Curve • History: startups can’t compete effectively because of cost of developing large scale or complex infrastructure • Full-text index for 200MM documents • Geocoded real-time messages • Quickly index and navigate mesh of millions of friends, compute shortest-path distance • Validate identity for secure transactions • Services provide components and provide pay as you go model • Scalable costs allow small companies to focus on core competency

Process: SupportDADO Evolution, 1 group Waterfall: Static Handoff Model, N groups New models of software development Assess Deploy Assess Deploy Develop Operate Develop Operate

Client Focused Development Light Weight Process for Small Teams Write Stories not Requirements Estimate Stories using Points Compute Your Velocity in Points per Day Scope Your Project Based on Your Velocity Spike Where You Can’t Estimate Agile (in 5 steps)

Real World Practices • Development vs. deployment environments & tools • Real version control (Subversion) • Automated deployment tools (Capistrano) • Regression testing techniques (Rspec, Selenium) • Database schema migration • User Stories. Customer focused requirements. • CRC cards (Class – Responsibility – Collaborator). Minimalist Object Modeling. • Iterations and velocity to scope releases. • Discussion of RoR and Web app entrepreneurship toward end of sem.

Prereqs & course format • “Guided individual study”, 2-4 units C/NC • Prereqs: CS 61A or equiv.; OOP concepts • Hybrid lecture/lab format • Realistic development environment • develop primarily on your own machines • Deploy to “production ISP” (RAD Lab cluster), also available for development use • Each project gets an external sub-domain • Option to purchase your own domain

Required Team Project • You pick the app • Tie-in to Enterpreneurship (Eng 190C) to help brainstorm • We help you scope & design the project • We help you DADO the project • You give live demo & 5 minute “pitch” at end of semester • “Best” project team members offered RAD Lab internships if they wish • Some previous projects already .com’s

Some projects from previous iterations • Spring 07 Projects: heavy comments; test rigs; Good GUI; Fine in browser; 1k to 3k LOC • WeJoinIn (real users!): staff your volunteer events • Calentopia: organize your academic life • 5um (next generation forum): organize & stay on top of all forums you read • Fall 07 Projects with real users (used 5um in class); • Yumify: where go to dinner? http://www.yumify.com/ • 2.5K LOC, 0.3K Test rig, 1K Lines of comments • Hesperian: multilingual community health access http://hesperian.rorclass.org/ • 0.5K LOC, 0.3K Test rig, 0.2K Lines of comments Students from “award winning” projects already working in RAD Lab as ugrad research assistants

Datacenter is the Computer • Program == Web search, Mail,… • Computer == Building full of 1000s of computers, networking & storage devices • Warehouse-sized facilities and workloads may be unusual today but are likely to be more common in the next few years • Re-inventing Client/Server Computing • The Datacenter is the (Server) Computer • The Laptop/Handheld is the (Client) Computer • HP shipped more laptops than desktops in 2007 • Million cells phones per day, increasing functionality • Apple iPhone as one example

Datacenter is the Computer • Interesting design point: datacenter composed of 20 ft. containers • Power/cooling for 200 KW of racked HW • External taps for electricity, network, water • ~250 Servers, 7 TB DRAM, or 1.5 PB disk • BlackBox to arrive next to Soda Spring 2008? Sun Project Blackbox10/17/06

RAD Lab research goal: Address Datacenter management challenges • Management == efficient use of resources under dynamic demand, cost model, reliability • Resources: Order(s) of magnitude more devices • “Dynamic demand”: peaks 5-10X averages, provisioning hard • “Reliability”: SW/HW failures commonplace • “Software Churn”: Google search rewritten twice in last 5 years; Ebay rewritten 4 times in 9 years • “Dynamic cost model”: nonlinear cost models for power, cooling, network traffic  Too large scale, too complex for manual administration by people  use Statistical Machine Learning to tackle

RoR as Datacenter Programming Language • We believe RoR will have lasting impact • Tastefully chosen features in language & framework combine best ideas from programming language literature • Intellectual/academic value in reinforcing key ideas of programming structures • Bootstrap RoR expertise within Berkeley EECS • Exceptionally rapid prototyping of Web 2.0 apps • Incremental/agile development support • Sophisticated UI support (Ajax, etc.) • Rapid bootstrapping doesn’t require learning bad habits • Maximizes programmer productivity, which is increasingly a scarce resource

So, why this course? • We (researchers) are great at creating technologies, terrible at creating apps • You create popular apps, your friends & colleagues flock to them • We get apps/data to test our ideas • You get fame, etc. • Example: two recent RAD Lab MS grads...

CS98/198 Development & Deployment Environment Mongrel Mongrel MySQLDev & Testing Shared MySQL instance ... Mongrel,Webrick,or IIS Capistrano Static con-tent Apache reverse proxy Apache reverse proxy Sub-versionrepo-sitory Internet RAD Lab/ Millennium cluster Your laptop www.MyVanityURL.com

Toolz U Will Uze • Lab 0: install these items from class page • RoR development environment • Capistrano deployment tools • An editor (emacs, vi, vim...) or IDE (Eclipse) • Subversion client for source code management (with SSL & Kerberos support compiled in) • MySQLAdmin or CocoaMySQL database GUI • and signup for class forum on 5um.rorclass.org

Movie time... • radlab.cs.berkeley.edu/wiki/RoR

Today: technical stuff review • Web Apps 101, from Web 1.0 to Web 2.0 • TCP/IP, HTTP, HTML, dynamic content (CGI) • Web 1.0 vs Web 2.0 • Model/view/controller • Lifecycle of a Web app • REST • Get RoR install, etc. running on your computer

The Web as remote procedure call server The Internet • RPC protocol == HTTP • ASCII based request/reply protocol run over TCP/IP • protocol headers specify metadata about the request • Stateless: notion of “session” must be synthesized separately • RPC arguments == URL’s, HTML-form contents • URL names a server & resource • URL may embed “argument values”, or these can be “uploaded” as encoded HTML form submission • REST goes a long way toward cleaning up this sorry state browser

A conversation with a Web server • Browser opens TCP connection to server on port 80 (default) and sends: GET /index.html HTTP/1.0 User-Agent: Mozilla/4.73 [en] (X11; U; Linux 2.0.35 i686) ...other boring headers... Cookie: B=2vsconq5p0h2n • Server replies: HTTP/1.0 200 OK Content-Length: 16018 Content-Type: text/html <html><head><title>Yahoo!</title><base href=http://www.yahoo.com/> …etc. • Repeat for any embedded images, scripts, ...

HTML in one slide • Roughly hierarchical collection of elements that make up a viewable page • inline (headings, tables, lists...) • embedded (images, video, Java applets, JavaScript code...) • forms—allow user to submit simple input (text, radio/check buttons, dropdown menus...) • Each element can have attributes (many optional) • of particular interest are id and class attributes • CSS (Cascading Style Sheets) allow specification of visual appearance of HTML pages based on the id’s and/or classes of elements • Old “styling” elements (like <b>, <i>, etc.) are deprecated; use CSS instead, or where possible, logical styling like <emph>, <strong> • Current incarnation, XHTML, more device-portable by being strict about syntax that went to pot in HTML • RoR and many other frameworks generate XHTML-compliant code

XHTML/CSS in 1 slide • Declarative language to specify visual appearance of your HTML • Style attributes include font color/size, background color/picture, border colors, padding, spacing, justification/position... • Attributes can apply to specific elements, elements of a given class... .bannerHead { background-color: #883333; color: white; padding: 8px; margin: 4px; border: 2px solid #d3bf8e; text-align: center; } .bannerHead a { color: white; text-decoration: none; } • pre { • background-color: #eee; • padding: 10px; • font-size: 11px; • }

Web 0.9 and Web 1.0 mysql Filesystem or database Ruby interp. your app CGI apache firefox • Web <1.0: web page == file (“static content”) • Web server (eg Apache) maps URL to file name • Web 1.0: web page == output of running a program • URL or HTML form specifies name of program and parameters to pass to it, via CGI (Common Gateway Interface) API • example: http://www.foo.com/search?term=white%20rabbit&show=10&page=1 • App runs and generates HTML • Server passes HTML back to client via HTTP • How to identify same user returning? • Server gives Web browser a cookie on 1st visit • Browser passes cookie back to server on each request • Server can modify cookie before returning it to browser • Cookie typically used to look up session info in database or other store • Various frameworks now capture this common structure

The MVC Design Pattern • Read data • Update data • User actions • Directives for rendering data .rb (Ruby) code .rhtml template (or .rjs, .rxml...) SQL table + Ruby class • Data provided to views • Goal: separate organization of data (model) from UI & presentation (view) by introducing controller • mediates user actions requesting access to data • presents data for rendering by the view • Web apps are “sort of” MVC by design Controller View Model

What is Ruby on Rails? • Ruby is a language that is... • dynamically typed, interpreted, object-oriented, functionally-inspired • Rails is a web application framework that... • embodies the MVC design pattern • emphasizes convention over configuration • leverages Ruby language features incl. dynamic typing, metaprogramming, & object-orientation to provide elegant support for both goals • Rails handles everything up to the point where your code is called • And everything past the point where your code delivers stuff to the user.

Web 1.0 • Web 1.0: user interaction == server roundtrip • Other than filling out form fields • Every user interaction causes server roundtrip • Every roundtrip causes full page redraw • Web 1.5: user interactions without contacting server • e.g. form validation before submit • e.g. selecting something from menu A causes contents of menu B to change • But every roundtrip still causes full page redraw

Web 2.0 • Separation of server roundtrip from page rendering • Initial load of page & draw page • User interaction causes background roundtrip to server • Response from server “captured” and passed to a programmer-defined JavaScript function • That function can redraw part of the page in place (using same mechanisms as Web 1.5) • Result: “desktop-like” responsive UI’s that can contact server • Auto completion • “Lazy” fetch of complicated parts of page • etc

Managing Your Source • Source/configuration management (SCM) • a/k/a revision control, version control • history in Unix world: sccs => rcs => cvs => svn • Idea: take “snapshots” of code at different points • roll back to snapshot if screw up • everyone can bring their working copies up-to-date relative to some recent snapshot • everyone can contribute changes to new snapshot (supports multiple developers working on same tree) • “conflicts” detected at check-in time

Basic operations of source code management • First time: Check out a copy of current revision (or some previous revision) of code from repository • Subsequent times: Update your working copy from repository (to get latest changes by others) • Edit code, etc. • Commit your changes • Conflicts during commit? (if others modified same files as you) • identify bug in communication among developers • 1 person manually edits file to resolve conflict; marks file "resolved" and commits definitive version • others update their version from newly-committed file

Lifecycle of Web app migrate Mongrel Mongrel 1. Checkoutor update 3. commit update MySQLDev & Testing Shared MySQL instance ... Mongrel,Webrick,or IIS 4. deploy Capistrano Static con-tent Apache reverse proxy Apache reverse proxy IDE or editor Sub-versionrepo-sitory 2. edit & test Internet Your laptop

REST is a Software Architectural Style • What's software architecture? • subdivides complex systems into components • assigns each component specific responsibilities • dictates how components communicate and which ones communicate with which others • Models the information (domain) into entities and specifies how it will be transformed and flow through the application • State management

REST • REST: Representational State Transfer • Documents are transferred around • Application state is contained entirely in the documents • URI's name documents and actions to be performed on them • The definitive document on REST is Roy Fielding's dissertation: http://www.ics.uci.edu/~fielding/pubs/dissertation/fielding_dissertation.pdf

Web Architecture • Network vs. Distributed • Distributed: A single system that distributes its work across multiple machines • Networked: A system that is utilized across a network • We are discussing networked architecture • A networked architecture can have a distributed architecture at the server

REST and CRUD • CRUD = basic operations on data objects (documents) • REST maps CRUD actions to HTTP methods (commands)

URI Refers to Resources • A resource is an object that can be operated upon • A URI will always refer to the same semantic resource • "customer with ID # 27" • "first order belonging to customer with ID # 27" • Resources are delivered in the manner best suited for the user agent • HTML • XML • WML • Etc…

CS 98/198: Intro. to Web 2.0 Development Using Ruby on Rails