1 / 52

CS 352: Computer Graphics

CS 352: Computer Graphics. Input and Interaction. What good is Computer Graphics?. JS1k canvas examples: 1 2 3 4 5 Games, visual demos…of what value? Is there a Christian perspective? Communication of information to the user Data visualization, simulation, GUI

Download Presentation

CS 352: Computer Graphics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. CS 352: Computer Graphics Input andInteraction

  2. What good is Computer Graphics? • JS1k canvas examples: 12345 • Games, visual demos…of what value? Is there a Christian perspective? • Communication of information to the user Data visualization, simulation, GUI Even a word processor is an "interactive graphics program" • Communication between users "Collaborative environments" are hot (multi-player games?) Social networking is transforming the world… • Interaction is an essential component

  3. Interaction • Much of the fun and utility of graphics is in interaction:accepting user input that affects the display • Paint programs, modeling, games, word processors, spreadsheets, powerpoint … • User initiates input events such as clicking on a menu or drawing a circle • Computer response by changing graphical display

  4. Projects 3 • Projects 3 will be a paint program • we'll learn 2-D graphics, interaction, event-loop programming, double-buffering simple animation, basic image processing • Features: • Pen, line, and rectangle tools • Color, pen size selectors • Image processing (sharpen, blur, detect edges) • Open, save images • Toolbar and menu for controlling application

  5. How to set up an application UI? • How to make a menu? • How to make a color picker? • How to make a toolbar? • How to make toolbar buttons pop in and out?

  6. How to paint? • How do I make a colored line follow the mouse or fingertip? • Interactive prog: • Input devices • Event handlers • Event loop

  7. Input devices • Interaction requires handling input devices • Physical:mouse, fingertip, keyboard, joystick, digitizer, accelerometer, head tracker… • Logical: • Text • Locator • Valuator (slider) • Stroke • Color picker • How to read?

  8. Input devices • Sample mode • Program reads the current state of input device (frequently) • Event mode • Each click or motion generates an event that goes on a queue • Program can process events from queue • HTML?

  9. Event-loop programming • Events: button press, slider value change, menu selection, mouse move, keypress • Event loop: • Poll (or wait) for events • Process events, update state • Update display • (ideally: wait for rest of frame time to elapse)

  10. State-driven • Typically, the event loop is driven by a big state table • e.g. depressing the paintbrush tool releases other tools, puts me in "paint mode" • Good libraries will handle some of the bookkeeping for you • You still typically have to handle some state

  11. State diagram

  12. Event loop in HTML/JS • HTML/JS provides event queue, support for many basic events (mousedown, mouseup mouseover, mousemove, keypress, key release, value change, button click, etc.) • You are on your own for higher-level events, e.g. clicking on a toolbar tool • It's also possible to set a function to run every 15 ms, sample input devices

  13. Events for painting? • mousedown: • go into paint mode • store mouse position • draw initial dot • mouseup: • exit paint mode • mousemove: • if in paint mode • draw a line from old mouse position to current • set old mouse position to current position

  14. Event handling with jQuery • Binding events to functions $(cpaint.canvas).bind('mousedown', cpaint.drawStart) • Processing events cpaint.drawStart = function(ev) { var x, y; x = ev.pageX - $(cpaint.canvas).offset().left; y = ev.pageY - $(cpaint.canvas).offset().top; ev.preventDefault(); cpaint.paintmode = true; cpaint.oldX = x; cpaint.oldY = y;

  15. Looking neat and spiffy • How to avoid crinkles in your paint strokes?

  16. Looking neat and spiffy • How to avoid crinkles in your paint strokes? • Draw connected paths • Or just use line caps

  17. Menus • There are many jQuery menu plug-ins… <ul id="mainmenu"> <li>File <ul> <li id="menuNew">New</li> <li id="menuOpen">Open</li> <li id="menuSave">Save</li> ------------------------------------- $('#menuNew').bind('click', cpaint.clear); $('#menuOpen').bind('click',cpaint.open); $('#menuSave').bind('click',cpaint.save);

  18. Toolbar • How to make a toolbar? • How should buttons behave? • State diagram?

  19. Buttons • Widgets may have several states • State should be evident in visual feedback • E.g. how exactly does a button work? States? • State transition diagram • Most buttons: six states, with six different appearances • neutral • neutral-highlighted • neutral-depressed • selected • selected-highlighted • selected-depressed • Events: mousedown, mouseup, enter, exit • Transitions: what happens in each state under each event?

  20. Button state diagram (buttons selectable, not unselectable) N S NH SH N: neutral H: highlighted (usually mouse over) S: selected D: mouse down NHD SHD Press Move

  21. Other button considerations • Could also consider • Tooltips • Whether button merely clicks and releases or can be selected • Whether button can be unselected (e.g. B/I/U vs. Left, Center, Right) • Want consistent appearance, behavior over whole program – or whole computer • Really need a library implementation and a strict set of UI guidelines…

  22. 4-State toolbar buttons: CSS .toolbarCell { background-color:#ddd; width:20pt; height:20pt; border: solid #eee 2px; -webkit-box-shadow: 2px 2px 2px #666; } #markerButton { background: url(img/paintbrush.png) no-repeat center center; } .selected { -webkit-box-shadow: inset 2px 2px 2px #666; } .toolbarCell:hover { border:solid #555 2px; }

  23. 4-State toolbar buttons: JS $('#markerButton').bind('click', {tool:"marker"}, cpaint.selectTool); ------------------- cpaint.selectTool = function(ev) { cpaint.tool = ev.data.tool; // get tool name $('.toolbarCell').each(function(index) { // unselect $(this).removeClass('selected'); // others }); var tool = '#' + cpaint.tool + 'Button'; // get ID $(tool).addClass('selected'); // select }

  24. Paintbrush size selector • How?

  25. Color picker • Google "jQuery color picker"…

  26. How draw a line? • What kind of feedback is normal? • Rubber Banding • Events and actions?

  27. Save & restore • Create your own off-screen canvas • Copy it back on each mouse movement • Events and actions?

  28. Save & Restore event handling • Mousedown • enter line mode • remember mouse position as startx, starty • save screen • draw initial dot • Mousemove • paste saved screen • draw line from startx, starty to current mouse pos • Mouseup • exit line mode

  29. Analysis • Drawbacks? • slow – eats memory bandwidth for breakfast • copy smallest possible rectangle? Only points from line? • possible flickering • use double buffering?

  30. Double buffering • Have two frame buffers, "front" and "back" • Draw into back buffer • At vertical retrace time, swap buffers • This is a fundamental graphics technique… • …not built into canvas… • …though, some drawing aggregation seems to happen automatically, behind the scenes; • …not usually necessary in canvas, but browser dependent

  31. Fake Double Buffering in Canvas • Create off-screen canvas canvasBuffer = document.createElement('canvas'); canvasBuffer.width = canvas.width; canvasBuffer.height= canvas.height; canvasBufferContext= canvasBuffer.getContext('2d'); • Draw into off-screen canvas canvasBufferContext.[drawSomeStuff…] • Copy onto display canvas context.drawImage(canvasBuffer, 0, 0);

  32. How to erase? • Draw background color…

  33. Save, Load? • Save • Copy pixels to an image so users can right-click, save-as • Load • What are the security risks? • Can only load files from the same server • Can use a file chooser if it's a local app • Security policies are not entirely in place

  34. Resizing • How would you allow the user to shrink, expand image? • What would happen to image resolution?

  35. Store drawing commands • Realistically, to redraw effectively, you have to save all drawing commands pen color red fill color blue line width 5 circle 5 10 12 textfont Times 12 text 10 10 "hello world" • Replay commands to redraw scene • Could store commands in a file (e.g. Mac PICT file) • For some kinds of drawings, files are smaller and more accurate than bitmaps

  36. Other toolkits • HTML offers basic UI capabilities in a cross-platform context • Writing your own UI extensions is tedious • jQuery plugins extend capabilities • Ideally, all UI elements ought to be built in • In the real world, they come in platform-specific 'toolkits' or 'windowing libraries' • E.g. MFC, QT, OpenLook, Cocoa …

  37. Image processing • Image processing examples: • blur, sharpen • detect edges • enhance contrast • noise reduction • posterize • fisheye • redeye reduction • find faces • …

  38. How to fade an image? • Could average pixel colors with white • Or just decrease alpha…

  39. How to blur an image? • Blurring is an averaging process • Convolution: apply a convolution kernel, e.g. • Center on pixel of interest • Multiply each value by color val underneath • Add results • Gaussian smoothing

  40. Gaussian smoothing

  41. How to sharpen an image? • It's a differentiation process • What's Unsharp Masking?

  42. How to sharpen an image? • It's a differentiation process • What's Unsharp Masking? • Subtract blurred version from original • Add back to original

  43. Sharpening • Subtract neighbors • Like subtracting a blurred version of the image • Unsharp Masking • E.g. convolve with a kernel like one of these:

  44. Edge detection? Wikipedia

  45. Sobel Edge Operators • Look for horizontal and vertical variation • Could do this at different sales or resolutions • Note: results maybe positive or negative numbers; must normalize

  46. Image Compression • What if you created say six different lower-resolution images and only stored the difference at each resolution? • Note: most of the data is in the highest-frequency component • An early image compression technique

  47. The alg that changed imagery… • Discrete cosinetransform (DCT) • (Wikipedia)

  48. JPEG image compression • DCT to convert to frequency domain • Perceptual modeling • Coefficient quantization • Entropy coding

More Related