Introduction to Programming 3D Applications CE0056-1

Introduction to Programming 3D ApplicationsCE0056-1 Lecture 15 Input and interaction in 3D Environments

Topics • Interaction • Input devices • Event-driven input • Display lists • Menus • Picking • Buffering • Virtual Trackball

1963 Sketchpad • The first system to explore pen-based user-interfaces • by Ivan Southerland (of Evans and …) • broached field of HCI • not graphics but important • OpenGL doesn’t support interaction directly • limits portability • not specifically graphics • we’ll use GLUT for device interactions

Input devices • Physical vs. logical • Physical • mouse, keyboard, etc. • perspective of how they interact with system • Logical • function • perspective of what they send application • higher-level

Input devices • Logical examples • Output: when we use printf in C, the actual string may be output to a monitor or printer • Input: we get the position of a user’s cursor whether we use a mouse or a data tablet • Our application doesn’t care at high level • Two categories, pointer and keyboard • pointers may be mice, pens, trackballs, etc. • pointers do not return character codes • can return relative or absolute positions

Input devices • Absolute-positioning (w.r.t. some fixed position) • pen returns position w.r.t. tablet (settable) • data glove always returns 3D position coords • Relative-positioning (w.r.t. current position) • mouse pos. always begins where cursor is • Trigger & measure • trigger: physical input on device (button, key) • measure: what device returns to application (button, state, x position, y position)

Input modes • Request & sample modes • request_locator (device_id, &measure); • after trigger, wait until input received • ex. get x,y pos. after right mouse button depressed • sample_locator (device_id, &measure); • assumes trigger, get input from a queue • ex. get current x,y position of mouse • limits user control of program • which device to sample from? • modal, user must specify context (not natural)

Input modes • Event mode - the most flexible input mode • triggers generate events (events store data) • measures are sent to 1)an event queue program checks queue events are examined and acted upon 2)a special-purpose function program associates function with trigger up front function is called with the event trigger generates the function is called a callback

Event-driven input • Callback for display • glutDisplayFunc(display) • Callback for mouse • glutMouseFunc(mouse) • void mouse (int btn, int state, int x, int y) { if (btn == GLUT_LEFT_BUTTON && state == GLUT_DOWN) drawSquare(x,y); if (btn == GLUT_MIDDLE_BUTTON && state == GLUT_DOWN) exit (); }

Account for different origins (90, 100) (90, 400) 500 500 Event-driven input • drawSquare() function • use new mouse position to draw new square drawSquare(int x, int y) { y=w*h-y; glColor3ub(r, g, b); glBegin(GL_POLYGON) glVertex(x+size, y+size); glVertex(x-size, y+size); glVertex(x-size, y-size); glVertex(x+size, y-size); glEnd(); glFlush(); } What are we doing here?

Event-driven input • Callback for keyboard • glutKeyboardFunc(keyboard); • Callback for display • glutDisplayFunc(display); • call in indirectly with glutPostRedisplay(); • after changing parameters, attributes, etc. • if nothing has changed, system will avoid drawing • Callback for idle • glutIdleFunc(idle) • perform tasks when nothing else is happening

Event-driven input • Callback for window resize event • glutReshapeFunc(reshape) • What do we do in reshape(width, height)? • What if new window is bigger or smaller? Do we grow, shrink, or resize the image? • What if aspect ratio has changed? Do we similarly resize the image, distort it, or clip it out? • Use the new width and height to • adjust clip rectangle and viewport sizes gluOrtho2D (0.0, width, 0.0, height); glViewport (0, 0, width, height);

Clients/server Model • In general • Servers perform tasks for clients • In OpenGL • assumes the OpenGL program is the client • assumes workstations are graphics servers that provide display and input services to the OpenGL program • for example, the client can display its output on any networked server

Display lists • History • refreshing the screen required special processing via a display processor that output a display list that was stored display memory • the display list was rendered fast enough to avoid flicker, leaving the host computer free • there is no longer any distinction between host and display processor (unless we count special graphics processing hardware)

Display lists • In immediate-mode, • we compute, rasterize, and display (cube.c) • we re-compute to redisplay • In retained-mode, • we store the results of our computation in a display-list • we send an instruction to redisplay • disadvantages • require memory on server • require overhead to create

Display list example • #define Box 1 • glNewList(Box, GL_COMPILE) • glBegin(GL_POLYGON) • glColor3f(1.0, 0.0, 0.0); • glVertex2f(-1.0, -1.0); • glVertex2f(1.0, -1.0); • glVertex2f(1.0, 1.0); • glVertex2f(-1.0, 1.0); • glEnd(GL_POLYGON); • glEndList();

Display list example • glMatrixMode(GL_PROJECTION); • for (i=1; i<5; i++) { • glLoadIdentity(); • gluOrtho2D(-2.0*i, 2.0*i, -2.0*i, 2.0*i); • glCallList(Box); • } • Clip rectangle results (left, right, bottom, top) • i=1: (-2.0, 2.0, -2.0, 2.0) • i=2: (-4.0, 4.0, -4.0, 4.0) • i=3: (-6.0, 6.0, -6.0, 6.0) What if we want to change the color each time through? What happens as clip rectangle grows? What if it was huge?

Display lists • Create multiple lists • glGenLists(number) generates multiple ids • glCallLists displays multiple lists

Menus • Glut provides pop-up windows • define entries • link menu to mouse button • define callback function for each menu entry

Menu example • Menu creation is order-dependent • /*Create a menu named sub_menu with callback named size_menu, with two entries*/ sub_menu = glutCreatMenu(size_menu); glutAddMenuEntry (“incr. square size”, 2); glutAddMenuEntry(“decr. square size”, 3); • /*Create a menu with callback named top_menu, with two entries*/ glutCreateMenu(top_menu); glutAddMenuEntry(“quit”, 1); glutAddSubMenu(“resize”, sub_menu); • /*Make right mouse button trigger menu event*/ glutAttachMenu(GLUT_RIGHT_BUTTON); incr. square size decr. square size quit resize incr. square size decr. square size

Menu example • You write the callback functions • /*top_menu callback checks for quit only*/ void top_menu (int id) { if (id == 1) exit(); } • /*sub_menu processes resize options */ void size_menu (int id) { if (id == 2) size = 2.0*size; else if (id == 3 && size > 1) size = size /2.0; else if (id == 3) printf(“At minimum square size”); glutPostRedisplay(); }

Picking • User selection of object on screen • In immediate mode, we compute, rasterize and display. How does this present a problem for identifying objects on screen? • we pick a pixel • how to work backward from pixel to object? • Several solutions • render each object to its own clip rectangle and viewport • for every position in frame buffer we add an obj id to a lookup table that is examined at each pick • for every object we compute a bounding rectangle or box (or extents)*

Picking • What if objects overlap? • return list of objects, not just one • What if cursor is near but not on object? • bounding box will accommodate some • also, you can leave slack in accuracy when you check position against object extents • What if pick is in a virtual 3D environment? • we will cast a ray into the scene and check each object for an intersection, returning list

Buffering • Single buffering • objects are always rendered into the same framebuffer, which is always being displayed • delay of clearing of and re-drawing into frame-buffer will cause flicker if re-draw takes longer than refresh or refresh and animation not synced • Double buffering • keep two buffers (front and back) • always display front, always render into back* • swap front and back when rendering complete

Virtual Trackball Following slides developed by Ed Angel Professor of Computer Science, Electrical and Computer Engineering, and Media Arts University of New Mexico

Physical Trackball • The trackball is an “upside down” mouse • If there is little friction between the ball and the rollers, we can give the ball a push and it will keep rolling yielding continuous changes • Two possible modes of operation • Continuous pushing or tracking hand motion • Spinning

A Trackball from a Mouse • Problem: we want to get the two behavior modes from a mouse • We would also like the mouse to emulate a frictionless (ideal) trackball • Solve in two steps • Map trackball position to mouse position • Use GLUT to obtain the proper modes

Trackball Frame origin at center of ball

Projection of Trackball Position • We can relate position on trackball to position on a normalized mouse pad by projecting orthogonally onto pad

Reversing Projection • Because both the pad and the upper hemisphere of the ball are two-dimensional surfaces, we can reverse the projection • A point (x,z) on the mouse pad corresponds to the point (x,y,z) on the upper hemisphere where y = if r  |x| 0, r  |z|  0

Computing Rotations • Suppose that we have two points that were obtained from the mouse. • We can project them up to the hemisphere to points p1 and p2 • These points determine a great circle on the sphere • We can rotate from p1 to p2 by finding the proper axis of rotation and the angle between the points

Using the cross product • The axis of rotation is given by the normal to the plane determined by the origin, p1 , and p2 n = p1  p2

Obtaining the angle • The angle between p1 and p2 is given by • If we move the mouse slowly or sample its position frequently, then q will be small and we can use the approximation | sin q| = sin q  q

Implementing with GLUT • We will use the idle, motion, and mouse callbacks to implement the virtual trackball • Define actions in terms of three booleans • trackingMouse: if true update trackball position • redrawContinue: if true, idle function posts a redisplay • trackballMove: if true, update rotation matrix

Example • In this example, we use the virtual trackball to rotate the color cube we modeled earlier • The code for the colorcube function is omitted because it is unchanged from the earlier examples

Initialization #define bool int /* if system does not support bool type */ #define false 0 #define true 1 #define M_PI 3.14159 /* if not in math.h */ int winWidth, winHeight; float angle = 0.0, axis[3], trans[3]; bool trackingMouse = false; bool redrawContinue = false; bool trackballMove = false; float lastPos[3] = {0.0, 0.0, 0.0}; int curx, cury; int startX, startY;

The Projection Step void trackball_ptov(int x, int y, int width, int height, float v[3]){ float d, a; /* project x,y onto a hemisphere centered within width, height , note z is up here*/ v[0] = (2.0*x - width) / width; v[1] = (height - 2.0F*y) / height; d = sqrt(v[0]*v[0] + v[1]*v[1]); v[2] = cos((M_PI/2.0) * ((d < 1.0) ? d : 1.0)); a = 1.0 / sqrt(v[0]*v[0] + v[1]*v[1] + v[2]*v[2]); v[0] *= a; v[1] *= a; v[2] *= a; }

glutMotionFunc (1) void mouseMotion(int x, int y) { float curPos[3], dx, dy, dz; /* compute position on hemisphere */ trackball_ptov(x, y, winWidth, winHeight, curPos); if(trackingMouse) { /* compute the change in position on the hemisphere */ dx = curPos[0] - lastPos[0]; dy = curPos[1] - lastPos[1]; dz = curPos[2] - lastPos[2];

glutMotionFunc (2) if (dx || dy || dz) { /* compute theta and cross product */ angle = 90.0 * sqrt(dx*dx + dy*dy + dz*dz); axis[0] = lastPos[1]*curPos[2] – lastPos[2]*curPos[1]; axis[1] = lastPos[2]*curPos[0] – lastPos[0]*curPos[2]; axis[2] = lastPos[0]*curPos[1] – lastPos[1]*curPos[0]; /* update position */ lastPos[0] = curPos[0]; lastPos[1] = curPos[1]; lastPos[2] = curPos[2]; } } glutPostRedisplay(); }

Idle and Display Callbacks void spinCube() { if (redrawContinue) glutPostRedisplay(); } void display() { glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT); if (trackballMove) { glRotatef(angle, axis[0], axis[1], axis[2]); } colorcube(); glutSwapBuffers(); }

Mouse Callback void mouseButton(int button, int state, int x, int y) { if(button==GLUT_RIGHT_BUTTON) exit(0); /* holding down left button allows user to rotate cube */ if(button==GLUT_LEFT_BUTTON) switch(state) { case GLUT_DOWN: y=winHeight-y; startMotion( x,y); break; case GLUT_UP: stopMotion( x,y); break; } }

Start Function void startMotion(int x, int y) { trackingMouse = true; redrawContinue = false; startX = x; startY = y; curx = x; cury = y; trackball_ptov(x, y, winWidth, winHeight, lastPos); trackballMove=true; }

Stop Function void stopMotion(int x, int y) { trackingMouse = false; /* check if position has changed */ if (startX != x || startY != y) redrawContinue = true; else { angle = 0.0; redrawContinue = false; trackballMove = false; } }

Quaternions • Because the rotations are on the surface of a sphere, quaternions provide an interesting and more efficient way to implement the trackball • Quaternions will be covered later

Introduction to Programming 3D Applications CE0056-1