[I/O] ³ Input/Output Cubed

[I/O]³ Input/OutputCubed Kevin Eykholt Tom Finneran Chris Pitoniak ShamitSom

Background • Current 3D modeling approaches are filled with many problems: • Physical 3D modeling: • Time – If you mess up, you have to start over • Healthrisks – Long term exposure to silica dust can lead to Potter’s Rot • Waste – Failed project materials are not reused • Virtual 3D modeling: • Non-intuitive interface – Mouse and keyboard are two dimensional interfaces while the models are three dimensional objects • Hard to learn – Computer programs have many different functions and sometimes involve lots of key hits or mouse clicks that are not intuitive

Project Focus • First Goal: Produce a tool for architectural design • Input doesn’t need to be intuitive so it can still feel similar to a touchscreen • 3D output

Project Focus • Second Goal: Produce a tool appropriate for a sculptor • Intuitive Input – Should give the user a similar feel to real life modeling without the sense of touch • 3D output On this We want this

Goal • Goal: To an intuitive 3D modeling system which includes the pros of both physical and virtual 3D modeling • We want make a tablet for 3D modeling

Our Solution - Block Diagram Input Leap Motion Leap Motion Sensor Data Output Input Parsing Data Processing 2D Visual Diagnostic Projector Output Data Planar Volumetric Sweep Device Output Processing Computer

Timeline MDR FPR 27 Oct 12 Nov 21 Nov 31 Dec 15 Jan 20 Feb 21 Mar Chris Kevin Shamit Tom Test gesture mapping Leap Motion gesture recognition Leap Motion gesture mapping Basic Shape Generation Scaling, Translation, Rotation Generate planar cross sections Integrate Full system and test Build volumetric sweep frame Add motor and ensure mechanical stability CDR CDR Test and optimize POV effect Implement and test interface Test POV Effect Add projector Conceptual design for software to projector interface

Proposed MDR Deliverables • Input: • Demonstrate Scalingand Rotation • Other functions are simpler to evolve after implementing these • Demonstrate interaction between Leap Motion and software, confirmed with 2D Visual Diagnostic • Major interface • Output: • Create a volumetric sweep device • Critical path to entire output system, must be completed before progressing further • Simple demonstration of POV effect

Modified MDR Deliverables • Input: • Demonstrate Scalingand Translation • Other functions are simpler to evolve after implementing these • Demonstrate interaction between Leap Motion and software, confirmed with 2D Visual Diagnostic • Major interface • Output: • Create a volumetric sweep device • Critical path to entire output system, must be completed before progressing further • Simple demonstration of POV effect • Initial Software to Projector Interface Implemented

Input Specifications • Input Requirements: • Respond to hand gestures • Intuitive gestures for object creation and manipulation • Recognize one unique gesture per function • 1 millimeter skeletal precision • Software Requirements: • Map input gestures to functions in the virtual 3D object space • Object functions: creation, translation, rotation, scaling • Control functions: save, undo • Display traditional 2D representation of virtual objects • Generate appropriate output for volumetric display

Hand Recognition Device - Leap Motion • Requirements: • 1 mm skeletal precision • Should be able to recognize multiple hand gestures • Prof. Koren’s concern – Need intuitive gestures • Challenges: • Trouble detecting fingers that are close in proximity • 14 inch max detection area • Trouble sensing fingers when they are perpendicular to the sensor • Completed for MDR: • Scaling • Translation • Demo Later

Software - JOGL • Requirements: • Map input gestures to functions in the virtual 3D object space • Display traditional 2D representation of virtual objects • Generate appropriate output for volumetric display • Challenges: • How to make cross sections based on vertices • Objects translate relative to the axis instead of the camera angle

Demo • Input: • Demonstrate scaling and translation • Demonstrate interaction between Leap Motion and software, confirmed with 2D Visual Diagnostic Hands On Demo After Presentation

Output Specifications • Output Requirements: • Distortion-free, volumetric image • 75 X 75 x 75 voxel resolution minimum • ≥20 FPS refresh rate • No smaller than a 2 inch cube display • Quiet operation (<30 dBa)

Software–Projector Interface Design • Fast, continuous flow of planar cross-sectional bit planes • Pool of worker threads read from parallel data pipe to encode image with bit planes • Encoded image displayed at 60Hz • Java timer event to lock-in refresh rate to meet projector specification 3D Software Parallel Data Pipe Encoded Image Thread Projector Thread Planar Cross-Sections Thread

Software-Projector Current State • Java application detects secondary output device (the projector) • Generates 608x684 RGB pixel image with randomized color data • Written sequentially over entire image to simulate processing delay of encoding multiple bit planes • Actual visible portion of projected image will be smaller and encoded in parallel to optimize speed • Refreshes at 60Hz with simple timing and minimal tearing • Also: POV effect demonstrated with two stripes projected at ~1440Hz as separate images but seen as one

Output Concept – Original Design • Design/build mechanical system to sweep a planar surface – settled on designing a system similar to a piston • Move projector with screen to mitigate focus problems • Ideally want 24fps, can be implemented with 12Hz reciprocation • p(t) = (1.5in)(.0254 m/in)cos(2π(12Hz)t) • v(t) = d/dt[p(t)] = -24 π*(1.5in)(.0254m/in)sin(2π(12Hz)t) • a(t) = d/dt[v(t)] = -24² π²(1.5in)(.0254m/in)cos(2 π(12Hz)t) • = 216.594 m/s² max acceleration = 22.08G! • At 10Hz, max acceleration = roughly 16G!

Output Device

Output Concept – Modifications • To mitigate possible projector destruction, decided to fix projector and just move screen • Focus error is negligible over 3” travel • Projected image size changes considerably (but linearly) with distance – will account for this in software • Still potentially possible to move projector, may attempt later if money/time permit 1 • Using regular paper sandwiched between acrylic to project onto – projected image sufficiently displayed on both surfaces of surface 1: According to TI Tech Journal in 1998, early DMD devices could handle 1500g shock tests, 20g vibration tests, and 10,000g acceleration (centrifugal force) tests without failure

Output Device • Due to (somewhat foreseen…) complications, including last-minute design changes, machine shop access, etc., device building encountered delays • Haven’t decided on motor yet, estimate around 20W motor necessary for load of 1kg • (1kg)(9.8 N/kg) * (3in[down] + 3in [up])(.0254m/in) * 12Hz = ~17.92W • Suspected overestimate, since no load besides weight of aluminum/acrylic platforms and friction • Next steps: • Assemble parts, test movement • Choose, procure, test motor • Attach motor, polish entire assembly • Build feedback system to control motor speed and report position • Build enclosure

PDR Change Review • The project has been split into two parts • The first part focuses on basic system functionality – Recognizing hand gestures, displaying image on 3D output • The second part includes that the hand gestures must be intuitive so as to seem similar to physical modeling • Projector is now fixed rather than moving with projection screen

Timeline MDR FPR 27 Oct 12 Nov 21 Nov 31 Dec 15 Jan 20 Feb 21 Mar Chris Kevin Shamit Tom Test gesture mapping Leap Motion gesture recognition Leap Motion gesture mapping Basic Shape Generation Scaling, Translation, Rotation Generate planar cross sections Integrate Full system and test Build volumetric sweep frame Add motor and ensure mechanical stability CDR Test and optimize POV effect Implement and test interface Test POV Effect Add projector Conceptual design for software to projector interface

CDR and FPR Deliverables CDR • All software functions have been implemented and have an associated gesture • Object Functions – Create, Delete, Rotate, Scale, Translate • Camera – Rotate, Zoom, and Translate • Control – Save, Load, and Undo • POV Structure Completed – Software Synced • Demonstrate input to output interface • A gesture detected on the input has the correct response displayed on the output FPR • Full system specifications met

[I/O] ³ Input/Output Cubed