1 / 45

The JavaFX API’s Synergy with JavaFX 3D

The JavaFX API’s Synergy with JavaFX 3D. Joe Andresen, Graphics Engineer JavaFX Alexander K., Software Engineer JavaFX. Program Agenda. Why Synergy Matters. JavaFX Images and Materials. “Brushes” Opacity and Layers Review of API as we go.

yannis
Download Presentation

The JavaFX API’s Synergy with JavaFX 3D

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The JavaFX API’s Synergy with JavaFX 3D Joe Andresen, Graphics Engineer JavaFXAlexander K., Software Engineer JavaFX

  2. Program Agenda • Why Synergy Matters. • JavaFX Images and Materials. • “Brushes” • Opacity and Layers Review of API as we go

  3. Synergy – more than one API part, do something awesome. No complexity!

  4. JavaFX 8 api (wo/ 3d) is a 2D renderer.

  5. JavaFX 8 3D api is a Renderer of 3d Content.

  6. JavaFX Canvas is a custom texture that you can draw to.

  7. Canvas And Materials Canvas! Effects: Per draw command of apply to the entire canvas. g.applyEffect(…); g.setEffect(…); Fill and Stroke commands. g.fillRect(…); g.strokeRect(…); Path drawing commands.

  8. Canvas And Materials Step 1) Canvas! canvas = new Canvas(300,300); g = canvas.getGraphicsContext2D(); //top left g.setFill(linearGradient); g.fillRect(0, 0, 150, 300); //more draw calls here…

  9. JavaFX Snapshot renders a node to an image.

  10. Canvas And Materials Snapshot SnapshotParametersparams = new SnapshotParameters(); params.setFill(Color.GREEN); params.setCamera(new PerspectiveCamera()); Image img = node.snapshot(params, null); Second Parameter is a WritableImage that can be written to by snapshot.

  11. Canvas And Materials Step 2) Snapshot patternCanvas.snapshot(params, null); Get an image of the pattern…

  12. Canvas And Materials Step 3) Canvas and ImagePattern! g.setFill(new ImagePattern(img, 0, 0, .04, .04, true)); g.fillRect(0, 0, 600, 600); //set fill with img as imagepattern and go!

  13. MeshView represents a 3d Surface in the scenegraph, specified by a mesh.

  14. Canvas And Materials MeshView MeshViewmeshView = new MeshView(mesh); meshView.setMaterial(phongMaterial); //Since meshView is a node, we can transform it. meshView.setTranslateX(width/2); root.getChildren().add(meshView);

  15. A material describes the way a 3d surface is rendered.

  16. Canvas And Materials PhongMaterial PhongMaterialmaterial = new PhongMaterial(); material.setDiffuseColor(Color.LIGHTGRAY); material.setSpecularColor(Color.rgb(30, 30, 30)); material.setBumpMap(img); material.setDiffuseMap(img2); material.setSpecularMap(img3);

  17. Canvas And Materials Step 4) Snapshot! Image snapshotImg= canvas.snapshot(new SnapshotParameters(), null); Get an image of the carbon fiber canvas texture.

  18. Canvas And Materials Step 5) Material & Apply to Mesh material = new PhongMaterial(); material.setDiffuseMap(snapshotImg); meshView.setMaterial(material); Material with diffuse map of image.

  19. Canvas And Materials Painting On A Model! Just Add Picking! -In Picking Handler, do a drawcall on the canvas based on UV’s in the PickResult.

  20. Canvas And Materials Triangle Mesh Data Layout Points : TriangleMesh. Num Components Per Point Num Components Per TexCoord Num Components Per Face

  21. Canvas And Materials Triangle Mesh Data Layout int a = triMesh.getFaces().get(t.getPickResult().getIntersectedFace() * TriangleMesh.NUM_COMPONENTS_PER_FACE) * TriangleMesh.NUM_COMPONENTS_PER_POINT; intb = triMesh.getFaces().get(t.getPickResult().getIntersectedFace() * TriangleMesh.NUM_COMPONENTS_PER_FACE + 2) * TriangleMesh.NUM_COMPONENTS_PER_POINT; int c = triMesh.getFaces().get(t.getPickResult().getIntersectedFace() * TriangleMesh.NUM_COMPONENTS_PER_FACE + 4) * TriangleMesh.NUM_COMPONENTS_PER_POINT;

  22. Canvas And Materials Painting On A Model! Cont.

  23. Canvas And Materials Picking a 3D Model - Hint PickResult returns a Point3D in local coordinate space. Remember to multiply the point by the node’s localToScene Transform: Point3D txPoint = meshView.getLocalToSceneTransform().transform(t.getPickResult().getIntersectedPoint());

  24. Canvas And Materials DRAW ON 3D DEMO

  25. Writable Image allows you to write to pixels using a Pixelwriter.

  26. WritableImage And Materials Height Map to Normal Map in JavaFX HeightMap – Wikipedia: … is a raster image used to store values, such as surface elevationdata… NormalMap – Wikipedia: “… is a technique used for faking the lighting of bumps and dents …”

  27. WritableImage And Materials Height Map to Normal Map in JavaFX

  28. WritableImage And Materials Step 1) Supply Height Map

  29. WritableImage And Materials Step 2) Get Pixel Reader From Image final PixelReader reader = heightMap.getPixelReader(); reader.getPixels( 0, 0, w, h,PixelFormat.getByteBgraInstance(),heightPixels,0,w*4);

  30. WritableImage And Materials Step 3) Bash Pixels! for (int y=0; y<h; y++) { for (int x=0; x<w; x++) { final intyAbove = Math.max(0,y-1); … Point3D pixelAbove = new Point3D(x,yAbove,pixelAboveHeight); … Point3D V = pixelAbove.subtract(pixelBelow); Point3D normal = H.crossProduct(V);

  31. WritableImage And Materials Step 3) Bash Pixels! Cont. normal = new Point3D( normal.getX()/w, normal.getY()/h, normal.getZ()); normalPixels[pixelIndex] = (byte)(255-(normal.getZ() * scale));

  32. Result!

  33. Canvas And Materials BUMP MAP 3D DEMO

  34. Canvas And Materials Picking a 3D Model - Hint PickResult returns a Point3D in local coordinate space. Remember to multiply the point by the node’s localToScene Transform: Point3D txPoint = meshView.getLocalToSceneTransform().transform(t.getPickResult().getIntersectedPoint());

  35. Procedural Images Painting a Model With Logic! When using a canvas to draw an a model, we can simulate many popular brushes used in apps. Color spray, shaped tips, hard edged, blurred, Path shaped, 3D shapes(!), etc. Any brush you can imagine!

  36. Procedural Images Color Spray Allow for subtle and random addition of color to a texture. Regenerate random spray (transparent radial gradient circles) every mouse event. Lots of varible inputs for control

  37. Procedural Images Color Spray - Result DEMO

  38. Performance - API synergy can be expensive.

  39. Synergy Performance This API img= Canvas.snapshot(…);

  40. int x = params.x; int y = params.y; int w = params.width; int h = params.height; if (w <= 0 || h <= 0) { return; } try { QuantumImagepImage = (params.platformImageinstanceofQuantumImage) ? (QuantumImage)params.platformImage : new QuantumImage(null); com.sun.prism.RTTexturert = pImage.getRT(w, h, rf); if (rt == null) { return; } Graphics g = rt.createGraphics(); draw(g, x, y, w, h); int[] pixels = pImage.rt.getPixels(); if (pixels != null) { pImage.setImage(com.sun.prism.Image.fromIntArgbPreData(pixels, w, h)); } else { IntBufferib = IntBuffer.allocate(w*h); if (pImage.rt.readPixels(ib, pImage.rt.getContentX(), pImage.rt.getContentY(), w, h)) { pImage.setImage(com.sun.prism.Image.fromIntArgbPreData(ib, w, h)); } else { pImage.dispose(); pImage = null; } } rt.unlock(); params.platformImage = pImage; } catch (Throwable t) { t.printStackTrace(System.err); } finally { Disposer.cleanUp(); } } }); final CountDownLatch latch = new CountDownLatch(1); re.setCompletionListener(new CompletionListener() { @Override public void done(final RenderJob job) { latch.countDown(); } }); addRenderJob(re); do { try { latch.await(); break; } catch (InterruptedException ex) { ex.printStackTrace(); } } while (true); Object image = params.platformImage; params.platformImage = saveImage; return image; } Synergy Performance @Override public Object renderToImage(ImageRenderingContext p) { Object saveImage = p.platformImage; final ImageRenderingContextparams = p; final com.sun.prism.paint.PaintcurrentPaint = p.platformPaintinstanceofcom.sun.prism.paint.Paint ? (com.sun.prism.paint.Paint)p.platformPaint : null; RenderJob re = new RenderJob(new Runnable() { private com.sun.prism.paint.ColorgetClearColor() { if (currentPaint == null) { return com.sun.prism.paint.Color.WHITE; } else if (currentPaint.getType() == com.sun.prism.paint.Paint.Type.COLOR) { return (com.sun.prism.paint.Color) currentPaint; } else if (currentPaint.isOpaque()) { return com.sun.prism.paint.Color.TRANSPARENT; } else { return com.sun.prism.paint.Color.WHITE; } } private void draw(Graphics g, int x, int y, int w, int h) { g.setDepthBuffer(params.depthBuffer); g.clear(getClearColor()); if (currentPaint != null && currentPaint.getType() != com.sun.prism.paint.Paint.Type.COLOR) { g.getRenderTarget().setOpaque(currentPaint.isOpaque()); g.setPaint(currentPaint); g.fillQuad(0, 0, w, h); } // Set up transform if (x != 0 || y != 0) { g.translate(-x, -y); } if (params.transform != null) { g.transform(params.transform); } if (params.root != null) { if (params.camera != null) { g.setCamera(params.camera); } NGNodengNode = params.root; ngNode.render(g); } } @Override public void run() { ResourceFactoryrf = GraphicsPipeline.getDefaultResourceFactory(); if (!rf.isDeviceReady()) { return; } Translates to this: private WritableImagedoSnapshot(SnapshotParametersparams, WritableImageimg) { if (getScene() != null) { getScene().doCSSLayoutSyncForSnapshot(this); } else { doCSSLayoutSyncForSnapshot(); } BaseTransform transform = BaseTransform.IDENTITY_TRANSFORM; if (params.getTransform() != null) { Affine3D tempTx = new Affine3D(); params.getTransform().impl_apply(tempTx); transform = tempTx; } double x; double y; double w; double h; Rectangle2D viewport = params.getViewport(); if (viewport != null) { // Use the specified viewport x = viewport.getMinX(); y = viewport.getMinY(); w = viewport.getWidth(); h = viewport.getHeight(); } else { // Get the bounds in parent of this node, transformed by the // specified transform. BaseBoundstempBounds = TempState.getInstance().bounds; tempBounds = getTransformedBounds(tempBounds, transform); x = tempBounds.getMinX(); y = tempBounds.getMinY(); w = tempBounds.getWidth(); h = tempBounds.getHeight(); } WritableImage result = Scene.doSnapshot(getScene(), x, y, w, h, this, transform, params.isDepthBufferInternal(), params.getFill(), params.getEffectiveCamera(), img); return result; } context.flushVertexBuffer(); GLContextglContext = context.getGLContext(); int id = glContext.getBoundFBO(); intfboID = getFboID(); booleanchangeBoundFBO = id != fboID; if (changeBoundFBO) { glContext.bindFBO(fboID); } boolean result = glContext.readPixels(pixels, x, y, width, height); if (changeBoundFBO) { glContext.bindFBO(id); } return result;

  41. Synergy Performance So… • Don’t do snapshot on each mouse event • Use AnimationTimer where applicable • Keep an eye on texture size • Reuse WritableImages

  42. Synergy Performance So… 3D UI demo

  43. Synergy Q & A

More Related