Integrating javafx with native technologies
Download
1 / 84

Integrating JavaFX with Native Technologies - PowerPoint PPT Presentation


  • 560 Views
  • Uploaded on
  • Presentation posted in: General

Integrating JavaFX with Native Technologies. Stephen Northover (Oracle) Felipe Heidrich (Oracle).

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha

Download Presentation

Integrating JavaFX with Native Technologies

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Integrating JavaFX with Native Technologies

Stephen Northover (Oracle)Felipe Heidrich (Oracle)


The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Program Agenda

  • FX Architecture: Overview

  • Native Code in Glass and Prism

  • Integrating OpenGL with JavaFX

  • Performance: The DiamondBenchmark


Why might you need Natives?

  • Access operating system specific operations

    • Use Win32, Cocoa and GTK API

  • Integrate pre-existing native code

  • Improve performance

    • Native code can be faster (tight loops etc.)

    • Access faster platform specific operations

      • OpenGL 3.x or greater, DirectX 11 or greater


JavaFX Architecture: Glass and Prism


JavaFX Architecture: Threading

start(Stage)

UI-Thread

(Glass, FX)

Sync

User Code

User Code

User Code

Sync

Render-Thread

(Prism)

Render Code

Render Code


JavaFX Architecture: Synchronizing

Render-Thread

(Prism)

UI-Thread

(Glass, FX)

Sync


Glass Architecture

  • Glass contains shared Java code (ie. View)

  • Glass contains platform specific Java code (ie. WinView)

  • Glass contains platform specific native peers (GlassView.cpp)

    • Native peers written in C/C++/Objective-C

    • Native peers contain native handles in their data structures

    • No API to get the native handles from Glass (*)

(*) An official API might be similar in concept to JAWT


What are the Native handles?

  • Windows objects (C++, COM objects)

    • HWND, HHOOK, IUnknown …

  • Mac and iOS objects (Objective-C objects)

    • NSApplication, NSWindow, NSView …

  • GTK and Lens objects (C objects)

    • GtkWidget *, GdkPixbuf *, ..


Where are the Native Handles?

  • Glass objects and native handles are not 1-to-1

  • View lightweight on Windows but not on Mac

    • No HWND on Windows, NSView on Mac … sometimes

    • In a Browser, on Mac, no NSView, only a CALayer

  • Need to write different code for different platforms

    • Glass uses different implementation languages

    • Glass uses different native objects to wrap handles


Problem: Get the Handle of a Stage

  • Get the Glass object from the FX object

    • Quantum exists between Glass and FX

    • No API: Glass/Quantum objects are not public

  • Get the Native object from the Glass object

    • No API: Native code structure is subject to change

    • No API: Must include Glass header files / fragments


Use Reflection: Java or Native Code

staticlonggetHandle (Stage stage) {

Object ws = stage.impl_getPeer();

try {

Method m1 = ws.getClass().getDeclaredMethod("getPlatformWindow");

m1.setAccessible(true);

Object w = m1.invoke(ws);

Method m2 = w.getClass().getSuperclass().getDeclaredMethod("getNativeWindow");

return(long) m2.invoke(w);

} catch (Throwableth) {}

return0;

}


Use Reflection: Java or Native Code

NOT API !

staticlonggetHandle (Stage stage) {

Object ws = stage.impl_getPeer();

try {

Method m1 = ws.getClass().getDeclaredMethod("getPlatformWindow");

m1.setAccessible(true);

Object w = m1.invoke(ws);

Method m2 = w.getClass().getSuperclass().getDeclaredMethod("getNativeWindow");

return(long) m2.invoke(w);

} catch (Throwableth) {}

return0;

}


Example: Open a Native Dialog on a Mac


Prism Architecture

  • Prism contains shared Java code (ie. BaseShaderContext)

  • Prism contains platform specific Java code (ie. ES2Conext)

  • Prism contains shared native code (ie. GLContext.c)

  • Prism contains platform specific native code (ie. MacGLContext.c)

    • Native code is written in C and C++ (a little Objective-C)

    • Native code contains native handles in their data structures

    • No API to get the native handles (same as Glass)


Prism Architecture: Graphics Pipelines

Prism

Software

D3D

ES2

Java2D

Mac

Linux

FB

iOS


Where are the OpenGL Prism handles?

  • OpenGL is a state machine

    • GL state is in a hidden platform specific “GL context”

    • Example: glScissor(x, y, width, height) // no GL context param

  • State can be queried when running in the Render-Thread

    • Query state using OpenGL calls and/or platform specific calls

  • State must be initialized when running in another thread

    • Share state (textures) with Prism using platform specific calls


Platform Calls to Share GL State

  • NSOpenGLContext>>initWithFormat:shareContext:

  • BOOL WINAPI wglShareLists( HGLRC hglrc1, HGLRC hglrc2 );

  • GLXContextglXCreateContext(Display *dpy,

    XVisualInfo *vis,

    GLXContextshareList,

    Booldirect);


Native Calls to Share GL State

  • EGLContexteglCreateContext(EGLDisplay display,

    EGLConfigconfig,

    EGLContextshare_context,

    EGLintconst*attrib_list)


To fill a shape with an image.

Use existing picture box, DO NOT delete and create new picture box.

Right click on the shape.

At the bottom of the submenu select “Format Shape”

Select “Fill” at the top of the “Format Shape” dialog box.

Select “Picture or Texture fill” from the options.

And select “File” under the “Insert from” option.

Navigate to the file you want to use and select “Insert”

On the “Format” tab, in the Size group, click on “Crop to Fill” in the Crop tool and drag the image bounding box to the desired size

DELETE THIS INSTRUCTION NOTE WHEN NOT IN USE

Integrating OpenGL with JavaFX


Java OpenGL Libraries

  • JOGL (Java Binding for the OpenGL API)

    • https://jogamp.org/jogl/www/

  • JLWGL (Light Weight Java Game Library)

    • http://www.lwjgl.org/

  • Eclipse Platform Generator

    • http://www.eclipse.org/swt/macgen.php

    • Not an official Eclipse component but useful


Problem: Draw OpenGL content in JavaFX

  • Solution 1:

    • Draw OpenGL content to a byte buffer

    • Upload the byte buffer to an Image (UI-Thread)

  • Solution 2:

    • Draw OpenGL content as part of Prism

    • Provide an OpenGL object (texture) for Prism to draw

      • Why not just draw directly? (more on this later)


Solution 1: Draw using a Byte Buffer

  • Advantages

    • Easy to understand and implement

    • Does not access internals of JavaFX and Prism

  • Disadvantages

    • Too slow when fps requirements are strict

    • Uses more memory than drawing as part of Prism


Use OpenGL calls to get the bytes

glGetTexImage(GL_TEXTURE_2D, 0, GL_RGBA, GL_UNSIGNED_SHORT, buffer);

glReadPixels(0, 0, WIDTH, HEIGHT, GL_RGBA, GL_UNSIGNED_BYTE, buffer);


Use FX code to write the bytes

PixelWriter pw = image.getPixelWriter();

PixelFormat<ByteBuffer> pf = PixelFormat.getByteBgraInstance();

pw.setPixels(0, 0, WIDTH, HEIGHT, pf, buffer, WIDTH*4);


Demo: Draw a Torus using a Byte Buffer


Solution 2: Draw as part of Prism

  • Advantages

    • Simple to understand/explain

    • Fastest possible solution (least overhead)

  • Disadvantages

    • Must run / coordinate with Render Thread

    • Uses Prism internals and relies on representation


Threading, Prism and OpenGL

  • Application code runs in the UI-thread

  • Prism code runs in the Render-thread

  • OpenGL threading rules:

    • OpenGL is not thread safe (it is thread aware)

    • Multiple threads are possible (each in their own apartment)

    • Platform specific code needed to create/share GL contexts

      • We’ve seen these calls before


In what thread should your code draw?

  • Draw in the Render-Thread

    • Must be careful not to conflict/hammer Prism state

  • Draw in another thread (even the UI-Thread)

    • Must create a GL context for that thread (apartment threading)

    • Must draw to a GL object (texture) for Render-Thread to draw

      • Cannot issue GL commands from “wrong thread”

      • Must synchronize around access to GL object (texture)


Drawing in the Render Thread

  • Risky because Prism drawing model might change

  • Must save/restore any GL state that you set

    • Slow to save and restore state one at a time

    • Doesn’t sound that hard but easy to get wrong

  • How to integrate with Prism effects? Transforms?

    • Transforms are done in software (might change)

    • You won’t draw in the right place or the right way


OpenGL Integration: The Prototype

  • Draw in the Render Thread

    • Create your own GL context (platform specific)

    • Save/Restore Prisms GL context

  • Draw to a GL object (Texture)

    • Allow Prism to create the texture (but you draw to it)

    • Prism then draws the texture (translations, effects work correctly)

    • Dispose the texture using Prism when no longer needed


Drawing in the Render Thread

  • Take part in the UI-Thread/Render-Thread Sync

  • Subclass an FX Class

    • Prototype uses Pane

    • Reimplementimpl_createPeer() to return your NGNode

  • Subclass an NGNode (your NGNode)

    • Prototype subclasses NGRegion

    • ReimplementNGNode.renderContent(Graphics g)


Drawing in the Render Thread

NOT API !

  • Take part in the UI-Thread/Render-Thread Sync

  • Subclass an FX Class

    • Prototype uses Pane

    • Reimplementimpl_createPeer() to return your NGNode

  • Subclass an NGNode (your NGNode)

    • Prototype subclasses NGRegion

    • ReimplementNGNode.renderContent(Graphics g)


Override renderContent()

@OverrideprotectedvoidrenderContent(Graphics g) {

// Save the Prism GL context, initialize the user GL context

Texture texture = getPrismTexture();

NSOpenGLContextprismContext = NSOpenGLContext.currentContext();

NSOpenGLContextuserContext = getUserContext(prismContext, texture);

// Set the user GL Context

userContext.makeCurrentContext();

// Execute GL commands in the user GL context, and flush (or nothing draws)

TorusLWJGL.resizeTorus(WIDTH, HEIGHT);

TorusLWJGL.drawTorus();

glFlush();

// Restore the Prism GL Context, draw the texture

prismContext.makeCurrentContext();

g.drawTexture(texture, 0, 0, WIDTH, HEIGHT, 0, 0, WIDTH, HEIGHT);

}


Implement getPrismTexture()

// Cache a single texture (released in setPrismTexture())

Texture prismTexture;

Texture getPrismTexture() {

if(prismTexture == null) {

ResourceFactoryf = GraphicsPipeline.getDefaultResourceFactory();

prismTexture= f.createTexture(PixelFormat.INT_ARGB_PRE,

Texture.Usage.DEFAULT,

Texture.WrapMode.CLAMP_NOT_NEEDED,

WIDTH, HEIGHT);

prismTexture.makePermanent();

}

returnprismTexture;

}


Implement getUserContext()

NSOpenGLContextgetUserContext(NSOpenGLContextprismContext, Texture prismTexture) {

if(userContext == null) {

userContext= (NSOpenGLContext) newNSOpenGLContext().alloc();

NSOpenGLPixelFormatpixelFormat = (NSOpenGLPixelFormat)newNSOpenGLPixelFormat().alloc();

pixelFormat.initWithAttributes(newint [] {0, });

userContext= userContext.initWithFormat(pixelFormat, prismContext);

userContext.makeCurrentContext();

try {GLContext.useContext(userContext, false);} catch (LWJGLException e) {e.printStackTrace();}

intuserFBO = glGenFramebuffersEXT();

glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, userFBO);

intprismTextureID = com.sun.prism.es2.SetID.getID(prismTexture);

glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT,

GL_COLOR_ATTACHMENT0_EXT, GL_TEXTURE_2D, prismTextureID, 0);

}

returnuserContext;

}


Important: Follow Threading Rules

  • Application code runs in the UI-Thread

  • renderContent() is called in the Render-Thread

  • Do not access UI-Thread state

    • Copy state from UI-Thread FX object to Render-Thread NGNode

    • It is easy to forget to do this but this leads to instability / crashes


Demo: Draw Torus inside Prism


The Way Forward

  • Implement GLNode for JavaFX

    • Provide minimal API (GL context, Prism texture)

    • Enough API to implement simple GL integration

  • Implement JOGL and LWJGL binding for JavaFX

    • Provide appropriate drawing mechanism for the library

    • Example: JOGL Autodrawable

NOTE: Only ES2 platforms supported (non-Windows)


To fill a shape with an image.

Use existing picture box, DO NOT delete and create new picture box.

Right click on the shape.

At the bottom of the submenu select “Format Shape”

Select “Fill” at the top of the “Format Shape” dialog box.

Select “Picture or Texture fill” from the options.

And select “File” under the “Insert from” option.

Navigate to the file you want to use and select “Insert”

On the “Format” tab, in the Size group, click on “Crop to Fill” in the Crop tool and drag the image bounding box to the desired size

DELETE THIS INSTRUCTION NOTE WHEN NOT IN USE

The DiamondBenchmark


The Diamond Benchmark

  • Strict requirements:

    • Must handle thousands of small objects (“diamonds”)

    • Objects are constantly moved and updated

    • Multi-monitor, large area needs to be drawn

    • Performance critical (must render at 60 fps)

  • Examples:

    • Sea urchins on the ocean floor, bees swarming around a hive, targets being tracked by missile defense system


First Version: Use Nodes

  • Node for each diamond

    • Use standard FX coding conventions

      • Subclass of Region

      • Use properties for variables

  • Result:

    • Too slow, animation is not smooth … why?

    • Too many nodes, too much data, too many updates


Example: DiamondBenchmarkFX


Second Version: Use Canvas

  • Lightweight object for each diamond

    • Use standard FX coding conventions

      • Subclass of Object (instead of Node)

      • Use properties for variables

  • Result:

    • Too slow (slower than using Nodes) … why?

    • Canvas uses immediate mode drawing (no nodes)

    • Prism caches node drawing (can’t cache immediate mode)


Example: DiamondBenchmarkCanvas


Third Version: Use Custom OpenGL

  • Lightweight object for each diamond

    • Use optimal Java coding conventions

    • Use optimal OpenGL conventions / data structures

    • Use LWJGL for convenience (recode in C and compare)

  • Result:

    • Meets requirements, animation is smooth ... why?

    • Uses an optimal OpenGL drawing strategy

    • Uses platform specific OpenGL extensions


Example: DiamondBenchmarkLWJGL


How Prism draws the Diamonds

  • A Diamond is a simple path (5 vertices)

    • There is a path for each diamond node

    • The diamond is transformed on the screen

  • Prism renders each path separately

  • Each path requires:

    • Vertex coordinates, texture coordinates, color data etc.

    • Rendering gets slower as the number of paths increase


A Faster way to draw the Diamonds

  • Use OpenGL to do instanced drawing

    • Put 5 path vertices on the graphics card and leave them

    • Use glDrawArraysInstanced() from OpenGL 3.1 to draw

  • Transfer the minimal amount of data (just x and y)

    • Use a vertex shader to apply the transform


Prism versus Custom OpenGL

“By knowing the content being rendered, an OpenGL developer can use techniques that Prism, as a general purpose graphics engine, does have the context to use.”

“At the price of portability, an OpenGL program can take advantage of extensions and platform specific mechanisms that are not available to Prism.”


Summary

  • Need an API like JAWT for Glass

    • Reflection and private header files don’t cut it

  • Need first class support for OpenGL in JavaFX

    • Drawing to a texture looks promising

    • Low level hacks / undocumented calls don’t cut it

  • OpenGL can be used to improve performance

    • Use of platform / GL version specific features

    • Lots of drawing / updates (you won’t beat Prism otherwise)


Questions?


Main Title Goes Here

Insert Presenter’s Name HereInsert Presenter’s Title Here


Program Agenda

  • Topic 1, Arial, 24 pt

  • Topic 2, Arial, 24 pt

  • Topic 3, Arial, 24 pt

  • Topic 4, Arial, 24 pt

  • Topic 5. More than 5 topics, add second agenda slide.


Slide Title: Arial, 28 PTTwo-Line Max

Subtitle: Arial, 20 pt, One-Line Max

  • Bullets are sentence case. Use Arial, 20 pt font.

    • Sub-bullets are Arial, 18 pt font.

  • Keep bullets short.

  • One idea per bullet.

  • No more than five bullets.

  • NOTE: Arial is the ONLY font that should be used in the Oracle corporate presentation template.Times and other serif fonts are not acceptable.

To ensure that slides are properly formatted to this template, see pages 7 and 8 for instructions.

Times

Arial


Slide Title: Arial, 28 PT, One-Line

Subtitle: Arial, 20 pt, One-Line Max

  • Bullets are sentence case. Use Arial, 20 pt font.

    • Sub-bullets are Arial, 18 pt font.

  • Keep bullets short.

  • One idea per bullet.

  • No more than five bullets.

  • NOTE: Arial is the ONLY font that should be used in the Oracle corporate presentation template.Times and other serif fonts are not acceptable.

To ensure that slides are properly formatted to this template, see pages 7 and 8 for instructions.


Graphic Section Divider


Safe Harbor Statements

  • One of the following slides must be used if your presentation covers material affected by Oracle’s Revenue Recognition Policy

  • To learn more about this policy, e-mail: Revrec-americasiebc_us@oracle.com


The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract.It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


To fill a shape with an image.

Use existing picture box, DO NOT delete and create new picture box.

Right click on the shape.

At the bottom of the submenu select “Format Shape”

Select “Fill” at the top of the “Format Shape” dialog box.

Select “Picture or Texture fill” from the options.

And select “File” under the “Insert from” option.

Navigate to the file you want to use and select “Insert”

On the “Format” tab, in the Size group, click on “Crop to Fill” in the Crop tool and drag the image bounding box to the desired size

DELETE THIS INSTRUCTION NOTE WHEN NOT IN USE

Image Section Divider


ORACLE

PRODUCT

LOGO

Announcement ALL CAPS, ARIAL 44 PT, 4-LINE MAX

DELETE IF NOT IN USE


ORACLE

PRODUCT

LOGO

Title, Arial, 28 PT, Two-Line Max

BODY/STATEMENT COPY, ALL CAPS, 24 PT, HIGHLIGHT TEXT IN BLUE, USED FOR EMPHASIS

DELETE IF NOT IN USE


Case Study Example with Photograph

  • Solutions

  • Loremipsum dolor sit amet, consecteturadipiscingelit. Fusce a sagittisorci.

  • Maecenas vehiculalorempharetraipsumsuscipitutvenenatisrisusfacilisis. Donecvelaugue vitae urnaaliquetcursus. Aliquamrutrumiaculis.


Case Study Example with Screenshot

  • Loremipsum dolor sit amet, consecteturadipiscingelit. Fusce a sagittisorci.

  • Maecenas vehiculalorempharetraipsumsuscipitutvenenatisrisusfacilisis. Donecvelaugue vitae urnaaliquetcursus. Aliquamrutrumiaculis.

  • Solutions


Case Study Example with Graphic

  • Loremipsum dolor sit amet, consecteturadipiscingelit. Fusce a sagittisorci.

  • Maecenas vehiculalorempharetraipsumsuscipitutvenenatisrisusfacilisis. Donecvelaugue vitae urnaaliquetcursus. Aliquamrutrumiaculis.

  • Solutions


Case Study Example with Partner Logos

  • Loremipsum dolor sit amet, consecteturadipiscingelit. Fusce a sagittisorci.

  • Maecenas vehiculalorempharetraipsumsuscipitutvenenatisrisusfacilisis. Donecvelaugue vitae urnaaliquetcursus. Aliquamrutrumiaculis.

  • Solutions


THIRD PARTY COMPANY LOGO

Insert Author Name Here

Position Title, Company Name

“This slide format serves to call attention to a quote from a prominent customer, executive, or thought leader regarding a particular topic.”

DELETE IF NOT IN USE


Classic Duke


Vertical Bar Chart

Contextual information about the chart

  • Bullet copy

  • Bullet copy

  • Bullet copy


Horizontal Bar Chart

Contextual information about the chart

  • Bullet copy

  • Bullet copy

  • Bullet copy


Stacked Chart

Contextual information about the chart

  • Bullet copy

  • Bullet copy

  • Bullet copy


Line Chart

Contextual information about the chart

  • Bullet copy

  • Bullet copy

  • Bullet copy


Pie Chart

Contextual information about the chart

  • Bullet copy

  • Bullet copy

  • Bullet copy


Tables


Tables


Tables


Tables


Tables


Tables


Tables


ad
  • Login