slide1
Download
Skip this Video
Download Presentation
Integrating JavaFX with Native Technologies

Loading in 2 Seconds...

play fullscreen
1 / 84

Integrating JavaFX with Native Technologies - PowerPoint PPT Presentation


  • 590 Views
  • Uploaded on

Integrating JavaFX with Native Technologies. Stephen Northover (Oracle) Felipe Heidrich (Oracle).

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Integrating JavaFX with Native Technologies' - miyo


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
integrating javafx with native technologies
Integrating JavaFX with Native Technologies

Stephen Northover (Oracle)Felipe Heidrich (Oracle)

slide3
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
program agenda
Program Agenda
  • FX Architecture: Overview
  • Native Code in Glass and Prism
  • Integrating OpenGL with JavaFX
  • Performance: The DiamondBenchmark
why might you need natives
Why might you need Natives?
  • Access operating system specific operations
    • Use Win32, Cocoa and GTK API
  • Integrate pre-existing native code
  • Improve performance
    • Native code can be faster (tight loops etc.)
    • Access faster platform specific operations
      • OpenGL 3.x or greater, DirectX 11 or greater
javafx architecture threading
JavaFX Architecture: Threading

start(Stage)

UI-Thread

(Glass, FX)

Sync

User Code

User Code

User Code

Sync

Render-Thread

(Prism)

Render Code

Render Code

javafx architecture synchronizing
JavaFX Architecture: Synchronizing

Render-Thread

(Prism)

UI-Thread

(Glass, FX)

Sync

glass architecture
Glass Architecture
  • Glass contains shared Java code (ie. View)
  • Glass contains platform specific Java code (ie. WinView)
  • Glass contains platform specific native peers (GlassView.cpp)
    • Native peers written in C/C++/Objective-C
    • Native peers contain native handles in their data structures
    • No API to get the native handles from Glass (*)

(*) An official API might be similar in concept to JAWT

what are the native handles
What are the Native handles?
  • Windows objects (C++, COM objects)
    • HWND, HHOOK, IUnknown …
  • Mac and iOS objects (Objective-C objects)
    • NSApplication, NSWindow, NSView …
  • GTK and Lens objects (C objects)
    • GtkWidget *, GdkPixbuf *, ..
where are the native handles
Where are the Native Handles?
  • Glass objects and native handles are not 1-to-1
  • View lightweight on Windows but not on Mac
    • No HWND on Windows, NSView on Mac … sometimes
    • In a Browser, on Mac, no NSView, only a CALayer
  • Need to write different code for different platforms
    • Glass uses different implementation languages
    • Glass uses different native objects to wrap handles
problem get the handle of a stage
Problem: Get the Handle of a Stage
  • Get the Glass object from the FX object
    • Quantum exists between Glass and FX
    • No API: Glass/Quantum objects are not public
  • Get the Native object from the Glass object
    • No API: Native code structure is subject to change
    • No API: Must include Glass header files / fragments
use reflection java or native code
Use Reflection: Java or Native Code

staticlonggetHandle (Stage stage) {

Object ws = stage.impl_getPeer();

try {

Method m1 = ws.getClass().getDeclaredMethod("getPlatformWindow");

m1.setAccessible(true);

Object w = m1.invoke(ws);

Method m2 = w.getClass().getSuperclass().getDeclaredMethod("getNativeWindow");

return(long) m2.invoke(w);

} catch (Throwableth) {}

return0;

}

use reflection java or native code1
Use Reflection: Java or Native Code

NOT API !

staticlonggetHandle (Stage stage) {

Object ws = stage.impl_getPeer();

try {

Method m1 = ws.getClass().getDeclaredMethod("getPlatformWindow");

m1.setAccessible(true);

Object w = m1.invoke(ws);

Method m2 = w.getClass().getSuperclass().getDeclaredMethod("getNativeWindow");

return(long) m2.invoke(w);

} catch (Throwableth) {}

return0;

}

prism architecture
Prism Architecture
  • Prism contains shared Java code (ie. BaseShaderContext)
  • Prism contains platform specific Java code (ie. ES2Conext)
  • Prism contains shared native code (ie. GLContext.c)
  • Prism contains platform specific native code (ie. MacGLContext.c)
    • Native code is written in C and C++ (a little Objective-C)
    • Native code contains native handles in their data structures
    • No API to get the native handles (same as Glass)
prism architecture graphics pipelines
Prism Architecture: Graphics Pipelines

Prism

Software

D3D

ES2

Java2D

Mac

Linux

FB

iOS

where are the opengl prism handles
Where are the OpenGL Prism handles?
  • OpenGL is a state machine
    • GL state is in a hidden platform specific “GL context”
    • Example: glScissor(x, y, width, height) // no GL context param
  • State can be queried when running in the Render-Thread
    • Query state using OpenGL calls and/or platform specific calls
  • State must be initialized when running in another thread
    • Share state (textures) with Prism using platform specific calls
platform calls to share gl state
Platform Calls to Share GL State
  • NSOpenGLContext>>initWithFormat:shareContext:
  • BOOL WINAPI wglShareLists( HGLRC hglrc1, HGLRC hglrc2 );
  • GLXContextglXCreateContext(Display *dpy,

XVisualInfo *vis,

GLXContextshareList,

Booldirect);

native calls to share gl state
Native Calls to Share GL State
  • EGLContexteglCreateContext(EGLDisplay display,

EGLConfigconfig,

EGLContextshare_context,

EGLintconst*attrib_list)

integrating opengl with javafx
To fill a shape with an image.

Use existing picture box, DO NOT delete and create new picture box.

Right click on the shape.

At the bottom of the submenu select “Format Shape”

Select “Fill” at the top of the “Format Shape” dialog box.

Select “Picture or Texture fill” from the options.

And select “File” under the “Insert from” option.

Navigate to the file you want to use and select “Insert”

On the “Format” tab, in the Size group, click on “Crop to Fill” in the Crop tool and drag the image bounding box to the desired size

DELETE THIS INSTRUCTION NOTE WHEN NOT IN USE

Integrating OpenGL with JavaFX
java opengl libraries
Java OpenGL Libraries
  • JOGL (Java Binding for the OpenGL API)
    • https://jogamp.org/jogl/www/
  • JLWGL (Light Weight Java Game Library)
    • http://www.lwjgl.org/
  • Eclipse Platform Generator
    • http://www.eclipse.org/swt/macgen.php
    • Not an official Eclipse component but useful
problem draw opengl content in javafx
Problem: Draw OpenGL content in JavaFX
  • Solution 1:
    • Draw OpenGL content to a byte buffer
    • Upload the byte buffer to an Image (UI-Thread)
  • Solution 2:
    • Draw OpenGL content as part of Prism
    • Provide an OpenGL object (texture) for Prism to draw
      • Why not just draw directly? (more on this later)
solution 1 draw using a byte buffer
Solution 1: Draw using a Byte Buffer
  • Advantages
    • Easy to understand and implement
    • Does not access internals of JavaFX and Prism
  • Disadvantages
    • Too slow when fps requirements are strict
    • Uses more memory than drawing as part of Prism
use opengl calls to get the bytes
Use OpenGL calls to get the bytes

glGetTexImage(GL_TEXTURE_2D, 0, GL_RGBA, GL_UNSIGNED_SHORT, buffer);

glReadPixels(0, 0, WIDTH, HEIGHT, GL_RGBA, GL_UNSIGNED_BYTE, buffer);

use fx code to write the bytes
Use FX code to write the bytes

PixelWriter pw = image.getPixelWriter();

PixelFormat pf = PixelFormat.getByteBgraInstance();

pw.setPixels(0, 0, WIDTH, HEIGHT, pf, buffer, WIDTH*4);

solution 2 draw as part of prism
Solution 2: Draw as part of Prism
  • Advantages
    • Simple to understand/explain
    • Fastest possible solution (least overhead)
  • Disadvantages
    • Must run / coordinate with Render Thread
    • Uses Prism internals and relies on representation
threading prism and opengl
Threading, Prism and OpenGL
  • Application code runs in the UI-thread
  • Prism code runs in the Render-thread
  • OpenGL threading rules:
    • OpenGL is not thread safe (it is thread aware)
    • Multiple threads are possible (each in their own apartment)
    • Platform specific code needed to create/share GL contexts
      • We’ve seen these calls before
in what thread should your code draw
In what thread should your code draw?
  • Draw in the Render-Thread
    • Must be careful not to conflict/hammer Prism state
  • Draw in another thread (even the UI-Thread)
    • Must create a GL context for that thread (apartment threading)
    • Must draw to a GL object (texture) for Render-Thread to draw
      • Cannot issue GL commands from “wrong thread”
      • Must synchronize around access to GL object (texture)
drawing in the render thread
Drawing in the Render Thread
  • Risky because Prism drawing model might change
  • Must save/restore any GL state that you set
    • Slow to save and restore state one at a time
    • Doesn’t sound that hard but easy to get wrong
  • How to integrate with Prism effects? Transforms?
    • Transforms are done in software (might change)
    • You won’t draw in the right place or the right way
opengl integration the prototype
OpenGL Integration: The Prototype
  • Draw in the Render Thread
    • Create your own GL context (platform specific)
    • Save/Restore Prisms GL context
  • Draw to a GL object (Texture)
    • Allow Prism to create the texture (but you draw to it)
    • Prism then draws the texture (translations, effects work correctly)
    • Dispose the texture using Prism when no longer needed
drawing in the render thread1
Drawing in the Render Thread
  • Take part in the UI-Thread/Render-Thread Sync
  • Subclass an FX Class
    • Prototype uses Pane
    • Reimplementimpl_createPeer() to return your NGNode
  • Subclass an NGNode (your NGNode)
    • Prototype subclasses NGRegion
    • ReimplementNGNode.renderContent(Graphics g)
drawing in the render thread2
Drawing in the Render Thread

NOT API !

  • Take part in the UI-Thread/Render-Thread Sync
  • Subclass an FX Class
    • Prototype uses Pane
    • Reimplementimpl_createPeer() to return your NGNode
  • Subclass an NGNode (your NGNode)
    • Prototype subclasses NGRegion
    • ReimplementNGNode.renderContent(Graphics g)
override rendercontent
Override renderContent()

@OverrideprotectedvoidrenderContent(Graphics g) {

// Save the Prism GL context, initialize the user GL context

Texture texture = getPrismTexture();

NSOpenGLContextprismContext = NSOpenGLContext.currentContext();

NSOpenGLContextuserContext = getUserContext(prismContext, texture);

// Set the user GL Context

userContext.makeCurrentContext();

// Execute GL commands in the user GL context, and flush (or nothing draws)

TorusLWJGL.resizeTorus(WIDTH, HEIGHT);

TorusLWJGL.drawTorus();

glFlush();

// Restore the Prism GL Context, draw the texture

prismContext.makeCurrentContext();

g.drawTexture(texture, 0, 0, WIDTH, HEIGHT, 0, 0, WIDTH, HEIGHT);

}

implement getprismtexture
Implement getPrismTexture()

// Cache a single texture (released in setPrismTexture())

Texture prismTexture;

Texture getPrismTexture() {

if(prismTexture == null) {

ResourceFactoryf = GraphicsPipeline.getDefaultResourceFactory();

prismTexture= f.createTexture(PixelFormat.INT_ARGB_PRE,

Texture.Usage.DEFAULT,

Texture.WrapMode.CLAMP_NOT_NEEDED,

WIDTH, HEIGHT);

prismTexture.makePermanent();

}

returnprismTexture;

}

implement getusercontext
Implement getUserContext()

NSOpenGLContextgetUserContext(NSOpenGLContextprismContext, Texture prismTexture) {

if(userContext == null) {

userContext= (NSOpenGLContext) newNSOpenGLContext().alloc();

NSOpenGLPixelFormatpixelFormat = (NSOpenGLPixelFormat)newNSOpenGLPixelFormat().alloc();

pixelFormat.initWithAttributes(newint [] {0, });

userContext= userContext.initWithFormat(pixelFormat, prismContext);

userContext.makeCurrentContext();

try {GLContext.useContext(userContext, false);} catch (LWJGLException e) {e.printStackTrace();}

intuserFBO = glGenFramebuffersEXT();

glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, userFBO);

intprismTextureID = com.sun.prism.es2.SetID.getID(prismTexture);

glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT,

GL_COLOR_ATTACHMENT0_EXT, GL_TEXTURE_2D, prismTextureID, 0);

}

returnuserContext;

}

important follow threading rules
Important: Follow Threading Rules
  • Application code runs in the UI-Thread
  • renderContent() is called in the Render-Thread
  • Do not access UI-Thread state
    • Copy state from UI-Thread FX object to Render-Thread NGNode
    • It is easy to forget to do this but this leads to instability / crashes
the way forward
The Way Forward
  • Implement GLNode for JavaFX
    • Provide minimal API (GL context, Prism texture)
    • Enough API to implement simple GL integration
  • Implement JOGL and LWJGL binding for JavaFX
    • Provide appropriate drawing mechanism for the library
    • Example: JOGL Autodrawable

NOTE: Only ES2 platforms supported (non-Windows)

the diamondbenchmark
To fill a shape with an image.

Use existing picture box, DO NOT delete and create new picture box.

Right click on the shape.

At the bottom of the submenu select “Format Shape”

Select “Fill” at the top of the “Format Shape” dialog box.

Select “Picture or Texture fill” from the options.

And select “File” under the “Insert from” option.

Navigate to the file you want to use and select “Insert”

On the “Format” tab, in the Size group, click on “Crop to Fill” in the Crop tool and drag the image bounding box to the desired size

DELETE THIS INSTRUCTION NOTE WHEN NOT IN USE

The DiamondBenchmark
the diamond benchmark
The Diamond Benchmark
  • Strict requirements:
    • Must handle thousands of small objects (“diamonds”)
    • Objects are constantly moved and updated
    • Multi-monitor, large area needs to be drawn
    • Performance critical (must render at 60 fps)
  • Examples:
    • Sea urchins on the ocean floor, bees swarming around a hive, targets being tracked by missile defense system
first version use nodes
First Version: Use Nodes
  • Node for each diamond
    • Use standard FX coding conventions
      • Subclass of Region
      • Use properties for variables
  • Result:
    • Too slow, animation is not smooth … why?
    • Too many nodes, too much data, too many updates
second version use canvas
Second Version: Use Canvas
  • Lightweight object for each diamond
    • Use standard FX coding conventions
      • Subclass of Object (instead of Node)
      • Use properties for variables
  • Result:
    • Too slow (slower than using Nodes) … why?
    • Canvas uses immediate mode drawing (no nodes)
    • Prism caches node drawing (can’t cache immediate mode)
third version use custom opengl
Third Version: Use Custom OpenGL
  • Lightweight object for each diamond
    • Use optimal Java coding conventions
    • Use optimal OpenGL conventions / data structures
    • Use LWJGL for convenience (recode in C and compare)
  • Result:
    • Meets requirements, animation is smooth ... why?
    • Uses an optimal OpenGL drawing strategy
    • Uses platform specific OpenGL extensions
how prism draws the diamonds
How Prism draws the Diamonds
  • A Diamond is a simple path (5 vertices)
    • There is a path for each diamond node
    • The diamond is transformed on the screen
  • Prism renders each path separately
  • Each path requires:
    • Vertex coordinates, texture coordinates, color data etc.
    • Rendering gets slower as the number of paths increase
a faster way to draw the diamonds
A Faster way to draw the Diamonds
  • Use OpenGL to do instanced drawing
    • Put 5 path vertices on the graphics card and leave them
    • Use glDrawArraysInstanced() from OpenGL 3.1 to draw
  • Transfer the minimal amount of data (just x and y)
    • Use a vertex shader to apply the transform
prism versus custom opengl
Prism versus Custom OpenGL

“By knowing the content being rendered, an OpenGL developer can use techniques that Prism, as a general purpose graphics engine, does have the context to use.”

“At the price of portability, an OpenGL program can take advantage of extensions and platform specific mechanisms that are not available to Prism.”

summary
Summary
  • Need an API like JAWT for Glass
    • Reflection and private header files don’t cut it
  • Need first class support for OpenGL in JavaFX
    • Drawing to a texture looks promising
    • Low level hacks / undocumented calls don’t cut it
  • OpenGL can be used to improve performance
    • Use of platform / GL version specific features
    • Lots of drawing / updates (you won’t beat Prism otherwise)
main title goes here
Main Title Goes Here

Insert Presenter’s Name HereInsert Presenter’s Title Here

program agenda1
Program Agenda
  • Topic 1, Arial, 24 pt
  • Topic 2, Arial, 24 pt
  • Topic 3, Arial, 24 pt
  • Topic 4, Arial, 24 pt
  • Topic 5. More than 5 topics, add second agenda slide.
slide title arial 28 pt two line max
Slide Title: Arial, 28 PTTwo-Line Max

Subtitle: Arial, 20 pt, One-Line Max

  • Bullets are sentence case. Use Arial, 20 pt font.
    • Sub-bullets are Arial, 18 pt font.
  • Keep bullets short.
  • One idea per bullet.
  • No more than five bullets.
  • NOTE: Arial is the ONLY font that should be used in the Oracle corporate presentation template.Times and other serif fonts are not acceptable.

To ensure that slides are properly formatted to this template, see pages 7 and 8 for instructions.

Times

Arial

slide title arial 28 pt one line
Slide Title: Arial, 28 PT, One-Line

Subtitle: Arial, 20 pt, One-Line Max

  • Bullets are sentence case. Use Arial, 20 pt font.
    • Sub-bullets are Arial, 18 pt font.
  • Keep bullets short.
  • One idea per bullet.
  • No more than five bullets.
  • NOTE: Arial is the ONLY font that should be used in the Oracle corporate presentation template.Times and other serif fonts are not acceptable.

To ensure that slides are properly formatted to this template, see pages 7 and 8 for instructions.

safe harbor statements
Safe Harbor Statements
  • One of the following slides must be used if your presentation covers material affected by Oracle’s Revenue Recognition Policy
  • To learn more about this policy, e-mail: [email protected]
slide61
The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract.It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
slide62
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
image section divider
To fill a shape with an image.

Use existing picture box, DO NOT delete and create new picture box.

Right click on the shape.

At the bottom of the submenu select “Format Shape”

Select “Fill” at the top of the “Format Shape” dialog box.

Select “Picture or Texture fill” from the options.

And select “File” under the “Insert from” option.

Navigate to the file you want to use and select “Insert”

On the “Format” tab, in the Size group, click on “Crop to Fill” in the Crop tool and drag the image bounding box to the desired size

DELETE THIS INSTRUCTION NOTE WHEN NOT IN USE

Image Section Divider
slide64
ORACLE

PRODUCT

LOGO

Announcement ALL CAPS, ARIAL 44 PT, 4-LINE MAX

DELETE IF NOT IN USE

title arial 28 pt two line max
ORACLE

PRODUCT

LOGO

Title, Arial, 28 PT, Two-Line Max

BODY/STATEMENT COPY, ALL CAPS, 24 PT, HIGHLIGHT TEXT IN BLUE, USED FOR EMPHASIS

DELETE IF NOT IN USE

case study example with photograph
Case Study Example with Photograph
  • Solutions
  • Loremipsum dolor sit amet, consecteturadipiscingelit. Fusce a sagittisorci.
  • Maecenas vehiculalorempharetraipsumsuscipitutvenenatisrisusfacilisis. Donecvelaugue vitae urnaaliquetcursus. Aliquamrutrumiaculis.
case study example with screenshot
Case Study Example with Screenshot
  • Loremipsum dolor sit amet, consecteturadipiscingelit. Fusce a sagittisorci.
  • Maecenas vehiculalorempharetraipsumsuscipitutvenenatisrisusfacilisis. Donecvelaugue vitae urnaaliquetcursus. Aliquamrutrumiaculis.
  • Solutions
case study example with graphic
Case Study Example with Graphic
  • Loremipsum dolor sit amet, consecteturadipiscingelit. Fusce a sagittisorci.
  • Maecenas vehiculalorempharetraipsumsuscipitutvenenatisrisusfacilisis. Donecvelaugue vitae urnaaliquetcursus. Aliquamrutrumiaculis.
  • Solutions
case study example with partner logos
Case Study Example with Partner Logos
  • Loremipsum dolor sit amet, consecteturadipiscingelit. Fusce a sagittisorci.
  • Maecenas vehiculalorempharetraipsumsuscipitutvenenatisrisusfacilisis. Donecvelaugue vitae urnaaliquetcursus. Aliquamrutrumiaculis.
  • Solutions
slide70
THIRD PARTY COMPANY LOGO

Insert Author Name Here

Position Title, Company Name

“This slide format serves to call attention to a quote from a prominent customer, executive, or thought leader regarding a particular topic.”

DELETE IF NOT IN USE

vertical bar chart
Vertical Bar Chart

Contextual information about the chart

  • Bullet copy
  • Bullet copy
  • Bullet copy
horizontal bar chart
Horizontal Bar Chart

Contextual information about the chart

  • Bullet copy
  • Bullet copy
  • Bullet copy
stacked chart
Stacked Chart

Contextual information about the chart

  • Bullet copy
  • Bullet copy
  • Bullet copy
line chart
Line Chart

Contextual information about the chart

  • Bullet copy
  • Bullet copy
  • Bullet copy
pie chart
Pie Chart

Contextual information about the chart

  • Bullet copy
  • Bullet copy
  • Bullet copy
ad