Download
1 / 84

Integrating JavaFX with Native Technologies - PowerPoint PPT Presentation


  • 583 Views
  • Uploaded on

Integrating JavaFX with Native Technologies. Stephen Northover (Oracle) Felipe Heidrich (Oracle).

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Integrating JavaFX with Native Technologies' - miyo


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Integrating javafx with native technologies
Integrating JavaFX with Native Technologies

Stephen Northover (Oracle)Felipe Heidrich (Oracle)


The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Program agenda
Program Agenda direction.

  • FX Architecture: Overview

  • Native Code in Glass and Prism

  • Integrating OpenGL with JavaFX

  • Performance: The DiamondBenchmark


Why might you need natives
Why might you need Natives? direction.

  • Access operating system specific operations

    • Use Win32, Cocoa and GTK API

  • Integrate pre-existing native code

  • Improve performance

    • Native code can be faster (tight loops etc.)

    • Access faster platform specific operations

      • OpenGL 3.x or greater, DirectX 11 or greater


Javafx architecture glass and prism
JavaFX direction. Architecture: Glass and Prism


Javafx architecture threading
JavaFX direction. Architecture: Threading

start(Stage)

UI-Thread

(Glass, FX)

Sync

User Code

User Code

User Code

Sync

Render-Thread

(Prism)

Render Code

Render Code


Javafx architecture synchronizing
JavaFX direction. Architecture: Synchronizing

Render-Thread

(Prism)

UI-Thread

(Glass, FX)

Sync


Glass architecture
Glass Architecture direction.

  • Glass contains shared Java code (ie. View)

  • Glass contains platform specific Java code (ie. WinView)

  • Glass contains platform specific native peers (GlassView.cpp)

    • Native peers written in C/C++/Objective-C

    • Native peers contain native handles in their data structures

    • No API to get the native handles from Glass (*)

(*) An official API might be similar in concept to JAWT


What are the native handles
What are the Native handles? direction.

  • Windows objects (C++, COM objects)

    • HWND, HHOOK, IUnknown …

  • Mac and iOS objects (Objective-C objects)

    • NSApplication, NSWindow, NSView …

  • GTK and Lens objects (C objects)

    • GtkWidget *, GdkPixbuf *, ..


Where are the native handles
Where are the Native Handles? direction.

  • Glass objects and native handles are not 1-to-1

  • View lightweight on Windows but not on Mac

    • No HWND on Windows, NSView on Mac … sometimes

    • In a Browser, on Mac, no NSView, only a CALayer

  • Need to write different code for different platforms

    • Glass uses different implementation languages

    • Glass uses different native objects to wrap handles


Problem get the handle of a stage
Problem: Get the Handle of a Stage direction.

  • Get the Glass object from the FX object

    • Quantum exists between Glass and FX

    • No API: Glass/Quantum objects are not public

  • Get the Native object from the Glass object

    • No API: Native code structure is subject to change

    • No API: Must include Glass header files / fragments


Use reflection java or native code
Use Reflection: Java or Native Code direction.

staticlonggetHandle (Stage stage) {

Object ws = stage.impl_getPeer();

try {

Method m1 = ws.getClass().getDeclaredMethod("getPlatformWindow");

m1.setAccessible(true);

Object w = m1.invoke(ws);

Method m2 = w.getClass().getSuperclass().getDeclaredMethod("getNativeWindow");

return(long) m2.invoke(w);

} catch (Throwableth) {}

return0;

}


Use reflection java or native code1
Use direction. Reflection: Java or Native Code

NOT API !

staticlonggetHandle (Stage stage) {

Object ws = stage.impl_getPeer();

try {

Method m1 = ws.getClass().getDeclaredMethod("getPlatformWindow");

m1.setAccessible(true);

Object w = m1.invoke(ws);

Method m2 = w.getClass().getSuperclass().getDeclaredMethod("getNativeWindow");

return(long) m2.invoke(w);

} catch (Throwableth) {}

return0;

}



Prism architecture
Prism Architecture direction.

  • Prism contains shared Java code (ie. BaseShaderContext)

  • Prism contains platform specific Java code (ie. ES2Conext)

  • Prism contains shared native code (ie. GLContext.c)

  • Prism contains platform specific native code (ie. MacGLContext.c)

    • Native code is written in C and C++ (a little Objective-C)

    • Native code contains native handles in their data structures

    • No API to get the native handles (same as Glass)


Prism architecture graphics pipelines
Prism Architecture: Graphics Pipelines direction.

Prism

Software

D3D

ES2

Java2D

Mac

Linux

FB

iOS


Where are the opengl prism handles
Where are the OpenGL Prism handles? direction.

  • OpenGL is a state machine

    • GL state is in a hidden platform specific “GL context”

    • Example: glScissor(x, y, width, height) // no GL context param

  • State can be queried when running in the Render-Thread

    • Query state using OpenGL calls and/or platform specific calls

  • State must be initialized when running in another thread

    • Share state (textures) with Prism using platform specific calls


Platform calls to share gl state
Platform Calls to Share GL State direction.

  • NSOpenGLContext>>initWithFormat:shareContext:

  • BOOL WINAPI wglShareLists( HGLRC hglrc1, HGLRC hglrc2 );

  • GLXContextglXCreateContext(Display *dpy,

    XVisualInfo *vis,

    GLXContextshareList,

    Booldirect);


Native calls to share gl state
Native Calls to Share GL direction. State

  • EGLContexteglCreateContext(EGLDisplay display,

    EGLConfigconfig,

    EGLContextshare_context,

    EGLintconst*attrib_list)


Integrating opengl with javafx

To fill direction. a shape with an image.

Use existing picture box, DO NOT delete and create new picture box.

Right click on the shape.

At the bottom of the submenu select “Format Shape”

Select “Fill” at the top of the “Format Shape” dialog box.

Select “Picture or Texture fill” from the options.

And select “File” under the “Insert from” option.

Navigate to the file you want to use and select “Insert”

On the “Format” tab, in the Size group, click on “Crop to Fill” in the Crop tool and drag the image bounding box to the desired size

DELETE THIS INSTRUCTION NOTE WHEN NOT IN USE

Integrating OpenGL with JavaFX


Java opengl libraries
Java OpenGL Libraries direction.

  • JOGL (Java Binding for the OpenGL API)

    • https://jogamp.org/jogl/www/

  • JLWGL (Light Weight Java Game Library)

    • http://www.lwjgl.org/

  • Eclipse Platform Generator

    • http://www.eclipse.org/swt/macgen.php

    • Not an official Eclipse component but useful


Problem draw opengl content in javafx
Problem: Draw OpenGL content in direction. JavaFX

  • Solution 1:

    • Draw OpenGL content to a byte buffer

    • Upload the byte buffer to an Image (UI-Thread)

  • Solution 2:

    • Draw OpenGL content as part of Prism

    • Provide an OpenGL object (texture) for Prism to draw

      • Why not just draw directly? (more on this later)


Solution 1 draw using a byte buffer
Solution 1: Draw using a Byte Buffer direction.

  • Advantages

    • Easy to understand and implement

    • Does not access internals of JavaFX and Prism

  • Disadvantages

    • Too slow when fps requirements are strict

    • Uses more memory than drawing as part of Prism


Use opengl calls to get the bytes
Use OpenGL calls to get the bytes direction.

glGetTexImage(GL_TEXTURE_2D, 0, GL_RGBA, GL_UNSIGNED_SHORT, buffer);

glReadPixels(0, 0, WIDTH, HEIGHT, GL_RGBA, GL_UNSIGNED_BYTE, buffer);


Use fx code to write the bytes
Use FX code to write the bytes direction.

PixelWriter pw = image.getPixelWriter();

PixelFormat<ByteBuffer> pf = PixelFormat.getByteBgraInstance();

pw.setPixels(0, 0, WIDTH, HEIGHT, pf, buffer, WIDTH*4);



Solution 2 draw as part of prism
Solution 2: Draw as part of Prism direction.

  • Advantages

    • Simple to understand/explain

    • Fastest possible solution (least overhead)

  • Disadvantages

    • Must run / coordinate with Render Thread

    • Uses Prism internals and relies on representation


Threading prism and opengl
Threading, Prism and OpenGL direction.

  • Application code runs in the UI-thread

  • Prism code runs in the Render-thread

  • OpenGL threading rules:

    • OpenGL is not thread safe (it is thread aware)

    • Multiple threads are possible (each in their own apartment)

    • Platform specific code needed to create/share GL contexts

      • We’ve seen these calls before


In what thread should your code draw
In what thread should your code draw? direction.

  • Draw in the Render-Thread

    • Must be careful not to conflict/hammer Prism state

  • Draw in another thread (even the UI-Thread)

    • Must create a GL context for that thread (apartment threading)

    • Must draw to a GL object (texture) for Render-Thread to draw

      • Cannot issue GL commands from “wrong thread”

      • Must synchronize around access to GL object (texture)


Drawing in the render thread
Drawing in the Render Thread direction.

  • Risky because Prism drawing model might change

  • Must save/restore any GL state that you set

    • Slow to save and restore state one at a time

    • Doesn’t sound that hard but easy to get wrong

  • How to integrate with Prism effects? Transforms?

    • Transforms are done in software (might change)

    • You won’t draw in the right place or the right way


Opengl integration the prototype
OpenGL Integration: The Prototype direction.

  • Draw in the Render Thread

    • Create your own GL context (platform specific)

    • Save/Restore Prisms GL context

  • Draw to a GL object (Texture)

    • Allow Prism to create the texture (but you draw to it)

    • Prism then draws the texture (translations, effects work correctly)

    • Dispose the texture using Prism when no longer needed


Drawing in the render thread1
Drawing in the Render Thread direction.

  • Take part in the UI-Thread/Render-Thread Sync

  • Subclass an FX Class

    • Prototype uses Pane

    • Reimplementimpl_createPeer() to return your NGNode

  • Subclass an NGNode (your NGNode)

    • Prototype subclasses NGRegion

    • ReimplementNGNode.renderContent(Graphics g)


Drawing in the render thread2
Drawing in the Render Thread direction.

NOT API !

  • Take part in the UI-Thread/Render-Thread Sync

  • Subclass an FX Class

    • Prototype uses Pane

    • Reimplementimpl_createPeer() to return your NGNode

  • Subclass an NGNode (your NGNode)

    • Prototype subclasses NGRegion

    • ReimplementNGNode.renderContent(Graphics g)


Override rendercontent
Override direction. renderContent()

@OverrideprotectedvoidrenderContent(Graphics g) {

// Save the Prism GL context, initialize the user GL context

Texture texture = getPrismTexture();

NSOpenGLContextprismContext = NSOpenGLContext.currentContext();

NSOpenGLContextuserContext = getUserContext(prismContext, texture);

// Set the user GL Context

userContext.makeCurrentContext();

// Execute GL commands in the user GL context, and flush (or nothing draws)

TorusLWJGL.resizeTorus(WIDTH, HEIGHT);

TorusLWJGL.drawTorus();

glFlush();

// Restore the Prism GL Context, draw the texture

prismContext.makeCurrentContext();

g.drawTexture(texture, 0, 0, WIDTH, HEIGHT, 0, 0, WIDTH, HEIGHT);

}


Implement getprismtexture
Implement direction. getPrismTexture()

// Cache a single texture (released in setPrismTexture())

Texture prismTexture;

Texture getPrismTexture() {

if(prismTexture == null) {

ResourceFactoryf = GraphicsPipeline.getDefaultResourceFactory();

prismTexture= f.createTexture(PixelFormat.INT_ARGB_PRE,

Texture.Usage.DEFAULT,

Texture.WrapMode.CLAMP_NOT_NEEDED,

WIDTH, HEIGHT);

prismTexture.makePermanent();

}

returnprismTexture;

}


Implement getusercontext
Implement direction. getUserContext()

NSOpenGLContextgetUserContext(NSOpenGLContextprismContext, Texture prismTexture) {

if(userContext == null) {

userContext= (NSOpenGLContext) newNSOpenGLContext().alloc();

NSOpenGLPixelFormatpixelFormat = (NSOpenGLPixelFormat)newNSOpenGLPixelFormat().alloc();

pixelFormat.initWithAttributes(newint [] {0, });

userContext= userContext.initWithFormat(pixelFormat, prismContext);

userContext.makeCurrentContext();

try {GLContext.useContext(userContext, false);} catch (LWJGLException e) {e.printStackTrace();}

intuserFBO = glGenFramebuffersEXT();

glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, userFBO);

intprismTextureID = com.sun.prism.es2.SetID.getID(prismTexture);

glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT,

GL_COLOR_ATTACHMENT0_EXT, GL_TEXTURE_2D, prismTextureID, 0);

}

returnuserContext;

}


Important follow threading rules
Important: Follow Threading Rules direction.

  • Application code runs in the UI-Thread

  • renderContent() is called in the Render-Thread

  • Do not access UI-Thread state

    • Copy state from UI-Thread FX object to Render-Thread NGNode

    • It is easy to forget to do this but this leads to instability / crashes



The way forward
The Way Forward direction.

  • Implement GLNode for JavaFX

    • Provide minimal API (GL context, Prism texture)

    • Enough API to implement simple GL integration

  • Implement JOGL and LWJGL binding for JavaFX

    • Provide appropriate drawing mechanism for the library

    • Example: JOGL Autodrawable

NOTE: Only ES2 platforms supported (non-Windows)


The diamondbenchmark

To fill direction. a shape with an image.

Use existing picture box, DO NOT delete and create new picture box.

Right click on the shape.

At the bottom of the submenu select “Format Shape”

Select “Fill” at the top of the “Format Shape” dialog box.

Select “Picture or Texture fill” from the options.

And select “File” under the “Insert from” option.

Navigate to the file you want to use and select “Insert”

On the “Format” tab, in the Size group, click on “Crop to Fill” in the Crop tool and drag the image bounding box to the desired size

DELETE THIS INSTRUCTION NOTE WHEN NOT IN USE

The DiamondBenchmark


The diamond benchmark
The Diamond Benchmark direction.

  • Strict requirements:

    • Must handle thousands of small objects (“diamonds”)

    • Objects are constantly moved and updated

    • Multi-monitor, large area needs to be drawn

    • Performance critical (must render at 60 fps)

  • Examples:

    • Sea urchins on the ocean floor, bees swarming around a hive, targets being tracked by missile defense system


First version use nodes
First Version: Use Nodes direction.

  • Node for each diamond

    • Use standard FX coding conventions

      • Subclass of Region

      • Use properties for variables

  • Result:

    • Too slow, animation is not smooth … why?

    • Too many nodes, too much data, too many updates


Example diamondbenchmarkfx
Example: direction. DiamondBenchmarkFX


Second version use canvas
Second Version: Use Canvas direction.

  • Lightweight object for each diamond

    • Use standard FX coding conventions

      • Subclass of Object (instead of Node)

      • Use properties for variables

  • Result:

    • Too slow (slower than using Nodes) … why?

    • Canvas uses immediate mode drawing (no nodes)

    • Prism caches node drawing (can’t cache immediate mode)


Example diamondbenchmarkcanvas
Example: direction. DiamondBenchmarkCanvas


Third version use custom opengl
Third Version: Use Custom OpenGL direction.

  • Lightweight object for each diamond

    • Use optimal Java coding conventions

    • Use optimal OpenGL conventions / data structures

    • Use LWJGL for convenience (recode in C and compare)

  • Result:

    • Meets requirements, animation is smooth ... why?

    • Uses an optimal OpenGL drawing strategy

    • Uses platform specific OpenGL extensions


Example diamondbenchmarklwjgl
Example: direction. DiamondBenchmarkLWJGL


How prism draws the diamonds
How Prism draws the Diamonds direction.

  • A Diamond is a simple path (5 vertices)

    • There is a path for each diamond node

    • The diamond is transformed on the screen

  • Prism renders each path separately

  • Each path requires:

    • Vertex coordinates, texture coordinates, color data etc.

    • Rendering gets slower as the number of paths increase


A faster way to draw the diamonds
A Faster way to draw the Diamonds direction.

  • Use OpenGL to do instanced drawing

    • Put 5 path vertices on the graphics card and leave them

    • Use glDrawArraysInstanced() from OpenGL 3.1 to draw

  • Transfer the minimal amount of data (just x and y)

    • Use a vertex shader to apply the transform


Prism versus custom opengl
Prism versus Custom OpenGL direction.

“By knowing the content being rendered, an OpenGL developer can use techniques that Prism, as a general purpose graphics engine, does have the context to use.”

“At the price of portability, an OpenGL program can take advantage of extensions and platform specific mechanisms that are not available to Prism.”


Summary
Summary direction.

  • Need an API like JAWT for Glass

    • Reflection and private header files don’t cut it

  • Need first class support for OpenGL in JavaFX

    • Drawing to a texture looks promising

    • Low level hacks / undocumented calls don’t cut it

  • OpenGL can be used to improve performance

    • Use of platform / GL version specific features

    • Lots of drawing / updates (you won’t beat Prism otherwise)


Questions
Questions? direction.


Main title goes here
Main Title Goes direction. Here

Insert Presenter’s Name HereInsert Presenter’s Title Here


Program agenda1
Program Agenda direction.

  • Topic 1, Arial, 24 pt

  • Topic 2, Arial, 24 pt

  • Topic 3, Arial, 24 pt

  • Topic 4, Arial, 24 pt

  • Topic 5. More than 5 topics, add second agenda slide.


Slide title arial 28 pt two line max
Slide Title: Arial, 28 direction. PTTwo-Line Max

Subtitle: Arial, 20 pt, One-Line Max

  • Bullets are sentence case. Use Arial, 20 pt font.

    • Sub-bullets are Arial, 18 pt font.

  • Keep bullets short.

  • One idea per bullet.

  • No more than five bullets.

  • NOTE: Arial is the ONLY font that should be used in the Oracle corporate presentation template.Times and other serif fonts are not acceptable.

To ensure that slides are properly formatted to this template, see pages 7 and 8 for instructions.

Times

Arial


Slide title arial 28 pt one line
Slide Title: Arial, 28 direction. PT, One-Line

Subtitle: Arial, 20 pt, One-Line Max

  • Bullets are sentence case. Use Arial, 20 pt font.

    • Sub-bullets are Arial, 18 pt font.

  • Keep bullets short.

  • One idea per bullet.

  • No more than five bullets.

  • NOTE: Arial is the ONLY font that should be used in the Oracle corporate presentation template.Times and other serif fonts are not acceptable.

To ensure that slides are properly formatted to this template, see pages 7 and 8 for instructions.


Graphic section divider
Graphic direction. Section Divider


Safe harbor statements
Safe Harbor Statements direction.

  • One of the following slides must be used if your presentation covers material affected by Oracle’s Revenue Recognition Policy

  • To learn more about this policy, e-mail: [email protected]


The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract.It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Image section divider

To fill direction. a shape with an image.

Use existing picture box, DO NOT delete and create new picture box.

Right click on the shape.

At the bottom of the submenu select “Format Shape”

Select “Fill” at the top of the “Format Shape” dialog box.

Select “Picture or Texture fill” from the options.

And select “File” under the “Insert from” option.

Navigate to the file you want to use and select “Insert”

On the “Format” tab, in the Size group, click on “Crop to Fill” in the Crop tool and drag the image bounding box to the desired size

DELETE THIS INSTRUCTION NOTE WHEN NOT IN USE

Image Section Divider


ORACLE direction.

PRODUCT

LOGO

Announcement ALL CAPS, ARIAL 44 PT, 4-LINE MAX

DELETE IF NOT IN USE


Title arial 28 pt two line max

ORACLE direction.

PRODUCT

LOGO

Title, Arial, 28 PT, Two-Line Max

BODY/STATEMENT COPY, ALL CAPS, 24 PT, HIGHLIGHT TEXT IN BLUE, USED FOR EMPHASIS

DELETE IF NOT IN USE


Case study example with photograph
Case Study Example with Photograph direction.

  • Solutions

  • Loremipsum dolor sit amet, consecteturadipiscingelit. Fusce a sagittisorci.

  • Maecenas vehiculalorempharetraipsumsuscipitutvenenatisrisusfacilisis. Donecvelaugue vitae urnaaliquetcursus. Aliquamrutrumiaculis.


Case study example with screenshot
Case Study Example direction. with Screenshot

  • Loremipsum dolor sit amet, consecteturadipiscingelit. Fusce a sagittisorci.

  • Maecenas vehiculalorempharetraipsumsuscipitutvenenatisrisusfacilisis. Donecvelaugue vitae urnaaliquetcursus. Aliquamrutrumiaculis.

  • Solutions


Case study example with graphic
Case Study Example with Graphic direction.

  • Loremipsum dolor sit amet, consecteturadipiscingelit. Fusce a sagittisorci.

  • Maecenas vehiculalorempharetraipsumsuscipitutvenenatisrisusfacilisis. Donecvelaugue vitae urnaaliquetcursus. Aliquamrutrumiaculis.

  • Solutions


Case study example with partner logos
Case Study Example with Partner Logos direction.

  • Loremipsum dolor sit amet, consecteturadipiscingelit. Fusce a sagittisorci.

  • Maecenas vehiculalorempharetraipsumsuscipitutvenenatisrisusfacilisis. Donecvelaugue vitae urnaaliquetcursus. Aliquamrutrumiaculis.

  • Solutions


THIRD PARTY COMPANY LOGO direction.

Insert Author Name Here

Position Title, Company Name

“This slide format serves to call attention to a quote from a prominent customer, executive, or thought leader regarding a particular topic.”

DELETE IF NOT IN USE


Classic Duke direction.


Vertical bar chart
Vertical Bar Chart direction.

Contextual information about the chart

  • Bullet copy

  • Bullet copy

  • Bullet copy


Horizontal bar chart
Horizontal Bar Chart direction.

Contextual information about the chart

  • Bullet copy

  • Bullet copy

  • Bullet copy


Stacked chart
Stacked Chart direction.

Contextual information about the chart

  • Bullet copy

  • Bullet copy

  • Bullet copy


Line chart
Line Chart direction.

Contextual information about the chart

  • Bullet copy

  • Bullet copy

  • Bullet copy


Pie chart
Pie Chart direction.

Contextual information about the chart

  • Bullet copy

  • Bullet copy

  • Bullet copy


Tables
Tables direction.


Tables1
Tables direction.


Tables2
Tables direction.


Tables3
Tables direction.


Tables4
Tables direction.


Tables5
Tables direction.


Tables6
Tables direction.


ad