Integrating javafx with native technologies
This presentation is the property of its rightful owner.
Sponsored Links
1 / 84

Integrating JavaFX with Native Technologies PowerPoint PPT Presentation


  • 509 Views
  • Uploaded on
  • Presentation posted in: General

Integrating JavaFX with Native Technologies. Stephen Northover (Oracle) Felipe Heidrich (Oracle).

Download Presentation

Integrating JavaFX with Native Technologies

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Integrating javafx with native technologies

Integrating JavaFX with Native Technologies

Stephen Northover (Oracle)Felipe Heidrich (Oracle)


Integrating javafx with native technologies

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Program agenda

Program Agenda

  • FX Architecture: Overview

  • Native Code in Glass and Prism

  • Integrating OpenGL with JavaFX

  • Performance: The DiamondBenchmark


Why might you need natives

Why might you need Natives?

  • Access operating system specific operations

    • Use Win32, Cocoa and GTK API

  • Integrate pre-existing native code

  • Improve performance

    • Native code can be faster (tight loops etc.)

    • Access faster platform specific operations

      • OpenGL 3.x or greater, DirectX 11 or greater


Javafx architecture glass and prism

JavaFX Architecture: Glass and Prism


Javafx architecture threading

JavaFX Architecture: Threading

start(Stage)

UI-Thread

(Glass, FX)

Sync

User Code

User Code

User Code

Sync

Render-Thread

(Prism)

Render Code

Render Code


Javafx architecture synchronizing

JavaFX Architecture: Synchronizing

Render-Thread

(Prism)

UI-Thread

(Glass, FX)

Sync


Glass architecture

Glass Architecture

  • Glass contains shared Java code (ie. View)

  • Glass contains platform specific Java code (ie. WinView)

  • Glass contains platform specific native peers (GlassView.cpp)

    • Native peers written in C/C++/Objective-C

    • Native peers contain native handles in their data structures

    • No API to get the native handles from Glass (*)

(*) An official API might be similar in concept to JAWT


What are the native handles

What are the Native handles?

  • Windows objects (C++, COM objects)

    • HWND, HHOOK, IUnknown …

  • Mac and iOS objects (Objective-C objects)

    • NSApplication, NSWindow, NSView …

  • GTK and Lens objects (C objects)

    • GtkWidget *, GdkPixbuf *, ..


Where are the native handles

Where are the Native Handles?

  • Glass objects and native handles are not 1-to-1

  • View lightweight on Windows but not on Mac

    • No HWND on Windows, NSView on Mac … sometimes

    • In a Browser, on Mac, no NSView, only a CALayer

  • Need to write different code for different platforms

    • Glass uses different implementation languages

    • Glass uses different native objects to wrap handles


Problem get the handle of a stage

Problem: Get the Handle of a Stage

  • Get the Glass object from the FX object

    • Quantum exists between Glass and FX

    • No API: Glass/Quantum objects are not public

  • Get the Native object from the Glass object

    • No API: Native code structure is subject to change

    • No API: Must include Glass header files / fragments


Use reflection java or native code

Use Reflection: Java or Native Code

staticlonggetHandle (Stage stage) {

Object ws = stage.impl_getPeer();

try {

Method m1 = ws.getClass().getDeclaredMethod("getPlatformWindow");

m1.setAccessible(true);

Object w = m1.invoke(ws);

Method m2 = w.getClass().getSuperclass().getDeclaredMethod("getNativeWindow");

return(long) m2.invoke(w);

} catch (Throwableth) {}

return0;

}


Use reflection java or native code1

Use Reflection: Java or Native Code

NOT API !

staticlonggetHandle (Stage stage) {

Object ws = stage.impl_getPeer();

try {

Method m1 = ws.getClass().getDeclaredMethod("getPlatformWindow");

m1.setAccessible(true);

Object w = m1.invoke(ws);

Method m2 = w.getClass().getSuperclass().getDeclaredMethod("getNativeWindow");

return(long) m2.invoke(w);

} catch (Throwableth) {}

return0;

}


Example open a native dialog on a mac

Example: Open a Native Dialog on a Mac


Prism architecture

Prism Architecture

  • Prism contains shared Java code (ie. BaseShaderContext)

  • Prism contains platform specific Java code (ie. ES2Conext)

  • Prism contains shared native code (ie. GLContext.c)

  • Prism contains platform specific native code (ie. MacGLContext.c)

    • Native code is written in C and C++ (a little Objective-C)

    • Native code contains native handles in their data structures

    • No API to get the native handles (same as Glass)


Prism architecture graphics pipelines

Prism Architecture: Graphics Pipelines

Prism

Software

D3D

ES2

Java2D

Mac

Linux

FB

iOS


Where are the opengl prism handles

Where are the OpenGL Prism handles?

  • OpenGL is a state machine

    • GL state is in a hidden platform specific “GL context”

    • Example: glScissor(x, y, width, height) // no GL context param

  • State can be queried when running in the Render-Thread

    • Query state using OpenGL calls and/or platform specific calls

  • State must be initialized when running in another thread

    • Share state (textures) with Prism using platform specific calls


Platform calls to share gl state

Platform Calls to Share GL State

  • NSOpenGLContext>>initWithFormat:shareContext:

  • BOOL WINAPI wglShareLists( HGLRC hglrc1, HGLRC hglrc2 );

  • GLXContextglXCreateContext(Display *dpy,

    XVisualInfo *vis,

    GLXContextshareList,

    Booldirect);


Native calls to share gl state

Native Calls to Share GL State

  • EGLContexteglCreateContext(EGLDisplay display,

    EGLConfigconfig,

    EGLContextshare_context,

    EGLintconst*attrib_list)


Integrating opengl with javafx

To fill a shape with an image.

Use existing picture box, DO NOT delete and create new picture box.

Right click on the shape.

At the bottom of the submenu select “Format Shape”

Select “Fill” at the top of the “Format Shape” dialog box.

Select “Picture or Texture fill” from the options.

And select “File” under the “Insert from” option.

Navigate to the file you want to use and select “Insert”

On the “Format” tab, in the Size group, click on “Crop to Fill” in the Crop tool and drag the image bounding box to the desired size

DELETE THIS INSTRUCTION NOTE WHEN NOT IN USE

Integrating OpenGL with JavaFX


Java opengl libraries

Java OpenGL Libraries

  • JOGL (Java Binding for the OpenGL API)

    • https://jogamp.org/jogl/www/

  • JLWGL (Light Weight Java Game Library)

    • http://www.lwjgl.org/

  • Eclipse Platform Generator

    • http://www.eclipse.org/swt/macgen.php

    • Not an official Eclipse component but useful


Problem draw opengl content in javafx

Problem: Draw OpenGL content in JavaFX

  • Solution 1:

    • Draw OpenGL content to a byte buffer

    • Upload the byte buffer to an Image (UI-Thread)

  • Solution 2:

    • Draw OpenGL content as part of Prism

    • Provide an OpenGL object (texture) for Prism to draw

      • Why not just draw directly? (more on this later)


Solution 1 draw using a byte buffer

Solution 1: Draw using a Byte Buffer

  • Advantages

    • Easy to understand and implement

    • Does not access internals of JavaFX and Prism

  • Disadvantages

    • Too slow when fps requirements are strict

    • Uses more memory than drawing as part of Prism


Use opengl calls to get the bytes

Use OpenGL calls to get the bytes

glGetTexImage(GL_TEXTURE_2D, 0, GL_RGBA, GL_UNSIGNED_SHORT, buffer);

glReadPixels(0, 0, WIDTH, HEIGHT, GL_RGBA, GL_UNSIGNED_BYTE, buffer);


Use fx code to write the bytes

Use FX code to write the bytes

PixelWriter pw = image.getPixelWriter();

PixelFormat<ByteBuffer> pf = PixelFormat.getByteBgraInstance();

pw.setPixels(0, 0, WIDTH, HEIGHT, pf, buffer, WIDTH*4);


Demo draw a torus using a byte buffer

Demo: Draw a Torus using a Byte Buffer


Solution 2 draw as part of prism

Solution 2: Draw as part of Prism

  • Advantages

    • Simple to understand/explain

    • Fastest possible solution (least overhead)

  • Disadvantages

    • Must run / coordinate with Render Thread

    • Uses Prism internals and relies on representation


Threading prism and opengl

Threading, Prism and OpenGL

  • Application code runs in the UI-thread

  • Prism code runs in the Render-thread

  • OpenGL threading rules:

    • OpenGL is not thread safe (it is thread aware)

    • Multiple threads are possible (each in their own apartment)

    • Platform specific code needed to create/share GL contexts

      • We’ve seen these calls before


In what thread should your code draw

In what thread should your code draw?

  • Draw in the Render-Thread

    • Must be careful not to conflict/hammer Prism state

  • Draw in another thread (even the UI-Thread)

    • Must create a GL context for that thread (apartment threading)

    • Must draw to a GL object (texture) for Render-Thread to draw

      • Cannot issue GL commands from “wrong thread”

      • Must synchronize around access to GL object (texture)


Drawing in the render thread

Drawing in the Render Thread

  • Risky because Prism drawing model might change

  • Must save/restore any GL state that you set

    • Slow to save and restore state one at a time

    • Doesn’t sound that hard but easy to get wrong

  • How to integrate with Prism effects? Transforms?

    • Transforms are done in software (might change)

    • You won’t draw in the right place or the right way


Opengl integration the prototype

OpenGL Integration: The Prototype

  • Draw in the Render Thread

    • Create your own GL context (platform specific)

    • Save/Restore Prisms GL context

  • Draw to a GL object (Texture)

    • Allow Prism to create the texture (but you draw to it)

    • Prism then draws the texture (translations, effects work correctly)

    • Dispose the texture using Prism when no longer needed


Drawing in the render thread1

Drawing in the Render Thread

  • Take part in the UI-Thread/Render-Thread Sync

  • Subclass an FX Class

    • Prototype uses Pane

    • Reimplementimpl_createPeer() to return your NGNode

  • Subclass an NGNode (your NGNode)

    • Prototype subclasses NGRegion

    • ReimplementNGNode.renderContent(Graphics g)


Drawing in the render thread2

Drawing in the Render Thread

NOT API !

  • Take part in the UI-Thread/Render-Thread Sync

  • Subclass an FX Class

    • Prototype uses Pane

    • Reimplementimpl_createPeer() to return your NGNode

  • Subclass an NGNode (your NGNode)

    • Prototype subclasses NGRegion

    • ReimplementNGNode.renderContent(Graphics g)


Override rendercontent

Override renderContent()

@OverrideprotectedvoidrenderContent(Graphics g) {

// Save the Prism GL context, initialize the user GL context

Texture texture = getPrismTexture();

NSOpenGLContextprismContext = NSOpenGLContext.currentContext();

NSOpenGLContextuserContext = getUserContext(prismContext, texture);

// Set the user GL Context

userContext.makeCurrentContext();

// Execute GL commands in the user GL context, and flush (or nothing draws)

TorusLWJGL.resizeTorus(WIDTH, HEIGHT);

TorusLWJGL.drawTorus();

glFlush();

// Restore the Prism GL Context, draw the texture

prismContext.makeCurrentContext();

g.drawTexture(texture, 0, 0, WIDTH, HEIGHT, 0, 0, WIDTH, HEIGHT);

}


Implement getprismtexture

Implement getPrismTexture()

// Cache a single texture (released in setPrismTexture())

Texture prismTexture;

Texture getPrismTexture() {

if(prismTexture == null) {

ResourceFactoryf = GraphicsPipeline.getDefaultResourceFactory();

prismTexture= f.createTexture(PixelFormat.INT_ARGB_PRE,

Texture.Usage.DEFAULT,

Texture.WrapMode.CLAMP_NOT_NEEDED,

WIDTH, HEIGHT);

prismTexture.makePermanent();

}

returnprismTexture;

}


Implement getusercontext

Implement getUserContext()

NSOpenGLContextgetUserContext(NSOpenGLContextprismContext, Texture prismTexture) {

if(userContext == null) {

userContext= (NSOpenGLContext) newNSOpenGLContext().alloc();

NSOpenGLPixelFormatpixelFormat = (NSOpenGLPixelFormat)newNSOpenGLPixelFormat().alloc();

pixelFormat.initWithAttributes(newint [] {0, });

userContext= userContext.initWithFormat(pixelFormat, prismContext);

userContext.makeCurrentContext();

try {GLContext.useContext(userContext, false);} catch (LWJGLException e) {e.printStackTrace();}

intuserFBO = glGenFramebuffersEXT();

glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, userFBO);

intprismTextureID = com.sun.prism.es2.SetID.getID(prismTexture);

glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT,

GL_COLOR_ATTACHMENT0_EXT, GL_TEXTURE_2D, prismTextureID, 0);

}

returnuserContext;

}


Important follow threading rules

Important: Follow Threading Rules

  • Application code runs in the UI-Thread

  • renderContent() is called in the Render-Thread

  • Do not access UI-Thread state

    • Copy state from UI-Thread FX object to Render-Thread NGNode

    • It is easy to forget to do this but this leads to instability / crashes


Demo draw torus inside prism

Demo: Draw Torus inside Prism


The way forward

The Way Forward

  • Implement GLNode for JavaFX

    • Provide minimal API (GL context, Prism texture)

    • Enough API to implement simple GL integration

  • Implement JOGL and LWJGL binding for JavaFX

    • Provide appropriate drawing mechanism for the library

    • Example: JOGL Autodrawable

NOTE: Only ES2 platforms supported (non-Windows)


The diamondbenchmark

To fill a shape with an image.

Use existing picture box, DO NOT delete and create new picture box.

Right click on the shape.

At the bottom of the submenu select “Format Shape”

Select “Fill” at the top of the “Format Shape” dialog box.

Select “Picture or Texture fill” from the options.

And select “File” under the “Insert from” option.

Navigate to the file you want to use and select “Insert”

On the “Format” tab, in the Size group, click on “Crop to Fill” in the Crop tool and drag the image bounding box to the desired size

DELETE THIS INSTRUCTION NOTE WHEN NOT IN USE

The DiamondBenchmark


The diamond benchmark

The Diamond Benchmark

  • Strict requirements:

    • Must handle thousands of small objects (“diamonds”)

    • Objects are constantly moved and updated

    • Multi-monitor, large area needs to be drawn

    • Performance critical (must render at 60 fps)

  • Examples:

    • Sea urchins on the ocean floor, bees swarming around a hive, targets being tracked by missile defense system


First version use nodes

First Version: Use Nodes

  • Node for each diamond

    • Use standard FX coding conventions

      • Subclass of Region

      • Use properties for variables

  • Result:

    • Too slow, animation is not smooth … why?

    • Too many nodes, too much data, too many updates


Example diamondbenchmarkfx

Example: DiamondBenchmarkFX


Second version use canvas

Second Version: Use Canvas

  • Lightweight object for each diamond

    • Use standard FX coding conventions

      • Subclass of Object (instead of Node)

      • Use properties for variables

  • Result:

    • Too slow (slower than using Nodes) … why?

    • Canvas uses immediate mode drawing (no nodes)

    • Prism caches node drawing (can’t cache immediate mode)


Example diamondbenchmarkcanvas

Example: DiamondBenchmarkCanvas


Third version use custom opengl

Third Version: Use Custom OpenGL

  • Lightweight object for each diamond

    • Use optimal Java coding conventions

    • Use optimal OpenGL conventions / data structures

    • Use LWJGL for convenience (recode in C and compare)

  • Result:

    • Meets requirements, animation is smooth ... why?

    • Uses an optimal OpenGL drawing strategy

    • Uses platform specific OpenGL extensions


Example diamondbenchmarklwjgl

Example: DiamondBenchmarkLWJGL


How prism draws the diamonds

How Prism draws the Diamonds

  • A Diamond is a simple path (5 vertices)

    • There is a path for each diamond node

    • The diamond is transformed on the screen

  • Prism renders each path separately

  • Each path requires:

    • Vertex coordinates, texture coordinates, color data etc.

    • Rendering gets slower as the number of paths increase


A faster way to draw the diamonds

A Faster way to draw the Diamonds

  • Use OpenGL to do instanced drawing

    • Put 5 path vertices on the graphics card and leave them

    • Use glDrawArraysInstanced() from OpenGL 3.1 to draw

  • Transfer the minimal amount of data (just x and y)

    • Use a vertex shader to apply the transform


Prism versus custom opengl

Prism versus Custom OpenGL

“By knowing the content being rendered, an OpenGL developer can use techniques that Prism, as a general purpose graphics engine, does have the context to use.”

“At the price of portability, an OpenGL program can take advantage of extensions and platform specific mechanisms that are not available to Prism.”


Summary

Summary

  • Need an API like JAWT for Glass

    • Reflection and private header files don’t cut it

  • Need first class support for OpenGL in JavaFX

    • Drawing to a texture looks promising

    • Low level hacks / undocumented calls don’t cut it

  • OpenGL can be used to improve performance

    • Use of platform / GL version specific features

    • Lots of drawing / updates (you won’t beat Prism otherwise)


Questions

Questions?


Main title goes here

Main Title Goes Here

Insert Presenter’s Name HereInsert Presenter’s Title Here


Program agenda1

Program Agenda

  • Topic 1, Arial, 24 pt

  • Topic 2, Arial, 24 pt

  • Topic 3, Arial, 24 pt

  • Topic 4, Arial, 24 pt

  • Topic 5. More than 5 topics, add second agenda slide.


Slide title arial 28 pt two line max

Slide Title: Arial, 28 PTTwo-Line Max

Subtitle: Arial, 20 pt, One-Line Max

  • Bullets are sentence case. Use Arial, 20 pt font.

    • Sub-bullets are Arial, 18 pt font.

  • Keep bullets short.

  • One idea per bullet.

  • No more than five bullets.

  • NOTE: Arial is the ONLY font that should be used in the Oracle corporate presentation template.Times and other serif fonts are not acceptable.

To ensure that slides are properly formatted to this template, see pages 7 and 8 for instructions.

Times

Arial


Slide title arial 28 pt one line

Slide Title: Arial, 28 PT, One-Line

Subtitle: Arial, 20 pt, One-Line Max

  • Bullets are sentence case. Use Arial, 20 pt font.

    • Sub-bullets are Arial, 18 pt font.

  • Keep bullets short.

  • One idea per bullet.

  • No more than five bullets.

  • NOTE: Arial is the ONLY font that should be used in the Oracle corporate presentation template.Times and other serif fonts are not acceptable.

To ensure that slides are properly formatted to this template, see pages 7 and 8 for instructions.


Graphic section divider

Graphic Section Divider


Safe harbor statements

Safe Harbor Statements

  • One of the following slides must be used if your presentation covers material affected by Oracle’s Revenue Recognition Policy

  • To learn more about this policy, e-mail: [email protected]


Integrating javafx with native technologies

The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract.It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Integrating javafx with native technologies

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Image section divider

To fill a shape with an image.

Use existing picture box, DO NOT delete and create new picture box.

Right click on the shape.

At the bottom of the submenu select “Format Shape”

Select “Fill” at the top of the “Format Shape” dialog box.

Select “Picture or Texture fill” from the options.

And select “File” under the “Insert from” option.

Navigate to the file you want to use and select “Insert”

On the “Format” tab, in the Size group, click on “Crop to Fill” in the Crop tool and drag the image bounding box to the desired size

DELETE THIS INSTRUCTION NOTE WHEN NOT IN USE

Image Section Divider


Integrating javafx with native technologies

ORACLE

PRODUCT

LOGO

Announcement ALL CAPS, ARIAL 44 PT, 4-LINE MAX

DELETE IF NOT IN USE


Title arial 28 pt two line max

ORACLE

PRODUCT

LOGO

Title, Arial, 28 PT, Two-Line Max

BODY/STATEMENT COPY, ALL CAPS, 24 PT, HIGHLIGHT TEXT IN BLUE, USED FOR EMPHASIS

DELETE IF NOT IN USE


Case study example with photograph

Case Study Example with Photograph

  • Solutions

  • Loremipsum dolor sit amet, consecteturadipiscingelit. Fusce a sagittisorci.

  • Maecenas vehiculalorempharetraipsumsuscipitutvenenatisrisusfacilisis. Donecvelaugue vitae urnaaliquetcursus. Aliquamrutrumiaculis.


Case study example with screenshot

Case Study Example with Screenshot

  • Loremipsum dolor sit amet, consecteturadipiscingelit. Fusce a sagittisorci.

  • Maecenas vehiculalorempharetraipsumsuscipitutvenenatisrisusfacilisis. Donecvelaugue vitae urnaaliquetcursus. Aliquamrutrumiaculis.

  • Solutions


Case study example with graphic

Case Study Example with Graphic

  • Loremipsum dolor sit amet, consecteturadipiscingelit. Fusce a sagittisorci.

  • Maecenas vehiculalorempharetraipsumsuscipitutvenenatisrisusfacilisis. Donecvelaugue vitae urnaaliquetcursus. Aliquamrutrumiaculis.

  • Solutions


Case study example with partner logos

Case Study Example with Partner Logos

  • Loremipsum dolor sit amet, consecteturadipiscingelit. Fusce a sagittisorci.

  • Maecenas vehiculalorempharetraipsumsuscipitutvenenatisrisusfacilisis. Donecvelaugue vitae urnaaliquetcursus. Aliquamrutrumiaculis.

  • Solutions


Integrating javafx with native technologies

THIRD PARTY COMPANY LOGO

Insert Author Name Here

Position Title, Company Name

“This slide format serves to call attention to a quote from a prominent customer, executive, or thought leader regarding a particular topic.”

DELETE IF NOT IN USE


Integrating javafx with native technologies

Classic Duke


Vertical bar chart

Vertical Bar Chart

Contextual information about the chart

  • Bullet copy

  • Bullet copy

  • Bullet copy


Horizontal bar chart

Horizontal Bar Chart

Contextual information about the chart

  • Bullet copy

  • Bullet copy

  • Bullet copy


Stacked chart

Stacked Chart

Contextual information about the chart

  • Bullet copy

  • Bullet copy

  • Bullet copy


Line chart

Line Chart

Contextual information about the chart

  • Bullet copy

  • Bullet copy

  • Bullet copy


Pie chart

Pie Chart

Contextual information about the chart

  • Bullet copy

  • Bullet copy

  • Bullet copy


Tables

Tables


Tables1

Tables


Tables2

Tables


Tables3

Tables


Tables4

Tables


Tables5

Tables


Tables6

Tables


  • Login