slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
The Closures Controversy PowerPoint Presentation
Download Presentation
The Closures Controversy

Loading in 2 Seconds...

play fullscreen
1 / 67

The Closures Controversy - PowerPoint PPT Presentation


  • 395 Views
  • Uploaded on

The Closures Controversy Joshua Bloch Chief Java Architect Google Inc. Disclaimer: Talk Represents My Opinion, Not Google’s!

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'The Closures Controversy' - bernad


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
the closures controversy

The Closures Controversy

Joshua Bloch

Chief Java Architect

Google Inc.

disclaimer talk represents my opinion not google s
Disclaimer: Talk Represents My Opinion, Not Google’s!

Google believes that the Java platform would likely benefit from some additional support for closures. Like the broader Java community, we are divided on what form this support should take. Some favor a lightweight approach that addresses the pain of anonymous inner classes without affecting the type system, VM, or libraries; others favor a heavyweight approach designed to provide for programmer-defined control structures. We believe it is premature to launch a JSR that forces us down either path.

www.javapolis.com

outline
Outline

I. Setting the Stage

II. Why Enhance Support for Closures?

III. BGGA Closures

IV. A Lightweight Approach

V. Where Do We Go From here?

www.javapolis.com

slide5

OOPSLA Invited Talk

October 8, 1996

Digitally reconstructed

from the Internet Archive

(AKA the Wayback Machine)

by Joshua Bloch

November 24, 2007

slide6
Oak
  • Started as a reimplementation of C++
  • Always a tool, never an end itself
  • Took on a life of its own
  • The Web happened...
    • serendipitous match!
  • and it became Java
the java language
The Java Language

Fusion of four kinds of programming

  • Object Oriented like Simula/C++/ ObjectiveC…
  • Numeric like FORTRAN
  • Systems like C
  • Distributed like nothing else
java a language for a job
Java - a language for a job
  • Doing language research:
    • an anti-goal
  • Started using C++
  • Broke down, needed:
    • Architecture neutral, portable, reliable, safe, long lived, multithreaded, dynamic, simple, ...
practical not theoretical
Practical, not theoretical
  • Driven by what people needed
    • (but hey, I spent too much time going to school!)
  • Theory provides
    • rigour
    • cleanliness
    • cohesiveness
no new ideas here
No new ideas here
  • Shamelessly ripped off ideas that worked in C, C++, Objective C, Cedar/Mesa, Modula, Simula, ...
  • (well, we slipped once or twice and invented something)
don t fix it until it chafes
Don’t fix it until it chafes
  • To keep it simple...
  • A procedural principle:
    • Require several real instances before including a feature
    • i.e. nothing goes in because it’s “nice”
  • (== “Just Say No, until threatened with bodily harm”)
java feels
Java feels...
  • Hyped :-(
  • Playful
  • Flexible
  • Deterministic
  • Non-threatening
  • Rich
  • Like I can just write code…

Hey! I left out “Object-Oriented”!

so how are we doing not so well unfortunately
So How Are We Doing? Not So Well, Unfortunately

Enum<E extends Enum<E>> { ... }

<T extends Object & Comparable<? super T>> T Collections.max(Collection<? extends T>)

public <V extends Wrapper<? extends Comparable<T>>>

Comparator<V> comparator() { ... }

error: equalTo(Box<capture of ?>) in Box<capture of ?> cannot be applied to (Box<capture of ?>)

equal = unknownBox.equalTo(unknownBox)

Arrays.asList(String.class, Integer.class) // Warning!

See Angelia Langer's 427-page (!) Java Generics FAQ for more:

http://www.angelikalanger.com/GenericsFAQ/JavaGenericsFAQ.pdf

www.javapolis.com

what the man on the web is saying
What the Man on the Web is Saying

“I am completely and totally humbled. Laid low. I realize now that I am simply not smart at all. I made the mistake of thinking that I could understand generics. I simply cannot. I just can't. This is really depressing. It is the first time that I've ever not been able to understand something related to computers, in any domain, anywhere, period.”

“I'm the lead architect here, have a PhD in physics, and have been working daily in Java for 10 years and know it pretty well. The other guy is a very senior enterprise developer (wrote an email system that sends 600 million emails/year with almost no maintenance). If we can't get [generics], it's highly unlikely that the ‘average’ developer will ever in our lifetimes be able to figure this stuff out.”

www.javapolis.com

where does the complexity come from
Where Does The Complexity Come From?

Feature Tuples

(exponential)

Feature Pairs

(quadratic)

Features

(linear)

www.javapolis.com

if the feel of java is to be preserved
If The Feel of Java is to be Preserved...
  • We simply cannot afford another wildcards
  • Further language additions must be undertaken with extreme caution
  • Minimal addition to conceptual surface area
  • High power-to-weight ratio

www.javapolis.com

outline18
Outline

I. Setting the Stage

II. Why Enhance Support for Closures?

II. BGGA Closures

III. A Lightweight Approach

IV. Where Do We Go From here?

www.javapolis.com

what is a closure
What is a Closure?
  • One definition: a function that is evaluated in an environment containing one or more bound variables [Wikipedia]
  • In English: a little snippet of code that can be passed around for subsequent execution
  • Limited support for closures since JDK 1.1, in the form of anonymous classes

www.javapolis.com

why are we considering better support for closures
Why Are We Considering Better Support for Closures?
  • Fine-Grained Concurrecy - Passing snippets of code to fork-join frameworks using anonymous classes is a pain
  • Resource Managememnt - Using try-finally blocks for resource management is a pain, and causes resource leaks

www.javapolis.com

1 fine grained fork join concurrency
1. Fine-grained (fork-join) concurrency

"It has to be easier to send snippets of code to frameworks for parallel execution; otherwise no one will use them .“

–Doug Lea, 2005

www.javapolis.com

here s how it looks today
Here’s How it Looks Today

class StudentStatistics {

ParallelArray<Student> students = ...

// ...

public double getMaxSeniorGpa() {

return students.withFilter(isSenior).

withMapping(gpaField).max();

}

// helpers:

static final class IsSenior implements Predicate<Student> {

public boolean evaluate(Student s) { return s.credits > 90; }

}

static final IsSenior isSenior = new IsSenior();

static final class GpaField implements MapperToDouble<Student> {

public double map(Student s) { return s.gpa; }

}

static final GpaField gpaField = new GpaField();

}

www.javapolis.com

2 automatic resource management
2. Automatic Resource Management

“The C++ destructor model is exactly the same as the Dispose pattern, except that it is far easier to use and a direct language feature and correct by default, instead of a coding pattern that is off by default and causing correctness or performance problems when it is forgotten.”

–Herb Sutter, OOPSLA 2004

www.javapolis.com

how it looks today manual resource management
How it Looks Today–Manual Resource Management

static String readFirstLineFromFile(String path)

throws IOException {

BufferedReader r = null;

String s;

try {

r = new BufferedReader(new FileReader(path));

s = r.readLine();

} finally {

if (r != null)

r.close();

}

return s;

}

www.javapolis.com

it s worse with multiple resources puzzler 41
It’s Worse With Multiple Resources (Puzzler 41)

static void copy(String src, String dest) throws IOException {

InputStream in = null;

OutputStream out = null;

try {

in = new FileInputStream(src);

out = new FileOutputStream(dest);

byte[] buf = new byte[1024];

int n;

while ((n = in.read(buf)) >= 0)

out.write(buf, 0, n);

} finally {

closeIgnoringException(in);

closeIgnoringException(out);

}

}

private static void closeIgnoringException(Closeable c) {

if (c != null) {

try { c.close(); } catch (IOException ex) { // ignore }

}

}

www.javapolis.com

automatic resource management
Automatic Resource Management

C++ Destructor

String^ ReadFirstLineFromFile( String^ path ) {

StreamReader r(path);

return r.ReadLine();

}

C# using Block

String ReadFirstLineFromFile( String path ) {

using ( StreamReader r = new StreamReader(path) ) {

return r.ReadLine();

}

}

www.javapolis.com

outline27
Outline

I. Setting the Stage

II. Why Enhance Support for Closures?

III. BGGA Closures

IV. A Lightweight Approach

V. Where Do We Go From here?

www.javapolis.com

controversial features in bgga
Controversial Features in BGGA
  • Function types
  • Non-local return
  • Non-local break and continue
  • Unrestricted access to nonfinal local variables
  • Design goal: library-defined control constructs

www.javapolis.com

function types
Function Types

The BGGA Spec says: “While the subtype rules for function types may at first glance appear arcane, they are defined this way for very good reason: [...]. And while the rules seem complex, function types do not add complexity to Java's type system because function types and their subtype relations can be understood as a straightforward application of generics and wildcards (existing constructs). From the programmer's perspective, function types just ‘do the right thing.’”

www.javapolis.com

function types are hard to read
Function Types are Hard to Read

static Pair<{ => int },{ => int }> joinedCounters(int initial) {

return Pair.<{ => int },{ => int }>of(

{ => initial++ }, { => initial++ });

}

interface BThunk extends {=>boolean} { }

static final {BThunk, { => void} => void} wihle =

{BThunk cond, { => void } action =>

while (cond.invoke()) action.invoke(); };

static <throws X> { {=> void throws X} => void throws X }foo() {

return { { => void throws X } block => block.invoke(); };

}

These examples come from test code that ships with BGGA Prototype

www.javapolis.com

function types encourage an exotic programming style
Function Types Encourage an “Exotic” Programming Style

static <A1, A2, R> {A1 => {A2 => R}} curry({A1, A2 => R} fn) {

return {A1 a1 => {A2 a2 => fn.invoke(a1, a2)}};

}

<A1, A2, A3, R> {A2, A3 => R} partial({A1, A2, A3 => R} fn, A1 a1);

static <A1, A2, A3, R> {A2, A3 => R}

partial({A1, A2, A3 => R} fn, A1 a1) {

return {A2 a2, A3 a3 => fn.invoke(a1, a2, a3)};

}

From Mark Mahieu's blog:

Currying and Partial Application with Java Closures

http://markmahieu.blogspot.com/2007/12/currying-and-partial-application-with.html

www.javapolis.com

nominal types are rich compared to function types
Nominal Types are RichCompared to Function Types

Name →

Known Implementations →

Documentation, including semantic constraints

{T, T => T}

www.javapolis.com

function types have unexpected interactions
Function Types Have Unexpected Interactions
  • Arrays don’t work

Foo.java:6: generic array creation

{ => int}[] closures = new { => int}[N];

  • Autoboxing doesn’t work

LoopBenchC.java:10: <E,X>forEach(java.util.Collection<E>, {E => void throws X}) in LoopBenchC cannot be applied to (java.util.List<java.lang.Integer>,{int => void})

forEach(list, {int i =>

  • Wildcards produce difficult error messages

NewtonWithClosures.java:26: invoke(capture#418 of ? super {double => double}) in {capture#418 of ? super {double => double} => capture#928 of ? extends {double => double}} cannot be applied to (double)

return fixedPoint(transform.invoke(guess));

^

www.javapolis.com

function types limit interoperability with sam types
Function Types Limit Interoperability With SAM Types
  • Closure conversion only works with interfaces
  • Unfortunately, existing APIs sometimes use abstract classes for functions
    • e.g., TimerTask, SwingWorker
  • These APIs would become 2nd class citizens

www.javapolis.com

summary pros and cons of function types
Summary - Pros and Cons of Function Types

+ Avoid need to define named interfaces

+ Avoid incompatibility among SAM types with same signatures

- Hard to read under moderate-to-heavy use

- Encourage “exotic” style of programming

- Don't reflect semantic constraints

- Don't provide the same level of documentation

- Don’t interact well with autocompletion (or grep)

- Limited interoperability may balkanize libraries

www.javapolis.com

bgga closures have two kinds of returns
BGGA Closures Have Two Kinds of Returns

static boolean test(boolean arg) {

{boolean => boolean} closure = { boolean arg =>

if (arg)

return true; // Non-local return

false // local return

};

return !closure.invoke(arg);

}

return means something completely different in a BGGA closure and an anonymous class

www.javapolis.com

what does test return
What Does test() Return?

static <E> Boolean contains(Iterable<E> seq, Predicate<E> pred) {

for (E e : seq)

if (pred.invoke(e))

return true;

return false;

}

static Boolean test() {

List<Character> list = Arrays.asList(

'h', 'e', 'l', 'l', 'o');

return contains(list, new Predicate<Character>() {

public Boolean invoke(Character c) {

return c > 'j';

}

});

}

interface Predicate<T> { Boolean invoke(T t); }

www.javapolis.com

now what does test return bgga
Now What Does test() Return? (BGGA)

static <E> Boolean contains(Iterable<E> seq, {E => Boolean} p) {

for (E e : seq)

if (p.invoke(e))

return true;

return false;

}

static Boolean test() {

List<Character> list = Arrays.asList(

'h', 'e', 'l', 'l', 'o');

return contains(list, { Character c => return c > 'j'; } );

}

www.javapolis.com

now what does test return bgga39
Now What Does test() Return? (BGGA)

static <E> Boolean contains(Iterable<E> seq, {E => Boolean} p) {

for (E e : seq)

if (p.invoke(e))

return true;

return false;

}

static Boolean test() {

List<Character> list = Arrays.asList(

'h', 'e', 'l', 'l', 'o');

return contains(list, { Character c => return c > 'j'; });

}

Accidental non-local return due to cut-and-paste from anonymous class can cause insidious bug

www.javapolis.com

suppose you wanted to translate this method to bgga
Suppose You Wanted to Translate this Method to BGGA

static <E> Predicate<Iterable<E>> contains(

final Predicate<E> pred) {

return new Predicate<Iterable<E>>() {

public Boolean invoke(Iterable<E> seq) {

for (E e : seq)

if (pred.invoke(e))

return true;

return false;

}

};

}

www.javapolis.com

it s awkward as only one local return is permitted
It’s Awkward, as Only One Local Return is Permitted

static <E> { Iterable<E> => Boolean } contains(

{ E => Boolean } pred) {

return { Iterable<E> seq =>

Boolean result = false;

for (E e : seq) {

if (pred.invoke(e)) {

result = true;

break;

}

}

result

};

}

www.javapolis.com

summary pros and cons of non local returns
Summary - Pros and Cons of Non-Local Returns

+ Permits library-defined control structures

- Having two kinds of returns is confusing

- Meaning of return has changed

- Unintentional non-local returns can cause bugs

- Only one local return permitted per closure

www.javapolis.com

what does this program print
What Does This Program Print?

public class Test {

private static final int N = 10;

public static void main(String[] args) {

List<{ => int}> closures = new ArrayList<{ => int}>();

for (int i = 0; i < N; i++)

closures.add( { => i } );

int total = 0;

for ({ => int} closure : closures)

total += closure.invoke();

System.out.println(total);

}

}

www.javapolis.com

what does this program print44
What does this program print?

public class Test {

private static final int N = 10;

public static void main(String[] args) {

List<{ => int}> closures = new ArrayList<{ => int}>();

for (int i = 0; i < N; i++)

closures.add( { => i } );

int total = 0;

for ({ => int} closure : closures)

total += closure.invoke();

System.out.println(total);

}

}

It prints 100, not 45. The same loop variable is captured by all 10 closures and evaluated after the loop is finished!

www.javapolis.com

summary pros and cons of access to nonfinal locals
Summary - Pros and Cons of Access to Nonfinal Locals

+ Permits library-defined control structures

+ Eliminates some uses of final modifier

- Semantics can be very confusing

- Locals can persist after their scope is finished

- Locals can be modified by other threads

- Performance model for locals will change

www.javapolis.com

library defined control constructs
Library-Defined Control Constructs
  • What are the compelling use-cases?
    • Custom loops
    • Automatic resource management blocks
    • Timer block

www.javapolis.com

custom for loops example from bgga spec
Custom for-loops (Example from BGGA Spec)

<K,V,throws X>

void for eachEntry(Map<K,V> map, {K,V=>void throws X} block)

throws X {

for (Map.Entry<K,V> entry : map.entrySet()) {

block.invoke(entry.getKey(), entry.getValue());

}

}

for eachEntry(String name, Integer value : map) {

if ("end".equals(name)) break;

if (name.startsWith("com.sun.")) continue;

System.out.println(name + ":" + value);

}

www.javapolis.com

what s wrong with this example
What's Wrong With This example?
  • BGGA loop competes with Java 5 for-each
    • Don't define a construct; just implement Iterable
  • Only compelling use is multiple loop variables
  • Last example doesn't offer power of for-each
    • Loop variable can’t be primitive (no auto-unboxing)
    • Would require 81 (!) overloadings to do fix this

www.javapolis.com

loop syntax tailored to for awkward for while
Loop syntax tailored to for; awkward for while

public static <throws X> void for myWhile(

{=> boolean throws X} cond, {=>void throws X} block) throws X {

while (cond.invoke()) {

block.invoke();

}

}

for myWhile( { => i < 7 } ) {

System.out.println(i++);

}

myWhile( { => i < 7 }, { =>

System.out.println(i++);

});

www.javapolis.com

automatic resource management block bgga spec
Automatic Resource Management Block (BGGA Spec)

<R, T extends Closeable, throws X>

R with(T t, {T=>R throws E} block) throws X {

try {

return block.invoke(t);

} finally {

try { t.close(); } catch (IOException ex) {}

}

}

with (FileReader in : makeReader()) { // Requires nesting

with (FileWriter out : makeWriter()) {

// code using in and out

}

}

www.javapolis.com

library defined control constructs are a double edged sword
Library-Defined Control Constructs are a Double-Edged Sword
  • A great feature of Java is “programmer portability”
    • All Java code looks pretty much alike
    • We can read and maintain each other's code with a minimum of effort
  • Library-Defined Control Constructs fosterdialects, which hinder programmer portability

www.javapolis.com

performance is an open question
Performance is an Open Question

Time to iterate over 108 elements(Take these numbers with a huge grain of salt)

Java HotSpot(TM) Server VM (build 1.6.0-b105, mixed mode)

Windows XP, Intel T2600 @ 2.16 GHz, 2GB RAM

BGGA prototype closures-2007-11-30 (I apologize)

www.javapolis.com

summary programmer defined control constructs
Summary - Programmer-Defined Control Constructs

+ They let you define new control constructs

- They don’t have the same syntax, semantics, or performance as built-in control constructs

- It’s not clear that you need the ability; Java already has a rich set of control structures

- Can lead to dialects

- Responsible for much of the complexity in BGGA

- Non-local return, break and continue

- Unrestricted access to mutable local variables

www.javapolis.com

outline54
Outline

I. Setting the Stage

II. Why Enhance Support for Closures?

III. BGGA Closures

IV. A Lightweight Approach

V. Where Do We Go From here?

www.javapolis.com

a lightweight approach
A Lightweight Approach
  • There is a much simpler approach to reducing the verbosity of anonymous classes and manual resource management
  • Attack the two problems head-on!
    • Concise syntax for anonymous class instance creation
    • Purpose-built construct for automatic resource management

www.javapolis.com

concise instance creation expressions cice
Concise Instance Creation Expressions (CICE)

This:

sort(list, Comparator<String>(String s1, String s2) {

return s1.length() - s2.length();

});

Expands to this:

sort(list, new Comparator<String>() {

public int compare(String s1, String s2) {

return s1.length() - s2.length();

}

});

I am not in love with this syntax; we can probably do better.

www.javapolis.com

automatic resource management arm blocks
Automatic Resource Management (ARM) Blocks

// One resource - readFirstLineFromFile

try (BufferedReader r = new BufferedReader(new FileReader(path)) {

String s = r.readLine();

}

// Multiple resources - copyFile

try (InputStream in = new FileInputStream(src);

OutputStream out = new FileOutputStream(dest)) {

byte[] buf = new byte[1024];

int n;

while ((n = in.read(buf)) >= 0)

out.write(buf, 0, n);

}

www.javapolis.com

more automatic resource management blocks
More Automatic Resource Management Blocks

Perhaps this:

protected (lock) {

... // access the resource protected by this lock

}

Should be shorthand for this:

lock.lock();

try {

// access the resource protected by this lock

} finally {

lock.unlock();

}

www.javapolis.com

for more information
For More Information
  • Automatic Resource Management Blocks (ARM)
    • http://docs.google.com/View?docid=dffxznxr_1nmsqkz
  • Concise Instance Creation Expressions (CICE)
    • http://docs.google.com/View?docid=k73_1ggr36h
  • Both of these docs are a bit sketchy 

www.javapolis.com

outline60
Outline

I. Setting the Stage

II. Why Enhance Support for Closures?

III. BGGA Closures

IV. Lightweight Closures

V. Where Do We Go From here?

www.javapolis.com

the closures spectrum
The Closures Spectrum

CICE+ARM

Bob Lee

www.javapolis.com

the big questions
The Big Questions
  • Do we really want function types in Java?
  • Do we really want library-defined control constructs in Java?
  • We should keep in mind that there’s already a fine programming language for the JVM that offers both of these things, and Java interoperability to boot: Scala

www.javapolis.com

the medium sized questions
The Medium-Sized Questions
  • Should we support non-local return, break, and continue? If so:
    • Should it be the default?
    • How do we prevent accidents?
  • Should we allow access to non-final local variables from closures? If so:
    • Should it be the default?
    • How do we prevent accidents?

www.javapolis.com

moving forward there are two approaches
Moving Forward, There are Two Approaches
  • Gain more experience with prototypes
    • When we come to an informed consensus, start a narrowly focused JSR
    • This might take a couple of years
    • Precedent established by generics
  • Start a broadly focused JSR in the near future
    • Must permit any point on the Closure Spectrum
    • First task is to answer the two Big Questions

www.javapolis.com

closing sermon
Closing Sermon
  • We have a big decision to make!
  • It will have a huge effect on the future of the Java platform
  • We must take our time and do what is right
  • We must not risk further damage to“the feel of Java”

www.javapolis.com

slide66

Q&A

View JavaPolis talks @ www.parleys.com