Core Java

The Dark Side Of Lambda Expressions in Java 8

blog_lambada_2

This post may not make me any new friends. Oh well, I was never really popular at school anyway. But let’s get to the point. Java 8’s biggest feature in terms of the language is undoubtedly Lambda expressions. It’s been a flagship feature for functional languages such as Scala and Clojure for a few years, and now Java has finally joined in.

The second biggest feature (depending of course on who you ask) is Nashorn – the new JVM JavaScript engine that’s supposed to bring Java up to par with other JS engines such as V8 and its node.js container.

But these new features have a dark side to them.

I’ll explain. The Java platform is built out of two main components. The JRE, which JIT compiles and executes bytecode, and the JDK which contains dev tools and the javac source compiler. These two components are fairly (but not fully) decoupled, which is what enables folks to write their own JVM languages, with Scala rising to prominence in the last few years. And therein lies some of the problem.

The JVM was built to be language agnostic in the sense that it can execute code written in any language, as long as it can be translated into bytecode. The bytecode specification itself is fully OO, and was designed to closely match the Java language. That means that bytecode compiled from Java source will pretty much resemble it structurally.

But the farther away you get from Java – the more that distance grows. When you look at Scala which is a functional language, the distance between the source code and the executed bytecode is pretty big. Large amounts of synthetic classes, methods and variables are added by the compiler to get the JVM to execute the semantics and flow controls required by the language.

When you look at fully dynamic languages such as JavaScript, that distance becomes huge.

And now with Java 8, this is beginning to creep into Java as well.

So why should I care?

I wish this could be a theoretical discussion, that while interesting, has no practical implication on our everyday work. Unfortunately it does, and in a very big way. With the push to add new elements into Java, the distance between your code and the runtime grows, which means that what you’re writing and what you’re debugging will be two different things.

To see how let’s (re)visit the example below.

Java 6 & 7

This is the traditional method by which we would iterate over a list of strings to map their lengths.

// simple check against empty strings
public static int check(String s) {
    if (s.equals("")) {
        throw new IllegalArgumentException();
    }
    return s.length();
}
 
//map names to lengths
 
List lengths = new ArrayList();
 
for (String name : Arrays.asList(args)) {
    lengths.add(check(name));
}

This will throw an exception if an empty string is passed. The stack trace will look like –

at LmbdaMain.check(LmbdaMain.java:19)
at LmbdaMain.main(LmbdaMain.java:34)

Here we see a 1:1 correlation between the stack trace we see and the code we wrote, which makes debugging this call stack pretty straightforward. This is what most Java devs are used to.

Now let’s look at Scala and Java 8.

Scala

Let’s look at the same code in Scala. Here we’ve got two big changes. The first is the use of a Lambda expression to map the lengths, and the second is that the iteration is carried out by the framework (i.e. internal iteration).

val lengths = names.map(name => check(name.length))

Here we really start to notice the difference between how the code you wrote looks, and how the JVM (and you) will see it at run time. If an exception is thrown, the call stack is an order of magnitude longer, and much harder to understand.

at Main$.check(Main.scala:6)
at Main$$anonfun$1.apply(Main.scala:12)
at Main$$anonfun$1.apply(Main.scala:12)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.List.foreach(List.scala:318)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at Main$delayedInit$body.apply(Main.scala:12)
at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App$$anonfun$main$1.apply(App.scala:71)
at scala.App$$anonfun$main$1.apply(App.scala:71)
at scala.collection.immutable.List.foreach(List.scala:318)
at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:32)
at scala.App$class.main(App.scala:71)
at Main$.main(Main.scala:1)
at Main.main(Main.scala)

* Remember, this example is very simple. With real-world nested Lambdas and complex structures you’ll be looking at much longer synthetic call stacks, from which you’ll need to understand what happened.

This has long been an issue with Scala, and one of the reasons we built the Scala Stackifier.

And now in Java 8

Up until now Java developers were pretty immune to this. This will change as Lambda expressions become an integral part of Java. Let’s look at the corresponding Java 8 code, and the resulting call stack.

Stream lengths = names.stream().map(name -> check(name));

at LmbdaMain.check(LmbdaMain.java:19)
at LmbdaMain.lambda$0(LmbdaMain.java:37)
at LmbdaMain$$Lambda$1/821270929.apply(Unknown Source)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:512)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:502)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.LongPipeline.reduce(LongPipeline.java:438)
at java.util.stream.LongPipeline.sum(LongPipeline.java:396)
at java.util.stream.ReferencePipeline.count(ReferencePipeline.java:526)
at LmbdaMain.main(LmbdaMain.java:39)

This is becoming pretty similar to Scala. We’re paying the price for shorter, more concise code with more complex debugging, and longer synthetic call stacks.

The reason is that while javac has been extended to support Lambda functions, the JVM still remains oblivious to them. This has been a design decision by the Java folks in order to to keep the JVM operating at a lower-level, and without introducing new elements into its specification.

And while you can debate the merits of this decision, it means that as Java developers the cost figuring out these call stacks when we get a ticket now sadly lies on our shoulders, whether we want to or not.

JavaScript in Java 8

Java 8 introduces a brand new JavaScript compiler. Now we can finally integrate Java + JS in an efficient and straightforward manner. However, nowhere is the dissonance between the code we write and the code we debug bigger than here.

Here’s the same function in Nashorn –

ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("nashorn");

String js = "var map = Array.prototype.map \n";
js += "var a = map.call(names, function(name) { return Java.type(\"LmbdaMain\").check(name) }) \n";
js += "print(a)";
engine.eval(js);

In this case the bytecode code is dynamically generated at runtime using a nested tree of Lambda expressions. There is very little correlation between our source code, and the resulting bytecode executed by the JVM. The call stack is now two orders of magnitude longer. In the poignant words of Mr.T – I pity the fools who will need to debug the call stack you’ll be getting here.

Questions, comments? (assuming you can scroll all the way below this call stack). Let me know in the comments section.

LmbdaMain [Java Application]
LmbdaMain at localhost:51287
Thread [main] (Suspended (breakpoint at line 16 in LmbdaMain))
LmbdaMain.wrap(String) line: 16
1525037790.invokeStatic_L_I(Object, Object) line: not available
1150538133.invokeSpecial_LL_I(Object, Object, Object) line: not available
538592647.invoke_LL_I(MethodHandle, Object[]) line: not available
1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available
LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147
LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625
LambdaForm.interpretWithArguments(Object...) line: 604
2150540.interpret_I(MethodHandle, Object, Object) line: not available
538592647.invoke_LL_I(MethodHandle, Object[]) line: not available
1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available
LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147
LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625
LambdaForm.interpretWithArguments(Object...) line: 604
92150540.interpret_I(MethodHandle, Object, Object) line: not available
38592647.invoke_LL_I(MethodHandle, Object[]) line: not available
1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available
LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147
LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625
LambdaForm.interpretWithArguments(Object...) line: 604
731260860.interpret_L(MethodHandle, Object, Object) line: not available
LambdaForm$NamedFunction.invoke_LL_L(MethodHandle, Object[]) line: 1108
1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available
LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147
LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625
LambdaForm.interpretWithArguments(Object...) line: 604
2619171.interpret_L(MethodHandle, Object, Object, Object) line: not available
1597655940.invokeSpecial_LLLL_L(Object, Object, Object, Object, Object) line: not available
LambdaForm$NamedFunction.invoke_LLLL_L(MethodHandle, Object[]) line: 1118
1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available
LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147
LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625
LambdaForm.interpretWithArguments(Object...) line: 604
2619171.interpret_L(MethodHandle, Object, Object, Object) line: not available
1353530305.linkToCallSite(Object, Object, Object, Object) line: not available
Script$\^eval\_._L3(ScriptFunction, Object, Object) line: 3
1596000437.invokeStatic_LLL_L(Object, Object, Object, Object) line: not available
1597655940.invokeSpecial_LLLL_L(Object, Object, Object, Object, Object) line: not available
LambdaForm$NamedFunction.invoke_LLLL_L(MethodHandle, Object[]) line: 1118
1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available
LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147
LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625
LambdaForm.interpretWithArguments(Object...) line: 604
484673893.interpret_L(MethodHandle, Object, Object, Object, Object, Object) line: not available
LambdaForm$NamedFunction.invoke_LLLLL_L(MethodHandle, Object[]) line: 1123
1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available
LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147
LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625
LambdaForm.interpretWithArguments(Object...) line: 604
282496973.interpret_L(MethodHandle, Object, Object, Object, long, Object) line: not available
93508253.invokeSpecial_LLLLJL_L(Object, Object, Object, Object, Object, long, Object) line: not available
1850777594.invoke_LLLLJL_L(MethodHandle, Object[]) line: not available
1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available
LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147
LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625
LambdaForm.interpretWithArguments(Object...) line: 604
282496973.interpret_L(MethodHandle, Object, Object, Object, long, Object) line: not available
293508253.invokeSpecial_LLLLJL_L(Object, Object, Object, Object, Object, long, Object) line: not available
1850777594.invoke_LLLLJL_L(MethodHandle, Object[]) line: not available
1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available
LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147
LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625
LambdaForm.interpretWithArguments(Object...) line: 604
1840903588.interpret_L(MethodHandle, Object, Object, Object, Object, long, Object) line: not available
2063763486.reinvoke(Object, Object, Object, Object, Object, long, Object) line: not available
850777594.invoke_LLLLJL_L(MethodHandle, Object[]) line: not available
1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available
LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147
LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625
LambdaForm.interpretWithArguments(Object...) line: 604
82496973.interpret_L(MethodHandle, Object, Object, Object, long, Object) line: not available
220309324.invokeExact_MT(Object, Object, Object, Object, long, Object, Object) line: not available
NativeArray$10.forEach(Object, long) line: 1304
NativeArray$10(IteratorAction).apply() line: 124
NativeArray.map(Object, Object, Object) line: 1315
1596000437.invokeStatic_LLL_L(Object, Object, Object, Object) line: not available
504858437.invokeExact_MT(Object, Object, Object, Object, Object) line: not available
FinalScriptFunctionData(ScriptFunctionData).invoke(ScriptFunction, Object, Object...) line: 522
ScriptFunctionImpl(ScriptFunction).invoke(Object, Object...) line: 207
ScriptRuntime.apply(ScriptFunction, Object, Object...) line: 378
NativeFunction.call(Object, Object...) line: 161
1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available
1740189450.invokeSpecial_LLL_L(Object, Object, Object, Object) line: not available
LambdaForm$NamedFunction.invoke_LLL_L(MethodHandle, Object[]) line: 1113
1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available
LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147
LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625
LambdaForm.interpretWithArguments(Object...) line: 604
2619171.interpret_L(MethodHandle, Object, Object, Object) line: not available
LambdaForm$NamedFunction.invoke_LLL_L(MethodHandle, Object[]) line: 1113
1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available
LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147
LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625
LambdaForm.interpretWithArguments(Object...) line: 604
323326911.interpret_L(MethodHandle, Object, Object, Object, Object) line: not available
LambdaForm$NamedFunction.invoke_LLLL_L(MethodHandle, Object[]) line: 1118
1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available
LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147
LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625
LambdaForm.interpretWithArguments(Object...) line: 604
323326911.interpret_L(MethodHandle, Object, Object, Object, Object) line: not available
263793464.invokeSpecial_LLLLL_L(Object, Object, Object, Object, Object, Object) line: not available
LambdaForm$NamedFunction.invoke_LLLLL_L(MethodHandle, Object[]) line: 1123
1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available
LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147
LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625
LambdaForm.interpretWithArguments(Object...) line: 604
1484673893.interpret_L(MethodHandle, Object, Object, Object, Object, Object) line: not available
587003819.invokeSpecial_LLLLLL_L(Object, Object, Object, Object, Object, Object, Object) line: not available
811301908.invoke_LLLLLL_L(MethodHandle, Object[]) line: not available
1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available
LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147
LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625
LambdaForm.interpretWithArguments(Object...) line: 604
484673893.interpret_L(MethodHandle, Object, Object, Object, Object, Object) line: not available
LambdaForm$NamedFunction.invoke_LLLLL_L(MethodHandle, Object[]) line: 1123
1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available
LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147
LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625
LambdaForm.interpretWithArguments(Object...) line: 604
323326911.interpret_L(MethodHandle, Object, Object, Object, Object) line: not available
2129144075.linkToCallSite(Object, Object, Object, Object, Object) line: not available
Script$\^eval\_.runScript(ScriptFunction, Object) line: 3
1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available
1709804316.invokeExact_MT(Object, Object, Object, Object) line: not available
FinalScriptFunctionData(ScriptFunctionData).invoke(ScriptFunction, Object, Object...) line: 498
ScriptFunctionImpl(ScriptFunction).invoke(Object, Object...) line: 207
ScriptRuntime.apply(ScriptFunction, Object, Object...) line: 378
NashornScriptEngine.evalImpl(ScriptFunction, ScriptContext, ScriptObject) line: 544
NashornScriptEngine.evalImpl(ScriptFunction, ScriptContext) line: 526
NashornScriptEngine.evalImpl(Source, ScriptContext) line: 522
NashornScriptEngine.eval(String, ScriptContext) line: 193
NashornScriptEngine(AbstractScriptEngine).eval(String) line: 264
LmbdaMain.main(String[]) line: 44
Reference: The Dark Side Of Lambda Expressions in Java 8 from our JCG partner Tal Weiss at the Takipi blog.

Tal Weiss

Tal is co-founder and CEO at Takipi where he leads a team of hackers that build tools that help fix Java and Scala code in production. His main interests are tools for developers, solving nasty bugs (preferably multi-threaded) and Jazz drumming.
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

21 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
dreadwolf
dreadwolf
9 years ago

Nice. Java joined the “write once – f*** you” group of languages.

Riccardo
9 years ago

Starting today, I’m your friend.
At last there is someone else who looks beyond the marketing; I totally agree with this post and think lambdas simply aren’t for general (developers) public.

Conrad Winchester
Conrad Winchester
9 years ago

Hi Tal, I was involved in commenting on some very early implementations of lambda functions is Java. I totally agree with you – but it’s even worse than you think. In languages that are truly functional you can have functions as objects. Java 8 lambdas makes you think that this is the case with method references. Here is a simple abstract example. Say you have an event producer object to which you can add listeners (maybe they are stored in a HashMap in the producer). With Java 8 you could ass a listener like this (using the new method reference… Read more »

Tal Weiss
Tal Weiss
9 years ago

Thanks Conrad! beautiful comment.

moi
moi
9 years ago

I just learned this the hard way. I wrote a bunch of code expecting removeListener to work, but it doesn’t :/ Unbelievable. There were millions of posts about adding listeners with method references and I loved the possibility… But none of them mentioned you can’t remove the damn thing. This makes method references completely useless in this scenario. I wish I read this comment earlier. So sad :((

Jake Zimmerman
Jake Zimmerman
9 years ago

Easy workaround:

EventListener listener = this::eventListener;

Pass that into eventProducer’s methods and viola!

You WOULD kind of hope that every time you reference a method, it would be the same object, but it’s not that big of a deal to actually set it into a variable (you know, like almost every other object you use). Make it final if you’re worried that it might accidentally get changed somewhere.

Conrad Winchester
Conrad Winchester
9 years ago

Sorry- the formatting got screwed up in my post – I hope it still makes sense

Yves
Yves
9 years ago

My point of view is that debugger has to show current stack trace in a smarter way, for instance with a small set of entities to be processed in each stream, a counter about how many entries has been processed, watch point and break point to intercept when a filter is applied on an entity… and so on. Sincerely that is a complex dynamic UI design that has to scale with the complexity of the lambda expression under debug, to avoid a developer to split his expression into parts for debugging, just losing the “benefits” of this new support in… Read more »

Nasruddin
9 years ago

You can add new friend in your list. Totally agree with the post. I would be glad to write boilerplate code till it makes my life easier while debugging. Thumbs mate for the post !

Shiva
Shiva
9 years ago

Nice article. I think we should have race condition for JVM programming languages.

Norman Richards
Norman Richards
9 years ago

Wow. If a little noise in stack traces is the greatest expense, it seems Java 8 Lambdas were very cheap. Java developers have long been used to long stack traces from: web/app servers and various frameworks. The stack traces are very clear about which lines are your application code and which aren’t, so I can’t imagine any Java developer spending more than a couple extra seconds to figure out what the offending lines were. And, did you know that when you can get an exception, you can get the stack trace data programmatically? You can easily write a stack trace… Read more »

Jake Zimmerman
Jake Zimmerman
9 years ago

Plus, there seams to be something that no one is taking into account: almost all of that stack trace has to do with the streams API, NOT the lambda stuff. Yes, there’s a lot going on behind the scenes with streams, but you gotta show me a big stack trace just from lambdas to throw me off. Even that barely will, because of what you said.

Jake Zimmerman
Jake Zimmerman
9 years ago

Just looked into it: there is only one additional class in the stack trace when it comes to a simple lambda call.

at LambdaStackTraceTest.lambda$0(LambdaStackTraceTest.java:19)
at LambdaStackTraceTest$$Lambda$2/1523554304.run(Unknown Source)
at LambdaStackTraceTest.referenceRunner(LambdaStackTraceTest.java:29)
at LambdaStackTraceTest.main(LambdaStackTraceTest.java:19)

The first one is the actual lambda I wrote, the second is the added thing. The third line is the method calling the lambda, and the last is the main.
If you use a method reference, the top line is simply replaced by the method being referenced.

Those huge stack traces are due to the complexity of streams, not because of lambdas.

Mike
Mike
9 years ago

Thank for this post. Hard times for maintainers come…

Dmytro
Dmytro
9 years ago

2 cents
if (s.equals(“”)) is not correct –
if (“”.equals(s)) is correct . You avoid nullpointer exception

Tal Weiss
Tal Weiss
9 years ago
Reply to  Dmytro

Thanks Dmytro. I’d still get the exception two lines later when I do s.length(), but in general you are right that there should be an additional check to protect against null values.

Tom Henricksen
9 years ago

I would agree with Riccardo that the general developer won’t embrace lambdas. Of course time will tell if this was a good addition to the Java language or not. Perhaps Java has grown too big and we need to remove some more things. Thanks for sharing Tal!

Tom

Anant Raman
Anant Raman
8 years ago

I am learning Scala now in preparation for Apache Spark. I noticed what Tal brings up in the Map Reduce also. Whenever there is an error, the system presents a huge list of errors that scans several pages.

Sean$4
Sean$4
8 years ago

I think Java 8 is very good. I think lambdas are an okay way to save a few lines of code but I don’t see them as super clarifying. Generics are much more confusing and I can only use them in simple way. I learnt Java around 1.5 if I remember correctly, it was somewhat uphill but doable. I wouldn’t like try starting out with Java now. It would take a massive effort to get to a basic industry level. On the other hand javafx 8 is easy and very good for GUI’s and there are a lot of helper… Read more »

Serhiy Brytskyy
Serhiy Brytskyy
4 years ago

Totally agree.
When the code looks simpler, it almost always hard to read it.
It becomes nightmare to understand logic, when you read the code, written not by you, or even by you, but while ago.
I would rather have some boilerplate code, than headache while trying to debug what is this code doing….

Jignesh
Jignesh
4 years ago

Basically, when the Java compiler sees a Lambda expression, it recursively searches through the list of interfaces known to it until it finds an interface that defines a single method whose arguments match the types of those used by the lambda. It takes the compiler (or IDE) a fraction of a second to do a brute-force search through the API to find a match. Humans, unfortunately, aren’t quite so agile at things like that, which is why we invented source code in the first place 50 years ago. Behind the scenes, the compiler is just automatically assembling an anonymous class… Read more »

Back to top button