SPONSORED BY IBM
Ryan welcomes Markus Eisele to the program to talk about why your coding agent should be writing Java. They talk about why the long history of Java both makes for a stable language and a source of training data for AI. On top of that, they discuss how agentic Java excels by working with powerful partners: a massive ecosystem of Java libraries and agentic harnesses.
Try out Bob, IBM’s coding agent that Markus talked about in the episode.
Connect with Markus on LinkedIn.
Congrats to Populist badge winner, Alex K, whose answer to ffmpeg usage to encode a video to H264 codec format outshined the accepted answer.
TRANSCRIPT
[intro music]
Ryan Donovan: Welcome to the Stack Overflow podcast, a place to talk all things software and technology. I'm your host, Ryan Donovan, and today we have an interesting episode sponsored by the fine folks at IBM talking about why Java is the best language for the agentic enterprise.
And my guest for that today is Markus Eisele, who is a developer advocate at IBM. So welcome to the show, Markus.
Markus Eisele: Thank you, Ryan. Pleasure to be here.
RD: So before we get into the topic, let's get to know you. How did you get into software and technology?
ME: Oh, well, I don't know how much time we actually have, but yeah. Okay. I pretty much started like everybody else out of university. Started with my first software development consulting engagements for a German-based ISV. Stuck around for many, many years. Ended up solving some very, very large problems in enterprise Java technologies, architectures for some pretty large customers. The one thing that actually stuck throughout my whole career is my idea of how people learn. So I want to share what I've learned, right? And I started very early giving internal presentations. And at some point, I basically deviated into what's now called developer advocacy, outbound product marketing, whatever you like stamp on it. The intrinsic desire to share knowledge. So now I'm an O'Reilly author, been credited as a Java champion. So all of this basically came because I wanted to share knowledge. And I still do development as of today. I contribute to open source, to Quarkus, Langchain4j, wherever I get my hands on and I find something that... with my limited experiences as a software developer lately, I can fix, I definitely still fix it.
RD: You do have some experience in Java. That was one of the first languages that I learned in school. I understand your position is that it's awesome for agentic engineering, but why?
ME: Oh, that's pretty simple. It's mainly two buckets. The first bucket is all about training data. Java has been around forever. I can't believe, but we'll end up having Java 26 enhanced very soonish. When I started with Java, it was more around 1.2, 1.3 releases, so very early releases.
That basically means all of Java has been in the open source, OpenJDK, since forever, right? So in every single model, you'll find the basics of Java. And on top, it has been and still is as of today, the premier language for enterprise problems, like solving enterprise problems, not making them, obviously. So the idea that we'll find training data examples, like even from your side, all over in these large language models, that is the one bucket. And the other bucket, it is a very strict language. It is pretty type safe. It has well-known structures that are very well-versed in terms of language constructs. And that basically means they are ideal not only to read and consume during training phases, but also to generate. And that makes it so much easier for AI agents, AI-driven tools to use Java. And I mean, modern Java is not exactly comparable with old-day Java. It made light years, light steps ahead, and it's really almost a complete rewrite in many cases, while still being mostly backward compatible. But having said that, that gives us the opportunity to just use agents with a high confidence on these code bases.
RD: Would it be fair to say that Java is a language with a lot of guardrails built into it?
ME: Absolutely. And not only the language. We can easily expand this into the platform.
I've said enterprise, right? And that always has kind of like a smell to it. It's old, it's boring. You know what? It's rock solid. It is something that has been proven to work through fires, storms, thunders, like that thing, whatever it is that still is in production based on a JVM that has been tested, built. It can be operated by teams that feel comfortable doing this. It can be scaled, right? So there is not only the language piece, it's also the platform and the tooling that we built around all of this, in particularly frameworks on top, right? Think about Spring, think about Quarkos. They are almost adding enterprise level of abstraction on top of the sheer standards platform, right? And that makes it just ideal.
RD: Yeah. I think, you know, you talk about it having a long, long history, and I also started on Java 1 in school. Does any of that history get in the way of, you know, good agentic engineering? Is it ever producing like, you know, old paradigms, old code?
ME: I have not seen this. I mean, at least not if you use agents the right way. We all know a little bit about context engineering, about how to create the correct grounding approaches for the task at hand, right? So, I mean, that's still up to every engineer out there. What I have seen occasionally are bad examples coming in the way. So I mentioned that as a pro in the very beginning, the large amount of training data. But honestly, there's also a lot of not exactly best practices out there, like historically seen, right? So if we compare that, we might see one or the other approach that is not exactly idiomatic with latest Java as it came out just a couple of months ago. But it's rare. I rarely see it in my own projects.
RD: So for some of the language features, like the types, there's various annotation add-ons. How do you get those to improve the quality of the Java code added? Are there things you can do in the prompting specs?
ME: Yeah, I don't think that this is even the main angle of the new features that have been added to the platform. If we look at Project Panama, for example, the goal is basically to expand the platform into natives that are also suitable to help with data scientist kind of problems, right? And I think that is even more interesting, to open the platform up from a programming perspective to new approaches. And we've seen inventions like Tornado VM. We've seen frameworks that actually run large language models on the JVM natively in Java. So that is a direction that Java is taking right now, which really brings a lot of these solidity of the JVM ecosystem tooling into the new world. Do I need to teach a JVM modern Java? An AI agent modern Java? I don't think so. I have not had to do that. There are a couple of famous tools that do use MCP servers. There's an amazing upstream project. I think it's called Grounded Docs that I personally love a lot. So I do curate my own little indices over certain documents that I find really high value. And if I want an agent to do a certain task, I'll actually let them do it. It's kind of the context seven of open source, right? So that's all things I do as an engineer to make sure agents don't go off rails. But yeah, the more interesting pieces are really where Java as a platform is headed.
RD: Yeah. I think you touched on it a little bit, but the sort of enterprise backbone that Java is. I remember my first job, we had a Java version, and it was so idiomatic on each, a different operating system that we had to test it thoroughly. And now it's just like running entire backends for service-oriented architectures. For the agentic coding, what complications are still there for Java?
ME: I think that's an unexpected answer. Java itself is not the complication, right? What actually is the complication is history. History, legacy, all the heritage that we basically inherited over years of large projects. I mean, I just saw a meme a couple of days ago, like when your project is older than the youngest intern. And that's kind of the stuff we actually have to deal with, right? There are developed software projects out there. And I'm guilty for at least one or two of them that are in production since 10 plus years. And these projects, they have plenty of challenges. And the sheer number of source code is just one of them. Documentation. That was last updated when the platform team was still called operations, right? We have... we have that specific exception in the code for one specific customer that nobody actually documented and everybody forgot because the one guy who figured that out after a year of work left. So that's kind of the challenges, right? So how can I extract the intent out of these large software projects and make it available for agents? And that is kind of one of the biggest challenges that we see. And yes, agents can give them time, give them loops. They'll just go cracking and come back at some point and tell you something about it. What we do know, though, is that this can be comparably costly and the results are not necessarily stable. So that's a field of tension that we see pretty often, very much motivated from modernization at large.
RD: Yeah. So I've definitely heard a lot of people talking about loops and iteration as the way
to get better at agentic engineering. But what is, besides the token spend, what is the sort of issue that people are running into in producing agentic code that way?
ME: Issue. I mean, it's fun, right? The first term that comes to mind is Vibe coding. And I mean, we've all done that, right? I'm guilty too. I threw TypeScript and Tailwind CSS at a large language model AI agent, and it spit out an insanely looking website. But honestly, I had no clue how it was built, how it even came to be, right? And if I just imagine my bank, do I want my bank to follow a vibe coding approach? No, no, really not, right? So systematic change, almost surgically changing pieces that are guaranteed to work is a completely different requirement in my perspective compared to vibe coding and loops. And I might get yelled at for that, right? But at the end of the day, I think it's a very fine balance to strike in, particularly when we're looking at changing established applications. And we need to find this balance between risk acceptance, stability. We have industries where none of this is an option at all. We need to have proof who changed what. Agentic coding has, I think, two sides of the scale, right? We go from all in loops, vibe coding, let's just like literally brute forcing solutions with tokens to precision engineering on the right side. I have to admit, I lean towards precision engineering because it is just so much better to know what is actually changing to be in control. Many people talk about this as the human in the loop, but that kind of focuses on on the final review, right? I mean, that means the agent finishes after whatever hours and just spits out a result and you get to review it. What a daunting task. I actually want to see the steps happening. So I want to be almost taken through a workflow. I want to be assisted, guided, partnered with instead of just like, hey, human, here's your review. Go take a crack at it.
RD: The way I've heard people put it is like, No, no human in the loop, man. It's my loop.
ME: Yeah, it's weird. That's not how I envision software engineering to happen. Human in the loop to me means human in control, making sure that the end result is auditable, understood, that we have the full documentation, even something boring like unit testing, right? I mean, that alone is worth another podcast talking about how much that adds on top of agentic engineering if you have a solid testing foundation.
RD: Yeah, absolutely. You talked about a lot of these Java backends being big legacy operations. And I wonder if there's something baked into the way that Java was released that made it a little harder. Because when I saw Java applications being made, people would wait huge, huge release gaps before upgrading. Each release was massive, and it was such a problem to update it. So I wonder if there's any sort of legacy effects of that in the code, any sort of disjointed nature of the code you're running to.
ME: Oh, absolutely. I mean, modernization is not modernization. You can simply bump the runtime and stick to old programming constructs and habits, and you can also apply the more idiomatic new features that are coming with a new version. And to be honest, knowing how costly it is to test enterprise solutions, I do understand a lot of customers who are just running their standard playbook, bumping the underlying version, not touching the code, at least not as much as they don't have to. What a sentence, Markus. It's really, really hot in Germany these days. What gives us the confidence to still do these updates, that that is something I think that people are underestimating. That's not only the manual testing effort. I said legacy applications barely have any automated testing, not alone documentation. I mentioned that, but what I think is interesting is that there are well-established technologies and approaches to do these modernizations, even code rewrites for more idiomatic patterns in the code base itself. Think about open rewrite as one of the premier tools, right? And having that power from a static rewriting approach basically enhanced by agentic approaches that are ingrained into workflows. So that is kind of what really delivers a result that is trusted and makes it a lot easier to not just bump the platform and be gone because support problems went away, or at least support costs, but also make sure... that maybe the code is getting less because you can actually implement something more idiomatic, right? That is something like the modernization workflow, not just changing a version number in a POM file, but actually doing a rewriting plus agentic soothing out. That is something that I love with the new Bob Java modernization workflow. That is something that I haven't seen before. And it's one of the really unique points that will make sure that enterprises are able to just not bump simple versions for a lot of money, but also profit from the functional changes that are underlying these Java version bumps.
RD: Yeah, I know Java is doing smaller releases more often now, but the rewrite stuff is never what people want to be doing.
ME: At least not a plain version bump because there's no return on investment, like literally you're just trying to catch up with supports and CVE, and like, there's, there's no new feature, it's literally the same code base that we had in on Java 8 now running on Java 25. Okay, it's 30% more performant, Operations will thank you. And that was a number I just made up because I don't know the actual numbers. I just know that it's more performant.
RD: 70% of statistics are made up.
ME: Love that.
RD: But so with all this history behind Java, obviously that Java runtime isn't operating alone, right? There's all these frameworks supporting it, the Springs, Quarkus. How are these frameworks sort of making it easier for agentic Java on the enterprise?
ME: The first piece of the story is very similar to the core Java JVM story, the specification angle. I have to admit, I'm a really big proponent for specifications, even if many people think they slow things down, it's like more agreeing than it is actually solving problems, but specifications do solve one problem for agents they deliver the common domain language. So every agent does understand what a certain CDI bean actually is, what a certain annotation is, right? So it's kind of the, the domain language they can all rely on and frameworks who implement specifications like MicroProfile, for example, or the Jakarta EE specifications, they do implicitly profit from all of that. And then there's other advantages. We briefly scratched Spring. It's out there since almost forever. Rod Johnson basically created it as a better J2EE back in the days, and that was like 100 million years ago. Modern frameworks like Quarkus do also embrace agentic engineering in a completely different way. Instead of just only optimizing the framework itself, there is an upstream initiative that's called the Quarkus Agent MCP. So it's literally an MCP server, but it comes with a bunch of tools that gives the agent access to your running app on your machine, like in development mode, and complements everything with documentation search. So it has like a doc search tool. It has an application control endpoint. It has a bunch of packaged skills. And these get automatically delivered via MCP to the agent.
So the agent is basically now a Quarkus-enabled agent. Awesome. The results, it's literally day and night. If you just ask a random agent to build me a Quarkus application, and Quarkus is one of these more faster-moving and releasing upstream projects, it's hardly possible that any agent is up-to-date, right? So grounding and actual access to up-to-date information, that's kind of the key to build an up-to-date and latest Quarkus application in development. So absolutely perfect for agentic approaches.
RD: Yeah, that's an interesting segue because we talked about the framework, but you're talking about getting the framework in the harness. And how much does the harness matter for agentic development here?
ME: Oh, we're getting into a territory that is heavily discussed, I'd say.
RD: Right.
ME: So I am lucky enough that I had the opportunity to not only use IBM Bob extensively, but also enough other tools out there, out of curiosity mostly, because A, I wanted to keep my skills sharp, and I'm a hands-on guy, so I really need something to do and not just read all day long about what amazing new loops or skills somebody just spit out. I actually wanted to build my own. So the harnesses are, I'd say, even more important than the models these days. It really is the core knowledge that is in these harnesses, how they interact, how they help you shape context, how they give you transparency into what's happening, what tools are called, right? And it’s different schools, I'd say, that follow different approaches. And I don't want to like black and white and male everything between vibe coding and controlled experiences. But as I said, there's a spectrum. And depending on where you want to go and how fast and if it's a team or if you're just alone having fun, there's definitely a large difference in how you use these harnesses individual products.
RD: In terms of keeping Java version straight, like the modernization, making sure, you know, we haven't even talked about the regular enterprise issues of security and reliability. How can the harness shape those sort of concerns?
ME: To me, it comes down to workflows, right? Some people say, hey, install this skill and gives you everything you need. I don't believe this. There is a lot of wishful thinking going on right now. I mean, at the end of the day, it's a bunch of text that gets shoved into the LLM to shape the context. But if we look at what IBM does, for example, in Bob, when we're talking about the Java modernization workflows, we have one workflow that is scanning for vulnerabilities. It does not just ask the LLM, hey, how's this source code? And use my find vulnerability skill and do magic, right? No, it actually goes out to the CVE database, looks for comparisons, finds candidates, and gives you a full rundown of the vulnerabilities found in your source code. So that is an ingrained workflow that has dedicated steps that you can go through. And it knows exactly what to look for. So also super important if we're looking at larger code bases. So the agents can't just pick a certain random path down the package path, right? Because they decide to ignore the rest of the application. If you couple these two aspects, so doing static stuff on your developer machine, enhanced by the agentic elements, basically glued together in a workflow, this is really something different. And that is a harness thing, like full on, right? There's no other way to do this. You need to be able to deliver this inside as an add-on to the harness.
RD: We talked about a workflow. Is it sort of a package deal? Is it like a one-click button? Or, as we talked about earlier, remaining in the loop, is that something you can observe at every level, kind of massage?
ME: So the workflows are packaged. They adopt to your infrastructure, to your project. So if it's just like the J2SE project, or if it's a full-blown Jakarta EE project, or even based on Open Liberty or whatever you choose, and it offers different activities that you can do on top of that, right? But it's not something that you can customize. And to be honest... maybe that's not even necessary. A, these workflows do not only ship the steps, they also ship a lot of knowledge. There's a lot of thoughts from a lot of teams and a lot of years of experience that went into creating these specific workflows that customers basically can profit from. Is it necessary to customize them? Mostly not because we try to design them in a way that they do fulfill the dreams and wishes. So speaking of the JUnit or the testing workflow, right? So it basically looks at your application, figures out if the tests are available or not, writes a testing strategy, lets you edit the testing strategy and even adopt. And you can go as far as even add code coverage to it. So you can influence it in the important pieces. The actual execution is something that's ingrained in the workflow. And you know what's also interesting? If you watch the token counter during these workflow execution runs. Because as I said, some of this runs on your laptop machine that doesn't do a model round trip. So there's just some execution happening and only the results will be shared with the model. So the Java JUnit workflow can actually estimate migration efforts like front up if you start running it.
RD: So talking about these individual workflows, But what does a good workflow look like for a Java developer on an enterprise? What's the full snout-to-tail workflow?
ME: That is a good question. I wrote about it on my personal blog on the main thread. I think this is a very personal decision. Everybody has its own works and ideas how software is developed. And I'm not speaking about methodologies or what the company wants you to do. Just the way a developer develops stuff, right? I personally like starting with a specification in agentic systems. I'm not saying that I do a test-driven development or even BDD. I'm basically creating an idea, a vision of what I want to build. You can call it a plan, but it's not super formalized. It has a couple of specific elements and sections that I usually do want to be included. And that's mostly non-functional requirements. What I find very challenging with all these spec-driven approaches is that at some point, you end up with a very detailed specification. And you know what a very detailed specification is pivoting towards to?
RD: What?
ME: A software program.
RD: Right.
ME: It's like the more detailed a specification becomes, the closer you get to source code ultimately, right? So it's really something that we all are still learning how to do, but figuring out where to put what is interesting. I believe in source code. So as I said, I start with mostly the non-functional requirements, go into a scaffolding phase, go into a stability phase. I do extract the design of the modules and the specific elements from it, and also do put this aside as an architectural guidance. That's kind of how I started to develop my software projects. And this is comparably successful. I mean, it's not that I'm developing banking software right now. As a developer advocate, I usually develop a ton of examples, but I also have some quite complex ones out on my main thread where you could just go to the GitHub repository and clone them to just see what I mean. It's an interesting new time.
RD: Yeah, and I thought about this really detailed spec basically being writing software, it's like, well, yeah, spec-driven development is programming by other means, right? But you're doing it in a language that doesn't have a lot of specificity.
ME: Not a lot of type safety either. I mean, it's really... this is why I love Java so much, because it's so much easier to specify something in Java, right? And there have been occasions in the past where I just implemented the interfaces. and let my agents do the implementations. So just to give them a direction. Plenty of approaches, very personal. Yeah, following through on your workflow question. So that's kind of the software development piece, right? Testing is something that I do absolutely include in software development. And I have to admit, I hate software tests. I, I really am not a big JUnit fan, I've never been– like it's, I, I kind of understand why, but it's, it's always it always felt like I'm testing the wrong thing. I, I really, I could throw test coverage at it. I was never felt productive, right? It always felt like, sure, so that is where agents really do help me a lot. Why is that important? Because I can basically tell them that they don't touch the tests. So the tests are part of my guardrail that I'm implying on my software development. That is new, and it gives them a little bit more meaning than they used to have, at least for me personally. And I also have a ton of physical guardrails, so repository guardrails. ZemGrep is one of them. for example. And there's plenty more out there. So my process is literally from the very beginning specification all the way a slightly different way of creating the artifacts into a more structured and enforced way on the repository level to make sure to catch things very early, like secret leaks, for example. Git leaks is probably a must-have. And there's probably plenty of other tools that I use here and there in various combinations. But yeah, having not only the software creation process, but also the guardrails in place is kind of the secret to delivering something that is actually a real piece of software, not just a demo.
RD: Yeah. And so when I've used agents, it's fun to see the green numbers and the red numbers go up. When that's done and it's got 100 new lines, 50 that it's removed, how do you prevent that? How do you make sure you're not just getting this massive diff that's really hard to review?
ME: Surgical precision. I've said that before, and I didn't really explain it properly. I use Bob Modes, for example. So Bob lets you define individual agent personas. And I have one that is... I'm not sure I really called it surgeon, but something similar. That is a combination of not only a caveman skill, but there's been a funny story about a skill named Ponytail just a couple of days ago. And I have a skill that's called the Grumpy German. So there are a couple of variations.
RD: Is that redundant?
ME: Almost. A couple of variations that I'm using to try to limit the blast radios. And I think this is... besides the humans as a partner during software development, limiting blast radius is kind of the second biggest topic that I'm looking at when we're changing existing software. And that is the only way how developers actually stay in control.
RD: Yeah.
[music]
Well, it's that time of the show where we shout out somebody who came on to Stack Overflow, dropped some knowledge, shared some curiosity, and earned themselves a badge. So today we're shouting out the winner of a populist badge, somebody who dropped an answer that was so good it outscored the accepted answer. So congrats to Alex K for answering FFmpeg usage to encode a video to H.264 codec format. If you're curious about that, we'll have the answer for you in the show notes. I am Ryan Donovan. I edit the blog, host the podcast here at Stack Overflow. If you have topics to cover, comments. If you just want to say hi, email me at podcast@stackoverflow.com. And if you want to reach out to me directly, you can find me on LinkedIn.
ME: Hey, I'm Markus Eisele. I'm a developer advocate at IBM. If you want to try out Bob, you go to bob.ibm.com. And if you want to find me online, just look for MyFear.
RD: All right. Thank you for listening, everyone. And we'll talk to you next time.
[outro music]