Jarring Build Procedure

zelmak

So someone -- a long time ago, in a galaxy too damned close for comfort -- decided that the plugin architecture built-in to the existing code base was somehow wrong.

Supposedly the lead developer at the time wanted the plugin code to be part of the mainline code base. Therefore, anyone wanting to make a plugin, had to submit the code to him/her for inclusion in the main code base.

Meanwhile, the plugin developers thought ... rightfully so ... WTF ITS PLUGINS! NOT PART OF MAIN CODE! HERP DERP! (actually, I don't know what they said, directly, but I'm sure much herping and derping ensued.)

They, of course, wanted to work in their own sandbox, develop plugins for the system, hand those plugins off to the integrator, who would simply add those plugins to the Java classpath where they could be picked up by the JVM as it kicked off and everything was hunky-dory.

The junior programmer (there were only the two developers working on the main code base, you see) steps in with a (supposedly) clever compromise: I'll write a script that takes all the jar-file based plugins, unjar them into a single directory, then jar the whole kit-and-kaboodle up into a single plugins.jar file and we only have one thing to reference on the classpath, the integrator never has to change the classpath in the configuration and/or scripts, etc. And to support old plugins, we'll unjar the plugins.jar into that directory first, then unjar the individual plugins on top of those files so that ... no I really don't understand the reason for using the previously built plugins.jar instead of starting afresh. But yes, once built, old plugins.jar became the basis for the new plugins.jar with all of the other individual plugin jar files copied over top of it.

Apparently the lead programmer thought that this was a Really Good Idea(tm), signed off on it, and off they went. Fast-forward six years and now I have to deal with it.

Oh, by the by, everything ... EVERYTHING had to be in a single package ( /plugins/ ) and had to coexist there with every other plugin. So, if someone made a Foo class in their plugin, and so did someone else, they'd have a name collision and "the last one in" wins.

I've gotten my boss to accept the fact that this whole process is dumb and frought with peril. That we should dump all plugins into a plugins directory, point the classpath to that directory (or write a script that generates appropriate classpath based on the content of the directory) and be done with all this jarring and unjarring of libraries.

boog

@zelmak said:

EVERYTHING had to be in a single package ( /plugins/ ) and had to coexist there with every other plugin.

Err? What? Are you saying that you couldn't even put classes under, for example, "plugins.useless" or "plugins.crap"?

Ugh. I bet it's because they wanted to put something like [code]import plugins.*;[/code] at the top of every file because "it's just easier" than thinking about dependencies.

zelmak

Yep, precisely that.

I think, actually it was more about 'discovering' the plugin ... if you've got stuff hither and yon, it takes more work to discover package names in a .jar file and find a given plugin by name or something.

If you ask for a plugin named Bar, the plugin architecture (as if) simply tries to load/instantiate plugins.Bar (Class.forName("plugins.Bar")) through reflection. If you did it with multiple-nested packages/directories, you'd have load each by name, and check if it implements the plugin interface ...

Also, the build scripts for the plugins is .csh scripts ... working with multiple layered directores would be ... hard?

Honestly, I don't understand why things are the way they are ... a lot of bad decisions by C (not C++) coders new to an OOP language ... with fixes and features piled on for > 10 years by people who understood the code base even less than those who built it, trying not to break what's there while enhancing it with new features and bugfixes.

It's ... fun?

Daid

@zelmak said:

point the classpath to that directory (or write a script that generates appropriate classpath based on the content of the directory)

Aren't classloaders designed to get around that? (I'm not a Java guy)

TheRider

@Daid said:

@zelmak said:
point the classpath to that directory (or write a script that generates appropriate classpath based on the content of the directory)
Aren't classloaders designed to get around that? (I'm not a Java guy)

Not quite. The classpath tells the class loader where it might find the compiled code for any class it should load. Classloaders are more used to group classes and also to permit unloading of classes if they are no longer used.

ubersoldat

ah! lovely! Well, today you can add plugins/* to your cp. Also, you might want to take a look at OSGI if you try to rewrite that mess. Good luck! And thank your favorite deity those people didn't need an EAR.

zelmak

@ubersoldat said:

ah! lovely! Well, today you can add plugins/* to your cp. Also, you might want to take a look at OSGI if you try to rewrite that mess. Good luck! And thank your favorite deity those people didn't need an EAR.

Ah, if only ... after a bit of digging I discovered that each plugin is supposed to implement a 'result' object as well as a 'response' object (I still don't understand the difference) and these objects live in a completely different package path -- let's just say for the sake of argument, foo.bar.result and fizz.buzz.response. Yes, that's right, they actually live in the main code base instead of the plugin jar.

So, in essence, the lead programmer got his way -- everytime a plugin is added, the main code base has to change to support it. No, they didn't realize that a jar file can have that same package path in it so that the plugin's result and response objects COULD still be loaded from the plugin jar file. In fact, this is what I'm hoping to do to fix this issue.

I was doing all this to avoid rebuilding the main code ... but it looks like I'll have to in order to remove the plugin's result and response objects from main code base and have them be present only in the plugin's jar.

:sigh:

As an aside, the brilliant folks who originally assembled this system went to great pains to build a comprehensive (i.e., huge) classpath with everything that could possibly be needed (including the default rt.jar and other things from standard Java -- even though those are covered by the bootstrap classpath, but I digress.) As I said, great pains, many, many lines of code to build the classpath environment variable in the most modular way possible.

And then, every call to run a program looks like:

java -Xmx512m -cp ${CLASSPATH} foo.bar.runner.ProcessRunner FIZZ_BUZZ

That is, every script which calls a Java-based program PASSES THE ENVIRONMENT VARIABLE to the Java runtime.

Master_Chief

@boog said:

@zelmak said:
EVERYTHING had to be in a single package ( /plugins/ ) and had to coexist there with every other plugin.
Err? What? Are you saying that you couldn't even put classes under, for example, "plugins.useless" or "plugins.crap"?
Ugh. I bet it's because they wanted to put something like import plugins.*; at the top of every file because "it's just easier" than thinking about dependencies.

And memory usage. And startup time. And name collision. And every sane idea behind OOP.

boomzilla

@Master Chief said:

And name collision.

I think it's fair to call their solution to name collision BANG BANG.

boog

@Master Chief said:

@boog said:
Ugh. I bet it's because they wanted to put something like import plugins.*; at the top of every file because "it's just easier" than thinking about dependencies.
And memory usage. And startup time. And name collision. And every sane idea behind OOP.

Well, they probably weren't thinking about those things to begin with. Whereas putting everything in one package to "cut back" on import statements was probably considered a [i]brilliant[/i] solution to a [i]truly[/i] non-trivial problem, as explained here:

"You mean I have to import every class that I'm using?"

"Yes, that's how programming works."

"Aw, lame! Ok, I'll just put them all in one package and include that."

zelmak

@boog said:

@Master Chief said:
@boog said:
Ugh. I bet it's because they wanted to put something like import plugins.*; at the top of every file because "it's just easier" than thinking about dependencies.
And memory usage. And startup time. And name collision. And every sane idea behind OOP.
Well, they probably weren't thinking about those things to begin with. Whereas putting everything in one package to "cut back" on import statements was probably considered a brilliant solution to a truly non-trivial problem, as explained here:

"You mean I have to import every class that I'm using?"

"Yes, that's how programming works."

"Aw, lame! Ok, I'll just put them all in one package and include that."

But ... these are plugins ... and therefore, wouldn't (oh, shit ... I'd better look -- SHOULDN'T) even be in an import statement.

boog

@zelmak said:

But ... these are plugins ... and therefore, wouldn't (oh, shit ... I'd better look -- SHOULDN'T) even be in an import statement.

True, that is TRWTF. My comment was focusing on the lesser WTF of putting everything in the same package, but I absolutely didn't intend to draw any attention away from the real problem of using includes to handle plugins.

boomzilla

@boog said:

@zelmak said:
But ... these are plugins ... and therefore, wouldn't (oh, shit ... I'd better look -- SHOULDN'T) even be in an import statement.

True, that is TRWTF

No, I'm fairly certain TRWTF is using csh instead of zsh.

zelmak

@boomzilla said:

@boog said:
@zelmak said:
But ... these are plugins ... and therefore, wouldn't (oh, shit ... I'd better look -- SHOULDN'T) even be in an import statement.
True, that is TRWTF
No, I'm fairly certain TRWTF is using csh instead of zsh.

:twitch: ... don't get me started on THAT conversation ...

boog

@boomzilla said:

No, I'm fairly certain TRWTF is using csh

Eww, I missed that part. Eww.

Sutherlands

Explanation on CSH/ZSH?

Xyro

@Sutherlands said:

Explanation on CSH/ZSH?

CSH is like the VB of Unix shells.

Their syntaxes suck, they're ... "special" in ways that don't make them better, and there are much superior alternatives sitting right beside them.

I would suggest creating a plugin that implements sane plugin management.

Hatshepsut

@Zecc said:

I would suggest creating a plugin that implements sane plugin management.

Yo d...

zelmak

@Zecc said:

I would suggest creating a plugin that implements sane plugin management.

plugception?