the tale of two plugins, or: the tale of one plugin from two providers

I work on an Eclipse plugin called CFEclipse.

CFEclipse has a built-in unit testing plugin, but it's old and pales in comparison to another available unit testing plugin called MXUnit.

So I wanted to replace the built-in plugin with the MXUnit plugin.

I needed to bundle MXUnit with CFEclipse, and still allow people to update from MXUnit if a newer version came out before I could bundle it with CFEclipse, however.

I thought that adding a "discovery" child to the "url" node (pointing at the MXUnit update site) as well as adding an "includes" node to the feature.xml would get me there:

<url>
<update label="CFEclipse Update Site" url="http://www.cfeclipse.org/update"/>
<discovery label="CFEclipse discovery site" url="http://cfeclipse.org/update"/>
<discovery label="MXUnit discovery site" url="http://mxunit.org/update"/>
</url>

<includes
id="org.mxunit.eclipseplugin.feature"
version="0.0.0"
search-location="both"
/>

I figured thusly because that's what I could find searching the web using terms like: "including 3rd party plugins" or "using two update sites" and whatnot.

The only problem was, this seemed to lock anyone who installed the bundled CFEclipse + MXUnit into whatever version of MXUnit we bundled (specifying 0.0.0.0 as a version number didn't help).

I figured if I could specify a revision range for MXUnit, that should "unlock" updates from the main MXUnit update site.

Unfortunately, looking through the docs, it appeared you could specify revision ranges for dependencies, but not for plugins you include in the feature.xml.

Back to the search-board!

The magic search string was: "feature.xml includes version ranges"

Which led me to this:

http://www.eclipse.org/forums/index.php?t=msg&&th=157425&goto=513692

Which had the following info:

There are two approaches, the first is to change the feature to require the plug-in instead of include it. This gets you the old update style ranges of equivalent[1.2.3, 1.3.0), compatible[1.2.3, 2.0.0), or greaterOrEqual. However, PDE/Build currently only builds included plug-ins, not required ones. So you would need to build the bundles separately and then have them (and their metadata) available to the product build.

The other and perhaps simpler option is for the feature to have a p2.inf beside the feature.xml. The p2.inf can add requirements to the feature, and they replace the generated requirements that came from included plug-ins if the name and namespace are the same.

feature: include org.eclipse.foo 0.0.0 p2.inf: requires.1.namespace = org.eclipse.equinox.p2.iu requires.1.name = org.eclipse.foo requires.1.range = [1.0.0, 2.0.0)

If you are changing the requirement ranges for include features, remember that the name for a feature's IU has ".feature.group" appended to it.

http://wiki.eclipse.org/Equinox/p2/Customizing_Metadata

Ah ha! I opted for the p2.inf approach, although I think the former might be better if you have to support pre Eclipse 3.3 plugins. Option two was definitely the easier of the two, too. :)

After adding the p2.inf, things worked like I'd wanted, where the update to MXUnit can come from either the main MXUnit update site or the CFEclipse update site.

This decoupling allows the end users to choose when and where to get their MXUnit updates, vs. forcing them into using whichever version CFEclipse ships with.

Score!

Ironic that it took a few hours of dicking around with stuff and searching to find a 3 line solution, but that's Eclipse development. Things that are simple in retrospect can be rather difficult to unearth initially.

I think this is a result of the size of the Eclipse codebase, and the constantly changing nature of it- neither of which would I trade in exchange for easier docs.

If Eclipse was closed source, I'd sing a different tune.

And targeting a different platform.

Reverse engineering the DLTK Python plugin example

Now that you've got Eclipse [link to installing eclipse be], I bet you wanna start kicking some ass, right?

Maybe you've been kicking some ass already, and just wish that you could kick a little more. Specifically, "if that thing would only do what you want it to do".

To that end, we're going to focus on the tools you use to do what you do, and in this case, Eclipse plugins. Yay!

Anyways, I'm enamored of some of the features in some of the editor plugins. Really swell stuff.

I use CFEclipse [link] for editing coldfusion files. It's pretty swell, but I wanted to make it sweller. Better than bee's knees. Or at least have occurrence marking capabilities.

So I took a stab at it, having taken a stab at a couple other plugins, and, as with the other plugins, saw ways to do things different. Not exactly originality, as I got the ideas from other places with similar constraints (like when trying to add key-binding to Subclipse, and looking to the CVS plugin for implementation ideas).

Anyways, a lot of this stuff is about abstraction, oddly enough. Instead of coding 5 specific bits of code, write one general one-- or, true-er; one fragile implementation of something core can become a lot of "work around" coding, sorta.

Anyhow, the less code the better, and CFE looked like there was a good bit of code we could get rid of.

I'm leaning towards getting rid of a lot, like a lot a lot, and relying on other plugins for some stuff we currently handle ourselves, but that's another story.

After looking around, I was liking DLTK for a base IDE type deal, and Mark Drew provided a little starting point-- He'd walked through part of the DLTK IDE how-to walk-thru, and put it in the source repository for people to play with.

Since the code for the example python IDE was available (and even though the real Python DLTK plugin is out now, and I could have used it, or a language really really close to CF for a basis, perhaps) I downloaded it, and sorta strolled around.

Mark Mandel had done some work on an ANTLR coldfusion parser, and the Python plugin used a ANTLR parser, so "I was like, hmmm... I wonder if I can just plug this other parser in?"

And because Eclipse JDT is so awesome, I was able to find out pretty quick. Ha ha! (4hrs+ quick)

What I did was, look for patterns, basically. (Good reason to use something nice and specific when naming stuff-- "PythonExampleParser" was pretty easy to refactor to "CFMLParser", roughly.)

Eclipse would ask me "rename all this crap, dude?" and since it was just the example, I was like, "sure".

Before I did that tho, I'd consolidated the 4 example plugins into two plugins-- a "core" and a "ui". I did that by just dragging the packages from one plugin to the other, and copy/paste the various bits of XML from each plugin into the respective plugin.

The idea was to keep the UI (eclipse presentation type stuff-- font colors or whatnot) apart from the core (I've since gone further, and created a plugin for the CF parser, but for dicking around, I could have just copied them all into one plugin, really-- compartmentalizing stuff was kinda an exercise, sorta.).

Annywho, after I consolidated the 4 plugins to 2, I then refactored all the "PythonExample"s to "CFML"s, which was pretty easy since JDT is so "aware".

Then I fired the plugin up in my usual manner: I'm editing the plugin.xml file for the plugin (any one-- they all fire up the same runtime application) with the Plugin Manifest Editor (right-click, "open with", if you got it but it's not a default-- probably comes with PDE, which you'd have to have by now to do any of this anyways), and on the "overview" screen, there's a "Testing" section, under which there's a nice little "Launch an Eclipse application in Debug mode".

I click on that link, and Eclipse fires up a test instance using all the open plugin projects. That's totally configurable, BTW-- under "Run" (top of screen) there's "Run Configurations" where you can configure the Eclipse Application instance that your code runs in-- use this to disable the loading of any plugins that you don't want running in that test/debug instance.

As the Python parser was still parsing, I was like "cool, now to try shoving the CF parser in"-- I was actually aiming for a quick switch, but it wasn't that pretty. Heh.

So I dropped in the CFMLParser, basically. Of course it didn't have the same kind of AST as the python parser (abstract syntax tree, apparently used to map parsed things to other things), so simply using the python stuff was out-- and the python stuff had all the DLTK stuff, for the outline view and whatnot. Sorta. Hell, look at the code [link], you'll see what I mean. Most of the DLTK stuff's in the parser, actually. Well, grammer file, I guess, technically. It's that grammer file that builds the parser and lexer.

Well now, this is getting long...

Luckily, I think that's pretty much it. This is piss poor, as far as a learning aid, I reckon, but maybe with a little editing or something... eh.

So, DLTK is an dynamic language IDE API, basically. It's got some generic-ish stuff you can use to do things like populate the outline view, create custom searches, and apparently mylyn integration (although I haven't explored that aspect yet), etc..

That generic-ish stuff was in the python parser itself (at least some of it), so to get the CF parser to talk to DLTK I think that's what we've got to do-- define functions components and whatnot, map them to the DLTK equivalents.

It really doesn't matter what happens now... I'm just amazed that things could work that simply. By "work", I mean that when I fired up that test instance, and opened a .cfm file with the "CFML Example Editor" *grin*, I saw output from the parser in the console of my main eclipse instance. "entering cfif, leaving cfif". Did a little more refactoring, put in the more abstract cf parser, that uses dictionary files... "isColdfusion: true" started coming out...

Eh. Maybe I should just do a screencast of the basic deal.

And I mean "doesn't matter" because in a few days, I've sorta kinda picked up just an amazing chunk of knowledge. When I first looked at that CFML.g file, or the python_v3.g parser grammer file... wow. I was like? WTF? Ya know?

Then it turns out it's like sorta, maybe, regular expression chaining or something. (Rough analogy, for sure). There's nice GUIs that make it obvious why X does Y or whatever.

Now I'm off onto another entry tho. Guess I'll cut this out, right about...

Installing Eclipse

So you want to work on open source projects, or maybe do some "work" with a kick ass IDE. Here's how I go about doing it!

First, I download the latest Eclipse version. If we're at, um, around an M4 release, I'll generally opt for that. I like to live on the edge, and really, these days, it's where it's at.

Everything is evolving really fast. I love it, although it probably pisses some people off. Eh.

Anyways: I like the package builds-- what I do most of the time is related to the interweb, so I go with the Java EE build, which comes with most everything I need. Mylyn, the Web whatnots, Data Tools... great stuff!

So, anyways, I download (http://phoenix.eclipse.org/packages/, currently) the Eclipse install I want, and then I fire it up, and add any plugins that don't come with it. For me that's a custom compiled "bleeding edge" version of Subclipse, and CFEclipse. That's it, these days. Oh, and Remote Target Management (for the Remote System Explorer, which does SFTP, FTP, Shells, and other cool things). Used to do QuantumDB, JSEclipse, SFTP...

I point it at a temporary workspace first, while installing plugins, and then I switch it to my "main" workspace (I've recently started using multiple workspaces-- it's easy, and keeps stuff faaaast!).

Here is the command I use to start up eclipse with some memory and aggressive garbage collection (yes, you can tune Eclipse just like CF, for what you're doing):

./eclipse3.4M6/eclipse -data Documents/workspace-cfe -clean -vmargs -Xverify:none -XX:+UseParallelGC -XX:PermSize=64M -XX:MaxNewSize=64M -XX:NewSize=64M -Xmx512m -Xms512m

docs about GC: http://wiki.jboss.org/wiki/TuneVMGarbageCollection

I'll generally have one eclipse directory, with the various versions within, and use a symlink or virtual folder to point at my "default" install, and various scripts for starting up specific workspaces. That line above starts up my CFE specific workspace.

Generally, you'll only start to need multiple workspaces if you're working with upwards of 30 or so /heavy duty/ projects. Up until recently, I probably had like 30 little website projects, 9 honking website applications, and the sources for like 20 open source projects. Mylyn makes stuff manageable, as do working sets.

Things were fine, which is fucking super impressive, you know? Seriously, I'd even leave them all open!

Every once in a while crap goes down tho, and a full workspace rebuild took minutes. Plus, searching /everything/ was a PITA. Just freaking oodles of files. Oodles.

So now I've got stuff broke up better. As you'll see in the next entry, where I talk about how I'm working on a DLTK version of CFE.

Mostly this is because, as I said earlier, I'm a mix-master. I'll check out the sources for 5 or 6 related projects while working on one project, since, to be honest, I'm standing on the shoulders of giants anyways.

See, I don't really know /why/ what I'm doing does what it does-- sure, I'm sorta do, perhaps, and it gets fleshed out as I go, but, seriously, I'm standing on the shoulders of giants, as mentioned previously.

Years of playing with random stuff gives me a certain spidy-sense, I reckon, but there is a lot that is a mystery to me, yet that I take advantage of anyways.

Bah. Talking eclipse install here, right? I'll save that other stuff for "how I guess at what to do" or a similarly titled next post.

So, with the new 3.4 update manager, equinox p2, having bundles of plugins is easy. I used to do the whole: create eclipse plugin extension folders for various things, to keep say, eclipse3.3-only plugins in one folder, and eclipse 3.2 and 3.3 plugins in another, or WTP in one, etc.

I stopped doing that a while ago tho... just got things to such a smooth process, I guess, that it was more trouble than it was worth.

Feel free to look into it tho, if you like to try out various plugins without having to hunt through the default giant eclipse plugin folder (for manual un-install/install, mostly).

If you're running a mac, the command line command I posted above will work. You need to edit (I think) a plist file if you want to be able to just click on the icon and have configuration settings work (the .ini file doesn't work on mac).

Since I'm running a macbook now, I created a ec.app folder, with Contents/MacOS/ folders within, and created a script called ec, which contained the command line stuff, so I can click on the icon... pretty retarded, really, because you can just click on script files too (if you tell Finder to open them with the Terminal app), but hey...

[note to self: see if a one-liner like:] echo thatStatCommand > ec.app/Contents/MacOS/ec chmod +x ec.app/Contents/MacOS/ec [would work]

anywaze... wow... guess that's about it, unless folks think of more eclipse questions, or I think of some more tips... look into mylyn, for sure, peoples... yup... that... is.. all... for... no-ow ow ow ow owww w w.

compiling subversion from trunk on os x for subclipse

First get the subclipse plugin sources, and compile them, using you current SVN version. You can do it pretty easy by using the team project set [add link] and then opening the update site with the manifest editor and selecting "build all".

Pretty easy, really. Just follow the INSTALL directions in trunk (grab the APR sources, etc.).

Just don't forget to enable javahl!

Here's my config string:

./configure --prefix=/usr/local --with-openssl --with-ssl --with-zlib=/usr/include/ --with-neon=/usr/local --enable-javahl

If you didn't install neon from source, you can try the included version (although I think it's too old, even with OS X 10.5):

./configure --prefix=/usr/local --with-openssl --with-ssl --with-zlib=/usr/include/ --with-neon=/usr --enable-javahl

Then do: make make javahl make install make install-javahl

If everything went well, you can start eclipse, and update your subclipse plugin using the local update site you built earlier.

Restart and enjoy subclipse bleeding edge with SVN 1.5 bleeding edge!

Using the TCP/IP Monitor in Eclipse 3.3

The tcp/ip monitor in Eclipse 3.3 is quite useful. I'm going to give a use case scenario, where the monitor made adapting my app connecting to another a snap.

I love FCKEditor for doing wysiwyg type HTML stuff. It's got a built-in file browser, for browsing, um, files.

I wanted to connect an fckeditor instance's file browser to my subversion repository service.

The nice thing is that the default fck file browser just issues requests for XML. There are actually examples of what it sends, and expects to get back, on the fckeditor site, but I love real world, variable testing, so figured, hey, lets just see what goes back and forth.

To do that, I utilized the quite nifty TCP/IP monitor, found in Eclipse 3.3 with, I think, the Server Tools plugin. If someone knows where it comes from, holler. For me, it's under Window > Show View > Debug > TCP/IP Monitor.

Right-clicking on the view and choosing "properties" (or going thru Window > Preferences > Run/Debug > TCP/IP Monitor) will let you define a new monitor.

It will ask for the local port, meaning, which port to you want to debug through. I choose 81, and for host, I enter localhost, since I've got FCKEditor running on local site. For port, I use 80, since that's the port my local apache install is listening on.

If I save this monitor, and then start it, I've got a proxy right within eclipse, that I can use to view all the traffic between my web browser, and my application.

So, I point my browser at http://localhost:81/, fire up a page that has an fckeditor instance on it, click the "browse server" button, and get a nice, live overview of requests and responses. I can even wank around with the requests and whatnot. Sweet!

Now I can create my cfsvn repository file browser connector, by responding to fckeditor's filebrowser with the XML it expects.

I used this same technique to create an interface to my issue tracking application for eclipse's mylyn, for trac's xmlrpc task repository connector.

I can now view the tasks in my issue application within eclipse, and use the rich task editor and whatnot- including mylyn contexts! And since I had a cf based svn browser, I have the auto-link to the source code from within task descriptions working y todo.

I'll have to write up another post about using the TCP/IP Monitor and the TPTP http request recording/playback stuff to run nifty tests and whatnot.

BlogCFC was created by Raymond Camden. This blog is running version 5.9.3.000. Contact Blog Owner