Antumbra, Comerge and Eclipse

RCP Update: No more "jammern" with InstallJammer

2010-01-28T08:25:00.000-08:00

Wow, I got a lot of feedback because of my last P2 bashing blog post. A guy from compeople even called me the other day, telling me they had the same problem and that they can provide a ready to use update mechanism. Not for free unfortunately. But it might be worth looking into it at some point. He also told me, that they are contributing to the eclipse Riena project which looks promising, have to look into this when I find some time.
I understand, that the situation is frustrating for the P2 team and that I better should help resolving the issues instead of complaining. But please also understand, that our customer doesn't care and I had to work through the weekend to write my own update mechanism.
Well the good news is that the InstallJammer approach works so far and even better: Comerge agreed to open source the update code under a new project called Refresh. In this blog I want to explain how Refresh works.

How to update an RCP with Refresh

Refresh connects to a set of updatesites and collects all available updates. An updatsite is an xml document hosted on a web server. The site contains a set of updates. An update is described by a product id, a version and a location of an executable. The updater can then download the executable and execute it. The executable is expected to update the client and then restart the client. We use InstallJammer to generate this executable. In the first part I describe how to program the updater in the second part I explain how to generate the executable.

Programming the updater

First you need to collect the updatesites:


HttpUpdateSitesReader reader= new HttpUpdateSitesReader(new File("updatesites.txt"));
List<UpdateSite> sites= reader.getSites();

The updatesites.txt can contain as many updatsites as you want:


http://foobar.com/maintenance/updatesite.xml
http://dev.foobar.com/milestones/updatesite.xml

I recommend to place this file next to the executable of your RCP. The user can then easily add new sites.

The updater will then connect to all the sites and collect all updates for a given product:

UpdateSearcher searcher= new UpdateSearcher("foo.bar.product", sites.toArray(new UpdateSite[sites.size()]));
List<Update> allUpdates= searcher.searchUpdates(new NullProgressMonitor());

The updatesite is a simple xml file like this:

<?xml version="1.0" encoding="UTF-8"?>
<updatesite>
   <update>
      <product>foo.bar.product</product>
      <version>1.0.1</version>
      <resource>http://foobar.com/update101.exe</resource>
   </update>
   <update>
      <product>foo.bar.product</product>
      <version>0.2.0</version>
      <resource>http://dev.foobar.com/update020.exe</resource>
   </update>
</updatesite>

You then need to determine if there is an update available:

List<Update> updates= EclipseVersionSchemaHelpers.getNewerVersions("0.1.0", allUpdates);
if (updates.size() == 0)
 //no update available
 return;

Update update= EclipseVersionSchemaHelpers.getNewestVersion(updates);

The code assumes, that you use the standard eclipse versioning schema but using your own is straight forward.

If there is an update available all what's left to do is to download the update and restart the workbench:

if (update.install(".", new NullProgressMonitor())) {
 PlatformUI.getWorkbench().close();
}

Of course you should use a ProgressDialog here and there and provide some useful error reporting. But you get the idea. The harder part is to generate the executable to update your client.

Generate InstallJammer updater

You can use the InstallJammer configuration GUI to define your updater. Defining an updater is quite simple, I only found two things which are not that obvious:
1. The default update type must be set to silence, otherwise the user will see the installer UI and has to press finish at the end. I was not yet able to show a progress bar only when executing the installer.

2. The folders which the installer is reading must be marked as active. Otherwise the content of the folders will be fixed and new files will not be picked up when generating the installer.

Once you have the installer defined you can use a simple bash script to generate the installer. My hudson build executes this script after the PDE build has finished:

mkdir hudson-jobs/foobar/workspace/pluginbuilder/installerworkspace
cd hudson-jobs/foobar/workspace/pluginbuilder/installerworkspace
cp ../results/foobar-*.zip foobar.zip
unzip foobar.zip
./installjammer -DAppName foobarupdate --build-for-release --output-dir ../results/ --build ../../rcp/foo.bar.installer/FooBarUpdate.mpi
mv ../results/foobarupdate--Setup.exe ../results/foobar-update.exe

That's it. I use the same script to generate an installer to install the RCP initially.

Things to do

All this is very simple, a bit too simple at the moment. There are certain must haves:
1. Show a progress while the update installer is running
2. Allow the installer to authenticate. Otherwise updates are available for everyone.
3. Make it work on other platforms

If you want to contribute or use the code go to the project page. The code is licensed under EPL.

Thanks

P2 or why I can't sleep anymore

2010-01-15T09:23:00.000-08:00

In the next one and a half year we are going to build a huge RCP for Switzerlands leading car sharer. They insisted to build a web frontend. We said that there is no way we are going to implement the beast they want in a web technology. They agreed under the condition that our rich client is able to update itself once deployed. I thought no problem: there is this new fancy updated mechanism called P2. This was about the time when I've started to worry and stopped sleeping.

Adding P2 self update to your RCP

There is a nice tutorial explaining how to convert your existing RCP to become a self updating P2 enabled RCP here: http://wiki.eclipse.org/Equinox/p2/Adding_Self-Update_to_an_RCP_Application.

First I had to change my hudson build to build a P2 enabled RCP. We use the plugin builder to configure the PDE headless build scripts. Even with the plugin builder it is far from trivial to set up a headless build. But once you get used to it it works quite well. Unfortunately the plugin builder doesn't seem to support P2 enabled builds. So I had to mess a bit around with the ant scripts. But after a day of swearing and sweating I was able to get it to work. Ok, the build now takes about twice as long as before but who cares?

Next step was to add the P2 dependencies to the product, by adding the p2.user.ui feature. This worked quite well. The RCP now is only like 30 Megabytes larger but who cares?

Next step was to add the check for update action as described in the above tutorial. This action is only about 100 lines long and uses almost exclusively internal P2 code plus it doesn't work out of the box because of this bug. But who cares?!

Now, I've increased the version number of the product, a feature and a plugin. Then I've add the location of the P2 repository which my brave little hudson builder crated as a touchpoint to the p2.inf file. For a start I've chosen to use a local repository located somewhere in my file system. All what's left is to start the old RCP and off we go with the automatic update! Yeah, whatever. It didn't work. Nothing, no update, no exception, no nothing. I've started to care!

I've played around with it, several hours, googled, read forums, read the p2 equinox wiki. I've added the P2 update UI to my RCP and realized that it actually found the update site but did refuse to update. Then I made an interesting discovery: If I created the p2 repository by exporting the product with my local eclipse install and used this repository it worked!

IT WORKED!!!

I assume something, no idea what, is wrong with my headless build. I thought ok, I can life with that for now: I only have to export the site every now and then. All what's left is to add the repository to some web server and point a touchpoint to the repository URL. Guess what: Didn't work! The exact same repository: If located in the file system it works, if added to a web server it doesn't. What's this good for!?

No, it doesn't work!

I had a tough night, got up at 4 o'clock in the morning and couldn't sleep no more. It was time to visit our CTO.

The CTO

Our CTO is a unix hacker, he likes scripts and configuration files, he thinks that a console is a more then sufficient user interface (they even have copy paste support nowadays you know). He doesn't like frameworks and of course he hates Java. He said: Fuck that shit! You want to update your RCP? Just make your RCP download a self extracting zip file, close the RCP, execute the self extracting zip file in the install folder and start the RCP again: Voila! Know what: This sounds like a hack but there is a golden rule when writing software:
GET THINGS DONE!
And his approach does the Job. We can even update the JRE which we ship with the RCP (not like P2 that tries to delete the JRE while the JRE is still used which of course doesn't work on windows).

But self extracting zips have some drawbacks:

We couldn't find an easy way to generate them for windows with our linux build server
You can only add files and not delete files.
They only work on windows (not a problem for us though)

We found this nice open source project: InstallJammer. With InstallJammer you can:

build windows installers with linux.
execute various actions on install, including deleting files
run the installer in silent mode such that the user doesn't realize that what is executed is actually an installer
start your program after the installer has finished
build installers for windows, linux and they say osx support is coming soon

The next night I've slept like a baby.

Conclusion

Here is my advice: If you need to update an RCP which you control, which can not be extended by users, then do not get into the P2 businesses. That's my opinion. I know it is possible to use P2 for this and I know there are people which used it successfully. But I like solutions which I can control and understand, and P2 I can not.

In a next post I will explain a bit more in detail how we did it with the installer approach.

Tom Schindl talks about E4 in Zuerich

2010-01-13T00:41:00.000-08:00

Learning about E4, the next generation of the eclipse platform, is on my (low priority) list for quite a while now. If you are located near Zurich: the Java User Group Switzerland will host a talk about e4 given by Tom Schindl tomorrow evening:

http://www.jugs.ch/html/events/2010/e4.html

I'll be there and hope Toms talk will give me an E4 kick start. Please be aware, that you need to register for the talk.

Also on my list for quite a while now is trying out following stuff:

I'm excited to learn more about E4.

Running Open Architecture Ware workflows on save

2010-01-08T06:07:00.000-08:00

We use open architecture ware for Aranea. With Aranea you can use a domain specific language (dsl) to model messages which can be send between Aranea nodes (computers). For example a message to send a status could look something like this:

[project AraneaDemo prefix=A_] { 
 [namespace Core] {
  [message Status] {
   properties={
    message: string notVoid
   }
  }
 }
}

Open Architecture Ware is then used to interpret the dsl, generate a model out of it, and then generate Java and Eiffel code out of this model. The Java class for the message above looks something like this:

package aranea.demo.core;
public class StatusMessage extends AraneaMessage {
public final String message;
/**
* Constructs a <code>Status</code> message without reply handler.
*
*/
public StatusMessage(String message) {
this(message, null);
}
}

Parsed in 0.093 seconds at 2.88 KB/s

A so called worklflow is used to script how to convert the dsl file.

The past

Until now it was required to invoke the workflow by hand, that is: right click and select 'run oaw workflow' from the context menu. That was not very cool and sometimes we forgot to regenerate the classes after changing the dsl. I thought a better solution would be to use a builder.

Ant builder

Did you know that eclipse can run any ant script on build? You can add an ant builder to your projects and define when to execute the ant scripts. Right click your project then open Properties and open the Builder property page:

Adding a new builder is trivial. The hard part was to write the ant script such that it can be executed. After a few days of trial and error I was able to create a script which can run any OAW workflow. The files are below and licensed under MIT license. I hope this is of use to someone. If you want to play with it read trough this site: http://aranea.origo.ethz.ch/wiki/how_to_create_an_aranea_message_project

<project name="project" default="main" basedir=".">
<import file="runOAW.xml"/>
<target name="main" description="the main entry point">
<antcall target="buildJava"/>
<antcall target="buildEiffel"/>
</target>
<target name="buildJava" description="build the java messages">
<property name="workflowLocation" value="${basedir}/src"/>
<property name="workflowFile" value="GenerateJavaMessages.oaw"/>
<antcall target="RunOaW.generate" />
</target>
<target name="buildEiffel" description="build the java messages">
<property name="workflowLocation" value="${basedir}/src"/>
<property name="workflowFile" value="GenerateEiffelMessages.oaw"/>
<antcall target="RunOaW.generate" />
</target>
</project>
<project name="RunOaW" default="generate" basedir=".">
<property name="eclipse" value="${eclipse.pdebuild.home}/../.."/>
<property name="plugins" value="${eclipse}/plugins"/>
<path id="deps.classpath">
<fileset dir="${plugins}/">
<include name="org.openarchitectureware.*.jar"/>
<include name="org.apache.commons.*.jar"/>
<include name="org.eclipse.*.jar"/>
</fileset>
<fileset dir="${plugins}/org.antlr_3.0.0">
<include name="*.jar"/>
</fileset>
<fileset dir="${plugins}/org.antlr.runtime_3.0.0">
<include name="*.jar"/>
</fileset>
<pathelement path="${workflowLocation}" />
<pathelement path="${basedir}/../araneaMessage.dsl.generator/bin"/>
<pathelement path="${basedir}/../araneaMessage.dsl/bin"/>
<fileset dir="${plugins}/">
<include name="araneaMessage.dsl*.jar" />
</fileset>
</path>
<target name="generate">
<taskdef name="workflow" classpathref="deps.classpath" classname="org.openarchitectureware.workflow.ant.WorkflowAntTask" >
</taskdef>
<workflow file="${workflowFile}">
<classpath>
<path refid="deps.classpath" />
</classpath>
</workflow>
</target>
</project>

Parsed in 0.016 seconds at 126.32 KB/s

This form post helped me a lot to get this script right.

Hello World

2010-01-08T02:07:00.000-08:00

Hello World

Finally: Yet Another Eclipse Blog! I will try this time to not let this blog become yet another zombi blog. I've use to blog about the Java Development Tooling and Platform Text back in the days when I worked as an active eclipse committer at IBM and I actually enjoyed writing blogs. But writing good blogs is time consuming and with customers on your back and milestones ahead writing blogs is never a priority. But this time everything is going to be different, promised;-)

In this blog I'd like to write about the stuff we do with Eclipse here at Comerge, the company I work for. At the moment we use Eclipse as a tool to develop Java code (what else:-) . We create GWT frontends and use the JDT to code that stuff. We also use the Eclipse Rich Client platforms to build applications for our customers. We also maintain and develop Aranea and its Java port. We are using Open Architecture Ware there. Then we also develop and maintain Origo and its Mylyn Origo connector. And in the spare time which I unfortunately don't have anymore I also work on another RCP: The Evolution Player.

I hope this will give me enough topics to blog about. Of course I'll try to blog only about stuff which is of interest to most of you.

Benno