Wednesday, 3 November 2010

OS X and Java Support

Apple have announced that Java will be deprecated on the next major OS X release, I think we can safely read that as basically broken. Will they give out their JVM source code for the community to continue with? Of course they won't. The reason they don't want Java applications on the Mac is they perceive the average Java application as having a poor/inconsistent UI compared to the Cocoa/Quartz standard. Java applications are also slower and more memory intensive than native applications. The community JVM will only be of worse quality than the Apple offering (unless some companies with heavy Mac/Java application investment step in). Java zealots can argue the difference can be reduced to a reasonably small amount, but the simple fact is that the average developer isn't skilled enough/lacks the time to achieve this. In essence, yes, I agree with Apple. Why should they have to maintain a technology that dilutes their  application quality, when overall quality in one of their unique selling points.

Are we going to rush off and create an objective C library component with exactly the same API as JGraphX so you can quickly recreate your application in Objective C? No. For one, we're not going to panic every time a company changes its mind and there's quite a lot that going on at the moment. We share Microsoft's view that HTML 5 canvas is going to be the lowest common denominator as a graphical platform and intend to build enough custom functionality on top of it, that for a hardware accelerated canvas you feel you could be working on a native application.

Is that not helpful to the Java developers (and only 3 people follow this blog to date, and I believe they are all Java developers)? Not really, we're continuing to support Java fully. We provide a very widely supported platform in the JavaScript mxGraph client. It's not free, like the Java client is, but the application written using JGraphX is only free if your time is worth nothing. The Java platform will remain a good way to prototype for the JavaScript client, having the same API. For academic and FOSS projects, we will be looking at some way to allow mxGraph to be freely used (free as in beer).

Ultimately, we feel that diagramming should be thin client with the server sitting virtualized in a cloud, be that public or private. I know these buzz words have been around for many years and never quite materialized, but it will arrive, one day.

Mobile Device Support, Native or HTML 5 Canvas?

All the rage now currently is support for mobile device, phones and tablets specifically. Is it the next big thing that we should all develop for? Do you need your workflow editor to work on a 4 inch touchscreen? I doubt it. Sure there are plenty of applications that make sense on smartphones, interactive diagramming most likely isn't one of them. Viewing diagrams, panning through them, maybe selecting cells, possibly. But this is possible using an image map and the processing on the server side.

Tablets are another matter. 7-10 inch screens make interaction possible, in fact, the whole process of diagram creation should be fundamentally different. But you'll see what I mean by that in the touchscreen examples we'll be releasing next year.

The question we're hearing a lot at the moment is are we going to support mobile platforms natively (iOS, Andriod, Symbian, Blackberry OS and Windows Mobile), or are we going to aim at getting mxGraph running on HTML 5 canvas and just have one implementation?

The second option would seem to be the obvious one, however, browsers implement things very differently. What looks like one JavaScript implementation in mxGraph is actually a heavyweight effort to smooth over the differences between browser under the hood of the mxGraph API. HTML 5 canvas is no different in this regard, the implementations vary wildly and the functionality available in HTML 5 canvas is really quite limited, especially when it comes to font rendering. 

On the iPad, for example, there isn't really any concept of a view port, from our initial investigations. This means having the graph extend beyond the visible region isn't possible with our current canvas prototype, and that will be some considerable effort to implement.

Ultimately, the question is do we focus on one thing that we consider to be our core work, or spread our bets amongst all the possible platforms? Our commercial strategy has always been to focus, even if the technology has started as being very niche. Why support the Java visualization client in addition? Because we always have and it serves as a very useful prototyping platform for us. Will Flash make it to production, no.

Production-wise the focus is do one thing and do it well. Doing one thing is the easy bit, just be too lazy to do a second...

Friday, 27 August 2010

Page Breaks/Layout in Diagramming Applications

Is it just me that hates them? Every time I run up the Java Swing version of GraphEditor the first thing I do is remove the page layout. What's the point of restricting the amount of available space the moment you start an application, is the idea to make sure the printed version comes out correctly? If so, with laptops and tablets now commonplace in meetings (printing from an iPad would be nice Apple), I suspect printing is in decline. Corporate environmental policies tend to see to that, as well. The other thing is, isn't the computer supposed to deal with details like this for us? How about diagramming application default to maximum screen size by default and intelligently sort out printing for us (by which I mean avoid vertices and text being split between pages and zooming as appropriate)?

Friday, 20 August 2010

PNG embedded diagrams

In the previous post I mentioned embedding the XML of mxGraph models into PNG. The PNG specification includes a zTXt element, which enables very large sections of compressed text to be placed in the PNG encoding. mxGraph and onwards contain functionality to embed the model XML in created PNGs and to decode PNGs created in this way, using the Java code base. The GraphEditor example of the Java Swing client now saves in this PNG+XML format (.png suffix) by default to demonstrate the idea. We felt this is a somewhat more intuitive way of storing and transferring diagrams than text XML, since suddenly every user has the tools to look at the diagram, without the need for mxGraph. It provides a simple means for the server side to display diagram thumbnails without having to process the diagram and removes the necessity to synchronise XML formats and their associated images.

You can still stick with text XML if you prefer, but we'll sulk if everyone doesn't use this shiny new feature. This does bring up the issue of whether the JavaScript display looks the same as the generated server diagram. By default, they are identical, but customisation of the JavaScript can cause duplication of implementation on the server technology to get image generation to look the same. The majority of cases where this occurs happen because of new shapes being added to the JavaScript client programmatically. Which is another important reason for adding the stencils mentioned in the last post.

We are going to add an additional option to add the text without compression, not everyone will want their servers unzipping every time a diagram is opened and it provides a mechanism to view and extract the XML using a text editor, if required. Given the text is likely to form the minority of the file size, we're inclined to make this the default save format.

If you have other cases where there is code duplication necessary (other than I/O), we'd be interested to hear.

Tuesday, 3 August 2010


We recently updated the Java shapes architecture to match the JavaScript implementation. It's better, but still not quite right (so apologies to Java Swing users for another API change, although, we think it might be possible to avoid one).

The improvement is to add stencils, exactly in the same way an application like Visio has them. It's not a rocket science idea, but clearly a big improvement in usability for both developers and users. We'll supply a central repository of a good number of stencils, which will be freely available under a creative commons style license. You'll be able to load one, or a set, and work with it as you would any under shape.

There are some details to be worked out, the registry system, a naming convention, etc. But, we've got the first version working in Java, so we're looking at a first release in the Java code base September-October 2010. We didn't want to invent our own description XML for stencils, and the Visio format is far too complex for our purposes, so we've gone with the shape format available on the excellent Dia diagramming tool. This is an SVG sub-set, wrapped by a few additional attributes needed. The format is clearly well thought out, enables the vast majority of required stencils to be drawn, yet keeps the format simple and fast to implement.

Dia also specifies a mechanism to define a set of stencils, we'll be using this, or something very similar. We'll also be re-using our embedded PNG concept we recently introduced to the Java code ( which is a posting on its own, and probably deserves a post after this one, since it needs a proper explanation ). The short story is that a set will comprise a set description XML and some number of PNG files only. The Dia format describing the XML of the shape will be embedded in the PNG itself.

As mentioned, the change to the API should be minimal (if anything). We intend to add the stencil as a new shape and change the other vertex shapes to call the correct stencil ( but leave the old shape classes and methods, only deprecated in Java ).

The additional ideas this project has brought up are: 1) Implement the edge end markers as stencils and 2) Implement a vertex perimeter description to position ports correctly. We're researching a mechanism to infer the perimeter from the SVG of the shape, but this is very much at the research stage.

In summary, the first release will be a test to see how developers find the idea in the Java client, after that to port the concept to the JavaScript client, once the design is finalised. We're looking at December 2010 for the JavaScript production version, assuming some number of iterations on the Java code.

Tuesday, 27 July 2010

iPad, performance and canvas

There has been a sharp increase in the number of users trying out the mxGraph demos using iPads. We took stock of a number of the devices last month to see just how well mxGraph works. There were two main observations, it was too slow and our fingers were too big. Displaying cells works fine, and so it should, we support Safari fully on non-touch devices. Some of the iPxxx specific gestures (like the two fingered zoom in/out) we've added very experimental support for, so you get a feel for how we intend to implement the various gestures. Handles are clearly unusable, so we need an alternative for grabbing the ends of edges to disconnect them. Touch and hold on a vertex to enter a connection mode makes sense, I believe. A lot of the other problems are more about the sizes of shapes on the palette and menu sizes on the examples, we'll create an iPad specific example to address this.

As an aside, the intention is to support the iPhone too. We are somewhat unconvinced that the iPhone is a serious platform for diagramming, interaction is painful in pretty much all similar applications that we've seen. We might look into simplifying the gestures on the iPhone to make viewing faster and easier, but at the expense of interaction (configurable, of course). iPhone work will be secondary to the iPad support, the iPad size makes it a far more likely device.

This bring me to the second issue mentioned, speed. The performance of mxGraph has always been limited by browser's ability to cope with x number of elements in the display DOM. Modern browsers have improved in this regard somewhat, but native support for Internet Explorer 6-8, as well as consistent look and programming API across all browsers, are part of the basic make-up of the JavaScript product. That said, with the only mainstream browser on the iPad providing full support for canvas (they basically invented it, after all), we are going to add canvas drawing where the browser makes it available. We have been testing canvas for some time now and we've been seeing orders of magnitude improvements in rendering speed in certain test cases.

The API should be very little effected by the addition, already we abstracted SVG and VML as the vector drawing mechanisms. One part of the change that will be more significant is the loss of the display DOM's hit detection and dirty region repaint. We have implemented these mechanisms fully in the Java client, so there are no research elements, it is still several weeks of porting effort. Timescale-wise, iPad support will be by November 2010 and full canvas support by January 2011.