Code Overview

NOTE This is out of date but has some relevance, class names have changed for example but the general info is still correct

QxCompiler is API based so the easiest way to get started is to checkout the demos in demos/js (the demos/json folder is for compiling via the CLI, but the CLI uses the same code as demos/js so it’s probably easier to get started with).

The principals are that there are targets (qxcompiler.targets.*) for source, build, etc (eg qxcompiler.targets.SourceTargetetc), libraries to be scanned for code (qxcompiler.Library), and a maker (eg qxcompiler.makers.AppMaker) which runs the whole process of building the application(s).

Internally, a qxcompiler.makers.Maker uses a qxcompiler.Analyser to scan class files and resources, compiling a database of dependencies and resource meta data (ie image sizes etc) and then generating the application(s). Each Maker has exactly one Target, and is configured with multiple qxcompiler.Library’s to be scanned by the Analyser; it is aware of locales, translations and has an output directory.

While a Maker’s job is to produce applications, this is not baked into the abstract Maker class - for example, AppMaker produces one or ordinary Qooxdoo apps, but there are other special things like the demo browser which is an app with a collection of not-quite-an-application-apps but which have dependencies etc.

Makers do produce and configure qxcompiler.Application instances which describe the application being output; they know about the Analyser that can find classes and resources, the theme, environment (as in the generator ”let” environment block), and a short name.

When the Maker outputs code, it will typically create a directory based on the application’s name, inside the Maker’s output directory; as a Maker is associated with exactly one target, this means that multiple applications will go into (eg) “build/app-one”, “build/app-two”, “source/app-one” etc

The main function of Application is to calculate class and asset dependencies; it starts with a list of class names given to the constructor, adds the theme classes, and uses the Analyser to determine dependencies and the correct loading order. This is a bit of a nightmare because Qooxdoo classes are not 100% deterministic via static analysis, or they are but only if you operate a fully stateful inspection of a javascript virtual machine as you recursively analyse the code - for example, a class's defer method has load dependencies for the class so if that defer method instantiates an object there is a dependency; if it calls a method on that object, then any dependencies of that method become load-time dependencies for the class. This makes it very difficult to accurately determine dependencies at compile time, and although the generator does do some recursive analysis (i.e. more than qxcompiler) it appears that this is not 100% fool proof because @use and @require are necessary. However, with some runtime-code additions already in master, I believe that the output code is backwards compatible with the generator, even though scripts are loaded in a slightly different order.

The Analyser is quite simple and does the heavy work of tracking down class files and resources to scan, caching the data in a database, and goes to some lengths to be super efficient. My goal is typical application compiles of <100ms with cache, and even from scratch without a cache it should be only a second or two. For example, it should be possible to edit Qooxdoo apps on the fly, hit refresh and have the server recompile as part of a refresh without a noticable lag. Given than qxcompiler now transpiles, then every single code change requires a recompile so this is an essential goal.

The Analyser creates qxcompiler.ClassFile instances to represent an individual class files, reading the source code to determine dependencies, translation strings, and to transpile into plain old ES5 code using Babel with a custom plugin. The Babel plug in manipulates Qooxdoo-specific extensions (eg this.base()) into the transpiled output.

Just as with the generator, while there may be a conceptual difference between a library such as a contrib or the Qooxdoo framework itself versus an end-user application created by create-application.py, as far as qxcompiler is concerned they are all Library’s. Each one has a set of code in a unique namespace, source files, resource files, translations, etc and the Analyser looks for class and resource files in these Library’s.

qxcompiler.ResourceManager scans a set of Library’s for resources, checking for and updating meta data files using ImageMagick tools, and collates an easy to use set of assets for the Analyzer and Maker’s. Again, it tries hard to be efficient about this and is able to run command line tools in parallel.

qxcompiler.Translation represents a parsed translation *.po file, exposing the entries and can write it out again. This write function is mostly used for debugging and testing the parser because some targets (eg qxcompiler.target.BuildTarget) will merge the translation files from multiple libraries into a single file using only those required strings, whereas of course SourceTarget uses the originals.

qxcompiler.cli.CommandLine is a working beta of an app that reads the kind of files in demos/json to put all of the above together without requiring code; I imagine it would either be used by or the basis for an npm version of qxcompiler, but for the moment it’s loosely becoming the ultimate demo as the .json files support more and more features.

qxcompiler.Cldr represents parsed CLDR data found in the Qooxdoo framework, and required for output by the targets.

qxcompiler.utils is logging, and confusingly there is also qxcompiler.files.Utils and qxcompiler.files.FindFiles, both of which are utils really ...

Finally, qxcompiler.generator.* is code that was going to read the config.json files for ultimate backwards compatibility - this is largely abandoned now because of problems interpreting the config.json files. It is surprisingly hard (and not fully documented) to interpret the config.json 100% accurately, and in the end I decided that the cost of backwards compatibility was too high; the demo in demos/json shows how much simpler config files can be.