Large Image Support

This project includes a baseline to load a series of enhancements to Pharo. These enhancements provide a better user experience when coping with large images. Large images are images with a lot of objects, this objects are not only objects representing our data but also it applies to images with a lot of code.

It relies on two projects that have been integrated in Pharo 9:

• Complishon a new completion engine for Pharo that provides better contextual answers and it is implemented to minimize the queries to the global system.

• Spotter an iteration on the processor model of GTSpotter adding new processors that uses a set of composable iterator to perform the queries incrementally.

These projects are already integrated and their maintainance will be done as part of Pharo project.

This baseline performs three diferent things:

1. It installs the following tools:

• Containers/CTTrie an implementation of Optimized Tries.

• Aleph an index system to provide better queries for Spotter and all the users of system navigation. It includes two different implementation, one using a dictionary and the other using a Optimized Trie.

• VM Tunning a tool to tune the Garbage Collection of the VM during the execution of the index generation. This is required to have correct usage of memory and generate the indexes in a short time.

2. It performs a series of settings to improve the behavior of the image:

• Disables the rendering of incomplete selectors and variables. This improves the speed of the syntax coloring, as it does not require to iterate the whole symbol table.

3. It installs the integration Spotter integration with the indexes generated by Aleph.

Instalation

The installation is as easy as installing the following baseline:

Metacello new
	baseline: 'LargeImageSupport';
	repository: 'github://pharo-project/largeImages/src';
	onConflictUseIncoming;
	load

This baseline installs all the tools and configure them, it also configures the image to apply the settings and executes the initial process of index creation. This process is only needed after the installation and to give an example, for a 2gb image it takes 3 minutes. Such an image has ~1.500.000 methods and ~95.000 classes. The final image with indexes created is about 2.4 gb in disk.

About the Indexes

This baseline creates indexes for the following elements:

• References
• Senders
• Implementors
• Classes

The indexes for references and senders are built using a hash table, the indexes for implementors and classes uses two optimized tries for each one. As the later requires to have full text search to implement a nice Spotter experience.

Memory footprint

The memory footprint depends on the nature of the indexed image. For the previously described image the impact is around ~400 MB. Different experiments shows a penalty of 20%, although it will depend of the ammount of symbols, classes and methods. And also how these methods interact with each other specially for senders and references. Other important differenciating element is the string distribution of the selectors and how much they have in common.

Activating / Deactivating

The indexing and the storage of the information can be activated / deactivated sending messages to AlpIndexManager

To activate:

AlpIndexManager activate

To deactivate:

AlpIndexManager deactivate

It is remarkable that the activation will trigger a recalculation of the indexes, and that the deactivation discards existing indexes. This is done, because once AlpIndexManager is deactivated it will not keep updated the indexes and the information will be invalid.