Editor cleanup #4363

For some toolbar buttons, when it is clicked while shift is pressed, its function changes. When handling the click event, this information is directly taken from KeyEvent.isShiftDown(). However, to also show the proper tooltip *before* clicking, EditorToolbar listened to key events on the main text area, to know when shift is (not) pressed. This approach means that pressing shift while the text area is not focused will not change the tooltip, and creates some unwanted coupling between the toolbar and the text area. This commit changes this approach to instead use the global KeyboardFocusManager. Any key presses pass through there before being dispatched to the currently focused component, so this makes sure that any shift presses are caught, as well as making EditorToolbar a bit more self-contained.

RSyntaxTextArea appears to support using a single instance and replacing the underlying text and document when switching between tabs, but in practice this support is not complete and even though the RSyntaxTextArea developers did some work to improve the situation, they recommend to just use a seperate instance for each tab. This commit implements exactly that. A new class EditorTab is introduce to wrap the RSyntaxTextArea and containing scroll pane, and to encapsulate the code related to handling the text area itself. Doing so removes some quirks and prepares for some later additions. In particular, error highlights are now no longer shared between all tabs, which was previously the case. This commit mostly moves code from Editor into EditorTab, and updates the callers to use getCurrentTab() and call methods on the result instead of calling them on Editor. Some code is added to take care of creating multiple EditorTab objects and switching between them. Some small changes have been made to make the flow of opening files work, though these are mostly a bit hacky. While moving code, changes to the rest of the code were kept minimal, retaining existing interfaces as much as possible. This sometimes result in less than ideal code, which should be cleaned up in subsequent commits. The SketchCodeDocument class has been pretty much emptied out, since it was mostly used to store things for tabs in the background, which are now just stored in each RSyntaxTextArea separately. The last remaining bits of this class can probably be moved or implemented differently later, so it can be removed. The entire flow of working with sketches and files needs to be cleaned up next, so no thorough attempt at testing this commit was done. It is likely that there are plenty of corner cases and race conditions, which will be fixed once the reset of the code is cleaned up. Fixes arduino#3441

It was not used anymore, and removing it makes subsequent refactoring easier.

Previously, EditorTab set the Document on the SketchCodeDocument, and the latter would listen for changes, only forwarding the modified status to SketchCode. This commit cuts out a step and lets EditorTab call SketchCode::setModified directly. Additionally, the DocumentTextChangedListener helper class is added, which wraps a simple (lambda) function to be called whenever anything about the document text is modified. This hides the verbosity of having to handle both insertion and deletion, and instead suffices with just having a single lambda function instead.

Now that each file in the sketch has its own text area in the GUI, it is no longer needed to store the (possibly modified) contents of each file inside SketchCode. Keeping the contents in the text area is sufficient. Doing so allows removing the code that dealt with copying contents from the text area into the SketchCode instance at the right time, which was fragile and messy. However, when compiling a sketch, the current (modified) file contents still should be used. To allow this, the TextStorage interface is introduced. This is a simple interface implemented by EditorTab, that allows the SketchCode class to query the GUI for the current contents. By using an interface, there is no direct dependency on the GUI code. If no TextStorage instance is attached to a SketchCode, it will just assume that the contents are always unmodified and the contents from the file will be used during compilation. When not using the GUI (e.g. just compiling something from the commandline), there is no need to load the file contents from disk at all, the filenames just have to be passed to arduino-builder and the compiler. So, the SketchCode constructor no longer calls its `load()` function, leaving this to the GUI code to call when appropriate. This also modifies the `SketchCode.load()` function to return the loaded text, instead of storing it internally. To still support adding new files to a sketch (whose file does not exist on disk yet), the EditorTab constructor now allows an initial contents to be passed in, to be used instead of loading from disk. Only the empty string is passed for new files now, but this could also be used for the bare minimum contents of a new sketch later (which is now down by creating a .ino file in a temporary directory). Another side effect of this change is that all changes to the contents now happen through the text area, which keeps track of modifications already. This allows removing all manual calls to `Sketch.setModified()` (even more, the entire function is removed, making `Sketch.isModified()` always check the modification status of the contained files).

Previously, some of the GUI code would use Editor.getSketch() to get the current sketch, and Sketch.getCurrentCode() to find out the currently selected tab. Since this code is really concerned with the currently open tab in the GUI, it makes more sense to query the Editor tabs list directly. This removes all references the current sketch code, as tracked by Sketch, external to Sketch itself. This prepares for removing the current tab tracking from Sketch later.

Instead of letting Sketch (also) keep track of the currently selected tab, this moves the responsibility to Editor instead. When Sketch need to know the current tab and file, it now asks Editor. Switching between tabs is still handled through Sketch methods, but that will be cleaned up later.

This lets all code directly call `Editor.selectTab()`, or the newly introduced `Editor.selectNextTab()` or `Editor.selectPrevTab()`. This also adds a new `Editor.findTabIndex(String)` to look up a tab based on the filename (what `Sketch.setCurrentCode(String)` used to do). At some point, this method might need to be removed, but for now it allows other code to keep working with minimal changes.

This class served no purpose anymore, so it can be removed. The `SketchCode.getMetadata()` and `setMetaData()` methods only served to keep track of a SketchCodeDocument instance (and were no longer used), so these are removed too, just like some SketchCode constructors dealing with this metadata object.

It was not used, and since it only updated the `name` attribute, but not the corresponding `file` attribute, nor actually handled renaming actual files, having this method around would actually be harmful, so just drop it.

This makes checking for the primary file easier, without having to know the index of a file in the list of tabs, or relying on the fact that the primary file is always first (it still is, though). This changes some places in Sketch to use the new `SketchCode.isPrimary()` method, but there probably are a lot more places in the code that could be start to use it as well.

These used to iterate over the list of SketchCodes to find the right one, and if so, let the List do the same again to remove it or find the index. This can be simplified to just let list take care of things instead. Technically, there is a small difference, since `List.remove()` and `List.indexOf()` will check using `equals()`, while the original code used `==`, but these should be effectively the same here. Also, the original code first used `==` to see if the object was present and then let List find it again using `equals()`, so that was a bit inconsistent anyway.

Instead of manually sorting the primary file at the start, and fiddling to keep it there during resorting, this just modifies the sorting comparator used to sort any primary files at the start. This is slightly more generic than needed, also supporting multiple primary files, to at least not break the Comparator preconditions when for some reason there are multiple primary files.

These exceptions were silently dropped, which is pretty much never a good idea.

Previously, the Sketch constructor called its `load()` function, which called the `SketchData.load()` function to load files and then `Editor.sketchLoaded()` to initialize the GUI with the loaded files. When external editing was enabled, `Sketch.load()` was called again when activating the Arduino app, to reload the entire sketch. With this commit, the `Sketch.load()` function is removed, and `SketchData.load()` is called from the SketchData constructor. Instead of Sketch calling `Editor.sketchLoaded()`, that method is renamed to `createTabs()` and called by `Editor.HandleOpenInternal()` directly after creating the Sketch object. Handling of external editor mode has also changed. When the Arduino application is activated, instead of fully reloading the sketch (through the now-absent `Sketch.load()` method), the new `SketchData.reload()` method is called to reload the list of files in the sketch. If it changed, all tabs are re-created. If not, only the current tab is reloaded. When the user switches from one tab to another, that tab is also reloaded. This ensures that the visible tab is always up-to-date, without needlessly reloading all tabs all the time. When external editing mode is enabled or disabled, all tabs are reloaded too, to make sure they are up-to-date. When re-creating all tabs, no attempt is made to preserve the currently selected tab. Since adding or removing files happens rarely, this should not be a problem. When files are changed, the currently selected tab is implicitly preserved (because the tab is reloaded, not recreated). The caret (and thus scroll) position is preserved by temporarily changing the caret update policy, so the caret does not move while the text is swapped out. This happens in `EditorTab.setText()` now, so other callers can also profit from it. To support checking for a changed list of files in `SketchData.reload()`, a `SketchCode.equals()` method is added, that just checks if the filenames are equal. Additionally, the loading of the file list for a sketch has now moved from `SketchData.load()` to `SketchData.listSketchFiles()`, so `reload()` can also use it. At the same time, this loading is greatly simplified by using a sorted Set and `FileUtils.listFiles()`. In external editor mode, to ensure that during compilation the version from disk is always used instead of the in-memory version, EditorTab detaches itself from its SketchCode, so SketchCode has no access to the (possibly outdated) in-memory contents of the file.

With this commit, any warnings about invalid sketch filenames are not shown when the sketch is reloaded. This reloading happens whenever the IDE window is focused, so re-logging warnings all the time isn't really helpful, so this hides them.

By now, all calls to `addCode()` were followed by a call to `sortCodes()`, and it seems like a task for SketchData to keep its list sorted. Previously, this separation made some sense, since `addCode()` was also used while loading a sketch, and you would only want to sort once. Now, sketch loading uses a SortedSet, so this is no longer a requirement.

The only change that happens is a single `setText()` call, which already results in a single undo action.

When completely replacing all text, all highlights will end up at the start of the file. Since keeping them at the right place is tricky (even impossible in some circumstances), just remove them now. This already happened in the autoformat code, so that part can be removed.

`EditorTab.setText()` now already preserves the caret position. The code used during auto-format tried a bit harder to preserve the position correctly, and probably worked better in a few specific cases, but for most cases they would both end up approximating the caret position anyway. To make the code simpler, better just stick to the simpler approach.

The former gives focus to the window in which a component is present, while the latter only changes the focus within the current window (not focusing the window itself if it is not focused yet). Java documentation recommends changing `requestFocusInWindow()` where possible, due to some platform-dependent behaviour in `requestFocus()`. When focusing the serial monitor and plotter, `requestFocus()` is still used, since then the focused window *should* change.

This was a remnant of Processing, this file has no special meaning for an Arduino sketch, so this code can just be removed.

When adding a file to a sketch (using drag and drop, or the Sketch -> Add file... menu item), .o, .a and .so files would be saved into a "code" subdirectory of the sketch. This seems to be a remnant of processing, where also .dll and .jar files could be added to a sketch to be used. In the Arduino IDE, these code files serve no special purpose, and are not treated specially, so it makes no sense to keep this code around. One implication of this is that when "save as" is used, a "code" subdirectory is no longer copied, which might affect people using this "code" subdirectory for other purposes. Similarly, there is support for a "data" subdirectory, in which all other files (that are not sketch source files) are stored, and which is also copied on "save as". Support for this folder is kept intact, since this appears occasionally used (the ESP8266 project uses it to store and upload additional data files, for example). This change was discussed on the mailing list in the "Anyone using "data" and "code" subdirectories in sketches?" thread: https://groups.google.com/a/arduino.cc/forum/#!msg/developers/zPlraPq55ho/ejrLqITnAgAJ

The only remaining thing that the method did was call `ensureExistence()`. However, a call the `prepare()` was always followed by a call to `build()`, which already calls `ensureExistence()`, so `prepare()` didn't have any remaining value.

Sketch is now called SketchController, since it didn't really represent a sketch, but just handled the GUI-related stuff for a given sketch (note that it is not strictly a controller in the MVC-sense, but it does have a similar function). SketchData more accurately represented the actual sketch, so it is now called Sketch. Below, the new names are used. Editor now keeps both a current Sketch and SketchController object, and the Sketch object is created by Editor and passed to SketchController, instead passing a File and letting SketchController create the Sketch. Wherever possible, code now uses the Sketch directly (or indirectly, through the new `SketchController.getSketch()`) and the accessors in SketchController that merely forwarded to Sketch have been removed. There are few things that now live in SketchController but should be moved to Sketch (`isModified()`, `isUntitled()`), so some of the code still has a dependency on SketchController that should be removed later. This commit mostly renames classes, methods and variables, it should not change the behaviour in any way.

Also, update any code that uses it, removing the dependency on SketchController entirely if possible.

This should be more reliable than comparing filenames or assuming the primary file is always at index 0.

Filename-based matching is a bit more fragile, so just do a lookup based on the SketchCode object we already have instead.

When renaming a file, the current tab is already the correct one, so no need to switch. This allows looking up the tab index based on the SketchCode object, instead of doing a filename lookup.

Before, `getPrettyName()` would return the extension-less name for all files. There were a lot of places that checked for .ino and/or .pde files and and called `getPrettyName()` for those, and `getFileName()` for others. By moving this check into `getPrettyName()`, all those callers become more simple, and more consistent (there were 5 different checks to basically achieve the same thing). There are small changes in behaviour, where .pde is now also hidden but was not before. Also, the print header now shows extensions for other files, which makes it more consistent with the tab names. For cases where the old behaviour was still required, `Sketch.getBaseName()` was added. At the same time, the actual handling of the filenames is simplified by using methods from FileUtils. With this change `Sketch.getFileNameWithExtensionIfNotIno()` and `SketchController.getHiddenExtensions()` are no longer needed and are removed.

This lets it use FileUtils.splitFilename and reference Sketch.EXTENSIONS and the new Sketch.DEFAULT_SKETCH_EXTENSION directly, allowing to remove a few helper functions.

For determining if the current file was a sketch file, it previously (indirectly) used a hardcoded "ino" comparison. Now, it uses `SKETCH_EXTENSIONS` so it also applies to .pde files and the hardcoded "ino" (and the methods leading up to it) can be removed.

Previously, the index of the SketchCode instance in the list kept by Sketch was kept, which isn't really robust. With this change, Sketch.indexOfCode is no longer needed and is removed.

That name more accurately reflects its purpose: It represents a single file within a sketch. This just updates the class name and variable names referring to these objects and some comments, so no behaviour should change.

This handler was only always assigned the DefaultStopHandler, which did nothing. It was called in a few places, but since it never does anything, better remove it. For properly supporting stopping of external processes, some better architecture should be added instead.

These methods dealt with opening up a sketch inside an existing editor window, but they were unused - all sketches open in a new window now.

This limitation was added a long time ago, when the build system did not cope with this. The current build system handles this situation just fine, so this limitation can be lifted.

Instead, just the File object when requested. It is not used during normal operation (just when adding files, or using save as), so no point in already creating the object in the constructor.

Previously, it returned a File object, which the Sketch separately stored from the primary SketchFile. By letting it just return the SketchFile, and let callers query that for the filename, Sketch does not need to store the File object itself and there is less chance of info getting out of sync.

Keeping filenames as File objects for as long as possible is generally a good idea and this removes a dependency on `Sketch.getMainFilePath()`, so it can be removed later.

Sketch already stores the sketch folder, and the sketch name should be identical to the folder name. In the case where the filename passed to the sketch constructor is not the primary .ino file (named after the sketch), this will slightly change behaviour. However, the calling code should prevent this from happening, and with the old code, some internal assumptions were probably violated, so this changes makes handling this situation a bit more robust. Since the actual filename passed to Sketch is no longer used, it is no longer required to pass the name of the primary .ino file. At some point, the constructor should probably be changed to accept a folder name instead of a filename, but that would require a lot of changes to trace this back through the code, so this is something for later.

There was already a nearly identical `FileUtils.copy()` that copies directories recursively. The only difference is that now hidden files *are* copied, but version control files (according the list in FileUtils) are not. Since this only affects the copying of the "data" directory during save as, this should not be much of a problem.

This makes a few related changes: - `FileUtils.replaceExtension()` is introduced to handle replacing the .pde extension with .ino. - Instead of iterating .pde files on disk, this iterates SketchFiles in memory, saving another lookup from filename -> SketchFile later. - `SketchController.renameCodeToInoExtension()` is removed. Now it no longer needs to look up the SketchFile and FileUtils handles the extension replacement, this method did not have any reason to exist anymore. - Instead of hardcoding the .pde extension, a new Sketch.OLD_SKETCH_EXTENSIONS constant is introduced.

No need to call the File constructor ourselves, if `File.getParentFile()` can just do that for us.

This commits replaces a significant part of the code handling these features. A lot of responsibilities are moved from SketchController to Sketch, though the code involved is rewritten mostly. Most of the handling now happens inside Sketch, including various checks against the new filename. Basically SketchController processes the user input to decide what needs to be done, and Sketch checks if it can be done and does it. If problems occur, an IOException is thrown, using a translated error message that is shown by SketchController as-is. This might not be the best way to transfer error messages (regular IOExceptions might contain less-friendly messages), so this might need further improvement later. In addition to moving around code and responsibilities, this code also changes behaviour in some places: - Because Sketch and SketchFile are now in control of renames and saves, they can update their internal state after a rename. This removes the need for reloading the entire sketch after a rename or save as and allows `Editor.handleOpenUnchecked()` to be removed. - When renaming the entire sketch, all files used to be saved before renaming, since the sketch would be re-opened after renaming. Since the re-opening no longer happens, there is no longer a need to save the sketch, so any unsaved changes remain unsaved in the editor after renaming the sketch. - When renaming or adding new files, duplicate filenames are detected. Initially, this happened case sensitively, but it was later changed to use case insensitive matching to prevent problems on Windows (where filenames cannot differ in just case). To prevent complexity, this did not distinguish between systems. In commit 5fbf962 (Sketch rename: allowig a case change rename if NOT on windows), the intention was to only do case insensitive checking on Windows, but it effectively disabled all checking on other systems, making the check not catch duplicate filenames at all. With this commit, all these checks are done using `File.equals()` instead of comparing strings, which is already aware of the case sensitivity of the platform and should act accordingly. - Some error messages were changed. - When adding a file, an empty file is not created directly, but only a SketchFile and EditorTab is added. When the sketch is saved, the file is created. - When importing a file that already exists (thus overwriting it), instead of replacing the SketchFile instance, this just lets the EditorTab reload its contents. This was broken since the introduction of EditorTab. The file would be replaced, but not this was not reflected in the editor, which is now fixed. This change allows `Sketch.replaceFile()` to be removed. - When importing a file that does not exist yet (thus adding it), a tab is now also added for it (in addition to a SketchFile). This was broken since the introduction of EditorTab, and would result in the file being added, but not shown in the editor. This commit adds a `Sketch.renameFileTo()` method, to rename a single file within the sketch. It would be better to integrate its contents into `Sketch.renameTo()`, but that does not have access to the `Sketch` instance it is contained in. This will be changed in a future commit.

These methods shouldn't really be in Base (or BaseNoGui, which did the actual work), especially since there is already a `FileUtils.recursiveDelete()` which just does the same thing. This commit removes the code from Base and BaseNoGui and instead uses the method from FileUtils. There is one difference between these methods: the Base methods did not delete files if the "compiler.save_build_files" preference was set. However, the Base methods were only used when deleting a sketch, or deleting an existing folder before overwriting it on save as, so this preference didn't actually do what it was supposed to anyway, so dropping it shouldn't be a problem.

This allows simplifying some other things later.

Now that SketchFile keeps a reference to its Sketch, `SketchFile.renameTo()` can call `Sketch.checkNewFilename()`, so there is no need for the renaming itself to go through Sketch. This changes the parameter for `SketchFile.renameTo()` from File to String, to enforce that only the filename is changed, not the directory name.

This isn't much code, but it makes deletion more consistent with renaming and saving with the SketchController handling the UI part and Sketch actually doing the delete.

The extra "File" in the name was a bit redundant, and this is more consistent with `Sketch.delete()`.

Previously, callers of `SketchFile.delete()` would also call `Sketch.removeFile()`, but letting SketchFile handle this is more robust. This is possible now that SketchFile keeps a reference to Sketch and makes updating the Sketch file list less fragile. Eventually this might be further decoupled by letting SketchFile broadcast a "deleted" event instead.

Previously, the caller of the constructor did this, but now that SketchFile keeps a reference to its containing sketch, it can figure it out itself.

Previously, everywhere where it was needed, the path was requested from BaseNoGui. Because the path is based on a hash of the sketch filename, every caller would get the same path for the same sketch. However, it makes more sense to store the path used for a given sketch inside the Sketch object. This prevents having to pass around or regenerate the build path everywhere, and no longer requires the build path to be deterministic (though it still is in this commit). This allows removing some methods and constructors of which two versions were available - one with a build path argument and one without.

Previously, this used a hash of the sketch filename, so the same build path would be generated for the same sketch between multiple compilations. Now that the build path is stored, this requirement has disappeared, so a random filename can be generated again. While here, this commit also changes the prefix from "build" to "arduino_build_", which makes it a bit more clear what the directory's purpose is.

Comparing a File object automatically takes care of filesystem case sensitivity, whereas strings do not, so this makes the comparison slightly more reliable.

This comment still talked about Processing related stuff which doesn't really apply anymore.

Turns out the approached used to keep caret and scroll position only keeps the scroll position. More code is needed to also keep caret position.

Turns out I dropped a setUndoManager call somewhere, as well typed Redo instead of Undo somewhere.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Editor cleanup #4363

Editor cleanup #4363

Commits on Jan 21, 2016