adding support of code completion [imported]

[cmaglie]

This is Issue 849 moved from a Google Code project.
Added by 2012-03-07T13:26:22.000Z by Wilfried...@gmail.com.
Please review that bug for more context and additional comments, but update this bug.

Original labels: Type-Enhancement, Priority-Medium

Original description

What change would like to see?
I would like to see a code completion in the IDE like other IDE Editors have. Especially it would be nice, when this feature could interact with the #include keyword.

Why?
Because of simpler programming.

Would this cause any incompatibilities with previous versions? If so, how
can these be mitigated?
I thing, there will be no incompatibilities.

[ricardojlrufino]

As was said before, but follows for registration.
This implementation has been carried out, we just need to integrate and perform some tests.

https://github.com/CriativaSoft/Arduino/tree/ide-1.5.x-autocomplete

[ricardojlrufino]

As we talked on the list of developers, did the new implementation based on the structure in branche 1.5.x.
The current implementation status is:

• Autocomplete (90%)
• Real-time Parser and Error Marker(90%)
• Documentation in Editor (in progress - 20%) - Documentation of libraries in the Editor #2692

Summary of changes and tests:
https://gist.github.com/ricardojlrufino/19f89884162be39d851a

How was launched from the IDE version 1.6, I believe you may have a few more adjustments.
But must first of a feeedback than has already been done to trace a plan for the other implementations (such as documentation on Editor)

PS: O Link of repo has changed: https://github.com/OpenDevice/Arduino

[ricardojlrufino]

Was made the merge with the 1.6.1 version of the master and created a new branche to facilitate integration:
https://github.com/OpenDevice/Arduino/tree/1.6.x-autocomplete-neweditor

Autocomplete (95%)
Real-time Parser and Error Marker(95%)
Documentation in Editor (removed from user perspective)

only need help to test

Summary of changes and tests:
https://gist.github.com/ricardojlrufino/19f89884162be39d851a

[lars-sorensen]

So... Has this been cancelled?

[PureSpider]

Would hate if this was cancelled. 👎

[ricardojlrufino]

I did not have any further feedback from the Arduino team. It seems to me that is not as priority

[PureSpider]

Can you provide a windows build for the current version of the Arduino IDE with your extensions? I would love to use that!

[Ede123]

Always the useful stuff is considered low priority 😞

Would be great if this could be implemented, since it would make coding a lot easier.

@arduino team: Please give some feedback on whether you're willing to merge this and what needs to be done if you think there are any issues! All the hard work was obviously done by @ricardojlrufino already...

[lmihalkovic]

IMHO, the code as it is somehow forces a choice between cpp/c & completion OR multiple file types (like editing keywords.txt or the lib descriptor). There might also be athe need for a legal advice on distributing the cdt code inside arduino.

[cmaglie]

sorry for being late, I tried to answer yesterday, but this is a very busy week.

I did not have any further feedback from the Arduino team. It seems to me that is not as priority

@ricardojlrufino autocomplete is in our priority and has not been cancelled. I've already reviewed and commented your PR, this is my last email from the developer list:
https://groups.google.com/a/bcmi-labs.cc/forum/#!msg/developers/rlbxoXmV9Q8/7d397_ZLRxkJOf course I'm still interested to hear your opinion and start a discussion.

@arduino team: Please give some feedback on whether you're willing to merge this and what needs to be done if you think there are any issues! All the hard work was obviously done by @ricardojlrufino already...

@Ede123
In general we do not accept patch like this in blindly: once a PR is merged it must be maintained, bugfixed, improved... and of course we cannot rely on the free time of Ricardo for doing this kind of work. This to say that it's not just a matter of clicking the "Merge" button on github.

To be more specific, the main reasons blocking this PR:

• It's too big and not stable. It must be split into smaller, incremental, chunks to make the review easier and to increase the chance to get merged (see my proposal above).
• It tries to fix or change a lot of collateral aspects of the IDE that are unrelated with the autocompletion. Don't misunderstand me, this is great, but it's better if those fixes are submitted separately and not bundled together to increase the size and complexity of an already big contribution.
• I'd like to know more about the C/C++ parser, in particular how is determined the list of files to be analysed. I've made some experiments with libclang and I know that this is a tricky problem. As a quick check I tried (with @ricardojlrufino patch) to auto-complete Serial it works for the AVR boards but not for the SAM, or if I #include <SPI.h> I cannot complete methods of the SPI. class. This confirms my suspects and makes me think that the way "suggestions" are build is not optimal or misses some pieces. This hasn't been discussed before the submission and it's hard to follow how the parser works by looking at a patch that changes more than 3500 lines of code. That's one of the reason why I asked to separate the "completion provider" from the "GUI" by creating a definite interface between the two, this way we can focus on fixing these non-trivial problems on the engine.
• Doing a separate contribution for the GUI part may help, on the other side, to get a bunch of nice thing merged independently from the autocompletion, the first thing that comes to mind is the improved error highlithing, but could be more. Again a definite interface may help to both understand the problems better and to allow to easily exchange implementations.

[lmihalkovic]

For having written an implementation of code completion, it is possible to shrink the contact surface to a couple lines of code, with all the logic in an external jar file. One of the added benefits is that you can also support different completion scheme based on the file type (i deal with keywords files, lib descriptors and c/c++ code). For having read the patch, I do not think that it will let you do that easily (not that this is what you have to do).
I already shared how IMHO the existing Editor and related classes could use a bit of cleanup (would also address a lot of known issues afor dealing with the compiler). The issue I see with the existing patch is that instead of keeping things together, it pushes more code in more places where it should not be added. Considering how the IDE is on its way out, maybe it is not such a bad thing anyhow (I'm sure Create will work easily on the desktop).

[lmihalkovic]

@cmaglie I think I might have a sense of the hit&miss that you described, as I did identify some inadequacies with the code driving the cdt parser: the problem is that it builds a model that is too simplistic as it is, and the complete cdt model layer depends on the eclipse workspace layer to run (if I recall there was also an issue with includes)

I made a test where I managed to start the workspace layer inside Arduino IDE (it actually mapped naturally to the ide's usage of a single root folder for work). It worked... but then might as well use eclipse. So then I wrote my own model of c++ that has more info than ricardo's but less than cdt. the code is not dependent on the parser as I was considering writing a libclang version (written as a standalone local cpp server) and does good enough for my own usage.

[Avamander]

If Arduino IDE (Processing fork) does not wish to implement this, what about creating official support for CLion and adding all the Arduino tools to it?

[ricardojlrufino]

Hi,

Since the discussion has started again. I made the implementation several times of the functionality of autocomplete, a had to adapt the changes that had been taking place of the IDE. Without the original team making the merge, it became impossible to maintain.

Yes, true, I had to do a lot of refactoring in IDE for it to be able to ACCEPT the auto-complete system. A lot of refactoring was accepted (like new code editor), but the autocomplete itself was not.

@lmihalkovic

For having written an implementation of code completion, it is possible to shrink the contact surface to a couple lines of code, with all the logic in an external jar file

Yes that may be possible. Part was made and isolated in the project:
https://github.com/ricardojlrufino/cplus-libparser

What I can do is to create the basis of the autocomplete system. And create the specific implementation you can use: keywords.txt (this would be the default) and C ++ code (cplus-libparser)

Remembering that the project can export the class definitions and functions for a JSON / XML, something like keywords, only without the manual work of the developer, and with the support to include the documentation of the methods.

I think this could avoid the need to distribute lib (/ cplus-libparser) along with Arduino

PS: Sorry for my bad english

[ricardojlrufino]

The main modifications in the IDE are to make the autocomplete smarter by dynamically parsing the main Sketch as the user modifies. This could be left for a second time, but with the loss of some functionality like not recognizing variables with different names of libraries such as: mySerial.xx!

I think it's a start, I've done this implementation for 2 years now and no progress has been made.

For a tool that is made for beginners, I think the auto-complete is essential, even if it is not perfect ...

@cmaglie What would be the way to start this project?
What would be the general guidelines for including this in Arduino

[per1234]

I think it would be a good feature to add but please add an option to completely disable it. If it's not desirable to do that in the GUI, editing preferences.txt is fine by me. Everyone has a different style of writing code and what's useful to some is an annoyance to others. An example is the automatic closing bracket feature. I'm sure some people love it but it causes way more problems than benefits for me and there's no way to turn it off.

[cmaglie]

Hi @ricardojlrufino,

I repeated so many times now... but for the sake of getting this thing done, let me be more pragmatic: may I ask you to make a PR to the current IDE that adds only the autocompletion dialog?

Something that takes the "code completions" from a CompletionProvider? I imagine something like:

package cc.arduino.autocomplete;

import java.util.List;
import processing.app.Sketch;

public interface CompletionProvider {
  List<CodeCompletion> getSuggestions(Sketch sketch, int fileIdx, int line, int col);
}

class CodeCompletion {
  // TODO: to be agreed
}

What I want is a patch that when I press Ctrl+Space (for example) makes the IDE calls getSuggestions method passing the Sketch with the cursor coordinates and get back the autocomplete suggestions that gets displayed (and used if the user click on one of them). We must also agree on the CodeCompletion class format.

Is that feasible?

For now I don't want the suggestions to be anything meaningful, to be clear it has to work with a FakeCompletionProvider:

class FakeCompletionProvider implements CompletionProvider {

  @Override
  public List<CodeCompletion> getSuggestions(Sketch sketch, int line, int col) {
    List<CodeCompletion> res = new ArrayList<>();
    res.add(new CodeCompletion("Hello!")); // or whatever fits
    return res;
  }
  
}