IDE is non-functional if package index can't be loaded

[per1234]

Describe the problem

Package indexes are used by the Arduino Boards Manager system to track the Arduino boards platforms that are available for installation.

These come from two types of sources:

• The official primary package index
• URLs specified by the user via the "Additional Boards Manager URLs" field of the IDE's "Preferences" dialog

The package indexes are downloaded from these URLs by Arduino IDE 2.x on startup.

🐛 Under the following conditions, the IDE is non-functional:

• A package index cannot be downloaded from its URL
• The package index is not already present in directories.data

To reproduce

1. Select File > Preferences... from the Arduino IDE menus.
2. Add the following to the "Additional Boards Manager URLs" field:

   https://downloads.arduino.cc/packages/package_nonexistent_index.json
   

   ⓘ The issue is not specific to this URL. Any URL that does not provide a package index URL download can be used.
3. Click the OK button.
4. Select File > Quit from the Arduino IDE menus.
5. Start the Arduino IDE.
6. Open the "Board Selector" menus.
   🐛 No boards are shown.
7. Open the Tools menu.
   🐛 There is no "Boards" menu.
8. Open the "Boards Manager" view.
   🐛 The view is empty, not even showing the currently installed platforms.
9. Open the "Library Manager" view.
   🐛 The view is empty.

🐛 The only way for the user to identify the problem is searching through the logs.

Expected behavior

Clearly communicate to the user that there was a problem with downloading the package index.

It would probably also make sense to retain general functionality of the Arduino IDE (though of course the Boards Manager functionality that relies on the missing package index in particular will not be available).

Arduino IDE version

2.0.0-rc7-snapshot-a715da3

Operating system

Windows

Operating system version

10

Additional context

I bisected the issue to 4c55807 (does not occur with the build from cb50d3a)

---

The logs contain the following:

root ERROR Uncaught Exception:
root ERROR Error: 13 INTERNAL: Error downloading index 'https://downloads.arduino.cc/packages/package_nonexistent_index.json': Server responded with: 404 Not Found
    at Object.callErrorFromStatus (C:\ide 2\tip\48-a715da3\resources\app\node_modules\@grpc\grpc-js\build\src\call.js:31:26)
    at Object.onReceiveStatus (C:\ide 2\tip\48-a715da3\resources\app\node_modules\@grpc\grpc-js\build\src\client.js:349:49)
    at Object.onReceiveStatus (C:\ide 2\tip\48-a715da3\resources\app\node_modules\@grpc\grpc-js\build\src\client-interceptors.js:328:181)
    at C:\ide 2\tip\48-a715da3\resources\app\node_modules\@grpc\grpc-js\build\src\call-stream.js:187:78
    at processTicksAndRejections (node:internal/process/task_queues:78:11)

---

Even though it is most likely to occur with package indexes provided by via the "Additional Boards Manager URLs" preference (whether by user error, no connection to Internet, or transient network outage on the server), it does also occur even with the primary package index so long as the conditions described previously are present.

---

Originally reported at #1036 (comment)

I mistakenly claimed that issue had been fixed by #1045 because I did not realize at that time that the package index's absence from directories.data was a required condition to reproduce it.

---

Related:

• https://forum.arduino.cc/t/arduino-2-0-0-rc-7-no-board-manager-or-library-manager/1004131/6
• https://forum.arduino.cc/t/ide-2-0-0-rc8-missing-tools-board-and-other-menus/1007606

Issue checklist

• I searched for previous reports in the issue tracker
• I verified the problem still occurs when using the latest nightly build
• My report contains all necessary details

[kittaakos]

This will be partially fixed with #1132. A follow-up issue is here: arduino/arduino-cli#1788

[Lorandil]

I'm experiencing the same behaviour on my machine since updating to 2.0 RC8.
IDE 1.8.16 is still working fine, so there are boards available.

I suppose I will have to go through all my board managers - yay!

http://arduino.esp8266.com/stable/package_esp8266com_index.json
https://dl.espressif.com/dl/package_esp32_index.json
http://drazzy.com/package_drazzy.com_index.json
http://raw.githubusercontent.com/damellis/attiny/ide-1.6.x-boards-manager/package_damellis_attiny_index.json
https://github.com/stm32duino/BoardManagerFiles/raw/master/STM32/package_stm_index.json
https://raw.githubusercontent.com/MrBlinky/Arduboy-homemade-package/master/package_arduboy_homemade_index.json

A hint in the output window would be nice ;)

[per1234]

Hi @Lorandil. There have been significant advancements since the 2.0.0-rc8 version of the Arduino IDE you are using. Please always test using the latest nightly build before submitting a report here on GitHub.

In this case, there is work in progress in a pull request to even further improve the handling of these URLs. The tester build from that pull request is the most advanced form of the Arduino IDE. Please give it a try and then tell us whether it solves your issue.

I'll provide instructions for how you can download the tester build:

1. Sign in to your GitHub account.
   (GitHub only allows downloads of the tester builds when you are signed in.)
2. Open this page in your browser:
   https://github.com/arduino/arduino-ide/pull/1132/checks
3. From the list of workflow runs on the left side of the page, click on "Arduino IDE" (this is the build workflow).
4. From the "Artifacts" section of the page, click the download link for your operating system.
   ⓘ For example, if you are using Windows, use the "Windows_X86-64_zip" link.
5. Wait for the download to finish.
6. Extract or install the downloaded file as usual.

[Lorandil]

Hi @per1234 ,

I tried the nightly from today and it didn't solve the problem for me.

[per1234]

@Lorandil I think it will be better for us to continue this investigation over on the Arduino Forum. Please post in the dedicated category for Arduino IDE 2.x:

https://forum.arduino.cc/c/software/arduino-ide-2-0-beta/93

I'm certain to see your post over there.

[per1234]

The issue was closed inadvertently during the merge of #1132 due to a chance occurrence of a GitHub issue closing keyword.

The IDE has been made much more resilient to failed package index downloads via #1132, and also does an excellent job of communicating about such failures to the user via the notification system.

The IDE now works just as expected when following the instructions I provided above. There remains one case where a failed package index download produces unexpected behavior:

1. Select File > Preferences... from the Arduino IDE menus.
2. Add the following to the "Additional Boards Manager URLs" field:

   https://downloads.arduino.cc/packages/package_nonexistent_index.json
   

3. Click the OK button.
4. Select File > Quit from the Arduino IDE menus.
5. Rename or delete (⚠ carefully!) the following folder:
   • Windows:

     %LOCALAPPDATA%\Arduino15\
     

     (e.g., C:\Users\<username>\AppData\Local\Arduino15\)
   • Linux:

     ~/.arduino15/
     

   • macOS:

     ~/Library/Arduino15/
     

6. Start the Arduino IDE.

🐛 The IDE startup hangs at the splash screen

Additional context

The IDE will start and function correctly if you close and then restart it.

[Lorandil]

Hi all,
just a quick update: The problem is fixed for me with the nightly of 2022-07-10.

The working boards are there and an error message is displayed - which exposes the root cause of the problem:
There was a blank after the esp8266 board manager link. Removing the blank solved the issue completely.
I suggest left/right trimming blanks from the boardmanager URLs.

Thanks for fixing the problem!

[per1234]

I'm glad you were able to find the problem @Lorandil

There was a blank after the esp8266 board manager link. Removing the blank solved the issue completely.
I suggest left/right trimming blanks from the boardmanager URLs.

Thanks for your report.

I did some experimentation but was not ever able to reproduce the problem. Please provide detailed instructions I can follow to reproduce it.