Implement support for updating plugins inside the editor

[bitbrain]

Describe the project you are working on

I am working on an untitled RPG (project name "cave"), as well as maintaining several Godot addons called beehave and pandora.

Describe the problem or limitation you are having in your project

While developing editor plugins and using them, keeping them up to date is a challenge. This involves manually checking either the asset library or Github to see if a new update is available.

Some plugins such as pandora and gdUnit4 have an auto-update functionality in-built, but for any other addon, the maintainer would require to build such functionality themselves.

The downside of this becomes apparent once using multiple addons, perhaps even in a double digit count. Keeping them up-to-date is a pain and requires serious effort to understand which ones to update, which ones may break compatibility etc. - to make matters worse, for the existing addons that support auto-updating, they need to be updated one by one and each brings an inconsistent user journey to how to update addons.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

This proposal aims to provide an auto-update experience to users of the Godot editor, while also not trying to force a specific way onto addon developers to update their addons. Inside the project settings there is a Plugins tab. Navigating there shows all the plugins that have a new version available, as well as a button to update them all:

Clicking "Update all" will automatically download the new versions of each addon and update it respectively. In case an addon requires an editor restart, it will prompt a popup that an editor restart may be required.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

The idea of this proposal also is to give the plugin developer the control over how their addon gets updated, rather than forcing a standardization on them. A developer of a plugin may choose to host their plugin code in a very specific place (like Github) and the Godot editor cannot infer that information always. This can be achieved in two ways:

Approach: dedicated AutoUpdatePlugin

Inside the EditorPlugin there could be two new methods being exposed as follows:

# gets called by Godot in a separate thread
func _check_plugin_version() -> bool:
   var plugin_config:ConfigFile = ConfigFile.new()
   plugin_config.load("res://addons/pandora/plugin.cfg")
   var current_version = plugin_config.get_value("plugin", "version")
   
   # check latest plugin.cfg on Github
   # or verify the latest git tag etc.
   return _get_github_version() > _version_to_number(current_version)

# gets called by Godot in a separate thread
func _auto_update_plugin() -> void:
   pass

In there, addon developers can add custom code to check if a new version is available, as well as code to auto-update it. This approach considers that each developer may have their own custom versioning (so it is up to them to know what it means to have a new version), as well as providing a way to check/download addon versions.

Q & A

• How can a user update addons individually?
  • this could be a button next to the version number
• How can a user understand what has changed during an update?
  • Another idea I had was to provide a dedicated update window (similar to IntelliJ) where you can select addon updates and see a dedicated patchnotes UI, rather than cluttering the existing Plugins tab.
• Many users still require the same code/mechanism to download their addons - how can we prevent code duplication?
  • Some of the helper functions, such as _version_to_number or _get_current_version() could be exposed somehow inside EditorPlugin or being made available via a helper node
• What about auto-updating plugins (e.g. on startup)?
  • Auto-updating is very handy but it could expose developers to very unsafe scenarios where certain plugins might be dodgy and unknowingly they update themselves and download malicious code. That being said, the responsibility is still on the user to ensure that they trust the addon

If this enhancement will not be used often, can it be worked around with a few lines of script?

N/A

Is there a reason why this should be core and not an add-on in the asset library?

This is already part of addons but it is annoying and cumbersome from an "update journey" perspective.

[coppolaemilio]

I like the idea, this was also recently implemented in Dialogic and it would have been nice to have a standardized way of doing it, but there is one thing that needs to be discussed:

How/where is it calling for updates? If the plugin is going to be "calling home", we need to inform the Godot user that it will be doing that and you have to approve/trust the dev individually. Because if I specify from my plugin to send data to my server to check for updates, you can also send some extra data without the user knowing.

I know this issue is also present in the current way; a plugin developer that implements their own auto-update feature could also do the same, but if we are providing a "built-in" way of doing it integrated from Godot's UI, we should consider how to properly manage permissions.

Other than that, I really like the idea of standardizing this, and it is something that I'm sure we'll have to be implemented in one way or another eventually.

[Ark2000]

totally agree. godot should have something like pip to python a decent way to manage dependencies.

[Jeremi360]

There are already few addons/tool that try to be npm for godot:

• https://github.com/imjp94/gd-plug
• https://github.com/imjp94/gd-plug-ui
• https://github.com/Jummit/godot-package-manager @Ark2000 something for you

Godot should use someting like this.

Also some addons like those for example:

• https://github.com/naiiveprojects/nv.gd.file_system
• https://github.com/Maran23/script-ide
• https://github.com/imjp94/gd-blender-3d-shortcuts

should be instalable on editor side and not that developer must to install it per project.

[bitbrain]

This proposal does not aim to provide a dependency management tool but rather to provide an update mechanism from within the editor.

[Structed]

I love how you make this specifically only about updating, and not dependency management. Dependency management is super hard and with the current add-on infrastructure even harder. Therefore this could be a simple way to provide the core functionality of updating in a very self-contained fashion!

To @coppolaemilio's point of where it checks for the version: I think we should limit this built-in functionality to the AssetLib, and therefore check the AssetLib for the version.

[Calinou]

This requires something like #7925 to be implemented, as there is no way for add-ons to declare a method to check for updates (let alone in a decentralized manner). Keep in mind some assets aren't offered on the asset library (such as some paid assets being offered on itch.io), and you should be able to check for updates for those too (even if there is no autoupdate functionality provided).

Making plugins integrate their own updater code is bound to cause a lot of issues (including potential security issues), so I think it should work in a purely declarative way.

[coppolaemilio]

To @coppolaemilio's point of where it checks for the version: I think we should limit this built-in functionality to the AssetLib, and therefore check the AssetLib for the version.

The issue with only checking with the Asset Library is that the user could change the lib they are pulling from, and a user could just download an addon directly from a website and add it to their projects without the AssetLib. So I imagine the plugin developer should be able to have a say about where it will be pulling the update info from.

I guess the default can be the asset lib, but we should also have in mind to provide other sources.

[bitbrain]

Making plugins integrate their own updater code is bound to cause a lot of issues (including potential security issues), so I think it should work in a purely declarative way.

This proposal purposefully goes down that path of having to integrate your own updater code to provide the flexibility of using various download sources and avoid early standardisation. Developers of plugins already offer updaters out of the box (which imposes security risks) so rather than preventing users from doing certain things, I advocate the opposite. However, the security concerns both raised by @coppolaemilio and @Calinou are very valid. To combat this, we may need some sort of functionality that when reaching out to a "non trusted" source, Godot itself prompts the user with a popup that cannot be circumvented by the addon developer. IntelliJ has something like this as well:

Plugin 'not-a-bitcoin-miner' trying to download file 98172937123.i-wanna-get-rich.ws/not-a-bitcoin-miner-i-swear.gz
from location 98172937123.i-wanna-get-rich.ws - are you trusting this host?

[NO] [ONCE] [ALWAYS]

This at least gives the user the control over when something gets downloaded. I believe this is the best trade off between allowing both users and addon developers enough freedom but also giving them responsibility of making sure that they don't go off and download dodgy files.

[MikeSchulze]

Thank you for creating this suggestion. This was exactly my problem, and I had to write my own update mechanism for my plugin.
What I would expect from an auto-update plugin in the editor would be:

• Allow to enable/disable for each installed plugin the auto update on/off
• Show a notification on editor startup for plugins that has update enabled and the installed version is lower than the latest release version, The notification should show the release nodes.
• An option to downgrade an installed version to a previous version.
• Auto restart option after plugin is installed.

I would therefore suggest introducing a new plugin tool class that you can optionally define in your plugin.
To start minimalistic, suggest supporting these three repository types.

enum Repository {
	GODOT_ASSET_LIB,
	GIT_HUB,
	GIT_LAB,
}

The new Godot tool class could be named EditorPluginUpdateProvider

@tool
extends EditorPluginUpdateProvider

## Provides the repository type used to request plugin update information
func _plugin_update_provider() -> Repository:
	return Repository.GIT_HUB

## Provides the repository URL used by the editor for the update request:
## * Collect the release versions (to select a version that you want to update/downgrade)
## * The plugin relase info to be displayed
## * To download and install the selected plugin version
func _plugin_update_provider_url() -> String:
	return "https://api.github.com/repos/MikeSchulze/gdUnit4"


## Converts the release tag provided by the repository to the Godot plugin version, if required.
func _plugin_update_version(release_version :String ) -> String:
	var plugin_version :String
	# do convert the given version into Godot plugin verion
	return plugin_version


## If this option is set to true, the Godot Editor is automatically restarted after the plugin has been updated.
func _plugin_update_restart() -> bool:
	return true

## Returns the Godot version that is compatible with this plugin.
## A range of minimum and maximum supported Godot versions.
func _plugin_update_compatiblity() -> Array[String]:
	return ["4.1.0", "4.2.0"]

[Loufe]

Godot is sorely lacking a more mature system for managing plugins. I think showing a pop-up that installed add-ons are out of date according to the assetlib version is a minimum step while waiting for something better to come along. Here's why I have issues with my team:

• The 6 assetlib addons I use in my project need to be excluded in git as it's a huge complexity add during merges we can't afford.
• For the 6 addons I want a member to install, how best do I get my team to install the same addons? Copy-pasting the URL from the website assetlib into the editor assetlib doesn't work. You have to post the name AND the author AND the version sometimes into the search to find the exact plugin we're using. I post a picture of each. Some addons still have broken paths and need to be manually fixed every time (so frustrating).
• If I instead gave them a zip to install, I'd have to maintain it up to date. The "update" feature doesn't work for me, and it would take just as much time to have a player update every single addon as to go and download them fresh.
• I can't include a "autodownload" list of addons in the definition of the project.
• Import file changes after an plugin updates are a mess and we oftentimes deal with hundreds of them being generated for a single version as one person on the team doesn't update one of their addons and it changes the import format when they work with it next.

I love Godot and I'm involved in helping maintain several addons, but the system needs to be improved.

[TokisanGames]

The 6 assetlib addons I use in my project need to be excluded in git as it's a huge complexity add during merges we can't afford.

@Loufe It sounds like all of your issues extend from not having your plugins in git. I have a large team and many plugins, all in git without issue. I maintain the exact version of each plugin I want in the repo. I'm also a plugin dev. When I'm ready with a new plugin version that I've tested, my own or others, I upgrade it in the repo. Everyone downloads updates as normal.

The only exception is we have one C# plugin that only 3 people use with the C# version of Godot. We have excluded that from git, and those 3 people have local installations of the plugin, after I send them the zip. This is purely because that plugin has a commercial license. Otherwise we would put it in our repo. Everyone else uses only the GDScript and C++ plugins.

Managing plugins in Git is not any different for us than any of the other code or assets in our project. They're just files in another directory.

[rsubtil]

@Loufe you can also check a plugin manager-like addon, seems like it would help in some of the pitfalls you're facing by not git-including all addons, such as https://github.com/imjp94/gd-plug

[donn-xx]

Godot can already go to a web-server (the Asset Library), so "phoning-home" is moot. This proposal is great.

[domske]

You can't even download the plugin again (from Asset Libary) to update. You need to delete the addon folder first, to resolve the conflicts. We need a overwrite option. But I would clear the folder first. Otherwise you'll have old dead files in the folder. A plugin download always contains all the required files.

The plugin.cfg contains the version and name. But the URL looks like so:
https://godotengine.org/asset-library/asset/3313
There is only the ID and the plugin name is no unique. Godot does not know the ID of installed addons, right? So it could either create a new meta file in the directory or add the ID to plugin.cfg. Then you have all information for the update. Btw. I would prefer to manually check for update. Then we could do the following:

1. Use the ID to get the information from server. Like the version.
2. Compare the plugin version from server with the locally installed version. If the string is different, then flag for update.
3. Implement the update process. Download already works. But we need a clear folder before install or overwrite. Voilà.

My current update process is to find the plugin in Asset Libary, then delete the addon folder and download again. Then restart Godot. So the updater needs to be reload the plugins or inform about restart.

[donn-xx]

Voilà.
Brilliant and KISS. Chef's kiss.

[MikeSchulze]

I don't love to look up the Godot asset Library, many plugins are developed in GitHub or GitLab.
To update the Asset Library is a kind of pain.
The update should be taken this into account.

I'm the dev of GdUnit4 it already provides an update tool, it works in these steps.

• check if a new release available on the configured GitHub URL
• download the release info and display it to the user
• if you press the update button than
  • download the release package
  • extract into temp
  • if all success
    • disable the plugin
    • delete the plugin
    • move temp plugin into the project
    • activate the plugin
    • restart Godot

This works fine since around 4 years.

[donn-xx]

True that. But if the Asset Library improves it will be the better source for trusted addons etc. I also worry that "Git**b" will get hardcoded in and relying on anything as much as we do on Github is always a bad idea.