Merge remote-tracking branch 'upstream/master' into devonfw#1198-abil…

…ity-to-download-custom-software-from-mounted-filesystem

.github/workflows/update-urls.yml

@@ -10,6 +10,8 @@ jobs:
     steps:
       - name: Checkout ide
         uses: actions/checkout@v3
+        with:
+          submodules: recursive
       - name: Checkout ide-urls
         uses: actions/checkout@v3
         with:
@@ -18,7 +20,7 @@ jobs:
           token: ${{ secrets.ACTION_PUSH_TOKEN }}
       - name: Build and run url updater
         run: |
-          cd url-updater
+          cd ide
           mvn -B -ntp -Dstyle.color=always install
           mvn -B -ntp -Dstyle.color=always exec:java -Dexec.mainClass="com.devonfw.tools.ide.url.UpdateInitiator" -Dexec.args="../ide-urls PT5H30M"
       - name: Commit and push to ide-urls

.mvn/maven.config

@@ -1 +1 @@
--Drevision=2023.07.003-SNAPSHOT
+-Drevision=2023.08.002-SNAPSHOT

CHANGELOG.asciidoc

@@ -2,12 +2,36 @@
 
 This file documents all notable changes to https://github.com/devonfw/ide[devonfw-ide].
 
+== 2023.08.002
+
+Release with new features and bugfixes:
+
+* TODO
+
+The full list of changes for this release can be found in https://github.com/devonfw/ide/milestone/48?closed=1[milestone 2023.08.002].
+
+== 2023.08.001
+
+Release with new features and bugfixes:
+
+* https://github.com/devonfw/ide/issues/1297[#1297]: doFail from debugging in productive code prevents MacOS installations
+
+The full list of changes for this release can be found in https://github.com/devonfw/ide/milestone/47?closed=1[milestone 2023.08.001].
+
 == 2023.07.003
 
 Release with new features and bugfixes:
 
+* https://github.com/devonfw/ide/issues/1287[#1287]: initial setup fails because urls is not cloned
+* https://github.com/devonfw/ide/issues/1279[#1279]: Improve rancher-desktop support and solve VPN issue
+* https://github.com/devonfw/ide/issues/1252[#1252]: Docker not working on linux
+* https://github.com/devonfw/ide/issues/1295[#1295]: Docker setup failing on MacOS
 * https://github.com/devonfw/ide/issues/1247[#1247]: Fix Jenkins, SonarQube, Jasypt is not working without args
 * https://github.com/devonfw/ide/issues/1246[#1246]: Fix yarn installation
+* https://github.com/devonfw/ide/issues/1283[#1283]: checksum verification prevents intellij plugin installation
+* https://github.com/devonfw/ide/issues/1276[#1276]: checksum verification for custom software repository
+* https://github.com/devonfw/ide/issues/1285[#1285]: Complete alignment of tool-names in scripts
+* https://github.com/devonfw/ide/issues/1164[#1164]: devon jenkins stop fails
 
 The full list of changes for this release can be found in https://github.com/devonfw/ide/milestone/46?closed=1[milestone 2023.07.003].
 

README.asciidoc

@@ -39,9 +39,8 @@ sudo yum install git-core curl
 *** On MacOS you only need to download and install https://git-scm.com/download/mac[git for mac].
 
 === Install
-NOTE: Currently this sonatype download service is broken (`here` link below will not work). As a workaround go to https://github.com/devonfw/ide/releases[releases] and click the latest direct download link. Sorry for any inconvenience.
 
-** Download the latest version of `devonfw-ide` from https://repository.sonatype.org/service/local/artifact/maven/redirect?r=central-proxy&g=com.devonfw.tools.ide&a=devonfw-ide-scripts&v=LATEST&p=tar.gz[here] and extract it to a folder of your choice.
+** Download the latest version of `devonfw-ide` from https://github.com/devonfw/ide/releases[here] and extract it to a folder of your choice.
 +
 (You can find all Releases in https://repo.maven.apache.org/maven2/com/devonfw/tools/ide/devonfw-ide-scripts/[Maven Central])
 ** Run the `setup` in this folder (On Windows you can double click on `setup.bat`)

documentation/DoD.asciidoc

@@ -11,7 +11,7 @@ However, chances to get your change merged quickly are higher if you address all
 ** [ ] PR title is of the form `#«issue-id»: «brief summary»` (e.g. `#921: fixed setup.bat`).
 ** [ ] PR top-level comment summarizes what has been done and contains link to addressed issue(s).
 ** [ ] PR is linked with the issue(s) that it implements and resolves (see sidebar, click on `Development` and enter issue ID)
-** [ ] PR and issue(s) have suitable labels (OS-specific like `windows`, `macOS`, or `linux` as well as other aspects like `commandlet`, `plugins`, `mirrors`, or tool specific labels such as `eclipse` or `java`)
+** [ ] PR and issue(s) have suitable labels (OS-specific like `windows`, `macOS`, or `linux` as well as other aspects like `commandlet`, `plugins`, `urls`, or tool specific labels such as `eclipse` or `java`)
 ** [ ] Issue is assigned to you (should actually be done before you start your implementation).
 ** [ ] Milestones are typically assigned by PO (Jörg) but have to be set before issues and PRs get closed. In case your issue is only addressed by a change in `ide-mirrors`/`ide-urls` or in `ide-settings` repository, please already assign the according milestone (switch to `closed` tab under `Milestone`).
 * [ ] All checks have passed. Otherwise if a check failed (red cross) you need to click the `Details` link, read the logs and fix the problem.
@@ -22,7 +22,7 @@ However, chances to get your change merged quickly are higher if you address all
 * [ ] You followed all coding conventions (indent with 2 spaces, no tabs, functions start with `do` prefix, variables are enclosed with curly braces, variables in functions are declared with `local`, etc.)
 * [ ] You have already added the issue implemented by your PR in https://github.com/devonfw/ide/blob/master/CHANGELOG.asciidoc[CHANGELOG.asciidoc] to the next open release (see milestones or https://github.com/devonfw/ide/blob/master/.mvn/maven.config[maven.config]). If there is no issue for your PR consider creating it or directly link the PR itself. Please note that the CHANGELOG shall only reflect public changes relevant for end-users. So e.g. if we implement a story and then add another PR as bugfix or improvement to the same story, we do not need to document this in the CHANGELOG to avoid spam and confusion.
 * [ ] In case your PR adds a new tool `«tool»` as commandlet there are the following additional checks:
-** [ ] The tool is already available as folder `«tool»` in https://github.com/devonfw/ide-mirrors[ide-mirrors] including `available-versions`.
+** [ ] A new urlUpdater named `«tool»` has been added to a a folder named `«tool»` in https://github.com/devonfw/ide/tree/master/url-updater/src/main/java/com/devonfw/tools/ide/url/updater[updater] that receives response bodies from URLs, updates tool versions, and checks if download URLs work.
 ** [ ] A new commandlet named `«tool»` has been added to https://github.com/devonfw/ide/tree/master/scripts/src/main/resources/scripts/command[command] that allows to install and launch the given software.
 ** [ ] The tool can be installed automatically (during setup via settings) or via the commandlet call.
 ** [ ] During the installation process (`devon «tool» setup`) reasonable progress and status information are displayed.

documentation/LICENSE.asciidoc

@@ -73,6 +73,7 @@ The following table shows the components that may be used. The column `inclusion
 |https://github.com/openrewrite/rewrite-maven-plugin[Openrewrite]|Optional|https://github.com/openrewrite/rewrite-maven-plugin/blob/main/LICENSE[ASL 2.0]
 |https://kotlinlang.org//[kotlin]|Optional|https://github.com/JetBrains/kotlin-web-site/blob/master/LICENSE[ASL 2.0]
 |https://tomcat.apache.org/[tomcat]|Optional|https://www.apache.org/licenses/LICENSE-2.0[ASL 2.0]
+| https://www.oracle.com/java/technologies/jdk-mission-control.html[Java Mission Control]|Optional|https://github.com/openjdk/jmc/blob/master/license/LICENSE.txt[UPL 1.0] and https://github.com/openjdk/jmc/blob/master/license/THIRDPARTYREADME.txt[EPL 2.0]
 |=======================
 
 == Apache Software License - Version 2.0
@@ -3026,3 +3027,70 @@ Exhibit B - “Incompatible With Secondary Licenses” Notice
       With Secondary Licenses”, as defined by
       the Mozilla Public License, v. 2.0.
 ```
+== The Universal Permissive License, Version 1.0
+```
+Copyright (c) 2018, 2022, Oracle and/or its affiliates. All rights reserved.
+
+The Universal Permissive License (UPL), Version 1.0
+
+Subject to the condition set forth below, permission is hereby granted to any
+person obtaining a copy of this software, associated documentation and/or data
+(collectively the "Software"), free of charge and under any and all copyright
+rights in the Software, and any and all patent rights owned or freely
+licensable by each licensor hereunder covering either (i) the unmodified
+Software as contributed to or provided by such licensor, or (ii) the Larger
+Works (as defined below), to deal in both
+
+(a) the Software, and
+
+(b) any piece of software and/or hardware listed in the lrgrwrks.txt file if
+    one is included with the Software (each a “Larger Work” to which the
+    Software is contributed by such licensors), without restriction, including
+    without limitation the rights to copy, create derivative works of,
+    display, perform, and distribute the Software and make, use, sell, offer
+    for sale, import, export, have made, and have sold the Software and the
+    Larger Work(s), and to sublicense the foregoing rights on either these or
+    other terms.
+
+This license is subject to the following condition:
+
+The above copyright notice and either this complete permission notice or at a
+minimum a reference to the UPL must be included in all copies or substantial
+portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+
+Copyright (c) 2018 Oracle America, Inc. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice,
+   this list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its contributors
+   may be used to endorse or promote products derived from this software
+   without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+```

documentation/cli.asciidoc

@@ -71,6 +71,7 @@ The following commandlets are currently available:
 * link:jasypt.asciidoc[jasypt]
 * link:java.asciidoc[java]
 * link:jenkins.asciidoc[jenkins]
+* link:jmc.asciidoc[jmc]
 * link:kotlinc.asciidoc[kotlinc]
 * link:kotlinc-native.asciidoc[kotlinc-native]
 * link:kubectl.asciidoc[kubectl]

documentation/configuration.asciidoc

@@ -12,7 +12,6 @@ The `devonfw-ide` aims to be highly configurable and flexible. The configuration
 5. `https://github.com/devonfw/ide-settings/blob/master/devon.properties[settings/devon.properties]` (`${SETTINGS_PATH}/devon.properties`) - project specific configurations from link:settings.asciidoc[settings].
 6. `workspaces/${WORKSPACE}/devon.properties` - optional workspace specific configurations (especially helpful in projects using docker).
 7. `https://github.com/devonfw/ide-settings/blob/master/devon/conf/devon.properties[conf/devon.properties]` - user specific configurations (e.g. `M2_REPO=~/.m2/repository`). During setup this file is created by copying a template from `${SETTINGS_PATH}/devon/conf/devon.properties`.
-8. `https://github.com/devonfw/ide-settings/tree/master/projects[settings/projects/*.properties]`- properties to configure link:projects.asciidoc[project checkout and import]
 
 == devon.properties
 

documentation/devonfw-ide-contribution-getting-started.asciidoc

@@ -79,29 +79,28 @@ For Mac and Linux, ShellCheck is automatically called during the build process (
 
 == Create a new commandlet
 
-=== IDE-Mirrors
-Many programs have a download link that depends on the operating system. 
+=== Urls
+Many programs have a download link that depends on the operating system.
 
-For a new tool create a folder in https://github.com/devonfw/ide-mirrors[ide-mirrors] repository named like the tool and the commandlet with the following files:
+For a new tool, create an url-updater in the https://github.com/devonfw/ide/tree/master/url-updater/src/main/java/com/devonfw/tools/ide/url/updater[updater] folder in the current repository. The Name needs to be equal to the one of both the tool and the commandlet.
 
-* `urls` contains the download URL(s) that typically contain placeholders like `${ext}` for extension, `${os}` for operating-system, `${arch}` for CPU architecture, or `${code}` for commandlet specific things. The file can hold multiple URLs - one per line. This is used if multiple https://en.wikipedia.org/wiki/Mirror_site[mirrors] are available so the load gets distributed. Therefore devonfw-ide will shuffle the URLs into random order and then try them sequentially so if one URL does not work (e.g. mirror unavailable) it will try the next one.
-* `ext-mapping` contains the mapping from operating system (OS) to the according file extension filled into the `${ext}` placeholder of the download for that OS.
-* `os-mapping` contains the remapping from OS to the value filled into the `${os}` placeholder of the URL. So you can e.g. remap `windows=win` or `mac=darwin` in case the download URLs do not follow common practices. You can even omit this file to fall back to the default https://github.com/devonfw/ide-mirrors/blob/master/os-mappings[os-mappings].
-* `arch-mapping` contains the remapping from architecture to the value filled into the `${arch}` placeholder of the URL. So you can e.g. remap `aarch64=arm64` or `x86_64=x64` in case the download URLs do not follow common practices. Simply omit this file if not needed.
-* `versions` in very special cases, you can create separate configurations in sub-folders for specific versions or version-ranges. Here you can specify a folder-name followed by a less-sign (`<`) and a version number to specify that if the version to install is less than this given version, the configuration will be read from that specified folder. The lines are processed top-down where the first match applies. An example can be found https://github.com/devonfw/ide-mirrors/blob/master/java/versions[here].
-* `versions-url` contains the url where all available versions can be found. If possible, check if the tool is available via the GitHub API (`https://api.github.com/repos/<organisation>/<repository>/git/refs/tags`). If that's not possible, provide a different url as well as the `versions-url-op` for this url.
-* `versions-url-op` contains the command used to extract the versions from the response of `curl $versions-url` in a suitable format. This file is always required if you didn't use the GitHub API for the `versions-url`. In case you used the GitHub API, you only have to create this file if the standardized approach doesn't work (e.g. versions have the wrong format, there are versions left that need to be removed, ...).
+There are four available parent classes of url-updaters, that can be used. They mainly differentiate in the way, available versions are detected.
 
-The file `available-versions` doesn't have to be created manually. Using the `versions-url` and `versions-url-op`, `available-versions` will be generated and updated automatically by a GitHub Actions workflow.
+* https://github.com/devonfw/ide/blob/master/url-updater/src/main/java/com/devonfw/tools/ide/url/updater/GithubUrlUpdater.java[`GithubUrlUpdater`]
 
-It is important that the line endings of all files are UNIX `\n`.
+* https://github.com/devonfw/ide/blob/master/url-updater/src/main/java/com/devonfw/tools/ide/url/updater/JsonUrlUpdater.java[`JsonUrlUpdater`]
 
-Note that all rules within a mapping are checked from top to bottom.
-If the first rule matches, the other rules are ignored. Therefore, the *-rule should always be last.
-With the `os-mapping` the *-rule should be used for the Linux distributions, since the OS type is not only linux, but varies strongly.
+* https://github.com/devonfw/ide/blob/master/url-updater/src/main/java/com/devonfw/tools/ide/url/updater/MavenBasedUrlUpdater.java[`MavenBasedUrlUpdater`]
 
-It is also possible to use variables in the mapping files (esp. in `os-mapping` that happens first before other variables get replaced). 
-For example, if the architecture in the download link only appears for Windows, `{arch}` can be used in the `os-mapping. For an example see https://github.com/devonfw/ide-mirrors/blob/master/rancher/os-mappings[here].
+* https://github.com/devonfw/ide/blob/master/url-updater/src/main/java/com/devonfw/tools/ide/url/updater/WebsiteUrlUpdater.java[`WebsiteUrlUpdater`]
+
+Note: The `WebsiteUrlUpdater` should not be used if other viable options are available.
+
+The updaters use the function `doAddVersion()`, where the downloadUrl, OS and architecture can be specified for each version. https://github.com/devonfw/ide/blob/master/url-updater/src/main/java/com/devonfw/tools/ide/url/updater/oc/OcUrlUpdater.java#L22C1-L24[Example]
+
+`VersionIdentifiers` can also be used if e.g. the downloadUrl changed after a specific version, or certain version are not indented to be available. https://github.com/devonfw/ide/blob/master/url-updater/src/main/java/com/devonfw/tools/ide/url/updater/helm/HelmUrlUpdater.java#L12[Example]
+
+Once a day, the `UpdateInitiator` will run the updaters through GitHub Actions and fill the https://github.com/devonfw/ide-urls[ide-URL] repository with new version/tools that are identified by the Updaters.
 
 === Commandlet
 Use an existing commandlet like https://github.com/devonfw/ide/blob/master/scripts/src/main/resources/scripts/command/helm[HELM] as a reference.

documentation/devonfw-ide-usage.asciidoc

@@ -56,6 +56,8 @@ include::java.asciidoc[leveloffset=3]
 
 include::jenkins.asciidoc[leveloffset=3]
 
+include::jmc.asciidoc[leveloffset=3]
+
 include::kotlinc.asciidoc[leveloffset=3]
 
 include::kotlinc-native.asciidoc[leveloffset=3]