Merge remote-tracking branch 'origin/main'

exasol · Oct 14, 2024 · c76ef87 · c76ef87
2 parents 898f06c + a381520
commit c76ef87
Show file tree

Hide file tree

Showing 8 changed files with 572 additions and 453 deletions.
diff --git a/doc/changes/changelog.md b/doc/changes/changelog.md
@@ -1,6 +1,7 @@
 # 📝 Changes
 
 * [unreleased](unreleased.md)
+* [0.8.0](changes_0.8.0.md)
 * [0.7.0](changes_0.7.0.md)
 * [0.6.0](changes_0.6.0.md)
 * [0.5.0](changes_0.5.0.md)
@@ -15,6 +16,7 @@
 hidden:
 ---
 unreleased
+changes_0.8.0
 changes_0.7.0
 changes_0.6.0
 changes_0.5.0

diff --git a/doc/changes/changes_0.8.0.md b/doc/changes/changes_0.8.0.md
@@ -0,0 +1,17 @@
+# 0.8.0 - 2024-10-14
+
+## Features
+
+* #52: Added timeout options for SLC deployer to CLI
+* #79 Added function `get_cli_arg` that makes a string CLI argument from an option and its value.
+  Also allowed passing an option name instead of the `StdParams` in the following two functions:
+  `create_std_option`, `check_params`.
+
+## Bug fixing
+
+* #78 Missing default value in the definition of `StdParams.path_in_bucket`.
+
+## Documentation
+
+* #81 Updated the documentation on the CLI commands, following the introduction of the standard
+  CLI parameters.
diff --git a/doc/changes/unreleased.md b/doc/changes/unreleased.md
@@ -1,12 +1 @@
 # Unreleased
-
-## Features
-
-* #52: Added timeout options for SLC deployer to CLI
-* #79 Added function `get_cli_arg` that makes a string CLI argument from an option and its value.
-  Also allowed passing an option name instead of the `StdParams` in the following two functions:
-  `create_std_option`, `check_params`.
-
-## Bug fixing
-
-* #78 Missing default value in the definition of `StdParams.path_in_bucket`.
diff --git a/doc/user_guide/user-guide.md b/doc/user_guide/user-guide.md
@@ -4,21 +4,21 @@
 
 An extension would typically use [UDF scripts](https://docs.exasol.com/db/latest/database_concepts/udf_scripts.htm)
 to enable certain custom functionality within a database. In most cases, UDF scripts must be backed by a
-[Script Language Container (SLC)](https://github.com/exasol/script-languages-release/), that must be installed in the
+[Script Language Container (SLC)](https://github.com/exasol/script-languages-release/), that needs to be installed in the
 Exasol Database. An SLC allows the installation of the chosen programming language and necessary dependencies in the
 Exasol Database.
 
-The language container for a particular Extension gets downloaded and installed by executing a deployment script
-similar to the one below.
+The deployment of the language container is typically included in the installation of the Extension. It should also
+be possible to install the SLC separately using the same or a different CLI command. Please check the user guide of
+a particular extension for details. Below is an example of an extension installation command.
 
   ```buildoutcfg
-  python -m <exasol_extension>.deploy language-container <options>
+  python -m <exasol_extension>.deploy <options>
   ```
 
-The name of the script (```<exasol_extension>.deploy``` in the above command) and the command name
-(e.g. ```language-container```) can vary from one extension to another. Please check the user guide of a particular
-extension. The rest of the command line will have a common format. It will include some of the options defined below.
-The choice of options is primarily determined by the storage backend being used - On-Prem or SaaS.
+The name of the script (```<exasol_extension>.deploy``` in the above command) can vary from one extension to another.
+The rest of the command line will have a common format. It will include some of the options defined below. The choice
+of options is primarily determined by the storage backend being used - On-Prem or SaaS.
 
 ### List of options
 
@@ -32,36 +32,38 @@ an option in the command line, without providing its value. In this case, the co
 interactively. For long values, such as the SaaS account id, it is more practical to copy/paste the value from
 another source.
 
-| Option name                  | On-Prem | SaaS | Comment                                                 |
-|:-----------------------------|:-------:|:----:|:--------------------------------------------------------|
-| dsn                          |   [x]   |      | i.e. <db_host:db_port>                                  |
-| db-user                      |   [x]   |      |                                                         |
-| db-pass                      |   [x]   |      | Env. [DB_PASSWORD]                                      |
-| bucketfs-name                |   [x]   |      |                                                         |
-| bucketfs-host                |   [x]   |      |                                                         |
-| bucketfs-port                |   [x]   |      |                                                         |
-| bucketfs-user                |   [x]   |      |                                                         |
-| bucketfs-password            |   [x]   |      | Env. [BUCKETFS_PASSWORD]                                |
-| bucketfs-use-https           |   [x]   |      | Optional boolean, defaults to False                     |
-| bucket                       |   [x]   |      |                                                         |
-| saas-url                     |         | [x]  | Env. [SAAS_HOST]                                        |
-| saas-account-id              |         | [x]  | Env. [SAAS_ACCOUNT_ID]                                  |
-| saas-database-id             |         | [x]  | Optional, Env. [SAAS_DATABASE_ID]                       |
-| saas-database-name           |         | [x]  | Optional, provide if the database_id is unknown         |
-| saas-token                   |         | [x]  | Env. [SAAS_TOKEN]                                       |
-| path-in-bucket               |   [x]   | [x]  |                                                         |
-| language-alias               |   [x]   | [x]  |                                                         |
+| Option name                  | On-Prem | SaaS | Comment                                                            |
+|:-----------------------------|:-------:|:----:|:-------------------------------------------------------------------|
+| dsn                          |   [x]   |      | i.e. <db_host:db_port>                                             |
+| db-user                      |   [x]   |      |                                                                    |
+| db-pass                      |   [x]   |      | Env. [DB_PASSWORD]                                                 |
+| bucketfs-name                |   [x]   |      |                                                                    |
+| bucketfs-host                |   [x]   |      |                                                                    |
+| bucketfs-port                |   [x]   |      |                                                                    |
+| bucketfs-user                |   [x]   |      |                                                                    |
+| bucketfs-password            |   [x]   |      | Env. [BUCKETFS_PASSWORD]                                           |
+| bucketfs-use-https           |   [x]   |      | Optional boolean, defaults to False                                |
+| bucket                       |   [x]   |      |                                                                    |
+| saas-url                     |         | [x]  | Env. [SAAS_HOST]                                                   |
+| saas-account-id              |         | [x]  | Env. [SAAS_ACCOUNT_ID]                                             |
+| saas-database-id             |         | [x]  | Optional, Env. [SAAS_DATABASE_ID]                                  |
+| saas-database-name           |         | [x]  | Optional, provide if the database_id is unknown                    |
+| saas-token                   |         | [x]  | Env. [SAAS_TOKEN]                                                  |
+| path-in-bucket               |   [x]   | [x]  |                                                                    |
+| language-alias               |   [x]   | [x]  |                                                                    |
 | schema                       |   [x]   | [x]  | Required if the user has no permission to create a database schema |
-| version                      |   [x]   | [x]  | Optional, provide for downloading SLC from GitHub       |
-| container-file               |   [x]   | [x]  | Optional, provide for uploading SLC file                |
-| ssl-cert-path                |   [x]   | [x]  | Optional                                                |
-| [no-]use-ssl-cert-validation |   [x]   | [x]  | Optional boolean, defaults to True                      |
-| ssl-client-cert-path         |   [x]   |      | Optional                                                |
-| ssl-client-private-key       |   [x]   |      | Optional                                                |
-| [no-]upload-container        |   [x]   | [x]  | Optional boolean, defaults to True                      |
-| [no-]alter-system            |   [x]   | [x]  | Optional boolean, defaults to True                      |
-| [no-]allow-override          |   [x]   | [x]  | Optional boolean, defaults to False                     |
-| [no-]wait_for_completion     |   [x]   | [x]  | Optional boolean, defaults to True                      |
+| version                      |   [x]   | [x]  | Optional, provide for downloading SLC from GitHub                  |
+| container-file               |   [x]   | [x]  | Optional, provide for uploading SLC file                           |
+| ssl-cert-path                |   [x]   | [x]  | Optional                                                           |
+| [no-]use-ssl-cert-validation |   [x]   | [x]  | Optional boolean, defaults to True                                 |
+| ssl-client-cert-path         |   [x]   |      | Optional                                                           |
+| ssl-client-private-key       |   [x]   |      | Optional                                                           |
+| [no-]upload-container        |   [x]   | [x]  | Optional boolean, defaults to True                                 |
+| [no-]alter-system            |   [x]   | [x]  | Optional boolean, defaults to True                                 |
+| [no-]allow-override          |   [x]   | [x]  | Optional boolean, defaults to False                                |
+| [no-]wait-for-completion     |   [x]   | [x]  | Optional boolean, defaults to True                                 |
+| deploy-timeout-minutes       |   [x]   | [x]  | Defaults to 10 minutes.                                            |
+| [no-]display-progress        |   [x]   | [x]  | Optional boolean, defaults to True                                 |
 
 ### Container selection
 
@@ -109,3 +111,13 @@ has already been uploaded one can use the `--no-upload-container` option to skip
 By default, overriding language activation is not permitted. If a language with the same alias has already
 been activated the command will result in an error. The activation can be overridden with the use of
 the `--allow-override` option.
+
+## BucketFS connection object
+
+Some extensions require the user to create a BucketFS connection object encapsulating the BucketFS credentials.
+The creation of such an object in the database would also be a part of the extension installation. Like the
+language container, the connection object can be created separately, using the same or a different command.
+Please check the documentation of a particular extension for details.
+
+Most of the options listed above in the Language Container Deployer sections are also relevant for the creation
+of the connection object, should it be performed in a separate command.
diff --git a/exasol/python_extension_common/cli/language_container_deployer_cli.py b/exasol/python_extension_common/cli/language_container_deployer_cli.py
@@ -1,6 +1,8 @@
 from pathlib import Path
+from datetime import timedelta
 
-from exasol.python_extension_common.deployment.language_container_deployer import LanguageContainerDeployer
+from exasol.python_extension_common.deployment.language_container_deployer import (
+    LanguageContainerDeployer, display_extract_progress, ExtractValidator)
 from exasol.python_extension_common.connections.pyexasol_connection import open_pyexasol_connection
 from exasol.python_extension_common.connections.bucketfs_location import create_bucketfs_location
 from exasol.python_extension_common.cli.std_options import StdParams
@@ -35,10 +37,17 @@ def __call__(self, **kwargs):
         alter_system = kwargs[StdParams.alter_system.name]
         allow_override = kwargs[StdParams.allow_override.name]
         wait_for_completion = kwargs[StdParams.wait_for_completion.name]
+        deploy_timeout_minutes = kwargs[StdParams.deploy_timeout_minutes.name]
+        display_progress = kwargs[StdParams.display_progress.name]
 
+        display_callback = display_extract_progress if display_progress else None
+        extract_validator = ExtractValidator(pyexasol_connection,
+                                             timedelta(minutes=deploy_timeout_minutes),
+                                             callback=display_callback)
         deployer = LanguageContainerDeployer(pyexasol_connection,
                                              language_alias,
-                                             bucketfs_location)
+                                             bucketfs_location,
+                                             extract_validator)
         if not upload_container:
             deployer.run(alter_system=alter_system,
                          allow_override=allow_override,