Merge pull request #36 from gqueiroz/feature-release-0.4.0

Review for release Version 0.4.0

CHANGES.rst

@@ -11,20 +11,51 @@ BDC-DB - Changes
 ================
 
 
+Version 0.4.0
+-------------
+
+
+Released 2020-08-31
+
+
+- New features:
+
+  - dynamic handling of triggers and scripts.
+
+  - new CLI commands: ``show-triggers``, ``create-triggers``, ``load-scripts``, and ``load-file``.
+
+
+- Improved documentation.
+
+
+
 Version 0.2.0
 -------------
 
+
 Released 2020-08-17
 
+
 - First experimental version.
+
 - Command Line Interface (CLI).
+
 - Alembic migration support.
+
 - Documentation system based on Sphinx.
+
 - Documentation integrated to ``Read the Docs``.
+
 - Installation and build instructions.
+
 - Package support through Setuptools.
+
 - Installation and usage instructions.
+
 - Travis CI support.
+
 - Unit-test environment set.
+
 - Source code versioning based on `Semantic Versioning 2.0.0 <https://semver.org/>`_.
-- License: `MIT <https://raw.githubusercontent.com/brazil-data-cube/bdc-db/master/LICENSE>`_.
+
+- License: `MIT <https://raw.githubusercontent.com/brazil-data-cube/bdc-db/master/LICENSE>`_.

USAGE.rst

@@ -16,38 +16,79 @@ Command-Line Interface (CLI)
 
 The ``BDC-DB`` extension installs a command line tool named ``bdc-db`` that groups a set of database commands under the group ``db``:
 
+- ``init``: Initialize a new database repository if it doesn't exist.
+
+- ``create-namespace``: Create the table namespace (schema) in database.
+
+- ``create-extension-postgis``: Enables the PostGIS extenion in the database.
+
 - ``create-schema``: Create the database schema (tables, primary keys, foreign keys).
 
-- ``drop-schema``: Drop the database schema (tables, primary keys, foreign keys).
+- ``create-triggers``: Create in the database all triggers registered in the extension.
 
-- ``init``: Initialize a new database repository if it doesn't exist.
+- ``load-scripts``: Load and execute database scripts.
+
+- ``drop-schema``: Drop the database schema (tables, primary keys, foreign keys).
 
 - ``destroy``: Drop the database repository.
 
-- ``create-namespace``: Create the table namespace (schema) in database.
+- ``show-triggers``: List all registred triggers.
 
-- ``create-extension-postgis``: Enables the PostGIS extenion in the database
+- ``load-file``: Load and execute a script file into database.
 
 
 Preparing a new Package with Alembic and BDC-DB
 -----------------------------------------------
 
 
-The Alembic commands are available trough the ``alembic`` command group. These are the the commands for managing upgrade recipes.
+The Alembic commands are available trough the ``alembic`` command group. These are the commands for managing upgrade recipes:
+
+- ``branches``: Show branch points.
+
+- ``current``: Show current revision.
+
+- ``downgrade``: Run downgrade migrations.
+
+- ``heads``: Show latest revisions.
+
+- ``log``: Show revision log.
+
+- ``merge``: Create merge revision.
+
+- ``mkdir``: Make migration directory.
+
+- ``revision``: Create new migration.
+
+- ``show``: Show the given revisions.
+
+- ``stamp``: Set current revision.
+
+- ``upgrade``: Run upgrade migrations.
+
+
+.. note::
+
+    For more information, type in the command line::
+
+        bdc-db alembic --help
 
 
 The ``BDC-DB`` follows the `Python Entry point specification <https://packaging.python.org/specifications/entry-points/>`_ to
 discover and load libraries dynamically.
 
 
-Basically, the ``BDC-DB`` has two major entry points to deal with dynamic SQLAlchemy models:
+Basically, the ``BDC-DB`` has the following entry points to deal with dynamic SQLAlchemy models and daabase scripts:
+
+- ``bdc_db.alembic``: The alembic migration folders.
+
+- ``bdc_db.models``: The initialization of your own models.
 
-- ``bdc_db.alembic`` - The alembic migration folders.
+- ``bdc_db.triggers``: A folder with SQL scripts to create triggers.
 
-- ``bdc_db.models`` - The initialization of your own models.
+- `bdc_db.scripts``: A folder with SQL scripts to be loaded and executed in the database.
 
 
-Both of them must be defined in the ``setup.py`` of your package if you would like to have database support.
+These entry points may be defined in the ``setup.py`` of your package if you would like to have database support.
 
 
 The following code is an example of an ``entry_points`` in ``setup.py`` file:
@@ -67,18 +108,15 @@ The following code is an example of an ``entry_points`` in ``setup.py`` file:
 
 .. note::
 
-    The package ``BDC-DB`` requires an instance of PostgreSQL listening up. You must set ``SQLALCHEMY_DATABASE_URI`` with your
-    own instance.
+    The package ``BDC-DB`` requires an instance of PostgreSQL listening up. You must set ``SQLALCHEMY_DATABASE_URI`` with your own instance.
 
 
 .. warning::
 
     When the entry points ``bdc_db.models`` and ``bdc_db.alembic`` is set, make sure you have the target values in your file system.
 
 
-To deal with migrations, you need to initialize the ``Alembic`` with the following command:
-
-.. code-block:: shell
+To deal with migrations, you need to initialize the ``Alembic`` with the following command::
 
     FLASK_APP=myapp flask alembic mkdir
 
@@ -101,10 +139,7 @@ You must follow the `SQLAlchemy Models <https://flask-sqlalchemy.palletsprojects
         version = db.Column(db.Integer())
 
 
-Once the model is set, you must generate a migration. To do that, use the command ``alembic revision``:
-
-
-.. code-block:: shell
+Once the model is set, you must generate a migration. To do that, use the command ``alembic revision``::
 
     FLASK_APP=myapp flask alembic revision "my app migration" --branch=myapp
 
@@ -118,8 +153,7 @@ The output will be something like::
 
 .. warning::
 
-    Whenever you create a revision with ``alembic revision`` command, make sure you have set the parameter ``--branch``
-    to ``BDC-DB`` put your migrations in the right place. Otherwise, it will move to ``site-packages/bdc_db/alembic``.
+    Whenever you create a revision with ``alembic revision`` command, make sure you have set the parameter ``--branch`` to ``BDC-DB``. This will put your migrations in the right place. Otherwise, it will move to ``site-packages/bdc_db/alembic``.
 
 
 Loading package SQL scripts SQLAlchemy and BDC-DB
@@ -129,7 +163,7 @@ Loading package SQL scripts SQLAlchemy and BDC-DB
 The ``BDC-DB`` also supports to load files ``.sql`` dynamically using `Python Entry point specification <https://packaging.python.org/specifications/entry-points/>`_.
 
 
-It is quite useful if you need to configure you environment, setting up `PostgreSQL Plpgsql Triggers <https://www.postgresql.org/docs/12/plpgsql-trigger.html>`_ and default script data.
+It is quite useful if you need to configure you environment, setting up `PostgreSQL PL/pgSQL Triggers <https://www.postgresql.org/docs/12/plpgsql-trigger.html>`_ and default script data.
 
 
 To do that, you must define the entrypoint ``bdc_db.triggers`` in your application ``setup.py`` file as following:
@@ -155,14 +189,14 @@ You can show the triggers loaded (In-Memory) by ``BDC-DB`` command line::
     bdc-db db show-triggers
 
 
-To load them into the database system, use the command::
+To register them into the database system, use the command::
 
-    bdc-db db load-triggers
+    bdc-db db create-triggers
 
 
 You can also load all data scripts with command::
 
-    bdc-db db create-data
+    bdc-db db load-scripts
 
 
 .. note::

bdc_db/cli.py

@@ -87,7 +87,7 @@ def create_schema(verbose):
     with click.progressbar(_db.metadata.sorted_tables) as bar:
         for table in bar:
             if verbose:
-                click.echo(' Creating table {0}'.format(table))
+                click.echo('\tCreating table {0}'.format(table))
             table.create(bind=_db.engine, checkfirst=True)
 
     click.secho('Database schema created!',
@@ -111,16 +111,15 @@ def drop_schema(verbose):
     with click.progressbar(reversed(_db.metadata.sorted_tables)) as bar:
         for table in bar:
             if verbose:
-                click.echo(' Dropping table {0}'.format(table))
+                click.echo('\tDropping table {0}'.format(table))
             table.drop(bind=_db.engine, checkfirst=True)
 
     click.secho('Database schema dropped!', bold=True, fg='green')
 
 
 @db.command()
-@click.option('-v', '--verbose', is_flag=True, default=False)
 @with_appcontext
-def create_namespace(verbose):
+def create_namespace():
     """Create the table namespace (schema) in database."""
     schema = _db.metadata.schema
 
@@ -138,9 +137,8 @@ def create_namespace(verbose):
 
 
 @db.command()
-@click.option('-v', '--verbose', is_flag=True, default=False)
 @with_appcontext
-def create_extension_postgis(verbose):
+def create_extension_postgis():
     """Enables the PostGIS extenion in the database."""
     click.secho(f'Creating extension postgis...', bold=True, fg='yellow')
 
@@ -152,64 +150,75 @@ def create_extension_postgis(verbose):
 
 
 @db.command()
-@click.option('-v', '--verbose', is_flag=True, default=False)
 @with_appcontext
-def show_triggers(verbose):
-    """List or load the database triggers available in ``BDC-DB``."""
+def show_triggers():
+    """List the trigger definition files registred in ``BDC-DB`` extension."""
     ext = current_app.extensions['bdc-db']
 
     for module_name, entry in ext.triggers.items():
         click.secho(f'Available triggers in "{module_name}"', bold=True, fg='green')
 
         for file_name, script in entry.items():
-            click.secho(f'  -> {script}', bold=True, fg='green')
+            click.secho(f'\t-> {script}', bold=True, fg='green')
 
 
 @db.command()
 @click.option('-v', '--verbose', is_flag=True, default=False)
 @with_appcontext
 def create_triggers(verbose):
-    """List or load the database triggers available in ``BDC-DB``."""
+    """Create in the database the triggers registered in ``BDC-DB`` extension."""
     ext = current_app.extensions['bdc-db']
 
     with _db.session.begin_nested():
         if len(ext.triggers.keys()) == 0:
             click.secho(f'No trigger configured.', bold=True, fg='yellow')
 
         for module_name, entry in ext.triggers.items():
-            click.secho(f'Executing trigger from "{module_name}"', bold=True, fg='green')
+            click.secho(f'Registering triggers from "{module_name}"', bold=True, fg='yellow')
 
             for file_name, script in entry.items():
                 with open(script) as f:
                     content = f.read()
 
-                click.secho(f'  -> {script}', bold=True, fg='green')
+                click.secho(f'\t-> {script}', bold=True, fg='green')
+
+                if verbose:
+                    click.secho(content)
+
                 _db.session.execute(content)
 
+            click.secho(f'Triggers from "{module_name}" registered', bold=True, fg='green')
+
     _db.session.commit()
 
 
 @db.command()
 @click.option('-v', '--verbose', is_flag=True, default=False)
 @with_appcontext
-def create_data(verbose):
-    """Load the database scripts available in ``BDC-DB`` package."""
+def load_scripts(verbose):
+    """Load the database scripts registred in ``BDC-DB`` extension."""
     ext = current_app.extensions['bdc-db']
 
     with _db.session.begin_nested():
         if len(ext.scripts.keys()) == 0:
             click.secho(f'No scripts configured.', bold=True, fg='yellow')
 
         for module_name, entry in ext.scripts.items():
-            click.secho(f'Executing script from "{module_name}"', bold=True, fg='green')
+            click.secho(f'Executing scripts from "{module_name}"', bold=True, fg='yellow')
 
             for file_name, script in entry.items():
                 with open(script) as f:
                     content = f.read()
 
-                click.secho(f'  -> {script}', bold=True, fg='green')
+                click.secho(f'\t-> {script}', bold=True, fg='yellow')
+
+                if verbose:
+                    click.secho(content)
+
                 _db.session.execute(content)
 
+            click.secho(f'Scripts from "{module_name}" executed!', bold=True, fg='green')
+
     _db.session.commit()
 
 
@@ -223,9 +232,14 @@ def load_file(verbose, file):
     """Load and execute a script file into database."""
     sql = file.read()
 
+    click.echo(f'Loading file {file}...', bold = True, fg = 'yellow')
+
+    if verbose:
+        click.echo(sql)
+
     with _db.session.begin_nested():
         _db.session.execute(sql)
 
     _db.session.commit()
 
-    click.echo("Data inserted with success!")
+    click.echo(f'File {file} loaded!', bold = True, fg = 'green')

bdc_db/version.py

@@ -8,4 +8,4 @@
 
 """Version information for BDC-DB."""
 
-__version__ = '0.4.0-alpha'
+__version__ = '0.4.0'

docs/sphinx/index.rst

@@ -6,8 +6,12 @@
     under the terms of the MIT License; see LICENSE file for more details.
 
 
-BDC-DB Extension
-================
+.. include:: ../../README.rst
+    :end-before: Installation
+
+
+This module is based on the source code of `Invenio-DB <https://invenio-db.readthedocs.io/en/latest/>`_.
+Thanks the Invenio Team for providing a scalable way to deal with dynamic database models with `Flask-SQLALchemy <https://flask-sqlalchemy.palletsprojects.com/en/2.x/>`_.
 
 
 .. toctree::
@@ -17,8 +21,8 @@ BDC-DB Extension
 
 
 .. toctree::
-   :maxdepth: 2
-   :caption: API Reference
+   :maxdepth: 1
+   :caption: Documentation
 
    installation
    configurations