Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 0.15.0 #153

Merged
merged 22 commits into from
Jan 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
66212d9
Merge pull request #138 from casework/release-0.14.0
kchason Nov 13, 2023
b86b084
Update SEMVER-related text
ajnelson-nist Nov 21, 2023
1bf1317
Merge pull request #139 from casework/update_semver_notes_in_readme
kchason Nov 21, 2023
6c1bb77
Export local_uuid
ajnelson-nist Dec 4, 2023
6f1bbdf
Run pre-commit autoupdate
ajnelson-nist Dec 11, 2023
016ac37
Run pre-commit autoupdate
ajnelson-nist Dec 12, 2023
26122e7
Merge pull request #140 from casework/export_local_uuid
kchason Dec 13, 2023
4f818d8
Deactivate List shape when TBox review is not requested
ajnelson-nist Dec 13, 2023
ead3b17
Merge branch 'develop' into deactivate_errant_rdf_list_shape
ajnelson-nist Dec 13, 2023
d4a868f
Remove exported local_uuid tests
ajnelson-nist Dec 13, 2023
d92c753
Fix module reference
ajnelson-nist Dec 13, 2023
d8a8b08
Replace case_utils.local_uuid usage in case_file
ajnelson-nist Dec 13, 2023
65cdcda
Remove re-export of local_uuid
ajnelson-nist Dec 13, 2023
cd178f3
Merge pull request #143 from casework/deactivate_errant_rdf_list_shape
kchason Dec 13, 2023
02dab3a
Run pre-commit autoupdate
ajnelson-nist Dec 15, 2023
ba3c6a2
Merge branch 'develop' into release-0.14.1
ajnelson-nist Dec 22, 2023
8b87ab8
Merge pull request #150 from casework/release-0.14.1
kchason Dec 22, 2023
afcb069
Merge pull request #152 from casework/release-0.14.1.post0
kchason Jan 9, 2024
07d1616
Merge branch 'develop' into export_local_uuid
ajnelson-nist Jan 9, 2024
6effb63
Bump cdo-local-uuid and update test calls
ajnelson-nist Jan 9, 2024
91bae5e
Merge pull request #144 from casework/export_local_uuid
kchason Jan 10, 2024
5cd31f4
Bump versions
ajnelson-nist Jan 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,9 @@ This repository can be installed from PyPI or from source.
pip install case-utils
```

Users who wish to install from PyPI should be aware that while CASE's ontology is in its pre-1.0.0 release state, backwards-incompatible ontology changes may occur. This may manifest as [`case_validate`](#case_validate) reporting data review errors after installing an updated `case_utils` version. Users may wish to pin `case_utils` within any dependent code bases to be less than the next unreleased SEMVER-minor version. (E.g. if `case_utils` version `0.8.0` is currently available, a newly adopting project might wish to track `case_utils<0.9.0` among its dependencies.)
Users who wish to install from PyPI should be aware that though CASE's ontology is in its post-1.0.0 release state, this Python project is in a pre-1.0.0 release state. Backwards-incompatible ontology changes will only occur in accordance with [SEMVER](https://semver.org/). This Python project is not yet committed to its API, and backwards-incompatiable changes may occur. They are likely to occur with advance notice.

Users may wish to pin `case-utils` within any dependent code bases to be less than the next unreleased SEMVER-minor version. (E.g. if `case-utils` version `0.14.0` is currently available, a newly adopting project might wish to track `case-utils<0.15.0` among its dependencies.)


### Installing from source
Expand Down Expand Up @@ -120,7 +122,7 @@ case_sparql_select output.md input.sparql input.json [input-2.json ...]

### `local_uuid`

This [module](case_utils/local_uuid.py) provides a wrapper UUID generator, `local_uuid()`. Its main purpose is making example data generate consistent identifiers, and intentionally includes mechanisms to make it difficult to activate this mode without awareness of the caller.
_Migration:_ Functionality previously in [`case_utils.local_uuid`](case_utils/local_uuid.py) has been exported to [`cdo-local-uuid`](https://github.com/Cyber-Domain-Ontology/CDO-Utility-Local-UUID). A future `case-utils` release will drop this re-export.


### Built versions
Expand Down
4 changes: 1 addition & 3 deletions case_utils/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,4 @@
#
# We would appreciate acknowledgement if the software is used.

__version__ = "0.14.1.post0"

from . import local_uuid # noqa: F401
__version__ = "0.15.0"
18 changes: 9 additions & 9 deletions case_utils/case_file/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
This module creates a graph object that provides a basic UCO characterization of a single file. The gathered metadata is among the more "durable" file characteristics, i.e. characteristics that would remain consistent when transferring a file between locations.
"""

__version__ = "0.5.1"
__version__ = "0.6.0"

import argparse
import datetime
Expand All @@ -28,7 +28,9 @@
import typing
import warnings

import cdo_local_uuid
import rdflib
from cdo_local_uuid import local_uuid

import case_utils.inherent_uuid
from case_utils.namespace import (
Expand Down Expand Up @@ -94,7 +96,7 @@ def create_file_node(
node_namespace = rdflib.Namespace(node_prefix)

if node_iri is None:
node_slug = "File-" + case_utils.local_uuid.local_uuid()
node_slug = "File-" + local_uuid()
node_iri = node_namespace[node_slug]
n_file = rdflib.URIRef(node_iri)
graph.add((n_file, NS_RDF.type, NS_UCO_OBSERVABLE.File))
Expand All @@ -110,7 +112,7 @@ def create_file_node(
n_file, NS_UCO_OBSERVABLE.FileFacet, namespace=node_namespace
)
else:
n_file_facet = node_namespace["FileFacet-" + case_utils.local_uuid.local_uuid()]
n_file_facet = node_namespace["FileFacet-" + local_uuid()]

graph.add(
(
Expand Down Expand Up @@ -144,9 +146,7 @@ def create_file_node(
n_file, NS_UCO_OBSERVABLE.ContentDataFacet, namespace=node_namespace
)
else:
n_contentdata_facet = node_namespace[
"ContentDataFacet-" + case_utils.local_uuid.local_uuid()
]
n_contentdata_facet = node_namespace["ContentDataFacet-" + local_uuid()]

graph.add((n_file, NS_UCO_CORE.hasFacet, n_contentdata_facet))
graph.add(
Expand Down Expand Up @@ -248,7 +248,7 @@ def create_file_node(
)
)
else:
hash_uuid = case_utils.local_uuid.local_uuid()
hash_uuid = local_uuid()
n_hash = node_namespace["Hash-" + hash_uuid]

graph.add((n_contentdata_facet, NS_UCO_OBSERVABLE.hash, n_hash))
Expand Down Expand Up @@ -291,7 +291,7 @@ def main() -> None:

logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)

case_utils.local_uuid.configure()
cdo_local_uuid.configure()

NS_BASE = rdflib.Namespace(args.base_prefix)

Expand All @@ -314,7 +314,7 @@ def main() -> None:
context_dictionary = {k: v for (k, v) in graph.namespace_manager.namespaces()}
serialize_kwargs["context"] = context_dictionary

node_iri = NS_BASE["File-" + case_utils.local_uuid.local_uuid()]
node_iri = NS_BASE["File-" + local_uuid()]
create_file_node(
graph,
args.in_file,
Expand Down
3 changes: 2 additions & 1 deletion case_utils/case_validate/validate_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
#
# We would appreciate acknowledgement if the software is used.

__version__ = "0.1.0"
__version__ = "0.2.0"

import importlib
import logging
Expand Down Expand Up @@ -197,6 +197,7 @@ def disable_tbox_review(graph: rdflib.Graph) -> None:
"Disjointedness-AP-OP-shape",
"Disjointedness-C-DT-shape",
"Disjointedness-DP-OP-shape",
"List-shape",
"ObjectProperty-shacl-constraints-shape",
"ontologyIRI-versionIRI-prerequisite-shape",
"versionIRI-nodeKind-shape",
Expand Down
4 changes: 2 additions & 2 deletions case_utils/inherent_uuid.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
2. A pattern based on inherence generates UUIDv5s based on how an inherent object (a.k.a. UcoInherentCharacterizationThing) structurally relates to the object in which it inheres. For instance, a Facet is understood to only relate to its UcoObject by linking with the uco-core:hasFacet property. So, a Facet's UUID is determined uniquely by (1) the "UUID namespace" of its corresponding UcoObject, and (2) its OWL Class.
A. The term "UUID namespace" is described in RFC 4122 Section 4.3 [#rfc4122s43]_ , and is not intended be confused with `rdflib.term.Namespace`. For any uco-core:UcoThing (or even owl:Thing), the function `inherence_uuid` defines the procedure for either extracting or generating a UUID for use as a namespace.

This module is independent of, and complements, `case_utils.local_uuid`, which provides deterministic UUIDs based on calling process's environment.
This module is independent of, and complements, `cdo_local_uuid.local_uuid`, which provides deterministic UUIDs based on calling process's environment.

References
==========
Expand Down Expand Up @@ -57,7 +57,7 @@
>>> assert str(n_file_facet)[-36:] == str(n_file_facet_2)[-36:]
"""

__version__ = "0.1.1"
__version__ = "0.1.2"

import binascii
import re
Expand Down
163 changes: 7 additions & 156 deletions case_utils/local_uuid.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,167 +15,18 @@
# We would appreciate acknowledgement if the software is used.

"""
This library is a wrapper for uuid, provided to generate repeatable UUIDs if requested.

The function local_uuid() should be used in code where a user could be expected to opt in to non-random UUIDs.
This library was a wrapper for uuid, provided to generate repeatable UUIDs if requested. It is now a temporary re-export of functionality migrated to cdo_local_uuid.
"""

__version__ = "0.4.1"
__version__ = "0.5.0"

__all__ = ["configure", "local_uuid"]

import logging
import os
import pathlib
import sys
import typing
import uuid
import warnings

DEMO_UUID_BASE: typing.Optional[str] = None

DEMO_UUID_COUNTER: int = 0

_logger = logging.getLogger(pathlib.Path(__file__).name)


def _is_relative_to(p1: pathlib.Path, p2: pathlib.Path) -> bool:
"""
This function provides pathlib.is_relative_to to Pythons before 3.9. After the End of Life of Python 3.8, this function can be removed.
"""
if sys.version_info < (3, 9):
try:
_ = p1.relative_to(p2)
return True
except ValueError:
return False
else:
return p1.is_relative_to(p2)


def configure() -> None:
"""
This function is part of setting up _demo_uuid() to generate non-random UUIDs. See _demo_uuid() documentation for further setup notes.
"""
global DEMO_UUID_BASE

# _logger.debug("sys.argv = %r.", sys.argv)

if os.getenv("DEMO_UUID_REQUESTING_NONRANDOM") == "NONRANDOM_REQUESTED":
warnings.warn(
"Environment variable DEMO_UUID_REQUESTING_NONRANDOM is deprecated. See case_utils.local_uuid._demo_uuid for usage notes on its replacement, CASE_DEMO_NONRANDOM_UUID_BASE. Proceeding with random UUIDs.",
FutureWarning,
)
return

env_base_dir_name = os.getenv("CASE_DEMO_NONRANDOM_UUID_BASE")
if env_base_dir_name is None:
return

base_dir_original_path = pathlib.Path(env_base_dir_name)
if not base_dir_original_path.exists():
warnings.warn(
"Environment variable CASE_DEMO_NONRANDOM_UUID_BASE is expected to refer to an existing directory. Proceeding with random UUIDs.",
RuntimeWarning,
)
return
if not base_dir_original_path.is_dir():
warnings.warn(
"Environment variable CASE_DEMO_NONRANDOM_UUID_BASE is expected to refer to a directory. Proceeding with random UUIDs.",
RuntimeWarning,
)
return

# Component: An emphasis this is an example.
demo_uuid_base_parts = ["example.org"]

# Component: Present working directory, relative to CASE_DEMO_NONRANDOM_UUID_BASE if that environment variable is an ancestor of pwd.
base_dir_resolved_path = base_dir_original_path.resolve()
srcdir_original_path = pathlib.Path(os.getcwd())
srcdir_resolved_path = srcdir_original_path.resolve()
# _logger.debug("base_dir_resolved_path = %r.", base_dir_resolved_path)
# _logger.debug("srcdir_resolved_path = %r.", srcdir_resolved_path)
try:
srcdir_relative_path = srcdir_resolved_path.relative_to(base_dir_resolved_path)
# _logger.debug("srcdir_relative_path = %r.", srcdir_relative_path)
demo_uuid_base_parts.append(str(srcdir_relative_path))
except ValueError:
# If base_dir is not an ancestor directory of srcdir, default to srcdir.
# _logger.debug("PWD is not relative to base path.")
demo_uuid_base_parts.append(str(srcdir_resolved_path))

# Component: Command of argument vector.
env_venv_name = os.getenv("VIRTUAL_ENV")
if env_venv_name is None:
demo_uuid_base_parts.append(sys.argv[0])
else:
command_original_path = pathlib.Path(sys.argv[0])
# _logger.debug("command_original_path = %r.", command_original_path)
command_resolved_path = command_original_path.resolve()
# _logger.debug("command_resolved_path = %r.", command_resolved_path)

# The command could be a command embedded in a virtual
# environment, or it could be a script external to any virtual
# environment.
venv_original_path = pathlib.Path(env_venv_name)
venv_resolved_path = venv_original_path.resolve()
if _is_relative_to(command_resolved_path, venv_resolved_path):
command_relative_path = command_resolved_path.relative_to(
venv_resolved_path
)
# _logger.debug("command_relative_path = %r.", command_relative_path)
demo_uuid_base_parts.append(str(command_relative_path))
else:
demo_uuid_base_parts.append(str(command_original_path))

if len(sys.argv) > 1:
# Component: Arguments of argument vector.
demo_uuid_base_parts.extend(sys.argv[1:])

# _logger.debug("demo_uuid_base_parts = %r.", demo_uuid_base_parts)

DEMO_UUID_BASE = "/".join(demo_uuid_base_parts)


def _demo_uuid() -> str:
"""
This function generates a repeatable UUID, drawing on non-varying elements of the environment and process call for entropy.

This function is not intended to be called outside of this module. Instead, local_uuid() should be called.

WARNING: This function was developed for use ONLY for reducing (but not eliminating) version-control edits to identifiers when generating sample data. It creates UUIDs that are decidedly NOT random, and should remain consistent on repeated calls to the importing script.

To prevent accidental non-random UUID usage, two setup steps need to be done before calling this function:

* An environment variable, CASE_DEMO_NONRANDOM_UUID_BASE, must be set to a string provided by the caller. The variable's required value is the path to some directory. The variable's recommended value is the equivalent of the Make variable "top_srcdir" - that is, the root directory of the containing Git repository, some parent of the current process's current working directory.
* The configure() function in this module must be called.
"""
global DEMO_UUID_BASE
global DEMO_UUID_COUNTER

if os.getenv("CASE_DEMO_NONRANDOM_UUID_BASE") is None:
raise ValueError(
"demo_uuid() called without CASE_DEMO_NONRANDOM_UUID_BASE in environment."
)

if DEMO_UUID_BASE is None:
raise ValueError("demo_uuid() called with DEMO_UUID_BASE unset.")

parts = [DEMO_UUID_BASE]

# Component: Incrementing counter.
DEMO_UUID_COUNTER += 1
parts.append(str(DEMO_UUID_COUNTER))

return str(uuid.uuid5(uuid.NAMESPACE_URL, "/".join(parts)))

from cdo_local_uuid import configure, local_uuid

def local_uuid() -> str:
"""
Generate either a UUID4, or if requested via environment configuration, a non-random demo UUID.
"""
global DEMO_UUID_BASE
if DEMO_UUID_BASE is None:
return str(uuid.uuid4())
else:
return _demo_uuid()
warnings.warn(
"case_utils.local_uuid has exported its functionality to cdo_local_uuid. Imports should be changed to use cdo_local_uuid. case_utils currently re-exports that functionality, but this will cease in a future release.",
DeprecationWarning,
)
1 change: 1 addition & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ license_files =
[options]
include_package_data = true
install_requires =
cdo-local-uuid >= 0.5.0, < 0.6.0
pandas
pyshacl >= 0.24.0
rdflib < 8
Expand Down
12 changes: 4 additions & 8 deletions tests/case_utils/case_file/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -117,11 +117,10 @@ sample.txt.json: \
$(tests_srcdir)/src/isomorphic_diff.py \
$(top_srcdir)/case_utils/case_file/__init__.py \
$(top_srcdir)/case_utils/inherent_uuid.py \
$(top_srcdir)/case_utils/local_uuid.py \
$(top_srcdir)/case_utils/namespace.py \
sample.txt-nocompact.json
rm -f $@ _$@ __$@
export CASE_DEMO_NONRANDOM_UUID_BASE="$(top_srcdir)" \
export CDO_DEMO_NONRANDOM_UUID_BASE="$(top_srcdir)" \
&& source $(tests_srcdir)/venv/bin/activate \
&& case_file \
--debug \
Expand Down Expand Up @@ -152,11 +151,10 @@ sample.txt.ttl: \
$(tests_srcdir)/.venv.done.log \
$(top_srcdir)/case_utils/case_file/__init__.py \
$(top_srcdir)/case_utils/inherent_uuid.py \
$(top_srcdir)/case_utils/local_uuid.py \
$(top_srcdir)/case_utils/namespace.py \
sample.txt.done.log
rm -f _$@ __$@
export CASE_DEMO_NONRANDOM_UUID_BASE="$(top_srcdir)" \
export CDO_DEMO_NONRANDOM_UUID_BASE="$(top_srcdir)" \
&& source $(tests_srcdir)/venv/bin/activate \
&& case_file \
--debug \
Expand All @@ -178,11 +176,10 @@ sample.txt-disable_hashes.ttl: \
$(tests_srcdir)/.venv.done.log \
$(top_srcdir)/case_utils/case_file/__init__.py \
$(top_srcdir)/case_utils/inherent_uuid.py \
$(top_srcdir)/case_utils/local_uuid.py \
$(top_srcdir)/case_utils/namespace.py \
sample.txt.done.log
rm -f _$@ __$@
export CASE_DEMO_NONRANDOM_UUID_BASE="$(top_srcdir)" \
export CDO_DEMO_NONRANDOM_UUID_BASE="$(top_srcdir)" \
&& source $(tests_srcdir)/venv/bin/activate \
&& case_file \
--debug \
Expand All @@ -206,11 +203,10 @@ sample.txt-nocompact.json: \
$(tests_srcdir)/src/isomorphic_diff.py \
$(top_srcdir)/case_utils/case_file/__init__.py \
$(top_srcdir)/case_utils/inherent_uuid.py \
$(top_srcdir)/case_utils/local_uuid.py \
$(top_srcdir)/case_utils/namespace.py \
sample.txt.done.log
rm -f _$@
export CASE_DEMO_NONRANDOM_UUID_BASE="$(top_srcdir)" \
export CDO_DEMO_NONRANDOM_UUID_BASE="$(top_srcdir)" \
&& source $(tests_srcdir)/venv/bin/activate \
&& case_file \
--debug \
Expand Down
Loading