Documentation for oneAPI data management backend primitives

docs/doxygen/oneapi/Doxyfile

@@ -902,8 +902,27 @@ EXCLUDE_SYMLINKS       = NO
 # Note that the wildcards are matched against the file with absolute path, so to
 # exclude all test directories for example use the pattern */test/*
 
-EXCLUDE_PATTERNS       = */backend/* \
-                         */detail/*
+EXCLUDE_PATTERNS       = */backend/interop/*                            \
+                         */backend/primitives/blas/*                    \
+                         */backend/primitives/distance/*                \
+                         */backend/primitives/heap/*                    \
+                         */backend/primitives/intersection/*            \
+                         */backend/primitives/lapack/*                  \
+                         */backend/primitives/objective_function/*      \
+                         */backend/primitives/optimizers/*              \
+                         */backend/primitives/placement/*               \
+                         */backend/primitives/reduction/*               \
+                         */backend/primitives/regression/*              \
+                         */backend/primitives/rng/*                     \
+                         */backend/primitives/search/*                  \
+                         */backend/primitives/selection/*               \
+                         */backend/primitives/sort/*                    \
+                         */backend/primitives/sparse_blas/*             \
+                         */backend/primitives/stat/*                    \
+                         */backend/primitives/test/*                    \
+                         */backend/primitives/voting/*                  \
+                         */detail/*                                     \
+                         */table/backend/*
 
 
 # The EXCLUDE_SYMBOLS tag can be used to specify one or more symbol names
@@ -2179,7 +2198,7 @@ INCLUDE_FILE_PATTERNS  =
 # recursively expanded use the := operator instead of the = operator.
 # This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
 
-PREDEFINED             =
+PREDEFINED             = ONEDAL_DATA_PARALLEL
 
 # If the MACRO_EXPANSION and EXPAND_ONLY_PREDEF tags are set to YES then this
 # tag can be used to specify a list of macro names that should be expanded. The

docs/source/api/data-management/backend-primitives.rst

@@ -0,0 +1,104 @@
+.. Copyright contributors to the oneDAL project
+..
+.. Licensed under the Apache License, Version 2.0 (the "License");
+.. you may not use this file except in compliance with the License.
+.. You may obtain a copy of the License at
+..
+..     http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+
+.. highlight:: cpp
+
+.. _api_backend_primitives:
+
+==================
+Backend Primitives
+==================
+
+Refer to :ref:`Developer Guide: Backend Primitives <dm_backend_primitives>`.
+
+.. _backend_primitives_programming_interface:
+
+---------------------
+Programming interface
+---------------------
+
+All types and functions in this section are declared in the
+``oneapi::dal::backend::primitives`` namespace and be available via inclusion of the
+``oneapi/dal/backend/primitives/ndarray.hpp`` header file.
+
+.. _api_ndorder:
+
+Multidimensional array order
+----------------------------
+
+Refers to data indexing order, or how a linear sequence is translated into a multi-dimensional array.
+
+.. onedal_enumclass:: oneapi::dal::backend::primitives::ndorder
+
+.. _api_ndshape:
+
+Multidimensional array shape
+----------------------------
+
+.. onedal_class:: oneapi::dal::backend::primitives::ndshape
+
+Multidimensional data view (ndview)
+-----------------------------------
+
+An implementation of a multidimensional data container that provides a view of the homogeneous
+data stored in an externally-managed memory block.
+
+All the ``ndview`` class methods can be divided into several groups:
+
+#. The group of ``wrap()`` methods that are used to create an ``ndview`` object from external,
+   mutable or immutable memory.
+
+#. The group of ``wrap_mutable()`` methods that are used to create a mutable ``ndview`` object from
+   ``dal::array`` object.
+
+#. The methods that are used to access the data.
+
+#. The methods like ``t()`` and ``reshape()`` that are used to change the shape and layout of the data view.
+
+#. The group of data slicing methods that are used to create a new ``ndview`` object that is a
+   view of the original data slice along some dimension.
+
+#. The group of data transfering methods that are used to produce a new ``ndview`` object that
+   contains the data copied from the original one, but at the different memory location.
+
+Multidimensional array (ndarray)
+--------------------------------
+
+An implementation of multidimensional data array that provides a way to store and manipulate
+homogeneous data in a multidimensional structure.
+
+All the ``ndarray`` class methods can be divided into several groups:
+
+#. The group of ``wrap()`` and ``wrap_mutable()`` methods that are used to create an ``ndarray``
+   object from external, mutable or immutable memory.
+
+#. The group of ``wrap()`` and ``wrap_mutable()`` methods that are used to create an ``ndarray``
+   that shares its data with another data object.
+
+#. The group of methods like ``zeros()``, ``full()``, ``arange()``, etc. that are used to create an ``ndarray``
+   object with the specified shape and values.
+
+#. The methods like ``t()`` and ``reshape()`` that are used to change the shape and layout
+   of the multidimensional array.
+
+#. The group of data slicing methods that are used to create a new ``ndarray`` object that is a view
+   of the original data slice along some dimension.
+
+#. The group of methods like ``fill()``, ``assign()``, ``assign_from_host()``, etc. that are used to
+   fill the array with the specified values.
+
+.. toctree::
+
+   backend/ndview.rst
+   backend/ndarray.rst

docs/source/api/data-management/backend/ndarray.rst

@@ -0,0 +1,33 @@
+.. Copyright contributors to the oneDAL project
+..
+.. Licensed under the Apache License, Version 2.0 (the "License");
+.. you may not use this file except in compliance with the License.
+.. You may obtain a copy of the License at
+..
+..     http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+
+.. _api_ndarray:
+
+======================
+Multidimensional array
+======================
+
+The ``ndarray`` class provides a way to store and manipulate homogeneous data
+in a multidimensional structure.
+The pointer to the data within the ``ndarray`` object is :term:`reference-counted <Reference-counted object>`:.
+
+---------------------
+Programming interface
+---------------------
+
+All types and functions in this section are declared in the
+``oneapi::dal::backend::primitives`` namespace and be available via inclusion of the
+``oneapi/dal/table/ndarray.hpp`` header file.
+
+.. onedal_class:: oneapi::dal::backend::primitives::ndarray

docs/source/api/data-management/backend/ndview.rst

@@ -0,0 +1,32 @@
+.. Copyright contributors to the oneDAL project
+..
+.. Licensed under the Apache License, Version 2.0 (the "License");
+.. you may not use this file except in compliance with the License.
+.. You may obtain a copy of the License at
+..
+..     http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+
+.. _api_ndview:
+
+=====================
+Multidimensional view
+=====================
+
+The ``ndview`` class provides a view of the homogeneous data as a multidimensional structure
+stored in an externally-managed memory block.
+
+---------------------
+Programming interface
+---------------------
+
+All types and functions in this section are declared in the
+``oneapi::dal::backend::primitives`` namespace and be available via inclusion of the
+``oneapi/dal/table/ndarray.hpp`` header file.
+
+.. onedal_class:: oneapi::dal::backend::primitives::ndview

docs/source/api/data-management/index.rst

@@ -25,4 +25,5 @@ Refer to :ref:`Developer Guide: Data Management <dg_data_management>`.
    data-sources.rst
    graphs.rst
    graph-service.rst
-   tables.rst
+   tables.rst
+   backend-primitives.rst

docs/source/conf.py

@@ -131,6 +131,7 @@
 nitpick_ignore = [
     # top level namespace
     ('cpp:identifier', 'dal'),
+    ('cpp:identifier', 'base'),
     # method
     ('cpp:identifier', 'method'),
     ('cpp:identifier', 'Method'),
@@ -139,6 +140,8 @@
     ('cpp:identifier', 'task::by_default'),
     ('cpp:identifier', 'Task'),
     # detail
+    ('cpp:identifier', 'dal::detail'),
+    ('cpp:identifier', 'dal::detail::empty_delete<T>'),
     ('cpp:identifier', 'detail'),
     ('cpp:identifier', 'detail::descriptor_base<>'),
     ('cpp:identifier', 'detail::descriptor_base<>::float_t'),
@@ -392,5 +395,29 @@
     ('cpp:identifier', 'kind::induced'),
     ('cpp:identifier', 'kind::non_induced'),
     ('cpp:identifier', 'preview'),
-    ('cpp:identifier', 'connected_components')
+    ('cpp:identifier', 'connected_components'),
+
+    # sycl
+    ('cpp:identifier', 'event_vector'),
+    ('cpp:identifier', 'sycl'),
+    ('cpp:identifier', 'sycl::event'),
+    ('cpp:identifier', 'sycl::queue'),
+    ('cpp:identifier', 'sycl::range'),
+    ('cpp:identifier', 'sycl::usm'),
+    ('cpp:identifier', 'sycl::usm::alloc'),
+    ('cpp:identifier', 'sycl::usm::alloc::shared'),
+
+    # backend primitives - data management
+    ('cpp:identifier', 'array_t'),
+    ('cpp:identifier', 'axis_count'),
+    ('cpp:identifier', 'ndarray'),
+    ('cpp:identifier', 'ndindex<axis_count>'),
+    ('cpp:identifier', 'ndorder'),
+    ('cpp:identifier', 'ndorder::c'),
+    ('cpp:identifier', 'ndshape'),
+    ('cpp:identifier', 'ndshape<new_axis_count>'),
+    ('cpp:identifier', 'ndview'),
+    ('cpp:identifier', 'order'),
+    ('cpp:identifier', 'shape_t'),
+    ('cpp:identifier', 'shared_t')
 ]

docs/source/contribution/cpu_features.rst

@@ -157,7 +157,7 @@ These files contain the definitions of one or several template classes that defi
 do the actual computations. Here is a variant of the ``Abc`` training algorithm kernel definition in the file
 `abc_classification_train_kernel.h`:
 
-.. include:: ../includes/cpu_features/abc-classification-train-kernel.rst
+.. include:: ../includes/cpu-features/abc-classification-train-kernel.rst
 
 Typical template parameters are:
 
@@ -176,7 +176,7 @@ These files contain the implementations of the computational functions defined i
 Here is a variant of ``method1`` implementation for ``Abc`` training algorithm that does not contain any
 instruction set specific code. The implementation is located in the file `abc_classification_train_method1_impl.i`:
 
-.. include:: ../includes/cpu_features/abc-classification-train-method1-impl.rst
+.. include:: ../includes/cpu-features/abc-classification-train-method1-impl.rst
 
 Although the implementation of the ``method1`` does not contain any instruction set specific code, it is
 expected that the developers leverage SIMD related macros available in |short_name|.
@@ -193,7 +193,7 @@ For example, the AVX-512 specific code should be gated on the value ``__CPUID__(
 
 Then the implementation of the ``method2`` in the file `abc_classification_train_method2_impl.i` will look like:
 
-.. include:: ../includes/cpu_features/abc-classification-train-method2-impl.rst
+.. include:: ../includes/cpu-features/abc-classification-train-method2-impl.rst
 
 \*_fpt_cpu.cpp
 --------------
@@ -203,7 +203,7 @@ These files contain the instantiations of the template classes defined in the fi
 The instantiation of the ``Abc`` training algorithm kernel for ``method1`` is located in the file
 `abc_classification_train_method1_batch_fpt_cpu.cpp`:
 
-.. include:: ../includes/cpu_features/abc-classification-train-method1-fpt-cpu.rst
+.. include:: ../includes/cpu-features/abc-classification-train-method1-fpt-cpu.rst
 
 `_fpt_cpu.cpp` files are not compiled directly into object files. First, multiple copies of those files
 are made replacing the ``fpt``, which stands for 'floating point type', and ``cpu`` parts of the file name

docs/source/onedal/data-management/backend-primitives.rst

@@ -0,0 +1,54 @@
+.. Copyright contributors to the oneDAL project
+..
+.. Licensed under the Apache License, Version 2.0 (the "License");
+.. you may not use this file except in compliance with the License.
+.. You may obtain a copy of the License at
+..
+..     http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+
+.. highlight:: cpp
+
+.. _dm_backend_primitives:
+
+==================
+Backend Primitives
+==================
+
+This section describes the types related to data management backend primitives.
+
+.. tabularcolumns::  |\Y{0.2}|\Y{0.8}|
+
+.. list-table:: Data Management Backend Primitives Types
+   :header-rows: 1
+   :widths: 10 70
+   :class: longtable
+
+   * - Type
+     - Description
+
+   * - :ref:`api_ndorder`
+     - An enumeration of multidimensional data orders used to store
+       contiguous data blocks inside the table.
+
+   * - :ref:`api_ndshape`
+     - A class that represents the shape of a multidimensional array.
+
+   * - :ref:`api_ndview`
+     - An implementation of a multidimensional data container that provides a view of the homogeneous
+       data stored in an externally-managed memory block.
+
+   * - :ref:`api_ndarray`
+     - A class that provides a way to store and manipulate homogeneous data
+       in a multidimensional structure.
+
+---------------------
+Programming interface
+---------------------
+
+Refer to :ref:`API: Data Management Backend Primitives <backend_primitives_programming_interface>`.

docs/source/onedal/data-management/index.rst

@@ -289,3 +289,4 @@ This section includes the detailed descriptions of all data management objects i
    data-sources.rst
    graphs.rst
    tables.rst
+   backend-primitives.rst