From f390eda54aa8b07eca6774fe32e5b628b87e2cef Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 23 Mar 2021 09:39:28 +0000
Subject: [PATCH 01/77] Add section for creating config files to docs

---
 docs/how_to_config_file.rst | 57 +++++++++++++++++++++++++++++++++++++
 docs/index.rst              |  1 +
 2 files changed, 58 insertions(+)
 create mode 100644 docs/how_to_config_file.rst

diff --git a/docs/how_to_config_file.rst b/docs/how_to_config_file.rst
new file mode 100644
index 000000000..7282c701e
--- /dev/null
+++ b/docs/how_to_config_file.rst
@@ -0,0 +1,57 @@
+###################
+Configuration files
+###################
+
+This page outlines how to construct configuration files to run your own routines
+with `~skypy.pipeline.Pipeline`.
+
+Remember that SkyPy not only is a library of astronomical models but a flexible
+tool with infrastructure for you to use with your own
+functions and pipelines!
+
+YAML syntax
+-----------
+YAML_ is a file format designed to be readable by both computers and humans.
+This guide introduces the main features of YAML relevant when writing
+a configuration file to use with ``SkyPy``.
+Fundamentally, a file written in YAML consists of a set of key-value pairs.
+Each pair is written as ``key: value``, where whitespace after the ``:`` is optional.
+
+
+
+Variables
+^^^^^^^^^
+* Define a variable
+* Reference a variable
+
+Functions
+^^^^^^^^^
+* Call a function
+* Define parameters: Variables that can be modified at execution
+
+Tables
+^^^^^^
+A dictionary of table names, each resolving to a dictionary of column names for that table
+
+* Create a table
+* Add a column
+* Multi-column assignment
+* Table.init and table.complete dependencies
+
+Cosmology, a special parameter
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+The cosmology to be used by functions within the pipeline.
+
+.. _YAML: https://yaml.org
+
+
+
+Walkthrough example
+-------------------
+
+This walkthrough example shows the natural flow of SkyPy pipelines and
+how to think through the process of creating a general configuration file.
+You can find more complex examples_ in our documentation.
+
+
+.. _examples: https://skypy.readthedocs.io/en/stable/examples/index.html
diff --git a/docs/index.rst b/docs/index.rst
index 84ee5e262..b6087297c 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -17,6 +17,7 @@ Getting Started
    :maxdepth: 1
 
    install
+   how_to_config_file
    examples/index
 
 .. _user-docs:

From 273ccecb595a81efe63ae5737c27bf00453ff44c Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 23 Mar 2021 10:03:46 +0000
Subject: [PATCH 02/77] Add guide for variables

---
 docs/how_to_config_file.rst | 32 ++++++++++++++++++++++++++++----
 1 file changed, 28 insertions(+), 4 deletions(-)

diff --git a/docs/how_to_config_file.rst b/docs/how_to_config_file.rst
index 7282c701e..2b8b4dfb7 100644
--- a/docs/how_to_config_file.rst
+++ b/docs/how_to_config_file.rst
@@ -12,17 +12,41 @@ functions and pipelines!
 YAML syntax
 -----------
 YAML_ is a file format designed to be readable by both computers and humans.
-This guide introduces the main features of YAML relevant when writing
-a configuration file to use with ``SkyPy``.
 Fundamentally, a file written in YAML consists of a set of key-value pairs.
 Each pair is written as ``key: value``, where whitespace after the ``:`` is optional.
 
+This guide introduces the main syntax of YAML relevant when writing
+a configuration file to use with ``SkyPy``. Essentially, it would start with
+definitions of individual variables at the top, followed by the tables to produce,
+and, within the table entries, the features of the objects to simulate are included.
 
 
 Variables
 ^^^^^^^^^
-* Define a variable
-* Reference a variable
+* `Define a variable`: a variable is defined as a key-value pair at the top of the file. You can define any type of variable:
+
+  .. code:: yaml
+
+      counter: 100 # An integer
+      miles: 1000.0 # A floating point
+      name: "Joy" # A string
+      mylist: [ 'abc', 789, 2.0e3 ] # A list
+
+
+* `Reference a variable`: variables can be referenced by their full name tagged with a dollar sign ``$``.
+  For example, if you previously defined a list of countries:
+
+  .. code:: yaml
+
+      country_list: [ 'UK', 'Spain', 'Zambia', 'Chile', 'Japan' ]
+
+  You would reference the variable;
+
+  .. code:: yaml
+
+      countries: $country_list
+
+
 
 Functions
 ^^^^^^^^^

From 832c6f548e1d33cf9af354e015b4ce5653bca84e Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 23 Mar 2021 10:19:01 +0000
Subject: [PATCH 03/77] Add guide for functions

---
 docs/how_to_config_file.rst | 35 ++++++++++++++++++++++++++++++++---
 1 file changed, 32 insertions(+), 3 deletions(-)

diff --git a/docs/how_to_config_file.rst b/docs/how_to_config_file.rst
index 2b8b4dfb7..662fa516f 100644
--- a/docs/how_to_config_file.rst
+++ b/docs/how_to_config_file.rst
@@ -50,12 +50,40 @@ Variables
 
 Functions
 ^^^^^^^^^
-* Call a function
-* Define parameters: Variables that can be modified at execution
+* `Call a function`: functions are defined as tuples where the first entry is the fully qualified function name tagged with and exclamation mark ``!`` and the second entry is either a list of positional arguments or a dictionary of keyword arguments.
+
+  For example, you need to call the ``log10()`` and ``linspace()`` NumPy_ functions, for this you define the following key-value pairs:
+
+  .. code:: yaml
+
+      log_of_2: !numpy.log10 [2]
+      myarray: !numpy.linspace [0, 2.5, 10]
+
+
+* `Define parameters`: parameters are variables that can be modified at execution. You can also define parameters of functions with a dictionary of keyword arguments.
+
+  Imagine you want to compute the total expense when buying a house (£230000) and a car (£15589.3). To run it with the `SkyPy` pipeline, you would call the function and define the parameters as an indented dictionary
+
+  .. code:: yaml
+
+      expense: !numpy.add
+        x1: 230000
+        x2: 15589.3
+
+  or you could also define the variables at the top and then reference them
+
+  .. code:: yaml
+
+      house_price: 230000
+      car_price: 15589.3
+      expense: !numpy.add
+        x1: $house_price
+        x2: $car_price
+
 
 Tables
 ^^^^^^
-A dictionary of table names, each resolving to a dictionary of column names for that table
+A dictionary of table names, each resolving to a dictionary of column names for that table.
 
 * Create a table
 * Add a column
@@ -67,6 +95,7 @@ Cosmology, a special parameter
 The cosmology to be used by functions within the pipeline.
 
 .. _YAML: https://yaml.org
+.. _NumPy: https://numpy.org
 
 
 

From 5da3aa9c8581ab7526f0048503ba84138670d979 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 23 Mar 2021 12:07:50 +0000
Subject: [PATCH 04/77] Add guide for tables and columns

---
 docs/how_to_config_file.rst | 75 ++++++++++++++++++++++++++++++++-----
 1 file changed, 66 insertions(+), 9 deletions(-)

diff --git a/docs/how_to_config_file.rst b/docs/how_to_config_file.rst
index 662fa516f..cf16efb18 100644
--- a/docs/how_to_config_file.rst
+++ b/docs/how_to_config_file.rst
@@ -59,9 +59,7 @@ Functions
       log_of_2: !numpy.log10 [2]
       myarray: !numpy.linspace [0, 2.5, 10]
 
-
-* `Define parameters`: parameters are variables that can be modified at execution. You can also define parameters of functions with a dictionary of keyword arguments.
-
+  You can also define parameters of functions with a dictionary of keyword arguments.
   Imagine you want to compute the total expense when buying a house (£230000) and a car (£15589.3). To run it with the `SkyPy` pipeline, you would call the function and define the parameters as an indented dictionary
 
   .. code:: yaml
@@ -83,16 +81,75 @@ Functions
 
 Tables
 ^^^^^^
-A dictionary of table names, each resolving to a dictionary of column names for that table.
 
-* Create a table
-* Add a column
-* Multi-column assignment
-* Table.init and table.complete dependencies
+* `Create a table`: a dictionary of table names, each resolving to a dictionary of column names for that table.
+
+  Let us create a table called lottery with a column to store the lottery results following a uniform distribution
+
+  .. code:: yaml
+
+      tables:
+        lottery:
+          results: !numpy.rand.random
+            low: 0
+            high: 9
+
+* `Add a column`: you can add as many columns to a table as you need.
+    Imagine you want to add a column to our lottery table to include whether you won the lottery (returning ``True`` or ``False``)
+
+  .. code:: yaml
+
+      tables:
+        lottery:
+          results: !numpy.rand.random
+            low: 0
+            high: 9
+          win: !bool
+            x: !numpy.random.randint [ 2 ]
+
+* `Reference a column`: columns in the pipeline can be referenced by their full name tagged with a dollar sign ``$``.
+  For example, you create a table  called ``motion`` with three columns storing the position, the time and the speed of the object.
+  The column ``speed`` will refer to the other columns
+
+  .. code:: yaml
+
+    tables:
+      motion:
+        position: !np.linspace
+          start: 0.
+          stop: 10.5
+          num: 5
+        time: !np.arange [0, 25, 5]
+        speed: !numpy.divide
+          x1: $motion.position
+          x2: $motion.time
+
+
+* `Multi-column assignment`: if a function returns multiple columns, you can chose to assign them to multiple columns with different names or to a muti-column object.
+
+  Example: imagine the function is a 2-dimensional ``numpy.ndarray``. You could choose a multi-column assignment
+
+  .. code:: yaml
+
+    tables:
+      mytable:
+        a, b: !numpy.ndarray [ [ 1,2,3 ] , [ 4,5,6 ] ]
+
+  or
+
+  .. code:: yaml
+
+    tables:
+      mytable:
+        my2darray: !numpy.ndarray [ [ 1,2,3 ] , [ 4,5,6 ] ]
+
+
+* `Table.init and table.complete dependencies`:
 
 Cosmology, a special parameter
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-The cosmology to be used by functions within the pipeline.
+* `Define parameters`: parameters are variables that can be modified at execution.
+* The cosmology to be used by functions within the pipeline.
 
 .. _YAML: https://yaml.org
 .. _NumPy: https://numpy.org

From 7d0ec30685538b5c733194c737062613649bfff4 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 23 Mar 2021 12:13:54 +0000
Subject: [PATCH 05/77] Fix typo

---
 docs/how_to_config_file.rst | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/how_to_config_file.rst b/docs/how_to_config_file.rst
index cf16efb18..64d850162 100644
--- a/docs/how_to_config_file.rst
+++ b/docs/how_to_config_file.rst
@@ -127,7 +127,7 @@ Tables
 
 * `Multi-column assignment`: if a function returns multiple columns, you can chose to assign them to multiple columns with different names or to a muti-column object.
 
-  Example: imagine the function is a 2-dimensional ``numpy.ndarray``. You could choose a multi-column assignment
+  Example: imagine the function is a 2-dimensional ``numpy.ndarray``. You could choose
 
   .. code:: yaml
 
@@ -135,7 +135,7 @@ Tables
       mytable:
         a, b: !numpy.ndarray [ [ 1,2,3 ] , [ 4,5,6 ] ]
 
-  or
+  or a multi-column assignment
 
   .. code:: yaml
 

From bc2945f7fd066dfa965846f3a5f1f49e88b1cb43 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 23 Mar 2021 12:18:49 +0000
Subject: [PATCH 06/77] Add guide for cosmology

---
 docs/how_to_config_file.rst | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/docs/how_to_config_file.rst b/docs/how_to_config_file.rst
index 64d850162..cf980c38d 100644
--- a/docs/how_to_config_file.rst
+++ b/docs/how_to_config_file.rst
@@ -148,8 +148,19 @@ Tables
 
 Cosmology, a special parameter
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
 * `Define parameters`: parameters are variables that can be modified at execution.
-* The cosmology to be used by functions within the pipeline.
+
+* The `cosmology` to be used by functions within the pipeline only needs to be set up at the top. If a function needs ``cosmology`` as an input, you need not define it again, it is automatically detected.
+
+  .. code:: yaml
+
+    parameters:
+      hubble_constant: 70
+      omega_matter: 0.3
+    cosmology: !astropy.cosmology.FlatLambdaCDM
+      H0: $hubble_constant
+      Om0: $omega_matter
 
 .. _YAML: https://yaml.org
 .. _NumPy: https://numpy.org

From 1f12a630aeb9fe30ad11a20efc6211ccb6a4834b Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 23 Mar 2021 12:29:50 +0000
Subject: [PATCH 07/77] Add walkthrough example

---
 docs/how_to_config_file.rst | 58 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 57 insertions(+), 1 deletion(-)

diff --git a/docs/how_to_config_file.rst b/docs/how_to_config_file.rst
index cf980c38d..2538560a6 100644
--- a/docs/how_to_config_file.rst
+++ b/docs/how_to_config_file.rst
@@ -162,6 +162,8 @@ Cosmology, a special parameter
       H0: $hubble_constant
       Om0: $omega_matter
 
+
+
 .. _YAML: https://yaml.org
 .. _NumPy: https://numpy.org
 
@@ -170,9 +172,63 @@ Cosmology, a special parameter
 Walkthrough example
 -------------------
 
-This walkthrough example shows the natural flow of SkyPy pipelines and
+This dialog-like walkthrough example shows the natural flow of SkyPy pipelines and
 how to think through the process of creating a general configuration file.
 You can find more complex examples_ in our documentation.
 
+* `SkyPy`: Hi! This is SkyPy, how can I help?
+* `User`: Hi! I need to sample redshifts and magnitudes from a Schechter function. I would like to run my own pipeline within SkyPy.
+* `SkyPy`: that’s brilliant! Do you have your own function or is it included in SkyPy or any other compatible package?
+* `User`: I choose the SkyPy luminosity function, `~skypy.galaxies.schechter_lf`
+* `SkyPy`: Nice choice! But remember you can always use other libraries, as SkyPy has the flexibility to interface with external softwares.
+  The parameters for the SkyPy luminosity function, `~skypy.galaxies.schechter_lf`
+  are: redshift, the characteristic absolute magnitude, the amplitude, faint-end slope parameter, the magnitude limit, the fraction of sky, cosmology and noise.
+  Would you need to reuse these parameters?
+* `User`: yes, all of them except for the Schechter parameters. I will also use the default value for noise.
+* `SkyPy`: brill! You can define these variables at the top of your config file
+
+  .. code:: yaml
+
+    cosmology: !astropy.cosmology.default_cosmology.get
+    z_range: !numpy.linspace [0, 2, 21]
+    magnitude_limit: 23
+    sky_area: 10 deg2
+
+* `User`: I would like to create a table with a column for the blue galaxies, as I intend to also include more features later on.
+* `SkyPy`: in that case, you can create the table `blue_galaxies` and for now add the columns for redshift and magnitude (note here the ``schechter_lf`` returns a 2D object)
+
+  .. code:: yaml
+
+    tables:
+      blue_galaxies:
+        redshift, magnitude: !skypy.galaxies.schechter_lf
+      		redshift: $z_range
+      		M_star: 20
+      		phi_star: 3e-3
+      		alpha: -1.3
+      		m_lim: $magnitude_limit
+      		sky_area: $sky_area
+
+* `User`: Why didn’t you define the cosmology parameter?
+* `SkyPy`: Aha! Good question! Remember, if cosmology is detected as parameter but is not set, it automatically uses the variable at the top of the file.
+  This is how your entire config file looks like! You can now save it as ``luminosity.yml`` and run it using our SkyPy `~skypy.pipeline.Pipeline`!
+
+  .. code:: yaml
+
+    cosmology: !astropy.cosmology.default_cosmology.get
+    z_range: !numpy.linspace [0, 2, 21]
+    magnitude_limit: 23
+    sky_area: 10 deg2
+    tables:
+      blue_galaxies:
+        redshift, magnitude: !skypy.galaxies.schechter_lf
+      		redshift: $z_range
+      		M_star: 20
+      		phi_star: 3e-3
+      		alpha: -1.3
+      		m_lim: $magnitude_limit
+      		sky_area: $sky_area
+
+Don’t forget to check out for more complete examples_!
 
 .. _examples: https://skypy.readthedocs.io/en/stable/examples/index.html

From f806f84144ea9a96d778d9463cda8120f0e1b7a2 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 23 Mar 2021 14:12:24 +0000
Subject: [PATCH 08/77] Change example for variables

---
 docs/how_to_config_file.rst | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/docs/how_to_config_file.rst b/docs/how_to_config_file.rst
index 2538560a6..3ee5d1ac2 100644
--- a/docs/how_to_config_file.rst
+++ b/docs/how_to_config_file.rst
@@ -34,17 +34,17 @@ Variables
 
 
 * `Reference a variable`: variables can be referenced by their full name tagged with a dollar sign ``$``.
-  For example, if you previously defined a list of countries:
+  For example, if you previously defined a list of galaxy properties:
 
   .. code:: yaml
 
-      country_list: [ 'UK', 'Spain', 'Zambia', 'Chile', 'Japan' ]
+      galaxy_properties: [ 'OBJECT_ID', 'RA', 'DEC', 'REDSHIFT', 'FLUX', 'FLUX_ERR' ]
 
-  You would reference the variable;
+  You could reference the variable:
 
   .. code:: yaml
 
-      countries: $country_list
+      catalog: $galaxy_properties
 
 
 
@@ -68,7 +68,7 @@ Functions
         x1: 230000
         x2: 15589.3
 
-  or you could also define the variables at the top and then reference them
+  or you could also define the variables at the top level and then reference them
 
   .. code:: yaml
 

From 2e65b25b9e4a09bc8c3dd030a098f1fce0587a89 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 23 Mar 2021 14:24:37 +0000
Subject: [PATCH 09/77] Change example for functions

---
 docs/how_to_config_file.rst | 16 +++++++---------
 1 file changed, 7 insertions(+), 9 deletions(-)

diff --git a/docs/how_to_config_file.rst b/docs/how_to_config_file.rst
index 3ee5d1ac2..52fe53d53 100644
--- a/docs/how_to_config_file.rst
+++ b/docs/how_to_config_file.rst
@@ -60,23 +60,21 @@ Functions
       myarray: !numpy.linspace [0, 2.5, 10]
 
   You can also define parameters of functions with a dictionary of keyword arguments.
-  Imagine you want to compute the total expense when buying a house (£230000) and a car (£15589.3). To run it with the `SkyPy` pipeline, you would call the function and define the parameters as an indented dictionary
+  Imagine you want to compute the comoving distance for a range of redshifts and an `Astropy` Planck 2015 cosmology.
+  To run it with the `SkyPy` pipeline, you would call the function and define the parameters as an indented dictionary
 
   .. code:: yaml
 
-      expense: !numpy.add
-        x1: 230000
-        x2: 15589.3
+      comoving_distance: !astropy.cosmology.Planck15.comoving_distance
+        z: !numpy.linspace [ 0, 1.3, num=10 ]
 
   or you could also define the variables at the top level and then reference them
 
   .. code:: yaml
 
-      house_price: 230000
-      car_price: 15589.3
-      expense: !numpy.add
-        x1: $house_price
-        x2: $car_price
+      redshift: !numpy.linspace [ 0, 1.3, num=10 ]
+      comoving_distance: !astropy.cosmology.Planck15.comoving_distance
+        z: $redshift
 
 
 Tables

From 6cadf8364b0211c424c8a88d9ebf987e761473c3 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 23 Mar 2021 15:13:37 +0000
Subject: [PATCH 10/77] Change example for tables and columns

---
 docs/how_to_config_file.rst | 50 ++++++++++++++++++++-----------------
 1 file changed, 27 insertions(+), 23 deletions(-)

diff --git a/docs/how_to_config_file.rst b/docs/how_to_config_file.rst
index 52fe53d53..c33a10c5a 100644
--- a/docs/how_to_config_file.rst
+++ b/docs/how_to_config_file.rst
@@ -82,45 +82,49 @@ Tables
 
 * `Create a table`: a dictionary of table names, each resolving to a dictionary of column names for that table.
 
-  Let us create a table called lottery with a column to store the lottery results following a uniform distribution
+  Let us create a table called ``telescope`` with a column to store the width of spectral lines that follow a normal distribution
 
   .. code:: yaml
 
       tables:
-        lottery:
-          results: !numpy.rand.random
-            low: 0
-            high: 9
+        telescope:
+          spectral_lines: !scipy.stats.norm
+            loc: 550
+            scale: 1.6
+            size: 100
 
 * `Add a column`: you can add as many columns to a table as you need.
-    Imagine you want to add a column to our lottery table to include whether you won the lottery (returning ``True`` or ``False``)
+    Imagine you want to add a column for the telescope collecting surface
 
   .. code:: yaml
 
       tables:
-        lottery:
-          results: !numpy.rand.random
-            low: 0
-            high: 9
-          win: !bool
-            x: !numpy.random.randint [ 2 ]
+        telescope:
+          spectral_lines: !scipy.stats.norm
+            loc: 550
+            scale: 1.6
+            size: 100
+          collecting_surface: !numpy.random.uniform
+            low: 6.9
+            high: 7.1
 
 * `Reference a column`: columns in the pipeline can be referenced by their full name tagged with a dollar sign ``$``.
-  For example, you create a table  called ``motion`` with three columns storing the position, the time and the speed of the object.
-  The column ``speed`` will refer to the other columns
+  Example: the radius of cosmic voids seem to follow a lognormal distribution. You could create a table ``cosmic_voids``
+  with a column ``radii`` where you sample 1000 void sizes and two other columns, ``mean`` and ``variance`` that reference
+  the first column
+
 
   .. code:: yaml
 
     tables:
-      motion:
-        position: !np.linspace
-          start: 0.
-          stop: 10.5
-          num: 5
-        time: !np.arange [0, 25, 5]
-        speed: !numpy.divide
-          x1: $motion.position
-          x2: $motion.time
+      cosmic_voids:
+        radii: !scipy.stats.lognorm.rvs
+          s: 200.
+          size: 10000
+        mean: !numpy.mean
+          a: $radii
+        variance: !numpy.var
+          a: $radii
 
 
 * `Multi-column assignment`: if a function returns multiple columns, you can chose to assign them to multiple columns with different names or to a muti-column object.

From 13ea9eafcf1fc71933c6e8dfdf0dc49e5426b3b1 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 23 Mar 2021 15:32:49 +0000
Subject: [PATCH 11/77] Change example for multicomlumn assignment

---
 docs/how_to_config_file.rst | 30 ++++++++++++++++++++++++------
 1 file changed, 24 insertions(+), 6 deletions(-)

diff --git a/docs/how_to_config_file.rst b/docs/how_to_config_file.rst
index c33a10c5a..2526bb340 100644
--- a/docs/how_to_config_file.rst
+++ b/docs/how_to_config_file.rst
@@ -110,7 +110,7 @@ Tables
 
 * `Reference a column`: columns in the pipeline can be referenced by their full name tagged with a dollar sign ``$``.
   Example: the radius of cosmic voids seem to follow a lognormal distribution. You could create a table ``cosmic_voids``
-  with a column ``radii`` where you sample 1000 void sizes and two other columns, ``mean`` and ``variance`` that reference
+  with a column ``radii`` where you sample 10000 void sizes and two other columns, ``mean`` and ``variance`` that reference
   the first column
 
 
@@ -129,21 +129,32 @@ Tables
 
 * `Multi-column assignment`: if a function returns multiple columns, you can chose to assign them to multiple columns with different names or to a muti-column object.
 
-  Example: imagine the function is a 2-dimensional ``numpy.ndarray``. You could choose
+  Imagine you want the distribution of the circular velocities of 1000 halos following a Maxwellian distribution.
+  The histogram NumPy_ returns a 2-dimensional object. Therefore, you could choose
 
   .. code:: yaml
 
     tables:
-      mytable:
-        a, b: !numpy.ndarray [ [ 1,2,3 ] , [ 4,5,6 ] ]
+      halos:
+        circular_velocities: !scipy.stats.maxwell.rvs
+          s: 250
+          size: 1000
+        hist, bin_edges: !numpy.histogram
+          a: $circular_velocities
+          density: True
 
   or a multi-column assignment
 
   .. code:: yaml
 
     tables:
-      mytable:
-        my2darray: !numpy.ndarray [ [ 1,2,3 ] , [ 4,5,6 ] ]
+      halos:
+        circular_velocities: !scipy.stats.maxwell.rvs
+          s: 250
+          size: 1000
+        histogram: !numpy.histogram
+          a: $circular_velocities
+          density: True
 
 
 * `Table.init and table.complete dependencies`:
@@ -153,6 +164,13 @@ Cosmology, a special parameter
 
 * `Define parameters`: parameters are variables that can be modified at execution.
 
+  For example,
+  .. code:: yaml
+
+    parameters:
+      hubble_constant: 70
+      omega_matter: 0.3
+
 * The `cosmology` to be used by functions within the pipeline only needs to be set up at the top. If a function needs ``cosmology`` as an input, you need not define it again, it is automatically detected.
 
   .. code:: yaml

From 57ea54d8b0e2069b3b6438909e5492ae11a06061 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 23 Mar 2021 15:43:28 +0000
Subject: [PATCH 12/77] Add example for .ini and depends

---
 docs/how_to_config_file.rst | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/docs/how_to_config_file.rst b/docs/how_to_config_file.rst
index 2526bb340..450bbe4b5 100644
--- a/docs/how_to_config_file.rst
+++ b/docs/how_to_config_file.rst
@@ -158,6 +158,29 @@ Tables
 
 
 * `Table.init and table.complete dependencies`:
+  This needs work
+
+  .. code:: yaml
+
+    tables:
+      halos:
+        halo_mass: !scipy.stats.maxwell.rvs
+          s: 250
+          size: 1000
+        galaxies:
+          luminosity: !numpy.histogram
+      matching:
+        init: !Colossus_abundance_matching_function_or_vstack
+          halos: $halos
+          galaxies: $galaxies
+          depends: [ halos.complete, galaxies.complete ]
+
+  Equivalently, you could replace the last line by
+
+  .. code:: yaml
+
+    depends: [ halos.halo_mass, galaxies.luminosity ]
+
 
 Cosmology, a special parameter
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -165,6 +188,7 @@ Cosmology, a special parameter
 * `Define parameters`: parameters are variables that can be modified at execution.
 
   For example,
+
   .. code:: yaml
 
     parameters:

From b28a40d445b0c165dd165f0253b502f2cda4e79d Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 25 Mar 2021 17:07:50 +0000
Subject: [PATCH 13/77] Add description for init and complete

---
 docs/how_to_config_file.rst | 58 +++++++++++++++++++++++++++----------
 1 file changed, 43 insertions(+), 15 deletions(-)

diff --git a/docs/how_to_config_file.rst b/docs/how_to_config_file.rst
index 450bbe4b5..03a739384 100644
--- a/docs/how_to_config_file.rst
+++ b/docs/how_to_config_file.rst
@@ -19,6 +19,7 @@ This guide introduces the main syntax of YAML relevant when writing
 a configuration file to use with ``SkyPy``. Essentially, it would start with
 definitions of individual variables at the top, followed by the tables to produce,
 and, within the table entries, the features of the objects to simulate are included.
+The main keywords: ``parameters``, ``cosmology``, ``tables``.
 
 
 Variables
@@ -157,29 +158,56 @@ Tables
           density: True
 
 
-* `Table.init and table.complete dependencies`:
-  This needs work
+* `Referencing tables: table.init and table.complete dependencies`.
+
+  Most of the time you would be referencing simple
+  variables. However there are times when your function depends on tables. In these
+  cases, you would need to ensure the referenced table has the necessary content and is not empty.
+
+  Imagine you want to perform a very simple abundance matching, painting galaxies within your halos.
+  For this you could create two tables ``halos`` and ``galaxies`` storing the halo mass and galaxy luminosities.
+  The idea is to stack these two tables and store it in a third table called ``matching``. In principle you could do:
 
   .. code:: yaml
 
     tables:
       halos:
-        halo_mass: !scipy.stats.maxwell.rvs
-          s: 250
-          size: 1000
-        galaxies:
-          luminosity: !numpy.histogram
-      matching:
-        init: !Colossus_abundance_matching_function_or_vstack
-          halos: $halos
-          galaxies: $galaxies
-          depends: [ halos.complete, galaxies.complete ]
-
-  Equivalently, you could replace the last line by
+        halo_mass: !mylibrary.my_halo_sampler
+          m_min: 1.0e8
+          m_max: 1.0e14
+      galaxies:
+        luminosity: !mylibrary.my_schechter_function
+          alpha: 1.20
+      matching_wrong:
+        match: !numpy.vstack
+          tup: [ $halos, $galaxies ]
+
+  This would probably not do what you intend.
+  For example, if you have a table called ``tableA`` with columns ``c1`` and ``c2``.
+  In configuration language, ``tableA`` is the name of the job.
+  That means, when executing the config file, the first thing that happens is call ``tableA``, second,  call ``tableA.c1`` and third, call ``tableA.c2``.
+  In our example, when you call the function ``numpy.vstack()`` and reference the tables ``$halos`` and ``$galaxies``, this is actually
+  referencing the job that initialises the empty table ``halos`` and ``galaxies``.
+  The potential risk is that the function could be called before the jobs ``halos`` and ``galaxies`` are finished, so the tables would be empty
+
+  To overcome this issue you can initialise your ``matching`` table with ``init``, specify their dependences with the keyword ``depends``
+  and ensure the tables are completed before calling the function with ``.complete``. The previous configuration file reads now:
 
   .. code:: yaml
 
-    depends: [ halos.halo_mass, galaxies.luminosity ]
+    tables:
+      halos:
+        halo_mass: !mylibrary.my_halo_sampler
+          m_min: 1.0e8
+          m_max: 1.0e14
+      galaxies:
+        luminosity: !mylibrary.my_schechter_function
+          alpha: 1.20
+      matching:
+        init: !numpy.vstack
+          tup: [ $halos, $galaxies ]
+          depends: [ tup.complete ]
+
 
 
 Cosmology, a special parameter

From ba8b27a27c7bed93cbf76bb2834661448ead107d Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 25 Mar 2021 17:14:38 +0000
Subject: [PATCH 14/77] Fix typos

---
 docs/how_to_config_file.rst | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/docs/how_to_config_file.rst b/docs/how_to_config_file.rst
index 03a739384..b85bffcd1 100644
--- a/docs/how_to_config_file.rst
+++ b/docs/how_to_config_file.rst
@@ -253,7 +253,8 @@ You can find more complex examples_ in our documentation.
 * `SkyPy`: that’s brilliant! Do you have your own function or is it included in SkyPy or any other compatible package?
 * `User`: I choose the SkyPy luminosity function, `~skypy.galaxies.schechter_lf`
 * `SkyPy`: Nice choice! But remember you can always use other libraries, as SkyPy has the flexibility to interface with external softwares.
-  The parameters for the SkyPy luminosity function, `~skypy.galaxies.schechter_lf`
+  T
+  he parameters for the SkyPy luminosity function, `~skypy.galaxies.schechter_lf`
   are: redshift, the characteristic absolute magnitude, the amplitude, faint-end slope parameter, the magnitude limit, the fraction of sky, cosmology and noise.
   Would you need to reuse these parameters?
 * `User`: yes, all of them except for the Schechter parameters. I will also use the default value for noise.
@@ -267,7 +268,7 @@ You can find more complex examples_ in our documentation.
     sky_area: 10 deg2
 
 * `User`: I would like to create a table with a column for the blue galaxies, as I intend to also include more features later on.
-* `SkyPy`: in that case, you can create the table `blue_galaxies` and for now add the columns for redshift and magnitude (note here the ``schechter_lf`` returns a 2D object)
+* `SkyPy`: in that case, you can create the table ```blue_galaxies`` and for now add the columns for redshift and magnitude (note here the ``schechter_lf`` returns a 2D object)
 
   .. code:: yaml
 
@@ -281,8 +282,9 @@ You can find more complex examples_ in our documentation.
       		m_lim: $magnitude_limit
       		sky_area: $sky_area
 
-* `User`: Why didn’t you define the cosmology parameter?
-* `SkyPy`: Aha! Good question! Remember, if cosmology is detected as parameter but is not set, it automatically uses the variable at the top of the file.
+* `User`: Why didn’t you define the parameter ``cosmology``?
+* `SkyPy`: Aha! Good question! Remember, if cosmology is detected as parameter but is not set, it automatically uses the variable at the top level of the file.
+
   This is how your entire config file looks like! You can now save it as ``luminosity.yml`` and run it using our SkyPy `~skypy.pipeline.Pipeline`!
 
   .. code:: yaml

From 8a4cb0eefd806ab6fea7cf2a4d94d657667228d4 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Mon, 29 Mar 2021 17:22:29 +0100
Subject: [PATCH 15/77] Fix name of halo function

---
 docs/how_to_config_file.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/how_to_config_file.rst b/docs/how_to_config_file.rst
index b85bffcd1..e2cd90d86 100644
--- a/docs/how_to_config_file.rst
+++ b/docs/how_to_config_file.rst
@@ -172,7 +172,7 @@ Tables
 
     tables:
       halos:
-        halo_mass: !mylibrary.my_halo_sampler
+        halo_mass: !mylibrary.my_halo_mass_function
           m_min: 1.0e8
           m_max: 1.0e14
       galaxies:

From 38bf2e78861143dfad8f6e65c47bba8b83a71a4e Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Mon, 29 Mar 2021 18:16:24 +0100
Subject: [PATCH 16/77] Change style for walkthrough example

---
 docs/how_to_config_file.rst | 41 +++++++++++++++++--------------------
 1 file changed, 19 insertions(+), 22 deletions(-)

diff --git a/docs/how_to_config_file.rst b/docs/how_to_config_file.rst
index e2cd90d86..31b481fe3 100644
--- a/docs/how_to_config_file.rst
+++ b/docs/how_to_config_file.rst
@@ -197,7 +197,7 @@ Tables
 
     tables:
       halos:
-        halo_mass: !mylibrary.my_halo_sampler
+        halo_mass: !mylibrary.my_halo_mass_function
           m_min: 1.0e8
           m_max: 1.0e14
       galaxies:
@@ -241,24 +241,21 @@ Cosmology, a special parameter
 
 
 
-Walkthrough example
--------------------
+SkyPy example
+-------------
+
+The guidelines above teach you how to write a configuration file when you have already thought about the physical problem you want to solve, the model and the functions to compute.
+These functions can be your own implementation, come from an external software or be part of the ``SkyPy`` library. Remember ``SkyPy`` is a library of astrophysical models, but mainly is infrastructure and a tool for you to run your own
+simulations of the Universe. In this last section, we show you how you can run a ``SkyPy`` pipeline. You can find more complex examples_ in our documentation.
 
-This dialog-like walkthrough example shows the natural flow of SkyPy pipelines and
-how to think through the process of creating a general configuration file.
-You can find more complex examples_ in our documentation.
+In this example, we sample redshifts and magnitudes from the SkyPy luminosity function, `~skypy.galaxies.schechter_lf`.
 
-* `SkyPy`: Hi! This is SkyPy, how can I help?
-* `User`: Hi! I need to sample redshifts and magnitudes from a Schechter function. I would like to run my own pipeline within SkyPy.
-* `SkyPy`: that’s brilliant! Do you have your own function or is it included in SkyPy or any other compatible package?
-* `User`: I choose the SkyPy luminosity function, `~skypy.galaxies.schechter_lf`
-* `SkyPy`: Nice choice! But remember you can always use other libraries, as SkyPy has the flexibility to interface with external softwares.
-  T
-  he parameters for the SkyPy luminosity function, `~skypy.galaxies.schechter_lf`
-  are: redshift, the characteristic absolute magnitude, the amplitude, faint-end slope parameter, the magnitude limit, the fraction of sky, cosmology and noise.
-  Would you need to reuse these parameters?
-* `User`: yes, all of them except for the Schechter parameters. I will also use the default value for noise.
-* `SkyPy`: brill! You can define these variables at the top of your config file
+- `Define variables`:
+
+From the documentation, the parameters for the `~skypy.galaxies.schechter_lf` function are: ``redshift``, the characteristic absolute magnitude ``M_star``, the amplitude ``phi_star``, faint-end slope parameter ``alpha``,
+the magnitude limit ``magnitude_limit``, the fraction of sky ``sky_area``, ``cosmology`` and ``noise``.
+If you are planning to reuse some of these parameters, you could define them at the top-level of your configuration file.
+Also, ``noise`` is an optional parameter and you could use its default value by omitting its definition.
 
   .. code:: yaml
 
@@ -267,8 +264,9 @@ You can find more complex examples_ in our documentation.
     magnitude_limit: 23
     sky_area: 10 deg2
 
-* `User`: I would like to create a table with a column for the blue galaxies, as I intend to also include more features later on.
-* `SkyPy`: in that case, you can create the table ```blue_galaxies`` and for now add the columns for redshift and magnitude (note here the ``schechter_lf`` returns a 2D object)
+- `Tables and columns`:
+
+You can create a table ``blue_galaxies`` and for now add the columns for redshift and magnitude (note here the ``schechter_lf`` returns a 2D object)
 
   .. code:: yaml
 
@@ -282,10 +280,9 @@ You can find more complex examples_ in our documentation.
       		m_lim: $magnitude_limit
       		sky_area: $sky_area
 
-* `User`: Why didn’t you define the parameter ``cosmology``?
-* `SkyPy`: Aha! Good question! Remember, if cosmology is detected as parameter but is not set, it automatically uses the variable at the top level of the file.
+Remember, if cosmology is detected as parameter but is not set, it automatically uses the variable at the top-level of the file.
 
-  This is how your entire config file looks like! You can now save it as ``luminosity.yml`` and run it using our SkyPy `~skypy.pipeline.Pipeline`!
+This is how the entire configuration file looks like! You can now save it as ``luminosity.yml`` and run it using our SkyPy `~skypy.pipeline.Pipeline`!
 
   .. code:: yaml
 

From 44b3c8aca9d47223c6df1cd02bdbcc1039110d0a Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 6 Apr 2021 15:53:24 +0100
Subject: [PATCH 17/77] Fix SkyPy example

---
 docs/how_to_config_file.rst | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/docs/how_to_config_file.rst b/docs/how_to_config_file.rst
index 31b481fe3..aeed22241 100644
--- a/docs/how_to_config_file.rst
+++ b/docs/how_to_config_file.rst
@@ -255,12 +255,15 @@ In this example, we sample redshifts and magnitudes from the SkyPy luminosity fu
 From the documentation, the parameters for the `~skypy.galaxies.schechter_lf` function are: ``redshift``, the characteristic absolute magnitude ``M_star``, the amplitude ``phi_star``, faint-end slope parameter ``alpha``,
 the magnitude limit ``magnitude_limit``, the fraction of sky ``sky_area``, ``cosmology`` and ``noise``.
 If you are planning to reuse some of these parameters, you could define them at the top-level of your configuration file.
+In our example, we are using ``Astropy`` linear and exponential models for the characteristic absolute magnitude and the amplitude, respectively.
 Also, ``noise`` is an optional parameter and you could use its default value by omitting its definition.
 
   .. code:: yaml
 
     cosmology: !astropy.cosmology.default_cosmology.get
     z_range: !numpy.linspace [0, 2, 21]
+    M_star: !astropy.modeling.models.Linear1D [-0.9, -20.4]
+    phi_star: !astropy.modeling.models.Exponential1D [3e-3, -9.7]
     magnitude_limit: 23
     sky_area: 10 deg2
 
@@ -274,8 +277,8 @@ You can create a table ``blue_galaxies`` and for now add the columns for redshif
       blue_galaxies:
         redshift, magnitude: !skypy.galaxies.schechter_lf
       		redshift: $z_range
-      		M_star: 20
-      		phi_star: 3e-3
+      		M_star: $M_star
+      		phi_star: $phi_star
       		alpha: -1.3
       		m_lim: $magnitude_limit
       		sky_area: $sky_area
@@ -288,17 +291,19 @@ This is how the entire configuration file looks like! You can now save it as ``l
 
     cosmology: !astropy.cosmology.default_cosmology.get
     z_range: !numpy.linspace [0, 2, 21]
+    M_star: !astropy.modeling.models.Linear1D [-0.9, -20.4]
+    phi_star: !astropy.modeling.models.Exponential1D [3e-3, -9.7]
     magnitude_limit: 23
     sky_area: 10 deg2
     tables:
       blue_galaxies:
         redshift, magnitude: !skypy.galaxies.schechter_lf
-      		redshift: $z_range
-      		M_star: 20
-      		phi_star: 3e-3
-      		alpha: -1.3
-      		m_lim: $magnitude_limit
-      		sky_area: $sky_area
+          redshift: $z_range
+          M_star: $M_star
+          phi_star: $phi_star
+          alpha: -1.3
+          m_lim: $magnitude_limit
+          sky_area: $sky_area
 
 Don’t forget to check out for more complete examples_!
 

From 32fae723b47d2e504e38668d0e44832828483bac Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 13 Apr 2021 15:55:39 +0100
Subject: [PATCH 18/77] Rename file

---
 docs/{how_to_config_file.rst => configuration_files} | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 rename docs/{how_to_config_file.rst => configuration_files} (100%)

diff --git a/docs/how_to_config_file.rst b/docs/configuration_files
similarity index 100%
rename from docs/how_to_config_file.rst
rename to docs/configuration_files

From c1cc9296e1706a1a2ff414ac15e02cdb725720f6 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 13 Apr 2021 15:56:30 +0100
Subject: [PATCH 19/77] Fix rename file extension

---
 docs/{configuration_files => configuration_files.rst} | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 rename docs/{configuration_files => configuration_files.rst} (100%)

diff --git a/docs/configuration_files b/docs/configuration_files.rst
similarity index 100%
rename from docs/configuration_files
rename to docs/configuration_files.rst

From f3b542bb5429708df34492db927193537c7f2a97 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 13 Apr 2021 15:58:18 +0100
Subject: [PATCH 20/77] Rename config file section on main doc page

---
 docs/index.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/index.rst b/docs/index.rst
index b6087297c..9b3470fba 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -17,7 +17,7 @@ Getting Started
    :maxdepth: 1
 
    install
-   how_to_config_file
+   configuration_files
    examples/index
 
 .. _user-docs:

From dd28a23a2c5ac135687979b6acb56a444906abcd Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 13 Apr 2021 16:06:41 +0100
Subject: [PATCH 21/77] Edit for positive wording :)

---
 docs/configuration_files.rst | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index aeed22241..4ca4b22de 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -5,9 +5,10 @@ Configuration files
 This page outlines how to construct configuration files to run your own routines
 with `~skypy.pipeline.Pipeline`.
 
-Remember that SkyPy not only is a library of astronomical models but a flexible
-tool with infrastructure for you to use with your own
-functions and pipelines!
+`SkyPy` is an astrophysical simulation pipeline tool that allows to define any
+arbitrary workflow and store data in table format. You may use `SkyPy Pipeline`
+to call any function --your own, from any compatible external software or from the `SkyPy library`.
+`SkyPy` deals with the data dependencies and provides a library of functions to be used with it.
 
 YAML syntax
 -----------

From b07dba39ff473c6caaf01ef575038e8c3852b643 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 13 Apr 2021 16:10:49 +0100
Subject: [PATCH 22/77] Change sections order to show example first

---
 docs/configuration_files.rst | 143 ++++++++++++++++++-----------------
 1 file changed, 73 insertions(+), 70 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 4ca4b22de..c9e1729ce 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -10,6 +10,79 @@ arbitrary workflow and store data in table format. You may use `SkyPy Pipeline`
 to call any function --your own, from any compatible external software or from the `SkyPy library`.
 `SkyPy` deals with the data dependencies and provides a library of functions to be used with it.
 
+This guidelines starts with an example using one of the `SkyPy` functions, and it follows
+the concrete YAML syntax necessary for you to write your own configuration files, beyond using the `SkyPy`
+functions.
+
+SkyPy example
+-------------
+
+The guidelines above teach you how to write a configuration file when you have already thought about the physical problem you want to solve, the model and the functions to compute.
+These functions can be your own implementation, come from an external software or be part of the ``SkyPy`` library. Remember ``SkyPy`` is a library of astrophysical models, but mainly is infrastructure and a tool for you to run your own
+simulations of the Universe. In this last section, we show you how you can run a ``SkyPy`` pipeline. You can find more complex examples_ in our documentation.
+
+In this example, we sample redshifts and magnitudes from the SkyPy luminosity function, `~skypy.galaxies.schechter_lf`.
+
+- `Define variables`:
+
+From the documentation, the parameters for the `~skypy.galaxies.schechter_lf` function are: ``redshift``, the characteristic absolute magnitude ``M_star``, the amplitude ``phi_star``, faint-end slope parameter ``alpha``,
+the magnitude limit ``magnitude_limit``, the fraction of sky ``sky_area``, ``cosmology`` and ``noise``.
+If you are planning to reuse some of these parameters, you could define them at the top-level of your configuration file.
+In our example, we are using ``Astropy`` linear and exponential models for the characteristic absolute magnitude and the amplitude, respectively.
+Also, ``noise`` is an optional parameter and you could use its default value by omitting its definition.
+
+  .. code:: yaml
+
+    cosmology: !astropy.cosmology.default_cosmology.get
+    z_range: !numpy.linspace [0, 2, 21]
+    M_star: !astropy.modeling.models.Linear1D [-0.9, -20.4]
+    phi_star: !astropy.modeling.models.Exponential1D [3e-3, -9.7]
+    magnitude_limit: 23
+    sky_area: 10 deg2
+
+- `Tables and columns`:
+
+You can create a table ``blue_galaxies`` and for now add the columns for redshift and magnitude (note here the ``schechter_lf`` returns a 2D object)
+
+  .. code:: yaml
+
+    tables:
+      blue_galaxies:
+        redshift, magnitude: !skypy.galaxies.schechter_lf
+      		redshift: $z_range
+      		M_star: $M_star
+      		phi_star: $phi_star
+      		alpha: -1.3
+      		m_lim: $magnitude_limit
+      		sky_area: $sky_area
+
+Remember, if cosmology is detected as parameter but is not set, it automatically uses the variable at the top-level of the file.
+
+This is how the entire configuration file looks like! You can now save it as ``luminosity.yml`` and run it using our SkyPy `~skypy.pipeline.Pipeline`!
+
+  .. code:: yaml
+
+    cosmology: !astropy.cosmology.default_cosmology.get
+    z_range: !numpy.linspace [0, 2, 21]
+    M_star: !astropy.modeling.models.Linear1D [-0.9, -20.4]
+    phi_star: !astropy.modeling.models.Exponential1D [3e-3, -9.7]
+    magnitude_limit: 23
+    sky_area: 10 deg2
+    tables:
+      blue_galaxies:
+        redshift, magnitude: !skypy.galaxies.schechter_lf
+          redshift: $z_range
+          M_star: $M_star
+          phi_star: $phi_star
+          alpha: -1.3
+          m_lim: $magnitude_limit
+          sky_area: $sky_area
+
+Don’t forget to check out for more complete examples_!
+
+.. _examples: https://skypy.readthedocs.io/en/stable/examples/index.html
+
+
 YAML syntax
 -----------
 YAML_ is a file format designed to be readable by both computers and humans.
@@ -239,73 +312,3 @@ Cosmology, a special parameter
 
 .. _YAML: https://yaml.org
 .. _NumPy: https://numpy.org
-
-
-
-SkyPy example
--------------
-
-The guidelines above teach you how to write a configuration file when you have already thought about the physical problem you want to solve, the model and the functions to compute.
-These functions can be your own implementation, come from an external software or be part of the ``SkyPy`` library. Remember ``SkyPy`` is a library of astrophysical models, but mainly is infrastructure and a tool for you to run your own
-simulations of the Universe. In this last section, we show you how you can run a ``SkyPy`` pipeline. You can find more complex examples_ in our documentation.
-
-In this example, we sample redshifts and magnitudes from the SkyPy luminosity function, `~skypy.galaxies.schechter_lf`.
-
-- `Define variables`:
-
-From the documentation, the parameters for the `~skypy.galaxies.schechter_lf` function are: ``redshift``, the characteristic absolute magnitude ``M_star``, the amplitude ``phi_star``, faint-end slope parameter ``alpha``,
-the magnitude limit ``magnitude_limit``, the fraction of sky ``sky_area``, ``cosmology`` and ``noise``.
-If you are planning to reuse some of these parameters, you could define them at the top-level of your configuration file.
-In our example, we are using ``Astropy`` linear and exponential models for the characteristic absolute magnitude and the amplitude, respectively.
-Also, ``noise`` is an optional parameter and you could use its default value by omitting its definition.
-
-  .. code:: yaml
-
-    cosmology: !astropy.cosmology.default_cosmology.get
-    z_range: !numpy.linspace [0, 2, 21]
-    M_star: !astropy.modeling.models.Linear1D [-0.9, -20.4]
-    phi_star: !astropy.modeling.models.Exponential1D [3e-3, -9.7]
-    magnitude_limit: 23
-    sky_area: 10 deg2
-
-- `Tables and columns`:
-
-You can create a table ``blue_galaxies`` and for now add the columns for redshift and magnitude (note here the ``schechter_lf`` returns a 2D object)
-
-  .. code:: yaml
-
-    tables:
-      blue_galaxies:
-        redshift, magnitude: !skypy.galaxies.schechter_lf
-      		redshift: $z_range
-      		M_star: $M_star
-      		phi_star: $phi_star
-      		alpha: -1.3
-      		m_lim: $magnitude_limit
-      		sky_area: $sky_area
-
-Remember, if cosmology is detected as parameter but is not set, it automatically uses the variable at the top-level of the file.
-
-This is how the entire configuration file looks like! You can now save it as ``luminosity.yml`` and run it using our SkyPy `~skypy.pipeline.Pipeline`!
-
-  .. code:: yaml
-
-    cosmology: !astropy.cosmology.default_cosmology.get
-    z_range: !numpy.linspace [0, 2, 21]
-    M_star: !astropy.modeling.models.Linear1D [-0.9, -20.4]
-    phi_star: !astropy.modeling.models.Exponential1D [3e-3, -9.7]
-    magnitude_limit: 23
-    sky_area: 10 deg2
-    tables:
-      blue_galaxies:
-        redshift, magnitude: !skypy.galaxies.schechter_lf
-          redshift: $z_range
-          M_star: $M_star
-          phi_star: $phi_star
-          alpha: -1.3
-          m_lim: $magnitude_limit
-          sky_area: $sky_area
-
-Don’t forget to check out for more complete examples_!
-
-.. _examples: https://skypy.readthedocs.io/en/stable/examples/index.html

From ea06960fe91c05851e1d14c5c5b2551d4c4b7ab5 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 13 Apr 2021 16:15:04 +0100
Subject: [PATCH 23/77] Fix typos

---
 docs/configuration_files.rst | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index c9e1729ce..7752f89f5 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -6,12 +6,12 @@ This page outlines how to construct configuration files to run your own routines
 with `~skypy.pipeline.Pipeline`.
 
 `SkyPy` is an astrophysical simulation pipeline tool that allows to define any
-arbitrary workflow and store data in table format. You may use `SkyPy Pipeline`
+arbitrary workflow and store data in table format. You may use `SkyPy` `~skypy.pipeline.Pipeline`.
 to call any function --your own, from any compatible external software or from the `SkyPy library`.
-`SkyPy` deals with the data dependencies and provides a library of functions to be used with it.
+Then `SkyPy` deals with the data dependencies and provides a library of functions to be used with it.
 
 This guidelines starts with an example using one of the `SkyPy` functions, and it follows
-the concrete YAML syntax necessary for you to write your own configuration files, beyond using the `SkyPy`
+the concrete YAML syntax necessary for you to write your own configuration files, beyond using `SkyPy`
 functions.
 
 SkyPy example

From 4afdec0aad47c3ef857df92c3dc4ea76fb155932 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 13 Apr 2021 16:22:41 +0100
Subject: [PATCH 24/77] Edit wording in example section as it now comes first

---
 docs/configuration_files.rst | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 7752f89f5..91ae71b70 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -10,17 +10,14 @@ arbitrary workflow and store data in table format. You may use `SkyPy` `~skypy.p
 to call any function --your own, from any compatible external software or from the `SkyPy library`.
 Then `SkyPy` deals with the data dependencies and provides a library of functions to be used with it.
 
-This guidelines starts with an example using one of the `SkyPy` functions, and it follows
+These guidelines start with an example using one of the `SkyPy` functions, and it follows
 the concrete YAML syntax necessary for you to write your own configuration files, beyond using `SkyPy`
 functions.
 
 SkyPy example
 -------------
 
-The guidelines above teach you how to write a configuration file when you have already thought about the physical problem you want to solve, the model and the functions to compute.
-These functions can be your own implementation, come from an external software or be part of the ``SkyPy`` library. Remember ``SkyPy`` is a library of astrophysical models, but mainly is infrastructure and a tool for you to run your own
-simulations of the Universe. In this last section, we show you how you can run a ``SkyPy`` pipeline. You can find more complex examples_ in our documentation.
-
+In this section, we exemplify how you can write a configuration file and use some of the `SkyPy` functions.
 In this example, we sample redshifts and magnitudes from the SkyPy luminosity function, `~skypy.galaxies.schechter_lf`.
 
 - `Define variables`:

From 48a12f18dfba9bec20a5dfffd4c8e161d8ad1a6c Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 13 Apr 2021 16:50:00 +0100
Subject: [PATCH 25/77] Change to active wording

---
 docs/configuration_files.rst | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 91ae71b70..ff716bc14 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -24,7 +24,7 @@ In this example, we sample redshifts and magnitudes from the SkyPy luminosity fu
 
 From the documentation, the parameters for the `~skypy.galaxies.schechter_lf` function are: ``redshift``, the characteristic absolute magnitude ``M_star``, the amplitude ``phi_star``, faint-end slope parameter ``alpha``,
 the magnitude limit ``magnitude_limit``, the fraction of sky ``sky_area``, ``cosmology`` and ``noise``.
-If you are planning to reuse some of these parameters, you could define them at the top-level of your configuration file.
+If you are planning to reuse some of these parameters, you can define them at the top-level of your configuration file.
 In our example, we are using ``Astropy`` linear and exponential models for the characteristic absolute magnitude and the amplitude, respectively.
 Also, ``noise`` is an optional parameter and you could use its default value by omitting its definition.
 
@@ -53,9 +53,9 @@ You can create a table ``blue_galaxies`` and for now add the columns for redshif
       		m_lim: $magnitude_limit
       		sky_area: $sky_area
 
-Remember, if cosmology is detected as parameter but is not set, it automatically uses the variable at the top-level of the file.
+`Important:` if cosmology is detected as a parameter but is not set, it automatically uses the cosmology variable defined at the top-level of the file.
 
-This is how the entire configuration file looks like! You can now save it as ``luminosity.yml`` and run it using our SkyPy `~skypy.pipeline.Pipeline`!
+This is how the entire configuration file looks like! You may now save it as ``luminosity.yml`` and run it using the `SkyPy` `~skypy.pipeline.Pipeline`!
 
   .. code:: yaml
 
@@ -87,10 +87,10 @@ Fundamentally, a file written in YAML consists of a set of key-value pairs.
 Each pair is written as ``key: value``, where whitespace after the ``:`` is optional.
 
 This guide introduces the main syntax of YAML relevant when writing
-a configuration file to use with ``SkyPy``. Essentially, it would start with
-definitions of individual variables at the top, followed by the tables to produce,
-and, within the table entries, the features of the objects to simulate are included.
-The main keywords: ``parameters``, ``cosmology``, ``tables``.
+a configuration file to use with ``SkyPy``. Essentially, it begins with
+definitions of individual variables at the top level, followed by the tables,
+and, within the table entries, the features of objects to simulate are included.
+Main keywords: ``parameters``, ``cosmology``, ``tables``.
 
 
 Variables
@@ -124,7 +124,7 @@ Functions
 ^^^^^^^^^
 * `Call a function`: functions are defined as tuples where the first entry is the fully qualified function name tagged with and exclamation mark ``!`` and the second entry is either a list of positional arguments or a dictionary of keyword arguments.
 
-  For example, you need to call the ``log10()`` and ``linspace()`` NumPy_ functions, for this you define the following key-value pairs:
+  For example, if you need to call the ``log10()`` and ``linspace()`` NumPy_ functions, then you define the following key-value pairs:
 
   .. code:: yaml
 
@@ -133,7 +133,7 @@ Functions
 
   You can also define parameters of functions with a dictionary of keyword arguments.
   Imagine you want to compute the comoving distance for a range of redshifts and an `Astropy` Planck 2015 cosmology.
-  To run it with the `SkyPy` pipeline, you would call the function and define the parameters as an indented dictionary
+  To run it with the `SkyPy` pipeline, call the function and define the parameters as an indented dictionary.
 
   .. code:: yaml
 
@@ -181,7 +181,7 @@ Tables
             high: 7.1
 
 * `Reference a column`: columns in the pipeline can be referenced by their full name tagged with a dollar sign ``$``.
-  Example: the radius of cosmic voids seem to follow a lognormal distribution. You could create a table ``cosmic_voids``
+  Example: the radius of cosmic voids seem to follow a lognormal distribution. You can create a table ``cosmic_voids``
   with a column ``radii`` where you sample 10000 void sizes and two other columns, ``mean`` and ``variance`` that reference
   the first column
 
@@ -202,7 +202,7 @@ Tables
 * `Multi-column assignment`: if a function returns multiple columns, you can chose to assign them to multiple columns with different names or to a muti-column object.
 
   Imagine you want the distribution of the circular velocities of 1000 halos following a Maxwellian distribution.
-  The histogram NumPy_ returns a 2-dimensional object. Therefore, you could choose
+  The histogram NumPy_ returns a 2-dimensional object. Therefore, you can choose
 
   .. code:: yaml
 
@@ -233,11 +233,11 @@ Tables
 
   Most of the time you would be referencing simple
   variables. However there are times when your function depends on tables. In these
-  cases, you would need to ensure the referenced table has the necessary content and is not empty.
+  cases, you need to ensure the referenced table has the necessary content and is not empty.
 
-  Imagine you want to perform a very simple abundance matching, painting galaxies within your halos.
-  For this you could create two tables ``halos`` and ``galaxies`` storing the halo mass and galaxy luminosities.
-  The idea is to stack these two tables and store it in a third table called ``matching``. In principle you could do:
+  Imagine you want to perform a very simple abundance matching, i.e. painting galaxies within your halos.
+  For this you can create two tables ``halos`` and ``galaxies`` storing the halo mass and galaxy luminosities.
+  The idea is to stack these two tables and store it in a third table called ``matching``. For example:
 
   .. code:: yaml
 

From 76299a25b7a279eb4776567d21e67c7e2f4756b1 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Mon, 19 Apr 2021 10:35:42 +0100
Subject: [PATCH 26/77] Add example plot

---
 docs/configuration_files.rst | 50 ++++++++++++++++++++++++------------
 docs/luminosity.yml          | 15 +++++++++++
 2 files changed, 48 insertions(+), 17 deletions(-)
 create mode 100644 docs/luminosity.yml

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index ff716bc14..bf6f4293b 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -55,25 +55,41 @@ You can create a table ``blue_galaxies`` and for now add the columns for redshif
 
 `Important:` if cosmology is detected as a parameter but is not set, it automatically uses the cosmology variable defined at the top-level of the file.
 
-This is how the entire configuration file looks like! You may now save it as ``luminosity.yml`` and run it using the `SkyPy` `~skypy.pipeline.Pipeline`!
+This is how the entire configuration file looks like!
+You may now save it as ``luminosity.yml`` and run it using the `SkyPy` `~skypy.pipeline.Pipeline`:
 
-  .. code:: yaml
+.. literalinclude:: luminosity.yml
+   :language: yaml
 
-    cosmology: !astropy.cosmology.default_cosmology.get
-    z_range: !numpy.linspace [0, 2, 21]
-    M_star: !astropy.modeling.models.Linear1D [-0.9, -20.4]
-    phi_star: !astropy.modeling.models.Exponential1D [3e-3, -9.7]
-    magnitude_limit: 23
-    sky_area: 10 deg2
-    tables:
-      blue_galaxies:
-        redshift, magnitude: !skypy.galaxies.schechter_lf
-          redshift: $z_range
-          M_star: $M_star
-          phi_star: $phi_star
-          alpha: -1.3
-          m_lim: $magnitude_limit
-          sky_area: $sky_area
+You may now save it as ``luminosity.yml`` and run it using the `SkyPy` `~skypy.pipeline.Pipeline`:
+
+.. plot::
+   :include-source: true
+   :context: close-figs
+
+    import matplotlib.pyplot as plt
+    from skypy.pipeline import Pipeline
+
+    # Execute SkyPy luminosity pipeline
+    pipeline = Pipeline.read("luminosity.yml")
+    pipeline.execute()
+
+    # Blue population
+    skypy_galaxies = pipeline['blue_galaxies']
+
+    # Plot histograms
+    fig, axs = plt.subplots(1, 2, figsize=(9, 3))
+
+    axs[0].hist(skypy_galaxies['redshift'], bins=50, histtype='step', color='purple')
+    axs[0].set_xlabel(r'$Redshift$')
+    axs[0].set_ylabel(r'$\mathrm{N}$')
+    axs[0].set_yscale('log')
+
+    axs[1].hist(skypy_galaxies['magnitude'], bins=50, histtype='step', color='green')
+    axs[1].set_xlabel(r'$Magnitude$')
+    axs[1].set_yscale('log')
+
+    plt.show()
 
 Don’t forget to check out for more complete examples_!
 
diff --git a/docs/luminosity.yml b/docs/luminosity.yml
new file mode 100644
index 000000000..1395bd68d
--- /dev/null
+++ b/docs/luminosity.yml
@@ -0,0 +1,15 @@
+cosmology: !astropy.cosmology.default_cosmology.get
+z_range: !numpy.linspace [0, 2, 21]
+M_star: !astropy.modeling.models.Linear1D [-0.9, -20.4]
+phi_star: !astropy.modeling.models.Exponential1D [3e-3, -9.7]
+magnitude_limit: 23
+sky_area: 10 deg2
+tables:
+  blue_galaxies:
+    redshift, magnitude: !skypy.galaxies.schechter_lf
+            redshift: $z_range
+            M_star: $M_star
+            phi_star: $phi_star
+            alpha: -1.3
+            m_lim: $magnitude_limit
+            sky_area: $sky_area

From e71cfde91c55480954f07e9cc6f257056002fb0c Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 22 Apr 2021 11:51:25 +0100
Subject: [PATCH 27/77] Commen on running pipeline from command line

---
 docs/configuration_files.rst | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index bf6f4293b..6082a8e17 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -91,6 +91,14 @@ You may now save it as ``luminosity.yml`` and run it using the `SkyPy` `~skypy.p
 
     plt.show()
 
+You can also run the pipeline directly from the command line:
+
+.. code-block:: bash
+
+    $ skypy luminosity.yml --format fits
+
+
+
 Don’t forget to check out for more complete examples_!
 
 .. _examples: https://skypy.readthedocs.io/en/stable/examples/index.html

From f2f9c39b31ba951f006c4432dd97e5626507b63f Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 22 Apr 2021 12:14:20 +0100
Subject: [PATCH 28/77] Correct variable type

---
 docs/configuration_files.rst | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 6082a8e17..386605a40 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -119,14 +119,25 @@ Main keywords: ``parameters``, ``cosmology``, ``tables``.
 
 Variables
 ^^^^^^^^^
-* `Define a variable`: a variable is defined as a key-value pair at the top of the file. You can define any type of variable:
+* `Define a variable`: a variable is defined as a key-value pair at the top of the file.
+  YAML is able to interpret any numeric data with the appropriate type: integer, float, boolean.
+  Similarly for lists and dictionaries.
+  In addition, SkyPy has added extra functionality to interpret and store Astropy Quantities_.
+  Everything else is stored as a string (whit or without explicitly using quotes)
 
   .. code:: yaml
 
+      # YAML interprets
       counter: 100 # An integer
       miles: 1000.0 # A floating point
       name: "Joy" # A string
+      planet: Earth # Another string
       mylist: [ 'abc', 789, 2.0e3 ] # A list
+      mydict: { 'fruit': 'orange', 'year': 2020 } # A dictionary
+
+      # SkyPy extra functionality
+      angle: 10 deg
+      distance: 300 kpc
 
 
 * `Reference a variable`: variables can be referenced by their full name tagged with a dollar sign ``$``.
@@ -333,3 +344,4 @@ Cosmology, a special parameter
 
 .. _YAML: https://yaml.org
 .. _NumPy: https://numpy.org
+.. _Quantities: https://docs.astropy.org/en/stable/units/

From 70584789d7e7ad3504f1e3b4e2aba65f942323c4 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 22 Apr 2021 13:19:27 +0100
Subject: [PATCH 29/77] Move variable referencing to the functions section

---
 docs/configuration_files.rst | 18 ++----------------
 1 file changed, 2 insertions(+), 16 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 386605a40..a8972e047 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -140,21 +140,6 @@ Variables
       distance: 300 kpc
 
 
-* `Reference a variable`: variables can be referenced by their full name tagged with a dollar sign ``$``.
-  For example, if you previously defined a list of galaxy properties:
-
-  .. code:: yaml
-
-      galaxy_properties: [ 'OBJECT_ID', 'RA', 'DEC', 'REDSHIFT', 'FLUX', 'FLUX_ERR' ]
-
-  You could reference the variable:
-
-  .. code:: yaml
-
-      catalog: $galaxy_properties
-
-
-
 Functions
 ^^^^^^^^^
 * `Call a function`: functions are defined as tuples where the first entry is the fully qualified function name tagged with and exclamation mark ``!`` and the second entry is either a list of positional arguments or a dictionary of keyword arguments.
@@ -175,7 +160,8 @@ Functions
       comoving_distance: !astropy.cosmology.Planck15.comoving_distance
         z: !numpy.linspace [ 0, 1.3, num=10 ]
 
-  or you could also define the variables at the top level and then reference them
+* `Reference a variable`: variables can be referenced by their full name tagged with a dollar sign ``$``.
+  In the previous example you could also define the variables at the top-level of the file and then reference them:
 
   .. code:: yaml
 

From 41543b889d43aced4b2ec2546852b8550d8cfb50 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 22 Apr 2021 13:29:43 +0100
Subject: [PATCH 30/77] Fix functions example

---
 docs/configuration_files.rst | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index a8972e047..70c7e0a97 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -158,14 +158,22 @@ Functions
   .. code:: yaml
 
       comoving_distance: !astropy.cosmology.Planck15.comoving_distance
-        z: !numpy.linspace [ 0, 1.3, num=10 ]
+        z: !numpy.linspace [ 0, 1.3, 10 ]
+
+  Similarly, you can specify the functions arguments as a dictionary:
+
+  .. code:: yaml
+
+      comoving_distance: !astropy.cosmology.Planck15.comoving_distance
+        z: !numpy.linspace {start: 0, stop: 1.3, num: 10}
+
 
 * `Reference a variable`: variables can be referenced by their full name tagged with a dollar sign ``$``.
   In the previous example you could also define the variables at the top-level of the file and then reference them:
 
   .. code:: yaml
 
-      redshift: !numpy.linspace [ 0, 1.3, num=10 ]
+      redshift: !numpy.linspace [ 0, 1.3, 10 ]
       comoving_distance: !astropy.cosmology.Planck15.comoving_distance
         z: $redshift
 

From 4b92db70613ff2723acac2da4bdcc73f8bb12a83 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 22 Apr 2021 16:17:42 +0100
Subject: [PATCH 31/77] Delete duplicated line

---
 docs/configuration_files.rst | 1 -
 1 file changed, 1 deletion(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 70c7e0a97..eb419c60a 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -56,7 +56,6 @@ You can create a table ``blue_galaxies`` and for now add the columns for redshif
 `Important:` if cosmology is detected as a parameter but is not set, it automatically uses the cosmology variable defined at the top-level of the file.
 
 This is how the entire configuration file looks like!
-You may now save it as ``luminosity.yml`` and run it using the `SkyPy` `~skypy.pipeline.Pipeline`:
 
 .. literalinclude:: luminosity.yml
    :language: yaml

From 138de054d0288473b91af75e1204edbc23ab56ee Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 22 Apr 2021 16:19:36 +0100
Subject: [PATCH 32/77] Fix cosmic voids example

---
 docs/configuration_files.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index eb419c60a..6b94e7b50 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -222,7 +222,7 @@ Tables
           s: 200.
           size: 10000
         mean: !numpy.mean
-          a: $radii
+          a: $cosmic_voids.radii
         variance: !numpy.var
           a: $radii
 

From 756e4c88fcd2f9e3bef79a740b7fac6fcc5a09c5 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 22 Apr 2021 16:20:26 +0100
Subject: [PATCH 33/77] Fix cosmic voids example variance

---
 docs/configuration_files.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 6b94e7b50..0baaf3079 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -224,7 +224,7 @@ Tables
         mean: !numpy.mean
           a: $cosmic_voids.radii
         variance: !numpy.var
-          a: $radii
+          a: $cosmic_voids.radii
 
 
 * `Multi-column assignment`: if a function returns multiple columns, you can chose to assign them to multiple columns with different names or to a muti-column object.

From 5608612e24f8d926006f683f4663bbb85bb848ee Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 22 Apr 2021 16:21:42 +0100
Subject: [PATCH 34/77] Fix circular velocity example

---
 docs/configuration_files.rst | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 0baaf3079..cdd1de8fa 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -240,7 +240,7 @@ Tables
           s: 250
           size: 1000
         hist, bin_edges: !numpy.histogram
-          a: $circular_velocities
+          a: $halos.circular_velocities
           density: True
 
   or a multi-column assignment
@@ -253,7 +253,7 @@ Tables
           s: 250
           size: 1000
         histogram: !numpy.histogram
-          a: $circular_velocities
+          a: $halos.circular_velocities
           density: True
 
 

From 8c728534b3fa6c7d93a55581b6c3b697e37825c2 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 22 Apr 2021 16:23:31 +0100
Subject: [PATCH 35/77] Fix typo

---
 docs/configuration_files.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index cdd1de8fa..523faedd7 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -122,7 +122,7 @@ Variables
   YAML is able to interpret any numeric data with the appropriate type: integer, float, boolean.
   Similarly for lists and dictionaries.
   In addition, SkyPy has added extra functionality to interpret and store Astropy Quantities_.
-  Everything else is stored as a string (whit or without explicitly using quotes)
+  Everything else is stored as a string (with or without explicitly using quotes)
 
   .. code:: yaml
 

From f42f22652c46c6afefd429f58040fdaaf4038fd4 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 27 Apr 2021 11:24:28 +0100
Subject: [PATCH 36/77] Restructure text around table referencing

---
 docs/configuration_files.rst | 48 ++++++++++++++++++++----------------
 1 file changed, 27 insertions(+), 21 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 523faedd7..ae41d2aa2 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -259,13 +259,23 @@ Tables
 
 * `Referencing tables: table.init and table.complete dependencies`.
 
-  Most of the time you would be referencing simple
-  variables. However there are times when your function depends on tables. In these
+  There are times when your function depends on tables. In these
   cases, you need to ensure the referenced table has the necessary content and is not empty.
 
-  Imagine you want to perform a very simple abundance matching, i.e. painting galaxies within your halos.
-  For this you can create two tables ``halos`` and ``galaxies`` storing the halo mass and galaxy luminosities.
-  The idea is to stack these two tables and store it in a third table called ``matching``. For example:
+  Example: you want to perform a very simple abundance matching, i.e. painting galaxies within your halos.
+  You can create two tables ``halos`` and ``galaxies`` storing the halo mass and galaxy luminosities.
+  Then you can stack these two tables and store it in a third table called ``matching``.
+
+  `Beware`: referencing tables is a bit different to variables or columns
+  -- where you simply tag a dollar sign.
+
+  The challenge: let ``tableA`` be a table with columns ``c1`` and ``c2``.
+  In configuration language, ``tableA`` is the name of the job.
+  *That means, when executing the configuration file, the first thing that happens is call ``tableA``, second,  call ``tableA.c1`` and third, call ``tableA.c2``.*
+  If you used the dollar sign to reference your ``tableA`` inside a function, this function might be called before the job ``tableA`` is complete, and the table will be empty.
+
+  `The solution`: to correctly reference tables, initialise your table with ``init``, specify their dependences with the keyword ``depends``
+  and ensure the tables are completed before calling the function with ``.complete``. Our example:
 
   .. code:: yaml
 
@@ -277,20 +287,13 @@ Tables
       galaxies:
         luminosity: !mylibrary.my_schechter_function
           alpha: 1.20
-      matching_wrong:
-        match: !numpy.vstack
-          tup: [ $halos, $galaxies ]
+      matching:
+        init: !numpy.vstack
+          tuple: [ $halos, $galaxies ]
+          depends: [ tuple.complete ]
 
-  This would probably not do what you intend.
-  For example, if you have a table called ``tableA`` with columns ``c1`` and ``c2``.
-  In configuration language, ``tableA`` is the name of the job.
-  That means, when executing the config file, the first thing that happens is call ``tableA``, second,  call ``tableA.c1`` and third, call ``tableA.c2``.
-  In our example, when you call the function ``numpy.vstack()`` and reference the tables ``$halos`` and ``$galaxies``, this is actually
-  referencing the job that initialises the empty table ``halos`` and ``galaxies``.
-  The potential risk is that the function could be called before the jobs ``halos`` and ``galaxies`` are finished, so the tables would be empty
 
-  To overcome this issue you can initialise your ``matching`` table with ``init``, specify their dependences with the keyword ``depends``
-  and ensure the tables are completed before calling the function with ``.complete``. The previous configuration file reads now:
+  `A non-working example`: this is an example of an incorrect table referencing.
 
   .. code:: yaml
 
@@ -302,10 +305,13 @@ Tables
       galaxies:
         luminosity: !mylibrary.my_schechter_function
           alpha: 1.20
-      matching:
-        init: !numpy.vstack
-          tup: [ $halos, $galaxies ]
-          depends: [ tup.complete ]
+      matching_wrong:
+        match: !numpy.vstack
+          tuple: [ $halos, $galaxies ]
+
+  Explanation: when calling the function ``numpy.vstack()`` and referencing the tables ``$halos`` and ``$galaxies``, this is actually
+  referencing the job that initialises the empty table ``halos`` and ``galaxies``.
+  The ``numpy.vstack()`` function is called before the jobs ``halos`` and ``galaxies`` are finished, so the tables are empty.
 
 
 

From 12498f113de8dba9f60a028e3626de362a91afce Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 27 Apr 2021 11:32:52 +0100
Subject: [PATCH 37/77] Set up other cosmologues section

---
 docs/configuration_files.rst | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index ae41d2aa2..d9f690a43 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -176,6 +176,8 @@ Functions
       comoving_distance: !astropy.cosmology.Planck15.comoving_distance
         z: $redshift
 
+  Please, check below in the `cosmology`_ section how to use different cosmologies with the SkyPy pipeline.
+
 
 Tables
 ^^^^^^
@@ -315,8 +317,8 @@ Tables
 
 
 
-Cosmology, a special parameter
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Cosmology
+^^^^^^^^^
 
 * `Define parameters`: parameters are variables that can be modified at execution.
 
@@ -339,6 +341,8 @@ Cosmology, a special parameter
       H0: $hubble_constant
       Om0: $omega_matter
 
+  * How to use `different cosmologies`
+
 
 
 .. _YAML: https://yaml.org

From c6ccf18a05bd5d2890351c0c36cc71d5d4b4e4e1 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 27 Apr 2021 11:50:16 +0100
Subject: [PATCH 38/77] Use numpy functions in abundance matching example

---
 docs/configuration_files.rst | 30 ++++++++++++++++++------------
 1 file changed, 18 insertions(+), 12 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index d9f690a43..d693ac06b 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -283,12 +283,15 @@ Tables
 
     tables:
       halos:
-        halo_mass: !mylibrary.my_halo_mass_function
-          m_min: 1.0e8
-          m_max: 1.0e14
+        halo_mass: !numpy.random.uniform
+          low: 1.0e8
+          high: 1.0e14
+          size: 20
       galaxies:
-        luminosity: !mylibrary.my_schechter_function
-          alpha: 1.20
+        luminosity: !numpy.random.uniform
+          low: 0.05
+          high: 10.0
+          size: 20
       matching:
         init: !numpy.vstack
           tuple: [ $halos, $galaxies ]
@@ -300,13 +303,16 @@ Tables
   .. code:: yaml
 
     tables:
-      halos:
-        halo_mass: !mylibrary.my_halo_mass_function
-          m_min: 1.0e8
-          m_max: 1.0e14
-      galaxies:
-        luminosity: !mylibrary.my_schechter_function
-          alpha: 1.20
+    halos:
+      halo_mass: !numpy.random.uniform
+        low: 1.0e8
+        high: 1.0e14
+        size: 20
+    galaxies:
+      luminosity: !numpy.random.uniform
+        low: 0.05
+        high: 10.0
+        size: 20
       matching_wrong:
         match: !numpy.vstack
           tuple: [ $halos, $galaxies ]

From 5bc5a5693dc3ef9ab36d458a61779dc4d1a496e3 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 27 Apr 2021 12:07:06 +0100
Subject: [PATCH 39/77] Add clone example in cosmology section

---
 docs/configuration_files.rst | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index d693ac06b..7976df2f4 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -347,10 +347,29 @@ Cosmology
       H0: $hubble_constant
       Om0: $omega_matter
 
-  * How to use `different cosmologies`
+* How to use `different cosmologies`: currently the cosmology parameter only accepts
+  functions, e.g.: ``astropy.cosmology.FlatLambdaCDM``. In order to use other cosmologies,
+  such as ``astropy.cosmology.Planck13`` or alike, you will need to use the Astropy `clone()`_ functionality.
+  This function returns a copy of the cosmology object, potentially with some changes:
 
+  .. code:: yaml
+
+    parameters:
+      hubble_constant: 70
+      omega_matter: 0.3
+    cosmology: !astropy.cosmology.Planck13.clone
+      name: 'Modified Planck 13'
+      H0: $hubble_constant
+      Om0: $omega_matter
+
+  `A non-working example`:
+
+  .. code:: yaml
+
+    cosmology: !astropy.cosmology.Planck13
 
 
 .. _YAML: https://yaml.org
 .. _NumPy: https://numpy.org
 .. _Quantities: https://docs.astropy.org/en/stable/units/
+.. _clone(): https://docs.astropy.org/en/stable/api/astropy.cosmology.FLRW.html?highlight=clone#astropy.cosmology.FLRW.clone

From d8b00651c34e63f3138014ce25e0d541a7da394d Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 27 Apr 2021 16:05:52 +0100
Subject: [PATCH 40/77] Update config style with new import functionality

---
 docs/configuration_files.rst | 40 +++++++++++++++++-------------------
 1 file changed, 19 insertions(+), 21 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 7976df2f4..8b3c4bcd1 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -166,6 +166,13 @@ Functions
       comoving_distance: !astropy.cosmology.Planck15.comoving_distance
         z: !numpy.linspace {start: 0, stop: 1.3, num: 10}
 
+  `N.B.` To call a function with no arguments, you should pass an empty list of
+  ``args`` or an empty dictionary of ``kwargs``. For example:
+
+  .. code:: yaml
+
+      cosmo: !astropy.cosmology.default_cosmology.get []
+
 
 * `Reference a variable`: variables can be referenced by their full name tagged with a dollar sign ``$``.
   In the previous example you could also define the variables at the top-level of the file and then reference them:
@@ -215,7 +222,6 @@ Tables
   with a column ``radii`` where you sample 10000 void sizes and two other columns, ``mean`` and ``variance`` that reference
   the first column
 
-
   .. code:: yaml
 
     tables:
@@ -322,6 +328,18 @@ Tables
   The ``numpy.vstack()`` function is called before the jobs ``halos`` and ``galaxies`` are finished, so the tables are empty.
 
 
+Import objects
+^^^^^^^^^^^^^^
+* The SkyPy configuration syntax allows objects to be imported directly from external
+  (sub)modules using the ``!`` tag and providing neither a list of arguments or a
+  dictionary of keywords. For example, this enables the import and usage of any Astropy cosmology:
+
+  .. code:: yaml
+
+    cosmo1: !astropy.cosmology.Planck13 # import the Planck18 object and bind to cosmo1
+    cosmo2: !astropy.cosmology.default_cosmology.get [] # call default_cosmology.get() and bind to cosmo2
+
+
 
 Cosmology
 ^^^^^^^^^
@@ -347,26 +365,6 @@ Cosmology
       H0: $hubble_constant
       Om0: $omega_matter
 
-* How to use `different cosmologies`: currently the cosmology parameter only accepts
-  functions, e.g.: ``astropy.cosmology.FlatLambdaCDM``. In order to use other cosmologies,
-  such as ``astropy.cosmology.Planck13`` or alike, you will need to use the Astropy `clone()`_ functionality.
-  This function returns a copy of the cosmology object, potentially with some changes:
-
-  .. code:: yaml
-
-    parameters:
-      hubble_constant: 70
-      omega_matter: 0.3
-    cosmology: !astropy.cosmology.Planck13.clone
-      name: 'Modified Planck 13'
-      H0: $hubble_constant
-      Om0: $omega_matter
-
-  `A non-working example`:
-
-  .. code:: yaml
-
-    cosmology: !astropy.cosmology.Planck13
 
 
 .. _YAML: https://yaml.org

From eeec85168e7ea338e689a7e171313598417ed77c Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Wed, 12 May 2021 11:25:23 +0100
Subject: [PATCH 41/77] Fix typ for Planck13 cosmology

---
 docs/configuration_files.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 8b3c4bcd1..bab2d4d8e 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -336,7 +336,7 @@ Import objects
 
   .. code:: yaml
 
-    cosmo1: !astropy.cosmology.Planck13 # import the Planck18 object and bind to cosmo1
+    cosmo1: !astropy.cosmology.Planck13 # import the Planck13 object and bind to cosmo1
     cosmo2: !astropy.cosmology.default_cosmology.get [] # call default_cosmology.get() and bind to cosmo2
 
 

From d7d4f0eaa74cb4c2db25d6747b1431075bb928c2 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Wed, 12 May 2021 11:28:32 +0100
Subject: [PATCH 42/77] Delete how-not-to-do example

---
 docs/configuration_files.rst | 24 ------------------------
 1 file changed, 24 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index bab2d4d8e..9a54c6ecf 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -304,30 +304,6 @@ Tables
           depends: [ tuple.complete ]
 
 
-  `A non-working example`: this is an example of an incorrect table referencing.
-
-  .. code:: yaml
-
-    tables:
-    halos:
-      halo_mass: !numpy.random.uniform
-        low: 1.0e8
-        high: 1.0e14
-        size: 20
-    galaxies:
-      luminosity: !numpy.random.uniform
-        low: 0.05
-        high: 10.0
-        size: 20
-      matching_wrong:
-        match: !numpy.vstack
-          tuple: [ $halos, $galaxies ]
-
-  Explanation: when calling the function ``numpy.vstack()`` and referencing the tables ``$halos`` and ``$galaxies``, this is actually
-  referencing the job that initialises the empty table ``halos`` and ``galaxies``.
-  The ``numpy.vstack()`` function is called before the jobs ``halos`` and ``galaxies`` are finished, so the tables are empty.
-
-
 Import objects
 ^^^^^^^^^^^^^^
 * The SkyPy configuration syntax allows objects to be imported directly from external

From 94c71606b26c78111718449eb8b66313cb0ca65c Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Wed, 12 May 2021 11:30:21 +0100
Subject: [PATCH 43/77] Fix plot rendering

---
 docs/configuration_files.rst | 1 +
 1 file changed, 1 insertion(+)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 9a54c6ecf..9cfc9bc37 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -88,6 +88,7 @@ You may now save it as ``luminosity.yml`` and run it using the `SkyPy` `~skypy.p
     axs[1].set_xlabel(r'$Magnitude$')
     axs[1].set_yscale('log')
 
+    plt.tight_layout()
     plt.show()
 
 You can also run the pipeline directly from the command line:

From c3f9c2731d29e53b9df39b95b8b78650feb55d34 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 13 May 2021 15:19:45 +0100
Subject: [PATCH 44/77] Fix typo argument cosmology

---
 docs/luminosity.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/luminosity.yml b/docs/luminosity.yml
index 1395bd68d..1b679327b 100644
--- a/docs/luminosity.yml
+++ b/docs/luminosity.yml
@@ -1,4 +1,4 @@
-cosmology: !astropy.cosmology.default_cosmology.get
+cosmology: !astropy.cosmology.default_cosmology.get []
 z_range: !numpy.linspace [0, 2, 21]
 M_star: !astropy.modeling.models.Linear1D [-0.9, -20.4]
 phi_star: !astropy.modeling.models.Exponential1D [3e-3, -9.7]

From 75dc98ec106cfbf121f12bd1751e263aabaa956a Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 13 May 2021 15:25:32 +0100
Subject: [PATCH 45/77] Fix typo init and depends

---
 docs/configuration_files.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 9cfc9bc37..bc7e47497 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -283,7 +283,7 @@ Tables
   *That means, when executing the configuration file, the first thing that happens is call ``tableA``, second,  call ``tableA.c1`` and third, call ``tableA.c2``.*
   If you used the dollar sign to reference your ``tableA`` inside a function, this function might be called before the job ``tableA`` is complete, and the table will be empty.
 
-  `The solution`: to correctly reference tables, initialise your table with ``init``, specify their dependences with the keyword ``depends``
+  `The solution`: to correctly reference tables, initialise your table with ``.init``, specify their dependences with the keyword ``.depends``
   and ensure the tables are completed before calling the function with ``.complete``. Our example:
 
   .. code:: yaml

From b71aa56450d292c48b50e43dcac160a817a1e334 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 13 May 2021 15:27:28 +0100
Subject: [PATCH 46/77] Fix reference table example

---
 docs/configuration_files.rst | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index bc7e47497..32270507d 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -300,9 +300,9 @@ Tables
           high: 10.0
           size: 20
       matching:
-        init: !numpy.vstack
-          tuple: [ $halos, $galaxies ]
-          depends: [ tuple.complete ]
+        .init: !astropy.table.vstack
+          tables: [ $halos, $galaxies ]
+          .depends: [ halos.complete, galaxies.complete ]
 
 
 Import objects

From 4d121eece55f00d15b05583f2ed9ed25935ad810 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Fri, 14 May 2021 17:07:22 +0100
Subject: [PATCH 47/77] Fix argument default cosmology

---
 docs/configuration_files.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 32270507d..41d5172d6 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -30,7 +30,7 @@ Also, ``noise`` is an optional parameter and you could use its default value by
 
   .. code:: yaml
 
-    cosmology: !astropy.cosmology.default_cosmology.get
+    cosmology: !astropy.cosmology.default_cosmology.get []
     z_range: !numpy.linspace [0, 2, 21]
     M_star: !astropy.modeling.models.Linear1D [-0.9, -20.4]
     phi_star: !astropy.modeling.models.Exponential1D [3e-3, -9.7]

From b26164260a0dec618dac705ca41fef7201fb1c16 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Fri, 14 May 2021 17:09:21 +0100
Subject: [PATCH 48/77] Fix header

---
 docs/configuration_files.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 41d5172d6..2089a971d 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -1,5 +1,5 @@
 ###################
-Configuration files
+Configuration Files
 ###################
 
 This page outlines how to construct configuration files to run your own routines

From b6d9dd2c51556014495183e13172e57f7b567bde Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Fri, 14 May 2021 17:10:12 +0100
Subject: [PATCH 49/77] Change to a smaller sky area

---
 docs/configuration_files.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 2089a971d..07d4b8b96 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -35,7 +35,7 @@ Also, ``noise`` is an optional parameter and you could use its default value by
     M_star: !astropy.modeling.models.Linear1D [-0.9, -20.4]
     phi_star: !astropy.modeling.models.Exponential1D [3e-3, -9.7]
     magnitude_limit: 23
-    sky_area: 10 deg2
+    sky_area: 0.1 deg2
 
 - `Tables and columns`:
 

From a0607acd5d2c146f8ae29209c6a0a7408466230a Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Fri, 14 May 2021 17:13:15 +0100
Subject: [PATCH 50/77] Change and explain the command line option

---
 docs/configuration_files.rst | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 07d4b8b96..c61a12c3c 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -91,11 +91,11 @@ You may now save it as ``luminosity.yml`` and run it using the `SkyPy` `~skypy.p
     plt.tight_layout()
     plt.show()
 
-You can also run the pipeline directly from the command line:
+You can also run the pipeline directly from the command line and write the outputs to a fits file:
 
 .. code-block:: bash
 
-    $ skypy luminosity.yml --format fits
+    $ skypy luminosity.yml luminosity.fits
 
 
 

From c9857e9c627c7464da4d6a125e120ce7e6ce1607 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Fri, 14 May 2021 17:18:32 +0100
Subject: [PATCH 51/77] Decrease sky area in the luminosity config file

---
 docs/luminosity.yml | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/docs/luminosity.yml b/docs/luminosity.yml
index 1b679327b..0ca5ed07f 100644
--- a/docs/luminosity.yml
+++ b/docs/luminosity.yml
@@ -3,13 +3,13 @@ z_range: !numpy.linspace [0, 2, 21]
 M_star: !astropy.modeling.models.Linear1D [-0.9, -20.4]
 phi_star: !astropy.modeling.models.Exponential1D [3e-3, -9.7]
 magnitude_limit: 23
-sky_area: 10 deg2
+sky_area: 0.1 deg2
 tables:
   blue_galaxies:
     redshift, magnitude: !skypy.galaxies.schechter_lf
-            redshift: $z_range
-            M_star: $M_star
-            phi_star: $phi_star
-            alpha: -1.3
-            m_lim: $magnitude_limit
-            sky_area: $sky_area
+      redshift: $z_range
+      M_star: $M_star
+      phi_star: $phi_star
+      alpha: -1.3
+      m_lim: $magnitude_limit
+      sky_area: $sky_area

From a75a57da75e9dc5fc306205f7f1268533561179b Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Fri, 14 May 2021 17:20:58 +0100
Subject: [PATCH 52/77] Fix telescope example

---
 docs/configuration_files.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index c61a12c3c..943005b74 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -198,7 +198,7 @@ Tables
 
       tables:
         telescope:
-          spectral_lines: !scipy.stats.norm
+          spectral_lines: !scipy.stats.norm.rvs
             loc: 550
             scale: 1.6
             size: 100

From e6749766a5edf22dd5c5002c4437155b95e2439e Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Fri, 14 May 2021 17:21:47 +0100
Subject: [PATCH 53/77] Fix telescope example 2

---
 docs/configuration_files.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 943005b74..6943f794d 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -210,7 +210,7 @@ Tables
 
       tables:
         telescope:
-          spectral_lines: !scipy.stats.norm
+          spectral_lines: !scipy.stats.norm.rvs
             loc: 550
             scale: 1.6
             size: 100

From 88bd0bc901184a3cabc8ebbba43d7bd7e419265d Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Fri, 14 May 2021 17:22:59 +0100
Subject: [PATCH 54/77] Fix telescope example to have coumns with same length

---
 docs/configuration_files.rst | 1 +
 1 file changed, 1 insertion(+)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 6943f794d..6206df567 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -217,6 +217,7 @@ Tables
           collecting_surface: !numpy.random.uniform
             low: 6.9
             high: 7.1
+            size: 100
 
 * `Reference a column`: columns in the pipeline can be referenced by their full name tagged with a dollar sign ``$``.
   Example: the radius of cosmic voids seem to follow a lognormal distribution. You can create a table ``cosmic_voids``

From abd98dea193bf3cc890b9fe58652a9764f3c279c Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Fri, 14 May 2021 17:26:01 +0100
Subject: [PATCH 55/77] Fix RuntimeWarning for the circular velocities examples

---
 docs/configuration_files.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 6206df567..d9df2c373 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -229,7 +229,7 @@ Tables
     tables:
       cosmic_voids:
         radii: !scipy.stats.lognorm.rvs
-          s: 200.
+          s: 5.
           size: 10000
         mean: !numpy.mean
           a: $cosmic_voids.radii

From 3459113acccea3acacb58c06bde69840a8471889 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Fri, 14 May 2021 17:32:06 +0100
Subject: [PATCH 56/77] Fix indentation problem in luminosity pipeline

---
 docs/configuration_files.rst | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index d9df2c373..a5e1ce6ad 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -46,12 +46,12 @@ You can create a table ``blue_galaxies`` and for now add the columns for redshif
     tables:
       blue_galaxies:
         redshift, magnitude: !skypy.galaxies.schechter_lf
-      		redshift: $z_range
-      		M_star: $M_star
-      		phi_star: $phi_star
-      		alpha: -1.3
-      		m_lim: $magnitude_limit
-      		sky_area: $sky_area
+          redshift: $z_range
+      	  M_star: $M_star
+      	  phi_star: $phi_star
+      	  alpha: -1.3
+      	  m_lim: $magnitude_limit
+      	  sky_area: $sky_area
 
 `Important:` if cosmology is detected as a parameter but is not set, it automatically uses the cosmology variable defined at the top-level of the file.
 

From 528ff21b9e1c99ab09284d073e3963351af0dcac Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Fri, 14 May 2021 17:41:50 +0100
Subject: [PATCH 57/77] Place import section as a second bullet point under
 variables

---
 docs/configuration_files.rst | 26 +++++++++++---------------
 1 file changed, 11 insertions(+), 15 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index a5e1ce6ad..f85d0f214 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -140,6 +140,17 @@ Variables
       distance: 300 kpc
 
 
+* `Import objects`:
+  the SkyPy configuration syntax allows objects to be imported directly from external
+  (sub)modules using the ``!`` tag and providing neither a list of arguments or a
+  dictionary of keywords. For example, this enables the import and usage of any Astropy cosmology:
+
+  .. code:: yaml
+
+      cosmo1: !astropy.cosmology.Planck13 # import the Planck13 object and bind to cosmo1
+      cosmo2: !astropy.cosmology.default_cosmology.get [] # call default_cosmology.get() and bind to cosmo2
+
+
 Functions
 ^^^^^^^^^
 * `Call a function`: functions are defined as tuples where the first entry is the fully qualified function name tagged with and exclamation mark ``!`` and the second entry is either a list of positional arguments or a dictionary of keyword arguments.
@@ -184,8 +195,6 @@ Functions
       comoving_distance: !astropy.cosmology.Planck15.comoving_distance
         z: $redshift
 
-  Please, check below in the `cosmology`_ section how to use different cosmologies with the SkyPy pipeline.
-
 
 Tables
 ^^^^^^
@@ -306,19 +315,6 @@ Tables
           .depends: [ halos.complete, galaxies.complete ]
 
 
-Import objects
-^^^^^^^^^^^^^^
-* The SkyPy configuration syntax allows objects to be imported directly from external
-  (sub)modules using the ``!`` tag and providing neither a list of arguments or a
-  dictionary of keywords. For example, this enables the import and usage of any Astropy cosmology:
-
-  .. code:: yaml
-
-    cosmo1: !astropy.cosmology.Planck13 # import the Planck13 object and bind to cosmo1
-    cosmo2: !astropy.cosmology.default_cosmology.get [] # call default_cosmology.get() and bind to cosmo2
-
-
-
 Cosmology
 ^^^^^^^^^
 

From 6f6f61b02cf1e852d3dc016c0d99515b85bd6b48 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Fri, 14 May 2021 17:47:29 +0100
Subject: [PATCH 58/77] Add placeholder for parameters section

---
 docs/configuration_files.rst | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index f85d0f214..62fda2770 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -150,6 +150,19 @@ Variables
       cosmo1: !astropy.cosmology.Planck13 # import the Planck13 object and bind to cosmo1
       cosmo2: !astropy.cosmology.default_cosmology.get [] # call default_cosmology.get() and bind to cosmo2
 
+Parameters
+^^^^^^^^^^
+
+* `Define parameters`: parameters are variables that can be modified at execution.
+
+  For example,
+
+  .. code:: yaml
+
+      parameters:
+        hubble_constant: 70
+        omega_matter: 0.3
+        
 
 Functions
 ^^^^^^^^^

From 1d2f201a25a3ff3ae200f6fd0384d9dac8bf0a6e Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Fri, 14 May 2021 17:50:14 +0100
Subject: [PATCH 59/77] Add cosmology bullet point in the functions section

---
 docs/configuration_files.rst | 35 ++++++++++-------------------------
 1 file changed, 10 insertions(+), 25 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 62fda2770..38377ca62 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -150,6 +150,7 @@ Variables
       cosmo1: !astropy.cosmology.Planck13 # import the Planck13 object and bind to cosmo1
       cosmo2: !astropy.cosmology.default_cosmology.get [] # call default_cosmology.get() and bind to cosmo2
 
+
 Parameters
 ^^^^^^^^^^
 
@@ -162,7 +163,7 @@ Parameters
       parameters:
         hubble_constant: 70
         omega_matter: 0.3
-        
+
 
 Functions
 ^^^^^^^^^
@@ -208,6 +209,14 @@ Functions
       comoving_distance: !astropy.cosmology.Planck15.comoving_distance
         z: $redshift
 
+* The `cosmology` to be used by functions within the pipeline only needs to be set up at the top. If a function needs ``cosmology`` as an input, you need not define it again, it is automatically detected.
+
+  .. code:: yaml
+
+      cosmology: !astropy.cosmology.FlatLambdaCDM
+        H0: 70
+        Om0: 0.3
+
 
 Tables
 ^^^^^^
@@ -328,30 +337,6 @@ Tables
           .depends: [ halos.complete, galaxies.complete ]
 
 
-Cosmology
-^^^^^^^^^
-
-* `Define parameters`: parameters are variables that can be modified at execution.
-
-  For example,
-
-  .. code:: yaml
-
-    parameters:
-      hubble_constant: 70
-      omega_matter: 0.3
-
-* The `cosmology` to be used by functions within the pipeline only needs to be set up at the top. If a function needs ``cosmology`` as an input, you need not define it again, it is automatically detected.
-
-  .. code:: yaml
-
-    parameters:
-      hubble_constant: 70
-      omega_matter: 0.3
-    cosmology: !astropy.cosmology.FlatLambdaCDM
-      H0: $hubble_constant
-      Om0: $omega_matter
-
 
 
 .. _YAML: https://yaml.org

From 4e163a3dc6adb9e8e590d8e66acee085a27413ec Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Fri, 14 May 2021 17:56:24 +0100
Subject: [PATCH 60/77] Add temporary example for the cosmology feature

---
 docs/configuration_files.rst | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 38377ca62..d888792e6 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -216,6 +216,8 @@ Functions
       cosmology: !astropy.cosmology.FlatLambdaCDM
         H0: 70
         Om0: 0.3
+      growth: !skypy.power-spectrum.growth_function
+        redshift: 0.2
 
 
 Tables

From 7d515d7810ddb6f481693e9c9f765053365108c0 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Fri, 14 May 2021 18:01:57 +0100
Subject: [PATCH 61/77] Delete blanck lines

---
 docs/configuration_files.rst | 2 --
 1 file changed, 2 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index d888792e6..58e7c2374 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -339,8 +339,6 @@ Tables
           .depends: [ halos.complete, galaxies.complete ]
 
 
-
-
 .. _YAML: https://yaml.org
 .. _NumPy: https://numpy.org
 .. _Quantities: https://docs.astropy.org/en/stable/units/

From 19d2ff80ab3587fe751d929bc1c90edaf9f5c9c4 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 20 May 2021 16:46:36 +0100
Subject: [PATCH 62/77] Change example or column reference

---
 docs/configuration_files.rst | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 58e7c2374..fd69a2bf3 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -253,21 +253,19 @@ Tables
             size: 100
 
 * `Reference a column`: columns in the pipeline can be referenced by their full name tagged with a dollar sign ``$``.
-  Example: the radius of cosmic voids seem to follow a lognormal distribution. You can create a table ``cosmic_voids``
-  with a column ``radii`` where you sample 10000 void sizes and two other columns, ``mean`` and ``variance`` that reference
-  the first column
+  Example: the galaxy mass that follows a lognormal distribution. You can create a table ``galaxies``
+  with a column ``mass`` where you sample 10000 object and a second column, ``radius`` which also follows a lognormal distribution
+  but the mean depends on how massive the galaxies are:
 
   .. code:: yaml
 
     tables:
-      cosmic_voids:
-        radii: !scipy.stats.lognorm.rvs
-          s: 5.
+      galaxies:
+        mass: !numpy.random.lognormal
+          mean: 5.
           size: 10000
-        mean: !numpy.mean
-          a: $cosmic_voids.radii
-        variance: !numpy.var
-          a: $cosmic_voids.radii
+        radius: !numpy.random.lognormal
+          mean: $galaxies.mass
 
 
 * `Multi-column assignment`: if a function returns multiple columns, you can chose to assign them to multiple columns with different names or to a muti-column object.

From 91550b10df555d4a4638bcf837455bc7ce1ec73e Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 20 May 2021 17:00:29 +0100
Subject: [PATCH 63/77] Edit multicolumn assignement and change examples

---
 docs/configuration_files.rst | 27 +++++----------------------
 1 file changed, 5 insertions(+), 22 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index fd69a2bf3..837995c3d 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -268,34 +268,17 @@ Tables
           mean: $galaxies.mass
 
 
-* `Multi-column assignment`: if a function returns multiple columns, you can chose to assign them to multiple columns with different names or to a muti-column object.
+* `Multi-column assignment`: multi-column assignment is performed with any 2d-array, where one of the dimensions is interpreted
+  as the rows of the table and the second dimension, as separate columns. Or you can do it from a function that returns a tuple.
 
-  Imagine you want the distribution of the circular velocities of 1000 halos following a Maxwellian distribution.
-  The histogram NumPy_ returns a 2-dimensional object. Therefore, you can choose
+  We use multi-column assignment in the following example where we sample different halo properties from a lognormal distribution (returns a tuple of three arrays):
 
   .. code:: yaml
 
     tables:
       halos:
-        circular_velocities: !scipy.stats.maxwell.rvs
-          s: 250
-          size: 1000
-        hist, bin_edges: !numpy.histogram
-          a: $halos.circular_velocities
-          density: True
-
-  or a multi-column assignment
-
-  .. code:: yaml
-
-    tables:
-      halos:
-        circular_velocities: !scipy.stats.maxwell.rvs
-          s: 250
-          size: 1000
-        histogram: !numpy.histogram
-          a: $halos.circular_velocities
-          density: True
+        mass, radius, concentration: !numpy.random.lognormal
+          size: [10000, 3]
 
 
 * `Referencing tables: table.init and table.complete dependencies`.

From e0404c6f0a31acb3572e10c587e95647d0a0db10 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 20 May 2021 17:17:31 +0100
Subject: [PATCH 64/77] Add .depends under functions and include example

---
 docs/configuration_files.rst | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 837995c3d..0df3cfe58 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -219,6 +219,20 @@ Functions
       growth: !skypy.power-spectrum.growth_function
         redshift: 0.2
 
+* `Job completion`: ``.depends`` can be used to force any function call to wait for completion
+  of any other job.
+
+  A simple example:
+
+  .. code:: yaml
+
+    redshift: !numpy.linspace [ 0, 1.3, 10 ]
+    comoving_distance: !astropy.cosmology.Planck15.comoving_distance
+      z: $redshift
+      .depends: redshift
+
+  By doing so, you force the function call ``redshift`` to be completed before is used to compute the comoving distance.
+
 
 Tables
 ^^^^^^

From 716414138d58ff84abe64e21bdd7dac7c003f63e Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 20 May 2021 17:37:24 +0100
Subject: [PATCH 65/77] Add table initialisation section and example

---
 docs/configuration_files.rst | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 0df3cfe58..d119f5f7a 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -295,7 +295,20 @@ Tables
           size: [10000, 3]
 
 
-* `Referencing tables: table.init and table.complete dependencies`.
+* `Table initialisation`: by default tables are initialised using ``astropy.table.Table()`` however this can be overridden using the `.init` keyword to initialise the table with any function call.
+
+  For example, you can stack galaxy properties such as radii and mass:
+
+  .. code:: yaml
+
+    radii: !numpy.logspace [ 1, 2, 100 ]
+    mass: !numpy.logspace [ 9, 12, 100 ]
+    tables:
+      .init: !astropy.table.vstack
+        galaxies: [ $radii, $mass ]
+
+
+* `Reference tables and table.complete`:
 
   There are times when your function depends on tables. In these
   cases, you need to ensure the referenced table has the necessary content and is not empty.

From 699f35db38043f938cef6a05951d2c3ebcff714b Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 20 May 2021 17:43:53 +0100
Subject: [PATCH 66/77] Edit table reference setion

---
 docs/configuration_files.rst | 19 +++----------------
 1 file changed, 3 insertions(+), 16 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index d119f5f7a..547ddfb8a 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -295,7 +295,7 @@ Tables
           size: [10000, 3]
 
 
-* `Table initialisation`: by default tables are initialised using ``astropy.table.Table()`` however this can be overridden using the `.init` keyword to initialise the table with any function call.
+* `Table initialisation`: by default tables are initialised using ``astropy.table.Table()`` however this can be overridden using the ``.init`` keyword to initialise the table with any function call.
 
   For example, you can stack galaxy properties such as radii and mass:
 
@@ -308,26 +308,13 @@ Tables
         galaxies: [ $radii, $mass ]
 
 
-* `Reference tables and table.complete`:
-
-  There are times when your function depends on tables. In these
-  cases, you need to ensure the referenced table has the necessary content and is not empty.
+* `Reference tables`: when a function call depends on tables, you need to ensure the referenced table has the necessary content and is not empty.
+  You can do that with ``.complete``.
 
   Example: you want to perform a very simple abundance matching, i.e. painting galaxies within your halos.
   You can create two tables ``halos`` and ``galaxies`` storing the halo mass and galaxy luminosities.
   Then you can stack these two tables and store it in a third table called ``matching``.
 
-  `Beware`: referencing tables is a bit different to variables or columns
-  -- where you simply tag a dollar sign.
-
-  The challenge: let ``tableA`` be a table with columns ``c1`` and ``c2``.
-  In configuration language, ``tableA`` is the name of the job.
-  *That means, when executing the configuration file, the first thing that happens is call ``tableA``, second,  call ``tableA.c1`` and third, call ``tableA.c2``.*
-  If you used the dollar sign to reference your ``tableA`` inside a function, this function might be called before the job ``tableA`` is complete, and the table will be empty.
-
-  `The solution`: to correctly reference tables, initialise your table with ``.init``, specify their dependences with the keyword ``.depends``
-  and ensure the tables are completed before calling the function with ``.complete``. Our example:
-
   .. code:: yaml
 
     tables:

From 58df263e05cf47ed5940b116694ea7aa8f8cd6bf Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 20 May 2021 17:48:28 +0100
Subject: [PATCH 67/77] Homogenise subsection headings

---
 docs/configuration_files.rst | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 547ddfb8a..8d3f512a7 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -119,7 +119,7 @@ Main keywords: ``parameters``, ``cosmology``, ``tables``.
 
 Variables
 ^^^^^^^^^
-* `Define a variable`: a variable is defined as a key-value pair at the top of the file.
+* `Variable definition`: a variable is defined as a key-value pair at the top of the file.
   YAML is able to interpret any numeric data with the appropriate type: integer, float, boolean.
   Similarly for lists and dictionaries.
   In addition, SkyPy has added extra functionality to interpret and store Astropy Quantities_.
@@ -154,7 +154,7 @@ Variables
 Parameters
 ^^^^^^^^^^
 
-* `Define parameters`: parameters are variables that can be modified at execution.
+* `Parameters definition`: parameters are variables that can be modified at execution.
 
   For example,
 
@@ -167,7 +167,7 @@ Parameters
 
 Functions
 ^^^^^^^^^
-* `Call a function`: functions are defined as tuples where the first entry is the fully qualified function name tagged with and exclamation mark ``!`` and the second entry is either a list of positional arguments or a dictionary of keyword arguments.
+* `Function call`: functions are defined as tuples where the first entry is the fully qualified function name tagged with and exclamation mark ``!`` and the second entry is either a list of positional arguments or a dictionary of keyword arguments.
 
   For example, if you need to call the ``log10()`` and ``linspace()`` NumPy_ functions, then you define the following key-value pairs:
 
@@ -200,7 +200,7 @@ Functions
       cosmo: !astropy.cosmology.default_cosmology.get []
 
 
-* `Reference a variable`: variables can be referenced by their full name tagged with a dollar sign ``$``.
+* `Variable reference`: variables can be referenced by their full name tagged with a dollar sign ``$``.
   In the previous example you could also define the variables at the top-level of the file and then reference them:
 
   .. code:: yaml
@@ -237,7 +237,7 @@ Functions
 Tables
 ^^^^^^
 
-* `Create a table`: a dictionary of table names, each resolving to a dictionary of column names for that table.
+* `Table creation`: a dictionary of table names, each resolving to a dictionary of column names for that table.
 
   Let us create a table called ``telescope`` with a column to store the width of spectral lines that follow a normal distribution
 
@@ -250,7 +250,7 @@ Tables
             scale: 1.6
             size: 100
 
-* `Add a column`: you can add as many columns to a table as you need.
+* `Column addition`: you can add as many columns to a table as you need.
     Imagine you want to add a column for the telescope collecting surface
 
   .. code:: yaml
@@ -266,7 +266,7 @@ Tables
             high: 7.1
             size: 100
 
-* `Reference a column`: columns in the pipeline can be referenced by their full name tagged with a dollar sign ``$``.
+* `Column reference`: columns in the pipeline can be referenced by their full name tagged with a dollar sign ``$``.
   Example: the galaxy mass that follows a lognormal distribution. You can create a table ``galaxies``
   with a column ``mass`` where you sample 10000 object and a second column, ``radius`` which also follows a lognormal distribution
   but the mean depends on how massive the galaxies are:
@@ -308,7 +308,7 @@ Tables
         galaxies: [ $radii, $mass ]
 
 
-* `Reference tables`: when a function call depends on tables, you need to ensure the referenced table has the necessary content and is not empty.
+* `Table reference`: when a function call depends on tables, you need to ensure the referenced table has the necessary content and is not empty.
   You can do that with ``.complete``.
 
   Example: you want to perform a very simple abundance matching, i.e. painting galaxies within your halos.

From 91aca08286f96d2059a80e845ff9db2f640f64cc Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Thu, 20 May 2021 17:52:37 +0100
Subject: [PATCH 68/77] Fix typo

---
 docs/configuration_files.rst | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 8d3f512a7..3bd821651 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -6,8 +6,8 @@ This page outlines how to construct configuration files to run your own routines
 with `~skypy.pipeline.Pipeline`.
 
 `SkyPy` is an astrophysical simulation pipeline tool that allows to define any
-arbitrary workflow and store data in table format. You may use `SkyPy` `~skypy.pipeline.Pipeline`.
-to call any function --your own, from any compatible external software or from the `SkyPy library`.
+arbitrary workflow and store data in table format. You may use `SkyPy` `~skypy.pipeline.Pipeline`
+to call any function --your own implementation, from any compatible external software or from the `SkyPy library`.
 Then `SkyPy` deals with the data dependencies and provides a library of functions to be used with it.
 
 These guidelines start with an example using one of the `SkyPy` functions, and it follows

From 319678e450844764a3cd273e9dc391c050891fbb Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 8 Jun 2021 13:34:36 +0100
Subject: [PATCH 69/77] Fix .depends example

---
 docs/configuration_files.rst | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 3bd821651..ba4b2dbaa 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -222,14 +222,16 @@ Functions
 * `Job completion`: ``.depends`` can be used to force any function call to wait for completion
   of any other job.
 
-  A simple example:
+  A simple example where, for some reason, the comoving distance needs to be called after
+  completion of the growth function:
 
   .. code:: yaml
 
-    redshift: !numpy.linspace [ 0, 1.3, 10 ]
+    growth: !skypy.power-spectrum.growth_function
+      redshift: 0.2
     comoving_distance: !astropy.cosmology.Planck15.comoving_distance
-      z: $redshift
-      .depends: redshift
+      z: !numpy.linspace [ 0, 1.3, 10 ]
+      .depends: growth
 
   By doing so, you force the function call ``redshift`` to be completed before is used to compute the comoving distance.
 

From 11d4a4fc7a21cebe1080b0732078a9e5cea7f5c9 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 8 Jun 2021 13:43:05 +0100
Subject: [PATCH 70/77] Fix .init example

---
 docs/configuration_files.rst | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index ba4b2dbaa..69e10def0 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -306,8 +306,8 @@ Tables
     radii: !numpy.logspace [ 1, 2, 100 ]
     mass: !numpy.logspace [ 9, 12, 100 ]
     tables:
-      .init: !astropy.table.vstack
-        galaxies: [ $radii, $mass ]
+      galaxies:
+        .init: !astropy.table.vstack [[ $radii, $mass ]]
 
 
 * `Table reference`: when a function call depends on tables, you need to ensure the referenced table has the necessary content and is not empty.

From 2266d20992d219f44e8945a08765c3a34cf9708c Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 8 Jun 2021 13:44:18 +0100
Subject: [PATCH 71/77] Fix .complete example

---
 docs/configuration_files.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 69e10def0..8d7e9837f 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -331,7 +331,7 @@ Tables
           high: 10.0
           size: 20
       matching:
-        .init: !astropy.table.vstack
+        .init: !astropy.table.hstack
           tables: [ $halos, $galaxies ]
           .depends: [ halos.complete, galaxies.complete ]
 

From 16e21fc06a20f57730f3114efb775bb74d8c72a9 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 8 Jun 2021 15:53:31 +0100
Subject: [PATCH 72/77] Change growth example for size

---
 docs/configuration_files.rst | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 8d7e9837f..f0d32c677 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -211,27 +211,31 @@ Functions
 
 * The `cosmology` to be used by functions within the pipeline only needs to be set up at the top. If a function needs ``cosmology`` as an input, you need not define it again, it is automatically detected.
 
+  For example, calculate the angular size of a galaxy with a given physical size, at a fixed redshift and for a given cosmology:
+
   .. code:: yaml
 
       cosmology: !astropy.cosmology.FlatLambdaCDM
         H0: 70
         Om0: 0.3
-      growth: !skypy.power-spectrum.growth_function
+      size: !skypy.galaxies.morphology.angular_size
+        physical_size: 10 kpc
         redshift: 0.2
 
 * `Job completion`: ``.depends`` can be used to force any function call to wait for completion
   of any other job.
 
   A simple example where, for some reason, the comoving distance needs to be called after
-  completion of the growth function:
+  completion of the angular size function:
 
   .. code:: yaml
 
-    growth: !skypy.power-spectrum.growth_function
+    size: !skypy.galaxies.morphology.angular_size
+      physical_size: 10 kpc
       redshift: 0.2
     comoving_distance: !astropy.cosmology.Planck15.comoving_distance
       z: !numpy.linspace [ 0, 1.3, 10 ]
-      .depends: growth
+      .depends: size
 
   By doing so, you force the function call ``redshift`` to be completed before is used to compute the comoving distance.
 

From d1abe32de73b274f20801402b9c6c0adf4292e15 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 8 Jun 2021 16:26:54 +0100
Subject: [PATCH 73/77] Fix .depends example by adding cosmology

---
 docs/configuration_files.rst | 1 +
 1 file changed, 1 insertion(+)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index f0d32c677..bda9a5c8d 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -230,6 +230,7 @@ Functions
 
   .. code:: yaml
 
+    cosmology: !astropy.cosmology.Planck15
     size: !skypy.galaxies.morphology.angular_size
       physical_size: 10 kpc
       redshift: 0.2

From db4fcc8bdb17cdfd8562a0648ace0db4ae96dd4c Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 8 Jun 2021 16:28:01 +0100
Subject: [PATCH 74/77] Rephrase text in multi -column assignment

---
 docs/configuration_files.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index bda9a5c8d..535465a3e 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -292,7 +292,7 @@ Tables
 * `Multi-column assignment`: multi-column assignment is performed with any 2d-array, where one of the dimensions is interpreted
   as the rows of the table and the second dimension, as separate columns. Or you can do it from a function that returns a tuple.
 
-  We use multi-column assignment in the following example where we sample different halo properties from a lognormal distribution (returns a tuple of three arrays):
+  We use multi-column assignment in the following example where we sample a two-dimensional array of values from a lognormal distribution and then store them as three columns in a table:
 
   .. code:: yaml
 

From 63e653269c062d7caa27e3c22953ebfb7da6584f Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 8 Jun 2021 16:30:20 +0100
Subject: [PATCH 75/77] Change import objects example

---
 docs/configuration_files.rst | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 535465a3e..773260d7a 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -147,8 +147,7 @@ Variables
 
   .. code:: yaml
 
-      cosmo1: !astropy.cosmology.Planck13 # import the Planck13 object and bind to cosmo1
-      cosmo2: !astropy.cosmology.default_cosmology.get [] # call default_cosmology.get() and bind to cosmo2
+      cosmology: !astropy.cosmology.Planck13 # import the Planck13 object and bind it to the variable named "cosmology"
 
 
 Parameters

From a429941f5b7dd3f9d0e4656c91927ae3aac28503 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 8 Jun 2021 16:32:39 +0100
Subject: [PATCH 76/77] Add line re: comments

---
 docs/configuration_files.rst | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 773260d7a..0cb50ec63 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -109,6 +109,9 @@ YAML syntax
 YAML_ is a file format designed to be readable by both computers and humans.
 Fundamentally, a file written in YAML consists of a set of key-value pairs.
 Each pair is written as ``key: value``, where whitespace after the ``:`` is optional.
+The hash character (#) denotes the start of a comment and all further text on that
+line is ignored by the parser.
+
 
 This guide introduces the main syntax of YAML relevant when writing
 a configuration file to use with ``SkyPy``. Essentially, it begins with

From 9f651f9e097e9cd15831966fe74321f035416783 Mon Sep 17 00:00:00 2001
From: "Lucia F. de la Bella" <55983939+Lucia-Fonseca@users.noreply.github.com>
Date: Tue, 8 Jun 2021 16:38:40 +0100
Subject: [PATCH 77/77] Fix typo

---
 docs/configuration_files.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/configuration_files.rst b/docs/configuration_files.rst
index 0cb50ec63..7e3d71039 100644
--- a/docs/configuration_files.rst
+++ b/docs/configuration_files.rst
@@ -109,7 +109,7 @@ YAML syntax
 YAML_ is a file format designed to be readable by both computers and humans.
 Fundamentally, a file written in YAML consists of a set of key-value pairs.
 Each pair is written as ``key: value``, where whitespace after the ``:`` is optional.
-The hash character (#) denotes the start of a comment and all further text on that
+The hash character ``#`` denotes the start of a comment and all further text on that
 line is ignored by the parser.