Skip to content

Commit

Permalink
Doc updates
Browse files Browse the repository at this point in the history
  • Loading branch information
bmcfee committed Jan 31, 2024
1 parent 7ca1d77 commit 0e2f7eb
Show file tree
Hide file tree
Showing 5 changed files with 10 additions and 10 deletions.
8 changes: 4 additions & 4 deletions docs/bufferedstreaming.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Buffered Streaming
==================

In a machine learning setting, it is common to train a model with multiple input datapoints simultaneously, in what are commonly referred to as "minibatches". To achieve this, pescador provides the :ref:`pescador.maps.buffer_stream` map transformer, which will "buffer" a data stream into fixed batch sizes.
In a machine learning setting, it is common to train a model with multiple input datapoints simultaneously, in what are commonly referred to as "minibatches". To achieve this, pescador provides the `pescador.maps.buffer_stream` map transformer, which will "buffer" a data stream into fixed batch sizes.

Following up on the first example, we use the `noisy_samples` generator.

Expand All @@ -17,17 +17,17 @@ Following up on the first example, we use the `noisy_samples` generator.
minibatch_size = 128
# Wrap your streamer
buffered_sample_gen = pescador.buffer_stream(streamer, minibatch_size)
buffered_sample_gen = pescador.maps.buffer_stream(streamer, minibatch_size)
# Generate batches in exactly the same way as you would from the base streamer
for batch in buffered_sample_gen:
...
A few important points to note about using :ref:`pescador.maps.buffer_stream`:
A few important points to note about using `pescador.maps.buffer_stream`:

- :ref:`pescador.maps.buffer_stream` will concatenate your arrays, adding a new sample dimension such that the first dimension contains the number of batches (`minibatch_size` in the above example). e.g. if your samples are shaped (4, 5), a batch size of 10 will produce arrays shaped (10, 4, 5)
- `pescador.maps.buffer_stream` will concatenate your arrays, adding a new sample dimension such that the first dimension contains the number of batches (`minibatch_size` in the above example). e.g. if your samples are shaped (4, 5), a batch size of 10 will produce arrays shaped (10, 4, 5)

- Each key in the batches generated will be concatenated (across all the samples buffered).

Expand Down
6 changes: 4 additions & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@

# The reST default role (used for this markup: `text`) to use for all
# documents.
#default_role = None
default_role = "autolink"

# If true, '()' will be appended to :func: etc. cross-reference text.
#add_function_parentheses = True
Expand Down Expand Up @@ -146,7 +146,9 @@

# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = 'default'
import sphinx_rtd_theme
html_theme = 'sphinx_rtd_theme'
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]

# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
Expand Down
2 changes: 0 additions & 2 deletions docs/example2.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,6 @@ cleaner syntax.
.. code-block:: python
:linenos:
from __future__ import print_function
import numpy as np
from sklearn.datasets import load_breast_cancer
Expand Down
2 changes: 1 addition & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ To make this all possible, Pescador provides the following utilities:
- :ref:`Mux` objects allow flexible sampling from multiple streams
- :ref:`ZMQStreamer` provides parallel processing with low communication overhead
- Transform or modify streams with Maps (see :ref:`processing-data-streams`)
- Buffering of sampled data into fixed-size batches (see :ref:`pescador.maps.buffer_stream`)
- Buffering of sampled data into fixed-size batches (see `pescador.maps.buffer_stream`)

************
Installation
Expand Down
2 changes: 1 addition & 1 deletion examples/mux/epoch.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
- If presentation order does not matter, and a deterministic
sequence is acceptable, then this can be achieved with
`ChainMux` as demonstrated in :ref:`Using ChainMux for repeatable streams`.
`ChainMux` as demonstrated in :ref:`sphx_glr_auto_examples_mux_chain_example.py`.
This is typically a good approach for validation or evaluation,
but not training, since the deterministic sequence order could bias
the model.
Expand Down

0 comments on commit 0e2f7eb

Please sign in to comment.