cfelton · cfelton · Sep 21, 2016 · Sep 21, 2016 · Sep 21, 2016 · Sep 21, 2016
diff --git a/.travis.yml b/.travis.yml
@@ -42,7 +42,7 @@ install:
 
 
 script:
-   - cd test
+   - cd tests
    - coverage run --source=jpegenc -m py.test --include-reference 
    - cp .coverage ../
 

diff --git a/README.md b/README.md
@@ -8,21 +8,67 @@
 
 Introduction
 ============
-This repository is a development sandbox.  The respository 
+The `test_jpeg` repository is in the middle of a change, originally 
+this repository was an example how to use MyHDL to setup a cosimulation 
+environment and the jpegenc cores were used (cosim with Verilog) ... 
+it is evolving to something more.
+
+During the summer of 2016 two students participated in 
+[MyHDL's GSoC](http://dev.myhdl.org/gsoc/gsoc_2016.html).  The 
+students worked on implementing the various JPEG Encoder subblocks 
+in MyHDL.  The sublocks were broken into frontend and backend. 
+The student's blogs for 
+[the frontend](https://myhdlgsoc2016.blogspot.gr/) 
+and 
+[the backend](https://vikram9866.wordpress.com/), 
+outline their 
+work over the summer.  The [latest documentation is available 
+on readthedocs](http://jpegenc.readthedocs.io/en/latest/)
+
+The primary goal of this repository is to have a working JPEG 
+Encoder (`jpegenc`) implemented in MyHDL.  But not simply 
+implemented in MyHDL but also utilizing methodologies such as 
+a focus on clean and reusable block-to-block interfaces and a full 
+regression test suite.
+
+This repository is a development sandbox.  The repository 
 contains three related development efforts:
 
-1. VHDL to Verilog conversion of the first JPEG-encoder reference 
-   (rational explained below).
+1. A MyHDL implementation of a JPEG-Encoder.
+2. Two Verilog reference implementations.
 2. MyHDL verification and co-simulation environment, tests will 
-   functionally verify and compare the reference JPEG-encoders.
-3. A MyHDL implementation of a JPEG-encoder.
+   functionally verify and compare the reference JPEG-Encoders. 
 
-The JPEG encoders used are the cores available at open-cores in 
-addition to JPEG encoders being developed.  This is a WIP and not 
-fully complete.
+The reference JPEG encoders are the cores available at open-cores.  
 
+This is a WIP and not fully complete (yet).
 
-Verification Environment
+
+Quick start guide
+=================
+
+Get the latest `jpegenc` package from the `test_jpeg` repository.
+
+>> git clone https://github.com/cfelton/test_jpeg
+
+Create a [virtualenv](https://www.davidfischer.name/2010/04/why-you-should-be-using-pip-and-virtualenv/) 
+to test the `jpegenc` package.
+
+>> virtualenv --no-site-packages env/test_jpeg
+>> source env/test_jpeg/bin/activate
+
+Install the required packages.
+
+>> pip install -r requirements.txt
+
+Run the test suite.
+
+>> cd tests
+>> py.test
+
+
+
+Verification environment
 ========================
 A stimulus and verification environment was created with Python and
 MyHDL.  An image is streamed to the encoder and the output is captured.
@@ -33,24 +79,28 @@ In the future the output of the various encoders will be compared to
 each other as well as the encoder performance (.e.g. max fps).
 
 
-JPEG Encoders
-=============
+Cosimulation
+============
+This repository also contains a cosimulation interface to two 
+reference designs.
 
-   - hdl/jpegenc_v1: [VHDL JPEG encoder](http://opencores.org/project,mkjpeg) converted to Verilog
-   - hdl/jpegenc_v2: [OC Verilog JPEG](http://opencores.org/project,jpegencode) encoder.
-   - jpegenc: A fresh (non-ported) MyHDL implementation, `jpegenc` is a Python package
-     being developed.
 
-<!-- (@todo: the above need better names?) -->
+Reference JPEG encoders
+-----------------------
 
+   - tests/reference_designs/jpegenc_v1: [VHDL JPEG encoder](http://opencores.org/project,mkjpeg) 
+     converted to Verilog
+   - tests/reference_designs/jpegenc_v2: [OC Verilog JPEG](http://opencores.org/project,jpegencode) 
+     encoder.
+
 
-Getting Started
-===============
-To run the test the following needs to be installed:
+Getting started with cosim
+--------------------------
+To run the cosim tests the following needs to be installed:
 
   * Icarus Verilog
-  * Python (currently using 2.7)
-  * MyHDL
+  * Python (currently using 3.5)
+  * MyHDL (1.0dev)
   * Python Imaging Library (e.g. pip install Pillow)
 
 
@@ -60,15 +110,15 @@ directory.
 ```
 >> cd myhdl/cosimulation/icarus
 >> make 
->> cp myhdl.vpi <somewhere>/test_jpeg/test
+>> cp myhdl.vpi <somewhere>/test_jpeg/tests
 ```  
 
-Once the tools and installed and the VPI module built the test can
+Once the tools are installed and the VPI module built the tests can
 be run.
 
 ```
 >> cd test
->> python test_jpecenc.py
+>> py.test test_jpecenc.py
 ```
 
 Depending on the test file the test can take significant time to run.
@@ -142,28 +192,32 @@ and fast ... correct?
 Progress Log
 ==============
 
-   **28-Dec-2014** : Conversion error was found in V1, V1 finishes
+   **09-Oct-2016**: A version of the JPEG encoder has been 
+     implemented in MyHDL.  This was completed by two students 
+     during GSoC 2016.
+
+   **28-Dec-2014**: Conversion error was found in V1, V1 finishes
      compressing the frame.  V2 has error with small images, this 
      might be a design limitation.
 
-   **07-Dec-2014** : Neither encoder completes successfully with a 
+   **07-Dec-2014**: Neither encoder completes successfully with a 
      small test image, currently debugging (design1 (v1) conversion 
      complete need to find any errors in conversion).  The test 
      environment will stream the image in and complete shortly after 
      the image has been streamed in.
 
-   **05-Dec-2014** : Design1 conversion to verilog is mostly complete, 
+   **05-Dec-2014**: Design1 conversion to verilog is mostly complete, 
      spending some time verifying.
 
-   **09-Nov-2014** : The test environment will stream an image to both
+   **09-Nov-2014**: The test environment will stream an image to both
      the design1 and design2 encoders.  The output is not interrogated
      (yet).  Design1 conversion to verilog is incomplete.
 
 
 Things to be completed
 ----------------------
 
-   1. [ ] Use Python3 as the default python verson.
+   1. [X] Use Python3 as the default python verson.
 
    1. [ ] Check encoded bitstreams, determine metrics to compare 
           encoders.

diff --git a/jpegenc/models/system/__init__.py → docs/jpegenc_v2_notes.md b/jpegenc/models/system/__init__.py → docs/jpegenc_v2_notes.md
diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -35,6 +35,7 @@
     'sphinx.ext.todo',
     'sphinx.ext.coverage',
     'sphinx.ext.mathjax',
+    'sphinx.ext.napoleon',
 ]
 
 # Add any paths that contain templates here, relative to this directory.
@@ -114,7 +115,8 @@
 
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
-html_theme = 'alabaster'
+# html_theme = 'alabaster'
+html_theme = 'default'
 
 # Theme options are theme-specific and customize the look and feel of a theme
 # further.  For a list of options available for each theme, see the

diff --git a/docs/source/getting_started.rst b/docs/source/getting_started.rst
@@ -0,0 +1,14 @@
+
+Quick start
+===========
+
+The easiest path to install the package is to use `pip`::
+
+  >> pip install git://github.com/cfelton/test_jpeg.git#egg=test_jpeg
+
+
+Once the package is installed the test suite can be run::
+
+  >> cd tests
+  >> py.test
+
diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -3,42 +3,17 @@
    You can adapt this file completely to your liking, but it should at least
    contain the root `toctree` directive.
 
-Welcome to jpegenc's documentation!
-===================================
 
-Backend Modules Documentation
-+++++++++++++++++++++++++++++++++++
-
-Backend Modules:
-
-.. toctree::
-   :maxdepth: 2
-
-    Quantizer<./subblocks/quantizer.rst>
-    RLE<./subblocks/rle.rst>
-    Huffman<./subblocks/huffman.rst>
-    ByteStuffer<./subblocks/bytestuffer.rst>
-    Backend Part<./subblocks/backend.rst>
-
-
-Test-Bench:
+JPEG encoder (test_jpeg) documentation
+=======================================
 
 .. toctree::
-   :maxdepth: 2
-
-    Quantizer Test <./subblocks/test_quantizer.rst>
-    RLE Test <./subblocks/test_rle.rst>
-    Huffman Test<./subblocks/test_huffman.rst>
-    ByteStuffer Test<./subblocks/test_bytestuffer.rst>
-    Backend Test<./subblocks/test_backend.rst>
-
+    :maxdepth: 1
 
-Results:
-
-.. toctree::
-   :maxdepth: 1
+    overview
+    getting_started
+    reference
 
-    Coverage<./subblocks/coverage_backend.rst>
 
 Indices and tables
 ==================

diff --git a/docs/source/overview.rst b/docs/source/overview.rst
@@ -0,0 +1,45 @@
+
+Overview
+========
+The `jpegenc` package is a JPEG encoder implemented in
+MyHDL_.  The JPEG encoder is intended to be
+flexible with reusable subblocks.  This project also includes a
+verification environment for the encoder.
+
+.. _MyHDL: http://www.myhdl.org
+
+The following figure outlines the main subblocks in the system.
+
+.. figure:: https://cloud.githubusercontent.com/assets/766391/18724671/e3eff6e8-8002-11e6-9dfe-9a03379a06fb.png
+    :scale: 80%
+
+    The JPEG encoder system diagram
+
+The subblocks were designed to be independent and process an
+image stream.
+
+
+Uses
+----
+   * (M)JPEG real-time video compression.
+   * Framework for investigation image and video compression.
+
+
+Goals
+-----
+
+   * Easy to use and understand JPEG encoder implementation.
+   * Flexible (modular) and reusable subblocks.
+   * Base set of blocks to build various image and video encoders.
+
+
+Measurements
+------------
+
+.. toctree::
+    :maxdepth: 1
+
+     Coverage Frontend<./subblocks/coverage.rst>
+     Coverage Backend<./subblocks/coverage_backend.rst>
+     Implementation Results<./subblocks/impl.rst>
+
diff --git a/docs/source/reference.rst b/docs/source/reference.rst
@@ -0,0 +1,67 @@
+
+JPEG encoder reference
+======================
+
+Interfaces
+----------
+.. todo::
+
+    This sections needs to be completed, the interfaces are in a state
+    of change.
+
+
+Frontend
+---------
+
+Interfaces
+++++++++++
+
+.. toctree::
+   :maxdepth: 1
+
+    Color Space Conversion Interfaces<./subblocks/test_color_converts_ints.rst>
+    1D-DCT Interfaces<./subblocks/dct_1d_ints.rst>
+    2D-DCT Interfaces<./subblocks/dct_2d_ints.rst>
+    Zig Zag Scan Interfaces<./subblocks/zig_zag_scan_ints.rst>
+    Frontend Part Interfaces<./subblocks/frontend_ints.rst>
+
+
+Subblocks
++++++++++
+
+.. toctree::
+   :maxdepth: 1
+
+    Color Space Conversion<./subblocks/color_converts.rst>
+    Color Space Conversion v2<./subblocks/color_converts_v2.rst>
+    1D-DCT<./subblocks/dct_1d.rst>
+    2D-DCT<./subblocks/dct_2d.rst>
+    Zig Zaz Scan Module<./subblocks/zig_zag.rst>
+    Frontend Part<./subblocks/frontend.rst>
+
+    Quantizer<./subblocks/quantizer.rst>
+    RLE<./subblocks/rle.rst>
+    Huffman<./subblocks/huffman.rst>
+    ByteStuffer<./subblocks/bytestuffer.rst>
+    Backend Part<./subblocks/backend.rst>
+
+
+Test units
+++++++++++
+
+.. toctree::
+   :maxdepth: 1
+
+    Color Space Conversion Test <./subblocks/test_color_converts.rst>
+    Color Space Conversion v2 Test <./subblocks/test_color_converts_v2.rst>
+    1D-DCT Test<./subblocks/test_dct_1d.rst>
+    2D-DCT Test<./subblocks/test_dct_2d.rst>
+    Zig Zag Scan Test<./subblocks/test_zig_zag.rst>
+    Frontend Part Test<./subblocks/test_frontend.rst>
+
+    Quantizer Test <./subblocks/test_quantizer.rst>
+    RLE Test <./subblocks/test_rle.rst>
+    Huffman Test<./subblocks/test_huffman.rst>
+    ByteStuffer Test<./subblocks/test_bytestuffer.rst>
+    Backend Test<./subblocks/test_backend.rst>
+
diff --git a/jpegenc/interfaces/__init__.py b/jpegenc/interfaces/__init__.py
@@ -0,0 +1,14 @@
+
+from __future__ import absolute_import
+
+from .object_with_blocks import ObjectWithBlocks
+from .datastream import DataStream
+from .pixelstream import PixelStream
+from .rgbstream import RGBStream
+from .ycbcrstream import YCbCrStream
+
+
+__all__ = [
+    'ObjectWithBlocks', 'DataStream', 'PixelStream',
+    'RGBStream', 'YCbCrStream',
+]