Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FEAT: +TOKEN modifier for side-effect functions #15

Open
wants to merge 102 commits into
base: master
Choose a base branch
from

Commits on Sep 28, 2019

  1. Configuration menu
    Copy the full SHA
    617e577 View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2019

  1. FIX(yahoo#13): BUG in plot-diagram writtin from PY2 era,

    were writing in text-mode in PY3. and failing as encoding error.
    ankostis committed Sep 29, 2019
    Configuration menu
    Copy the full SHA
    f58d148 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    c75a2c0 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    a005bd6 View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2019

  1. FEAT: +TOKEN modifier for side-effect functions

    Tokens work as usual while solving the DAG but
    they are never assigned any values to/from the operation's functions.
    
    + TC included.
    + Docs updated.\+ Added `modifiers` superclass to facilitate identification code.
    + refactored FunctionalOperation._compute().
    ankostis committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    506be80 View commit details
    Browse the repository at this point in the history
  2. fix(func): 1-item outs broke with token modifiers...

    + _norm_kwargs:
      + use isinstance() instead of type-equality checks,
        to support new modier classes;
        + avoid excessive dict searches with local vars.
    ankostis committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    4e6d543 View commit details
    Browse the repository at this point in the history
  3. refact(token): undo adding of new modifier class...

    not really needed, better be explicit which modifier is searched.
    ankostis committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    62cf0a0 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    47c6c1d View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    94b7291 View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2019

  1. enh(test): + x2 TC breaking UNSATISFIED operations...

    receiving partial inputs, needed for other operations.
    ankostis committed Oct 1, 2019
    Configuration menu
    Copy the full SHA
    52c0d77 View commit details
    Browse the repository at this point in the history
  2. ENH(net,yahoo#18): ignore UN-SATISFIABLE operations with partial inputs

    + The x2 TCs added just before are now passing.
    ankostis committed Oct 1, 2019
    Configuration menu
    Copy the full SHA
    bc4c221 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    b8daa07 View commit details
    Browse the repository at this point in the history
  4. ENH(core): ORDERED SETs for DETERMINISTIC results

    NOTE dict are not deterministic in <PY3.6.
    So this commit would not improve determinism
    in those pythons.
    
    + build: add `boltons` dependency for ndexedSet.
    + doc: mark all set usage if affect determinism.
    + e.g. see reproducibility problem in yahoo#14.
    ankostis committed Oct 1, 2019
    Configuration menu
    Copy the full SHA
    12bdfe4 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    b8377ca View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    489b32c View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2019

  1. Configuration menu
    Copy the full SHA
    b102d44 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    de02885 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    e1454fd View commit details
    Browse the repository at this point in the history
  4. MERGE (prune_unsatified, ordered_sets) into fix-pruning ...

    needed to refactor the new pruning algorithm.
    
    - expected 2 TCs fail for yet-to-be-solved yahoo#24 & yahoo#25 bugs.
    ankostis committed Oct 2, 2019
    Configuration menu
    Copy the full SHA
    3736738 View commit details
    Browse the repository at this point in the history
  5. DOC(net): explain DAG solution & compilation...

    the later described in yahoo#21.
    ankostis committed Oct 2, 2019
    Configuration menu
    Copy the full SHA
    c273068 View commit details
    Browse the repository at this point in the history
  6. TEST(prune): +Failing x2 TCs multi-out must run but not...

    override intermediate data.
    
    More changes only for newer pruning TCs:
    + refact(test): rename graph-->netop vars for results of compose(),
      to avoid of `graph.net.graph`.
    + Explain failure modes in v1.2.4 & this merged branch (yahoo#19 + yahoo#23).
    ankostis committed Oct 2, 2019
    Configuration menu
    Copy the full SHA
    16d42f1 View commit details
    Browse the repository at this point in the history
  7. refact(dag): call compile() before compute.compute...

    not after compose().
    
    + All TCs pass ok.
    + NOTE this is not yet what is described in yahoo#21.
    ankostis committed Oct 2, 2019
    Configuration menu
    Copy the full SHA
    b92f103 View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2019

  1. Configuration menu
    Copy the full SHA
    6d1884e View commit details
    Browse the repository at this point in the history
  2. ENH(net): move compile() after SOLVE DAG ...

    to pass +TC checking DeleteInst vary when inputs change.
    
    - x4 TCs still failing, and need revamp of dag-solution.
    ankostis committed Oct 3, 2019
    Configuration menu
    Copy the full SHA
    619cae7 View commit details
    Browse the repository at this point in the history
  3. REFACT(NET) COMPILE+COMPUTE...

    + Read the next doc-only commit to understand changes.
    + Renamed:
      + net.steps --> net.execution_plan.
      + (old)compile() --> _build_execution_plan()
      + _find_necessary_steps() --> (new)compile() + _solve_dag()
        compile() became the master function invoking _solve_dag &
        _build-execution_plan(), and do the caching.
    + refact(compute()): extract common tasks from sequential/parallel.
    + refact show_layers() to allow full-print, geting also string
      (not just printing), and using custom classes for representation.
    + Raise AssertionError when invalid class in plan.
      it's a logic error, not a language type-error.
    ankostis committed Oct 3, 2019
    Configuration menu
    Copy the full SHA
    eff351d View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    d959485 View commit details
    Browse the repository at this point in the history
  5. FIX(net): new Ops invalidate execution-plan cache...

    Probaly unreported bug in v1.2.4 for '_neccessary_steps_cache`.
    ankostis committed Oct 3, 2019
    Configuration menu
    Copy the full SHA
    17eb2fd View commit details
    Browse the repository at this point in the history
  6. ENH(DAG): NEW SOLVER

    + Pruning behaves correctly also when outputs given;
      this happens by breaking incoming provide-links
      to any given intermedediate inputs.
    + Unsatisfied detection now includes those without outputs
      due to broken links (above).
    + Remove some uneeded "glue" from unsatisfied-detection code,
      leftover from previous compile() refactoring.
    + Renamed satisfiable --> satisfied.
    + Improved unknown output requested raise-message.
    + x3 TCs PASS, x1 in yahoo#24 and the first x2 in yahoo#25.
    - 1x TCs in yahoo#25 still FAIL, and need "Pinning" of given-inputs
      (the operation MUST and MUST NOT run in these cases).
    ankostis committed Oct 3, 2019
    Configuration menu
    Copy the full SHA
    0830b7c View commit details
    Browse the repository at this point in the history
  7. enh(build): replace numpy with pytest...

    numpy was used just for its assert_raise
    ankostis committed Oct 3, 2019
    Configuration menu
    Copy the full SHA
    32409f6 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    f606ed1 View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2019

  1. WIP/FIX(prune,yahoo#26): PIN intermediate inputs if operation before …

    …must run
    
    - WIP: PARALLEL execution not adding PINS!
    + Insert "PinInstructions" in the execution-plan to avoid overwritting.
    + Add `_overwrite_collector` in `compose()` to collect re-calculated values.
    + FIX the last TC in yahoo#25.
    ankostis committed Oct 4, 2019
    Configuration menu
    Copy the full SHA
    0dc1293 View commit details
    Browse the repository at this point in the history
  2. REFACT(net): part 3 of new dag-solver & pin refactoring

    - STILL buggy PIN on PARALLEL, 2 DISABLED TCs FAIL:
      - test_pruning_with_given_intermediate_and_asked_out()
      - test_unsatisfied_operations_same_out()
    + move check if value in asked outputs before cache-evicting it
      in build-execution-plan time - compute methods
      don't need outputs anymore.
    + test: speed up parallel/multihtread TCs
      by reducing delays & repetitions.
    + refact: network rightfully adopted stray functions
      for parallel processing - they all worke on the net.graph,
    + upd: networkx api by indexing on `dag.nodes` views.
    + enh: add log message when deleting in parallel
      (in par with sequential code).
    + refact: var-renames, if-then-else simplifications, pythonisms.
    + doc: A lot!
    ankostis committed Oct 4, 2019
    Configuration menu
    Copy the full SHA
    06f6554 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    1cc733e View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    cd1370b View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    f676662 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    65d1816 View commit details
    Browse the repository at this point in the history
  7. WIP/FIX(PIN): PARALLEL DELs decide on PRUNED-dag (not full)...

    - WIP: x4 TCs FAIL and still not discovered th bug :-(
    + BUT ALL+AUGMENTED PARALLEL TCs pass
      (yahoo#26 were failing some)
    + refact: net stores also `pruned_dag` (not only `steps`).
    + refact: _solve_dag() --> _prune_dag().
    + doc: +a lot.
    + TODO: store pruned_dag in own ExePlan class.
    ankostis committed Oct 4, 2019
    Configuration menu
    Copy the full SHA
    d403783 View commit details
    Browse the repository at this point in the history
  8. WIP/+ExecPlan class...

    - SAME x4 TCs FAIL like parent.
    + refact: revive net.steps --> net.last_plan
    + BREAK: inverse the order of outputs/inputs args in net.compute()
    + refact: use iset() to merge.
    ankostis committed Oct 4, 2019
    Configuration menu
    Copy the full SHA
    64e0028 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    4e55b30 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    47b50f6 View commit details
    Browse the repository at this point in the history

Commits on Oct 5, 2019

  1. refact(plot): extract plot function out of Network class...

    to use it also on stary DAGs.  Keep delegation.
    ankostis committed Oct 5, 2019
    Configuration menu
    Copy the full SHA
    b1d02a1 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    c11af2a View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    344490b View commit details
    Browse the repository at this point in the history
  4. ENH(plot): +inputs, +outputs, +solution modify plotting

    (see yahoo#13 for an example):
    ankostis committed Oct 5, 2019
    Configuration menu
    Copy the full SHA
    23ef81e View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    4e8601c View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    834a8b0 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    c2e28a4 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    e38c8ad View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    d855bf6 View commit details
    Browse the repository at this point in the history
  10. test(plot): enhance plot test to try all yahoo#13 features;

    + test all chained plot() methods from netop.
    ankostis committed Oct 5, 2019
    Configuration menu
    Copy the full SHA
    ca5d243 View commit details
    Browse the repository at this point in the history
  11. test(optional): +x1 TC to check selective optionals with same out...

    + enh TC with parallel.
    + Scavenged from yahoo#20.
    ankostis committed Oct 5, 2019
    Configuration menu
    Copy the full SHA
    f25f189 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    a2de9ef View commit details
    Browse the repository at this point in the history
  13. FIX(PLOT.TC): TC was always testing PNG, ...

    + retorfitted to try all available formats.
    + list of forbidden formats based on my failres
    ankostis committed Oct 5, 2019
    Configuration menu
    Copy the full SHA
    dc5a21a View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    782d9b9 View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    7d389c3 View commit details
    Browse the repository at this point in the history
  16. ENH(plot): return SVG rendered in JUPYTER, ...

    + doc: rename in sample code: netop --> pipeline.
    + enh(build): add `ipython` in test dependencies.
    + include it in the plot TC.
    ankostis committed Oct 5, 2019
    Configuration menu
    Copy the full SHA
    3fe0b40 View commit details
    Browse the repository at this point in the history
  17. Configuration menu
    Copy the full SHA
    1471551 View commit details
    Browse the repository at this point in the history
  18. Configuration menu
    Copy the full SHA
    af7ae0f View commit details
    Browse the repository at this point in the history
  19. Configuration menu
    Copy the full SHA
    b440196 View commit details
    Browse the repository at this point in the history
  20. Configuration menu
    Copy the full SHA
    bde9b64 View commit details
    Browse the repository at this point in the history
  21. Configuration menu
    Copy the full SHA
    8e361e6 View commit details
    Browse the repository at this point in the history
  22. DROP PY3.4 - add PY3.6, PY3.7...

    ...pytest has problems with 3.4.
    ankostis committed Oct 5, 2019
    Configuration menu
    Copy the full SHA
    b08a363 View commit details
    Browse the repository at this point in the history
  23. Configuration menu
    Copy the full SHA
    3a87959 View commit details
    Browse the repository at this point in the history
  24. Configuration menu
    Copy the full SHA
    17f18c2 View commit details
    Browse the repository at this point in the history
  25. Configuration menu
    Copy the full SHA
    4d250d3 View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2019

  1. ENH(plot,net): +plot() on ExecPlan; +PlotMixin ...

    to avoid copying plot() doc+sig around
    ankostis committed Oct 6, 2019
    Configuration menu
    Copy the full SHA
    18191e4 View commit details
    Browse the repository at this point in the history
  2. enh(plan): repr()

    ankostis committed Oct 6, 2019
    Configuration menu
    Copy the full SHA
    7341529 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    231ada5 View commit details
    Browse the repository at this point in the history
  4. FIX(plot): distinguish Del/Pin cmds; +choice utils

    - still x4 TCs fail...
    ankostis committed Oct 6, 2019
    Configuration menu
    Copy the full SHA
    80f110a View commit details
    Browse the repository at this point in the history
  5. refact(plot): inline imports, not to cycle with base...

    to bring back PlotMixin to change together stuff.
    ankostis committed Oct 6, 2019
    Configuration menu
    Copy the full SHA
    32eaa80 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    4b70cfb View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    b4fa5e0 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    c6f2155 View commit details
    Browse the repository at this point in the history
  9. ENH(plan,plot): executed operations drawn as filled

    + enh(plan): also sequential execution collects accounts executed.
    + refact(plan): executed_nodes-->executed.
    ankostis committed Oct 6, 2019
    Configuration menu
    Copy the full SHA
    ec69090 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    24a3d1e View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    77fc887 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    da087df View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    ac73bbb View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    c2829a3 View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    c61947b View commit details
    Browse the repository at this point in the history
  16. REFACT(PLOT): Plotter builds dot & renders...

    ... based on Law of Demeter simplify kwargs,
    defined in one place.
    
    + enh: plotters "suggest" kwargs, possinly to override them
      (no duplcate arg when attempted).
    ankostis committed Oct 6, 2019
    Configuration menu
    Copy the full SHA
    434d4e2 View commit details
    Browse the repository at this point in the history
  17. Configuration menu
    Copy the full SHA
    dabc787 View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2019

  1. FIX(plot): VISUAL fixes & COLOR-palettes:

    + FIX: apply thickness as in legend also for operators.
    + enh: don't cluster if no nodes pruned.
    + enh: netop includes its name a graph-title.
    + color palette: wheat filled nodes.
    + reuse common func.
    ankostis committed Oct 7, 2019
    Configuration menu
    Copy the full SHA
    c5b9167 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    1d443ce View commit details
    Browse the repository at this point in the history
  3. FIX(NET): were FORGETTING PRUNED ASKED-OUTPUTs...

    ... bugged in the opening commit d403783 of this PR, and
    discovered 68(!) commits later, and all that time had to live
    with x4 broken TCs with asked-outputs.
    ankostis committed Oct 7, 2019
    Configuration menu
    Copy the full SHA
    77bec49 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    89e4edb View commit details
    Browse the repository at this point in the history
  5. FIX(plot.TC): formats-TC were testing dot-file, not rendered...

    + ENH: strongly refatctored TC to detect nulls & dupes.
    ankostis committed Oct 7, 2019
    Configuration menu
    Copy the full SHA
    cef7526 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    f9b2415 View commit details
    Browse the repository at this point in the history
  7. FIX(<PY3.5): ORDERED DiGRAPH for old Python to fix TCs

    + Partial fix deterministic results (yahoo#22-2.4.3i).
    ankostis committed Oct 7, 2019
    Configuration menu
    Copy the full SHA
    fce0515 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    d8ba34f View commit details
    Browse the repository at this point in the history
  9. feat(plot): +legend & TC

    ankostis committed Oct 7, 2019
    Configuration menu
    Copy the full SHA
    e8fec22 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    7a87d61 View commit details
    Browse the repository at this point in the history

Commits on Oct 8, 2019

  1. DOC(PLOT): +sample GRAPH & LEGEND into README...

    + test: some refacts on vars & for the uploaded images.
    + MARK SPURIOUS FAILS in < PY3.6 due to unordered dicts
      eg https://travis-ci.org/ankostis/graphkit/jobs/594813119
    ankostis committed Oct 8, 2019
    Configuration menu
    Copy the full SHA
    53c6ce5 View commit details
    Browse the repository at this point in the history
  2. DROP(plot) jupyter kw, simpler to monkeypath Dot class...

    to support _repr_svg(), and it works without any flag.
    ankostis committed Oct 8, 2019
    Configuration menu
    Copy the full SHA
    ff754a1 View commit details
    Browse the repository at this point in the history
  3. FIX(plot): LEGEND mistakes, SVGize, egg graphs, ...

    + graphop label renames,
    + DOT shortening.
    ankostis committed Oct 8, 2019
    Configuration menu
    Copy the full SHA
    f787f26 View commit details
    Browse the repository at this point in the history
  4. FIX(DOC): don't use graph in sample code, crash DOT, +more ...

    + FIX: `graph` is a DOT keyword, eg  pydot/pydot#111
      + Replaced `graph` --> `graphop`, inline with future refactoring.
    + Refactored example code.
    + New "Plotting" section under Quickstart.
    + Copy README instructions to main docs.
    + Abandoned `example_graph.svg`, replaced with `intro.svg`.
    + Recommend `.png` --> `.svg`, to save storage space.
    - discovered BUG in MERGE saumple code (empty!).
    ankostis committed Oct 8, 2019
    Configuration menu
    Copy the full SHA
    5cf7189 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    7f637fe View commit details
    Browse the repository at this point in the history
  6. chore(TCs): mark SLOW tests, when in hurry, +setup.cfg

    ... to adopt pytest args
    + mark wheel a universal.
    ankostis committed Oct 8, 2019
    Configuration menu
    Copy the full SHA
    ae01163 View commit details
    Browse the repository at this point in the history
  7. FIX(TCs): MERGE TCs were not ASSERTING...

    got values from v1.2.4
    - ALL MERGE TCs FAIL!
    ankostis committed Oct 8, 2019
    Configuration menu
    Copy the full SHA
    64838a5 View commit details
    Browse the repository at this point in the history
  8. FIX(MERGE): broken by NEW_DAG_SOLVER (yahoo#26 ...

    many commits ago.
    
    Never got it bc TC were not checking merges!
    ankostis committed Oct 8, 2019
    Configuration menu
    Copy the full SHA
    8c41066 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    6b452f7 View commit details
    Browse the repository at this point in the history
  10. FIX(sideffect.TX): old TC was not with unsatisfied in mind,

    needed to give the sidefeect as input.
    ankostis committed Oct 8, 2019
    Configuration menu
    Copy the full SHA
    3e06148 View commit details
    Browse the repository at this point in the history