Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add ImageNumber distinction through TableNumber capabilities #59

Closed
wants to merge 66 commits into from
Closed

Add ImageNumber distinction through TableNumber capabilities #59

wants to merge 66 commits into from

Commits on Apr 20, 2023

  1. increase duckdb threads

    @d33bs
    d33bs committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    cb0440b View commit details
    Browse the repository at this point in the history

  2. dynamic threading for duckdb

    @d33bs
    d33bs committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    f0a5bc8 View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2023

  1. add adbc

    @d33bs
    d33bs committed Apr 21, 2023
    Configuration menu
    Copy the full SHA
    d1b0bfd View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2023

  1. move to parquet for _read_data

    @d33bs
    d33bs committed Apr 22, 2023
    Configuration menu
    Copy the full SHA
    c8bf5f9 View commit details
    Browse the repository at this point in the history

  2. move prepend col ops to data read

    @d33bs
    d33bs committed Apr 22, 2023
    Configuration menu
    Copy the full SHA
    edfd4c6 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2023

  1. pathing

    @d33bs
    d33bs committed Apr 23, 2023
    Configuration menu
    Copy the full SHA
    105e735 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2023

  1. remove extension from pathing; move column naming

    @d33bs
    d33bs committed Apr 24, 2023
    Configuration menu
    Copy the full SHA
    e3eaa63 View commit details
    Browse the repository at this point in the history

  2. enable list of list for concat

    @d33bs
    d33bs committed Apr 24, 2023
    Configuration menu
    Copy the full SHA
    065cdd4 View commit details
    Browse the repository at this point in the history

  3. move dest path to tables reference

    @d33bs
    d33bs committed Apr 24, 2023
    Configuration menu
    Copy the full SHA
    b8aa21f View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2023

  1. fix joins

    @d33bs
    d33bs committed Apr 25, 2023
    Configuration menu
    Copy the full SHA
    b245a2c View commit details
    Browse the repository at this point in the history

  2. raw data parquet chunks; csv duckdb reader 

    + preset overrides
    @d33bs
    d33bs committed Apr 25, 2023
    Configuration menu
    Copy the full SHA
    c3bbd08 View commit details
    Browse the repository at this point in the history

  3. memory safety for parquet joins

    @d33bs
    d33bs committed Apr 25, 2023
    Configuration menu
    Copy the full SHA
    3b6e99a View commit details
    Browse the repository at this point in the history

  4. concurrency limits; sequential execution

    @d33bs
    d33bs committed Apr 25, 2023
    Configuration menu
    Copy the full SHA
    8ef35c2 View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2023

  1. corrected sequenced reads; concurrency limit

    @d33bs
    d33bs committed Apr 26, 2023
    Configuration menu
    Copy the full SHA
    1b60ca7 View commit details
    Browse the repository at this point in the history

  2. re-generalize

    @d33bs
    d33bs committed Apr 26, 2023
    Configuration menu
    Copy the full SHA
    769a897 View commit details
    Browse the repository at this point in the history

  3. limited concurrent extracts

    @d33bs
    d33bs committed Apr 26, 2023
    Configuration menu
    Copy the full SHA
    02893b7 View commit details
    Browse the repository at this point in the history

  4. concurrency setting update

    @d33bs
    d33bs committed Apr 26, 2023
    Configuration menu
    Copy the full SHA
    3e15472 View commit details
    Browse the repository at this point in the history

Commits on Apr 27, 2023

  1. wrap functionality in local prefect api default

    @d33bs
    d33bs committed Apr 27, 2023
    Configuration menu
    Copy the full SHA
    72c3e9e View commit details
    Browse the repository at this point in the history

  2. prefect server context and avoiding subflows

    @d33bs
    d33bs committed Apr 27, 2023
    Configuration menu
    Copy the full SHA
    2f3fc7c View commit details
    Browse the repository at this point in the history

  3. migrate from prefect to parsl

    @d33bs
    d33bs committed Apr 27, 2023
    Configuration menu
    Copy the full SHA
    eb704c4 View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2023

  1. parsl refinement

    @d33bs
    d33bs committed Apr 28, 2023
    Configuration menu
    Copy the full SHA
    e8c5b83 View commit details
    Browse the repository at this point in the history

  2. avoid hte serialization issues

    @d33bs
    d33bs committed Apr 28, 2023
    Configuration menu
    Copy the full SHA
    669b503 View commit details
    Browse the repository at this point in the history

  3. gitignore; pre-commit deps updates; linting

    @d33bs
    d33bs committed Apr 28, 2023
    Configuration menu
    Copy the full SHA
    f456162 View commit details
    Browse the repository at this point in the history

  4. further linting

    @d33bs
    d33bs committed Apr 28, 2023
    Configuration menu
    Copy the full SHA
    27c4c5c View commit details
    Browse the repository at this point in the history

  5. test updates

    @d33bs
    d33bs committed Apr 28, 2023
    Configuration menu
    Copy the full SHA
    0f64dcb View commit details
    Browse the repository at this point in the history

  6. further testing revisions

    @d33bs
    d33bs committed Apr 28, 2023
    Configuration menu
    Copy the full SHA
    8feaeea View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2023

  1. s3 test update

    @d33bs
    d33bs committed Apr 29, 2023
    Configuration menu
    Copy the full SHA
    4bd197e View commit details
    Browse the repository at this point in the history

  2. move to join_app

    @d33bs
    d33bs committed Apr 29, 2023
    Configuration menu
    Copy the full SHA
    adfcf91 View commit details
    Browse the repository at this point in the history

Commits on Apr 30, 2023

  1. flatten loop

    @d33bs
    d33bs committed Apr 30, 2023
    Configuration menu
    Copy the full SHA
    a71323d View commit details
    Browse the repository at this point in the history

  2. refactor to nonblocking parsl flow

    @d33bs
    d33bs committed Apr 30, 2023
    Configuration menu
    Copy the full SHA
    d1c5547 View commit details
    Browse the repository at this point in the history

Commits on May 2, 2023

  1. updated tests and exception handling

    @d33bs
    d33bs committed May 2, 2023
    Configuration menu
    Copy the full SHA
    8c8aca6 View commit details
    Browse the repository at this point in the history

  2. linting

    @d33bs
    d33bs committed May 2, 2023
    Configuration menu
    Copy the full SHA
    d32a2c7 View commit details
    Browse the repository at this point in the history

  3. update documentation

    @d33bs
    d33bs committed May 2, 2023
    Configuration menu
    Copy the full SHA
    1ab86c6 View commit details
    Browse the repository at this point in the history

  4. remove unnecessary deps

    @d33bs
    d33bs committed May 2, 2023
    Configuration menu
    Copy the full SHA
    94029f8 View commit details
    Browse the repository at this point in the history

  5. docs; linting; AppBase for sphinx; add max_threads

    @d33bs
    d33bs committed May 2, 2023
    Configuration menu
    Copy the full SHA
    096d8c1 View commit details
    Browse the repository at this point in the history

  6. linting

    @d33bs
    d33bs committed May 2, 2023
    Configuration menu
    Copy the full SHA
    7cde0d2 View commit details
    Browse the repository at this point in the history

Commits on May 3, 2023

  1. readd python 3.8; case sensitive join work 

    move from logging.warn to logging.warning (avoid deprecated calls)
    @d33bs
    d33bs committed May 3, 2023
    Configuration menu
    Copy the full SHA
    9201484 View commit details
    Browse the repository at this point in the history

Commits on May 4, 2023

  1. Apply suggestions from code review 

    Co-authored-by: Gregory Way <gregory.way@gmail.com>
    @d33bs @gwaybio
    d33bs and gwaybio authored May 4, 2023
    Configuration menu
    Copy the full SHA
    851bbda View commit details
    Browse the repository at this point in the history

  2. remove dask section of gitignore

    @d33bs
    d33bs committed May 4, 2023
    Configuration menu
    Copy the full SHA
    7f5c6b7 View commit details
    Browse the repository at this point in the history

  3. update _prepend_column_name docstring 

    Co-Authored-By: Gregory Way <gregory.way@gmail.com>
    @d33bs @gwaybio
    d33bs and gwaybio committed May 4, 2023
    Configuration menu
    Copy the full SHA
    3562067 View commit details
    Browse the repository at this point in the history

  4. fix concat dir cleanup and test

    @d33bs
    d33bs committed May 4, 2023
    Configuration menu
    Copy the full SHA
    8c7af62 View commit details
    Browse the repository at this point in the history

Commits on May 6, 2023

  1. import safety for python_apps

    @d33bs
    d33bs committed May 6, 2023
    Configuration menu
    Copy the full SHA
    7a8955a View commit details
    Browse the repository at this point in the history

Commits on May 7, 2023

  1. further imports for parsl apps non-threaded use 

    Co-Authored-By: Faisal Alquaddoomi <faisal.alquaddoomi@cuanschutz.edu>
    @d33bs @falquaddoomi
    d33bs and falquaddoomi committed May 7, 2023
    Configuration menu
    Copy the full SHA
    86dfe82 View commit details
    Browse the repository at this point in the history

  2. further parsl app imports reconfiguration 

    Co-Authored-By: Faisal Alquaddoomi <faisal.alquaddoomi@cuanschutz.edu>
    @d33bs @falquaddoomi
    d33bs and falquaddoomi committed May 7, 2023
    Configuration menu
    Copy the full SHA
    c461d35 View commit details
    Browse the repository at this point in the history

Commits on May 9, 2023

  1. add mixed type handler for sqlite sources

    @d33bs
    d33bs committed May 9, 2023
    Configuration menu
    Copy the full SHA
    174d164 View commit details
    Browse the repository at this point in the history

  2. add duckdb mixed type exception handling for the chunked query

    @d33bs
    d33bs committed May 9, 2023
    Configuration menu
    Copy the full SHA
    52273ac View commit details
    Browse the repository at this point in the history

  3. add docstring

    @d33bs
    d33bs committed May 9, 2023
    Configuration menu
    Copy the full SHA
    68243c4 View commit details
    Browse the repository at this point in the history

  4. add test for _sqlite_mixed_type_query_to_parquet

    @d33bs
    d33bs committed May 9, 2023
    Configuration menu
    Copy the full SHA
    e6ee34b View commit details
    Browse the repository at this point in the history

Commits on May 15, 2023

  1. Merge remote-tracking branch 'upstream/main' into sqlite-mixed-types-… 

    …handling
    @d33bs
    d33bs committed May 15, 2023
    Configuration menu
    Copy the full SHA
    cf9b4d4 View commit details
    Browse the repository at this point in the history

Commits on May 18, 2023

  1. enhance modularity

    @d33bs
    d33bs committed May 18, 2023
    Configuration menu
    Copy the full SHA
    ab0185e View commit details
    Browse the repository at this point in the history

  2. modularize tests

    @d33bs
    d33bs committed May 18, 2023
    Configuration menu
    Copy the full SHA
    9282113 View commit details
    Browse the repository at this point in the history

  3. testing docs and linting

    @d33bs
    d33bs committed May 18, 2023
    Configuration menu
    Copy the full SHA
    2861f0a View commit details
    Browse the repository at this point in the history

  4. increase test performance with xdist

    @d33bs
    d33bs committed May 18, 2023
    Configuration menu
    Copy the full SHA
    d1e7160 View commit details
    Browse the repository at this point in the history

  5. placeholder and docs for _gather_tablenumber

    @d33bs
    d33bs committed May 18, 2023
    Configuration menu
    Copy the full SHA
    3f2acfb View commit details
    Browse the repository at this point in the history

  6. linting

    @d33bs
    d33bs committed May 18, 2023
    Configuration menu
    Copy the full SHA
    d63d605 View commit details
    Browse the repository at this point in the history

Commits on May 19, 2023

  1. add checksum; pytest xdist defaults

    @d33bs
    d33bs committed May 19, 2023
    Configuration menu
    Copy the full SHA
    323057d View commit details
    Browse the repository at this point in the history

  2. linting

    @d33bs
    d33bs committed May 19, 2023
    Configuration menu
    Copy the full SHA
    ebef287 View commit details
    Browse the repository at this point in the history

  3. add tablenumber to source extraction; adjust tests

    @d33bs
    d33bs committed May 19, 2023
    Configuration menu
    Copy the full SHA
    f4efaef View commit details
    Browse the repository at this point in the history

  4. update docs; slight renaming

    @d33bs
    d33bs committed May 19, 2023
    Configuration menu
    Copy the full SHA
    8138847 View commit details
    Browse the repository at this point in the history

  5. optional preset tablenumber; add a test

    @d33bs
    d33bs committed May 19, 2023
    Configuration menu
    Copy the full SHA
    eea8f04 View commit details
    Browse the repository at this point in the history

  6. cast tablenumber; cytominer-db tablenumber testing

    @d33bs
    d33bs committed May 19, 2023
    Configuration menu
    Copy the full SHA
    190153b View commit details
    Browse the repository at this point in the history

  7. use github images for github tests

    @d33bs
    d33bs committed May 19, 2023
    Configuration menu
    Copy the full SHA
    66cc5f5 View commit details
    Browse the repository at this point in the history

Commits on May 20, 2023

  1. readd xdist

    @d33bs
    d33bs committed May 20, 2023
    Configuration menu
    Copy the full SHA
    25e4d86 View commit details
    Browse the repository at this point in the history

  2. linting

    @d33bs
    d33bs committed May 20, 2023
    Configuration menu
    Copy the full SHA
    fa1b880 View commit details
    Browse the repository at this point in the history

  3. distinct parsl_config load for testing

    @d33bs
    d33bs committed May 20, 2023
    Configuration menu
    Copy the full SHA
    73a9e53 View commit details
    Browse the repository at this point in the history

  4. run single-core pytest for github testing

    @d33bs
    d33bs committed May 20, 2023
    Configuration menu
    Copy the full SHA
    172cad9 View commit details
    Browse the repository at this point in the history