Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add subqueries, attribute modifiers & interactive mode #27

Merged
merged 42 commits into from
Jun 4, 2017
Merged

Commits on May 17, 2017

  1. Add subquery tokenizer

    The currently solution requires us to track both the tokenizer's "previous"
    Token as well as each Token's preceding Token.
    
    When we reach an opening paren, we check if it's preceding token is IN, and if
    so, the next block is read as a Subquery.
    This is just a string representing another input, so it'll be tokenized,
    parsed, and evalulated as an individual query.
    
    If the subquery includes parens, this method of reading breaks, since we stop
    reading as soon as we reach a closing paren. So, we'll have to count opening
    parens and only exit when the count is down to 0.
    kashav committed May 17, 2017
    Configuration menu
    Copy the full SHA
    91b7cc1 View commit details
    Browse the repository at this point in the history

Commits on May 18, 2017

  1. Minor tokenizer refactor

    Addresses FIXMEs for parsing parens in subqueries and quotes/backticks.
    kashav committed May 18, 2017
    Configuration menu
    Copy the full SHA
    cfb7824 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7437a25 View commit details
    Browse the repository at this point in the history
  3. Fix bug with missing ending identifier in parsed subqueries

    Subqueries were missing an identifier; resolved by adding the final "word"
    to the query string before breaking (in Tokenizer.readQuery).
    kashav committed May 18, 2017
    Configuration menu
    Copy the full SHA
    6050e0c View commit details
    Browse the repository at this point in the history

Commits on May 20, 2017

  1. Add prelim. implementation of subqueries; update package structure

    Subquery implementation
    =======================
    
      - Evaluates subqueries as individual queries on parse step. Currently
        lets you do something like:
    
        SELECT
          all
        FROM
          ~/Desktop
        WHERE
          name IN (
            SELECT
              name
            FROM
              $GOPATH/src
            WHERE
              name = %.go) AND
          size > 2mb
    
        Some points for discussion:
    
          - No support for SELECTing and comparing more than one attribute in a
            subquery yet, I'm not really sure _how_ to implement this (or even
            if it's necessary).
    
      - Also implemented boilerplate for superquery/subquery references (as
        discussed in #4#issuecomment-302104954). Currently runs **terribly**
        though, not really sure if there's a way to improve performance. Also,
        haven't added support for substituting directory/file names yet.
    
    IN operator (non-subquery)
    ==========================
    
      - Haven't decided if I'm going to keep this. Currently only works for `name`.
        Accepts a comma-separated list of strings and checks if filename is in list.
    
      - Example:
    
        SELECT
          all
        FROM
          .
        WHERE
          name IN [main.go, query.go]
    
    Project layout changes
    ======================
    
      - Introduce individual packages for tokenizer and parser;
      - Move compare package to query.
    
    Unit tests
    ==========
    
      - Adds tests for tokenizer and compare functions.
      - Some tokenizer tests are currently failing (reflect.DeepEqual is saying
        identical strings don't match, might be a type issue (interface{} vs.
        string)).
    
    Bug fixes
    =========
    
      - Minor: Fix less-than-equal raw text (was >= before, doesn't really change
        anything, since we're not using the raw text for anything).
    kashav committed May 20, 2017
    Configuration menu
    Copy the full SHA
    4aa37d3 View commit details
    Browse the repository at this point in the history

Commits on May 22, 2017

  1. Configuration menu
    Copy the full SHA
    f9e8caf View commit details
    Browse the repository at this point in the history

Commits on May 24, 2017

  1. Configuration menu
    Copy the full SHA
    f5fc17b View commit details
    Browse the repository at this point in the history
  2. gofmt

    kashav committed May 24, 2017
    Configuration menu
    Copy the full SHA
    adaa2e7 View commit details
    Browse the repository at this point in the history
  3. Misc. edits to transformation implementation

    - Rename Transformation -> Modifier.
    - Update PerformTransformations (renamed to ApplyModifiers) to run on all
      attributes at once.
    - Each modifier function receives access to the key as well as path and
      os.FileInfo, so we're able to run our operation based on the key instead of
      type (i.e. format(time, ...) vs format(size, ...) shouldn't be a problem
      anymore).
    
    Note that nothing here is final. I plan to merge these changes with the updated
    parser structure, which means we'll a large portion of this will have to be
    rewritten to fit the new structure.
    kashav committed May 24, 2017
    Configuration menu
    Copy the full SHA
    6a3ecf0 View commit details
    Browse the repository at this point in the history
  4. Use regex to detect exclusions instead of string.Contains (#21)

    * Adds excluder interface that can be used to determine file paths to exclude from output
    * Implements excluder in regexExclude to detect if a filepath should be excluded
    beeceej authored and kashav committed May 24, 2017
    Configuration menu
    Copy the full SHA
    bb9efe2 View commit details
    Browse the repository at this point in the history
  5. Minor clean-up

    Further simplify Exlcuder.buildRegex, make test cases more concise
    kashav committed May 24, 2017
    Configuration menu
    Copy the full SHA
    dc730f9 View commit details
    Browse the repository at this point in the history
  6. Merge branch 'feature/subquery' into feature/attribute-modifiers

    * feature/subquery:
      Minor clean-up
      Use regex to detect exclusions instead of string.Contains (#21)
      Fix failing tests; track tokens with slice instead of linked list
      Add prelim. implementation of subqueries; update package structure
      Fix bug with missing ending identifier in parsed subqueries
      Replace stack implementation with Lane (oleiade/lane)
      Minor tokenizer refactor
      Add subquery tokenizer
    kashav committed May 24, 2017
    Configuration menu
    Copy the full SHA
    e5954f2 View commit details
    Browse the repository at this point in the history
  7. Remove print statements

    kashav committed May 24, 2017
    Configuration menu
    Copy the full SHA
    39a87bf View commit details
    Browse the repository at this point in the history
  8. Add format function for time

    Currently supports 'iso' and 'unix' modes.
    kashav committed May 24, 2017
    Configuration menu
    Copy the full SHA
    0f882c8 View commit details
    Browse the repository at this point in the history
  9. Support uppercase modifiers

    kashav committed May 24, 2017
    Configuration menu
    Copy the full SHA
    8bf9820 View commit details
    Browse the repository at this point in the history

Commits on May 25, 2017

  1. Refactor attribute modifier implementation

    - Add support for attribute modifiers in WHERE clause. The Condition struct
      holds any and all attribute modifiers used in that condition.
    - Offset all "transformation" work to transform package. This package is split
      into two parts: format and parse. Parse is use to parse input values (i.e.
      WHERE clause attributes) and format is used to format output values (i.e.
      SELECT clause attributes).
    - Currently support modifiers: FORMAT for time (iso/unix), size (kb/mb/gb),
      UPPER and LOWER (for name), and FULLPATH (also for name).
    - Note that attribute modifiers are not fully implemented for subqueries (both
      evaluated and unevaluated).
    kashav committed May 25, 2017
    Configuration menu
    Copy the full SHA
    8dc8418 View commit details
    Browse the repository at this point in the history
  2. Add .travis.yml

    kashav committed May 25, 2017
    Configuration menu
    Copy the full SHA
    a2145ac View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    2ee74f8 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    2111283 View commit details
    Browse the repository at this point in the history
  5. Merge pull request #25 from jonesbrianc26/bugfix/clean-file-paths

    Use filepath.Clean on user given paths before processing
    kashav authored May 25, 2017
    Configuration menu
    Copy the full SHA
    7221e38 View commit details
    Browse the repository at this point in the history
  6. Add support for modifiers in subqueries and list values

    Now supports the following cases:
    
      1) SELECT all FROM . WHERE UPPER(name) IN (SELECT name FROM foo)
      2) SELECT all FROM . WHERE LOWER(name) IN [foo, BAR, BaZ]
    
    Decided to make the parsers for these as abstract as possible; ended up using
    reflect _quite_ heavily (mainly for working with "interfaced" slices/maps).
    From my brief testing, it's working as it should, but this is probably
    slightly unsafe.
    
    Also removed tokenizer.readList in favour of parsing everything related
    to conditions/attributes in parser.parseNextCond.
    
    Note: Still no support for queries with super/sub-relationships (even without
    attribute modifiers). Going to try to get this squashed next.
    kashav committed May 25, 2017
    Configuration menu
    Copy the full SHA
    e4686b9 View commit details
    Browse the repository at this point in the history
  7. Add boilerplate for ! token

    kashav committed May 25, 2017
    Configuration menu
    Copy the full SHA
    5b56312 View commit details
    Browse the repository at this point in the history

Commits on May 28, 2017

  1. Configuration menu
    Copy the full SHA
    5535abe View commit details
    Browse the repository at this point in the history

Commits on May 29, 2017

  1. Run tests with verbose flag

    kashav committed May 29, 2017
    Configuration menu
    Copy the full SHA
    d5faced View commit details
    Browse the repository at this point in the history
  2. Add parser & token unit tests; various structure tweaks

    - Rename Condition.Comparator -> Condition.Operator.
    - Change type of ConditionNode.Type to *TokenType (was TokenType).
    - Clean directory filepaths when parsing source list instead of during
      traversal (exclusions are normalized as well now).
    - Handle aliased exclusions (throws error, since it'd make no sense to give
      excluded directories an alias).
    - Also removes a ton of dead code (notably size parsing, which was moved to
      the transform package).
    kashav committed May 29, 2017
    Configuration menu
    Copy the full SHA
    d301448 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    4081f98 View commit details
    Browse the repository at this point in the history
  4. Fix failing build; add lint, vet, and test cmds to README.

    Removes `go vet` cmd from Travis pre-script, since was getting a different
    result locally, will take a look into this.
    kashav committed May 29, 2017
    Configuration menu
    Copy the full SHA
    6f4afd7 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    04c58df View commit details
    Browse the repository at this point in the history
  6. Fix bug with chained functions in WHERE clause

    Recursive modifiers (e.g. `FULLPATH(UPPER(name))`) weren't being applied since
    we were passing c.Value each time instead of the previously-altered value (so
    only the innermost modifier was having an effect).
    kashav committed May 29, 2017
    Configuration menu
    Copy the full SHA
    11c095d View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    ff97b71 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    0d33a6e View commit details
    Browse the repository at this point in the history
  9. Add interactive mode with channel-based prompt

    - Invoked with -interactive.
    - No arrow key support yet (for moving fwd/back in query, accessing previous
      queries, etc). Not sure how to go about implementing this right now.
    - Slight change in structure: flag/input parsing moved to cmd/fsql, while
      top-level package (fsql) handles running the query and printing output.
    - Adds new package with channel-based shell prompt. Allows for quote-less
      AND escape-less queries. End queries with single semicolon.
    kashav committed May 29, 2017
    Configuration menu
    Copy the full SHA
    d4d767a View commit details
    Browse the repository at this point in the history
  10. Add support for formatting/parsing custom time layouts

    Layout must be set according to 2006-01-02T15:04:05.999999-07:00, some sample
    queries:
    
      $ fsql SELECT all FROM . WHERE format(time, 2006-01-02) > 2017-05-29
      $ fsql SELECT name, format(time, "2016-01-02 15:04:05") FROM .
    kashav committed May 29, 2017
    Configuration menu
    Copy the full SHA
    b4a8d99 View commit details
    Browse the repository at this point in the history

Commits on Jun 2, 2017

  1. Add *.go to sources wildcard

    kashav committed Jun 2, 2017
    Configuration menu
    Copy the full SHA
    67a73d7 View commit details
    Browse the repository at this point in the history
  2. Minor clean-up

    kashav committed Jun 2, 2017
    Configuration menu
    Copy the full SHA
    e289fc9 View commit details
    Browse the repository at this point in the history

Commits on Jun 4, 2017

  1. Use new path in build step

    kashav committed Jun 4, 2017
    Configuration menu
    Copy the full SHA
    5140c9d View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a82df1a View commit details
    Browse the repository at this point in the history
  3. Use buffer to read query

    kashav committed Jun 4, 2017
    1 Configuration menu
    Copy the full SHA
    c1c6a15 View commit details
    Browse the repository at this point in the history
  4. Update README to reflect changes

    Subqueries, attribute modifiers & interactive mode!
    kashav committed Jun 4, 2017
    Configuration menu
    Copy the full SHA
    adc1ed7 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    0783806 View commit details
    Browse the repository at this point in the history
  6. Resolve merge conflicts

    * master:
      Add Homebrew installation instructions
      Bump version
      Add note for selecting from dirs that begin with hypen
    kashav committed Jun 4, 2017
    Configuration menu
    Copy the full SHA
    6056684 View commit details
    Browse the repository at this point in the history
  7. Various tweaks

    - Add Travis badge to README
    - Organize .gitignore
    - Bump version to 0.2.0
    kashav committed Jun 4, 2017
    Configuration menu
    Copy the full SHA
    5d60e0e View commit details
    Browse the repository at this point in the history