-
Notifications
You must be signed in to change notification settings - Fork 119
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add subqueries, attribute modifiers & interactive mode #27
Commits on May 17, 2017
-
The currently solution requires us to track both the tokenizer's "previous" Token as well as each Token's preceding Token. When we reach an opening paren, we check if it's preceding token is IN, and if so, the next block is read as a Subquery. This is just a string representing another input, so it'll be tokenized, parsed, and evalulated as an individual query. If the subquery includes parens, this method of reading breaks, since we stop reading as soon as we reach a closing paren. So, we'll have to count opening parens and only exit when the count is down to 0.
Configuration menu - View commit details
-
Copy full SHA for 91b7cc1 - Browse repository at this point
Copy the full SHA 91b7cc1View commit details
Commits on May 18, 2017
-
Addresses FIXMEs for parsing parens in subqueries and quotes/backticks.
Configuration menu - View commit details
-
Copy full SHA for cfb7824 - Browse repository at this point
Copy the full SHA cfb7824View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7437a25 - Browse repository at this point
Copy the full SHA 7437a25View commit details -
Fix bug with missing ending identifier in parsed subqueries
Subqueries were missing an identifier; resolved by adding the final "word" to the query string before breaking (in Tokenizer.readQuery).
Configuration menu - View commit details
-
Copy full SHA for 6050e0c - Browse repository at this point
Copy the full SHA 6050e0cView commit details
Commits on May 20, 2017
-
Add prelim. implementation of subqueries; update package structure
Subquery implementation ======================= - Evaluates subqueries as individual queries on parse step. Currently lets you do something like: SELECT all FROM ~/Desktop WHERE name IN ( SELECT name FROM $GOPATH/src WHERE name = %.go) AND size > 2mb Some points for discussion: - No support for SELECTing and comparing more than one attribute in a subquery yet, I'm not really sure _how_ to implement this (or even if it's necessary). - Also implemented boilerplate for superquery/subquery references (as discussed in #4#issuecomment-302104954). Currently runs **terribly** though, not really sure if there's a way to improve performance. Also, haven't added support for substituting directory/file names yet. IN operator (non-subquery) ========================== - Haven't decided if I'm going to keep this. Currently only works for `name`. Accepts a comma-separated list of strings and checks if filename is in list. - Example: SELECT all FROM . WHERE name IN [main.go, query.go] Project layout changes ====================== - Introduce individual packages for tokenizer and parser; - Move compare package to query. Unit tests ========== - Adds tests for tokenizer and compare functions. - Some tokenizer tests are currently failing (reflect.DeepEqual is saying identical strings don't match, might be a type issue (interface{} vs. string)). Bug fixes ========= - Minor: Fix less-than-equal raw text (was >= before, doesn't really change anything, since we're not using the raw text for anything).
Configuration menu - View commit details
-
Copy full SHA for 4aa37d3 - Browse repository at this point
Copy the full SHA 4aa37d3View commit details
Commits on May 22, 2017
-
Configuration menu - View commit details
-
Copy full SHA for f9e8caf - Browse repository at this point
Copy the full SHA f9e8cafView commit details
Commits on May 24, 2017
-
Configuration menu - View commit details
-
Copy full SHA for f5fc17b - Browse repository at this point
Copy the full SHA f5fc17bView commit details -
Configuration menu - View commit details
-
Copy full SHA for adaa2e7 - Browse repository at this point
Copy the full SHA adaa2e7View commit details -
Misc. edits to transformation implementation
- Rename Transformation -> Modifier. - Update PerformTransformations (renamed to ApplyModifiers) to run on all attributes at once. - Each modifier function receives access to the key as well as path and os.FileInfo, so we're able to run our operation based on the key instead of type (i.e. format(time, ...) vs format(size, ...) shouldn't be a problem anymore). Note that nothing here is final. I plan to merge these changes with the updated parser structure, which means we'll a large portion of this will have to be rewritten to fit the new structure.
Configuration menu - View commit details
-
Copy full SHA for 6a3ecf0 - Browse repository at this point
Copy the full SHA 6a3ecf0View commit details -
Use regex to detect exclusions instead of string.Contains (#21)
* Adds excluder interface that can be used to determine file paths to exclude from output * Implements excluder in regexExclude to detect if a filepath should be excluded
Configuration menu - View commit details
-
Copy full SHA for bb9efe2 - Browse repository at this point
Copy the full SHA bb9efe2View commit details -
Further simplify Exlcuder.buildRegex, make test cases more concise
Configuration menu - View commit details
-
Copy full SHA for dc730f9 - Browse repository at this point
Copy the full SHA dc730f9View commit details -
Merge branch 'feature/subquery' into feature/attribute-modifiers
* feature/subquery: Minor clean-up Use regex to detect exclusions instead of string.Contains (#21) Fix failing tests; track tokens with slice instead of linked list Add prelim. implementation of subqueries; update package structure Fix bug with missing ending identifier in parsed subqueries Replace stack implementation with Lane (oleiade/lane) Minor tokenizer refactor Add subquery tokenizer
Configuration menu - View commit details
-
Copy full SHA for e5954f2 - Browse repository at this point
Copy the full SHA e5954f2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 39a87bf - Browse repository at this point
Copy the full SHA 39a87bfView commit details -
Currently supports 'iso' and 'unix' modes.
Configuration menu - View commit details
-
Copy full SHA for 0f882c8 - Browse repository at this point
Copy the full SHA 0f882c8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8bf9820 - Browse repository at this point
Copy the full SHA 8bf9820View commit details
Commits on May 25, 2017
-
Refactor attribute modifier implementation
- Add support for attribute modifiers in WHERE clause. The Condition struct holds any and all attribute modifiers used in that condition. - Offset all "transformation" work to transform package. This package is split into two parts: format and parse. Parse is use to parse input values (i.e. WHERE clause attributes) and format is used to format output values (i.e. SELECT clause attributes). - Currently support modifiers: FORMAT for time (iso/unix), size (kb/mb/gb), UPPER and LOWER (for name), and FULLPATH (also for name). - Note that attribute modifiers are not fully implemented for subqueries (both evaluated and unevaluated).
Configuration menu - View commit details
-
Copy full SHA for 8dc8418 - Browse repository at this point
Copy the full SHA 8dc8418View commit details -
Configuration menu - View commit details
-
Copy full SHA for a2145ac - Browse repository at this point
Copy the full SHA a2145acView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2ee74f8 - Browse repository at this point
Copy the full SHA 2ee74f8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2111283 - Browse repository at this point
Copy the full SHA 2111283View commit details -
Merge pull request #25 from jonesbrianc26/bugfix/clean-file-paths
Use filepath.Clean on user given paths before processing
Configuration menu - View commit details
-
Copy full SHA for 7221e38 - Browse repository at this point
Copy the full SHA 7221e38View commit details -
Add support for modifiers in subqueries and list values
Now supports the following cases: 1) SELECT all FROM . WHERE UPPER(name) IN (SELECT name FROM foo) 2) SELECT all FROM . WHERE LOWER(name) IN [foo, BAR, BaZ] Decided to make the parsers for these as abstract as possible; ended up using reflect _quite_ heavily (mainly for working with "interfaced" slices/maps). From my brief testing, it's working as it should, but this is probably slightly unsafe. Also removed tokenizer.readList in favour of parsing everything related to conditions/attributes in parser.parseNextCond. Note: Still no support for queries with super/sub-relationships (even without attribute modifiers). Going to try to get this squashed next.
Configuration menu - View commit details
-
Copy full SHA for e4686b9 - Browse repository at this point
Copy the full SHA e4686b9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5b56312 - Browse repository at this point
Copy the full SHA 5b56312View commit details
Commits on May 28, 2017
-
Configuration menu - View commit details
-
Copy full SHA for 5535abe - Browse repository at this point
Copy the full SHA 5535abeView commit details
Commits on May 29, 2017
-
Configuration menu - View commit details
-
Copy full SHA for d5faced - Browse repository at this point
Copy the full SHA d5facedView commit details -
Add parser & token unit tests; various structure tweaks
- Rename Condition.Comparator -> Condition.Operator. - Change type of ConditionNode.Type to *TokenType (was TokenType). - Clean directory filepaths when parsing source list instead of during traversal (exclusions are normalized as well now). - Handle aliased exclusions (throws error, since it'd make no sense to give excluded directories an alias). - Also removes a ton of dead code (notably size parsing, which was moved to the transform package).
Configuration menu - View commit details
-
Copy full SHA for d301448 - Browse repository at this point
Copy the full SHA d301448View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4081f98 - Browse repository at this point
Copy the full SHA 4081f98View commit details -
Fix failing build; add lint, vet, and test cmds to README.
Removes `go vet` cmd from Travis pre-script, since was getting a different result locally, will take a look into this.
Configuration menu - View commit details
-
Copy full SHA for 6f4afd7 - Browse repository at this point
Copy the full SHA 6f4afd7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 04c58df - Browse repository at this point
Copy the full SHA 04c58dfView commit details -
Fix bug with chained functions in WHERE clause
Recursive modifiers (e.g. `FULLPATH(UPPER(name))`) weren't being applied since we were passing c.Value each time instead of the previously-altered value (so only the innermost modifier was having an effect).
Configuration menu - View commit details
-
Copy full SHA for 11c095d - Browse repository at this point
Copy the full SHA 11c095dView commit details -
Configuration menu - View commit details
-
Copy full SHA for ff97b71 - Browse repository at this point
Copy the full SHA ff97b71View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0d33a6e - Browse repository at this point
Copy the full SHA 0d33a6eView commit details -
Add interactive mode with channel-based prompt
- Invoked with -interactive. - No arrow key support yet (for moving fwd/back in query, accessing previous queries, etc). Not sure how to go about implementing this right now. - Slight change in structure: flag/input parsing moved to cmd/fsql, while top-level package (fsql) handles running the query and printing output. - Adds new package with channel-based shell prompt. Allows for quote-less AND escape-less queries. End queries with single semicolon.
Configuration menu - View commit details
-
Copy full SHA for d4d767a - Browse repository at this point
Copy the full SHA d4d767aView commit details -
Add support for formatting/parsing custom time layouts
Layout must be set according to 2006-01-02T15:04:05.999999-07:00, some sample queries: $ fsql SELECT all FROM . WHERE format(time, 2006-01-02) > 2017-05-29 $ fsql SELECT name, format(time, "2016-01-02 15:04:05") FROM .
Configuration menu - View commit details
-
Copy full SHA for b4a8d99 - Browse repository at this point
Copy the full SHA b4a8d99View commit details
Commits on Jun 2, 2017
-
Configuration menu - View commit details
-
Copy full SHA for 67a73d7 - Browse repository at this point
Copy the full SHA 67a73d7View commit details -
Configuration menu - View commit details
-
Copy full SHA for e289fc9 - Browse repository at this point
Copy the full SHA e289fc9View commit details
Commits on Jun 4, 2017
-
Configuration menu - View commit details
-
Copy full SHA for 5140c9d - Browse repository at this point
Copy the full SHA 5140c9dView commit details -
Configuration menu - View commit details
-
Copy full SHA for a82df1a - Browse repository at this point
Copy the full SHA a82df1aView commit details -
1
Configuration menu - View commit details
-
Copy full SHA for c1c6a15 - Browse repository at this point
Copy the full SHA c1c6a15View commit details -
Update README to reflect changes
Subqueries, attribute modifiers & interactive mode!
Configuration menu - View commit details
-
Copy full SHA for adc1ed7 - Browse repository at this point
Copy the full SHA adc1ed7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0783806 - Browse repository at this point
Copy the full SHA 0783806View commit details -
* master: Add Homebrew installation instructions Bump version Add note for selecting from dirs that begin with hypen
Configuration menu - View commit details
-
Copy full SHA for 6056684 - Browse repository at this point
Copy the full SHA 6056684View commit details -
- Add Travis badge to README - Organize .gitignore - Bump version to 0.2.0
Configuration menu - View commit details
-
Copy full SHA for 5d60e0e - Browse repository at this point
Copy the full SHA 5d60e0eView commit details