Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scala support #399

Merged
merged 31 commits into from
Jan 8, 2022
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
c329560
scala support: initial commit
SCdF Dec 17, 2021
26ac29c
className should work here
SCdF Dec 17, 2021
08b602d
class, className, ifStatement, string, comment
SCdF Dec 17, 2021
f24eb6c
Merge branch 'main' into scala
SCdF Dec 17, 2021
d83fcdf
very basic list and call support
SCdF Dec 20, 2021
4d580a8
class changes to support traits and test case classes
SCdF Dec 20, 2021
04ae183
(some) lambdas, maps, interpolated strings
SCdF Dec 21, 2021
7b9d03e
argument and parameter support
SCdF Dec 21, 2021
c3bf94a
names, functions and function names
SCdF Dec 21, 2021
22b9d50
basic types (no generic specific work yet)
SCdF Dec 21, 2021
2ae25db
basic value support
SCdF Dec 21, 2021
8f032d5
conditions
SCdF Dec 21, 2021
c738eee
cleaning up temp work
SCdF Dec 21, 2021
6c99cca
doc changes
SCdF Dec 21, 2021
54b59fa
adding fn names
SCdF Dec 21, 2021
12e69bf
Merge remote-tracking branch 'upstream/main' into scala
SCdF Dec 21, 2021
b56c559
Fix scala language id missing
pokey Dec 23, 2021
0bfcfc1
Merge branch 'main' into scala
pokey Dec 23, 2021
5f6f9ce
more matching values tests
SCdF Jan 6, 2022
065d064
More condition tests
SCdF Jan 6, 2022
f4b8b8c
Dropping list and map, improving types
SCdF Jan 6, 2022
db2f0b6
paired delimiters
SCdF Jan 6, 2022
4fa8085
Dropping comment about partial functions
SCdF Jan 6, 2022
f47a418
cleaning up types
SCdF Jan 7, 2022
3dee158
More type tests
SCdF Jan 7, 2022
491dff6
Fixing up types
SCdF Jan 8, 2022
7df7ac5
cleaned up notes
SCdF Jan 8, 2022
f35bfb8
Merge branch 'main' into scala
pokey Jan 8, 2022
8ac2acb
clarifying our type matching strategy
SCdF Jan 8, 2022
f83f8ea
Merge branch 'scala' of https://github.com/SCdF/cursorless-vscode int…
SCdF Jan 8, 2022
1f53671
Merge branch 'main' into scala
pokey Jan 8, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions data/playground/scala.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
case class Point(x: Int, y: Int)

trait Greeter {
def greet(name: String): Unit
}

class TypesAhoy {
def ++[B >: A](suffix: IterableOnce[B]): ArrayBuffer[B] = ???
val foo: String = "foo"
val bar: List[String] = List("123")
bar.map((c: Char) => c.toDigit)
}

class ExampleClass() {
def realFunction() {
// Lists
val foo: Seq[Int] = List(1,2,3)
val fancyList = 1 :: (2 :: (3 :: Nil))
foo == fancyList

// Maps
var foo: Map[String, String] = Map("red" -> "#FF0000", "azure" -> "#F0FFFF")
foo + = ('I' -> 1)
foo + = ('J' -> 5)
foo + = ('K' -> 10)
foo + = ('L' -> 100)

// lambdas
foo.map(a => a + 1)
foo.map((a: Int) => a + 1)
foo.map(a => {
// so many lines!
a + 1
})

// partial functions (are they lambdas?)
foo.map(_ + 1)
foo.map({case x => x + 1})

// calls with partial functions
foo.map(_ + 1) // works
foo.map {_ + 1}
foo map(_ + 1)
foo map {_ + 1}
foo.size
}

val xml = <p>Hello</p>

realFunction(1,2,3)
}
9 changes: 7 additions & 2 deletions docs/contributing/adding-a-new-language.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,20 @@ for how to add support for a new parser

## 2. Define parse tree patterns in Cursorless

Minimum changes that each language needs:

- new file in `/src/languages/<yourlanguage>.ts`. Take a look at [existing languages](../../src/languages) as a base. At its core you're implementing your language's version of the `nodeMatchers` const, mapping scope types found in [`Types.ts:ScopeType`](../../src/typings/Types.ts) with matching expressions that align with the parse tree output.
- new entry in [`getNodeMatcher.ts:languageMatchers`](../../src/languages/getNodeMatcher.ts), importing your new file above
- new entry in [`constants.ts`](../../src/languages/constants.ts)
- new text fragment extractor (default is likely fine) in [`getTextFragmentExtractor.ts:textFragmentExtractors`](../../src/languages/getTextFragmentExtractor.ts)

SCdF marked this conversation as resolved.
Show resolved Hide resolved
The parse trees exposed by tree-sitter are often pretty close to what we're
looking for, but we often need to look for specific patterns within the parse
tree to get the scopes that the user expects. Fortunately, we have a
domain-specific language that makes these definitions fairly compact.

- Check out the [docs](parse-tree-patterns.md) for the syntax tree pattern
matcher
- You may also find it helpful to look at an existing language, such as
[java](../../src/languages/java.ts).
- If you look in the debug console, you'll see debug output every time you move
your cursor, which might be helpful.
- You will likely want to look at `node-types.json` for your language, (eg [java](https://github.com/tree-sitter/tree-sitter-java/blob/master/src/node-types.json)). This file is generated from `grammar.js`, which might also be helpful to look at (eg [java](https://github.com/tree-sitter/tree-sitter-java/blob/master/grammar.js)).
Expand Down
1 change: 1 addition & 0 deletions src/languages/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ export const supportedLanguageIds = [
"json",
"jsonc",
"python",
"scala",
"typescript",
"typescriptreact",
] as const;
Expand Down
2 changes: 2 additions & 0 deletions src/languages/getNodeMatcher.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import { patternMatchers as typescript } from "./typescript";
import java from "./java";
import { patternMatchers as html } from "./html";
import python from "./python";
import scala from "./scala";
import go from "./go";
import { UnsupportedLanguageError } from "../errors";
import { SupportedLanguageId } from "./constants";
Expand Down Expand Up @@ -59,6 +60,7 @@ const languageMatchers: Record<
json,
jsonc: json,
python,
scala,
typescript,
typescriptreact: typescript,
};
Expand Down
1 change: 1 addition & 0 deletions src/languages/getTextFragmentExtractor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ const textFragmentExtractors: Record<
jsonStringTextFragmentExtractor
),
python: constructDefaultTextFragmentExtractor("python"),
scala: constructDefaultTextFragmentExtractor("scala"),
typescript: constructDefaultTextFragmentExtractor(
"typescript",
typescriptStringTextFragmentExtractor
Expand Down
64 changes: 64 additions & 0 deletions src/languages/scala.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
import {
createPatternMatchers,
argumentMatcher,
leadingMatcher,
conditionMatcher,
trailingMatcher,
cascadingMatcher,
} from '../util/nodeMatchers';
import { NodeMatcherAlternative, ScopeType } from '../typings/Types';

const nodeMatchers: Partial<Record<ScopeType, NodeMatcherAlternative>> = {
// treating classes = classlike
class: ['class_definition', 'object_definition', 'trait_definition'],
className: ['class_definition[name]', 'object_definition[name]', 'trait_definition[name]'],

ifStatement: 'if_expression',

string: ['interpolated_string_expression', 'string'],
comment: 'comment',

// lists basic definition is just a function call to a constructor, eg List(1,2,3,4)
// MISSING: fancy list style: val foo = 1 :: (2 :: (3 :: Nil)) // List(1,2,3)
list: 'call_expression',
map: 'call_expression',
SCdF marked this conversation as resolved.
Show resolved Hide resolved

// list.size(), does not count foo.size (field_expression), or foo size (postfix_expression)
functionCall: 'call_expression',
namedFunction: 'function_definition',
// Do we want to consider partial functions as lambdas? eg
SCdF marked this conversation as resolved.
Show resolved Hide resolved
// foo.map(_ + 1)
// foo.map {_ + 1}
// foo.map {case x => x + 1}
// are not under `lambda_expression`
anonymousFunction: 'lambda_expression',

argumentOrParameter: argumentMatcher('arguments', 'parameters', 'class_parameters', 'bindings'),

name: ['*[name]', '*[pattern]'],
pokey marked this conversation as resolved.
Show resolved Hide resolved
functionName: 'function_definition[name]',

// MISSING: function/class/trait generics
type: leadingMatcher([
'generic_type.type_identifier',
'type_identifier',
'type_parameters',
SCdF marked this conversation as resolved.
Show resolved Hide resolved
], [':']),
value: leadingMatcher(['*[value]', '*[default_value]'], ['=']),
SCdF marked this conversation as resolved.
Show resolved Hide resolved
condition: conditionMatcher('*[condition]'),

// Pulled from the complete list that isn't implemented above

// collectionItem: "???"
// collectionKey: "???",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't you want to support "AL" -> "Alabama" expressions? For these, "key" would be "AL", "value" would be "Alabama", and "item" would be the pair.

collectionItem is also supposed to include list elements, tho since those are just function calls you would basically use an argument matcher and check that the parent is a List. For fancy stuff like this, you might check out the Clojure implementation. That one took tons of custom matchers to get working

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd have to write something custom for that: ("AL" -> "Alabama") comes out as:

          arguments: (arguments [1, 19] - [1, 38]
            (infix_expression [1, 20] - [1, 37]
              left: (string [1, 20] - [1, 24])
              operator: (operator_identifier [1, 25] - [1, 27])
              right: (string [1, 28] - [1, 37]))))))))

So I'd need to take the left or right of an operator, when that operator === '->'

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that looks like it should be doable, but a bit custom. I think I'd be ok with shipping without it for now but filing an issue to capture all the missing things. I believe cursorless will generate a helpful message so that users can just see it hasn't been implemented yet

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw I'd capture your example here into the follow-up issue for remaining scala features; it's quite helpful


// regularExpression: "???",

// attribute: "???",
// xmlElement: "???",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't scala support inline html elements?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does (well, XML), but it's deprecated and will be dropped in 3.x, to be replaced with string formatting (I think, I don't know much about 3.x). I'm not sure how much people use it. The tree sitter implemention also errors with it.

// xmlStartTag: "???",
// xmlEndTag: "???",
// xmlBothTags: "???",
};

export default createPatternMatchers(nodeMatchers);
6 changes: 5 additions & 1 deletion src/test/runTest.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,11 @@ import {
downloadAndUnzipVSCode,
} from "vscode-test";

const extensionDependencies = ["pokey.parse-tree", "ms-toolsai.jupyter"];
const extensionDependencies = [
"pokey.parse-tree",
"ms-toolsai.jupyter",
"scalameta.metals",
SCdF marked this conversation as resolved.
Show resolved Hide resolved
];

async function main() {
try {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
languageId: scala
command:
version: 1
spokenForm: chuck arg fine
action: remove
targets:
- type: primitive
modifier: {type: containingScope, scopeType: argumentOrParameter, includeSiblings: false}
mark: {type: decoratedSymbol, symbolColor: default, character: f}
initialState:
documentContents: |
class ExampleClass(foo: Int, bar: Int) {}
selections:
- anchor: {line: 0, character: 0}
active: {line: 0, character: 0}
marks:
default.f:
start: {line: 0, character: 19}
end: {line: 0, character: 22}
finalState:
documentContents: |
class ExampleClass(bar: Int) {}
selections:
- anchor: {line: 0, character: 0}
active: {line: 0, character: 0}
thatMark:
- anchor: {line: 0, character: 19}
active: {line: 0, character: 19}
fullTargets: [{type: primitive, mark: {type: decoratedSymbol, symbolColor: default, character: f}, selectionType: token, position: contents, insideOutsideType: outside, modifier: {type: containingScope, scopeType: argumentOrParameter, includeSiblings: false}, isImplicit: false}]
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
languageId: scala
command:
version: 1
spokenForm: chuck arg fine
action: remove
targets:
- type: primitive
modifier: {type: containingScope, scopeType: argumentOrParameter, includeSiblings: false}
mark: {type: decoratedSymbol, symbolColor: default, character: f}
initialState:
documentContents: |
class ExampleClass() {
def example(foo: Int, bar: Int) = foo + bar
}
selections:
- anchor: {line: 0, character: 0}
active: {line: 0, character: 0}
marks:
default.f:
start: {line: 1, character: 14}
end: {line: 1, character: 17}
finalState:
documentContents: |
class ExampleClass() {
def example(bar: Int) = foo + bar
}
selections:
- anchor: {line: 0, character: 0}
active: {line: 0, character: 0}
thatMark:
- anchor: {line: 1, character: 14}
active: {line: 1, character: 14}
fullTargets: [{type: primitive, mark: {type: decoratedSymbol, symbolColor: default, character: f}, selectionType: token, position: contents, insideOutsideType: outside, modifier: {type: containingScope, scopeType: argumentOrParameter, includeSiblings: false}, isImplicit: false}]
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
languageId: scala
command:
version: 1
spokenForm: chuck arg fine
action: remove
targets:
- type: primitive
modifier: {type: containingScope, scopeType: argumentOrParameter, includeSiblings: false}
mark: {type: decoratedSymbol, symbolColor: default, character: f}
initialState:
documentContents: |
class ExampleClass() {
example(foo, bar)
}
selections:
- anchor: {line: 0, character: 0}
active: {line: 0, character: 0}
marks:
default.f:
start: {line: 1, character: 10}
end: {line: 1, character: 13}
finalState:
documentContents: |
class ExampleClass() {
example(bar)
}
selections:
- anchor: {line: 0, character: 0}
active: {line: 0, character: 0}
thatMark:
- anchor: {line: 1, character: 10}
active: {line: 1, character: 10}
fullTargets: [{type: primitive, mark: {type: decoratedSymbol, symbolColor: default, character: f}, selectionType: token, position: contents, insideOutsideType: outside, modifier: {type: containingScope, scopeType: argumentOrParameter, includeSiblings: false}, isImplicit: false}]
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
languageId: scala
SCdF marked this conversation as resolved.
Show resolved Hide resolved
command:
version: 1
spokenForm: chuck type look
action: remove
targets:
- type: primitive
modifier: {type: containingScope, scopeType: type, includeSiblings: false}
mark: {type: decoratedSymbol, symbolColor: default, character: l}
initialState:
documentContents: |
class TypesAhoy(foo: List[String]) {}
selections:
- anchor: {line: 0, character: 0}
active: {line: 0, character: 0}
marks:
default.l:
start: {line: 0, character: 21}
end: {line: 0, character: 25}
finalState:
documentContents: |
class TypesAhoy(foo) {}
selections:
- anchor: {line: 0, character: 0}
active: {line: 0, character: 0}
thatMark:
- anchor: {line: 0, character: 19}
active: {line: 0, character: 19}
fullTargets: [{type: primitive, mark: {type: decoratedSymbol, symbolColor: default, character: l}, selectionType: token, position: contents, insideOutsideType: outside, modifier: {type: containingScope, scopeType: type, includeSiblings: false}, isImplicit: false}]
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
languageId: scala
command:
version: 1
spokenForm: chuck type look
action: remove
targets:
- type: primitive
modifier: {type: containingScope, scopeType: type, includeSiblings: false}
mark: {type: decoratedSymbol, symbolColor: default, character: l}
initialState:
documentContents: |
class TypesAhoy() {
def example(foo: List[String]): List[String] = foo
}
selections:
- anchor: {line: 0, character: 0}
active: {line: 0, character: 0}
marks:
default.l:
start: {line: 1, character: 19}
end: {line: 1, character: 23}
finalState:
documentContents: |
class TypesAhoy() {
def example(foo): List[String] = foo
}
selections:
- anchor: {line: 0, character: 0}
active: {line: 0, character: 0}
thatMark:
- anchor: {line: 1, character: 17}
active: {line: 1, character: 17}
fullTargets: [{type: primitive, mark: {type: decoratedSymbol, symbolColor: default, character: l}, selectionType: token, position: contents, insideOutsideType: outside, modifier: {type: containingScope, scopeType: type, includeSiblings: false}, isImplicit: false}]
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
languageId: scala
command:
version: 1
spokenForm: chuck type look
action: remove
targets:
- type: primitive
modifier: {type: containingScope, scopeType: type, includeSiblings: false}
mark: {type: decoratedSymbol, symbolColor: default, character: l}
initialState:
documentContents: |
class TypesAhoy() {
val foo: List[String] = List("foo")
}
selections:
- anchor: {line: 0, character: 0}
active: {line: 0, character: 0}
marks:
default.l:
start: {line: 1, character: 11}
end: {line: 1, character: 15}
finalState:
documentContents: |
class TypesAhoy() {
val foo = List("foo")
}
selections:
- anchor: {line: 0, character: 0}
active: {line: 0, character: 0}
thatMark:
- anchor: {line: 1, character: 9}
active: {line: 1, character: 9}
fullTargets: [{type: primitive, mark: {type: decoratedSymbol, symbolColor: default, character: l}, selectionType: token, position: contents, insideOutsideType: outside, modifier: {type: containingScope, scopeType: type, includeSiblings: false}, isImplicit: false}]
Loading