GSoC 2021 Final Report: Add synthetics and symbol information for semanticdb in Scala 3

Rikito Taniguchi (rikiriki1238@gmail.com)

Overview

SemanticDB is a data model for semantic information such as symbols and types in Scala programs. It is widely used for developing Scala's devtools such as scalafix and Metals. However, the SemanticDB extractor for Scala3 was a work in progress, and some features in devtools were unavailable for Scala3.

In this project, I worked on extracting more information for SemanticDB from the Scala3 compiler to improve the developer experience of Scala3. My contributions consists of mainly five parts.

• Generate Scala3 code for SemanticDB data models
• Support Signature information
• Support Synthetic information
• Support some missing fields of SymbolInformation and new Scala3 modifiers
• Some other contributions

Work

(1) Generate Scala3 code for SemanticDB data models (completed)

• Discussion scalameta/scalameta#2367
• New project https://github.com/tanishiking/semanticdb-for-scala3
• PR to lampepfl/dotty scala/scala3#12780

The first thing I did was automating the process of generating Scala3 code from the protobuf scheme of SemanticDB.

Previously, the code for SemanticDB data models in Scala3 was copy-and-pasted and adjusted by hand to remove dependencies to ScalaPB and protobuf library (see the discussion). It was a plausible way of generating Scala3 code when all the information we extracted for SemanticDB was limited. However, once we start extracting more information for SemanticDB, we need more classes to generate, and hand-crafting all the code and maintaining them was unrealistic.

So I proposed some ways to improve the code generation process, and we decided to generate Scala code using ScalaPB and automatically adjust them for Scala3 compiler using scalafix. See the discussion.

Here's a project to generate and adjust Scala code, and generated code had been merged into dotty.

(2) Support Signature information (completed)

• Issue scala/scala3#12963
• PR scala/scala3#12885 (MERGED)

This is the main contribution of this project.

Signature in SemanticDB is a data structure that represents definition signatures (type information) for class, method, type, and local value. This information is consumed by, for example, metals to show inferred types as decoration, and scalafix rules that utilize type information. Previously, Scala3 doesn't extract this information, and Metals couldn't show the inferred type for Scala3 programs.

In this project, I extended Scala3 to extract signature information. The PR involves

• Convert definition types of Scala3 into SemanticDB Types.
• Convert Signature information of Scala3 to SemanticDB Signature
• Pretty Print Signature information for regression testing.

The most tricky part of this work is converting complicated types such as TypeParamRef and TermParamRef, and RefinedType. In SemanticDB, they should be represented as TypeRef, TermRef, and StructuralType respectively, and they should contain the references to the symbols. However, we can't retrieve the symbol information directly from those types. Therefore we work-around it by

• Traverse the children nodes first, construct symbol tables, and then lookup the symbol for TypeParamRef, TermParamRef, and RefinedType.
• If the symbols are unavailable from the symbol tables, create a dummy symbol and convert it to a SemanticDB symbol.

(3) Support Synthetic infromation (not yet merged)

• Issue scala/scala3#13135
• PR
  • scala/scala3#13288
  • tanishiking/scala3#5 (WIP)

Synthetic is information that represents trees synthesized by the compiler. For example, Metals uses this information to show implicit parameters, show inferred type applications, and find references.

First, I summarized what kind of synthetic information we should extract because Scala3 has different syntax and semantics than Scala2. During the coding period, I submit PRs to support several synthetics, but they are not yet merged.

(4) Support some missing fields of SymbolInformation and new Scala3 modifiers

• Issue scala/scala3#12963
• PR
  • Access information scala/scala3#12964
  • Overridden Symbols scala/scala3#13295
  • Support Scala3 modifiers
    • scalameta/scalameta#2439
    • scala/scala3#13239

While working on this project, I realized some useful information is missing from SymbolInformation in Scala3. So I worked on it. Also, supported new Scala3 modifiers such as inline, transparent, and given.

(5) Other contributions

• Support missing symbol occurrences in type annotations.
  • scala/scala3#13284
• Related to overriddenSymbols, I refactored Metals' goto parent code lense feature using overiddenSybol
  • scalameta/metals#3045
• Aside from extracting semantic information from Scala3, I started working on a personal project to detect unused symbols in Scala3 using SemanticDB generated by Scala3 (which is implemented by this project).
  • https://github.com/tanishiking/scalafix-unused

Links to all the contributions during GSoC

• lampepfl/dotty
• scalameta/scalameta
• scalameta/metals
• New repositories
  • tanishiking/semanticdb-for-scala3
  • tanishiking/scalafix-unused

Future Work

• Support MatchType and TypeLambda
  • While most of the types in Scala3 are covered, some new Scala3 types such as TypeLambdas and MatchType are not yet supported because it requires extending the SemanticDB schema.
  • Here's a WIP PR to support those types in SemanticDB scalameta/scalameta#2414
• Extend Signatures to have multiple type parameter clauses
• Support nested annotations in SemanticDB signatures
• Support missing fields in SymbolInformation
  • annotations and document fields are missing at this moment.