JavaScript in TypeScript compilations

[billti]

JavaScript in TypeScript compilations

(See parent issue #4789 for an overview)

There are a number of considerations for the TypeScript compiler to handle JavaScript code:

• Should the compiler automatically pick up JavaScript files in a folder as it does for TypeScript?
• If so, how to avoid compiling JavaScript files that were emitted by a prior TypeScript compilation?
• Should the compiler emit JavaScript files for provided JavaScript source?
  • This would be required in any "transpile" scenarios, for example emitting ES5 JavaScript from ES6 source
  • In this scenario a new filename or location would be needed to avoid overwriting the input
• Should the compiler use the same compiler options for TypeScript and JavaScript?
  • For example, the target ECMAScript mode, or the JSX flags for handling React/JSX code.
• Does the resolving of module imports crawl the node_modules folder?
  • If so, how deep through dependencies does this go?
  • How can it determine if a module (such as express) already has a .d.ts declaration, and crawling the JavaScript files is unnecessary?
• Is the file extension constrained to .js, or can other file types be provided (e.g. .es)?

JavaScript files for input

There are four ways in which source files may be included in a compilation:

1. A tsconfig.json file exists in a folder that is compiled, and it does not explicitly list files to include. In this case, all source files in this folder and subfolders are crawled for source files to compile.
2. The list of source files is provided explicitly, either via a files property in a tsconfig.json file, or via files listed on the tsc command line.
3. Files are included via /// <reference ...> tags in other source files.
4. Files are included as imported modules for other source files.

Implementation note: Currently, files are parsed and their dependencies included and parsed, before the next input file is parsed. For example, if a compilation includes a.ts and b.ts (in that order), but a.ts imports .\c, then c will be parsed and its dependencies resolved and included before b.ts is processed.

Steps 1 - 3 above are unambiguous as to which file to include in the set to compile, as the file extension is part of the given or discovered file name. For step 4 (module resolution), the file extension is not given, and precedence rules are followed (module resolution will attempt to location .ts, then .d.ts files for the module name. The .js extension would go on the end of this list).

Once the set of source files is gathered, the list of emit files can be calculated. If there is a collision (i.e. src\foo.js is both a source and emit file), then it should be removed from the list of source files if it was found via discovery (i.e. crawling the folder or resolving a module path discovered the file to be emitted). If it was listed explicitly as an input file, yet would be overwritten on emit, then an error should be raised.

Transpiling JavaScript

It would be valuable to allow developers to write the same ES6 constructs they can use in TypeScript (e.g. arrow functions, classes, etc.) and provide the same conversion to ES5/ES3 that is available for TypeScript. As this should be relatively simple to implement, this would be worth adding. Of note is that due to the input/output file conflict outlined above, the emitting files would need to go to a separate folder (--outDir), or be combined into a separate output file (--outFile) in order to avoid attempting to overwrite the source file on emit (else input and output will have the same file path).

Compiler flags

There are two categories of compiler flags to consider:

1. The behavior of existing compiler flags
2. Any new flags that should be added

Existing flags of interest include:

• target: should the compiler/langauge service care about the targeted ECMAScript level (especially if not transpiling)?
• jsx: should the compiler/language service treat JavaScript files written in JSX syntax the same as source .tsx files?
• outDir: should the compiler copy the input JavaScript files to the output directory (especially if not transpiling)?
• declaration: should the compiler emit a .d.ts file for JavaScript files it has parsed and gathered types from?
• module: should the compiler care about module type for .js files?

TODO

• Outline any new flags needed
• Describe any limit on crawling depth and handling node_modules
• If and how to also handle non-js file extensions

[ldminoc]

Here are some opinions:

• the environment should provide syntactic support for ES5 (or lower?) but should not compile. Alternatively provide a compiler flag to ignore (i.e not compile). There are reason to want to NOT re-compile third-party sources.
• definitely support transpilation of es6 files unless an alternative transpiler has been configured
• definitely support jsx, by default or compiler flag
• node modules should be resolved in exactly the same way that node does it, to avoid surprises (e.g. incompatible version being picked up).
• the compiler should not constrain the file extensions but should provide compilation or configuration options to specify extensions to be processed
• tsconfig file should not automatically process all files in its directory and below, but should respect the configured extensions mentioned above. Also consider supporting an 'exclude' configuration, .i.e. exclude files/directories specified by name or pattern/regex
• jsx treated as .tsx, hmm, first thought is probably yes
• copy input js to outDir even if not transpiled - my instinct says no, don't do unnecessary work, also because the environment or third-party tools may have a 'watch' on the directory. I'm thinking of something like webpack watching the directory for bundling; don't want it to do something when nothing has changed.
• explicit list of files to compile is ok for small projects but leads to busy work and increases the potential for errors in large projects; i'm not a fan
• emitting .d.ts should be done on-demand, not automatically unless the compiler is smart enough to differentiate interface changes from body changes; comprise by adding a compiler flag

Side note: I've experience Visual Studio Code erroring on es6 files unless the 'target' compiler option was set to es6. It should be using a 'source' specifier; target is what we want to transpile to.

[mhegazy]

This should be available now with #5471 using --allowJs

[amcdnl]

Any usage docs?

[RobCannon]

I think the support of globs in tsconfig.json file is really necessary for this feature to be useful. How do I keep typescript from pulling in a lot of devtime dependencies (like gulp modules)?

[mhegazy]

specify "exclude" property in your tsconfig.