Add support for F#-style Pipeline Operator #38305

Pokute · 2020-05-03T02:09:57Z

PR adding pipeline operator support to TypeScript. The proposal is currently a stage 1 proposal for EcmaScript. It "introduces a new operator |> similar to F#, OCaml, Elixir, Elm, Julia, Hack, and LiveScript, as well as UNIX pipes and Haskell's &. It's a backwards-compatible way of streamlining chained function calls in a readable, functional manner, and provides a practical alternative to extending built-in prototypes."

const square = (n: number) => Math.pow(n, 2);
4 |> square |> console.log; // 16

Currently this is a minimal implementation of pipeline operator, but I'm planning on making a F# variant support and after that a Topic/Smart pipeline variant PRs.

Partial application is a good pair for minimal/F# pipeline operator and has it's own PR at #37973. I'm planning on making a separate draft PR that has both features in them.

There is also a PR for Hack-style pipelines with # as a placeholder token.

Code status

This works minimally. The TS transformer generates valid JS and there's working type inference. There are many bugs, inconsistencies with specs, insufficient error messages currently. Most of failing tests are not included in this PR since Playground won't get generated with any failures and having faulty baselines is shady. I'm eagerly awaiting feedback, testing and even help with this PR. Even using and talking of these features will help advance them through TC39 proposal process!

await is currently not supported.

I used @rbuckton 's work on pipelineStage1 branch as a guide.

Testing

Playground link

To test it in your project, add following to your package.json:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/100059/artifacts?artifactName=tgz&fileId=D3836CF7DD5F6438481C5BCDBEE7A63AB21880164FEFD2BD75085B15EB974FCE02&fileName=/typescript-4.3.0-insiders.20210404.tgz"
    }
}

and then run npm install.

PR state

There is an associated issue in the Backlog milestone (required)
Code is up-to-date with the master branch
You've successfully run gulp runtests locally
There are new or updated unit tests validating the change

Why a draft PR for stage 1 proposal?

The TypeScript officially follows EcmaScript / stage 3 proposals. However, I'm opening this draft pull request despite it being only stage 1. The partial application and pipeline proposals now are in a stage where they want people to test the features - write it in their code, find problems, test existing codebases. The usual way these would be done is through a Babel plugin, but since a few years ago, just a plain text editor with syntax highlighting is no longer the default JS development environment.

TypeScript is no longer just a build step. It's used by a significant portion of JS developers for all their development, even when prototyping new code. If the tool chain of these developers requires TypeScript, then a simple Babel plugin won't help. This means that writing TypeScript with those features is impossible without first-class support for them in TypeScript.

But then could we ask them to write just plain JS instead? It turns out that TypeScript is more and more part of normal JavaScript development process with Language Server Protocol pointing out errors in code and providing typing information. Having people encounter red squigglies and disappearing type information as soon as they use the new language feature will not count favourably to the user experience. I believe having TypeScript support for new syntax proposals as important or even more important than having a Babel plugin.

I was considering creating and upkeeping a separate fork, but that would just mean duplication of work. There's already a great infrastructure for TypeScript with playgrounds, CI, etc. With a draft PR, new features can more easily get more eyes, testers and feedback. There will eventually be issues and questions on new features for TypeScript and github.com/microsoft/TypeScript is where anyone would first look up info on partial application.

Of course, I'm not expecting merging until the proposal reaches stage 3.

orta · 2020-05-03T03:27:07Z

@typescript-bot pack this

typescript-bot · 2020-05-03T03:28:09Z

Heya @orta, I've started to run the tarball bundle task on this PR at 19fd8b1. You can monitor the build here.

typescript-bot · 2020-05-03T03:43:23Z

Hey @orta, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/72713/artifacts?artifactName=tgz&fileId=5FDC6B0458B0184B85D790D1EB8AA47C0924EADA6FBC0575C4425A46F70973E002&fileName=/typescript-4.0.0-insiders.20200503.tgz"
    }
}

and then running npm install.

There is also a playground for this build.

markusjohnsson · 2020-05-03T06:59:46Z

Nice! I have wondered myself how pipeline operators would play with typescript, especially how the types would flow. You could have went the way of only parsing and type checking, and letting e.g. babel do translation, but since you have also done translation this is really easy to try out.

I did test it out and ran in to a type inference limitation which I hope can be fixed or it will be hard to use with complex types:

const ns = [1, 2, 3, 4, 5];

function map<T, R>(f: (t: T) => R) {
    return (ts: T[]): R[] => ts.map(f);
}

const result = ns |> map(x => x * x);

index.ts:8:16 - error TS2571: Object is of type 'unknown'.

8 ns |> map(x => x * x);

Here T must be inferred from the type of ns.

rbuckton · 2020-05-03T20:07:22Z

@Pokute: If it helps at all I have two old branches where I started to look into implementing pipeline as well:

https://github.com/microsoft/TypeScript/tree/pipelineStage1 - Limited to only the pipeline operator
https://github.com/microsoft/TypeScript/tree/rbuckton/pipeline - Pipeline, partial application, and a few other concepts. Older than pipelineStage1 and much more experimental

Pokute · 2020-05-04T07:47:00Z

Nice! I have wondered myself how pipeline operators would play with typescript, especially how the types would flow. You could have went the way of only parsing and type checking, and letting e.g. babel do translation, but since you have also done translation this is really easy to try out.

Writing the transformer was actually pretty simple as soon as I had the scanning & parsing in place.

I did test it out and ran in to a type inference limitation which I hope can be fixed or it will be hard to use with complex types:
Here T must be inferred from the type of ns.

I think that the form that TypeScript seems to internally handle it is map(x => x * x)(ns).
This is an interesting case where the type of a function would be resolved from it's usage and it's not directly either. I'm interested whether there are existing cases of this happening within TypeScript.

In the simplest cases, it feels like it should be possible to resolve the type. Beyond that is out of my expertise. :-)

Either way, thank you for an interesting test case!

phaux · 2020-05-04T15:33:38Z

Is there a way to try pipeline operator and partial application together?

dustinlacewell · 2020-05-08T07:46:03Z

When I found this PR I nearly cried. Thank you.

lukewestby · 2020-05-15T01:14:13Z

Hey folks, I stumbled upon this PR from https://github.com/tc39/proposal-pipeline-operator#build-tools. I'm not able to access the tarball mentioned here #38305 (comment) and I'm just wondering if it's been removed intentionally.

Pokute · 2020-05-15T06:17:05Z

Hey folks, I stumbled upon this PR from https://github.com/tc39/proposal-pipeline-operator#build-tools. I'm not able to access the tarball mentioned here #38305 (comment) and I'm just wondering if it's been removed intentionally.

At least I haven't removed the tarball. I wonder if those have limited lifetimes. Any ideas @orta?

orta · 2020-05-15T14:59:11Z

Yep, looks like they timeout - we're having a chat about what to do in the long term (normally this is not a problem for us because they are more active PRs), but for now I'll give it another 10-12 days.

@typescript-bot pack this

typescript-bot · 2020-05-15T14:59:20Z

Heya @orta, I've started to run the tarball bundle task on this PR at 19fd8b1. You can monitor the build here.

typescript-bot · 2020-05-15T15:14:55Z

Hey @orta, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/74014/artifacts?artifactName=tgz&fileId=53B4D99407A93B4AF6FAEF8437B02126A8C7FEC62F1C3307DAA098F75D2CE5D602&fileName=/typescript-4.0.0-insiders.20200515.tgz"
    }
}

and then running npm install.

There is also a playground for this build.

mikearnaldi · 2020-05-29T21:48:40Z

Nice! I have wondered myself how pipeline operators would play with typescript, especially how the types would flow. You could have went the way of only parsing and type checking, and letting e.g. babel do translation, but since you have also done translation this is really easy to try out.

I did test it out and ran in to a type inference limitation which I hope can be fixed or it will be hard to use with complex types:
const ns = [1, 2, 3, 4, 5];

function map<T, R>(f: (t: T) => R) {
    return (ts: T[]): R[] => ts.map(f);
}

const result = ns |> map(x => x * x);
index.ts:8:16 - error TS2571: Object is of type 'unknown'.

8 ns |> map(x => x * x);
Here T must be inferred from the type of ns.

Running in the same issue, without this sort of inference the usage of the operator is very limited, not sure if this can help but currently the only place I sow proper inference with a pipe implementation is this (not the operator a simple pipe function):
https://github.com/gcanti/fp-ts/blob/master/src/pipeable.ts#L97

xixixao · 2020-06-16T04:43:28Z

This is great, I might take a stab at adding the Hack-style proposal (see tc39/proposal-pipeline-operator#167 (comment) - it's the Smart proposal without the minimal proposal) to Flow.

ken-okabe · 2020-06-19T01:36:29Z

One of the greatest updates.
Just in case, here's the working:

 "devDependencies": {
    "@types/node": "*",
    "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/74014/artifacts?artifactName=tgz&fileId=53B4D99407A93B4AF6FAEF8437B02126A8C7FEC62F1C3307DAA098F75D2CE5D602&fileName=/typescript-4.0.0-insiders.20200515.tgz"
  }

sgioia9 · 2020-07-14T01:21:24Z

YESSSS

chenglou · 2020-07-14T11:23:11Z

Funny that this is coming back full circle. I've made a comment a while ago regarding the type inference/tooling problem with the pipe, and how to solve it: tc39/proposal-pipeline-operator#143 (comment)

Jopie64 · 2020-07-15T21:40:45Z

About that backward type inference problem:

In RxJS, TypeScript works just fine for a situation like this:

const int$ = of('hello').pipe(map(x => x.length));
// type of int$ is inferred to be Observable<number>

This would be equivalent to:

const int$ = of('hello') |> map(x => x.length);

Why would it be impossible to infer int$ in the latter case while it is inferrable in the former? In both cases map is something of type:

type Map = <T, U>(f: T => U) => Observable<T> => Observable<U>

And hence the receiver is the last curried argument. So it feels like it should be inferrable in both cases..?

benlesh · 2020-07-17T19:59:49Z

FYI: It appears the type inference isn't working that great for more advanced cases like RxJS see this playground example

Copied here for convenience:

interface Subscription {
  unsubscribe(): void;
}

interface Observer<T> {
  next(value: T): void;
  error(err: any): void;
  complete(): void;
}

interface Observable<T> {
  subscribe(observer: Partial<Observer<T>>): Subscription;
}

type Op<T, R> = (source: Observable<T>) => Observable<R>;

declare function map<T, R>(fn: (value: T, index: number) => R): Op<T, R>;

declare function filter<T>(fn: (value: T, index: number) => boolean): Op<T, T>;

declare const source: Observable<number>;

const result = source
  |> map(x => x + x) // <-- x is `unknown`
  |> map(x => x + '!!!') // <-- x is `unknown`
  |> filter(x => x.length < 20); // <-- x is `unknown`

This is a real shame, because it would be a killer language feature for libraries like RxJS.

lozandier · 2020-07-22T14:05:28Z

@benlesh @markusjohnsson Seems to be a pattern of type inference needing to be added for its use w/ generics; generic use cases don't seem to be covered yet in cases/conformance nor baselines/reference yet by @Pokute.

ken-okabe · 2020-08-08T13:58:36Z

@typescript-bot pack this

orta · 2020-08-17T14:33:56Z

@typescript-bot pack this

typescript-bot · 2020-08-17T14:33:59Z

Heya @orta, I've started to run the tarball bundle task on this PR at 19fd8b1. You can monitor the build here.

kiprasmel · 2020-08-18T07:12:47Z

Testing features like this out in a project and having to manually update the packed bundle because it expires in about 10 days is very annoying - could we get a feature to tell typescript bot to pack things indefinitely? Thanks!

/cc @orta

orta · 2021-04-18T07:32:56Z

@typescript-bot pack this

typescript-bot · 2021-04-18T07:32:59Z

Heya @orta, I've started to run the tarball bundle task on this PR at 50ce993. You can monitor the build here.

typescript-bot · 2021-04-18T07:54:00Z

Hey @orta, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/101226/artifacts?artifactName=tgz&fileId=52B72C8EFE9367109126FF631C5367AC4E9A15482B3AD34AB472EEEEED73464F02&fileName=/typescript-4.3.0-insiders.20210418.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@4.3.0-pr-38305-57".;

Lonli-Lokli · 2021-04-18T07:55:28Z

Will it be possible to use pipeline with iterators, sync and async?

lozandier · 2021-04-18T08:18:16Z

@Lonli-Lokli Yes, see the following examples from the proposal's github repo

The simple pipeline handles the former 2 trivially by design; F#, Smart, & Hack-style proposals build on top of the simple proposal to accommodate async functions / promises / iterables a variety of ways w/ pros & cons.

rjhilgefort · 2021-05-31T19:57:47Z

Is anyone aware of a working version of this PR? I tried the playground link posted by the bot, but the playground doesn't appear to be working with this branch?

https://www.staging-typescript.org/play?ts=4.3.0-pr-38305-57

Pokute · 2021-06-02T13:08:18Z

Is anyone aware of a working version of this PR? I tried the playground link posted by the bot, but the playground doesn't appear to be working with this branch?

There might be a problem with operator precedence. You need to wrap the inline arrow function within parenthesis: |> ((x: string) => x.toUpperCase()) |>

Thank you for testing this out, @rjhilgefort.

rjhilgefort · 2021-06-02T14:10:05Z

@Pokute ah, thanks, I didn't realize you had to wrap the lamda in parens. That seems to work, mostly. The compiler accepts the expression, but still has an error.

It doesn't seem to properly infer that x is a string and I would have to tell the compiler that it's a string explicitly.

Is that expected for this build? Is the plan to have that inference work?

ken-okabe · 2021-06-02T14:44:19Z

@Pokute ah, thanks, I didn't realize you had to wrap the lamda in parens. That seems to work, mostly. The compiler accepts the expression, but still has an error.

It doesn't seem to properly infer that x is a string and I would have to tell the compiler that it's a string explicitly.

Is that expected for this build? Is the plan to have that inference work?

I think it's natural that the pipeline operator has a high precedence.

and the error is common in typescript, nothing to do with the pipeline operator.

Pokute · 2021-06-02T15:24:12Z

The compiler [...] still has an error.
It doesn't seem to properly infer that x is a string and I would have to tell the compiler that it's a string explicitly.

Is that expected for this build? Is the plan to have that inference work?

and the error is common in typescript, nothing to do with with to the pipeline operator.

This should be inferable as shown in a above screenshot. It does require more work though, but I'm not sure when I can work on it.

I'm not sure about the precedence either. I'll need to check it out.

timbuckley · 2021-09-10T00:24:48Z

TC39 is moving forward with the Hack-proposal version (stage 1 -> 2).

Therefore, this PR should be closed in favor of the Hack-style approach #43617

ken-okabe · 2021-09-14T06:35:48Z

TC39 is moving forward with the Hack-proposal version (stage 1 -> 2).

Therefore, this PR should be closed in favor of the Hack-style approach #43617

I firstly thought it's good news, but investigating "hack-style" that for some reason I didn't notice (I only knew miminal/F#/Smart).

I think if a programmer knows mathematics or functional programming, the one will notice it's just a mess and night-mare.

I wrote this:

https://gist.github.com/tabatkins/1261b108b9e6cdab5ad5df4b8021bcb5#gistcomment-3892109

ken-okabe · 2021-09-27T21:17:17Z

I made an issue:
Inconsistency of type inference of Hack-style pipeline-operator proposal (Stage2) #46101
also an article:
TC39/proposal-pipeline-operator Hack-style |> hijacks Grouping operator ( )
https://dev.to/stken2050/the-current-tc39-proposal-pipeline-operator-hack-style-hijacks-grouping-operator-1dam

andrewbranch · 2021-09-27T21:32:30Z

@stken2050 please do not post any comments here that are unrelated to the specific implementation of this PR. General discussion of the pipeline proposal should be done on the pipeline proposal’s repository. Seeing as you’re already participating there, one backlink for visibility was ok, but at this point you’re bordering on spam.

ken-okabe · 2021-09-27T21:43:23Z

@andrewbranch Thanks for your comments,

Seeing as you’re already participating there, one backlink for visibility was ok

however, this is not exactly true, unfortunately, they are deleting comments.

sandersn · 2022-05-24T22:39:32Z

This experiment is pretty old, so I'm going to close it to reduce the number of open PRs.

Pokute marked this pull request as draft May 3, 2020 02:12

Pokute mentioned this pull request May 3, 2020

What is the current status? tc39/proposal-pipeline-operator#157

Closed

RyanCavanaugh added the Experiment A fork with an experimental idea which might not make it into master label Jun 16, 2020

jamiebuilds mentioned this pull request Jul 14, 2020

Negated types #29317

Closed

orta mentioned this pull request Aug 19, 2020

Sandbox: Support making customizations to monaco-typescript microsoft/TypeScript-Website#191

Closed

Pokute and others added 6 commits April 17, 2021 01:49

Pipeline - F# - checker, parser, transformer.

cf19047

Pipeline - Remove pipelineTopic.ts

b47feb9

Pipeline operator - Add PipelineOperator

921be32

Pipeline - Remove PipelineOperator.

960d0e0

Pipeline - tests.

65855bb

Pipeline - updated baselines

50ce993

Pokute force-pushed the pipeline-operator branch from dac1dcb to 50ce993 Compare April 17, 2021 21:46

bfu4 approved these changes Apr 28, 2021

View reviewed changes

This was referenced Sep 12, 2021

Bikeshedding the Hack topic token tc39/proposal-pipeline-operator#91

Open

Hack is dead. Long live F#. tc39/proposal-pipeline-operator#205

Closed

This was referenced Sep 14, 2021

tc39/proposal-pipeline-operator debate concern tc39/how-we-work#95

Closed

Question about ^ when a "child pipeline" is present. tc39/proposal-pipeline-operator#208

Closed

kawazoe mentioned this pull request Sep 24, 2021

Frustrations of people looking forward to tacit syntax tc39/proposal-pipeline-operator#215

Open

sandersn closed this May 24, 2022

Add support for F#-style Pipeline Operator #38305

Add support for F#-style Pipeline Operator #38305

Conversation

Pokute commented May 3, 2020 • edited Loading

Code status

Testing

PR state

Why a draft PR for stage 1 proposal?

orta commented May 3, 2020 • edited Loading

typescript-bot commented May 3, 2020 • edited Loading

typescript-bot commented May 3, 2020 • edited Loading

markusjohnsson commented May 3, 2020

rbuckton commented May 3, 2020

Pokute commented May 4, 2020 • edited Loading

phaux commented May 4, 2020

dustinlacewell commented May 8, 2020

lukewestby commented May 15, 2020

Pokute commented May 15, 2020

orta commented May 15, 2020 • edited Loading

typescript-bot commented May 15, 2020 • edited Loading

typescript-bot commented May 15, 2020 • edited Loading

mikearnaldi commented May 29, 2020

xixixao commented Jun 16, 2020

ken-okabe commented Jun 19, 2020 • edited Loading

sgioia9 commented Jul 14, 2020

chenglou commented Jul 14, 2020

Jopie64 commented Jul 15, 2020

benlesh commented Jul 17, 2020

lozandier commented Jul 22, 2020

ken-okabe commented Aug 8, 2020

orta commented Aug 17, 2020

typescript-bot commented Aug 17, 2020 • edited Loading

kiprasmel commented Aug 18, 2020 • edited Loading

orta commented Apr 18, 2021

typescript-bot commented Apr 18, 2021 • edited Loading

typescript-bot commented Apr 18, 2021 • edited Loading

Lonli-Lokli commented Apr 18, 2021

lozandier commented Apr 18, 2021

rjhilgefort commented May 31, 2021

Pokute commented Jun 2, 2021 • edited Loading

rjhilgefort commented Jun 2, 2021

ken-okabe commented Jun 2, 2021 • edited Loading

Pokute commented Jun 2, 2021

timbuckley commented Sep 10, 2021 • edited Loading

ken-okabe commented Sep 14, 2021

ken-okabe commented Sep 27, 2021 • edited Loading

andrewbranch commented Sep 27, 2021

ken-okabe commented Sep 27, 2021 • edited Loading

sandersn commented May 24, 2022

Pokute commented May 3, 2020 •

edited

Loading

orta commented May 3, 2020 •

edited

Loading

typescript-bot commented May 3, 2020 •

edited

Loading

typescript-bot commented May 3, 2020 •

edited

Loading

Pokute commented May 4, 2020 •

edited

Loading

orta commented May 15, 2020 •

edited

Loading

typescript-bot commented May 15, 2020 •

edited

Loading

typescript-bot commented May 15, 2020 •

edited

Loading

ken-okabe commented Jun 19, 2020 •

edited

Loading

typescript-bot commented Aug 17, 2020 •

edited

Loading

kiprasmel commented Aug 18, 2020 •

edited

Loading

typescript-bot commented Apr 18, 2021 •

edited

Loading

typescript-bot commented Apr 18, 2021 •

edited

Loading

Pokute commented Jun 2, 2021 •

edited

Loading

ken-okabe commented Jun 2, 2021 •

edited

Loading

timbuckley commented Sep 10, 2021 •

edited

Loading

ken-okabe commented Sep 27, 2021 •

edited

Loading

ken-okabe commented Sep 27, 2021 •

edited

Loading