Skip to content

Commit

Permalink
generators
Browse files Browse the repository at this point in the history
Behavior:

Generators are iterable. Each value yielded from the generator function
is iterated.

Generators are iterators. Values can be passed in through the `next`
method. The iterator result values will be either a) the yielded values,
b) the value returned from the generator (the completion value) or c)
`undefined`, if `next` is called after completion.

Generators can delegate to other iterables, including other generators.
Values passed to `next` on the delegator generator will be passed
through to the delegatee. Similarly, values yielded by the delegatee
will be returned along the iterable/iterator interfaces.

The "delegate yield" expression, `yield *`, returns the completion value
of the delegatee.

Implementation:

While normal functions' return type is the type of the `return`
statement, generators are different. The return type is always a
Generator, where the second type argument is the type of the `return`
statement.

We use the same process to infer this type--an internal tvar named
"return"--but then override the return type when creating the function
type, if the function is a generator.

In order to track the `next` and `yield` types, we introduce new
internal tvars, which accumulate lower bounds according to their use,
and upper bounds from explicit annotation of the Generator type.

Caveats:

The generator interface is highly dynamic, and not particularly amenable
to static typing.

The type of Generator<Y,R,N> enforces that all next values and all
yielded values be described by a single type, but it's quite normal to
use values of different types. Consider the following example:

```javascript
function *userGen() {
  var name = yield("What is your name?");
  var age = yield("How old are you?");
  return { name, age }
}

var gen = userGen();
var name, age, user;
if (name = prompt(gen.next().value)) {
  if (age = prompt(gen.next(name).value)) {
    user = gen.next(parseInt(age, 10)).value;
    // user is { name: string|number, age: string|number }
    // but we "want" { name: string, age: number }
  }
}
```

To escape this pitfall, you can use `Generator<any,any,any>` or write
the necessary dynamic type tests. Ideally, Flow would enforce that you
first pass a `string` to `next`, then a `number`, but that isn't
possible in general, and in specific instances is still very hard, so
I'm leaving it for future work (if ever).

Another difficulty is the fact that the argument to `next` is optional.
This is because, in order to "start" the generator, you need to call
`next` once. Put differently, the first argument to `next` is not
returned by the first `yield` expression; the second argument to `next`
is. Consider the following example:

```javascript
function *bmiCalc() {
  var height: number = yield("What is your height, in meters?");
  var weight: number = yield("What is your weight, in kilograms?");
  return weight / (height * height);
}

// The first argument to next is always `void`. The value of this
// expression is the string "What is your height..."
bmiCalc.next();

// This call to `next` expects the value for `height`, above, but
// because the type of `next` marks its argument optional, we allow
// this. :(
bmiCalc.next();
```

In this implementation, I wanted to make things strict, so the return
type of a yield expression is always `Y|void`, and the generator needs
to do a dynamic type test in order to get at the expected `Y`.

The above caveats, and the strict interpretation of this implementation,
has the consequence that most, if not all, generator code "in the wild"
will result in type errors from Flow. We might consider sacrificing
correctness here for practical purposes, but I think it will be easier
to go from strict to less strict than the other way around.
  • Loading branch information
samwgoldman committed Aug 6, 2015
1 parent 44984e0 commit c9ddfd6
Show file tree
Hide file tree
Showing 10 changed files with 297 additions and 59 deletions.
22 changes: 15 additions & 7 deletions lib/core.js
Original file line number Diff line number Diff line change
Expand Up @@ -295,18 +295,26 @@ declare class JSON {
}

/* Iterators */
interface IteratorResult<T> {
interface IteratorResult<Y,R> {
done: boolean;
value?: T;
value?: Y|R;
}

interface Iterator<T> {
next(): IteratorResult<T>;
@@iterator(): Iterator<T>;
interface $Iterator<Y,R,N> {
@@iterator(): $Iterator<Y,R,N>;
next(value?: N): IteratorResult<Y,R>;
}
type Iterator<T> = $Iterator<T,void,void>;

interface Iterable<T> {
@@iterator(): Iterator<T>;
interface $Iterable<Y,R,N> {
@@iterator(): $Iterator<Y,R,N>;
}
type Iterable<T> = $Iterable<T,void,void>;

/* Generators */
interface Generator<Y,R,N> {
@@iterator(): $Iterator<Y,R,N>;
next(value?: N): IteratorResult<Y,R>;
}

/* Maps and Sets */
Expand Down
123 changes: 95 additions & 28 deletions src/typing/type_inference_js.ml
Original file line number Diff line number Diff line change
Expand Up @@ -801,7 +801,6 @@ let rec convert cx map = Ast.Type.(function
mk_nominal_type cx reason map (c, params)
)

(* TODO: unsupported generators *)
| loc, Function { Function.params; returnType; rest; typeParameters } ->
let typeparams, map_ = mk_type_param_declarations cx ~map typeParameters in
let map = SMap.fold SMap.add map_ map in
Expand Down Expand Up @@ -1976,8 +1975,8 @@ and statement cx = Ast.Statement.(
let o = Flow_js.get_builtin_typeapp
cx
(mk_reason "iteration expected on Iterable" loc)
"Iterable"
[element_tvar] in
"$Iterable"
[element_tvar; AnyT.at loc; AnyT.at loc] in

Flow_js.flow cx (t, o); (* null/undefined are NOT allowed *)

Expand Down Expand Up @@ -2031,19 +2030,19 @@ and statement cx = Ast.Statement.(
| (loc, Debugger) ->
()

(* TODO: unsupported generators *)
| (loc, FunctionDeclaration {
FunctionDeclaration.id;
params; defaults; rest;
body;
generator;
returnType;
typeParameters;
async;
_
}) ->
let reason = mk_reason "function" loc in
let this = Flow_js.mk_tvar cx (replace_reason "this" reason) in
let fn_type = mk_function None cx reason ~async
let fn_type = mk_function None cx reason ~async ~generator
typeParameters (params, defaults, rest) returnType body this
in
Hashtbl.replace cx.type_table loc fn_type;
Expand Down Expand Up @@ -2609,11 +2608,11 @@ and object_prop cx map = Ast.Expression.Object.(function
_ }) ->
Ast.Expression.Function.(
let { params; defaults; rest; body;
returnType; typeParameters; id; async; _ } = func
returnType; typeParameters; id; async; generator; _ } = func
in
let reason = mk_reason "function" vloc in
let this = Flow_js.mk_tvar cx (replace_reason "this" reason) in
let ft = mk_function id cx ~async reason typeParameters
let ft = mk_function id cx ~async ~generator reason typeParameters
(params, defaults, rest) returnType body this
in
Hashtbl.replace cx.type_table vloc ft;
Expand Down Expand Up @@ -2663,7 +2662,7 @@ and object_prop cx map = Ast.Expression.Object.(function
let { body; returnType; _ } = func in
let reason = mk_reason "getter function" vloc in
let this = Flow_js.mk_tvar cx (replace_reason "this" reason) in
let function_type = mk_function None cx ~async:false reason None
let function_type = mk_function None cx ~async:false ~generator:false reason None
([], [], None) returnType body this
in
let return_t = extract_getter_type function_type in
Expand All @@ -2688,7 +2687,7 @@ and object_prop cx map = Ast.Expression.Object.(function
let { params; defaults; body; returnType; _ } = func in
let reason = mk_reason "setter function" vloc in
let this = Flow_js.mk_tvar cx (replace_reason "this" reason) in
let function_type = mk_function None cx ~async:false reason None
let function_type = mk_function None cx ~async:false ~generator:false reason None
(params, defaults, None) returnType body this
in
let param_t = extract_setter_type function_type in
Expand Down Expand Up @@ -3360,22 +3359,23 @@ and expression_ ~is_cond cx loc e = Ast.Expression.(match e with
params; defaults; rest;
body;
async;
generator;
returnType;
typeParameters;
_
} ->
let desc = (if async then "async " else "") ^ "function" in
let reason = mk_reason desc loc in
let this = Flow_js.mk_tvar cx (replace_reason "this" reason) in
mk_function id cx reason ~async
mk_function id cx reason ~async ~generator
typeParameters (params, defaults, rest) returnType body this

(* TODO: unsupported generators *)
| ArrowFunction {
ArrowFunction.id;
params; defaults; rest;
body;
async;
generator;
returnType;
typeParameters;
_
Expand All @@ -3384,8 +3384,8 @@ and expression_ ~is_cond cx loc e = Ast.Expression.(match e with
let reason = mk_reason desc loc in
let this = this_ cx reason in
let super = super_ cx reason in
mk_arrow id cx reason ~async
typeParameters (params, defaults, rest) returnType body this super
mk_arrow id cx reason ~async ~generator typeParameters
(params, defaults, rest) returnType body this super

| TaggedTemplate {
TaggedTemplate.tag = _, Identifier (_,
Expand Down Expand Up @@ -3428,8 +3428,37 @@ and expression_ ~is_cond cx loc e = Ast.Expression.(match e with
let reason = mk_reason name name_loc in
mk_class cx loc reason c

| Yield { Yield.argument; delegate = false } ->
let reason = mk_reason "yield" loc in
let yield = Env_js.get_var cx (internal_name "yield") reason in
let t = expression cx argument in
Flow_js.flow cx (t, yield);
let next = Env_js.get_var cx (internal_name "next") reason in
OptionalT next

| Yield { Yield.argument; delegate = true } ->
let reason = mk_reason "yield* delegate" loc in
let next = Env_js.get_var cx
(internal_name "next")
(prefix_reason "next of parent generator in " reason) in
let yield = Env_js.get_var cx
(internal_name "yield")
(prefix_reason "yield of parent generator in " reason) in
let t = expression cx argument in

let ret = Flow_js.mk_tvar cx
(prefix_reason "return of child generator in " reason) in

(* widen yield with the element type of the delegated-to iterable *)
let iterable = Flow_js.get_builtin_typeapp cx
(mk_reason "iteration expected on Iterable" loc)
"$Iterable"
[yield; ret; next] in
Flow_js.flow cx (t, iterable);

ret

(* TODO *)
| Yield _
| Comprehension _
| Generator _
| Let _ ->
Expand Down Expand Up @@ -4220,7 +4249,7 @@ and react_create_class cx loc class_props = Ast.Expression.(
returnType; typeParameters; _ } = func
in
let reason = mk_reason "defaultProps" vloc in
let t = mk_method cx reason ~async:false (params, defaults, rest)
let t = mk_method cx reason ~async:false ~generator:false (params, defaults, rest)
returnType body this (MixedT reason)
in
(match t with
Expand All @@ -4241,7 +4270,7 @@ and react_create_class cx loc class_props = Ast.Expression.(
returnType; typeParameters; _ } = func
in
let reason = mk_reason "initialState" vloc in
let t = mk_method cx reason ~async:false (params, defaults, rest)
let t = mk_method cx reason ~async:false ~generator:false (params, defaults, rest)
returnType body this (MixedT reason)
in
let override_state =
Expand All @@ -4261,10 +4290,10 @@ and react_create_class cx loc class_props = Ast.Expression.(
_ }) ->
Ast.Expression.Function.(
let { params; defaults; rest; body;
returnType; typeParameters; async; _ } = func
returnType; typeParameters; async; generator; _ } = func
in
let reason = mk_reason "function" vloc in
let t = mk_method cx reason ~async (params, defaults, rest)
let t = mk_method cx reason ~async ~generator (params, defaults, rest)
returnType body this (MixedT reason)
in
fmap, SMap.add name t mmap
Expand Down Expand Up @@ -5063,6 +5092,8 @@ and mk_class_elements cx instance_info static_info body = Ast.Class.(
let this, super, method_sigs, getter_sigs, setter_sigs =
if static then static_info else instance_info
in
let yield = MixedT (mk_reason "no yield" loc) in
let next = MixedT (mk_reason "no next" loc) in

let sigs_to_use = match kind with
| Method.Constructor
Expand All @@ -5082,7 +5113,7 @@ and mk_class_elements cx instance_info static_info body = Ast.Class.(
let ret = Flow_js.subst cx map_ ret in

mk_body None cx ~async
param_types_map param_loc_map ret body this super;
param_types_map param_loc_map ret body this super yield next;
);
ignore (Abnormal.swap Abnormal.Return save_return_exn);
ignore (Abnormal.swap Abnormal.Throw save_throw_exn)
Expand Down Expand Up @@ -5433,22 +5464,39 @@ and mk_interface cx reason_i typeparams map (sfmap, smmap, fmap, mmap) extends s
signature consisting of type parameters, parameter types, parameter names,
and return type, check the body against that signature by adding `this` and
`super` to the environment, and return the signature. *)
and function_decl id cx (reason:reason) ~async
and function_decl id cx (reason:reason) ~async ~generator
type_params params ret body this super =

let typeparams, type_params_map = mk_type_param_declarations cx type_params in

let (params, pnames, ret, param_types_map, param_types_loc) =
mk_params_ret cx type_params_map params (body_loc body, ret) in

(* If this is a generator function, the return type annotation can be an
application of the Generator type. We don't want to flow the explicit or
phantom return type into the Generator typeapp, but we still want to be
able to flow the Generator type constructed below into the annotation, so
we store off the converted annotation in _ret until then and proceed with a
tvar in its place. *)
let _ret = ret in
let (yield,ret,next) = if generator then (
Flow_js.mk_tvar cx (prefix_reason "yield of " reason),
Flow_js.mk_tvar cx (prefix_reason "return of " reason),
Flow_js.mk_tvar cx (prefix_reason "next of " reason)
) else (
MixedT (replace_reason "no yield" reason),
ret,
MixedT (replace_reason "no next" reason)
) in

let save_return_exn = Abnormal.swap Abnormal.Return false in
let save_throw_exn = Abnormal.swap Abnormal.Throw false in
Flow_js.generate_tests cx reason typeparams (fun map_ ->
let param_types_map =
param_types_map |> SMap.map (Flow_js.subst cx map_) in
let ret = Flow_js.subst cx map_ ret in

mk_body id cx ~async param_types_map param_types_loc ret body this super;
mk_body id cx ~async param_types_map param_types_loc ret body this super yield next;
);

ignore (Abnormal.swap Abnormal.Return save_return_exn);
Expand All @@ -5460,6 +5508,23 @@ and function_decl id cx (reason:reason) ~async
else ret
in

(* If this is a generator function, we don't want to use the type from the
return statement as the return type of the function. Instead, we want to
return a Generator typeapp where the inferred return type flows into the
Generator's R type param. Since generator functions can have explicit type
annotations, flow the inferred type into the annotation as well. *)
let ret =
if generator then
let t = Flow_js.get_builtin_typeapp
cx
reason
"Generator"
[yield; ret; next] in
Flow_js.flow cx (t, _ret);
t
else ret
in

(typeparams,params,pnames,ret)

and is_void cx = function
Expand All @@ -5470,7 +5535,7 @@ and is_void cx = function
and mk_upper_bound cx locs name t =
Scope.create_entry t t (SMap.get name locs)

and mk_body id cx ~async param_types_map param_locs_map ret body this super =
and mk_body id cx ~async param_types_map param_locs_map ret body this super yield next =
let ctx = Env_js.get_scopes () in
let new_ctx = Env_js.clone_scopes ctx in
Env_js.update_env cx new_ctx;
Expand All @@ -5492,6 +5557,8 @@ and mk_body id cx ~async param_types_map param_locs_map ret body this super =
(* special bindings for super, this, return value slot *)
add (internal_name "super") (create_entry super super None) scope;
add (internal_name "this") (create_entry this this None) scope;
add (internal_name "yield") (create_entry yield yield None) scope;
add (internal_name "next") (create_entry next next None) scope;
add (internal_name "return") (create_entry ret ret None) scope;
scope
) in
Expand Down Expand Up @@ -5659,18 +5726,18 @@ and extract_type_param_instantiations = function
| Some (_, typeParameters) -> typeParameters.Ast.Type.ParameterInstantiation.params

(* Process a function definition, returning a (polymorphic) function type. *)
and mk_function id cx reason ~async type_params params ret body this =
and mk_function id cx reason ~async ~generator type_params params ret body this =
(* Normally, functions do not have access to super. *)
let super = MixedT (replace_reason "empty super object" reason) in
let signature =
function_decl id cx reason ~async type_params params ret body this super
function_decl id cx reason ~async ~generator type_params params ret body this super
in
mk_function_type cx reason this signature

(* Process an arrow function, returning a (polymorphic) function type. *)
and mk_arrow id cx reason ~async type_params params ret body this super =
and mk_arrow id cx reason ~async ~generator type_params params ret body this super =
let signature =
function_decl id cx reason ~async type_params params ret body this super
function_decl id cx reason ~async ~generator type_params params ret body this super
in
(* Do not expose the type of `this` in the function's type. The call to
function_decl above has already done the necessary checking of `this` in
Expand Down Expand Up @@ -5703,9 +5770,9 @@ and mk_function_type cx reason this signature =

(* This function is around for the sole purpose of modeling some method-like
behaviors of non-ES6 React classes. It is otherwise deprecated. *)
and mk_method cx reason ~async params ret body this super =
and mk_method cx reason ~async ~generator params ret body this super =
let (_,params,pnames,ret) =
function_decl None cx ~async reason None params ret body this super
function_decl None cx ~async ~generator reason None params ret body this super
in
FunT (reason, Flow_js.dummy_static, Flow_js.dummy_prototype,
Flow_js.mk_functiontype2
Expand Down
14 changes: 7 additions & 7 deletions tests/async/async.exp
Original file line number Diff line number Diff line change
Expand Up @@ -3,29 +3,29 @@ async.js:12:3,11: async return
Error:
async.js:12:10,10: number
This type is incompatible with
[LIB] core.js:372:32,45: union type
[LIB] core.js:380:32,45: union type

async.js:30:30,35: number
This type is incompatible with
[LIB] core.js:372:32,45: union type
[LIB] core.js:380:32,45: union type

async.js:45:22,25: undefined
This type is incompatible with
[LIB] core.js:352:1,379:1: Promise
[LIB] core.js:360:1,387:1: Promise

async2.js:6:3,12: async return
Error:
async2.js:6:10,11: number
This type is incompatible with
[LIB] core.js:372:32,45: union type
[LIB] core.js:380:32,45: union type

async2.js:29:21,24: undefined
This type is incompatible with
[LIB] core.js:352:1,379:1: Promise
[LIB] core.js:360:1,387:1: Promise

async2.js:43:28,31: undefined
This type is incompatible with
[LIB] core.js:352:1,379:1: Promise
[LIB] core.js:360:1,387:1: Promise

async2.js:48:3,17: undefined
This type is incompatible with
Expand All @@ -35,7 +35,7 @@ async3.js:23:11,21: await
Error:
async3.js:12:10,11: number
This type is incompatible with
[LIB] core.js:372:32,45: union type
[LIB] core.js:380:32,45: union type

await_not_in_async.js:5:9,9: Unexpected number

Expand Down
Loading

0 comments on commit c9ddfd6

Please sign in to comment.