Clarifying question: can await be used in top-level code?

[davidbau]

In the proposal, it looks like "await" is to be used inside async functions.

Can "await" be used in top-level code?

The reason is, when teaching students, it makes a lot of sense to start with top-level code that does not contain function definitions, and it also makes sense to "await input", such as in this example using Iced CoffeeScript's "await". What would this example look like in this proposal?

http://pencilcode.net/edit/guessmynumber

[lukehoban]

Good question.

There are three notions of "top-level code" that are interesting here;

1. A JavaScript REPL
2. The ES6 Script grammar
3. The ES6 Module grammar

I think you are mostly asking about the REPL case (or, as in the case you linked, a hosted execution environment of same kind where the embedder can choose how to execute the user provided code). Here it should be mostly up to the host whether they want to allow await or not. If they do, they would execute the code as though wrapped in:

(async function() {
  // <embed user provided code here>
})();

For example like your link, this would likely work. However, it has two downsides. First, it hides all declarations inside the function scope, so there is no shared global across REPL inputs. Second, the return value is delivered asynchronously, so the embedding would have to decide how to handle that.

The other two cases are the more formal ones that the language would have to decide on.

For an ES6 Script, there are a couple of problems. First is that scripts synchronously return a value. I suspect there is significant dependence on this in the web platform and other JS host environments. Second is that scripts synchronously modify a shared global scope. Is await was allowed, this could lead to some confusing beahviour. For example:

<script>
var x = await userEnteredAValue();
function foo() { return x; }
var y = "hello";
</script>
<script>
console.log(y) // is this 'hello'?
console.log(foo()) // is this the value the user entered?
</script>

There is already a <script async> in HTML, which offers semantics that don't guarantee the ordering of the <script> blocks. It may be that a new ScriptAsync top-level production could be added to the grammar, and this could be the target production for <script async>, but I'm not sure that's a good idea.

In the ES6 Module case, the grammar production already allows the async import and does not have a synchronous return value that can be consumed anywhere. So await may be able to be added here. The question would be whether the linking phase of the module pipeline would chain to the promise implicitly created by the Module production. I think it could, but this could make it too easy to unnecessarily delay the module loading phase, and I'm not sure in practice it would be justified.

Ultimately, I'm not sure it would make sense to add await to either the Program or Module grammar, but REPLs or other embeddings could likely still provide environments that supported this at their own top-level.

[davidbau]

Thanks for the answer!

From a user (and teacher) point of view, I like either the possibility that
Script could just support it (nailing down some reasonable synchronous
semantics, such as eval to undefined after encountering async) or that we
define <script async> to support it. Ideally just Script should support it
by default.

The reason is this:
(1) "async" solves a key user problem for beginners learning in Javascript
(allowing them to write code like "guess my number" that "blocks" on input).
(2) but much of the benefit for beginners is lost if they need learn what a
function is before they can use async.

Pedagogically, it makes sense for the lesson on "input" to come before the
lesson on "function". It should certainly come before the lesson on "use
an anonymous closure function to wrap your whole program, and execute it
immediately".

If done with good top-level script support, async could make Javascript a
significantly better platform for learning.

David

On Sat, Mar 29, 2014 at 12:29 PM, Luke Hoban notifications@github.comwrote:

Good question.

There are three notions of "top-level code" that are interesting here;

1. A JavaScript REPL
2. The ES6 Script grammarhttps://people.mozilla.org/%7Ejorendorff/es6-draft.html#sec-scripts
3. The ES6 Module grammarhttps://people.mozilla.org/%7Ejorendorff/es6-draft.html#sec-modules

I think you are mostly asking about the REPL case (or, as in the case you
linked, a hosted execution environment of same kind where the embedder can
choose how to execute the user provided code). Here it should be mostly up
to the host whether they want to allow await or not. If they do, they
would execute the code as though wrapped in:

(async function() {
// })();

For example like your link, this would likely work. However, it has two
downsides. First, it hides all declarations inside the function scope, so
there is no shared global across REPL inputs. Second, the return value is
delivered asynchronously, so the embedding would have to decide how to
handle that.

The other two cases are the more formal ones that the language would have
to decide on.

For an ES6 Script, there are a couple of problems. First is that scripts
synchronously return a valuehttps://people.mozilla.org/%7Ejorendorff/es6-draft.html#sec-runtime-semantics-script-evaluation.
I suspect there is significant dependence on this in the web platform and
other JS host environments. Second is that scripts synchronously modify a
shared global scope. Is await was allowed, this could lead to some
confusing beahviour. For example:

<script>var x = await userEnteredAValue();function foo() { return x; }var y = "hello";</script><script>console.log(y) // is this 'hello'?console.log(foo()) // is this the value the user entered?</script>

There is already a <script async> in HTML, which offers semantics that
don't guarantee the ordering of the <script> blocks. It may be that a new
ScriptAsync top-level production could be added to the grammar, and this
could be the target production for <script async>, but I'm not sure
that's a good idea.

In the ES6 Module case, the grammar production already allows the async
import and does not have a synchronous return value that can be consumed
anywhere. So await may be able to be added here. The question would be
whether the linking phase of the module pipeline would chain to the promise
implicitly created by the Module production. I think it could, but this
could make it too easy to unnecessarily delay the module loading phase, and
I'm not sure in practice it would be justified.

Ultimately, I'm not sure it would make sense to add await to either the
Program or Module grammar, but REPLs or other embeddings could likely
still provide environments that supported this at their own top-level.

Reply to this email directly or view it on GitHubhttps://github.com//issues/9#issuecomment-39000615
.

[getify]

scripts synchronously return a value. I suspect there is significant dependence on this in the web platform

Can you give an example of this? It's the first I've ever realized that scripts have a return value.

[zenparsing]

Since do-expressions are on the table for ES7, I wonder if it might make more sense to provide for this use case through an async version of do-expressions:

// In a script (or even a module)
let result = do async {
    await something;
}

[lukehoban]

@getify - Good question.

The HTML5 spec has dependencies on the SourceElements and FunctionBody. productions, for scripts and event handler content attributes respectively. In the first case, it looks like it doesn't do anything with the value. In the second case, I can't tell - but that would be different anyway.

So it may be that the web platform could get away with chaging the meaning of script tags to execute the script, if it returns a value, chain off that value (if its a promise, continue asynchonosly, else if its not, continue synchrnously).

I actually cant' think of anything that would obviously break here. The biggest concern might just be the ability to "accidentally" slow down the load path. But in practice, that's something that is happening already in module loaders, so maybe not a critical concern.

[lukehoban]

@zenparsing - Yes, that might be an option, but I expect that it wouldn't pass @davidbau's simplicity test. If you are willing to do that, it's not much of a stretch to go to:

let result = (async function() {
  await something;
})();

[getify]

So it may be that the web platform could get away with chaging the meaning of script tags to execute the script, if it returns a value, chain off that value (if its a promise, continue asynchonosly, else if its not, continue synchrnously).

How exactly does a script "return a value"? You can't call return in a top-level script program. Is it returning a value from the last evaluated statement/expression implicitly (like dev-tools consoles do)?

[lukehoban]

How exactly does a script "return a value"? You can't call return in a top-level script program. Is it returning a value from the last evaluated statement/expression implicitly (like dev-tools consoles do)?

Yes, that appears to be the case. See https://people.mozilla.org/~jorendorff/es6-draft.html#sec-runtime-semantics-script-evaluation.

[getify]

Yes, that appears to be the case

Interesting, I wonder if that's there for the eval(..) use case? I've never heard of there being any other usage of this result. Learn something new every day!

[zenparsing]

@lukehoban I included the result capture for generality, but we'd leave it off for this use case:

do async {
    await whatever();
}

Pretty simple, I think. And much cleaner that an IIAF.

There will be no getting around sync/async duality in Javascript. The simple thing to do is to structure the language so that contexts are synchronous unless explicitly otherwise.

[getify]

for consistency, it would seem like...

async do {
    await whatever();
}

...would make more sense, no?

[davidbau]

So it may be that the web platform could get away with chaging the meaning
of script tags to execute the script, if it returns a value, chain off that
value (if its a promise, continue asynchonosly, else if its not, continue
synchrnously).

(To help me understand - ) If the web platform did this, can you give a
couple examples of what could go in a script tag and what the semantics
could be?

On Tue, Apr 8, 2014 at 9:23 PM, Kyle Simpson notifications@github.comwrote:

for consistency, it would seem like...

async do {
await whatever();}

...would make more sense, no?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/9#issuecomment-39920133
.

[lukehoban]

@davidbau Here's an example - note that it uses await at top level in a script.

<script>
var x = await fetchAsync("www.bing.com");
</script>
<script>
document.write(x);
</script>
<img src="someimage.png"></img>

This should result in the contents of bing.com followed by an tag. Normal script tags like these have to run to completion before DOM parsing and construction can continue. If you allow await in a script tag, you would have to wait for all the awaits to complete before considering the script to have run to completion, and only then continue on with DOM parsing and construction.

This is not unlike what has to happen when you have a <script src='http://example.org/'> which can asynchronously delay DOM parsing, and is thus encouraged to appear as late as possible in HTML. I am not an expert on this though, there may be hairier issues.

[davidbau]

Thank you for the example. It would behave the same way with a single
script tag instead of two, right?

This model would be terrific and simple, and it would give beginners a very
understandable way to learn (and write classical learning programs) using
straight-line code in a script tag.

David

On Wed, Apr 9, 2014 at 6:38 PM, Luke Hoban notifications@github.com wrote:

@davidbau https://github.com/davidbau Here's an example - note that it
uses await` at top level in a script.

<script>var x = await fetchAsync("www.bing.com");</script><script>document.write(x);</script>

This should result in the contents of bing.com followed by an tag. Normal
script tags like these have to run to completion before DOM parsing and
construction can continue. If you allow await in a script tag, you would
have to wait for all the awaits to complete before considering the script
to have run to completion, and only then continue on with DOM parsing and
construction.

This is not unlike what has to happen when you have a <script src=' http://example.org/'> which can asynchronously delay DOM parsing, and is
thus encouraged to appear as late as possible in HTML. I am not an expert
on this though, there may be hairier issues.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/9#issuecomment-40025261
.

[tomyan]

Perhaps a related question is whether native promises have a done method. In promise libraries like Q, there is a done method with behaviour equivalent to:

Promise.prototype.done = function (callback, errback) {
    var promise = callback || errback ? this.then(callback, errback) : this;
    promise.then(null, function (error) {
        throw error;
    });
};

Without this it's easy to miss error handling from your promise chain and have your async operations silently fail. If you were able to await at the top-level, the promise could simply be awaited and errors handled with try/catch, making this method redundant. If there was a way to do this an maintain backwards compatibility I think it would be a big win.

I expect most people wouldn't want to do this in top-level code (as it would need to block the page from rendering). It would be more useful in event handlers IMO.

Tom

[zenparsing]

@tomyan In es6now we deal with this issue by introducing the concept of a "main" function. If that main function returns a promise (as an async function would) and if that promise ends up being rejected, then the error is propagated back to the user just like a synchronous error is.

export async function main() {
    throw new Error("oops");
}

This approach seems to "just work" and keeps things separated and simple.

Also, it's off-topic here but since you mentioned it, we are experimentally doing the same thing with event handlers: if the event handler returns a rejected promise, it gets propagated back to the user in the usual expected way. This seems to me to be the best way so far to deal with the done issue.

[tomyan]

@zenparsing sounds sensible, I'll give es6now a spin

[lukehoban]

Summarizing where I think we are on this - there were three possible places we could deal with await in top-level code:

1. A JavaScript REPL
2. The ES6 Script grammar
3. The ES6 Module grammar

I suggest that we update the proposal to include the semantics for (3) as an extension of the async module loader, and that we include an informative note on (1) that REPL's should aim to support await at top-level in a natural way for the REPL environment.

I think we should not try to address (2) in the current proposal.

/cc @bterlson

[getify]

Speaking from the perspective of someone who teaches JS, I very much dislike that you would be creating yet another variation between normal JS and what the REPL does. Developers learn JS by trying things in REPLs (either CLI or in browser), and it's consistently confusing to them when they run across these various places where a REPL works differently from the global scope of a normal program.

There are necessary differences, but we shouldn't be going out of our way to make more and more differences without really strong reason. I wrote at length about this frustration.

In this case, I see no reason why we should encourage REPLs to diverge from the global script context. That is, if (2) isn't going to be handled, neither should (1). If (2) eventually gets handled in a later spec, like with async do... expressions, that should cover (1) as well.

[domenic]

REPLs should behave like modules, not scripts.

[arv]

This is getting off topic...

It is not that easy. You want the following to work:

> function f() { return 1; }
< undefined
> f()
< 1

if each line is a Script it works but if each line is a Module it does not.

[domenic]

Right, each line should not be a module, the entire REPL should be a module. Similarly, var x = 5 should not create a global variable, and so on.

That said this is off topic indeed so I'll stop here.

[lukehoban]

@getify - I don't know of any way for await to work in top-level of scripts in a general way. async do doesn't solve this, it is just sugar over a workaround (how to most easily put the code not at top-level). So I don't think (2) is an option at all.

That leaves the question of whether you want to allow await in a REPL or not. This isn't a spec question really - it's a question for various REPL implementers. But my guess is they will want to support it just because it makes the REPL more useful and broadly usable. Also, for better consistency with top level in modules, which will presumably be increasingly common.

[getify]

include an informative note on (1) that REPL's should aim to support await at top-level

My objection was to including a note, and specifically to including a note (even a non-normative one) that says "should". If the spec doesn't aim to provide any guidance on how (for REPLs), then it shouldn't aim to provide any guidance on what (for REPLs) either. If a REPL wants to try it, having such a non-normative note in there isn't going to sway them one direction or the other.

---

Despite the assertion "the entire REPL should be a module", that's not in any way reflective of any real world requirement (or even present/near-future reality). Some REPLs will, I'm sure, but others won't.

One problem with making a REPL session a module is that modules don't have access to the global scope, and yet many developers do in fact use things like the dev console's "REPL" to interact with the global scope. Same goes with interactive debugging sessions. Also, if a REPL has an await (or any other blocking mechanism, like import) in it, and the user types it in and it fires off a long running process, will that REPL just sit there appearing to be hung? I'm skeptical there really will be that many REPLs which go that direction, if for no other reason but UX.

My only point is that it's in no way a foregone conclusion that all or even most REPLs will significantly differ from main scripts in terms of how they deal with top-level asynchrony (blocking, etc). Since this spec doesn't want to tackle that beast, it should remain silent on the topic altogether.

[bterlson]

This was discussed at the July TC39. Await at top level introduces a large number of complexities with the module loader - complexities that need to be worked out there before they can be considered here. As such, TC39 wants to see async functions without await at top level. Await will still be reserved so we can pursue this in later editions of 262.

[pannous]

can't just the whole Top Level scope be marked async behind the scene?

[towerofnix]

@pannous I believe part of the problem is that in Node.js, scripts must be loaded and evaluated synchronously (no async) - because require is not an asynchronous function; it immediately returns the module.exports definition in the file path you give it.

I think it was discussed that, with ES6 modules, the top level scope could be marked async, because there is no "require function" - it's just a piece of syntax. But I haven't followed that discussion much, and I haven't taken much of a look at ES6 modules, so don't take my word for it!

[pannous]

@towerofnix interesting and good point. But what about the 'Script' context, wouldn't an async default be reasonable there?

[josephg]

Just to throw my hat in the ring - that this feature is blocked because of concerns about module loading is ridiculous. I don't care about the semantics of how require works. I do care about the code I write afterwards - eg:

const fetch = require('node-fetch')

const someval = await fetch('https://my.example/someurl.json')
// Use someval.

Top-level await makes writing simple scripts like this 900% better, because I don't have to wrap all my code in a (async () {})(); function block just to use all the features of the language.

[ljharb]

@josephg this repo is for the stage 4 proposal that landed in ES2017. You might be more interested in https://github.com/tc39/proposal-top-level-await ?

[josephg]

Ah, thanks!