Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish native es modules #1766

Closed
bennypowers opened this issue Oct 20, 2017 · 28 comments
Closed

Publish native es modules #1766

bennypowers opened this issue Oct 20, 2017 · 28 comments
Labels
feature-request A feature should be added or improved. needs-major-version Can only be considered for the next major release

Comments

@bennypowers
Copy link

Build steps for dev servers don't have to be a thing anymore. With native modules and relative path module specifiers, we could just run the code in the browser as-is.

import CognitoIdentityServiceProvider from '../../aws-sdk/es/clients/cognitoidentityserviceprovider';

Having native es modules in this library would really be swell.

Thanks for your consideration.

@AllanZhengYP
Copy link
Contributor

Thank you for your advice. It's very nice and intuitive to have this feature. However, this may need a major version update. 😉

@bennypowers
Copy link
Author

Pardon my ignorance of the internal workings of your library, but it seems to me that you could release a minor version that doesn't break backwards, only adds an es-module version in /dist/es/

@jeskew jeskew added the feature-request A feature should be added or improved. label Oct 23, 2017
@jeskew
Copy link
Contributor

jeskew commented Oct 23, 2017

The semantics of ES modules are different enough from script-based module formats like CommonJS, UMD, and AMD that I don't think we could automate a conversion of the SDK from its current format to an ESM-based format. Since we release several times a week, if the conversion cannot be automated, then I'm not sure it's a feasible solution.

The conversion from ESM to CommonJS, however, is fairly straightforward, so if the SDK were written following ES module semantics, we could release both versions. That, however, would require a major version bump.

@bennypowers
Copy link
Author

bennypowers commented Oct 23, 2017 via email

@sholladay
Copy link

There is one tool I am aware of that takes CommonJS modules as input and outputs ES modules, which is exactly what you need to automate the conversion.

https://github.com/rollup/rollup-plugin-commonjs

That Rollup plugin is intended to be used as part of a build process to import CommonJS dependencies, but I suspect you could hack it to suit your needs relatively easily.

By the way, I disagree that offering a new type of build would demand a major version bump. It's a new feature and should not impact existing users whatsoever.

@bennypowers
Copy link
Author

bennypowers commented Oct 26, 2017

On that note, I'm experimenting with using that plugin to get the ball rolling, and actually successfully bundled using:

import sourcemaps from 'rollup-plugin-sourcemaps';
import resolve from 'rollup-plugin-node-resolve';
import builtins from 'rollup-plugin-node-builtins';
import globals from 'rollup-plugin-node-globals';
import commonjs from 'rollup-plugin-commonjs';
import json from 'rollup-plugin-json';

export default {

  input: 'src/eve-redux/eve-redux-store.js',

  output: {
    file: 'src/bundle.js',
    name: 'EVE',
    format: 'iife',
  },

  watch: {
    include: 'src/**',
  },

  plugins: [
    resolve(),
    commonjs({
      include: 'node_modules/**',
      ignoreGlobal: true,
      namedExports: {
        'node_modules/inferno-redux/index.js': ['Provider'],
        'node_modules/aws-sdk/global.js': ['util'],
      },
    }),
    sourcemaps(),
    json(),
    builtins(),
    globals(),
  ],

};

@sholladay
Copy link

sholladay commented Oct 26, 2017

Are you able to trick Rollup into converting the SDK to ES modules by doing...

format: 'es'  // instead of 'iife'

?

That might actually work! It would still be bundled, though, presumably. So I guess we'd have to then figure out a way to de-bundle it, in order to get source code to start work on a PR.

@bennypowers
Copy link
Author

Well, rollup doesn't give any errors, although browser complains Uncaught TypeError: Failed to resolve module specifier 'aws-sdk'

bundle.js:1

import { S3 } from 'aws-sdk';

@bennypowers
Copy link
Author

see also #1769

@thesoftwarephilosopher
Copy link

The AWS JS SDK having a native ESM format is especially useful for those of us who are starting to migrate away from "mandatory build phase" web development while still using "modern" libraries like React and HTM. In particular, the forward-thinking Pika tooling says that aws-sdk isn't compatible.

@thesoftwarephilosopher
Copy link

@jeskew After re-reading what you wrote here:

The conversion from ESM to CommonJS, however, is fairly straightforward, so if the SDK were written following ES module semantics, we could release both versions. That, however, would require a major version bump.

It seems like this would be a reasonable plan moving towards this goal:

  1. Rewrite all require() calls as import statements throughout the codebase
  2. Create a build script that converts the newly ESM codebase to UMD for backwards compatibility
  3. Include the raw ESM codebase in new releases so that ESM-based tools can consume it.

This would involve some planning, to ensure that the code behaves the same as before, the execution of this plan happens during the course of a single day, and preparation to create or modify the relevant build tools ahead of time.

But this issue has been open for almost 2 years. I know it's probably not high priority for AWS but ES Modules are already here, the web is ready for this, and making this issue a reality instead of a dream will help projects that use ESM tooling to move forward more easily.

@juliankrispel
Copy link

This would be pretty great especially for usage with deno. I believe the change could be scripted using a tool like jscodeshift would be happy to contribute if it is something that would be considered 👍

@antoniotorres
Copy link

Is there a plan or roadmap to move to ES modules?

@FredKSchott
Copy link

If the data helps: We're seeing more people having trouble using aws-sdk with Snowpack, Rollup, and ESM in general. Shipping native es modules would go a long way in unblocking these users!

@wperron
Copy link

wperron commented Sep 27, 2020

I'm also interested in using this in Deno, though that would require more than simply using ESmodules syntax; imports would need to be re-written to include file extensions as well (which also means re-writing some imports to import /index.js rather than the directory name).

Any ETA available yet on this issue?

@sebiniemann
Copy link

Are there any help wanted tickets as first steps? I would like to contribute towards this goal 😃

Now that AWS Lambda supports NodeJS 14 and also loads native JS modules (as long as "type": "module" has been set within an additionally uploaded package.json), it would be great to do something like this

import * as AWS from 'aws-sdk';

That this is not possible is in fact the only reason why we have been (automatically) downgrading our entire backend to the require syntax for years 😅 (and also the reason why we have to include and maintain a build system like Rollup at all).

@sholladay
Copy link

@sebiniemann if you're using "type": "module", you can actually do that already. You probably want to remove the * as part, though.

Unfortunately, since the AWS SDK is still CommonJS, there are no named exports, so doing a wildcard import means you have to use AWS.default everywhere. If you instead do import AWS from ..., then you can just use AWS.

@sebiniemann
Copy link

sebiniemann commented Feb 4, 2021

Thanks for the feedback :)

We just tried this (including without * as), but the pre-installed SDK on AWS Lambda seems not to be compatible (or we are still missing something):

"Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /var/task/index.js",
"require() of ES modules is not supported",
"trace": [
"Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /var/task/index.js",
"require() of ES modules is not supported.",
"require() of /var/task/index.js from /var/runtime/UserFunction.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.",
"Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /var/task/package.json.",
"",
" at Object.Module._extensions..js (internal/modules/cjs/loader.js:1080:13)",
" at Module.load (internal/modules/cjs/loader.js:928:32)",
" at Function.Module._load (internal/modules/cjs/loader.js:769:14)",
" at Module.require (internal/modules/cjs/loader.js:952:19)",
" at require (internal/modules/cjs/helpers.js:88:18)",
" at _tryRequire (/var/runtime/UserFunction.js:75:12)",
" at _loadUserApp (/var/runtime/UserFunction.js:95:12)",
" at Object.module.exports.load (/var/runtime/UserFunction.js:140:17)",
" at Object. (/var/runtime/index.js:43:30)",
" at Module._compile (internal/modules/cjs/loader.js:1063:30)"
]

We created a basic NodeJS 14 Lambda (no layers or additional configuration) with the following minimal code (besides the package.json for module support):

import AWS from 'aws-sdk';

async function handler() {
    return {
        statusCode: 200,
        body: JSON.stringify('Hello from Lambda!'),
    };
};

export {
    handler,
};

// EDIT

We just tried it without import AWS from 'aws-sdk'; and got the same error. It seems as if the AWS Lambda NodeJS 14 environment itself is not ready for ES6 imports.

@sholladay
Copy link

That error implies that something is transpiling your code or at least converting your import statements to require() calls. Make sure you get rid of any bundlers or other tooling that might be doing that. You might be right that Lambda just isn't ready for this yet, but it's more likely something else.

@sebiniemann
Copy link

The code was just pasted into the AWS Lambda editor. No build process involved (at least from our side).

@sebiniemann
Copy link

sebiniemann commented Feb 14, 2021

After consulting with AWS support, it was confirmed that AWS Lambda is currently not compatible with "type": "module" and therefore not ready for ES6 imports.

It would be possible to be compatible with both CommonJS and ECMAScript Modules by adding another package.json that does not contain "type": "module", explicitly indicating that CommonJS is used within AWS' own Lambda runtime scripts. As far as I could see, this would need to be placed outside of /var/task (in /var, based on the error message). However, it seems that this is unfortunately not possible for the user.

@pfried
Copy link

pfried commented Jan 19, 2022

@sebiniemann https://aws.amazon.com/about-aws/whats-new/2022/01/aws-lambda-es-modules-top-level-await-node-js-14/?nc1=h_ls

@ignoramous
Copy link

It would be possible to be compatible with both CommonJS and ECMAScript Modules by adding another package.json that does not contain "type": "module",

This is we did with package.json w/ "type": "module", and it seems to work for S3 if nothing else:

import * as awscjs from 'aws-sdk';

const AWS = awscjs.default;

const S3 = new AWS.S3(...

Unsure if there are any pitfalls here.

@sholladay
Copy link

sholladay commented Oct 17, 2022

@ignoramous you can simplify your code to this:

import AWS from 'aws-sdk';

const S3 = new AWS.S3();

That works ^^

The thing that's unfortunately not currently supported is named imports:

import { S3 } from 'aws-sdk';

const S3 = new S3();

@btakita
Copy link

btakita commented Jan 20, 2023

Please add ESM support. Lack of ESM support is a major stumbling block in using this library, particularly with vite. If you have to do a major version upgrade, please do so.

I'm getting the following error when attempting to use a dynamic import to support isomorphic javascript using vite & solid-js:

    at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/aws-sdk@2.1298.0/node_modules/aws-sdk/lib/aws.js:1:14
    at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/vite@4.0.4_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15)
    at eval (/src/_esg/urth__ctx_.ts:30:15)
Trace: ReferenceError: require is not defined
    at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/aws-sdk@2.1298.0/node_modules/aws-sdk/lib/aws.js:1:14
    at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/vite@4.0.4_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15)
    at eval (/src/_esg/mi_eco_20221130__ctx_.ts:34:15)
An unhandled error occured: ReferenceError: require is not defined
    at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/aws-sdk@2.1298.0/node_modules/aws-sdk/lib/aws.js:1:14
    at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/vite@4.0.4_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15)
An unhandled error occured: ReferenceError: require is not defined
    at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/aws-sdk@2.1298.0/node_modules/aws-sdk/lib/aws.js:1:14
    at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/vite@4.0.4_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15)
An unhandled error occured: ReferenceError: require is not defined
    at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/aws-sdk@2.1298.0/node_modules/aws-sdk/lib/aws.js:1:14
    at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/vite@4.0.4_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15)
An unhandled error occured: ReferenceError: require is not defined
    at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/aws-sdk@2.1298.0/node_modules/aws-sdk/lib/aws.js:1:14
    at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/vite@4.0.4_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15)
An unhandled error occured: Error: Timed out after 10000ms.
    at promise_timeout (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/@ctx-core+function@21.6.0/node_modules/@ctx-core/function/src/promise_timeout/index.js:1:60)
    at Proxy.subscribe_wait (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/@ctx-core+nanostores@2.2.13/node_modules/@ctx-core/nanostores/src/subscribe_wait/index.js:15:30)
    at esg_mi_eco__preload (/home/brian/work/portfoliome/censible-core/apps/mi-web/src/_http/preload.ts:19:2)
    at async Object.handle (/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/itty-router@3.0.11/node_modules/itty-router/dist/itty-router.js:1:658)
    at async handleRequest (/home/brian/work/portfoliome/censible-core/apps/mi-web/src/entry-server.tsx:49:8)
    at async eval (/home/brian/work/portfoliome/censible-core/vendor/solid-start/packages/start/entry-server/StartServer.tsx:45:11)
    at async devFetch (file:///home/brian/work/portfoliome/censible-core/vendor/solid-start/packages/start/dev/server.js:46:12)
    at async startHandler (file:///home/brian/work/portfoliome/censible-core/vendor/solid-start/packages/start/dev/server.js:115:20)

Using createRequire renders it unable to inline the library to the lambda deployment using vite. Unless there is another solution, this will potentially require major reengineering. Perhaps there are example repos out there with solutions to these problems.

@colelloa
Copy link

Please add ESM support. Lack of ESM support is a major stumbling block in using this library, particularly with vite. If you have to do a major version upgrade, please do so.

I'm getting the following error when attempting to use a dynamic import to support isomorphic javascript using vite & solid-js:

    at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/aws-sdk@2.1298.0/node_modules/aws-sdk/lib/aws.js:1:14
    at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/vite@4.0.4_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15)
    at eval (/src/_esg/urth__ctx_.ts:30:15)
Trace: ReferenceError: require is not defined
    at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/aws-sdk@2.1298.0/node_modules/aws-sdk/lib/aws.js:1:14
    at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/vite@4.0.4_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15)
    at eval (/src/_esg/mi_eco_20221130__ctx_.ts:34:15)
An unhandled error occured: ReferenceError: require is not defined
    at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/aws-sdk@2.1298.0/node_modules/aws-sdk/lib/aws.js:1:14
    at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/vite@4.0.4_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15)
An unhandled error occured: ReferenceError: require is not defined
    at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/aws-sdk@2.1298.0/node_modules/aws-sdk/lib/aws.js:1:14
    at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/vite@4.0.4_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15)
An unhandled error occured: ReferenceError: require is not defined
    at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/aws-sdk@2.1298.0/node_modules/aws-sdk/lib/aws.js:1:14
    at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/vite@4.0.4_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15)
An unhandled error occured: ReferenceError: require is not defined
    at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/aws-sdk@2.1298.0/node_modules/aws-sdk/lib/aws.js:1:14
    at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/vite@4.0.4_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15)
An unhandled error occured: Error: Timed out after 10000ms.
    at promise_timeout (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/@ctx-core+function@21.6.0/node_modules/@ctx-core/function/src/promise_timeout/index.js:1:60)
    at Proxy.subscribe_wait (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/@ctx-core+nanostores@2.2.13/node_modules/@ctx-core/nanostores/src/subscribe_wait/index.js:15:30)
    at esg_mi_eco__preload (/home/brian/work/portfoliome/censible-core/apps/mi-web/src/_http/preload.ts:19:2)
    at async Object.handle (/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/itty-router@3.0.11/node_modules/itty-router/dist/itty-router.js:1:658)
    at async handleRequest (/home/brian/work/portfoliome/censible-core/apps/mi-web/src/entry-server.tsx:49:8)
    at async eval (/home/brian/work/portfoliome/censible-core/vendor/solid-start/packages/start/entry-server/StartServer.tsx:45:11)
    at async devFetch (file:///home/brian/work/portfoliome/censible-core/vendor/solid-start/packages/start/dev/server.js:46:12)
    at async startHandler (file:///home/brian/work/portfoliome/censible-core/vendor/solid-start/packages/start/dev/server.js:115:20)

Using createRequire renders it unable to inline the library to the lambda deployment using vite. Unless there is another solution, this will potentially require major reengineering. Perhaps there are example repos out there with solutions to these problems.

+1 to this comment - does AWS not want this project to be widely adopted? this is a killer for Vite users....

@btakita
Copy link

btakita commented Apr 24, 2023

The @aws-sdk/* libraries are published with esm.

@kellertk
Copy link
Contributor

Hello, the v3 of the SDK has native ES modules. We have no plans to implement this in v2.

@kellertk kellertk closed this as not planned Won't fix, can't repro, duplicate, stale Apr 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature-request A feature should be added or improved. needs-major-version Can only be considered for the next major release
Projects
None yet
Development

No branches or pull requests