Ability to use against devDependencies

[rmmeans]

Hi! First off, thank you so much for this great project! I noticed on Gitter that devDependencies (https://gitter.im/ScalablyTyped/community?at=5e36b2586f9d3d349817a0da) are not parsed when using ScalablyTypedConverterExternalNpmPlugin.

I have a use case that needs this. Let me explain -

We are making heavy use of Scala.js on AWS Lambda. On lambda, it is not uncommon to run into situations where a dependency is provided via a Lambda Layer or by the lambda runtime itself (e.g. the aws-sdk). Since NPM doesn't have what we have in maven land with a provided scope, Lambda developers will use devDependencies for resources that they actually do need at runtime, but they don't need built into their output source of their lambda artifact since the npm package they need will be provided either by the runtime or via a lambda layer.`

Any ideas on how we can make it possible to allow ScalablyTyped to parse devDependencies, or named devDependencies via a config option?

[oyvindberg]

Hi there and thanks. This surfaced also in #340 now, so it'll be fixed somehow.
Just for my understanding, is there a problem if you just add these dependencies as normal npm dependencies?

[armanbilge]

We are making heavy use of Scala.js on AWS Lambda.

@rmmeans Very cool! Besides ST, are you using any frameworks for this? We're currently working on a concept for this in Typelevel. If you have a moment, would be awesome to get your feedback on this :)
https://gist.github.com/djspiewak/37a4ea0d7a5237144ec8b56a76ed080d

[rmmeans]

Hi there and thanks. This surfaced also in #340 now, so it'll be fixed somehow.
Just for my understanding, is there a problem if you just add these dependencies as normal npm dependencies?

@oyvindberg The problem is minor - and that is bundle size which has impact on coldstart times. On Lambda, the runtime your code runs on will already have some dependencies for you, things such as the aws-sdk for Javascript will already be on the lambda runtime for node. The aws-sdk can be quite large and you are duplicating what the runtime already has if you specify your own. Thus, you can avoid pulling it in as a normal npm dependency - you only need it for compile time for typings, but it doesn't need to be bundled up with your assets after running npm prune for production.

[armanbilge]

Bundle size was my concern as well.

[rmmeans]

We are making heavy use of Scala.js on AWS Lambda.

@rmmeans Very cool! Besides ST, are you using any frameworks for this? We're currently working on a concept for this in Typelevel. If you have a moment, would be awesome to get your feedback on this :)
https://gist.github.com/djspiewak/37a4ea0d7a5237144ec8b56a76ed080d

@armanbilge yup! We make HEAVY use of this facade library: https://github.com/exoego/aws-sdk-scalajs-facade it's very lightweight and feels pretty natural from an AWS API point of view. At some point this fall, I hope to do a writeup about our experience of building a large serverless app using Scala.js.

At present, I can say this in summary about our work of using Scala for serverless:

• Mono Repo with 4 backend sub-projects, and one front-end sub-project. Both front-end and back-end is built on Scala.js with a few Scala JVM lambda's running for a specific use cases (using sbt cross project for these)
• Common subproject for type system that share common data types from backend and frontend. Serialization of shared types between FE and BE is done with Circe.
• Using AWS SAM for backend deployment pipelines of each of the 4 sub projects (each sub-project is its own SAM stack deployed independently). Plain scala.js (no scalajs bundler) builds the backend components
• 60+ lambda functions across the 4 backend sub projects, ~150 cloud watch alarms, 30+ SQS FIFO queues, etc
• Architecture is event driven and follows domain driven design patterns. There is SQS FIFO queues all over the place tieing the lambda functions together between Dynamo Tables with dynamodb streams, etc. I think we have ~ 6 aggregates and 3 saga's now in the full domain.

I think the more tutorials that can be written on how to do large scale scala.js on lambda, the better! The get started templates can be useful but I don't think they are practical for application unless the show how to use common serverless tooling such as Serverless Framework or AWS SAM. In reality, serverless apps have just as many, if not more native cloud resources in them as lambda's themselves - we should embrace that workflow as a scala community and show how to use scala.js with AWS SAM or Serverless framework.

For us, once we got the core structure ironed out for our mono repo and the first commits rolling up through a pipeline, on-boarding new team members, etc hasn't been much different than any other scala project.

[armanbilge]

Thank you for taking the time to write this all up!! Very very helpful to hear your experience with this.

show how to use scala.js with AWS SAM or Serverless framework.

What are your thoughts an SBT plugin that handles this deployment aspect?

[oyvindberg]

Fixed this now in 88ec013 , the next release will have an stIncludeDev flag

[oyvindberg]

@rmmeans hey that's very interesting. Looking forward to a blog post on that :)

also if you have any feedback on what it would take for for instance the generated aws-sdk (or the modular version if you prefer) to be in the same ballpark as the wrapper you're using that would also be very useful

[armanbilge]

@rmmeans you may find these new projects helpful:
https://github.com/typelevel/feral
https://github.com/disneystreaming/smithy4s