Skip to content
This repository has been archived by the owner on Dec 15, 2021. It is now read-only.

nodejs, remove runtime internal data from sandbox, avoid code concat #633

Merged
merged 6 commits into from
Mar 19, 2018
Merged

nodejs, remove runtime internal data from sandbox, avoid code concat #633

merged 6 commits into from
Mar 19, 2018

Conversation

cscheiber
Copy link

@cscheiber cscheiber commented Mar 14, 2018

Issue Ref:
#632
Description:
Please see issue for details.

TODOs:

  • Ready to review
  • Automated Tests
  • Docs

@murali-reddy murali-reddy force-pushed the function-trigger-runtimes branch 2 times, most recently from 09cb7f7 to c4ad19a Compare March 15, 2018 10:55
Copy link
Contributor

@andresmgot andresmgot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just submitted some small suggestions. The approach looks good so far :)

else if (libDeps.includes(p))
return require(path.join(libPath, p));
else if (p === 'kubeless')
return (handler) => modExecute(handler, req, res, end);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here is where we could add the possibility of importing different functions. What I have tried:

    else if (p === 'kubeless')
        return (handler) => {
            let func = null
            if (typeof handler === "function") {
                func = handler;
            } else if (typeof handler === "object") {
                func = handler[funcHandler];
            } else {
                throw new Error(`Unable to load ${handler}`);
            }
            modExecute(handler, req, res, end)
        };

Copy link
Author

@cscheiber cscheiber Mar 15, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, would work. Map could also be supported. Better check also for null, e.g.

} else if (handler && typeof handler === "object") {

Just a question: if there are multiple functions in one file, would kubeless also allow to run it in the same pod, e.g. take the FUNC_NAME from request and use it as key? Or is it just about developers convenience to use one file.

Copy link
Author

@cscheiber cscheiber Mar 15, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And maybe put (p === 'kubeless') to the begin to override any 'kubeless' from functions package.json.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

modExecute(func, req, res, end)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a question: if there are multiple functions in one file, would kubeless also allow to run it in the same pod, e.g. take the FUNC_NAME from request and use it as key? Or is it just about developers convenience to use one file.

The second, it is just for convenience but only one function will run per pod.

'event-namespace': req.get('event-namespace'),
'data': req.body.toString('utf-8'),
'extensions': { request: req },
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you missed the data parsing here:

        if (req.get('content-type') === 'application/json') {
            event.data = JSON.parse(event.data)
        }

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, could be a helper function.

'data': req.body.toString('utf-8'),
'extensions': { request: req },
};
const context = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would move the context initialization out of the modExecute function so it is executed just once (instead of one per request). That way we can save some ms.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. And at this point we could also check the Promise callbacks. Instead of (err) => { handleError(err); } just err => handleError(err, ...) at least under node 6 a little faster to my knowledge. Of course also the typical case, the success with then result => handleResult(result) using an additional plain function.

@murali-reddy murali-reddy force-pushed the function-trigger-runtimes branch from c4ad19a to 01518b7 Compare March 15, 2018 11:59
@cscheiber
Copy link
Author

Added a commit according to comments:

  • handler check is done inside modExecute, to keep concise arrow function body
  • timer end is also called in handleError()

@andresmgot
Copy link
Contributor

andresmgot commented Mar 15, 2018

Thank you @cscheiber. The code LGTM. Now we need some testing before we merge this. Could you:

  • Update the examples under ./examples/nodejs/*
  • Build images for node6 and node8 and push them to your personal repository (or any public docker hub repository)
  • Modify the references in kubeless.jsonnet for the images you just pushed
  • Extra: In the file ./examples/Makefile there is a task timeout-nodejs that writes a nodejs function with an infinite loop, you will need to adapt it to the new format.

Once you do the above the CI will test everything with your changes included.

@cscheiber
Copy link
Author

cscheiber commented Mar 16, 2018

As discussed I adjusted the examples, but did not commit so far. The file 'index.js' leads to a question: if module.exports=myHandler is replaced by kubeless(myHandler) in the single files - how to deal with index.js in general?

@cscheiber
Copy link
Author

To summarize today's discussions:

  • one code line will be generated at the end of the main file require('kubeless')(module.exports);
  • function file structure does not need to change
  • index.js can be used, however it is not recommended to provide e.g. 50 functions with one index file where runtime will then select just the one to be used
  • sandbox works only for the main file, not for any required module (unchanged behavior)

A final commit has been provided.

@andresmgot
Copy link
Contributor

@cscheiber I have pushed the images to my repository, do you mind changing the reference of the images in the kubeless.jsonnet file to the following?

    "ID": "nodejs",
    "versions": [
      {
       	"name": "node6",
        "version": "6",
        "runtimeImage": "andresmgot/nodejs@sha256:37d496adee0896a4a192f7116d8075ae88ee14b1a3c41974d79609e0863ae464",
        "initImage": "node:6.10"
      },
      {
        "name": "node8",
	"version": "8",
        "runtimeImage": "andresmgot/nodejs@sha256:4327d0f7781463a8b0a843000592c2c0df17988662b38b33dbb35b1bdd5ccc2b",
        "initImage": "node:8"
      }
    ],

@murali-reddy murali-reddy force-pushed the function-trigger-runtimes branch 2 times, most recently from de9e4e9 to 358f8cb Compare March 18, 2018 17:15
@cscheiber
Copy link
Author

The file has been updated accordingly. Again, sorry that I had no own repository available for now.

@andresmgot
Copy link
Contributor

Everything green in the CI so I pushed the images to the kubeless repository. I will leave the CI to run one more time and I will merge the changes. Thanks @cscheiber !

@andresmgot andresmgot merged commit 81cba44 into vmware-archive:function-trigger-runtimes Mar 19, 2018
murali-reddy added a commit that referenced this pull request Mar 22, 2018
* CRD API objects  for functions, http, kafka, cronjob triggers and
corresponding auto-generated informer, listers, clienset and deep copy
generated function

* CRD controllers for the function, kafka triggers, cronjob triggers, http
triggers CRD objects.

* new server side binaries kubeless-controller-manager and kafka-controller

* updated CLI to incorporate implict creation of triggers with
"kubeless function deploy" and move ingress object creation logic
to http trigger controller

* updated to build scripts

* updated to manifests

Co-authored-by: Andres <andres.mgotor@gmail.com>
Co-authored-by: Tuna <ng.tuna@gmail.com>

* Updates to examples used in integration tests
Co-authored-by: Andres <andres.mgotor@gmail.com>
Co-authored-by: Tuna <ng.tuna@gmail.com>

* kafka even consumer with new go kafka lib dependencies
Co-authored-by: Tuna <ng.tuna@gmail.com>

* update vendor libs
Co-authored-by: Tuna <ng.tuna@gmail.com>

* update docker files
Co-authored-by: Tuna <ng.tuna@gmail.com>
Co-authored-by: Andres <andres.mgotor@gmail.com>

* updates to utils and langruntime

* update .travis.yaml

* delete old controller docker file

* better kafka even consumer logging

* fix travis ci issue related to kafka broker

* keep the old GKE version 1.7.12-gke.1

* Use bitnami-bot docker account

* Fix http request method for the Kafka consumer

* Push Kafka image

* Fix content-type in consumer HTTP message

* Fix GKE version

* Removed duplicated files

* Update runtime docs

* Do proper check for topic update

* skip IT test with trigger update through

* Add CPU limit argument to function commands (#606)

* Add CPU limit argument to function commands

* Fix doc typo

* Add better description to --cpu argument

* fix glide.lock and vendor libs, that got messed up with master merge

* addressing review comments

* update .gitignore with new controller images

* Review

* Review

* use 3-way merge Patch only to update the CRD objects

* nodejs, remove runtime internal data from sandbox, avoid code concat (#633)

* nodejs, remove runtime internal data from sandbox, avoid code concat

* update according to comments

* keep compatibility with module.exports

* adjust kubeless.jsonnet

* Move images to the kubeless repository

* Revert "use 3-way merge Patch only to update the CRD objects"

This reverts commit 358f8cb.

* Avoid reflect.DeepEqual for deployment comparison

* Fix empty body for NodeJS functions

* fix GKE CI failures due to upstream issue kubernetes/kubernetes#60538

* Fix error catching in Python runtime. Remove old files

* Send message as jsons if necessary in kafka-consumer

* Move serviceSpec to function object. Fix service modifications
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants