Add ballerina runtime support #788

anuruddhal · 2018-05-31T09:38:58Z

Issue Ref: #744

Description:
Add Ballerina runtime support to kubeless.

[PR Description]
This PR contains implementation, examples, and integration tests for ballerina runtime.

TODOs:

Ready to review
Automated Tests
Docs

sebgoa · 2018-05-31T13:03:01Z

A short documentation would be great.

anuruddhal · 2018-06-01T06:00:15Z

Added documentation with commit dad7d86.

anuruddhal · 2018-06-04T04:52:00Z

@sebgoa
Ballerina has Prometheus compatible endpoint to collect metrics. Will Kubless be able to collect data if I enable it?
https://ballerina.io/learn/how-to-observe-ballerina-code/

sebgoa · 2018-06-04T08:00:32Z

@anuruddhal if you check the python runtime you will see that we use a prometheus client to collect metrics:

https://github.com/kubeless/kubeless/blob/master/docker/runtime/python/kubeless.py#L20

So yes, if you collect metrics in ballerina and expose them via a /metrics route kubeless will be able to use it for auto-scaling.

andresmgot

thank you for the PR @anuruddhal, great job! I sent a few comments.

Regarding the Prometheus integration we can do like Ruby and PHP, add the Golang proxy to redirect requests (see the Dockerfile and the Makefile build task). That way we can easily get timeout handling (that I think is missing from the current implementation) and Prometheus statistics. This proxy redirect incoming requests to localhost:8090 so that port 8090 should be hardcoded in the ballerina runtime.

andresmgot · 2018-06-04T08:28:11Z

docker/runtime/ballerina/kubeless_run.tpl.bal

+        kubeless:Event event = {};
+
+        //Read environment variables and set them to context
+        context.function_name = config:getAsString("FUNC_HANDLER");


can you move the context initialization outside the handler? So it doesn't need to execute this code every time the function is called.

andresmgot · 2018-06-04T08:32:54Z

docs/runtimes.md

+
+Ballerina functions should import the package `kubeless`. This [package](../docker/runtime/ballerina/kubeless/kubeless.bal) contains two types `Event` and `Context`. 
+
+When using the Ballerina runtime, it is possible to provide the `ballerina.conf` file. The values in conf file is available for the function. 


I think we shouldn't use the flag --dependencies for other thing than installing external dependencies. Since ballerina doesn't require a different file to do so I think it's better to just ignore this flag. If people wants to use a configuration file they can bundle their files in a zip that will be uncompressed.

andresmgot · 2018-06-04T08:34:02Z

kubeless-non-rbac.jsonnet

+       {
+          "name": "ballerina0.970.1",
+          "version": "0.970.1",
+          "runtimeImage": "kubeless/ballerina:0.970.1",


push the images to a temporary repository just to do the first testing, we can push them later on to the kubeless repository.

andresmgot · 2018-06-04T08:36:35Z

pkg/langruntime/langruntime.go

@@ -240,6 +240,8 @@ func (l *Langruntimes) GetBuildContainer(runtime, depsChecksum string, env []v1.
 	case strings.Contains(runtime, "java"):
 		command = appendToCommand(command,
 			"mv /kubeless/pom.xml /kubeless/function-pom.xml")
+	case strings.Contains(runtime, "ballerina"):
+		command = appendToCommand(command, "ls /kubeless/ballerina.conf")


I would throw an error here to make it clear that ballerina doesn't require this file. something like:

return v1.Container{}, fmt.Errorf("Ballerina does not require a dependencies file")

andresmgot · 2018-06-04T08:38:23Z

pkg/langruntime/langruntime.go

@@ -309,6 +311,16 @@ func (l *Langruntimes) GetCompilationContainer(runtime, funcName string, install
 			"cp /kubeless/*.java /kubeless/function/src/main/java/io/kubeless/ && " +
 			"cp /kubeless/function-pom.xml /kubeless/function/pom.xml 2>/dev/null || true && " +
 			"mvn package > /dev/termination-log 2>&1 && mvn install > /dev/termination-log 2>&1"
+	case strings.Contains(runtime, "ballerina"):
+		command = fmt.Sprintf(
+			"mkdir -p /kubeless/kubeless/ && mkdir -p /kubeless/func/ && "+


mkdir -p accepts more than one argument: mkdir -p /kubeless/kubeless/ /kubeless/func/

andresmgot · 2018-06-05T10:17:07Z

pkg/langruntime/langruntime.go

@@ -315,6 +315,7 @@ func (l *Langruntimes) GetCompilationContainer(runtime, funcName string, install
 		command = fmt.Sprintf(
 			"mkdir -p /kubeless/kubeless/ /kubeless/func/ && "+
 				"cp -r /ballerina/files/kubeless/*.bal /kubeless/kubeless/ && "+
+				"ls -altr && "+


I guess this was for debug?

andresmgot

This looks good to me now, thank you again for the contribution!

anuruddhal changed the title ~~Add ballerina runtime~~ Add ballerina runtime support May 31, 2018

anuruddhal force-pushed the master branch from 614fb02 to 8103aeb Compare June 1, 2018 04:43

andresmgot reviewed Jun 4, 2018

View reviewed changes

anuruddhal force-pushed the master branch 2 times, most recently from 76ca944 to aefb48e Compare June 5, 2018 08:30

andresmgot reviewed Jun 5, 2018

View reviewed changes

anuruddhal force-pushed the master branch 5 times, most recently from a0e1053 to e74f3a7 Compare June 6, 2018 07:24

anuruddhal added 7 commits June 6, 2018 13:51

Add ballerina runtime

5b32f5c

change ballerina controller file name

5fc4d61

Add ballerina run time documentation

5121a25

Fix typo

ff1394a

Add proxy to ballerina service

bd43d37

Add kubeless.toml file support

9e679de

Fix test failures with zip file

4dca46a

anuruddhal force-pushed the master branch from e74f3a7 to 4dca46a Compare June 6, 2018 08:21

anuruddhal and others added 3 commits June 6, 2018 15:31

Validate content-length before reading payload

ecb4880

push images to kubeless repository

eff61cf

fix typo

c31ef47

andresmgot approved these changes Jun 6, 2018

View reviewed changes

andresmgot merged commit b06c647 into vmware-archive:master Jun 6, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ballerina runtime support #788

Add ballerina runtime support #788

anuruddhal commented May 31, 2018 •

edited

Loading

sebgoa commented May 31, 2018

anuruddhal commented Jun 1, 2018

anuruddhal commented Jun 4, 2018 •

edited

Loading

sebgoa commented Jun 4, 2018

andresmgot left a comment •

edited

Loading

andresmgot Jun 4, 2018

andresmgot Jun 4, 2018

andresmgot Jun 4, 2018

andresmgot Jun 4, 2018 •

edited

Loading

andresmgot Jun 4, 2018

andresmgot Jun 5, 2018

andresmgot left a comment


		Ballerina functions should import the package `kubeless`. This [package](../docker/runtime/ballerina/kubeless/kubeless.bal) contains two types `Event` and `Context`.

		When using the Ballerina runtime, it is possible to provide the `ballerina.conf` file. The values in conf file is available for the function.

Add ballerina runtime support #788

Add ballerina runtime support #788

Conversation

anuruddhal commented May 31, 2018 • edited Loading

sebgoa commented May 31, 2018

anuruddhal commented Jun 1, 2018

anuruddhal commented Jun 4, 2018 • edited Loading

sebgoa commented Jun 4, 2018

andresmgot left a comment • edited Loading

Choose a reason for hiding this comment

andresmgot Jun 4, 2018

Choose a reason for hiding this comment

andresmgot Jun 4, 2018

Choose a reason for hiding this comment

andresmgot Jun 4, 2018

Choose a reason for hiding this comment

andresmgot Jun 4, 2018 • edited Loading

Choose a reason for hiding this comment

andresmgot Jun 4, 2018

Choose a reason for hiding this comment

andresmgot Jun 5, 2018

Choose a reason for hiding this comment

andresmgot left a comment

Choose a reason for hiding this comment

anuruddhal commented May 31, 2018 •

edited

Loading

anuruddhal commented Jun 4, 2018 •

edited

Loading

andresmgot left a comment •

edited

Loading

andresmgot Jun 4, 2018 •

edited

Loading