Instanced Functions

[lkingland]

Instanced Functions

Discussed herein is the background context for Function Instances before outlining the Instanced Function method signatures, and a preliminary implementation.

Background

Functions are, from the perspective of the runtime environment, a running network service. This service is loaded into memory when the autoscaler creates a new instance of the service, and requests are routed to the service for processing until such time as idleness leads the autoscaler to potentially release the resources by killing the process.

It is for this reason that a Function, from the perspective of a developer, be considered "stateless." A Function should not write any important data to its local storage, because the Function instance could end at any time, erasing its RAM and local storage.

However, this transience is sometimes misunderstood to be an extreme version of statelessness where a Function is expected to be instantiated (its entire process and memory space) for each request; subsequently exiting the process and releasing all resources after the request is handled.

All programs have some state when running: variables, open connections, etc. Perhaps a more accurate term for the situation is "ephemeral." A Function can of course have state, or even write to local storage, but none of this data should be important to retain if the instance is scaled down or dies unexpectedly. A Function can, and should, be thought of as an evolution of a full Service, retaining all its benefits while simplifying development and deployment. A few simple examples being caching, large-memory-footprint services, and services needing persistent connections.

An ephemeral (in memory) cache is an important way to improve service execution times. Only with instanced, re-entrant Functions is an ephemeral cache possible. If the process exits for every request, there is nowhere local to store the cache for subsequent requests.

Services with a large memory footprint, such as an AI service loading its inference model, need only perform the slow task of loading their model into memory when the service is instantiated. Handling individual requests is comparatively fast. This is only possible with instanced Functions. Otherwise the model must be reloaded for every request.

Services with persistent connections, such as database connections, have a somewhat slow initial setup time, but can process individual operations comparatively quickly. Without Instanced Functions, the connection would need to be torn down and recreated for every request.

I hope it is clear that terminating a Function instance is no different than deploying a website that starts a new webserver for every HTTP request. So how did this seemingly expectation come to be?

The confusion is due of course first to early implementations. However, it persists because method signatures expected by the Function platform authors have hitherto always been static. Therefore, to treat them like the running service they are could only be accomplished using global variables. This is generally an antipattern, so it is only logical that an extreme version of isolation and statelessness would be assumed. The static signature implies that the incoming request will have the entire memory space to itself; the only time a global variable might be warranted.

Therefore it is prerequisite to creating performant, reentrant services using Functions that the method signatures provided to the developer accurately represent the underlying resource allocation and available features. This means enabling developers to create Function instances, using appropriate method signatures, rather than relying on static methods only.

Signatures

Below are the proposed minimum set of function signatures to support for each language in pseudocode. The actual signature will differ by language:

Handle()	         # Static Base Case
Handle(request) response # Static HTTP Handler
Handle(event) event 	 # Static CloudEvent Handler

f = New() Function	 # Constructor for a Function Instance
f.Handle(req) res	 # instanced version of HTTP request handler
f.Handle(event) event	 # instanced version of CloudEvent handler

This set of functions includes both the currently supported static and the proposed instanced functions (for HTTP and CloudEvents). Also included are two “base case” signatures which have no arguments and no return (akin to a fire-and-forget job). Some languages will offer additional signatures and features such as mapping input data directly to language-native data structures,
etc. Some libraries, such as the Go CloudEvents library, offer many more supported signatures. However, these extended signatures are not included in this base minimum set to be supported by all languages. By keeping the expected base set small, we will greatly improve our odds at correctness and compatibility.

Example

To see how this works in action, an initial implementation has been completed for the Go language, and is demonstrated as a screencast.

Summary

Input on this initiative would be greatly appreciated. It is no exaggeration to say that defining the base set of method signatures required for a Function to be deployed by the toolkit is one of the most important decisions we can make. This is the contract between the Functions Toolchain and Functions Developers.

Please see the associated Epic Issue #2023 where individual tasks are in the process of being enumerated.