This contains a few different builder manager fixes:
* Update the zip file structure to avoid an extra subdirectory
* Annotate packages with what functions are using them
* Only trigger builds when there is no deployment archive
* Python environment loadpath bugfixes
* Other bugfixes
Add a flag to the environment to control multiple specialization -- the max number of functions per container. This can be set to 1 or infinity.
Add an api proxy to workflow apiserver from the controller.
Add function metadata to FunctionLoadRequest; every v2 environment now knows which function it's loading (but can ignore that information if it wants to).
Add function identity headers to router. This is useful for multiple specialization, so the router can disambiguate between different function calls. (If this turns out to be a non-trivial perf overhead, we could add these headers conditionally, but for now they are always added.)
This change orchestrates function builds.
Environments (in v2) define a builder image, just like they do a runtime image. The builder image contains a build script that's invoked with source and deployment paths (as env vars).
The buildermgr watches for environments with build images defined, and creates build deployments and services.
Functions can define source and deployment. Buildermgr watches for functions with source code (and build status == pending) and invokes the environment's builder when appropriate. It captures logs from the build and sets the build status (success/failure) and build lots into the PackageStatus.
Introduce _Archives_ as a type to reference arbitrary blobs.
_Packages_ are a pair of Source and Deployment archives. Packages have an environment reference.
Functions reference a package. Multiple functions can reference the same package.
Split out the Package type into a first class Kubernetes resource. Before this change, packages were implicitly tied to functions.
This wasn't ideal because:
* Functions will need to share packages
* A package storage system may be more generally useful than just
functions (for example, for storing static assets)
This change does the following:
* Updates the fission and tpr types to add a new Package and PackageSpec. It also creates a PackageRef type, and a FunctionPackageRef type. The PackageRef simply references a package, but the FunctionPackageRef includes the name of a function within the package. This allows us to share packages between different functions.
* Updates fetcher and other components for first-class packages
* Allows customization of fetcher image pull policy in the helm charts
This changes the core fission function, environment and trigger types. It also changes Fission's storage to use ThirdPartyResources.
- Functions are now specified by packages. Functions can also have both source and deployment packages. A package can be specified by a literal, or by a URL.
- Environments have a build and runtime component.
- Triggers reference functions by a FunctionReference. This is a layer of indirection between triggers and functions, and will allow things like incremental function upgrades in future releases.
See Documentation/wip/env-v2.md for design discussion about points 1 and 2.
Changes:
* V2 Types
All types now have a spec, following the pattern of K8s objects.
Functions now have source and deployment packages. A Package can be
specified by literal, or by URL.
Environments now have a builder and runtime component.
All triggers use a new FunctionReference to specify the function. This
for now only uses a function name, but in the future can be extended
to be more flexible.
A new FunctionLoadRequest type is added for specialization requests to
the environment runtime.
* TPR types, TPR init code, and a "fission client"
Implements TPR types using the spec types in fission/types.go.
Adds code for adding creating TPR types, and convenient types for crud
operations on each of our resource types.
Adds code for connecting to K8s API and configuring a REST client with
fission types set up.
* Change old stateful controller into a thin apiserver
This apiserver is now simply a stateless api layer on top of the TPR
types. At the moment it doesn't do anything that couldn't be done by
simply talking to the TPR types. In the future we can have better
validation and potentially some higher level APIs (like versioning for
example) in here.
* Split controller client into files and update for v2 types.
* Update CLI for v2 types.
As far as possible we keep the CLI flags the same. We'll have to add
flags for source/deploy packages and builder/runtime
environments. That will come in the next change.
* Update poolmgr and fetcher for v2 types.
Also adds a poolmgr_test.
* Update router for new types.
Also adds a function reference resolver, which separates out the job
of resolving a FunctionReference to a function.
* Update kubewatcher and timer for v2 types.
* Update Message Queue trigger type for v2 types.
* Minor odds and ends.
* Fission bundle CLI updates
Remove controllerUrl flag, since we don't need it any more.
* Remove etcd deployment (replaced by storing state in TPR)
Also update the poolmgr commandline, and use an env var for the
fetcher image URL.
* Explicit ChecksumType and consts
* Clarify separation of environment interface types
Adds support for message queue triggers based on the NATS-Streaming message queue.
This lets the user map a queue topic to a function, and optionally send the function's response into another queue topic.
The message queue trigger manager is designed to be message queue independent, so we should be able add support for more queues by adding implementations for the relatively simple mqtrigger.MessageQueue interface.
This change adds a timer trigger API, client, and implementation. Users can trigger a function with a cron-compatible string.
This change also moves the publisher interface and webhook-based publisher implementation out of kubewatcher and into a separate `publisher` package.
Motivation: The output when an error occurs shows an integer error code,
which isn't very helpful. And in the case of an error when creating a
resource, you also get a redundant log message from the controller
client.
Modifications:
* the controller client no longer logs errors
* the generated String() allows for the enum name to be displayed in the
error output
* each errorCode enum member has a hand-generated description
* the format of the error message from the CLI client was modified to
display the error description with the error message
Related to #112
