This change contains CLI support for uploading large functions to the storage service.
It also adds a reverse proxy into the storage service to the fission API.
The helm chart is not yet updated to actually run the storage service -- that will come in the next change.
A very rudimentary storage service for archives larger than what can
fit in TPR/CRD resources. The library used allows for using local
files, AWS S3, or other cloud storage APIs. For now only local paths
are set up.
The storage service doesn't know anything about functions or packages;
it can be used for arbitrary archives.
There's no CLI integration yet.
This changes the core fission function, environment and trigger types. It also changes Fission's storage to use ThirdPartyResources.
- Functions are now specified by packages. Functions can also have both source and deployment packages. A package can be specified by a literal, or by a URL.
- Environments have a build and runtime component.
- Triggers reference functions by a FunctionReference. This is a layer of indirection between triggers and functions, and will allow things like incremental function upgrades in future releases.
See Documentation/wip/env-v2.md for design discussion about points 1 and 2.
Changes:
* V2 Types
All types now have a spec, following the pattern of K8s objects.
Functions now have source and deployment packages. A Package can be
specified by literal, or by URL.
Environments now have a builder and runtime component.
All triggers use a new FunctionReference to specify the function. This
for now only uses a function name, but in the future can be extended
to be more flexible.
A new FunctionLoadRequest type is added for specialization requests to
the environment runtime.
* TPR types, TPR init code, and a "fission client"
Implements TPR types using the spec types in fission/types.go.
Adds code for adding creating TPR types, and convenient types for crud
operations on each of our resource types.
Adds code for connecting to K8s API and configuring a REST client with
fission types set up.
* Change old stateful controller into a thin apiserver
This apiserver is now simply a stateless api layer on top of the TPR
types. At the moment it doesn't do anything that couldn't be done by
simply talking to the TPR types. In the future we can have better
validation and potentially some higher level APIs (like versioning for
example) in here.
* Split controller client into files and update for v2 types.
* Update CLI for v2 types.
As far as possible we keep the CLI flags the same. We'll have to add
flags for source/deploy packages and builder/runtime
environments. That will come in the next change.
* Update poolmgr and fetcher for v2 types.
Also adds a poolmgr_test.
* Update router for new types.
Also adds a function reference resolver, which separates out the job
of resolving a FunctionReference to a function.
* Update kubewatcher and timer for v2 types.
* Update Message Queue trigger type for v2 types.
* Minor odds and ends.
* Fission bundle CLI updates
Remove controllerUrl flag, since we don't need it any more.
* Remove etcd deployment (replaced by storing state in TPR)
Also update the poolmgr commandline, and use an env var for the
fetcher image URL.
* Explicit ChecksumType and consts
* Clarify separation of environment interface types
Adds support for message queue triggers based on the NATS-Streaming message queue.
This lets the user map a queue topic to a function, and optionally send the function's response into another queue topic.
The message queue trigger manager is designed to be message queue independent, so we should be able add support for more queues by adding implementations for the relatively simple mqtrigger.MessageQueue interface.
This change adds a timer trigger API, client, and implementation. Users can trigger a function with a cron-compatible string.
This change also moves the publisher interface and webhook-based publisher implementation out of kubewatcher and into a separate `publisher` package.
Add function log aggregation and persistence using Fluentd and InfluxDB.
Fluentd and a helper sidecar run as a daemonset. The poolmgr sets up logging for each function pod, using the helper sidecar. Fluentd forwards logs to InfluxDB, which is run as a deployment and service. The client CLI directly queries InfluxDB for logs.
Fluentd supports many outputs besides InfluxDB, so we aren't very tied to InfluxDB.
The setup is somewhat manual, which we should be able to improve by integrating this into the helm chart.
Diagram of component interactions: https://cloud.githubusercontent.com/assets/202578/23100399/b0e3ea00-f6ba-11e6-8f2f-6588cfef2e84.png
fission-bundle is a single binary that can be run as one or more of
the router, controller and poolmgr. It's built into one docker image
which can then be run with different commands.
