Introduce _Archives_ as a type to reference arbitrary blobs.
_Packages_ are a pair of Source and Deployment archives. Packages have an environment reference.
Functions reference a package. Multiple functions can reference the same package.
This change contains CLI support for uploading large functions to the storage service.
It also adds a reverse proxy into the storage service to the fission API.
The helm chart is not yet updated to actually run the storage service -- that will come in the next change.
Split out the Package type into a first class Kubernetes resource. Before this change, packages were implicitly tied to functions.
This wasn't ideal because:
* Functions will need to share packages
* A package storage system may be more generally useful than just
functions (for example, for storing static assets)
This change does the following:
* Updates the fission and tpr types to add a new Package and PackageSpec. It also creates a PackageRef type, and a FunctionPackageRef type. The PackageRef simply references a package, but the FunctionPackageRef includes the name of a function within the package. This allows us to share packages between different functions.
* Updates fetcher and other components for first-class packages
* Allows customization of fetcher image pull policy in the helm charts
This changes the core fission function, environment and trigger types. It also changes Fission's storage to use ThirdPartyResources.
- Functions are now specified by packages. Functions can also have both source and deployment packages. A package can be specified by a literal, or by a URL.
- Environments have a build and runtime component.
- Triggers reference functions by a FunctionReference. This is a layer of indirection between triggers and functions, and will allow things like incremental function upgrades in future releases.
See Documentation/wip/env-v2.md for design discussion about points 1 and 2.
Changes:
* V2 Types
All types now have a spec, following the pattern of K8s objects.
Functions now have source and deployment packages. A Package can be
specified by literal, or by URL.
Environments now have a builder and runtime component.
All triggers use a new FunctionReference to specify the function. This
for now only uses a function name, but in the future can be extended
to be more flexible.
A new FunctionLoadRequest type is added for specialization requests to
the environment runtime.
* TPR types, TPR init code, and a "fission client"
Implements TPR types using the spec types in fission/types.go.
Adds code for adding creating TPR types, and convenient types for crud
operations on each of our resource types.
Adds code for connecting to K8s API and configuring a REST client with
fission types set up.
* Change old stateful controller into a thin apiserver
This apiserver is now simply a stateless api layer on top of the TPR
types. At the moment it doesn't do anything that couldn't be done by
simply talking to the TPR types. In the future we can have better
validation and potentially some higher level APIs (like versioning for
example) in here.
* Split controller client into files and update for v2 types.
* Update CLI for v2 types.
As far as possible we keep the CLI flags the same. We'll have to add
flags for source/deploy packages and builder/runtime
environments. That will come in the next change.
* Update poolmgr and fetcher for v2 types.
Also adds a poolmgr_test.
* Update router for new types.
Also adds a function reference resolver, which separates out the job
of resolving a FunctionReference to a function.
* Update kubewatcher and timer for v2 types.
* Update Message Queue trigger type for v2 types.
* Minor odds and ends.
* Fission bundle CLI updates
Remove controllerUrl flag, since we don't need it any more.
* Remove etcd deployment (replaced by storing state in TPR)
Also update the poolmgr commandline, and use an env var for the
fetcher image URL.
* Explicit ChecksumType and consts
* Clarify separation of environment interface types
Adds support for message queue triggers based on the NATS-Streaming message queue.
This lets the user map a queue topic to a function, and optionally send the function's response into another queue topic.
The message queue trigger manager is designed to be message queue independent, so we should be able add support for more queues by adding implementations for the relatively simple mqtrigger.MessageQueue interface.
* Fix controller timer handler not return when error occurred
* Fix controller not close http response correctly
This PR aims to fix controller not close response correctly. Also, I’ve check all over the project there is no more resource leak issues exist after this patch.
* Fix controller not return immediately when error occurred
* Sort import
Establish a proxy server from the controller to the log database.
Redirect query commands send from client to database then proxy back the db response.
Use parameter binding instead of fmt.Sprintf to prevent SQL injection.
This change adds a timer trigger API, client, and implementation. Users can trigger a function with a cron-compatible string.
This change also moves the publisher interface and webhook-based publisher implementation out of kubewatcher and into a separate `publisher` package.
When the request is handled successfully, json data is returned, this
patch changes the response Content-Type header from text/plain to
application/json
Fixes: #144
If no uid is provided, all versions of function files and function metadata will be deleted. If a uid is provided, only that specific version is deleted, and the "latest" version of the function moves back to the previous one.
Fixes#152
Motivation: The output when an error occurs shows an integer error code,
which isn't very helpful. And in the case of an error when creating a
resource, you also get a redundant log message from the controller
client.
Modifications:
* the controller client no longer logs errors
* the generated String() allows for the enum name to be displayed in the
error output
* each errorCode enum member has a hand-generated description
* the format of the error message from the CLI client was modified to
display the error description with the error message
Related to #112
Motivation: Creating duplicate resources currently results in a 500
error being displayed by the client, with no information about the true
nature of the failure.
Modifications:
* ResourceStore now converts any errors from the etcd client into
fission errors, capturing the reason for the error in the case of a
duplicate key (as ErrorNameExists)
* ErrorNameExists errors are signaled to the client with a
409 (Conflict) HTTP status
* MakeErrorFromHTTP() now reads the body of the error response to
retrieve the actual error message instead of using the HTTP status
message
* the controller client now uses MakeErrorFromHTTP() to centralize
status code -> error code mapping
* tests for all of the resources now check that duplicate resources are
reported properly
Result: The client can now provide more context when a duplicate name is
given
related to #112
Environments for functions. For now this is just the image URL for the
function run container. In the future, Environments will also have
build container image URLs, and perhaps parameters to those
containers.
Create/read/update/delete/list. Versioning will be added later. For
now, only the latest update is stored.
This implemenents create/read/update/delete/list for http triggers.
This isn't quite the final implementation: you can't look up old
versions by UUID since it only stores the latest update. I'll get to
that later.
FunctionStore uses fileStore for function code and resourceStore for
the Function's metadata. Environment- and HTTPTrigger- Store are just
thin wrappers on resourceStore.
There is probably a better way to organize this code, maybe with
reflection.
- A generic Resource interface, which is serializable and has
a key() method to get an identifying key.
- A resourceStore, which has create, read, update, delete and list.
This wraps etcd key operations and serialization/deserialization in
a convenient service. Serialization is provided to the
resourceStore as a 'serialization' interface. This can be JSON or
anything else.
- The resourceStore also has file operations -- this has read, write
and delete operations. The general idea is to store metadata in
etcd and data on the fileStore (which could also be an object store,
we don't really use any file-specific properties). Files are also
versioned -- each writeFile writes to a new UUID and that UUID is
stored in an etcd in-order list. readFile can retrieve an arbitrary
version by UUID, or just get the latest contents.
Soam Vasani