Nail/fission-src
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

V2 types and TPR (#266)

Browse Source 
This changes the core fission function, environment and trigger types. It also changes Fission's storage to use ThirdPartyResources.

 - Functions are now specified by packages. Functions can also have both source and deployment packages. A package can be specified by a literal, or by a URL.
 - Environments have a build and runtime component.
 - Triggers reference functions by a FunctionReference. This is a layer of indirection between triggers and functions, and will allow things like incremental function upgrades in future releases.

See Documentation/wip/env-v2.md for design discussion about points 1 and 2.

Changes:

* V2 Types

All types now have a spec, following the pattern of K8s objects.

Functions now have source and deployment packages. A Package can be
specified by literal, or by URL.

Environments now have a builder and runtime component.

All triggers use a new FunctionReference to specify the function. This
for now only uses a function name, but in the future can be extended
to be more flexible.

A new FunctionLoadRequest type is added for specialization requests to
the environment runtime.

* TPR types, TPR init code, and a "fission client"

Implements TPR types using the spec types in fission/types.go.

Adds code for adding creating TPR types, and convenient types for crud
operations on each of our resource types.

Adds code for connecting to K8s API and configuring a REST client with
fission types set up.

* Change old stateful controller into a thin apiserver

This apiserver is now simply a stateless api layer on top of the TPR
types. At the moment it doesn't do anything that couldn't be done by
simply talking to the TPR types. In the future we can have better
validation and potentially some higher level APIs (like versioning for
example) in here.

* Split controller client into files and update for v2 types.

* Update CLI for v2 types.

As far as possible we keep the CLI flags the same. We'll have to add
flags for source/deploy packages and builder/runtime
environments. That will come in the next change.

* Update poolmgr and fetcher for v2 types.

Also adds a poolmgr_test.

* Update router for new types.

Also adds a function reference resolver, which separates out the job
of resolving a FunctionReference to a function.

* Update kubewatcher and timer for v2 types.

* Update Message Queue trigger type for v2 types.

* Minor odds and ends.

* Fission bundle CLI updates

Remove controllerUrl flag, since we don't need it any more.

* Remove etcd deployment (replaced by storing state in TPR)

Also update the poolmgr commandline, and use an env var for the
fetcher image URL.

* Explicit ChecksumType and consts

* Clarify separation of environment interface types
This commit is contained in:
Soam Vasani
2017-08-05 01:18:30 -07:00
committed by GitHub
parent 37aa266a4d
commit e238776bf7
81 changed files with 4649 additions and 3588 deletions
Download Patch File Download Diff File Expand all files Collapse all files
types.go
+192 -53
View File
@@ -17,72 +17,155 @@ limitations under the License.
package fission
type (
// Metadata is used as the general identifier for all kinds of
// resources managed by the controller.
Metadata struct {
//
// Functions and packages
//
// ChecksumType specifies the checksum algorithm, such as
// sha256, used for a checksum.
ChecksumType string
// Checksum of package contents when the contents are stored
// outside the Package struct. Type is the checksum algorithm;
// "sha256" is the only currently supported one. Sum is hex
// encoded.
Checksum struct {
Type ChecksumType `json:"type"`
Sum string `json:"sum"`
}
// PackageType is either literal or URL, indicating whether
// the package is specified in the Package struct or
// externally.
PackageType string
// Package contains or references a collection of source or
// binary files.
Package struct {
// Type defines how the package is specified: literal or URL.
Type PackageType `json:"type"`
// Literal contents of the package. Can be used for
// encoding packages below TODO (256KB?) size.
Literal []byte `json:"literal"`
// URL references a package.
URL string `json:"url"`
// Checksum ensures the integrity of packages
// refereced by URL. Ignored for literals.
Checksum Checksum `json:"checksum"`
// EntryPoint optionally specifies an entry point in
// the package. Each environment defines a default
// entry point, but that can be overridden here.
EntryPoint string `json:"entrypoint"`
}
// FunctionSpec describes the contents of the function.
FunctionSpec struct {
// EnvironmentName is the name of the environment that this function is associated
// with. An Environment with this name should exist, otherwise the function cannot
// be invoked.
EnvironmentName string `json:"environmentName"`
// Source is an source package for this function; it's used for the build step if
// the environment defines a build container.
Source Package `json:"source"`
// Deployment is a deployable package for this function. This is the package that's
// loaded into the environment's runtime container.
Deployment Package `json:"deployment"`
}
FunctionReferenceType string
FunctionReference struct {
// Type indicates whether this function reference is by name or selector. For now,
// the only supported reference type is by name. Future reference types:
// * Function by label or annotation
// * Branch or tag of a versioned function
// * A "rolling upgrade" from one version of a function to another
Type FunctionReferenceType `json:"type"`
// Name of the function.
Name string `json:"name"`
Uid string `json:"uid,omitempty"`
}
// Function is a unit of executable code. Though it's called
// a function, the code may have more than one function; it's
// usually some sort of module or package.
Function struct {
Metadata `json:"metadata"`
Environment Metadata `json:"environment"`
Code string `json:"code"`
//
// Environments
//
Runtime struct {
// Image for containing the language runtime.
Image string `json:"image"`
// LoadEndpointPort defines the port on which the
// server listens for function load
// requests. Optional; default 8888.
LoadEndpointPort int32 `json:"loadendpointport"`
// LoadEndpointPath defines the relative URL on which
// the server listens for function load
// requests. Optional; default "/specialize".
LoadEndpointPath string `json:"loadendpointpath"`
// FunctionEndpointPort defines the port on which the
// server listens for function requests. Optional;
// default 8888.
FunctionEndpointPort int32 `json:"functionendpointport"`
}
Builder struct {
Image string `json:"image"`
Command string `json:"command"`
}
EnvironmentSpec struct {
// Environment API version
Version int `json:"version"`
// Runtime container image etc.; required
Runtime Runtime `json:"runtime"`
// Optional
Builder Builder `json:"builder"`
// Optional, but strongly encouraged. Used to populate
// links from UI, CLI, etc.
DocumentationURL string `json:"documentationurl"`
}
// Environment identifies the language and OS specific
// resources that a function depends on. For now this
// includes only the function run container image. Later,
// this will also include build containers, as well as support
// tools like debuggers, profilers, etc.
Environment struct {
Metadata `json:"metadata"`
RunContainerImageUrl string `json:"runContainerImageUrl"`
//
// Triggers
//
HTTPTriggerSpec struct {
Host string `json:"host"`
RelativeURL string `json:"relativeurl"`
Method string `json:"method"`
FunctionReference FunctionReference `json:"functionref"`
}
// HTTPTrigger maps URL patterns to functions. Function.UID
// is optional; if absent, the latest version of the function
// will automatically be selected.
HTTPTrigger struct {
Metadata `json:"metadata"`
UrlPattern string `json:"urlpattern"`
Method string `json:"method"`
Function Metadata `json:"function"`
KubernetesWatchTriggerSpec struct {
Namespace string `json:"namespace"`
Type string `json:"type"`
LabelSelector map[string]string `json:"labelselector"`
FunctionReference FunctionReference `json:"functionref"`
}
MessageQueueTrigger struct {
Metadata `json:"metadata"`
Function Metadata `json:"function"`
MessageQueueType string `json:"messageQueueType"`
Topic string `json:"topic"`
ResponseTopic string `json:"respTopic,omitempty"`
}
// Watch is a specification of Kubernetes watch along with a URL to post events to.
Watch struct {
Metadata `json:"metadata"`
Namespace string `json:"namespace"`
ObjType string `json:"objtype"`
LabelSelector string `json:"labelselector"`
FieldSelector string `json:"fieldselector"`
Function Metadata `json:"function"`
Target string `json:"target"` // Watch publish target (URL, NATS stream, etc)
// MessageQueueTriggerSpec defines a binding from a topic in a
// message queue to a function.
MessageQueueTriggerSpec struct {
FunctionReference FunctionReference `json:"functionref"`
MessageQueueType string `json:"messageQueueType"`
Topic string `json:"topic"`
ResponseTopic string `json:"respTopic,omitempty"`
}
// TimeTrigger invokes the specific function at a time or
// times specified by a cron string.
TimeTrigger struct {
Metadata `json:"metadata"`
Cron string `json:"cron"`
Function Metadata `json:"function"`
TimeTriggerSpec struct {
Cron string `json:"cron"`
FunctionReference `json:"functionref"`
}
// Errors returned by the Fission API.
@@ -94,6 +177,54 @@ type (
errorCode int
)
//
// Fission-Environment interface. The following types are not
// exposed in the Fission API, but rather used by Fission to
// talk to environments.
//
type (
FunctionLoadRequest struct {
// FilePath is an absolute filesystem path to the
// function. What exactly is stored here is
// env-specific. Optional.
FilePath string `json:"filepath"`
// Entrypoint has an environment-specific meaning;
// usually, it defines a function within a module
// containing multiple functions. Optional; default is
// environment-specific.
EntryPoint string `json:"entrypoint"`
// URL to expose this function at. Optional; defaults
// to "/".
URL string `json:"url"`
}
)
const (
ChecksumTypeSHA256 ChecksumType = "sha256"
)
const (
// PackageTypeLiteral means the package contents are specified in the Literal field of
// resource itself.
PackageTypeLiteral PackageType = "literal"
// PackageTypeUrl means the package contents are at the specified URL.
PackageTypeUrl PackageType = "url"
)
const (
// FunctionReferenceFunctionName means that the function
// reference is simply by function name.
FunctionReferenceTypeFunctionName = "name"
// Other function reference types we'd like to support:
// Versioned function, latest version
// Versioned function. by semver "latest compatible"
// Set of function references (recursively), by percentage of traffic
)
const (
ErrorInternal = iota
@@ -103,6 +234,8 @@ const (
ErrorInvalidArgument
ErrorNoSpace
ErrorNotImplmented
ErrorChecksumFail
ErrorSizeLimitExceeded
)
// must match order and len of the above const
@@ -114,4 +247,10 @@ var errorDescriptions = []string{
"Invalid argument",
"No space",
"Not implemented",
"Checksum verification failed",
"Size limit exceeded",
}
const (
PackageLiteralSizeLimit int64 = 256 * 1024
)