This feature allows creation of fission objects in different namespaces, in addition to retaining the existing behavior of creating fission objects in default namespace if user doesnt provide one.
It also removes cluster admin roles for fission-fetcher and fission-builder Service Accounts and grants them only those privileges that they need.
Environment Specs so far had only an image URL to specify a container image.
This was fine for public images but fell short in a few of cases:
(a) Using private image registries
(b) Specifying environment variables (this is needed for workflows helm install)
(c) Setting a SecurityContext for the container
This change adds the Container object to both build and runtime Environments.
Compatibility is preserved -- the existing ImageURL field is still used. See the comments in types.go for the overriding rules in the case that both Container and ImageURL are specified.
A newdeploy backend which uses new deployment to serve requests. This is the second phase of #193 and builds on top of changes in #384 .
* Executor layer added on top of pool manager
* Removed the external server for executor
* Minor changes to keep existing semantics as much possible
* Separating the executor vs. poolmgr backend functionality and associated data members
* Executor logic separated from Poolmgr backend completely, placeholder for new backend
* Changed references to poolmgr in tests
* Moved poolmgr to it's package, as a side effect moved Cache to its's package (was causing cyclical dependency) and had to make some data structures exposed outside package
* Rebased from master and changed references to tpr -> crd
* Executor layer added on top of pool manager
* Executor logic separated from Poolmgr backend completely, placeholder for new backend
* Changed podName to a generic objectReference in fscache (#391)
Changed podName to a generic objectReference in function service cache implementation.
* Moved poolmgr to it's package, as a side effect moved Cache to its's package (was causing cyclical dependency) and had to make some data structures exposed outside package
* Rebased from master and changed references to tpr -> crd
* Merged from master with latest changes
* Executor layer added on top of pool manager
* Removed the external server for executor
* Minor changes to keep existing semantics as much possible
* Separating the executor vs. poolmgr backend functionality and associated data members
* Executor logic separated from Poolmgr backend completely, placeholder for new backend
* Changed references to poolmgr in tests
* update compiling.md to use helm
* Compile instructions: changed pullPolicy to IfNotPresent (#378)
Containers will get stuck in ErrImagePull/ImagePullBackOff state otherwise
* Moved poolmgr to it's package, as a side effect moved Cache to its's package (was causing cyclical dependency) and had to make some data structures exposed outside package
* Fetcher called when pod is created for newDeploy backend but also supports older way, this is WIP and still needs pod specialization and creating & exposing a service so the URL can be hit by end user
* WIP Specializing the POD as part of startup along with fetching
* Working specialization of a new deployment. Needs some work on caching, cleanup etc.
* Switched to service based address instead of POD address
* Minor formating issue fixed
* Added logging to pods and a readiness check, the readiness check is flaky though ATM
* Fixed some rebase issues that were failing build
* Better names for K8S objects and methods
* Switched usage of FuncSvc in backends from pod to api.ObjectReference
* Adding retry to fetcher request, for now just using default retry client which might need tweaking in future
* Switching to plain old retry, some issue in getting retryablehttp with glide import
* Removed stale executor service & deployment from previous merge
* Addressed review comments, still testing some areas
* Added types in FunctionSpec
* Resolved conflicts due to merge from executor_abstraction branch
* Added backend type on EnvironmentSpec along with operations for create/list/update, the pools are created/destroyed based on change in backend type
* Backend from types and a minor err return issue fixed
* Draft version of CPU and memory parameters added to environment
* Added resourceReq to newDeploy, though it has some issues
* Issue with resourceName fixed, now newdeploy pods also pick up resources from the environment config
* Adding scale params, removing validation on CPU params for now
* Fixed a formatting issue
* Checking if slight more delay helps in the test which is currently failing for internal routes
* The resourceList newly added in Env can not be compared by compiler, hence must use breakdown comparison instead
* Added strategy selection on client side
* Added caching, informers, delete operations for newdeploy backend functions
* Deleted a stale directory
* A simple HPA based on scale parameters, testing still WIP
* Fixed a small issue in delete function, added HPA delete too when deleting a function
* Previous merge missed the pkg flag for update fn command somehow, fixed that
* Fixed comments from review
* Changed poolmgr cleanup to be generic cleanup and moved to executor, added instanceID labels to newdeploy so that cleanup works
* Moved instanceIdLabel to types to avoid cyclic dependency
* More review fixes
* Tweaking sleep to see results
* If user does not provide poolsize, then it should not default to zero
* Switched to naming convention for now, fixed default poolsize if not provided
* Changed error return behaviour in delete fn, also changed cleanup to look based on obj type though support for additional type will need more work
* Changed check location so avoid false logging
* Test for newdeploy backend
* Adding tests for poolmgr backend
* Fixed an issue with glide dependency version, already fixed in master
* Added instanceId for NewDeploy, Initial cleanup now cleans older objects of newdeploy backend, removed eagercreate flag and instead using minScale to drive eager creation
* Moved cleanup to executor layer with cleanup for newDeploy backend, changes to use the new Cache impl
* Cleaning up pod & rs along with deployment for newdeploy backend
* Enhanced fn and env listing to show min/maxscale and resuorces respectively
* Added conditional heapster deployment and fixed a small issue with resources for fetcher container in function pod
* Addressed review comments from previous change
* Addressed some more review comments - majorly create only on NotFoundError
* Added TargetCPU as an input for scaling
* Bumped target CPU to be greater than 0 and added a default value
* Min replicas should be 1 even if the minScale is 0 when creating deployment
* Changed name from 'backend' to executorType, added additional test for minscale 0 case, changed TargetCPU to TargetCPUPercent
Switch Fission's storage over to the new CustomResourceDefinitions, from the deprecated ThirdPartyResources. This allows us to be compatible with Kubernets 1.8 and onwards.
This also adds a CLI tool for dumping state from an old fission version and restoring state into new CRDs.
The storage service is unaffected by this change.
Introduce _Archives_ as a type to reference arbitrary blobs.
_Packages_ are a pair of Source and Deployment archives. Packages have an environment reference.
Functions reference a package. Multiple functions can reference the same package.
Split out the Package type into a first class Kubernetes resource. Before this change, packages were implicitly tied to functions.
This wasn't ideal because:
* Functions will need to share packages
* A package storage system may be more generally useful than just
functions (for example, for storing static assets)
This change does the following:
* Updates the fission and tpr types to add a new Package and PackageSpec. It also creates a PackageRef type, and a FunctionPackageRef type. The PackageRef simply references a package, but the FunctionPackageRef includes the name of a function within the package. This allows us to share packages between different functions.
* Updates fetcher and other components for first-class packages
* Allows customization of fetcher image pull policy in the helm charts
This changes the core fission function, environment and trigger types. It also changes Fission's storage to use ThirdPartyResources.
- Functions are now specified by packages. Functions can also have both source and deployment packages. A package can be specified by a literal, or by a URL.
- Environments have a build and runtime component.
- Triggers reference functions by a FunctionReference. This is a layer of indirection between triggers and functions, and will allow things like incremental function upgrades in future releases.
See Documentation/wip/env-v2.md for design discussion about points 1 and 2.
Changes:
* V2 Types
All types now have a spec, following the pattern of K8s objects.
Functions now have source and deployment packages. A Package can be
specified by literal, or by URL.
Environments now have a builder and runtime component.
All triggers use a new FunctionReference to specify the function. This
for now only uses a function name, but in the future can be extended
to be more flexible.
A new FunctionLoadRequest type is added for specialization requests to
the environment runtime.
* TPR types, TPR init code, and a "fission client"
Implements TPR types using the spec types in fission/types.go.
Adds code for adding creating TPR types, and convenient types for crud
operations on each of our resource types.
Adds code for connecting to K8s API and configuring a REST client with
fission types set up.
* Change old stateful controller into a thin apiserver
This apiserver is now simply a stateless api layer on top of the TPR
types. At the moment it doesn't do anything that couldn't be done by
simply talking to the TPR types. In the future we can have better
validation and potentially some higher level APIs (like versioning for
example) in here.
* Split controller client into files and update for v2 types.
* Update CLI for v2 types.
As far as possible we keep the CLI flags the same. We'll have to add
flags for source/deploy packages and builder/runtime
environments. That will come in the next change.
* Update poolmgr and fetcher for v2 types.
Also adds a poolmgr_test.
* Update router for new types.
Also adds a function reference resolver, which separates out the job
of resolving a FunctionReference to a function.
* Update kubewatcher and timer for v2 types.
* Update Message Queue trigger type for v2 types.
* Minor odds and ends.
* Fission bundle CLI updates
Remove controllerUrl flag, since we don't need it any more.
* Remove etcd deployment (replaced by storing state in TPR)
Also update the poolmgr commandline, and use an env var for the
fetcher image URL.
* Explicit ChecksumType and consts
* Clarify separation of environment interface types
This change adds a timer trigger API, client, and implementation. Users can trigger a function with a cron-compatible string.
This change also moves the publisher interface and webhook-based publisher implementation out of kubewatcher and into a separate `publisher` package.
When the request is handled successfully, json data is returned, this
patch changes the response Content-Type header from text/plain to
application/json
Fixes: #144
If no uid is provided, all versions of function files and function metadata will be deleted. If a uid is provided, only that specific version is deleted, and the "latest" version of the function moves back to the previous one.
Fixes#152
Motivation: Creating duplicate resources currently results in a 500
error being displayed by the client, with no information about the true
nature of the failure.
Modifications:
* ResourceStore now converts any errors from the etcd client into
fission errors, capturing the reason for the error in the case of a
duplicate key (as ErrorNameExists)
* ErrorNameExists errors are signaled to the client with a
409 (Conflict) HTTP status
* MakeErrorFromHTTP() now reads the body of the error response to
retrieve the actual error message instead of using the HTTP status
message
* the controller client now uses MakeErrorFromHTTP() to centralize
status code -> error code mapping
* tests for all of the resources now check that duplicate resources are
reported properly
Result: The client can now provide more context when a duplicate name is
given
related to #112
Environments for functions. For now this is just the image URL for the
function run container. In the future, Environments will also have
build container image URLs, and perhaps parameters to those
containers.
Create/read/update/delete/list. Versioning will be added later. For
now, only the latest update is stored.
This implemenents create/read/update/delete/list for http triggers.
This isn't quite the final implementation: you can't look up old
versions by UUID since it only stores the latest update. I'll get to
that later.
