Various e2e test bugfixes. The test runner now compiles fission bundle and cli, installs helm, installs kubectl, and uses helm to install fission, run all scripts under test/tests/, and uninstalls fission.
The change also removes go tip tests -- I think we can get by without them, but we'll add them back if the next go version update brings some surprises.
Build fission-bundle and fission cli. Install the fission-bundle using
the fission-all helm chart. Run all files matching test_*.sh from the
tests/ directory.
This change doesn't contain any real tests, just the build and test
runner scripts.
Skips the tests when there are no cluster credentials available.
Travis doesn't pass secret env vars to tests run on forks, so we need
a way to skip the tests when there are no credentials passed in.
This change unsets KUBECONFIG when there's no reachable cluster. The
Go tests already check for KUBECONFIG and skip when it's not defined.
Remove hardcoded "fission" namespace in various places.
Remove etcd deployment and service, since we use K8s APIs now.
Add ClusterRoleBinding for the fission service account, to allow fission components to access TPRs. We currently make the service account a cluster admin. This is more permissive than it should be; we should define a specific role for just fission resources.
This was initially added to avoid restarts if poolmgr starts before
fission api; but we don't really need that since k8s will restart
it. Also, poolmgr now talks to the k8s api directly instead of the
fission api.
Sets up Travis CI tests to use a Kubernetes cluster on GKE. All tests have access to $KUBECONFIG which points at a kubeconfig file with credentials to a cluster.
I had to skip the poolmgr unit test because it currently assumes NodePort services; that will be fixed in another change.
Also, because some of the tests don't work properly in parallel, I've ended up reducing the concurrency to 1. If this becomes a problem we'll have to fix the tests, but for now I don't mind just waiting a few minutes longer.
This changes the core fission function, environment and trigger types. It also changes Fission's storage to use ThirdPartyResources.
- Functions are now specified by packages. Functions can also have both source and deployment packages. A package can be specified by a literal, or by a URL.
- Environments have a build and runtime component.
- Triggers reference functions by a FunctionReference. This is a layer of indirection between triggers and functions, and will allow things like incremental function upgrades in future releases.
See Documentation/wip/env-v2.md for design discussion about points 1 and 2.
Changes:
* V2 Types
All types now have a spec, following the pattern of K8s objects.
Functions now have source and deployment packages. A Package can be
specified by literal, or by URL.
Environments now have a builder and runtime component.
All triggers use a new FunctionReference to specify the function. This
for now only uses a function name, but in the future can be extended
to be more flexible.
A new FunctionLoadRequest type is added for specialization requests to
the environment runtime.
* TPR types, TPR init code, and a "fission client"
Implements TPR types using the spec types in fission/types.go.
Adds code for adding creating TPR types, and convenient types for crud
operations on each of our resource types.
Adds code for connecting to K8s API and configuring a REST client with
fission types set up.
* Change old stateful controller into a thin apiserver
This apiserver is now simply a stateless api layer on top of the TPR
types. At the moment it doesn't do anything that couldn't be done by
simply talking to the TPR types. In the future we can have better
validation and potentially some higher level APIs (like versioning for
example) in here.
* Split controller client into files and update for v2 types.
* Update CLI for v2 types.
As far as possible we keep the CLI flags the same. We'll have to add
flags for source/deploy packages and builder/runtime
environments. That will come in the next change.
* Update poolmgr and fetcher for v2 types.
Also adds a poolmgr_test.
* Update router for new types.
Also adds a function reference resolver, which separates out the job
of resolving a FunctionReference to a function.
* Update kubewatcher and timer for v2 types.
* Update Message Queue trigger type for v2 types.
* Minor odds and ends.
* Fission bundle CLI updates
Remove controllerUrl flag, since we don't need it any more.
* Remove etcd deployment (replaced by storing state in TPR)
Also update the poolmgr commandline, and use an env var for the
fetcher image URL.
* Explicit ChecksumType and consts
* Clarify separation of environment interface types
Keep install docs in one place INSTALL.md instead of two.
Use github releases for all YAMLs and CLIs. This has a few advantages:
(a) it makes releasess more transparent
(b) all yaml/cli files are transferred over HTTPS
(c) CLIs and YAMLs being from the same release ensures that people
always get compatible versions (we will still occasionally break
api compatibility, until beta)
Before this change, if the user doesn't define a route for /, the
router happily 404s at that path. This breaks GKE Ingress, which uses
that request as a health check. So this change adds a handler for
"GET /", unless the user already has one.
Also, log the delay for slow requests.
Test script that sets up fission on a cluster, runs tests and
uninstalls it.
Yet to come:
1. The actual tests
2. Builds (for fission-bundle, client, and envs)
3. Automation with github pull requests
Remove our non-standard namespace variable and use .Release.Namespace
to put fission deployments and services into.
(We still need the functionNamespace parameter, to customize what
namespace fission puts functions into.)
Addresses issue #113. poolmgr.MakeGenericPool does not need to wait
for a ready pod. Kubernetes already retries image fetching, so
there's no need to repeat the task in fission. And if it's a
non-retriable error (like the image URL being wrong), then the
deployment will stay broken, and the user can fix the environment with
'fission env update' (or 'fission env delete' followed by 'fission env
create').
Poolmgr removed the default-latest-version semantics to simplify
caching. But this broke the router's internal function routes, which
are used by kubeWatcher. The router now exposes a versionless url
and regularly updates that to point to the latest version.
Label all poolmgr-created resources with an id that's unique to a
running poolmgr instance. On poolmgr start up, clean up resources
created by old instances. Resources that are idle are killed
immediately; resources that could be running a user function are
killed after the maximum function timeout.
Addresses issue #51. Makes versioning in poolmgr explicit, so its
cache needs no invalidation on version update. Functions in poolmgr
are always cached by name and UID.
Router now updates implictly versioned routes with the latest version
of a function.
This means that users will see requests to implicitly-versioned routes
go to the latest version of a function within 3 seconds. (Those 3 sec
will go away when we use a real watch instead of polling the
controller.)
There's no change in behaviour for routes that explictly specify a
function UID.
This allows users to add --method and --url params to their "fission
fn create" command line. It sets up the route to call the latest
version of the function; user can edit the route with the "fission
route update" command, if they want to.
Soam Vasani