Various e2e test bugfixes. The test runner now compiles fission bundle and cli, installs helm, installs kubectl, and uses helm to install fission, run all scripts under test/tests/, and uninstalls fission.
The change also removes go tip tests -- I think we can get by without them, but we'll add them back if the next go version update brings some surprises.
Build fission-bundle and fission cli. Install the fission-bundle using
the fission-all helm chart. Run all files matching test_*.sh from the
tests/ directory.
This change doesn't contain any real tests, just the build and test
runner scripts.
Skips the tests when there are no cluster credentials available.
Travis doesn't pass secret env vars to tests run on forks, so we need
a way to skip the tests when there are no credentials passed in.
This change unsets KUBECONFIG when there's no reachable cluster. The
Go tests already check for KUBECONFIG and skip when it's not defined.
Remove hardcoded "fission" namespace in various places.
Remove etcd deployment and service, since we use K8s APIs now.
Add ClusterRoleBinding for the fission service account, to allow fission components to access TPRs. We currently make the service account a cluster admin. This is more permissive than it should be; we should define a specific role for just fission resources.
This was initially added to avoid restarts if poolmgr starts before
fission api; but we don't really need that since k8s will restart
it. Also, poolmgr now talks to the k8s api directly instead of the
fission api.
Sets up Travis CI tests to use a Kubernetes cluster on GKE. All tests have access to $KUBECONFIG which points at a kubeconfig file with credentials to a cluster.
I had to skip the poolmgr unit test because it currently assumes NodePort services; that will be fixed in another change.
Also, because some of the tests don't work properly in parallel, I've ended up reducing the concurrency to 1. If this becomes a problem we'll have to fix the tests, but for now I don't mind just waiting a few minutes longer.
This changes the core fission function, environment and trigger types. It also changes Fission's storage to use ThirdPartyResources.
- Functions are now specified by packages. Functions can also have both source and deployment packages. A package can be specified by a literal, or by a URL.
- Environments have a build and runtime component.
- Triggers reference functions by a FunctionReference. This is a layer of indirection between triggers and functions, and will allow things like incremental function upgrades in future releases.
See Documentation/wip/env-v2.md for design discussion about points 1 and 2.
Changes:
* V2 Types
All types now have a spec, following the pattern of K8s objects.
Functions now have source and deployment packages. A Package can be
specified by literal, or by URL.
Environments now have a builder and runtime component.
All triggers use a new FunctionReference to specify the function. This
for now only uses a function name, but in the future can be extended
to be more flexible.
A new FunctionLoadRequest type is added for specialization requests to
the environment runtime.
* TPR types, TPR init code, and a "fission client"
Implements TPR types using the spec types in fission/types.go.
Adds code for adding creating TPR types, and convenient types for crud
operations on each of our resource types.
Adds code for connecting to K8s API and configuring a REST client with
fission types set up.
* Change old stateful controller into a thin apiserver
This apiserver is now simply a stateless api layer on top of the TPR
types. At the moment it doesn't do anything that couldn't be done by
simply talking to the TPR types. In the future we can have better
validation and potentially some higher level APIs (like versioning for
example) in here.
* Split controller client into files and update for v2 types.
* Update CLI for v2 types.
As far as possible we keep the CLI flags the same. We'll have to add
flags for source/deploy packages and builder/runtime
environments. That will come in the next change.
* Update poolmgr and fetcher for v2 types.
Also adds a poolmgr_test.
* Update router for new types.
Also adds a function reference resolver, which separates out the job
of resolving a FunctionReference to a function.
* Update kubewatcher and timer for v2 types.
* Update Message Queue trigger type for v2 types.
* Minor odds and ends.
* Fission bundle CLI updates
Remove controllerUrl flag, since we don't need it any more.
* Remove etcd deployment (replaced by storing state in TPR)
Also update the poolmgr commandline, and use an env var for the
fetcher image URL.
* Explicit ChecksumType and consts
* Clarify separation of environment interface types
Improve fission helm chart; split it into "fission core" and "fission all". fission-core is the minimal install, and fission-all will have all the bells and whistles.
* Update/Add fission-core & fission-all helm charts
* Fix incompatible types for comparison
* Add message queue trigger
* Rename openshiftRBAC to openshift
* Update documentation & comments
* Fix logger & kubewatcher failed to access kubernetes resource due to RBAC
Adds support for message queue triggers based on the NATS-Streaming message queue.
This lets the user map a queue topic to a function, and optionally send the function's response into another queue topic.
The message queue trigger manager is designed to be message queue independent, so we should be able add support for more queues by adding implementations for the relatively simple mqtrigger.MessageQueue interface.
* include path parameters in params hash for ruby environment
* Improve code organization and documentation for ruby environment.
* Document Ruby examples
Keep install docs in one place INSTALL.md instead of two.
Use github releases for all YAMLs and CLIs. This has a few advantages:
(a) it makes releasess more transparent
(b) all yaml/cli files are transferred over HTTPS
(c) CLIs and YAMLs being from the same release ensures that people
always get compatible versions (we will still occasionally break
api compatibility, until beta)
Addresses #158. Adds URL params into HTTP headers. A URL param named `KEY` with the value `VALUE` in the actual URL will appear as the header "X-Fission-Params-KEY: VALUE" in the fission function.
The poolmgr client now aggregates successive tapService() calls and sends them to the poolmgr service every 5 sec, instead of sending them immediately.
* Fix controller timer handler not return when error occurred
* Fix controller not close http response correctly
This PR aims to fix controller not close response correctly. Also, I’ve check all over the project there is no more resource leak issues exist after this patch.
* Fix controller not return immediately when error occurred
* Sort import
The generic pool manager creates a generic pool with the reference of environment metadata. Since the reference is address of loop variable, so the content of metadata will be replaced with the last element of envs. So pool manager sets up logging with wrong env.
Establish a proxy server from the controller to the log database.
Redirect query commands send from client to database then proxy back the db response.
Use parameter binding instead of fmt.Sprintf to prevent SQL injection.
Soam Vasani