This contains a few different builder manager fixes:
* Update the zip file structure to avoid an extra subdirectory
* Annotate packages with what functions are using them
* Only trigger builds when there is no deployment archive
* Python environment loadpath bugfixes
* Other bugfixes
Switch Fission's storage over to the new CustomResourceDefinitions, from the deprecated ThirdPartyResources. This allows us to be compatible with Kubernets 1.8 and onwards.
This also adds a CLI tool for dumping state from an old fission version and restoring state into new CRDs.
The storage service is unaffected by this change.
This change orchestrates function builds.
Environments (in v2) define a builder image, just like they do a runtime image. The builder image contains a build script that's invoked with source and deployment paths (as env vars).
The buildermgr watches for environments with build images defined, and creates build deployments and services.
Functions can define source and deployment. Buildermgr watches for functions with source code (and build status == pending) and invokes the environment's builder when appropriate. It captures logs from the build and sets the build status (success/failure) and build lots into the PackageStatus.
Watch functions in the router and use that to trigger functionReferenceResolver cache invalidation.
We might want to rate-limit syncTriggers in a future change, since we might be triggering it more often than needed.
Fetcher now uncompresses archive files.
This is triggered by automatically detecting the file format, although maybe we should make that explicit or at least move the detection into the client.
Build server and client for environments.
This build server will run inside environment builders and invoke environment-specific scripts for doing builds. It provides a uniform language-independent API for builds to the fission build manager.
The change also includes an implementation of a builder for Python -- this "builder" acts on requirements.txt and runs `pip install` to collect all function deps.
Improve fission helm chart; split it into "fission core" and "fission all". fission-core is the minimal install, and fission-all will have all the bells and whistles.
* Update/Add fission-core & fission-all helm charts
* Fix incompatible types for comparison
* Add message queue trigger
* Rename openshiftRBAC to openshift
* Update documentation & comments
* Fix logger & kubewatcher failed to access kubernetes resource due to RBAC
Adds support for message queue triggers based on the NATS-Streaming message queue.
This lets the user map a queue topic to a function, and optionally send the function's response into another queue topic.
The message queue trigger manager is designed to be message queue independent, so we should be able add support for more queues by adding implementations for the relatively simple mqtrigger.MessageQueue interface.
* Fix controller timer handler not return when error occurred
* Fix controller not close http response correctly
This PR aims to fix controller not close response correctly. Also, I’ve check all over the project there is no more resource leak issues exist after this patch.
* Fix controller not return immediately when error occurred
* Sort import
The generic pool manager creates a generic pool with the reference of environment metadata. Since the reference is address of loop variable, so the content of metadata will be replaced with the last element of envs. So pool manager sets up logging with wrong env.
Establish a proxy server from the controller to the log database.
Redirect query commands send from client to database then proxy back the db response.
Use parameter binding instead of fmt.Sprintf to prevent SQL injection.
Add function log aggregation and persistence using Fluentd and InfluxDB.
Fluentd and a helper sidecar run as a daemonset. The poolmgr sets up logging for each function pod, using the helper sidecar. Fluentd forwards logs to InfluxDB, which is run as a deployment and service. The client CLI directly queries InfluxDB for logs.
Fluentd supports many outputs besides InfluxDB, so we aren't very tied to InfluxDB.
The setup is somewhat manual, which we should be able to improve by integrating this into the helm chart.
Diagram of component interactions: https://cloud.githubusercontent.com/assets/202578/23100399/b0e3ea00-f6ba-11e6-8f2f-6588cfef2e84.png
Fixes#139. The code before this change was saving up pointers to the loop variable in a slice. So loop was useless and we always chose the last pod in the list, whether it was ready or not. With this fix, choosePod should always return a ready pod.
Also adds a check for the pod phase, before looping over the containers in the pod, and discard pods that aren't running yet.
