* Change Ingress version from v1beta1 to v1 for K8s 1.22+ compatibility
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Minor fix
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
PR adds a new executortype which supports running containers as functions. New CLI under functions is added to create container as functions.
Co-authored-by: Harsh Thakur <harshthakur9030@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
1. Support for adding labels and annotations to functions & environment via fission CLI through create & update command.
2. Change ensures labels and annotations assigned to the environment would reflect on pods created via executor type pool manager and new deploy.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Retrieve pod metrics only if metrics server is running
Currently we query pod metrics every 30 sec which floods executor logs,
added check which confirms if metrics server is running then only we start
querying pod metrics for identifying CPU utilization.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Fixed couple of typos and misspells with Go CI
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Remove unnecessary conversions with Go CI
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* add multi env and multi k8s support on integraation tests
* test workflow
* test workflow
* test workflow
* remove test branch name
* correct kind action with image and version
* use same kind version as per kind images
* use OS as ubuntu-latest
* fetch upstream
* add .github/workflows to paths
* resolve conflicts
* make kindimage appear first on workflow
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
* Correct release name in release script for non helm install
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Increase test timeout to 20 minutes
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Enable codeql for master only
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Added GCP PubSub Image, enabled Keda by default and updated versions of all Keda connector images
Signed-off-by: Vishal Biyani <vrbiyani@gmail.com>
* Fixed the hyphens, it's been long since I have written Helm templates :)
Signed-off-by: Vishal Biyani <vrbiyani@gmail.com>
* Fixed the hyphens, it's been long since I have written Helm templates :)
Signed-off-by: Vishal Biyani <vrbiyani@gmail.com>
* Update charts/fission-all/values.yaml
Co-authored-by: Harsh Thakur <harshthakur9030@gmail.com>
* Update charts/fission-core/values.yaml
Co-authored-by: Harsh Thakur <harshthakur9030@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: Harsh Thakur <harshthakur9030@gmail.com>
* Add a SharedIndexInformer for services and deployments to NewDeploy executor.
This brings the NewDeploy executor behaviour into sync with GenericPoolManager behaviour
by caching Kubernetes services and deployments used in per-request function validation.
* Create informers in executer by executor label
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: James Sinclair <irgeek@btc.com.au>
* Enable prefix based routing
Signed-off-by: Harsh Thakur <harshthakur9030@gmail.com>
* Optimize checking condition
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Disable few tests
* disable router modification for now
* run code generator
* Collect fission dump in CI
* Enable all tests back
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Remove unwanted code
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Add prefix support at more places and couple of todo's
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Few more changes
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Improve function trim support
* Support for prefix based urls in fission function test
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* multi route for fission function test
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Improve validations in trigger creations
* Adjust leading / in url from fission
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Change suburl to subpath for function test
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
* Add a warning if spec is applied on a dirty worktree
* Update go dependencies after rebase
* Use the console package for showing the warning
* Fix merge conflicts
* Handle errors and use better messages
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
Since we recently migrated all fission examples to fission/examples repository, we need to change
respective references.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
We are facing issue in CI for multiple PRs due to mlock. It depends on kernel version or ulimit. Trying to switch to latest go version from 1.14 and see if it helps!
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Series are sorted internal by "order by", but the set of all series returned is not. Tried some ways to sort completely on database, found no way .. client side sort works at least
Co-authored-by: Vishal <vishal-biyani@users.noreply.github.com>
Adding once only execution pattern to pool manager based functions. This is for use cases where you don't want to share the samne pod for another execution instance!
In test_mqtrigger.sh and test_mqtrigger_error.sh tests, we timeout program
if not received right output. Ensuring we capture timeout signal and do program
cleanup accordingly. Also printing due to which signal program failed.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Moved out environments and examples out of main Fission repo to their own repo. This is to allow changes to environment releases to be independent of main Fission releases.
Websocket event support for cleaning up pods only after WS connection is terminated. The support for websocket is right now in the environment itself and router simply acts as a proxy for WS communication!
Increase the default concurrency to 500 from 5 set in the function. Usually, it is good to error on the higher side and if needed, users can always fine-tune it to their needs.
This feature enables routing more than one request to a pod at the same time. This is the first draft of the work and might involve more optimizations later.
Passing information of function from router to executor is more efficient than calling the K8S API. This change does that instead of passing only metadata and then executor calling the K8S API again.
A ready pod which can be specialized was fetched for every function earlier, this has been changed to a queue and cache implementation in client-go to improve performance.
Poolmanager when tested at high load had some issues and this PR fixes one set of them which were found so far.
Co-authored-by: Vishal <vishal-biyani@users.noreply.github.com>
Concurrency in the pool manager allows specializing pods concurrently based on a specified limit.
Co-authored-by: Vishal <vishal-biyani@users.noreply.github.com>
Attribute enableServiceLinks can be now configured in PodSpec so that environment variables are not injected in function pods and DNS is used for discovery
This PR let the controller returns the detail error message
of kubernetes error which makes users understand what's
happening instead of meanless error msg.
This PR changes the behavior of controller API which
wrongly exposes sensitive data to the client. Now, the
API only returns success if secret/configmap exists;
otherwise, an error will be returned.
In some cases, creating a package and querying the package info
immediately the Kubernetes API server will return "not found"
error. So retry the query again after some time.
In some cases (#1384), users may want to access the file in the directory of deploy archive, however, we use the function UID name as the directory name and make it difficult for users to use as they don't know the real path of the directory.
This PR uses the fix name "deployarchive" to make things easier.
If the user applies package YAML file has no status field,
the package won't be able to be compiled or deployed due
to lack of status. This PR aims to add a check at buildermgr
to set initial package status to those packages.
Executor wrongly deletes role bindings if the user
creates an environment in the reserved namespaces. This
PR is a quick fix to solve the problem by checking if
an environment is under reserved namespaces.
This PR adds a fake controller client for local CLI operations like
offline spec generation or for unit test purposes. The fake client
now only implements the "Version()" function and more functions
will be implemented once we figure out how to achieve the goals
mentioned above.
The root cause of the problem is that eagerPoolCreator
tries to create the deployment when the poolmanager
is trying to delete it. To avoid this, start eager pool
creator after executor starts serving requests.
Adding fission spec list functionality. It lists Functions, Environment, Packages, HTTPTrigger, MessageQueueTriggers, Canary Config, TimeTrigger, KubeWatch created by an application specification.
The 3rd party package version used by the go server
may be different from the one in the user's source code
and will cause plugin version mismatched. Hence, we
should never import any external packages except the
Fission or built-in packages.
This PR replaces all 3rd-party packages with built-in packages.
This PR adds an interface for controller API client, it allows us to
implement mock API client for unit testing with ease and we are
able to generate spec file without accessing the real Fission server.
The pod template is embedded inside the deployment. So if
the pod annotation contains instance-id, the deployment
will get updated and thus triggers a rolling update whenever
a new executor starts which is unwanted.
After this PR, poolmanager will patches instance-id when a
pod is chosen for a function.
For newdeploy, unlike poolmanager manages the lifecycle
of function pod directly, newdeploy is only responsible
to create the deployment so we append instance-id to top-
level controller (deployment) only.
We used to update timestamp in the deployment environment field
in order to trigger a rolling update when the function referenced
resources get updated. However, use timestamp means we are not
able to avoid triggering a rolling update when executor tries to adopt
orphaned deployment due to timestamp changed which is unwanted.
In order to let executor adopt deployment without triggering a rolling
update, we need an identical way to get a value that can reflect res-
ources changed without affecting by time.
To achieve this goal, the sum of the resource version of all referenced
resources is a good fit for our scenario since the sum of the resource
version is always the same as long as no resources changed.
When a new executor starts up, it adopts the orphan kubernetes resources created
by the old executor instance. However, the adopted resource won't reflect the changes
come with the new executor, for example, the fetcher image inside won't be changed.
To solve this, executor updates the resource spec (HPA/Deployment/Service) with the
latest resources spec. By doing this, we can prevent the inconsistency between resources
created by different executor instance, also minimizes the impact on users.
Previously, once the executor is deleted for reasons (like upgrade or cluster scale-in),
the new executor deletes all existing resources created by the old executor and creates
new one. This mechanism becomes a problem when there are requests connecting to the
existing pods. Also in the worst case, the cluster may not have enough resources to create
new pods and cause service downtime.
This PR let each executor type adopts existing resources before starting the executor
API services, and so the alive connections won't experience failure. However, the requests
send to the function that doesn't have alive function pods will still fail due to the
executor is in bootstrapping.
The pool manager keeps terminating function pod periodically even there are
traffic to the function. The root cause is that executor, poolmgr, newdeploy
manage their own functionServiceCache separately. And when router taps a
function, executor updates the access time of the function service entry in its
own cache without notifying executor types to do the update as well. Hence,
the access time of function service entry in poolmanager cache never gets updated.
Due to the access time never gets updated, the idle pod reaper in poolmanager
then thinks the function pod is in idle state and recycle it.
This PR removes the cache in executor itself, and when router tries to tap a function,
executor will call executor type to tap the function and update access time.
The router taps function service one by one which is inefficient and
increases the burden of executor. This PR aggregates all requests into
one to solve the problem mentioned above.
When a function is created before the creation of the environment it's used, the newdeploy will not be able to create kube objs. Hence no function service record is inserted into the cache.
When getFuncSvc is called, the newdeploy tries to find the record in service cache in order to create kube objs with the same name used in previous kubeobjs creation. However, due to no record in the cache, a NotFound error is returned directly and causes the problem. To solve this, we use fn meta UID to ensure we always get the same obj name instead of getting it from the cache.
If the context of request is closed before ReverseProxy finishing writing
a huge response body to the response writer, the client will only receive
a truncated response body.
To solve this, move the context cancel after ReverseProxy finished.
To keep archive creation implementation simple and prevent any
confusion, we decided to remove `--keepurl` flag and embed URL
directly without downloading the file from it . In this way, we can
ensure consistent behavior in either package creation or spec file
creation. Also, it increases the portability of spec file.
1. The records are stored in redis which is not migratable to another cluster for the testing purposes.
2. Some of the requests fields are not recorded.
3. People should consider using https://github.com/buger/goreplay which is an existing mature and well-tested solution for testing purposes.
The function implementations of controller client package are
inconsistent. This PR lets functions reuse the functions
that already implemented and able to set additional headers to
request.
When `fn test` failed to retrieve logs from the function
pod, it turns to query logs from the log database. However,
the logdb type flag is not set to `fn test`, hence the logdb
type is empty (even without default value) and cause `fn logs`
returns an error due to unable to find the corresponding log
database type.
This PR adds logdb type flag to `fn test` to resolve the problem.
Jaeger preserves the trace data of router healthz endpoints
which is not so helpful for monitoring function metrics and
consumes storage to store such trace data.
This PR uses `GetStartOptions` to examine the request's path and
check if it's a request to healthz endpoint. If yes, skip and not
to trace it.
To follow 12-factor app rules, make jaeger collector endpoint
as an environment variable instead of CLI args. It's easier to
replace the value in different deployments. Also, we can utilize
valueFrom to get value from the configmap.
The helm pre-upgrade check failed due to the program tries to
check whether a multierr is a nil pointer instead of errors it contains.
This PR changes the way the program checks multierr in order not to block the upgrade.
Helm treats the single line "extraCoreComponentPodConfig" in
values.yaml as a key value pair config.
```
extraCoreComponentPodConfig:
```
And when a user tries to install helm with providing config
like following, it failed.
```
extraCoreComponentPodConfig:
nodeSelector:
serverless: fission
```
The root cause is that helm treats config in default values.yaml
and providing config as two different types of config, which is
key-value vs. value table. And so it's unable for helm to merge these
two configs even with the same key name.
This PR comments out the single line config in default values.yaml
to prevent the issue.
Before this PR, CLI functions fatal out when encountering error
instead of returning it. Such behavior makes it hard to reuse
the functions nor writing unit tests. This PR aims to make functions
return errors instead of error out.
Istio sidecar proxy blocks all requests sent through the reverse proxy
to the target service if the request.Host is not properly set to the
internal target service host. This PR sets the target service hosts
before establishing the proxy for the client in order to pass the
Istio sidecar proxy check.
Tensorflow Serving supports three kinds of API: predict, classify, regress. In order to fully support possible API types, this PR adds code to split the entry point with separator ":" to get model name and the API type separately. And append the API type to proxy URL when receiving the requests.
$ fission fn create --name t1 --pkg <pkg name> \
--env tensorflow --entrypoint "half_plus_two:regress"
The utility function GetMetadata uses wrong flag text to get the resource namespace and could cause the wrong results returned from the API server. This PR changes the function signature that allows users to pass in the flag text in order to get the correct value.
In the case of large files, it takes a long time for the user to download
the source from the URL and upload it to StorgeSvc through CLI.
This PR allows a user to use URL as the function source when creating a function
and provides a new flag "--keeparchiveurl" to let the user to decided
whether the CLI should download the file first or store the file URL in the
archive directly. If "--keeparchiveurl" is true, then no checksum will be
generated, it's the user's responsibility to ensure the file won't be changed.
When a function with executor type newdeploy got created, Newdeploy
is expected to create deployment/HPA/service for it and insert serviceEntry
to the cache for later use. Once clients call the function, newdeploy returns
the serviceEntry to the router.
However, the log shows that the newdeploy was unable to find the entry and
prints "Resource not found - key 'xxx' not found". The root cause is that the
informer controller instead of processing items in parallel, it dispatches XXFunc
to process items one by one. So if there is any problem during the creation of the
kubernetes resource, it takes a longer time to process the next item and hence
the serviceEntry was not inserted before clients call the function.
This PR lets the newdeploy to process items in extra goroutines instead of blocking
the process loop. It's a workaround to solve the problem above, we should consider
using workqueue to solve it in the future.
Encoded path means the URL path contains encoded string like "/foo%2Fbar".
Gorilla/Mux by default doesn't enable encoded path support, you need to enable
it when initializing the router.
This PR adds a new environment variable USE_ENCODED_PATH to the router
deployment to enable encoded path support.
If a user deploys fission in the namespace which is different from the one in the single YAML file generated by helm, fission components won't be able to talk to each other due to the wrong namespace appends after the service address.
This PR adds --namespace when generating the YAML file to prevent the mismatch problem.
ReplicaSet generated by Deployment distributes pods to nodes based
on nodes resource usage, which means in some cases the newly created
pods may be assigned to only a few nodes. The requests will go to the
same node if there multiple pods on it and may increase the node
resource usage and overall request latency.
To solve the problem mentioned above, this PR allows users to deploy
the router as DaemonSet so that we can distribute requests across all
nodes for better workload distribution and lower latency.
The merge function executor used wasn't merge container correctly, and it
didn't merge all fields in spec except volumeMount & Env which confused people.
To apply the user-configured container correctly, this PR changes the way of merge
and follows rules:
1. Slices are merged and return an error if the elements in the slice have name conflicts.
2. Maps are merged, the value of map of dst container are overridden if the key is the same.
3. The rest of the fields of dst container are overridden directly.
In 1.9, kubernetes prompted Deployment to apps/v1 and finally deprecated
extensions/v1beta1 support in 1.16. Users will see the following error message
in log when fission components try to submit Deployment to the k8s API server
and specify extensions/v1beta1 as API group.
error: unable to recognize "deployment": no matches for kind "Deployment" in version "extensions/v1beta1"
This PR updates call to k8s API server and apiKind in chart files to avoid fission failure in 1.16.
Router readiness probe failed due to kubelet cannot find router healthz endpoint.
This is caused that the healthz endpoint is added only when there is any update request
send to the updateRouterRequestChannel, and makes router failed.
This PR sends an update request right before the router is started to avoid the problem.
The router prints error no matter what error type it is.
It's useful for troubleshooting, however, it also prints
the context canceled error, which means that users abort
request before reply and its really normal nowadays. Also,
the router returns 502 if error is not nil and may confused
client if it's a timeout error.
To solve these problems, this PR adds an error handler to
reverse proxy to examine the return error and change the
status code when needed.
Previously, the spec doesn't update the package status if nothing
in the spec file changed. Due to this, the failed package will
always stay in the failed state.
This PR adds a check to see whether a package is in the failed state.
If yes, then changes the state to pending for builder manager to rebuild it.
Previously, a user has to delete HTTP triggers point to
the same function one by one. This PR adds new flag --function
to list & delete commands. So that user can delete or list all
triggers with the same function. However, this feature is not
working for triggers with canary deployment setup.
For each ingress controller, the format of ingress host,
path and annotations are different. To support different
kinds of controller, this PR adds new ingress config field
to http trigger spec. A user can set annotations, host and
path based on the type of underlying ingress controller
with CLI.
Command example:
fission route create --name foo \
--url /foo/{bar} --function foofn --createingress \
--ingressannotation "nginx.ingress.kubernetes.io/ssl-redirect=false" \
--ingressannotation "nginx.ingress.kubernetes.io/use-regex=true" \
--ingressrule "*=/foo/*"
* Add specializationtimeout flag to function create/update
* Set specialization timeout of 120 seconds if not present
* Add default newdeploy timeout for rest of the test cases
* Comment out specialization timeout in validations for compatibility
* Add warning if specializationtimeout is lower than default value
This fix allows the empty repository tag in values.yaml file. The changes are done in the templates of deployment and fluentbit where repository tag will be added in the docker image name only when specified with some value in case of empty repository tag image name contain the name of image and tag only.
* Config and secret change to invoke function update
* Added recycle function for pool manager as well - where it recycles the specialized pods
* Switched to rolling update of pods and env variable based change instead of deleting pods in new deploy executor
This PR removes not so useful logs and changes most of Info level
log to Debug/Error level in hot path while preserving some of them
that is helpful for troubleshooting.
Currently, only one CI build can be started at the same time due to we only have only one testing cluster. This PR aims to enable multiple concurrent builds can be triggered to reduce the waiting time for CI builds. The major changes listed below:
1. Each build generate unique images for testing and debugging
2. Check whether the testing cluster is being used by another build.
3. Cache docker image & go mod after build finished
4. Improve dockerfiles for reusing docker build cache
* Use terraform to manage ci cluster. Upgrade ci cluster to 1.13
* Upgrade helm version
* Do not break the test if no previous resources
* Do not skip install helm if helm is old version
* Add support for setting nodeSelector/affinity/tolerations on all deployments of fission components.
* Add extraCoreComponmentPodConfig helm value to allow setting of affinty/tolerations/nodeSelectors/etc on core pods
The prometheus server may not come up in a short time or when a cluster doesn’t have a storage class config. Because of this, the connectivity test will increase controller initialization time and so the controller will fail at readiness probe detection. Since we don’t want to delay the time that controller comes up also canary config manager prints error log when it failed at connecting to prometheus server, remove the connectivity test here.
Consolidates the addition of the fetcher container to the pod into a new type FetcherConfig and takes care of serviceAccountName if not set. Also added PreStop lifecycle handler if pod set TerminationGracePeriodSeconds
Currently the implementation of the CLI (or any Fission component) specifies any timeout on
the fission fn test functionality. If a user function or environment does not specify any
timeout the command will stall. Nor is there any parameter to set a timeout if the user
wanted to.
This change sets the sensible default for the timeout to 30 seconds, to avoid new
users having to figure this out the hard way. The alternative could be to set the timeout to
0 (no timeout).
Future work: extend this timeout parameter to a global flag, and support
timeouts on all CLI commands.
From go release history (https://golang.org/doc/devel/release.html#go1.11)
and travis ci build history, looks like 1.11.7 and 1.11.8 contains unknown
bugs that breaks build. To avoid this, fix go version to 1.11.6 for now.
* Added support for --plugin parameter in Fission CLI
This fixes the issue that renaming fission to fission-XYZ would cause a fork bomb, due to the CLI recursively calling itself to discover plugins.
As of K8s 1.8, RBAC mode is stable and backed by the rbac.authorization.k8s.io/v1
API. Using v1beta1 API version for RBAC based objects fails helm deployment
of fission on minikube
It should be possible to keep zero pool size when the environment is created so that there are no idle resources for a newdeploy function. Fixes: https://github.com/fission/fission/issues/1104
This removes fluentd in favor of using fluentbit, which is lighter (in
memory usage) and seems to be more actively maintained.
Fluentbit's config file format is different from fluentd's. It also
doesn't support the same record modification stuff that fluentd
supports, so we have to change the influxdb query slightly. This
means that after an upgrade, the new CLI may won't work for querying older
logs. Hopefully, this slight breakage is acceptable; if users
really need older logs they can use the older CLI.
The root cause of the issue was introduced by PR https://github.com/fission/fission/pull/1009/files .
To be short, even the CRD of environment was delete, it still takes time for executor (poolmgr) to destroy env pool. In our cases, the previous test creates an env and delete it when test finished, then the next one creates the same name env, but failed to create pool due to the deploy name conflict. So the executor selects the pod from the first created env pool. Then, executor starts to delete the env pool, and makes the pod state became Termination state. To fix this problem, a unique name of deployment will be returned after this PR to prevent the name conflict.
* Send the error message to user when enabling canary feature fails.
* Fix controller doesn’t reply canary error when promClient tries connect to invalid prometheus server
* Changed Kafka topic name validation so that capital characters are also allowed
* Removing support for \ in Kafka topic name
* Using raw string some specific sequences
Added support for Kafka headers to be passed and retrieved from functions. The headers and supported and work only for Kafka version 0.11.0.0 and higher.
* Refactor specialization process
* Separate readiness and liveness probe to different routes
For newdeploy, readiness probe should check whether a fetcher specializes env container successfully or not. In this commit, fetcher returns the actual state of current specialization status instead of returning 200ok directly.
Archives with multiple files should have all files in the inputs.
Also clean up some inconsistent variable naming -- archive name,
archive file path, and archive input files were all stored in
incorrectly-named variables.
* Remove version from release name since it contains illegal chars for names
* Put version in release name for uniqueness, but remove invalid chars
* Remove prometheus=false yaml
* Optimize router response time by adding update lock
In 0.9.2, the router sends multiple GetServiceForFunction requests to executor
to get the service URL. However, the response time of executor will increase
under high-concurrency situation due to too many requests are waiting for
processing.
To solve the problem, an update lock was added to the router. All of goroutines belongs
to the same function need to grab the update lock before sending the request.
Only the first goroutine which gets the update lock is allowed to send request.
In this way, we reduce the burden of executor and lower the failure rate.
A simple test for Kafka integration which runs locally provided cluster has Kafka and Fission install has Kafka integratiom enabled (mqtrigger-kafka deployment present).
Improved some of warnings/messages shown to the user when using `fission spec` command. The `spec validate` command now shows appropriate warnings instead
This PR introduces extensibility to the Fission CLI. The principle behind the design of this extensibility is taken from the approach git uses: the main binary (fission) calls other binaries (fission-workflows) - which it discovers using simple prefix-rules - based on specific commands (fission workflow <command>).
* Fix CLI failed to set up port-forwarding when multiple controller pods exist in the same namespace
* Check pod healthy state before establishing the connection
Java environment builder with support for Maven builds. Default build comman runs `mvn clean package` and picks up target/*with-dependencies.jar for function.
This changes image pull policy from Always to IfNotPresent. The previous
policy forbid customer to use the exact image existing on K8S Node.
And it should be OK to use fixed policy here. Customer need not set policy by
their own.
The Round Tripper parameters of timeout, keep alive time and Max retry
times is configured in router system environment.
And they are set to Round Tripper when router service initializing.
This setup new nested structure `tsRoundTripperParams` to transfer them.
This line warns the user that dependent environment doesn’t exist, and presents the command line
for creating an environment.
When the variable `envNamespace` is specified, the `envns` option should be available as well.
* Python Environment: add gevent based WSGI server framework
This supply gevent based WSGI server framework to Python Environment, to
allow customer to run their functions under multi-threads.
* classify the flask based application
This set up the flask based application as individual class, to make
sure the consistent and integrity.
* Add instruction of WSGI server selection applied in Python Environment
* Python Environment starting logger: provide framework name
For certain environments, the archive should not be extracted and kept as it is (For ex. for Java the jar file should not be extracted). This change enables an environment level flag to control this behaviour.
For archives bigger than 256K, the Storage service was called from the client side, this needed few environment variables to be set. This change uses port forwarding to achieve the same and does not need environment variables to be set.
This use constant to instead hard code for ArchiveLiteralSizeLimit. That ensure if the limitation
value is changed in cli, the controller server can follow it. Otherwise, the condition will return
error, and make error HTTP response.
Later, we should delete it in fission-cli, and make it configurable it chart, to work in fission-
servers only.
It is mandatory (from this release onwards) for function to refer to secrets, config-maps and packages in its own namespace to ensure isolation of users sharing the same cluster.
This change runs a pre-upgrade job to verify function references for all functions created prior to this release and fails the upgrade by printing a list of functions that violate this restriction.
Ingress integration to allow the optional creation of ingress for a given route. The ingress controller needs to be set up by the user separately so that ingress path is accessible outside the cluster.
This feature allows creation of fission objects in different namespaces, in addition to retaining the existing behavior of creating fission objects in default namespace if user doesnt provide one.
It also removes cluster admin roles for fission-fetcher and fission-builder Service Accounts and grants them only those privileges that they need.
* Fixes the issue with fn test and adds relevant test cases, fixes#650
* Checking log size for invalid function
* Testing the invalid function logs a few time beforr quitting
* Disabling the test for now, linked to issue #653
The transport in RoundTripper is the default transport and there's just one object shared between different http requests and releasing the idle Connections ended up releasing the idle connections that were established with the executor service address too.
So every time a http request was made to get a service for function, the transport layer ended up having to dial a new connection (that was timing out due to various reasons).
By removing the CloseIdleConnections, we minimize the need for transport to dial a new connection to executor for every request, thereby minimizing the occurrence of dial timeouts.
The router's cache entry for a function might become stale if the pod that had the function specialized gets deleted somehow. In such a case, we'd retry getting a new service for the function from executor and retry forwarding the user request to the newly created service.
Adds a validator, invoked through `fission spec validate` and before `fission spec apply`. The validator checks for duplicate names, dangling references, and unused resources.
Also attempts to fix a race between CLI and builder. Avoids replacing a package while it's building, instead waiting for the build status to change from pending before updating it.
The cli spec dir flag name should be `specdir`, not `specs`, that should match to:
```go
specDirFlag := cli.StringFlag{Name: "specdir", Usage: "Directory to store specs, defaults to ./specs"}
```
in fission/main.go
Currently it cause the specific specs dir not to work.
```
$ fission spec apply --specdir <my spec dir>
Everything up to date.
```
Tests for updates to a function of new deployment executor type. Tests check for changes in the environment, scale, secrets etc. Also checks conversion of function executor type from new deployment to pool manager and vice versa
The Deployment definition for executor, controller, router contains readines/liveness probes. The port definition for those probes should be numbers, instead of string, i.e. port: "8888" should be port 8888.
Environment Specs so far had only an image URL to specify a container image.
This was fine for public images but fell short in a few of cases:
(a) Using private image registries
(b) Specifying environment variables (this is needed for workflows helm install)
(c) Setting a SecurityContext for the container
This change adds the Container object to both build and runtime Environments.
Compatibility is preserved -- the existing ImageURL field is still used. See the comments in types.go for the overriding rules in the case that both Container and ImageURL are specified.
Newdeploy manager now checks the existence of function service cache by function UID before trying to create a new deployment. And return the cached fsvc directly if the cache exists.
Remove the hard coded default "fission" namespace that was added in the last PR, replacing it with some logic to figure out where fission is installed and connecting to it if there's exactly one installation. If there is more than one fission installation, error out with a useful message.
Also improve the error message for a missing kubeconfig file.
This makes set up easier for new users. Users can still set
FISSION_NAMESPACE but it will default to "fission". Users can also
still set KUBECONFIG, but it will default to $HOME/.kube/config.
Also, update the post-install chart notes.txt and the install guide to
use "fission function test" as the first step after install. This
means that if there's anything wrong with the setup, the user will see
useful errors instead of "internal server error". Also, they can test
their setup without worrying about nodeports or ingresses or whatever.
All existing functionality of FISSION_URL and FISSION_ROUTER continues
to work.
This the very first step for fission to integrate with Istio, which is an open platform to connect, manage, and secure microservices. With Istio, users are able to monitor functions usage and trace requests latency through dashboards. For more information, please visit http://fission.io/docs/
setting serviceType to ClusterIP as default for fission controller and corresponding changes in fission cli to be able to port-forward the controller pod.
The fetcher needs a relatively smaller set of resources and does not have to be same as the function container/defaults. This change adds defaults for fetcher containers in function pods.
* prevent leaking packages; add new clis to list and delete orphan packages.
* Fixes after testing.
* Address review comment.
* Removing deleting orphan packages from fnUpdate and fnDelete.
* goFmt and removing unwanted code.
These commits implement support for consuming messages from an Azure storage queue to trigger Fission functions.
* Add stubbed Azure message queue implementation and modify Helm charts.
This commit stubs an implementation for an Azure storage message queue trigger
that will be completed by future commits.
It also modifies the Helm chart to add support for deploying Fission with an
mqtrigger configured for Azure storage queue triggers.
* Add Azure Go SDK to glide.
This commit adds the Azure Go SDK to glide for the upcoming work to support
Azure storage queue triggers.
* Implement Azure message queue trigger.
This commit implements a message queue trigger based on Azure storage queues.
Required message queue trigger manager environment variables:
* AZURE_STORAGE_ACCOUNT_NAME - the Azure storage account to use.
* AZURE_STORAGE_ACCOUNT_KEY - the Azure storage account key.
When creating a message queue trigger, the topic will be the Azure storage
queue to receive messages from.
* Add CA certificates to fission-bundle.
This commit adds the root CA certificates to the fission-bundle image. This
allows Fission to contact third-party APIs that use HTTPS with root CA
signed certificates.
* Add Makefile to build and test.
This commit adds a simple Makefile for building the client and bundle, running
tests, creating the Docker image, and pushing the Docker image.
This change enables users to have declarative specifications for Fission resources. Users can specify their "app" in a set of spec files, and use a new fission CLI command to "apply" these specs to a running cluster.
The new "spec" CLI also includes archiving of local source files, and a file watcher that re-builds archives and uploads them on file changes, and a package build watcher that waits for package builds on the CLI.
A "--spec" option is also added to "function create", and will be added to other resources in future changes. This option causes a YAML to be outputted to the specs directory instead of the resource being created on the cluster.
The CLI "fission spec --help" outputs usage information.
This commit solves part of the issue #52 , functions are able to access secrets/configmaps specified by the user. For now, CLI only accept one secret/configmap. For advanced users, it will be able to use YAML to declare multiple secrets/configmaps in later changes.
A newdeploy backend which uses new deployment to serve requests. This is the second phase of #193 and builds on top of changes in #384 .
* Executor layer added on top of pool manager
* Removed the external server for executor
* Minor changes to keep existing semantics as much possible
* Separating the executor vs. poolmgr backend functionality and associated data members
* Executor logic separated from Poolmgr backend completely, placeholder for new backend
* Changed references to poolmgr in tests
* Moved poolmgr to it's package, as a side effect moved Cache to its's package (was causing cyclical dependency) and had to make some data structures exposed outside package
* Rebased from master and changed references to tpr -> crd
* Executor layer added on top of pool manager
* Executor logic separated from Poolmgr backend completely, placeholder for new backend
* Changed podName to a generic objectReference in fscache (#391)
Changed podName to a generic objectReference in function service cache implementation.
* Moved poolmgr to it's package, as a side effect moved Cache to its's package (was causing cyclical dependency) and had to make some data structures exposed outside package
* Rebased from master and changed references to tpr -> crd
* Merged from master with latest changes
* Executor layer added on top of pool manager
* Removed the external server for executor
* Minor changes to keep existing semantics as much possible
* Separating the executor vs. poolmgr backend functionality and associated data members
* Executor logic separated from Poolmgr backend completely, placeholder for new backend
* Changed references to poolmgr in tests
* update compiling.md to use helm
* Compile instructions: changed pullPolicy to IfNotPresent (#378)
Containers will get stuck in ErrImagePull/ImagePullBackOff state otherwise
* Moved poolmgr to it's package, as a side effect moved Cache to its's package (was causing cyclical dependency) and had to make some data structures exposed outside package
* Fetcher called when pod is created for newDeploy backend but also supports older way, this is WIP and still needs pod specialization and creating & exposing a service so the URL can be hit by end user
* WIP Specializing the POD as part of startup along with fetching
* Working specialization of a new deployment. Needs some work on caching, cleanup etc.
* Switched to service based address instead of POD address
* Minor formating issue fixed
* Added logging to pods and a readiness check, the readiness check is flaky though ATM
* Fixed some rebase issues that were failing build
* Better names for K8S objects and methods
* Switched usage of FuncSvc in backends from pod to api.ObjectReference
* Adding retry to fetcher request, for now just using default retry client which might need tweaking in future
* Switching to plain old retry, some issue in getting retryablehttp with glide import
* Removed stale executor service & deployment from previous merge
* Addressed review comments, still testing some areas
* Added types in FunctionSpec
* Resolved conflicts due to merge from executor_abstraction branch
* Added backend type on EnvironmentSpec along with operations for create/list/update, the pools are created/destroyed based on change in backend type
* Backend from types and a minor err return issue fixed
* Draft version of CPU and memory parameters added to environment
* Added resourceReq to newDeploy, though it has some issues
* Issue with resourceName fixed, now newdeploy pods also pick up resources from the environment config
* Adding scale params, removing validation on CPU params for now
* Fixed a formatting issue
* Checking if slight more delay helps in the test which is currently failing for internal routes
* The resourceList newly added in Env can not be compared by compiler, hence must use breakdown comparison instead
* Added strategy selection on client side
* Added caching, informers, delete operations for newdeploy backend functions
* Deleted a stale directory
* A simple HPA based on scale parameters, testing still WIP
* Fixed a small issue in delete function, added HPA delete too when deleting a function
* Previous merge missed the pkg flag for update fn command somehow, fixed that
* Fixed comments from review
* Changed poolmgr cleanup to be generic cleanup and moved to executor, added instanceID labels to newdeploy so that cleanup works
* Moved instanceIdLabel to types to avoid cyclic dependency
* More review fixes
* Tweaking sleep to see results
* If user does not provide poolsize, then it should not default to zero
* Switched to naming convention for now, fixed default poolsize if not provided
* Changed error return behaviour in delete fn, also changed cleanup to look based on obj type though support for additional type will need more work
* Changed check location so avoid false logging
* Test for newdeploy backend
* Adding tests for poolmgr backend
* Fixed an issue with glide dependency version, already fixed in master
* Added instanceId for NewDeploy, Initial cleanup now cleans older objects of newdeploy backend, removed eagercreate flag and instead using minScale to drive eager creation
* Moved cleanup to executor layer with cleanup for newDeploy backend, changes to use the new Cache impl
* Cleaning up pod & rs along with deployment for newdeploy backend
* Enhanced fn and env listing to show min/maxscale and resuorces respectively
* Added conditional heapster deployment and fixed a small issue with resources for fetcher container in function pod
* Addressed review comments from previous change
* Addressed some more review comments - majorly create only on NotFoundError
* Added TargetCPU as an input for scaling
* Bumped target CPU to be greater than 0 and added a default value
* Min replicas should be 1 even if the minScale is 0 when creating deployment
* Changed name from 'backend' to executorType, added additional test for minscale 0 case, changed TargetCPU to TargetCPUPercent
All functions have a pkg reference. This can be a package with either source and a deploy archives, or, a deploy archive. Everytime a function is updated, a new package is created. With archive pruner, the archives that are pointed to by old pkg reference can be deleted from the storage.
* High level spec for package pruning.
* Skeleton for archive pruning
* Adding meat 1 to skeleton.
* Adding meat #2. Separated storage service into a httpHandler component and
Storage Layer component.
* Adding meat #3. getOrphanedArchives in pruner and getItems on
stowClient.
* Restructured archivePruner methods.
* Commiting the day's work. Ready for testing #1.
* Fixing compile errors.
* Test ready. added a few logs for debugging.
* Adding a filter for getItems in stowClient.
* After testing.
* Added a test for archivePruner.
* Adding helm value pruneInterval for testing.
* Modified test.
* Final test.
* Fixing interval from seconds to minutes.
* Small change.
* Changing debugs to info.
* Removing the WIP design
* Ran gofmt on all these files.
* Fixing prune_interval as string in ENV var.
* Addressing all comments, but one.
* changing getFile method in stowClient to stream it into a response.
* All comments incorporated.
* Introducing a new flag for running archivePruner.
1. This flag is disabled for archivePruner to run in unit test.
2. This flag is enabled for archivePruner to run in production.
3. Also disabling test_archive_pruner.sh in this PR. Follow up with
next PR to enable it.
* Addressing review comments.
* Changing the command to generate a file dynamically.
* Enabling arching_pruner_test
* giving execute permissions to test_archive_pruner.sh
* Making changes of positional parameters after recent commit.
Change test case permission and removing kubectlPortForward.
* Adding debug to see why test_utils.sh passed junk pruneInterval.
* shell needs special handling for positional parameters from 10.
By default, flask turns redirects into absolute URLs. This is
undesired behaviour on Kubernetes, because the Pod's own IP address is
not really useful to any client that's not in the cluster (and even
within the cluster most clients will want to use the address of a
Service, not a Pod).
* Add readiness probe
* Remove builder manager http api interface since we don’t use/need it
* Check environment builder status and block build requests until builder is ready
* Replace deprecated api extension interface
* Add healthy check to python env
Modifies the fsCache to support a list of arbitrary kubernetes objects instead of just one pod. This allows better support for the newdeploy backend.
Also removes the byKubeObject cache index; cleanup is changed to use the byFunction index, and byKubeObject is no longer needed.
A few improvements to the integration test:
* Search for test files recursively in the test dir
* Output a test report at the end for a quick summary of what tests passed/failed/were skipped
* Add support skipping test files; skip the logging test since it's broken (#446 is tracking this)
* Switch to latest helm version
* Cleanup namespace
tested fission on openshift 3.7.0 and installed fine, without having to use openshift variables
those variables were used prior to switching to custom resource definitions, and as such are obsolete
Add package command support: This PR adds package command to CLI. package provides some useful subcommands to use such as packages CRUD and display package detail information. Also it is able to reuse existing package at function creation.
* Check function existence before creating package
* Check package existence
* Add support for downloading archive from given url
The functionality was supported before e238776bf7. Add this functionality back for more flexible usage.
* Add output flag to save archive content in specific file
* Add logs when package create/update
* Fix fnCreate requires environment argument when a pkg is specified
* Fix fnUpdate failed to update function pkg info when a package is specified
* Add package command test
* Fix wrong python test image in test script
* Remove package description fields
* Fix package command not update package but creating a new one
* Set package as pending status only when there is no deploy archive
* Rename function from fetchArchiveFromArbitraryURL to downloadToTempFile
* Use io.Copy to prevent loading all body into memory
* Revise some messages
* Allow user to update package build command
* Allow user to update package content when using function update
* Retrieve pkgName from function packageref if it’s not specified
* Fix function failed to update due to resource conflict
* Fix test case failure due to single quote
* kick ci
* Fix failed to update function packageRef
* kick ci
* kick ci
Before this change, poolmgr tried to hit fetcher just once. However if the pod has just started, fetcher may not be up yet, causing the initial request to fail with a connection refused error. This change adds a similar retry loop that we currently use on the specialize endpoint of the runtime environment container.
Create a container image with the buidler, so we can avoid copying the builder binary into the env container during the env container build.
Gather both stdout and stderr of a build into the logs.
* Add Dockerfile for builder + update release scripts
* Improve logging of buildermgr and builder
* Add support for environment-scoped buildcmd
* Fix fork/exec wd when package is a file
* Update tests to also include builder image
* Add logging to builder's fork/exec
This change adds a layer of abstraction over poolmgr. Poolmgr is now just one of the ways to turn a function into a service; other implementations will be added. The executor abstraction is a uniform API over all these implementations.
* Executor layer added on top of pool manager
* Removed the external server for executor
* Minor changes to keep existing semantics as much possible
* Separating the executor vs. poolmgr backend functionality and associated data members
* Executor logic separated from Poolmgr backend completely, placeholder for new backend
* Changed references to poolmgr in tests
* Moved poolmgr to it's package, as a side effect moved Cache to its's package (was causing cyclical dependency) and had to make some data structures exposed outside package
* Rebased from master and changed references to tpr -> crd
* Executor layer added on top of pool manager
* Executor logic separated from Poolmgr backend completely, placeholder for new backend
* Changed podName to a generic objectReference in fscache (#391)
Changed podName to a generic objectReference in function service cache implementation.
* Moved poolmgr to it's package, as a side effect moved Cache to its's package (was causing cyclical dependency) and had to make some data structures exposed outside package
* Rebased from master and changed references to tpr -> crd
* Merged from master with latest changes
* Removed stale executor service & deployment from previous merge
* Addressed review comments, still testing some areas
Remove the `logger` container from the logging daemonset.
Remove the outgoing call from the poolmgr to the logger. Use Fluentd's Kubernetes filter to add function name and UID to influx metadata.
This means fluentd now figures out when to start collecting function logs on its own, without being informed by poolmgr. This is great for other execution strategies, and for autoscaling, where fission isn't in direct control of function pod creation.
Also adds an integration test to make sure logging keeps working.
This fixes a bug where functionReferenceResolver returned out-of-date function metadata and caused the router to proxy requests to old function pods.
It also uses the go context package to shutdown the controller when the router is shutting down.
This contains a few different builder manager fixes:
* Update the zip file structure to avoid an extra subdirectory
* Annotate packages with what functions are using them
* Only trigger builds when there is no deployment archive
* Python environment loadpath bugfixes
* Other bugfixes
Switch Fission's storage over to the new CustomResourceDefinitions, from the deprecated ThirdPartyResources. This allows us to be compatible with Kubernets 1.8 and onwards.
This also adds a CLI tool for dumping state from an old fission version and restoring state into new CRDs.
The storage service is unaffected by this change.
closes#319
* Make default node-env use alpine, and support separate debian env. List envs in documentation.
* Copy ONBUILD base image commands into node Dockerfiles
* Copy more commands from onbuild image
* Tidy dockerfiles
Add a flag to the environment to control multiple specialization -- the max number of functions per container. This can be set to 1 or infinity.
Add an api proxy to workflow apiserver from the controller.
Add function metadata to FunctionLoadRequest; every v2 environment now knows which function it's loading (but can ignore that information if it wants to).
Add function identity headers to router. This is useful for multiple specialization, so the router can disambiguate between different function calls. (If this turns out to be a non-trivial perf overhead, we could add these headers conditionally, but for now they are always added.)
This change orchestrates function builds.
Environments (in v2) define a builder image, just like they do a runtime image. The builder image contains a build script that's invoked with source and deployment paths (as env vars).
The buildermgr watches for environments with build images defined, and creates build deployments and services.
Functions can define source and deployment. Buildermgr watches for functions with source code (and build status == pending) and invokes the environment's builder when appropriate. It captures logs from the build and sets the build status (success/failure) and build lots into the PackageStatus.
Since `mz` has already bumped for several times without
breaking change, it's better to follow the updates.
```console
$ npm outdated
Package Current Wanted Latest Location
mz 2.1.0 2.1.0 2.7.0 fission-nodejs-runtime
```
Fixes: https://github.com/fission/fission/issues/167
This adds partial support for multiple specialization to the
function service cache. It allows Add() to succeed if the pod and
address maps already contain entries.
But it doesn't deal with DeletePod and TouchByAddress. That's ok for
now, because the workflow engine never deletes the environment pod
(that pod is the workflow engine).
However, if/when we want full support for multiple specialization,
that will require updating both DeletePod and TouchByAddress.
Allows dumping v0.1 state to a json file and restoring it into a new v0.2.1 fission installation.
Usage guide is at: /Documentation/docs-site/content/upgrade-from-v0.1.md
Introduce _Archives_ as a type to reference arbitrary blobs.
_Packages_ are a pair of Source and Deployment archives. Packages have an environment reference.
Functions reference a package. Multiple functions can reference the same package.
Watch functions in the router and use that to trigger functionReferenceResolver cache invalidation.
We might want to rate-limit syncTriggers in a future change, since we might be triggering it more often than needed.
Fetcher now uncompresses archive files.
This is triggered by automatically detecting the file format, although maybe we should make that explicit or at least move the detection into the client.
This change contains CLI support for uploading large functions to the storage service.
It also adds a reverse proxy into the storage service to the fission API.
The helm chart is not yet updated to actually run the storage service -- that will come in the next change.
Build server and client for environments.
This build server will run inside environment builders and invoke environment-specific scripts for doing builds. It provides a uniform language-independent API for builds to the fission build manager.
The change also includes an implementation of a builder for Python -- this "builder" acts on requirements.txt and runs `pip install` to collect all function deps.
A very rudimentary storage service for archives larger than what can
fit in TPR/CRD resources. The library used allows for using local
files, AWS S3, or other cloud storage APIs. For now only local paths
are set up.
The storage service doesn't know anything about functions or packages;
it can be used for arbitrary archives.
There's no CLI integration yet.
Split out the Package type into a first class Kubernetes resource. Before this change, packages were implicitly tied to functions.
This wasn't ideal because:
* Functions will need to share packages
* A package storage system may be more generally useful than just
functions (for example, for storing static assets)
This change does the following:
* Updates the fission and tpr types to add a new Package and PackageSpec. It also creates a PackageRef type, and a FunctionPackageRef type. The PackageRef simply references a package, but the FunctionPackageRef includes the name of a function within the package. This allows us to share packages between different functions.
* Updates fetcher and other components for first-class packages
* Allows customization of fetcher image pull policy in the helm charts
Various e2e test bugfixes. The test runner now compiles fission bundle and cli, installs helm, installs kubectl, and uses helm to install fission, run all scripts under test/tests/, and uninstalls fission.
The change also removes go tip tests -- I think we can get by without them, but we'll add them back if the next go version update brings some surprises.
Build fission-bundle and fission cli. Install the fission-bundle using
the fission-all helm chart. Run all files matching test_*.sh from the
tests/ directory.
This change doesn't contain any real tests, just the build and test
runner scripts.
Skips the tests when there are no cluster credentials available.
Travis doesn't pass secret env vars to tests run on forks, so we need
a way to skip the tests when there are no credentials passed in.
This change unsets KUBECONFIG when there's no reachable cluster. The
Go tests already check for KUBECONFIG and skip when it's not defined.
Remove hardcoded "fission" namespace in various places.
Remove etcd deployment and service, since we use K8s APIs now.
Add ClusterRoleBinding for the fission service account, to allow fission components to access TPRs. We currently make the service account a cluster admin. This is more permissive than it should be; we should define a specific role for just fission resources.
This was initially added to avoid restarts if poolmgr starts before
fission api; but we don't really need that since k8s will restart
it. Also, poolmgr now talks to the k8s api directly instead of the
fission api.
Sets up Travis CI tests to use a Kubernetes cluster on GKE. All tests have access to $KUBECONFIG which points at a kubeconfig file with credentials to a cluster.
I had to skip the poolmgr unit test because it currently assumes NodePort services; that will be fixed in another change.
Also, because some of the tests don't work properly in parallel, I've ended up reducing the concurrency to 1. If this becomes a problem we'll have to fix the tests, but for now I don't mind just waiting a few minutes longer.
This changes the core fission function, environment and trigger types. It also changes Fission's storage to use ThirdPartyResources.
- Functions are now specified by packages. Functions can also have both source and deployment packages. A package can be specified by a literal, or by a URL.
- Environments have a build and runtime component.
- Triggers reference functions by a FunctionReference. This is a layer of indirection between triggers and functions, and will allow things like incremental function upgrades in future releases.
See Documentation/wip/env-v2.md for design discussion about points 1 and 2.
Changes:
* V2 Types
All types now have a spec, following the pattern of K8s objects.
Functions now have source and deployment packages. A Package can be
specified by literal, or by URL.
Environments now have a builder and runtime component.
All triggers use a new FunctionReference to specify the function. This
for now only uses a function name, but in the future can be extended
to be more flexible.
A new FunctionLoadRequest type is added for specialization requests to
the environment runtime.
* TPR types, TPR init code, and a "fission client"
Implements TPR types using the spec types in fission/types.go.
Adds code for adding creating TPR types, and convenient types for crud
operations on each of our resource types.
Adds code for connecting to K8s API and configuring a REST client with
fission types set up.
* Change old stateful controller into a thin apiserver
This apiserver is now simply a stateless api layer on top of the TPR
types. At the moment it doesn't do anything that couldn't be done by
simply talking to the TPR types. In the future we can have better
validation and potentially some higher level APIs (like versioning for
example) in here.
* Split controller client into files and update for v2 types.
* Update CLI for v2 types.
As far as possible we keep the CLI flags the same. We'll have to add
flags for source/deploy packages and builder/runtime
environments. That will come in the next change.
* Update poolmgr and fetcher for v2 types.
Also adds a poolmgr_test.
* Update router for new types.
Also adds a function reference resolver, which separates out the job
of resolving a FunctionReference to a function.
* Update kubewatcher and timer for v2 types.
* Update Message Queue trigger type for v2 types.
* Minor odds and ends.
* Fission bundle CLI updates
Remove controllerUrl flag, since we don't need it any more.
* Remove etcd deployment (replaced by storing state in TPR)
Also update the poolmgr commandline, and use an env var for the
fetcher image URL.
* Explicit ChecksumType and consts
* Clarify separation of environment interface types
Improve fission helm chart; split it into "fission core" and "fission all". fission-core is the minimal install, and fission-all will have all the bells and whistles.
* Update/Add fission-core & fission-all helm charts
* Fix incompatible types for comparison
* Add message queue trigger
* Rename openshiftRBAC to openshift
* Update documentation & comments
* Fix logger & kubewatcher failed to access kubernetes resource due to RBAC
Adds support for message queue triggers based on the NATS-Streaming message queue.
This lets the user map a queue topic to a function, and optionally send the function's response into another queue topic.
The message queue trigger manager is designed to be message queue independent, so we should be able add support for more queues by adding implementations for the relatively simple mqtrigger.MessageQueue interface.
* include path parameters in params hash for ruby environment
* Improve code organization and documentation for ruby environment.
* Document Ruby examples
Keep install docs in one place INSTALL.md instead of two.
Use github releases for all YAMLs and CLIs. This has a few advantages:
(a) it makes releasess more transparent
(b) all yaml/cli files are transferred over HTTPS
(c) CLIs and YAMLs being from the same release ensures that people
always get compatible versions (we will still occasionally break
api compatibility, until beta)
Addresses #158. Adds URL params into HTTP headers. A URL param named `KEY` with the value `VALUE` in the actual URL will appear as the header "X-Fission-Params-KEY: VALUE" in the fission function.
The poolmgr client now aggregates successive tapService() calls and sends them to the poolmgr service every 5 sec, instead of sending them immediately.
* Fix controller timer handler not return when error occurred
* Fix controller not close http response correctly
This PR aims to fix controller not close response correctly. Also, I’ve check all over the project there is no more resource leak issues exist after this patch.
* Fix controller not return immediately when error occurred
* Sort import
The generic pool manager creates a generic pool with the reference of environment metadata. Since the reference is address of loop variable, so the content of metadata will be replaced with the last element of envs. So pool manager sets up logging with wrong env.
Establish a proxy server from the controller to the log database.
Redirect query commands send from client to database then proxy back the db response.
Use parameter binding instead of fmt.Sprintf to prevent SQL injection.
This change adds a timer trigger API, client, and implementation. Users can trigger a function with a cron-compatible string.
This change also moves the publisher interface and webhook-based publisher implementation out of kubewatcher and into a separate `publisher` package.
Adds the minimal OpenShift components needed to deploy Fission with the
same functionality as with Kubernetes, using only a deployment template
file (fission-openshift.yaml). No need of objects creation via CLI.
Update deployment instructions in README.md
Primarily this involved removing the release namespace, which is
controlled by helm. If you want resources in a namespace you set
--namespace= with helm install, anything else leaves helm unable
to properly manage the resources.
I think the functions namespace won't work either on a delete.
Add function log aggregation and persistence using Fluentd and InfluxDB.
Fluentd and a helper sidecar run as a daemonset. The poolmgr sets up logging for each function pod, using the helper sidecar. Fluentd forwards logs to InfluxDB, which is run as a deployment and service. The client CLI directly queries InfluxDB for logs.
Fluentd supports many outputs besides InfluxDB, so we aren't very tied to InfluxDB.
The setup is somewhat manual, which we should be able to improve by integrating this into the helm chart.
Diagram of component interactions: https://cloud.githubusercontent.com/assets/202578/23100399/b0e3ea00-f6ba-11e6-8f2f-6588cfef2e84.png
Minimal golang runtime.
Functions are built as Go plugins, and the plugin is uploaded to fission.
See `environments/go/README.md` for instructions and `examples/go` for a usage example.
For now, we're re-using the 'code' field of the function to store the binary plugin. With v2 environments, this will be stored separately as a binary package.
When the request is handled successfully, json data is returned, this
patch changes the response Content-Type header from text/plain to
application/json
Fixes: #144
Upgrades the node environment to NodeJS 7.6.0. Functions can now use async/await and promises; they can return a promise instead of using a callback. The change preserves compatibility with the callback style.
* Upgrade node environment to Node.js 7.6.0+
* Bump package version
* Preserve compatibility for callbacks
* Cleanup
* Re-add hello callback example
* Make sure not returning won't blow everything up
* Update stock example with request promise
If no uid is provided, all versions of function files and function metadata will be deleted. If a uid is provided, only that specific version is deleted, and the "latest" version of the function moves back to the previous one.
Fixes#152
Fixes#139. The code before this change was saving up pointers to the loop variable in a slice. So loop was useless and we always chose the last pod in the list, whether it was ready or not. With this fix, choosePod should always return a ready pod.
Also adds a check for the pod phase, before looping over the containers in the pod, and discard pods that aren't running yet.
Before this change, if the user doesn't define a route for /, the
router happily 404s at that path. This breaks GKE Ingress, which uses
that request as a health check. So this change adds a handler for
"GET /", unless the user already has one.
Also, log the delay for slow requests.
Motivation: The output when an error occurs shows an integer error code,
which isn't very helpful. And in the case of an error when creating a
resource, you also get a redundant log message from the controller
client.
Modifications:
* the controller client no longer logs errors
* the generated String() allows for the enum name to be displayed in the
error output
* each errorCode enum member has a hand-generated description
* the format of the error message from the CLI client was modified to
display the error description with the error message
Related to #112
Test script that sets up fission on a cluster, runs tests and
uninstalls it.
Yet to come:
1. The actual tests
2. Builds (for fission-bundle, client, and envs)
3. Automation with github pull requests
Remove our non-standard namespace variable and use .Release.Namespace
to put fission deployments and services into.
(We still need the functionNamespace parameter, to customize what
namespace fission puts functions into.)
Motivation: Creating duplicate resources currently results in a 500
error being displayed by the client, with no information about the true
nature of the failure.
Modifications:
* ResourceStore now converts any errors from the etcd client into
fission errors, capturing the reason for the error in the case of a
duplicate key (as ErrorNameExists)
* ErrorNameExists errors are signaled to the client with a
409 (Conflict) HTTP status
* MakeErrorFromHTTP() now reads the body of the error response to
retrieve the actual error message instead of using the HTTP status
message
* the controller client now uses MakeErrorFromHTTP() to centralize
status code -> error code mapping
* tests for all of the resources now check that duplicate resources are
reported properly
Result: The client can now provide more context when a duplicate name is
given
related to #112
Addresses issue #113. poolmgr.MakeGenericPool does not need to wait
for a ready pod. Kubernetes already retries image fetching, so
there's no need to repeat the task in fission. And if it's a
non-retriable error (like the image URL being wrong), then the
deployment will stay broken, and the user can fix the environment with
'fission env update' (or 'fission env delete' followed by 'fission env
create').
* Cleaned up project.json
* Updated README with example on how to deserialize json request body
* Added reference to System.Runtime.Serialization.Json as this would seems a common usage
* Added C# example files to the /Examples directory
* Add initial support for PHP7
* Update PHP env with now an PSR3 logger (Monolog). Now you can also create a handler function with PSR7 Http message and PSR3 logger in parameter.
* Add examples for PHP7 environement
* For development you can create a function in this path /app/userfuncdev.php it will be loaded
* Switch to alpine base image
* handler now receive an array with request, response and logger object
Added FissionContext class
Changed name on both class and function for input code to avoid namespace clashes when adding FissionContext
Updated README
Added section on how to develop/debug the code to README, might be obvious for some but not all
Changed code input from static function to method
Poolmgr removed the default-latest-version semantics to simplify
caching. But this broke the router's internal function routes, which
are used by kubeWatcher. The router now exposes a versionless url
and regularly updates that to point to the latest version.
Label all poolmgr-created resources with an id that's unique to a
running poolmgr instance. On poolmgr start up, clean up resources
created by old instances. Resources that are idle are killed
immediately; resources that could be running a user function are
killed after the maximum function timeout.
Addresses issue #51. Makes versioning in poolmgr explicit, so its
cache needs no invalidation on version update. Functions in poolmgr
are always cached by name and UID.
Router now updates implictly versioned routes with the latest version
of a function.
This means that users will see requests to implicitly-versioned routes
go to the latest version of a function within 3 seconds. (Those 3 sec
will go away when we use a real watch instead of polling the
controller.)
There's no change in behaviour for routes that explictly specify a
function UID.
This allows users to add --method and --url params to their "fission
fn create" command line. It sets up the route to call the latest
version of the function; user can edit the route with the "fission
route update" command, if they want to.
These routes allow us to call any function without explicitly defining
a route for them. TODO: if we can serve these routes from a separate
instance of router (controlled by a commandline flag or env var), we'd
be able to keep this instance of the router "private" by not assigning
it a loadbalancer/nodeport or other externally-visible service.
Kubewatcher watches the Kubernetes API for Watches that have been
configured on the fission controller. It then posts that event to the
function configured in the watch.
The new command invokes editor on a copy of the source code, and
uploads the new source code. It's an easy way to manually edit
functions from a terminal.
* The easiest way to start is to look at existing [issues](https://github.com/fission/fission/issues) and see if there's something there that you'd like to work on. You can filter issues with label "[Good first issue](https://github.com/fission/fission/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)" which are relatively self sufficient issues and great for first time contributors.
- If you are going to pick up an issue, it would be good to add a comment stating the intention.
- If the contribution is a big change/new feature, please raise an issue and discuss the needs, design in the issue in detail.
* For contributing a new Fission environment, please check the [environments repo](https://github.com/fission/environments)
* For contributing a new Keda Connector, please check the [Keda Connectors repo](https://github.com/fission/keda-connectors)
### Get Help.
Do reach out on Slack or Twitter and we are happy to help.
* Drop by the [slack channel](http://slack.fission.io).
* Say hi on [twitter](https://twitter.com/fissionio).
# Contributing - building & deploying
## Prerequisite
- You'll need the `go` compiler and tools installed. Currently version 1.12.x of Go is needed.
- You'll also need [docker](https://docs.docker.com/install) for building images locally.
- You will need a Kubernetes cluster and you can use one of options from below.
`cmd` package is entry point for all runtime components and also has Dockerfile for each component. The actual logic here will be pretty light and most of logic of each component is in `pkg` (Discussed later)
| Component | Runtime Component |Used in|
| :------------- |:------------- |:-|
| fetcher | Docker Image |Environments|
| fission-bundle | Docker Image |Binary for all components|
**fetcher** : is a very lightweight component and all of related logic is in fetcher package itself. Fetcher helps in fetching and uploading code and in specializing environments.
**fission-bundle** : is a component which is a single binary for all components. Based on arguments you pass to fission-bundle - it becomes that component. For ex.
So most server side components running on server side are fission-bundle binary wrapped in container and used with different arguments. Various arguments and environment variables are passed from manifests/helm chart
**fission-cli** : is the cli used by end user to interact Fission
**preupgradechecks** : is again a small independent component to do pre-install upgrade tasks.
### pkg
Pkg is where most of core components and logic reside. The structure is fairly self-explanatory for example all of executor related functionality will be in executor package and so on.
```
.
├── pkg
│ ├── apis
│ ├── builder
│ ├── buildermgr
│ ├── cache
│ ├── canaryconfigmgr
│ ├── controller
│ ├── crd
│ ├── error
│ ├── executor
│ ├── fetcher
│ ├── fission-cli
│ ├── generator
│ ├── info
│ ├── kubewatcher
│ ├── logger
│ ├── mqtrigger
│ ├── plugin
│ ├── publisher
│ ├── router
│ ├── storagesvc
│ ├── throttler
│ ├── timer
│ └── utils
```
### Charts
Fission currently has two charts - and we recommend using fission-all for development.
```
.
├── charts
│ ├── README.md
│ ├── fission-all
│ └── fission-core
```
### Environments
Each of runtime environments is in fission/environments repo and fairly independent. If you are enhancing or creating a new environment - most likely you will end up making changes in that repo.
## Web UI (tracked separately in the fission-ui repo)
## Performance and Scalability ([area-perf](https://github.com/fission/fission/labels/area-perf))
- Autoscaling
- Cold-start optimization -- optimistically choose from pool, save about ~20msec
- Cold-start optimization -- preload funcs in fetcher
- Cold-start optimization -- preload libraries in envs (v2) -- mem vs. speed tradeoff
## Function extensibility ([area-ext](https://github.com/fission/fission/labels/area-ext))
- Env v2: easy addition of dependencies etc.
- Integration with Service Broker
## Multi-area stuff
- Execution strategies: cold-start pool vs create-pod-on-cold-start -- one size doesn't fit all, at least with current tech; abstract over execution strategies according to requirements
Multi-tenancy in Fission allows users to create Fission objects, i.e functions, packages, environments and triggers in different namespaces.
It mandates that a function reference secrets, configmaps and its package (if explicitly referenced during function create/update operation) to be present in the same namespace as the function.
This allows user separation and prevents in-advertent access to sensitive data of other users sharing the same cluster.
However, users are allowed and encouraged to share environments to ensure optimal utilization of cluster resources. To achieve this, users can create all the necessary environments in a ns, say ns1 and then go on to create functions in different namespaces and refer to env in ns1.
Users that prefer complete isolation can create their env, functions in the same ns.
## Roles and privileges
1. Cluster-Admin Role : Fission's services need cluster-admin privileges to monitor, create, update and delete resources across namespaces.
2. Package-getter Role : This role has privileges to do a get, watch and list on fission package objects.
3. Secret-Configmap-getter Role : This role has privileges to do a get, watch and list on secrets and configmaps.
## Service Accounts
1. fission-fetcher
This SA is created in every namespace that a user creates runtime environments in.
Also created in function namespaces where a user creates functions that use NewDeploy executor backend.
2. fission-builder
This SA is created in every namespace that a user creates builder environments in.
## Role-bindings
1. Package-getter-binding
Every time a user creates a package explicitly in a namespace, this role binding is created in package's namespace (which is also function's namespace). This grants package-getter role to fission-fetcher SA present in the referenced environment's namespace.
If the package is a source package, then, fission-builder SA present in the environment namespace is also added to this role binding.
Next when the user creates a function in the same namespace, if the function's executor type is newdeploy, then, the fission-fetcher SA present in function namespace is also added to the same role binding.
Note : For functions that have executor type poolmgr, the env pods are created in the namespace that env object is created. Whereas, for those functions that have executor type New deploy mgr, the function pods are created in the namespace that function object is created in.
This is because, poolmgr allows env sharing and optimal resource utilization. so generic env pools are created in a different namespace and all functions that prefer sharing this env pool can reference these pools.
If users require strict isolation, they can either create functions with new deploy backend, or, create envs in different namespaces and not share them across functions.
2. Secret-Configmap-getter-binding
Every time a user creates a function in a namespace, Secret-Configmap-getter-binding is created in the same namespace, granting secret-configmap-getter role to fission-fetcher SA present in the referenced environment's namespace in case the executor type is poolmgr.
If the executor type is newdeploymgr, then the same role binding is created in the same namespace as the function, granting the same secret-configmap-getter role to fission-fetcher SA present in the function namespace.
## Examples
1. create a generic python runtime env in ns 1 and function with poolmgr executor type in ns 2 that references it.
2. create a builder and runtime environment in ns3, a source pkg in ns3 and a function referring to this src pkg also in ns3. (for complete isolation, all objects are in ns3)
1. To maintain backward compatibility, fission objects that are created without the ns flags are created in default namespace. Also, the run time env pods in such a case will continue to live in fission-function ns and builder env pods in fission-builder ns
2. Since all envs in a namespace have the same fission-fetcher SA mounted in them, even though multiple envs are created in a namespace and referenced by functions in different namespaces, the SA will have privileges to view those function's secrets if any.
3. Similarly, if there are multiple functions in different namespaces but all sharing an env in one namespace, the fission-fetcher SA in that namespace will have privileges to see all of their secrets.
Annotations are used by the core Kubernetes system and to even larger extent by projects such as Istio Ingress Controllers and Prometheus and such.
Users want to add annotations to some objects such as ingress (https://github.com/fission/fission/issues/989).
To enable the users to use annotations, here are some thoughts and ideas:
## Defining annotations
Annotations can be defined fairly easily in the spec file for any object as part of metadata.
``` yaml
apiVersion: fission.io/v1
kind: HTTPTrigger
metadata:
creationTimestamp: null
name: spectest
namespace: default
annotations:
test-anno: some-test-value
spec:
createingress: true
```
These annotations can be merged to target object using a merging mechanism - so that additional annotations put by Fission can also be preserved.
## Considerations
- More often than not the annotations are needed by a Kubernetes objects created by one of the function CRDs/controllers. For example ingress created by route needs the annotation and not the route object itself.
### Implementation 1
- Most annotations use a convention which we can use to determine if an annotation is meant for an ingress object or to be applied on a pod.
For example. look at annotations:
|Annotation name| Description|
|:-------------|:-------------|
|`prometheus.io/scrape`| Prometheus - applied to pod|
|`sidecar.istio.io/inject`|Istio - applied to pod|
|`helm.sh/hook`| Used by helm to apply to pods/jobs|
|`traefik.ingress.kubernetes.io/app-root`|Used by Trafeik ingress controller, applied to ingress|
|`nginx.ingress.kubernetes.io/add-base-url`|Used by Nginx ingress controller, applied to ingress|
So we can write a simple logic - to check if a annotation is applicable for an ingress and based on that apply or not apply annotations to ingress.
### Implementation 2
- One of the side effects is that the annotations will still stay on the source CRD object - for example annotation will stay on the httptrigger as well as the ingress object. This can cause problems in certain cases where something like Prometheus uses annotations to scrape objects. So instead we wrap the annotations needed by an object into another annotation name. This also solves problem of having to guess which annotations to apply to which object.
This is specifically important because for example newdeploy function will create a deployment, service and HPA and all three might have different set of annotations.
Based on discussion in the team there is a additional option of adding a explicit field in the spec to hold the annotations. For now this assumes that we are only considering HTTPTriggers for annotations and not other objects such as Functions.
- The implementation idea 2 & 3 look better than 1. The third option involves HTTPTrigger Spec change.
- For both (2) & (3) - if in future we have to implement annotations for Functions etc. we will have to consider the fact that a function will in turn create 3 objects (Service, Pod & HPA) and annotations for all three would need to be accommodated.
# Continuous Integration and Delivery of Fission functions
This document outlines a simple CI/CD process for Fission functions which can be extended to any CI/CD tool. Before we start, some level setting for terminology used as the terms are used rather broadly in industry.
## Continuous Integration
CI is made up of a series of broad areas:
- The first step is to compile the source code and convert into artifact which is pushed to artifact repository. Traditionally this has been building a Py or wheel package (Python) or Jar file (Java) but as containers became mainstream the container image became the package. The traditional artifact repositories were replaced by the Docker registries.
- Execution and reporting of unit testing has been a crucial part of the CI cycle and is done after the source code can be compiled successfully.
- Running a static/dynamic code analyzer is the next step in continuous integration. The static code analysis is usually used to measure and report quality metrics and dynamic code scanning/analysis for security.
In the draft version of this proposal we will only consider the source to artifact conversion part and will not dive into unit testing or code scanning/analysis cycles of the CI.
## Continuous Delivery
CD is also composed of a few broad areas focusing on different aspects:
- After CI cycle completes successfully - deploying the artifact to a Dev/Staging environment so that it can be tested by developers and QA teams.
- Once the tests & teams have verified that a function works, the same function should be promoted from Dev/Stage to higher/production environment. The number of environment that a organization maintains varies but the idea of promotion from one environment to higher environment does exist. There are very few organizations who deploy the newer versions of functions directly in production with a A/B setup but that is as of this writing is an exception and not the norm.
- Another aspect of promoting from one environment to higher environment is the configuration for both environments will be different. For ex. the DB connection string will be different for each environment. Or the "maxscale" property for production environment could be higher than that for Dev. The ability to store all these environment specific configurations in some sort of system (Github for normal values and some sort of KMS for sensitive data) and being able to combine the logic and configurations for each environment when deploying is important.
Beyond these points there are integration/automation points such as being able to call a test suite after deployment is done etc. but we will skip for now for brevity.
## 1 Fission specs in a container
Let's start with a simple Fission function which uses specs. A typical directory structure looks like below:
```
.
├── multifile
│ ├── README.md
│ ├── __init__.py
│ ├── main.py
│ ├── message.txt
│ └── readfile.py
└── specs
├── README
├── env-python.yaml
├── fission-deployment-config.yaml
└── function-pyz.yaml
```
Irrespective of if source code that needs to be built or not, there is a simple command to fire to update the function, which will build (if applicable) and deploy the function:
```
$ fission spec apply
```
If we look at this from CI/CD perspective this process requires:
1. Source code & specs
2. A github push event when any one of the two change
3. Fission CLI
4. Kubernetes Config so that the apply command can be run
So if we build a container - which has the above requirements met as installed software (Ex. Fission & Kubectl CLI) or available as environment variable (Github pull token or Kubeconfig), the container can be used as part of CI workflow in any tool such as - Jenkins, Argo, Github Actions, GitLab etc.
The idea is to build a generic container with Fission CLI, Kubernetes CLI and a way to read Github token and Kubeconfig from env variable/mounted files and being able to run `fission spec apply` command.
### 1.1
Instead of building a container in previous section - the same can be achieved by a function. The Github webhook can call a function endpoint which in turn can execute the process similar to inside the container.
## 2 Environment Configurations
There are use cases and reasons to have environment configuration different for each environment such as Dev/Staging etc. Let's assume that we want to vary the `maxscale` in functions and `DB_CONNECTION` in environment
```
spec:
InvokeStrategy:
ExecutionStrategy:
ExecutorType: newdeploy
MaxScale: 2 // <-- Varies based on environment deployed in
MinScale: 1
```
```
container:
env:
- name: DB_CONNECTION
value: "http://database.url" // <-- Varies based on environment deployed in
```
Without changing anything in Fission spec it is possible to change these things from environment to environment and some of strategies used by people are:
1. Generate and maintain specs for each environment. This is not a best practice as it leads to drift in code and configuration between environment over time.
2. Use placeholder variables (i.e. $DB_CONNECTION_VALUE) and replace them for each environment before deploying. This is better in the sense that you are combining changes specific to each environment with spec code but is still a work around sort of.
For environment specific configurations, it is possible to use some sort of templates or overlay mechanism. One of interesting projects using overlays is [Kustomize](https://github.com/kubernetes-sigs/kustomize). In any case as of today the fission spec command does not have a way to use template or modify values using overlay and it is worth exploring this approach for fission spec.
## 3 Promotion from one environment to another
This necessarily does not fall in the area of Fission per se but it would be fairly easy to build a pipeline in the the tool used for CI/CD if we have container mentioned in (1) and even work around mentioned in (2).
## Action Items
As a first step it would be good to build a simple container mentioned in (1) and use it in various tools to understand the value it adds and any unknowns. The next steps would be to build a full end to end pipeline from source to production.
timetrigger, tt, timer Manage Time triggers (timers)for functions
mqtrigger, mqt, messagequeue Manage message queue triggers for functions
environment, env Manage environments
watch, w Manage watches
package, pkg Manage packages
spec, specs Manage a declarative app specification
upgrade Upgrade tool from fission v0.1
tpr2crd Migrate tool for TPR to CRD
help, h Shows a list of commands or helpfor one command
PLUGINS:
workflows, wf Inspect and manage workflow executions
ui Start the user interface
GLOBAL OPTIONS:
--server value Fission server URL (default: "http://127.0.0.1:65356")
--help, -h show help
--version, -v print the version
```
Of course Fission needs to be able to find all plugins for this. There are several ways in which we can provide discoverability. The simplest one is for Fission to look in the path for all binaries starting with the `fission-*` prefix. Optionally, fission could invoke a specific command on the subcommand to get info about the plugin (such as version, help text, aliases...)
With Fission Workflows this info would look something like this:
```bash
$ fission-workflows --plugin
name: workflows
version: 0.4.0
help: Inspect and manage workflow executions
```
The idea is that this plugin info is all completely optional.
If it is not available, we simply degrade the results to user.
This way users/we can easily prototype or add plugins without having to worry about adhering to some interface.
**List version**
```bash
$ fission --version
client:
fission: 0.8.0
fission-workflows: 0.4.0
server:
fission: 0.8.1
fission-workflows: 0.3.0
```
Again, versioning info for fission-workflows is taken from the plugin info of the commands.
Note: a related issue is to have some more formalized plugin support/discoverability on the server-side,
but that is out of the scope of this issue.
### Other (optional) extensions and notes
- Like git we could setup a preferred binary path, where fission looks first when searching for the subcommand.
This could optionally be defined with a `FISSION_EXEC_PATH`.
- With the current approach we cannot have aliases for commands---fission will not be able to find fission-workflows
when the user calls `fission wf`. This might be UX issue, with these long path names. One option is let the user fix
it themselves by symlinking `fission-wf` to `fission-workflows`; using the plugin info Fission can recognize and
merge aliases together.
- To help detect versioning conflicts (old version of fission, too new version of fission workflows). We could add
a `requires` field to the fission-workflows plugin info. Then we could throw a warning or error, when two out of sync
versions are being used.
- To avoid unhelpful errors to the user when they have not installed a plugin, we could add a heuristic to check
`https://github.com/fission/SUBCOMMAND` to see if the subcommand might be an uninstalled plugin.
OR, we could lookup a simple text file that contains common plugins `https://github.com/fission/fission/plugins.txt`
and list them as suggestions to the user. OR we could of course just default to a bit help text that says something
like `unknown subcommand 'foo'. If this is a plugin, ensure that it is present on your PATH`.
---
### Motivation
The proposed approach is to use the git-based plugin system for now. Reasons for this approach over a sophisticated,
integrated plugin-based approach:
- It is low effort to implement.
- It is easy to extend with minimal to no required interface.
- The binaries remain standalone, allowing users to separate them if needed and make independent development on the
binaries easy.
Limitations of the proposed approach:
- The user still has to do some work, adding binaries to the PATH; ensuring that permissions are correct; ensuring
that the binary is executable; how to deal with duplicate binaries on the PATH. All this makes this approach assume
basic/intermediate knowledge of the OS from the user.
- I have to admit: I am not entirely sure if this approach requires any changes for Windows. Probably not.
- Upgrading fission with many plugins could be cumbersome, as you would need to upgrade each binary one by one.
Improving this is probably best left to future work.
The more heavyweight solution solves some of these limitations to an extent, but these do not way up to the increased
development and maintenance cost IMO. If needed we could explore this option (or some hybrid option) in the future.
Fission's currently uses a pool of running "environments" and specialized them for execution of a function. This design served the cold start use cases well but this is not the only strategy for creation and execution of functions. For example requirements for a new execution backend have been discussed in https://github.com/fission/fission/issues/193. This document aims to discuss the currently under development "newdeploy" backend and related thoughts
# Executor
A new layer - executor now sits between the router and actual backends are responsible for all of heavy lifting for execution of functions. Executor layer is responsible for accepting requests from router and checking with cache before calling on a backend for execution of a function.
# Backend
A backend is responsible for execution of a function - which can involve provisioning appropriate objects in Kubernetes. So with the new design Pool manager becomes one of the backends. As of this writing there are two backends which are described as:
### Pool Manager Backend
Pool manager backend uses a pool of environment pods and specialized them when a function is invoked. The specialized pods are cleaned up if not in use after a few minutes. More details on Pool manager can be found here: https://github.com/fission/fission/blob/5c470735185b980c1f7987921db360e91c65573b/Documentation/Architecture.md
### New Deploy Backend
New Deploy backend create a Kubernetes deployment, a Kubernetes Service for a given function. It additionally creates a HorizontalPodAutoscaler if scale parameters are provided. The creation of deployment and service can be eager or lazy based on input.
### Execution Strategy
While this is still a WIP, parameters that affect execution behavior of function are based on `InvokeStrategy`. A invoke strategy defines the `strategyType` and actual strategy parameters encapsulated in the strategy object.
```
InvokeStrategy struct {
ExecutionStrategy ExecutionStrategy
StrategyType StrategyType
}
```
For example in above case the strategy type is `ExecutionStrategy` and the corresponding parameters are listed below.
```
ExecutionStrategy struct {
Backend BackendType
MinScale int
MaxScale int
EagerCreation bool
}
```
In future there could be more strategies for different use cases.
## Dispatch to backend
As of now one of the backends is chosen based on a simple flag in `ExecutionStrategy`. In future there might be a intelligent/hybrid ways of choosing a backend. For example initial requests of a function could be served from a pool manager while later scaling could be served by a NewDeploy backend
This document documents the design and thoughts that lead to design of Java environment. Before we dive deeper, some important points:
- When we say Java, we really mean JVM. That does not mean that all languages will work seamlessly, so support will be added gradually based on validation. Some of popular languages as of today are:
- Scala
- Groovy
- Kotlin (Server side with Spring)
- In Java there are a few prominent frameworks which have a ecosystem of their own (See list below). How these framework fit in the environment will be detailed later, but it is important to understand their place in ecosystem and design for it.
- A large percentage of enterprise developers use [Spring framework](https://spring.io/) as has been shown by multiple surveys
- Reactive has taken up recently with data intensive operations [Reactive extensions for JVM](https://github.com/ReactiveX/RxJava)
- [Spark](http://sparkjava.com/) is a micro web framework
A draft implementation of the Java environment design is in the branch [java_env_alpha](https://github.com/fission/fission/tree/java_env_alpha). Also a earlier implementation based on Vort.x framework [can be found here](https://github.com/tobias/fission-java-env/)
## Function interface
The goal is here is to minimize the lock in for the user into any framework as much as possible. Java 8 introduced an interface called ```Function``` which could be a great fit here. The user has to implement the ```Function<T, R>``` class and to meet the contract implement the apply method:
```
public class HelloWorld implements Function<T, R> {
public R apply(T str) {
```
Now - the T & R could be different things and we discuss some options below:
### Body in request and response
From the early implementation in the branch mentioned above, the environment extracts the body and send it as a JSON string. The JSON then can be transformed into the appropriate object by the function.
This works well, but has one major limitation: the function does not get access to other things like headers etc. The same thing applies to the response: function can send the body but looses control over status code etc.
### HttpServletRequest and HttpServletResponse
It is possible to send the [HttpServeletRequest](https://docs.oracle.com/javaee/6/api/javax/servlet/http/HttpServletRequest.html) request object as it is to the function class but then the interface becomes a bit too low level. For example the function user has to retrieve the body of request using ```getInputStream``` which gives raw input stream and needs additional work.
Also most enterprise applications today use a framework of some sort for web applications instead of dealing with the raw HttpServlet
### Custom/Context Object
A custom object which encapsulates all needed fields etc. can be used to pass the data from environment to the function. But this means the user has to import a Fission object/library for this object in the application code.
This approach has been taken in the implementation done earlier for a Java environment in Fission and [object interface can be found here](https://github.com/tobias/fission-java-env/blob/master/src/main/java/io/fission/api/Context.java). Related discussion is in the [issue](https://github.com/fission/fission/issues/91)
AWS Lambda also uses a context object, but the purpose is very different, [details of context object here](https://docs.aws.amazon.com/lambda/latest/dg/java-context-object.html).
### Spring's HttpEntity
If we have to depend on a class/library, it is probably better to depend on a class which is part of ecosystem. So instead of using the low level interface of Servlet, we can use [HttpEntity's subclasses RequestEntity and ResponseEntity](https://docs.spring.io/spring/docs/5.0.5.RELEASE/javadoc-api/org/springframework/http/HttpEntity.html). This ensures that the function user is not locked in the Fission object contract, but also gets the full access to request/response object.
The Spring cloud function project also discusses the issue of not having access to other things in request and [related issues are here](https://github.com/spring-cloud/spring-cloud-function/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+header)
### Thoughts
- If we only intend to pass request/response object to function - then using HttpEntity might be a good choice
- If there is a need for additional exchange of information between the environment and function execution in future, then a custom/context object is a better option. We can wrap the HttpEntity's fields and additional fields in the custom context object
## Environment Design
JVM environment design is based on Spring boot and Spring MVC frameworks. The details can be found in branch, but here are some key points:
- All classes in the function and dependent classes are loaded into JVM. Which means the user should supply the uber/fat jar for execution.
- The entrypoint class is specified by the user as ```entrypoint``` flag on the class. The method is by convention (```apply``` as per the Function interface contract)
Fission now has rich functionality supported by multiple services, however, it brings the complexity of troubleshooting.
This proposal tends to give a picture of fission support tool that can help both user and developer to locate the problem in short time.
To achieve this, the support tool will dump related kubernetes objects, fission resources and pod logs from the given cluster.
# Functionality
## Environment Information Collection
Before troubleshooting, some of the basic information is needed to give others an overview of kubernetes/fission user test with so that we can locate the problem in short time.
* Fission version
* Client/Server version
* Kubernetes cluster version
* Cluster version (i.e v1.9.7-gke.0)
* Running environment (i.e GKE, AKS and minikube)
* Nodes version and other information
## Service Logs Collection
The component logs and the logs of interaction between components are important for people to understand what really happened in cluster. Following are components need to collect logs from.
* All fission component pods
* Function pods
* Builder pods
* Environment pods
## Object dumping
Fission is deeply coupled with Kubernetes, most of the objects are created and maintained by it. There is two major type of objects need to be dumped from kubernetes:
* K8S objects
* CRD resources
All objects should be dumped into a readable file format. It will be great if people can reproduce similar environment with these files.
## Information upload
Upload dump files to the specific backend server for support channel to analysis
# CLI Interface
```
$ fission support collect
NAME:
fission support collect - Collect pod logs, fission resources and related kubernetes objects for troubleshooting
USAGE:
fission support collect [command options] [arguments...]
OPTIONS:
--dumpdir value Directory to save dump kubernetes objects and fission resources (default: "fission-dump")
--fissionns value Namespace of fission installation (default: "fission")
--builderns value Namespace of fission package builder (default: "fission-builder")
--funcns value Namespace of fission function pod (default: "fission-function")
```
# Thoughts?
1. What to do with sensitive objects like secrets and configmap? Ignore the dump for such objects?
2. The functionality is necessary but not listed above?
This proposal was initially started as a upgrade testing proposal but soon problems that were posed resulted in a bigger proposal.
### Kinds of testing
Most of current integration tests are CLI driven. Fission CLI is used to test execute various test cases. In future we would have to also focus on API level testing as a UI is built for Fission.
## Needs & patterns
This section only explains the problems/best practices without going into tooling and language used for implementation.
### Separating the test & data
Separating the tests from test data has two aspects - one is separation of concerns and second is scaling the tests without touching the test logic. The test data is a simple data structure which holds all information and test can take data and execute the logic.
As an example today we test "Hello world" for nodejs environment with a simple hello.js like this:
fission route create --function $fn --url /$fn --method GET
response=$(curl http://$FISSION_ROUTER/$fn)
```
The variables here are environment image, function code & route URL.
If tomorrow if we had to scale this test for all environments, we will have to repeat ourselves. (Violate DRY principle). Instead of that if we encapsulate the test setup & test in a simple function:
```
test_hello_env(envImage, codePath, routeURL){
}
```
And feed it with a dictionary which has all possible combination of tests:
- Separate the test execution logic from the data it needs clearly.
- For adding new kind of environments, you just need to add one more entry into data structure.
Testing all environments may not be most apt example for this, but there can be potential use cases like this.
### Separating the test & setup/teardown
When we run a test there are typically three distinct phases:
- Setup (Create env, fn, route)
- Test (Curl the function)
- Cleanup (Delete fn, route & env)
It should be possible to separate the before test and after test parts from actual tests at two levels:
- Each test
- A whole test suite
The ability to have clean and separate before and after blocks, apart from separation of concerns, enables:
- Running a suite of tests for same setup (See tagging for suite of tests)
### Tagging tests & running a selection
Over a period of time as tests grow, there will be unit, smoke, integration, performance, soak tests and so on. Ability to run a particular test suite only or a combination of them makes it easy to run for specific purpose.
### Measuring test times
[Good to have, not a must] Measuring time for tests and reporting somewhere helps over time to monitor trends. Although this job is better done by performance/benchmark tests so it is not a strict requirement
### Cleaner Logging
It would be good to have cleaner/relevant logging as part of build & test. For example something that Ginkgo framework does is it shows error logs only for failed tests.
### Tests in Parallel
It would be good to be able to run tests in parallel.
## Evaluating the tools/alternatives
### BATS
Bash Automated Testing System is like a enhanced version of bash with support for @test tags and before and after steps & ability to skip tests etc. While it enhances the bash to certain extent, the overall improvement is only marginal.
The testing package of Go also is quite feature rich for most of the use cases we need. Go 1.7 onwards there is support for setup & teardown parts and parallelism etc.
#### Go - Testing package
- Support for setup and teardown based on https://golang.org/pkg/testing/#hdr-Main
- Go testing already supports and has examples of table driven tests (Separating test & data), measuring test times and parallel tests
#### Shell execution: Go's Exec Library
GO provides a built in Exec library for working with CLI commands. The package seems good enough for us to work, though a few working examples will help decide better
https://golang.org/pkg/os/exec
### Using CLI package
Currently we build a CLI and then execute the tests. The tests basically call one of functions from CLI package. If we decide to use a go lang based framework, then we can import the CLI package and then call those functions by providing them context. This is as good as calling the Fission from CLI, with added benefit of programmability of Go language.
```
func TestSomething(t *testing.T) {
// Build the Cli context with flags etc.
ctx := cli.Context{}
// Pass the ctx to create function
fnCreate(ctx)
}
```
Some of benefits of using above pattern are:
- We can build a small framework around above core where we can pass various flag combinations etc. and exercise all flags in great detail
- We can use rest of Go testing library and other libraries to build matchers, looping, parallelism etc.
- It allows us to exercise the logic in CLI as well as validate the API at the same time.
### Ginkgo & Gomega
Ginkgo is a BDD framework which works with Gomega matcher library. I will state relevant portions of these two frameworks which can be utilized:
From Ginkgo:
- Global `BeforeSuite` and after `AfterSuite` can be used to have global setup and tear down phases
- For tests `BeforeEach` and `AfterEach` and more such variants to do before and after test tasks.
From Gomega:
Gomega is a matcher library but the `gexec` library makes it really easy to interact with OS execution environment. Some working examples:
- Build and cleanup the Fission CLI before & after the tests
- Gomega provides quite a few built in matchers - so you don't have to code those small usual checks, for example:
```
Ω(ACTUAL).Should(BeTrue()) // The output should be true
Ω(ACTUAL).Should(BeAnExistingFile()) // The file should already exist
```
There are many more matchers which cane be found here: http://onsi.github.io/gomega/#provided-matchers
- We can build custom matchers in Go language for reusable logic.
#### Links
Ginkgo: http://onsi.github.io/ginkgo/
Gomega: http://onsi.github.io/gomega/
## Thoughts & Next actions
Based on the discussion with team, here are current thoughts and next action items:
### Thoughts
- As far as possible we should stick to Go's built in testing package
- Ginkgo's cleaner logging feature (Onlu log if there are errors) - is very useful. We can decide to incorporate this in future.
- Gomega's (gexec)[http://onsi.github.io/gomega/#gexec-testing-external-processes] is really neat and some matchers can be used if necessary
### Action items
- How will upgrade test for Fission fit in the framework?
- How will migration of tests happen over time:
- Aim is to keep existing tests around so that enough validation is in place
- May be migrate one test at a time
- How much of current setup etc. will move into framework? For example it is clear that helm commands should be part of test framework as part of setup/teardown. But other sections may or may not be. A RCA needs to be done to analyze and come up with clear demarcation.
* See the [installation guide](https://docs.fission.io/docs/installation/) for installing and running Fission.
* You can learn more about Fission and get started from [Fission Docs](https://docs.fission.io/docs).
* See the [troubleshooting guide](https://docs.fission.io/docs/trouble-shooting/) for debugging your functions and Fission installation.
Fission is in early alpha. Watch this space for announcements soon!
## Contributing
Check out the [contributing guide](CONTRIBUTING.md).
Performance
===========
## Get Help & Community Meeting
The alpha release has focussed on cold-start latency performance. For
requests that don't have a running instance, i.e. a "cold start",
fission has a latency overhead of less than 100 msec for the NodeJS
environment.
Fission is a project by [many contributors](https://github.com/fission/fission/graphs/contributors).
Reach us on [slack](https://join.slack.com/t/fissionio/shared_invite/enQtOTI3NjgyMjE5NzE3LTllODJiODBmYTBiYWUwMWQxZWRhNDhiZDMyN2EyNjAzMTFiYjE2Nzc1NzE0MTU4ZTg2MzVjMDQ1NWY3MGJhZmE) or [twitter](https://twitter.com/fissionio).
A regular community meeting takes place every other Thursday at 09:00 AM PT (Pacific Time). [Convert to your local timezone](http://www.thetimezoneconverter.com/?t=09:00&tz=PT%20%28Pacific%20Time%29).
Meeting Link: https://zoom.us/j/413921817
The meeting agenda for next meeting and notes from past meetings are maintained in [this document](https://docs.google.com/document/d/1E-xw4KJgka4sUpETHxr9BJBYntzrtxlAN_CE3Wt8kws). You are welcome to join to discuss direction of project, design and implementation reviews and general questions about project etc.
## Official Releases
Official releases of Fission can be found on [the releases page](https://github.com/fission/fission/releases).
Please note that it is strongly recommended that you use official releases of Fission, as unreleased versions from
the master branch are subject to changes and incompatibilities that will not be supported in the official releases.
## Sponsors
The following companies, organizations, and individuals support Fission's ongoing maintenance and development. If you are using/contributing to Fission, we would be happy to list you here, please raise a Pull request.
| `fission-core` | FaaS essentials, and triggers for HTTP, Timers and Kubernetes Watches |
| `fission-all` | Log aggregation with fluentd and InfluxDB; NATS for message queue triggers; Fission-UI |
## Installing the chart
To install the chart with the release name `my-release`,
```bash
$ helm install --name my-release fission-all
```
## Uninstalling the chart
To uninstall/delete chart,
```bash
$ helm delete my-release
```
## Configuration
The following table lists the configurable parameters of the Fission chart and their default values.
Parameter | Description | Default
--------- | ----------- | -------
`serviceType` | Type of Fission Controller service to use. For minikube, set this to NodePort, elsewhere use LoadBalancer or ClusterIP. | `ClusterIP`
`routerServiceType` | Type of Fission Router service to use. For minikube, set this to NodePort, elsewhere use LoadBalancer or ClusterIP. | `LoadBalancer`
`repository` | Image base repository | `index.docker.io`
`analytics` | Analytics let us count how many people installed fission. Set to false to disable analytics | `true`
`analyticsNonHelmInstall` | Internally used for generating an analytics job for non-helm installs | `false`
`pruneInterval` | The frequency of archive pruner (in minutes) | `60`
`preUpgradeChecksImage` | Fission pre-install/pre-upgrade checks live in this image | `fission/pre-upgrade-checks`
`debugEnv` | If there are any pod specialization errors when a function is triggered and this flag is set to true, the error summary is returned as part of http response | `true`
`prometheus.enabled` | Set to true if prometheus needs to be deployed along with fission | `true` in `fission-all`, `false` in `fission-core`
`prometheus.serviceEndpoint` | If prometheus.enabled is false, please assign the prometheus service URL that is accessible by components. | `nil`
`canaryDeployment.enabled` | Set to true if you need canary deployment feature | `true` in `fission-all`, `false` in `fission-core`
`extraCoreComponentPodConfig` | Extend the container specs for the core fission pods. Can be used to add things like affinty/tolerations/nodeSelectors/etc. | None
`executor.adoptExistingResources` | If true, executor will try to adopt existing resources created by the old executor instance. | `false`
`router.deployAsDaemonSet` | Deploy router as DaemonSet instead of Deployment | `false`
`router.extraCoreComponentPodConfig` | Extend the container specs for the router fission pods. Can be used to add things like affinty/tolerations/nodeSelectors/etc. Overrides the generic `extraCoreComponentPodConfig`| None
`router.svcAddressMaxRetries` | Max retries times for router to retry on a certain service URL returns from cache/executor | `5`
`router.svcAddressUpdateTimeout` | The length of update lock expiry time for router to get a service URL returns from executor | `30`
`router.svcAnnotations` | Annotations for router service | None
`router.useEncodedPath` | For router to match encoded path. If true, "/foo%2Fbar" will match the path "/{var}"; Otherwise, it will match the path "/foo/bar". | `false`
`router.traceSamplingRate` | Uniformly sample traces with the given probabilistic sampling rate | `0.5`
`router.roundTrip.disableKeepAlive` | Disable transport keep-alive for fast switching function version | `true`
`router.roundTrip.keepAliveTime` | The keep-alive period for an active network connection to function pod | `30s`
`router.roundTrip.timeout` | HTTP transport request timeout | `50ms`
`router.roundTrip.timeoutExponent` | The length of request timeout will multiply with timeoutExponent after each retry | `2`
`router.roundTrip.maxRetries` | Max retries times of a failed request | `10`
### Extra configuration for `fission-all`
Parameter | Description | Default
--------- | ----------- | -------
`createNamespace` | If true, create `fission-function` and `fission-builder` namespaces | ` true`
For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/{{ .Chart.Version }}/fission-{{ .Chart.Version }}-windows-amd64.exe
For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/{{ .Chart.Version }}/fission-{{ .Chart.Version }}-windows-amd64.exe
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
