To follow 12-factor app rules, make jaeger collector endpoint
as an environment variable instead of CLI args. It's easier to
replace the value in different deployments. Also, we can utilize
valueFrom to get value from the configmap.
If a user deploys fission in the namespace which is different from the one in the single YAML file generated by helm, fission components won't be able to talk to each other due to the wrong namespace appends after the service address.
This PR adds --namespace when generating the YAML file to prevent the mismatch problem.
ReplicaSet generated by Deployment distributes pods to nodes based
on nodes resource usage, which means in some cases the newly created
pods may be assigned to only a few nodes. The requests will go to the
same node if there multiple pods on it and may increase the node
resource usage and overall request latency.
To solve the problem mentioned above, this PR allows users to deploy
the router as DaemonSet so that we can distribute requests across all
nodes for better workload distribution and lower latency.
In 1.9, kubernetes prompted Deployment to apps/v1 and finally deprecated
extensions/v1beta1 support in 1.16. Users will see the following error message
in log when fission components try to submit Deployment to the k8s API server
and specify extensions/v1beta1 as API group.
error: unable to recognize "deployment": no matches for kind "Deployment" in version "extensions/v1beta1"
This PR updates call to k8s API server and apiKind in chart files to avoid fission failure in 1.16.
This fix allows the empty repository tag in values.yaml file. The changes are done in the templates of deployment and fluentbit where repository tag will be added in the docker image name only when specified with some value in case of empty repository tag image name contain the name of image and tag only.
* Add support for setting nodeSelector/affinity/tolerations on all deployments of fission components.
* Add extraCoreComponmentPodConfig helm value to allow setting of affinty/tolerations/nodeSelectors/etc on core pods
Consolidates the addition of the fetcher container to the pod into a new type FetcherConfig and takes care of serviceAccountName if not set. Also added PreStop lifecycle handler if pod set TerminationGracePeriodSeconds
As of K8s 1.8, RBAC mode is stable and backed by the rbac.authorization.k8s.io/v1
API. Using v1beta1 API version for RBAC based objects fails helm deployment
of fission on minikube
* Optimize router response time by adding update lock
In 0.9.2, the router sends multiple GetServiceForFunction requests to executor
to get the service URL. However, the response time of executor will increase
under high-concurrency situation due to too many requests are waiting for
processing.
To solve the problem, an update lock was added to the router. All of goroutines belongs
to the same function need to grab the update lock before sending the request.
Only the first goroutine which gets the update lock is allowed to send request.
In this way, we reduce the burden of executor and lower the failure rate.
The Round Tripper parameters of timeout, keep alive time and Max retry
times is configured in router system environment.
And they are set to Round Tripper when router service initializing.
This setup new nested structure `tsRoundTripperParams` to transfer them.
For archives bigger than 256K, the Storage service was called from the client side, this needed few environment variables to be set. This change uses port forwarding to achieve the same and does not need environment variables to be set.
Ingress integration to allow the optional creation of ingress for a given route. The ingress controller needs to be set up by the user separately so that ingress path is accessible outside the cluster.
This feature allows creation of fission objects in different namespaces, in addition to retaining the existing behavior of creating fission objects in default namespace if user doesnt provide one.
It also removes cluster admin roles for fission-fetcher and fission-builder Service Accounts and grants them only those privileges that they need.
The Deployment definition for executor, controller, router contains readines/liveness probes. The port definition for those probes should be numbers, instead of string, i.e. port: "8888" should be port 8888.
This makes set up easier for new users. Users can still set
FISSION_NAMESPACE but it will default to "fission". Users can also
still set KUBECONFIG, but it will default to $HOME/.kube/config.
Also, update the post-install chart notes.txt and the install guide to
use "fission function test" as the first step after install. This
means that if there's anything wrong with the setup, the user will see
useful errors instead of "internal server error". Also, they can test
their setup without worrying about nodeports or ingresses or whatever.
All existing functionality of FISSION_URL and FISSION_ROUTER continues
to work.
This the very first step for fission to integrate with Istio, which is an open platform to connect, manage, and secure microservices. With Istio, users are able to monitor functions usage and trace requests latency through dashboards. For more information, please visit http://fission.io/docs/
setting serviceType to ClusterIP as default for fission controller and corresponding changes in fission cli to be able to port-forward the controller pod.
All functions have a pkg reference. This can be a package with either source and a deploy archives, or, a deploy archive. Everytime a function is updated, a new package is created. With archive pruner, the archives that are pointed to by old pkg reference can be deleted from the storage.
* High level spec for package pruning.
* Skeleton for archive pruning
* Adding meat 1 to skeleton.
* Adding meat #2. Separated storage service into a httpHandler component and
Storage Layer component.
* Adding meat #3. getOrphanedArchives in pruner and getItems on
stowClient.
* Restructured archivePruner methods.
* Commiting the day's work. Ready for testing #1.
* Fixing compile errors.
* Test ready. added a few logs for debugging.
* Adding a filter for getItems in stowClient.
* After testing.
* Added a test for archivePruner.
* Adding helm value pruneInterval for testing.
* Modified test.
* Final test.
* Fixing interval from seconds to minutes.
* Small change.
* Changing debugs to info.
* Removing the WIP design
* Ran gofmt on all these files.
* Fixing prune_interval as string in ENV var.
* Addressing all comments, but one.
* changing getFile method in stowClient to stream it into a response.
* All comments incorporated.
* Introducing a new flag for running archivePruner.
1. This flag is disabled for archivePruner to run in unit test.
2. This flag is enabled for archivePruner to run in production.
3. Also disabling test_archive_pruner.sh in this PR. Follow up with
next PR to enable it.
* Addressing review comments.
* Changing the command to generate a file dynamically.
* Enabling arching_pruner_test
* giving execute permissions to test_archive_pruner.sh
* Making changes of positional parameters after recent commit.
Change test case permission and removing kubectlPortForward.
* Adding debug to see why test_utils.sh passed junk pruneInterval.
* shell needs special handling for positional parameters from 10.
* Add readiness probe
* Remove builder manager http api interface since we don’t use/need it
* Check environment builder status and block build requests until builder is ready
* Replace deprecated api extension interface
* Add healthy check to python env
tested fission on openshift 3.7.0 and installed fine, without having to use openshift variables
those variables were used prior to switching to custom resource definitions, and as such are obsolete
This change adds a layer of abstraction over poolmgr. Poolmgr is now just one of the ways to turn a function into a service; other implementations will be added. The executor abstraction is a uniform API over all these implementations.
* Executor layer added on top of pool manager
* Removed the external server for executor
* Minor changes to keep existing semantics as much possible
* Separating the executor vs. poolmgr backend functionality and associated data members
* Executor logic separated from Poolmgr backend completely, placeholder for new backend
* Changed references to poolmgr in tests
* Moved poolmgr to it's package, as a side effect moved Cache to its's package (was causing cyclical dependency) and had to make some data structures exposed outside package
* Rebased from master and changed references to tpr -> crd
* Executor layer added on top of pool manager
* Executor logic separated from Poolmgr backend completely, placeholder for new backend
* Changed podName to a generic objectReference in fscache (#391)
Changed podName to a generic objectReference in function service cache implementation.
* Moved poolmgr to it's package, as a side effect moved Cache to its's package (was causing cyclical dependency) and had to make some data structures exposed outside package
* Rebased from master and changed references to tpr -> crd
* Merged from master with latest changes
* Removed stale executor service & deployment from previous merge
* Addressed review comments, still testing some areas
Switch Fission's storage over to the new CustomResourceDefinitions, from the deprecated ThirdPartyResources. This allows us to be compatible with Kubernets 1.8 and onwards.
This also adds a CLI tool for dumping state from an old fission version and restoring state into new CRDs.
The storage service is unaffected by this change.
This change orchestrates function builds.
Environments (in v2) define a builder image, just like they do a runtime image. The builder image contains a build script that's invoked with source and deployment paths (as env vars).
The buildermgr watches for environments with build images defined, and creates build deployments and services.
Functions can define source and deployment. Buildermgr watches for functions with source code (and build status == pending) and invokes the environment's builder when appropriate. It captures logs from the build and sets the build status (success/failure) and build lots into the PackageStatus.
Split out the Package type into a first class Kubernetes resource. Before this change, packages were implicitly tied to functions.
This wasn't ideal because:
* Functions will need to share packages
* A package storage system may be more generally useful than just
functions (for example, for storing static assets)
This change does the following:
* Updates the fission and tpr types to add a new Package and PackageSpec. It also creates a PackageRef type, and a FunctionPackageRef type. The PackageRef simply references a package, but the FunctionPackageRef includes the name of a function within the package. This allows us to share packages between different functions.
* Updates fetcher and other components for first-class packages
* Allows customization of fetcher image pull policy in the helm charts
Remove hardcoded "fission" namespace in various places.
Remove etcd deployment and service, since we use K8s APIs now.
Add ClusterRoleBinding for the fission service account, to allow fission components to access TPRs. We currently make the service account a cluster admin. This is more permissive than it should be; we should define a specific role for just fission resources.
