* Add fixes for failure in specialization
* reduce specialization in progress and remove expired requests from queue when specialization is timed out
* rename markSpecializationFailure and remove logger from the queue
* refactor clean up code in api.go and add test case for queue
Details:
- Cleanup svc waiting for the counter in the pool manager if specialization fails
- Cleanup active requests counter in pool manager if client exists the demand for function service while we have allocated function service
- Consider specialization timeout if pod ready timeout > specialization timeout in waiting for ready pod. We also consider if the request to choosePod is cancelled.
- We ensure if we have requests waiting for service requests but if there is no pod in the specialization we clean up those.
---------
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: Pranoy Kundu <pranoy1998k@gmail.com>
* add functionality to wait for specialization by keeping track of incoming requests
* format executor package
* fix required capacity to specialise new pod condition
* move handling concurrency logic into pool cache from executor
* remove unused methods and structs
* implement queue in to store the svc wait
* create a queue struct and its methods to handle concurrent inputs
* use newly created queue to store waiting for svc requests
* add waiting requests in queue and use them when a svc is ready
* set function to request in queue if the context is still alive
* remove concurrency approach to set svc for waiting requests
* update the active requests whenever requests from pool are assigned a svc
* add doc to define why the conditions exist
* remove unwanted params in strcut and clean up code
* set error while getting svc value if sum of specialization in progress and specialized is only more than concurrency limit
* remove duplicate functions and unnecessary values in struct
* close svc channel on set value and create constants for default concurrency and rpp
* get next value in queue in case context is timed out for fetched value
* remove specializationInProgress counter from pool cache
* return in case the queue is empty wihle setting func to svc
* test getSvcVaue and setSvcValue in poolcache
* add unit tests for GetConcurrent and GetRequestsPerPod methods
* reorder imports
* add fuzzy testing for getSVCValue and setSVCValue in poolcache
* restructure go mod file and update pool cache test cases
* Add tests and bug fixes
* refactor code and add test cases
* add svcWaiting check while setting svc value
---------
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
Merged pool cache package into fscache to avoid import cycle.
Also changed all types in pool cache from interface to specific
types.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* service account changes for fission-builder
* moved code into executor
* changed default timing to 30 min
* code refactor and add new roles for executor
* create role if serviceAccountCheck is enabled
* Use informerfactory across executor
* Run function informer for poolpodcontroller if istio enabled
* Use same namespace for secret as keda mqtriggers
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Remove cluster role references
* Convert secret/package getter cluster roles to role
* Remove all cluster role binding permissions
* Remove unwanted permission
* Try removing all RBAC-related code
* Remove additional constants
* Add functionality for service account checks
* Restrict permission across services
* Remove package informer factory from the executor
* Revert service account check code for now
* Skip adding roles in place of cluster roles
* Remove additional permission from the router
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Consider specific namespaces mentioned by the user in building informers in the executor
- Confimaps
- Secrets
- Deployments
- Services
- Pods
- Replicasets
Currently, we create Fission resources in the default namespace, function-related resources are created in the fission-function namespace, whereas builder resources are created in the fission-builder namespace. This causes confusion for a lot of users.
In this fix, we allow the user to set the function and builder namespace empty so that function and builder resources are created in the same namespace as the function resource always.
If the user desires older behaviour they can functionNamespace and builderNamespace the same previous before the upgrade.
* use default namespace for fission function and builder
* support for existing fission namespaces
* Replace builder and function namespace with template
* Fix namespace creation template
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
We allow functions to refer to configmap/secrets. We are monitoring all namespaces for config maps and secrets and also allow cross-namespace references.
This fix monitors configmaps/secret updates in specific namespaces. Also, we ignore cross-namespace references for configmap/secret updates.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Added properties to configure object reaper interval, global and specific to exec type.
OBJECT_REAPER_INTERVAL - global
NEWDEPLOY_OBJECT_REAPER_INTERVAL - for new deploy type
CONTAINER_OBJECT_REAPER_INTERVAL - for container type
POOLMGR_OBJECT_REAPER_INTERVAL - for poolmgr
Configmap inside pods for newdeploy and pool manager executor type were not being updated if the user update the configmap.
This fix will help to update the pods for both executor type with new configmap. As per the changes if there is any configmap update then pods will get restarted for both executor type and then it will refer new configmap.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
* Updated all Go language dependencies to latest version available
* Formatted all files as per gofmt
* Update Golangci-lint version to 1.48.0
* Updated action version wherer application in Github workflows
* Updated Kubernetes version to latest available
* Remove "io/ioutil" references and replace with "io"/"os"
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
The users can now set the pod spec for builder and fn pods via helm chart.
Currently we have set some default securitycontext for the pods. Before there were no permissions set and the user would by default enter root when kubectl exec into pod. Now the permissions have been set and the user will not be able to access root directory in poolmgr and newdeploy pods.
* Add support for custom metrics for HPA
* Cleanup TargetCPUPercent references from possible places
* HPA v2beta has 80% default cpu limit if not set
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Migrate HPA v1 to v2beta2
HPA v2beta2 is defined and supported from 1.19+ onwards.
Also HPA v2 is stable from 1.23 onwards. As we support 1.19+
onwards using HPA v2beta2.
This change is base for custom metrics support we want to add
later by modifying Function spec.
* Add unit tests for hpa operations
* Use constants instead of strings
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Created separate file for authmiddleware fn
* Optimize auth login and middleware
* Added unittests for authmiddleware
* Fixed authURL
* Removed featureConfig as global variable
* Fix integration test according to examples repo changes
* Fix integration test path for go module-example
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
Using interface makes it easy to create a fake client and unit test
a specific portion of the code. We should be able to more write unit
test and increase coverage of code with this change.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
In pool pod controller we were using pool namespace
rather than pod namespace in cleanup which was causing
issue in few scenarios. Using pod namespace now instead.
Also add unit test for scenario which was failing.
Using kubernetes client interface now across instead of
kubernetes ClientSet for testing.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Defining httpserver package to capture httpserver shutdown and
introduces uniform running of http server across codebase.
* Add unit tests for httpserver
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
- Enabled metrics in storagesvc, buildermgr and controller.
- Added a middleware in storagesvc, router, executor and controller to monitor total number of http requests, each request's duration and number of requests that are currently being served. These requests can be filtered on their path, method or statuscode.
- Removed functionCallDuration and functionCallResponseSize metrics from router.
- Removed funcAliveSummary, funcIsAlive, funcReapTime and idleTime metrics.
- Replaced function calls for collecting metrics to direct metric calls.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
We should avoid tap service call to executor if service URL retrieved from executor is empty.
Added sanity checks to ensure that.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Update Go version to 1.17 and dependencies
* Changes in ProbeHandler and LifeCycleHandler
* Update CRD definitions
* Update Go version to 1.17 in Github Actions
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
