* skeleton for envtest fission
* Refactor code and add CLI test
* hack
* Update server test
* remove skip-ci for lint tests
* Pass client go storagesvc
* Add clientGen interface across code
* Fix storagesvc test
* Fix cmd client
* add retry in server test
* Fix concurrenct access to pool deployment
* Remove old executor test
* get rid of ginkgo/gomega
* disable flaky test
* flaky test
* revert ci change
* handle err from ParseBool
---------
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: Pranoy Kundu <pranoy1998k@gmail.com>
- Update kubernets version 1.23+
- Update post chart install notes with namespace info
- Update upgrade notes in chart
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
- added retainPods flag to take in the number of specialized pods to retain
- add retainPods in both the create function and update function command
- modify crd keys to be typed instead of string
- keep track of function generation in case of update function operation
- add delete handler function to make sure specialized pods are deleted in case function is deleted
---------
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Signed-off-by: Pranoy Kundu <pranoy1998k@gmail.com>
Co-authored-by: Pranoy Kundu <pranoy1998k@gmail.com>
the FISSION_..._NAMESPACE environment variables are used by all services during init
to check if the CRDs are loaded.
Move them to the 'fission-resource-namespace' macro to ensure all services know the relavant namespaces.
Signed-off-by: Anthony King <anthony@datapane.com>
* Add fixes for failure in specialization
* reduce specialization in progress and remove expired requests from queue when specialization is timed out
* rename markSpecializationFailure and remove logger from the queue
* refactor clean up code in api.go and add test case for queue
Details:
- Cleanup svc waiting for the counter in the pool manager if specialization fails
- Cleanup active requests counter in pool manager if client exists the demand for function service while we have allocated function service
- Consider specialization timeout if pod ready timeout > specialization timeout in waiting for ready pod. We also consider if the request to choosePod is cancelled.
- We ensure if we have requests waiting for service requests but if there is no pod in the specialization we clean up those.
---------
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: Pranoy Kundu <pranoy1998k@gmail.com>
* add functionality to wait for specialization by keeping track of incoming requests
* format executor package
* fix required capacity to specialise new pod condition
* move handling concurrency logic into pool cache from executor
* remove unused methods and structs
* implement queue in to store the svc wait
* create a queue struct and its methods to handle concurrent inputs
* use newly created queue to store waiting for svc requests
* add waiting requests in queue and use them when a svc is ready
* set function to request in queue if the context is still alive
* remove concurrency approach to set svc for waiting requests
* update the active requests whenever requests from pool are assigned a svc
* add doc to define why the conditions exist
* remove unwanted params in strcut and clean up code
* set error while getting svc value if sum of specialization in progress and specialized is only more than concurrency limit
* remove duplicate functions and unnecessary values in struct
* close svc channel on set value and create constants for default concurrency and rpp
* get next value in queue in case context is timed out for fetched value
* remove specializationInProgress counter from pool cache
* return in case the queue is empty wihle setting func to svc
* test getSvcVaue and setSvcValue in poolcache
* add unit tests for GetConcurrent and GetRequestsPerPod methods
* reorder imports
* add fuzzy testing for getSVCValue and setSVCValue in poolcache
* restructure go mod file and update pool cache test cases
* Add tests and bug fixes
* refactor code and add test cases
* add svcWaiting check while setting svc value
---------
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
* use flag FnExecutionTimeout instead of FnTestTimeout
* use flag FnExecutionTimeout instead of FnTestTimeout
* keep FnTestTimeout and FunctionTimeout both
* code review changes
Signed-off-by: Shubham Bansal <shubhambansaliimtgn@gmail.com>
---------
Signed-off-by: Shubham Bansal <shubhambansaliimtgn@gmail.com>
* changes to push images in github container registry
* reverting changes for helm chart to pass CI test
* use gitURL instead of whole URL
* use gitURL instead of whole URL
* added release process for docker.io
* removed trailing spaces
* added support for multi arch images in GHCR
* set authorization header for fission version and check command
* display warning if response status is unauthorize or forbidden
* passing context to http request
* create roles for custom resource in function and builder namespace
* Create roles for CR for builder in function and builder namespace
* convert warn to info to remove error stack trace
Merged pool cache package into fscache to avoid import cycle.
Also changed all types in pool cache from interface to specific
types.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Fix issue with updating time trigger without --cron flag
* Add check with ResourceVersion
* Change in UpdateFunc
Signed-off-by: Shubham Nazare <shubham4443@gmail.com>
Signed-off-by: Shubham Nazare <shubham4443@gmail.com>
* service account changes for fission-builder
* moved code into executor
* changed default timing to 30 min
* code refactor and add new roles for executor
* create role if serviceAccountCheck is enabled
* Use informerfactory across executor
* Run function informer for poolpodcontroller if istio enabled
* Use same namespace for secret as keda mqtriggers
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Remove cluster role references
* Convert secret/package getter cluster roles to role
* Remove all cluster role binding permissions
* Remove unwanted permission
* Try removing all RBAC-related code
* Remove additional constants
* Add functionality for service account checks
* Restrict permission across services
* Remove package informer factory from the executor
* Revert service account check code for now
* Skip adding roles in place of cluster roles
* Remove additional permission from the router
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* trigger test job to ensure backward compatibility
* update builder and function namespace in the CI test
* resolve namespace to show logs for fission-CLI
* export function-namespace variable
* Use a common script for an old and new test run
* Revert empty namespace values in the chart
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
* Replace all cluster roles with autogenerated namespaced roles
* Use cluster role only for preupgrade api extension permissions
* Correct role definition typos
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: shaunak_deshmukh <shaunak@infracloud.io>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
Consider specific namespaces mentioned by the user in building informers in the executor
- Confimaps
- Secrets
- Deployments
- Services
- Pods
- Replicasets
We enable security context by default now, so that the fission pod do not have root access to the system.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* handle error condition in fission fn log command
* use the single stream for log exclude fetcher logs
* add all pods in the fn logs command
* update the previous stable version
* remove proxy to the controller for influxdb call
* stop running controller if influxdb flag turned true
* port-forward for influx DB
* remove controller dependency from test case
* port forward to influx DB if URL not provided by the user
* archive pruner test
* Backup Prometheus data from CI run
* use `kube-prometheus-stack` instead of `prometheus`
* Add dump-analyzer script
* Fix monitor helm chart values
* Update readme for dump-analyzer
* Disable grafana and alertmanager
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* handle error condition in fission fn log command
* use single stream for log exclude fetcher logs
* add all-pods in fn logs command
* update previous stable version
* add controller enablement flag
* throw an error if service not found
* add logs from Kubernetes in function log command
* pass context in function param
* add pod-namespace in function log command
* pass context in function param
* search for the pod in fn ns in the test
Currently, we create Fission resources in the default namespace, function-related resources are created in the fission-function namespace, whereas builder resources are created in the fission-builder namespace. This causes confusion for a lot of users.
In this fix, we allow the user to set the function and builder namespace empty so that function and builder resources are created in the same namespace as the function resource always.
If the user desires older behaviour they can functionNamespace and builderNamespace the same previous before the upgrade.
* use default namespace for fission function and builder
* support for existing fission namespaces
* Replace builder and function namespace with template
* Fix namespace creation template
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Convert ClusterRoles to Roles for all components for multiple namespaces
* Added templates for rules and role generator to dynamically generate roles for each component
* Fix role-generator component templating
* Enable namespaces variable for preupgradechecks
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: shaunak_deshmukh <shaunak@infracloud.io>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
* Add variables for function/builder namespace in the test
* Allow different namespaces in integration tests
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
Use the Kubernetes and Fission Client from CLI instead of Controller API.
This removes port-forwarding for the controller across Fission CLI mostly.
* Use configurable client in CLI
* Move resource namespace under cmd client
* use server to get fission version
* get archive with URL
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
We allow functions to refer to configmap/secrets. We are monitoring all namespaces for config maps and secrets and also allow cross-namespace references.
This fix monitors configmaps/secret updates in specific namespaces. Also, we ignore cross-namespace references for configmap/secret updates.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* added unit test for setup role binding method
* changes to throw an error in case of mismatch in cluster role name
* changes to add a check for role kind
* Role and RoleBinding for fission-fetcher and fission-builder can be created in additional namespaces
* Added template file for fetcher and builder role and rolebinding
This PR moves fission CLI as closer as possible to kubectl command behaviour. We have improved namespace handling behaviour across CLI.
* add namespace param for fn and env
* use common fn for ns check
* update validation
* default namespace for httpTrigger, env and package, config and triggers
* use default ns
* add namespace filter to spec
* add forceNamespace flag
* set current namespace
* add default namespace in config
* add namespace specific destroy
* add all namespace in the list of resources
* add namespace as global tag
* use %s instead of %v
* add test cases for namespace
* use ns in get all functions
* Separate service accounts for each component
* Permission changes per component
* Fixed fluentbit permissions
* added hooks to pre-upgrade service account and role
* Adjusted hook weights for pre-upgrade hook
* Replaced * with explicit permissions
Added properties to configure object reaper interval, global and specific to exec type.
OBJECT_REAPER_INTERVAL - global
NEWDEPLOY_OBJECT_REAPER_INTERVAL - for new deploy type
CONTAINER_OBJECT_REAPER_INTERVAL - for container type
POOLMGR_OBJECT_REAPER_INTERVAL - for poolmgr
These changes have specifically been made for Kafka connector. This will expose a new metric named `fission_mqt_message_lag`, which will show a number of messages lag per topic and partition.
We can use this metric in the auto-scaling of the pod for the new deploy type executor function. While creating a new deploy function we need to add hpa metrics of external type inside the function definition.
* allow two HTTP triggers with no URL and different prefix
* update dependency
* Fix controller existing tests
* Ensure namespace cleanup in API test
* update test cases
* handle error conditions in test
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
* Add grafana dashboards and configmap creation
* Fix Data source, job and template rules
* Add units to all panels
* fix units and fix linting issues
* Enable verbose linting for dashboards
Co-authored-by: shaunak_deshmukh <shaunak@infracloud.io>
Configmap inside pods for newdeploy and pool manager executor type were not being updated if the user update the configmap.
This fix will help to update the pods for both executor type with new configmap. As per the changes if there is any configmap update then pods will get restarted for both executor type and then it will refer new configmap.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
* Updated all Go language dependencies to latest version available
* Formatted all files as per gofmt
* Update Golangci-lint version to 1.48.0
* Updated action version wherer application in Github workflows
* Updated Kubernetes version to latest available
* Remove "io/ioutil" references and replace with "io"/"os"
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
1. Use websocket support introduced in Openetelemetry Librarries
2. Use propagators settings introduced in Opentelemetry Go SDK
3. Use samplers settings introduced in Openetelemetry Go SDK
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
The users can now set the pod spec for builder and fn pods via helm chart.
Currently we have set some default securitycontext for the pods. Before there were no permissions set and the user would by default enter root when kubectl exec into pod. Now the permissions have been set and the user will not be able to access root directory in poolmgr and newdeploy pods.
* Add API for listing storage service archives
If id is mentioned we allow user to download specific
archive. If id is not mentioned we list all archives
present with storage service.
This will allow us to build CLI with storage service
and help users to debug storage service.
* Added commands for storagesvc cli and functionalities
* Reusing code and added geturl and download.
* Fixed geturl for localstorage.
* Fixed description of fission archive command
* Added unit tests for function getstorageurl
* Added integration test for archive cli
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
* allow buildcmd command to process arguments
* Add unit tests for builder with different scenarios
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
* Add pre-check flag to root cmd
* Make precheck flag optional
* Remove pre from global optional flag
Signed-off-by: Shubham Nazare <shubham4443@gmail.com>
Following component has been updated with securityContext -
1. buildermgr
2. controller
3. executor
4. kubewatcher
5. router
6. timer
7. storagesvc
* Removed the security context from fetcher as per the offline discussion
* Add support for custom metrics for HPA
* Cleanup TargetCPUPercent references from possible places
* HPA v2beta has 80% default cpu limit if not set
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Migrate HPA v1 to v2beta2
HPA v2beta2 is defined and supported from 1.19+ onwards.
Also HPA v2 is stable from 1.23 onwards. As we support 1.19+
onwards using HPA v2beta2.
This change is base for custom metrics support we want to add
later by modifying Function spec.
* Add unit tests for hpa operations
* Use constants instead of strings
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Use latest goreleaser with GOAMD64 support
* Update Go version to 1.18.1
* Hardcode GOAMD64 in cli install
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Created separate file for authmiddleware fn
* Optimize auth login and middleware
* Added unittests for authmiddleware
* Fixed authURL
* Removed featureConfig as global variable
* Fix integration test according to examples repo changes
* Fix integration test path for go module-example
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
Using interface makes it easy to create a fake client and unit test
a specific portion of the code. We should be able to more write unit
test and increase coverage of code with this change.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
In pool pod controller we were using pool namespace
rather than pod namespace in cleanup which was causing
issue in few scenarios. Using pod namespace now instead.
Also add unit test for scenario which was failing.
Using kubernetes client interface now across instead of
kubernetes ClientSet for testing.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Defining httpserver package to capture httpserver shutdown and
introduces uniform running of http server across codebase.
* Add unit tests for httpserver
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
- Enabled metrics in storagesvc, buildermgr and controller.
- Added a middleware in storagesvc, router, executor and controller to monitor total number of http requests, each request's duration and number of requests that are currently being served. These requests can be filtered on their path, method or statuscode.
- Removed functionCallDuration and functionCallResponseSize metrics from router.
- Removed funcAliveSummary, funcIsAlive, funcReapTime and idleTime metrics.
- Replaced function calls for collecting metrics to direct metric calls.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
Currently we dial on localport and wait until, port forwarding
is done. Using channel instead from forwarder to wait explicitly.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
We are removing Fission deprecated Azure Storage Queue connector and
planning to adopt Keda going forward to have better
delegated functionality and more rich support.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
We are removing Fission deprecated Nats connector and
planning to adopt Keda going forward to have better
delegated functionality and more rich support.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
We should avoid tap service call to executor if service URL retrieved from executor is empty.
Added sanity checks to ensure that.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
- Use mqtrigger watch instead of polling mqtrigger every 5 seconds
- Added metrics to monitor no of subscriptions, and no of messages per subscription
- Add standard go metrics exported by prometheus
- Enable prometheus discovery for mqtrigger pod
- Optimized mqtrigger manager cache
- Add unit tests for mqtrigger cache
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
* Update Go version to 1.17 and dependencies
* Changes in ProbeHandler and LifeCycleHandler
* Update CRD definitions
* Update Go version to 1.17 in Github Actions
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Removing Prometheus dependency from fission chart
* User should install their own Prometheus installation
* Install Prometheus chart in Github CI workflow
* Change Prometheus query logging to debug
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
This change mainly fixes few things around router and executor
exposed metrices.
1. We are trying to follow standard in metric names.
2. Lables such as namespace are colliding with kube-prometheus standards
so they are getting relabled to exported_namespace. Added function prefix
to resolve this.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* fix helm templating on resources of router deployment
* add resources attribute to other fission deployments
Co-authored-by: crmendes <carlos.mendes@kelvininc.com>
* Change RBAC resource names work for multiple Fission release
* Fetch secret configmap and package cluster role based on the release name
* Remove default namespace hardcoding from helm chart
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* If a bucket already exists with the same name, allow the user to use that one
* Abstracted create or get a storage container
* Check err when the con is empty and return container name with error
- Currently, fission components don't handle shutdown signals.
So we don't get any to do the required cleanup before the fission process
exits. Adding signal capture process with cancelling context so
that all dependent processes stop working when the process gets term
signal.
- Set log level to error in otel shutdown function
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Currently when any of fission component exists, we fail to sync
log as logger.Sync is not called before exiting.
Restructured code so that we can logger.Sync before existing from
the fission bundle component execution.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Reactored ready pod controller code to user lister and cache sync checks,
so that we avoid querying lister if cache is not synced in choodPod function.
Also, as noticied in #2258 we were initializing workqueue in goroutine
which was causing nil pointer reference. We have moved it out of goroutine
and kept specific parts in goroutine.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Currently default pod termination logs go to /dev/termination-log.
With CI we need to change customize path somewhere to /var/log for
exporting logs with the kind export logs command.
Setting FallbackToLogsOnError as termination policy for skaffold.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
In Fission CI as well in a couple of scenarios, noticed Fission pods
getting preempted when Function pods are getting created. As both
Fission components and function pods have the same priority class
fission component pods might get preempted. It to be recommended to
set higher priority class for components such as Fission router, executor,
controller so that Fission functioning doesn't get disrupted.
Most of the Fission components work fine with restart except the executor. So
user should set priorityClass for Fission executor.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
- New flag `--specignore` has been added for command fission spec [list | validate | apply | destroy]
- This flag is optional.
- The default spec ignore file name is `.specignore`, if the flag is not used
- The spec ignore file existence will be checked in the `--specdir` path
- If spec ignore file is passed to the flag, with a name other than .specignore and if it does not exist in the `--specdir` path then an error will be returned.
- `--specdir` will be the root path against which the patterns will be evaluated
- The behaviour of the spec ignore file will be similar to .gitignore (only for yml and yaml files). For reference: http://git-scm.com/docs/gitignore
We would like to deprecate Fission Nats Integration and recommend
user to use Fission Keda Nats connector to handle Keda events.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Added options to allow disabling of pre-upgrade checks,
also providing options to mention image and image tag
for pre-upgrade.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Sync a few files from fission-all to fission-core chart
* Change values in fission-all chart
We keep the following components by default disabled now, which were enabled by default earlier.
1. nats - Fission Nats integration
2. influxdb - Influxdb and logger component
3. prometheus - Prometheus disabled by default
4. canaryDeployment - Disabled by default
This change reduces the need for a fission-core chart and we can configure
values of fission-all so that it can provide the behaviour of fission-all
as well as fission-core.
* Remove fission-core chart
* Add README in the fission-all chart
* Rename clusterrolebinding fission-crd to fission-cr-admin
* Add icon and sources in helm chart
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* security: Update go-uuid to recommended version
* security: Update mholt/archiver dep to recommended version
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Samplers: Check PR/helm values for supported types
* Propagators: Check PR/helm values for supported types
* Added tracing support via fission CLI
* Use parentbased_traceidratio as default sampler with 0.1 ratio
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Capture important open telemetry events with span in fission
* Add context to missing HTTP calls
* Add Trace ID in logs
* capture trace id in the proxy handler function
* Always registry tracer to get traceID
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
When fetch request is received by fetcher, it fetches
package information. Checking if package resource version mentioned
in fetch req matches with one received. Adding warning for now,
in future we need to ensure we give error if resource version
are not matching.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
In upgrade tests, sometimes in race condition pool manager podlister
takes time to sync in which case, poolpodcontrolller get nil reference
for gpm.podLister. Passing podInformer to poolpodcontroller so that
we wait for podInformer cache to sync.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
With addition of prefix based routes in 1.13.1, we have turned
all internal routes prefix based. This is causing wrong responses
in functions names with common prefixes such as `hello-world` and
`hello-world-consumer`. This patch fixes it where we add a fix route
for function name and prefix path for function name with `/`.
We improve overall prefix path handing with fix and prevent breakage
for existing flow.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Using controller-runtime zap integration which is aware of Kubernetesobjects and logs only name+namespace of the object when complete runtime.Object compatible type is given to logger.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Signed-off-by: Sanket Sudake sanketsudake@gmail.com
- Use informers and listers in executors
- Passing context properly in pool manager executor
- Environment updates in the pool manager would not cause updates in the deployment
- Environment update minimizing downtime
- Use replicaset controller and environment delete triggers to cleanup specialized pods
Generally using typed informers is more standard practise
than using SharedIndexInformer(SII) implicity. SII also lack
listers provided by informer factory and few other high level
abstractions.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Currently most of informers used in Fission are using resync
period of 30sec which is quite low and causes lot of disturbances
in overall system. Since informers already have watchers which would
be delivering events to the handlers. If any handlers require lower
sync period they should register handler with `AddEventHandlersWithResyncPeriod`
instead of we generaling informer.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* add opentracing section and otelCollectorEndpoint
* initialize OTLP exporter
* pkg/controller: changes for context propagation
* pkg/executor: changes for context propagation
* pkg/fetcher: changes for context propagation
* pkg/router: changes for context propagation
* pkg/storagesvc: changes for context propagation
* set no default value for otel collector endpoint
* update readme and add notes to charts
* move common code to pkg/utils/otel
* adding fn and env as attributes
* don't use otelhttp transport for websocket
* URL ignore with common filter UrlsToIgnore
Note: The web socket example does not work when using OTEL HTTP. Here is an issue related to that on open-telemetry/opentelemetry-js-contrib.
Signed-off-by: Gaurav Gahlot <gauravgahlot0107@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
The patch adds few improvements in pool manager and adds better
function composability by reorganizing code.
1. Added created status in get pool call
2. Improved logging in certain areas and having logger per component
3. Separated deployment-specific code in diff file for extensibility
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Currently when we update env, even if we dont have newdeploy related
functions env is getting is getting triggered. Added filter in finding
related functions to avoid this behaviour.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Change Ingress version from v1beta1 to v1 for K8s 1.22+ compatibility
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Minor fix
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
PR adds a new executortype which supports running containers as functions. New CLI under functions is added to create container as functions.
Co-authored-by: Harsh Thakur <harshthakur9030@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
1. Support for adding labels and annotations to functions & environment via fission CLI through create & update command.
2. Change ensures labels and annotations assigned to the environment would reflect on pods created via executor type pool manager and new deploy.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Retrieve pod metrics only if metrics server is running
Currently we query pod metrics every 30 sec which floods executor logs,
added check which confirms if metrics server is running then only we start
querying pod metrics for identifying CPU utilization.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Fixed couple of typos and misspells with Go CI
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Remove unnecessary conversions with Go CI
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* add multi env and multi k8s support on integraation tests
* test workflow
* test workflow
* test workflow
* remove test branch name
* correct kind action with image and version
* use same kind version as per kind images
* use OS as ubuntu-latest
* fetch upstream
* add .github/workflows to paths
* resolve conflicts
* make kindimage appear first on workflow
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
* Correct release name in release script for non helm install
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Increase test timeout to 20 minutes
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Enable codeql for master only
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Added GCP PubSub Image, enabled Keda by default and updated versions of all Keda connector images
Signed-off-by: Vishal Biyani <vrbiyani@gmail.com>
* Fixed the hyphens, it's been long since I have written Helm templates :)
Signed-off-by: Vishal Biyani <vrbiyani@gmail.com>
* Fixed the hyphens, it's been long since I have written Helm templates :)
Signed-off-by: Vishal Biyani <vrbiyani@gmail.com>
* Update charts/fission-all/values.yaml
Co-authored-by: Harsh Thakur <harshthakur9030@gmail.com>
* Update charts/fission-core/values.yaml
Co-authored-by: Harsh Thakur <harshthakur9030@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: Harsh Thakur <harshthakur9030@gmail.com>
* Add a SharedIndexInformer for services and deployments to NewDeploy executor.
This brings the NewDeploy executor behaviour into sync with GenericPoolManager behaviour
by caching Kubernetes services and deployments used in per-request function validation.
* Create informers in executer by executor label
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: James Sinclair <irgeek@btc.com.au>
* Enable prefix based routing
Signed-off-by: Harsh Thakur <harshthakur9030@gmail.com>
* Optimize checking condition
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Disable few tests
* disable router modification for now
* run code generator
* Collect fission dump in CI
* Enable all tests back
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Remove unwanted code
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Add prefix support at more places and couple of todo's
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Few more changes
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Improve function trim support
* Support for prefix based urls in fission function test
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* multi route for fission function test
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Improve validations in trigger creations
* Adjust leading / in url from fission
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Change suburl to subpath for function test
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
* Add a warning if spec is applied on a dirty worktree
* Update go dependencies after rebase
* Use the console package for showing the warning
* Fix merge conflicts
* Handle errors and use better messages
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
Since we recently migrated all fission examples to fission/examples repository, we need to change
respective references.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
We are facing issue in CI for multiple PRs due to mlock. It depends on kernel version or ulimit. Trying to switch to latest go version from 1.14 and see if it helps!
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Series are sorted internal by "order by", but the set of all series returned is not. Tried some ways to sort completely on database, found no way .. client side sort works at least
Co-authored-by: Vishal <vishal-biyani@users.noreply.github.com>
Adding once only execution pattern to pool manager based functions. This is for use cases where you don't want to share the samne pod for another execution instance!
In test_mqtrigger.sh and test_mqtrigger_error.sh tests, we timeout program
if not received right output. Ensuring we capture timeout signal and do program
cleanup accordingly. Also printing due to which signal program failed.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
Moved out environments and examples out of main Fission repo to their own repo. This is to allow changes to environment releases to be independent of main Fission releases.
Websocket event support for cleaning up pods only after WS connection is terminated. The support for websocket is right now in the environment itself and router simply acts as a proxy for WS communication!
Increase the default concurrency to 500 from 5 set in the function. Usually, it is good to error on the higher side and if needed, users can always fine-tune it to their needs.
This feature enables routing more than one request to a pod at the same time. This is the first draft of the work and might involve more optimizations later.
Passing information of function from router to executor is more efficient than calling the K8S API. This change does that instead of passing only metadata and then executor calling the K8S API again.
A ready pod which can be specialized was fetched for every function earlier, this has been changed to a queue and cache implementation in client-go to improve performance.
Poolmanager when tested at high load had some issues and this PR fixes one set of them which were found so far.
Co-authored-by: Vishal <vishal-biyani@users.noreply.github.com>
Concurrency in the pool manager allows specializing pods concurrently based on a specified limit.
Co-authored-by: Vishal <vishal-biyani@users.noreply.github.com>
Attribute enableServiceLinks can be now configured in PodSpec so that environment variables are not injected in function pods and DNS is used for discovery
This PR let the controller returns the detail error message
of kubernetes error which makes users understand what's
happening instead of meanless error msg.
This PR changes the behavior of controller API which
wrongly exposes sensitive data to the client. Now, the
API only returns success if secret/configmap exists;
otherwise, an error will be returned.
In some cases, creating a package and querying the package info
immediately the Kubernetes API server will return "not found"
error. So retry the query again after some time.
In some cases (#1384), users may want to access the file in the directory of deploy archive, however, we use the function UID name as the directory name and make it difficult for users to use as they don't know the real path of the directory.
This PR uses the fix name "deployarchive" to make things easier.
If the user applies package YAML file has no status field,
the package won't be able to be compiled or deployed due
to lack of status. This PR aims to add a check at buildermgr
to set initial package status to those packages.
Executor wrongly deletes role bindings if the user
creates an environment in the reserved namespaces. This
PR is a quick fix to solve the problem by checking if
an environment is under reserved namespaces.
This PR adds a fake controller client for local CLI operations like
offline spec generation or for unit test purposes. The fake client
now only implements the "Version()" function and more functions
will be implemented once we figure out how to achieve the goals
mentioned above.
The root cause of the problem is that eagerPoolCreator
tries to create the deployment when the poolmanager
is trying to delete it. To avoid this, start eager pool
creator after executor starts serving requests.
Adding fission spec list functionality. It lists Functions, Environment, Packages, HTTPTrigger, MessageQueueTriggers, Canary Config, TimeTrigger, KubeWatch created by an application specification.
The 3rd party package version used by the go server
may be different from the one in the user's source code
and will cause plugin version mismatched. Hence, we
should never import any external packages except the
Fission or built-in packages.
This PR replaces all 3rd-party packages with built-in packages.
This PR adds an interface for controller API client, it allows us to
implement mock API client for unit testing with ease and we are
able to generate spec file without accessing the real Fission server.
The pod template is embedded inside the deployment. So if
the pod annotation contains instance-id, the deployment
will get updated and thus triggers a rolling update whenever
a new executor starts which is unwanted.
After this PR, poolmanager will patches instance-id when a
pod is chosen for a function.
For newdeploy, unlike poolmanager manages the lifecycle
of function pod directly, newdeploy is only responsible
to create the deployment so we append instance-id to top-
level controller (deployment) only.
We used to update timestamp in the deployment environment field
in order to trigger a rolling update when the function referenced
resources get updated. However, use timestamp means we are not
able to avoid triggering a rolling update when executor tries to adopt
orphaned deployment due to timestamp changed which is unwanted.
In order to let executor adopt deployment without triggering a rolling
update, we need an identical way to get a value that can reflect res-
ources changed without affecting by time.
To achieve this goal, the sum of the resource version of all referenced
resources is a good fit for our scenario since the sum of the resource
version is always the same as long as no resources changed.
When a new executor starts up, it adopts the orphan kubernetes resources created
by the old executor instance. However, the adopted resource won't reflect the changes
come with the new executor, for example, the fetcher image inside won't be changed.
To solve this, executor updates the resource spec (HPA/Deployment/Service) with the
latest resources spec. By doing this, we can prevent the inconsistency between resources
created by different executor instance, also minimizes the impact on users.
Previously, once the executor is deleted for reasons (like upgrade or cluster scale-in),
the new executor deletes all existing resources created by the old executor and creates
new one. This mechanism becomes a problem when there are requests connecting to the
existing pods. Also in the worst case, the cluster may not have enough resources to create
new pods and cause service downtime.
This PR let each executor type adopts existing resources before starting the executor
API services, and so the alive connections won't experience failure. However, the requests
send to the function that doesn't have alive function pods will still fail due to the
executor is in bootstrapping.
The pool manager keeps terminating function pod periodically even there are
traffic to the function. The root cause is that executor, poolmgr, newdeploy
manage their own functionServiceCache separately. And when router taps a
function, executor updates the access time of the function service entry in its
own cache without notifying executor types to do the update as well. Hence,
the access time of function service entry in poolmanager cache never gets updated.
Due to the access time never gets updated, the idle pod reaper in poolmanager
then thinks the function pod is in idle state and recycle it.
This PR removes the cache in executor itself, and when router tries to tap a function,
executor will call executor type to tap the function and update access time.
The router taps function service one by one which is inefficient and
increases the burden of executor. This PR aggregates all requests into
one to solve the problem mentioned above.
When a function is created before the creation of the environment it's used, the newdeploy will not be able to create kube objs. Hence no function service record is inserted into the cache.
When getFuncSvc is called, the newdeploy tries to find the record in service cache in order to create kube objs with the same name used in previous kubeobjs creation. However, due to no record in the cache, a NotFound error is returned directly and causes the problem. To solve this, we use fn meta UID to ensure we always get the same obj name instead of getting it from the cache.
If the context of request is closed before ReverseProxy finishing writing
a huge response body to the response writer, the client will only receive
a truncated response body.
To solve this, move the context cancel after ReverseProxy finished.
To keep archive creation implementation simple and prevent any
confusion, we decided to remove `--keepurl` flag and embed URL
directly without downloading the file from it . In this way, we can
ensure consistent behavior in either package creation or spec file
creation. Also, it increases the portability of spec file.
1. The records are stored in redis which is not migratable to another cluster for the testing purposes.
2. Some of the requests fields are not recorded.
3. People should consider using https://github.com/buger/goreplay which is an existing mature and well-tested solution for testing purposes.
The function implementations of controller client package are
inconsistent. This PR lets functions reuse the functions
that already implemented and able to set additional headers to
request.
When `fn test` failed to retrieve logs from the function
pod, it turns to query logs from the log database. However,
the logdb type flag is not set to `fn test`, hence the logdb
type is empty (even without default value) and cause `fn logs`
returns an error due to unable to find the corresponding log
database type.
This PR adds logdb type flag to `fn test` to resolve the problem.
Jaeger preserves the trace data of router healthz endpoints
which is not so helpful for monitoring function metrics and
consumes storage to store such trace data.
This PR uses `GetStartOptions` to examine the request's path and
check if it's a request to healthz endpoint. If yes, skip and not
to trace it.
To follow 12-factor app rules, make jaeger collector endpoint
as an environment variable instead of CLI args. It's easier to
replace the value in different deployments. Also, we can utilize
valueFrom to get value from the configmap.
The helm pre-upgrade check failed due to the program tries to
check whether a multierr is a nil pointer instead of errors it contains.
This PR changes the way the program checks multierr in order not to block the upgrade.
Helm treats the single line "extraCoreComponentPodConfig" in
values.yaml as a key value pair config.
```
extraCoreComponentPodConfig:
```
And when a user tries to install helm with providing config
like following, it failed.
```
extraCoreComponentPodConfig:
nodeSelector:
serverless: fission
```
The root cause is that helm treats config in default values.yaml
and providing config as two different types of config, which is
key-value vs. value table. And so it's unable for helm to merge these
two configs even with the same key name.
This PR comments out the single line config in default values.yaml
to prevent the issue.
Before this PR, CLI functions fatal out when encountering error
instead of returning it. Such behavior makes it hard to reuse
the functions nor writing unit tests. This PR aims to make functions
return errors instead of error out.
Istio sidecar proxy blocks all requests sent through the reverse proxy
to the target service if the request.Host is not properly set to the
internal target service host. This PR sets the target service hosts
before establishing the proxy for the client in order to pass the
Istio sidecar proxy check.
Tensorflow Serving supports three kinds of API: predict, classify, regress. In order to fully support possible API types, this PR adds code to split the entry point with separator ":" to get model name and the API type separately. And append the API type to proxy URL when receiving the requests.
$ fission fn create --name t1 --pkg <pkg name> \
--env tensorflow --entrypoint "half_plus_two:regress"
The utility function GetMetadata uses wrong flag text to get the resource namespace and could cause the wrong results returned from the API server. This PR changes the function signature that allows users to pass in the flag text in order to get the correct value.
In the case of large files, it takes a long time for the user to download
the source from the URL and upload it to StorgeSvc through CLI.
This PR allows a user to use URL as the function source when creating a function
and provides a new flag "--keeparchiveurl" to let the user to decided
whether the CLI should download the file first or store the file URL in the
archive directly. If "--keeparchiveurl" is true, then no checksum will be
generated, it's the user's responsibility to ensure the file won't be changed.
When a function with executor type newdeploy got created, Newdeploy
is expected to create deployment/HPA/service for it and insert serviceEntry
to the cache for later use. Once clients call the function, newdeploy returns
the serviceEntry to the router.
However, the log shows that the newdeploy was unable to find the entry and
prints "Resource not found - key 'xxx' not found". The root cause is that the
informer controller instead of processing items in parallel, it dispatches XXFunc
to process items one by one. So if there is any problem during the creation of the
kubernetes resource, it takes a longer time to process the next item and hence
the serviceEntry was not inserted before clients call the function.
This PR lets the newdeploy to process items in extra goroutines instead of blocking
the process loop. It's a workaround to solve the problem above, we should consider
using workqueue to solve it in the future.
Encoded path means the URL path contains encoded string like "/foo%2Fbar".
Gorilla/Mux by default doesn't enable encoded path support, you need to enable
it when initializing the router.
This PR adds a new environment variable USE_ENCODED_PATH to the router
deployment to enable encoded path support.
If a user deploys fission in the namespace which is different from the one in the single YAML file generated by helm, fission components won't be able to talk to each other due to the wrong namespace appends after the service address.
This PR adds --namespace when generating the YAML file to prevent the mismatch problem.
ReplicaSet generated by Deployment distributes pods to nodes based
on nodes resource usage, which means in some cases the newly created
pods may be assigned to only a few nodes. The requests will go to the
same node if there multiple pods on it and may increase the node
resource usage and overall request latency.
To solve the problem mentioned above, this PR allows users to deploy
the router as DaemonSet so that we can distribute requests across all
nodes for better workload distribution and lower latency.
The merge function executor used wasn't merge container correctly, and it
didn't merge all fields in spec except volumeMount & Env which confused people.
To apply the user-configured container correctly, this PR changes the way of merge
and follows rules:
1. Slices are merged and return an error if the elements in the slice have name conflicts.
2. Maps are merged, the value of map of dst container are overridden if the key is the same.
3. The rest of the fields of dst container are overridden directly.
In 1.9, kubernetes prompted Deployment to apps/v1 and finally deprecated
extensions/v1beta1 support in 1.16. Users will see the following error message
in log when fission components try to submit Deployment to the k8s API server
and specify extensions/v1beta1 as API group.
error: unable to recognize "deployment": no matches for kind "Deployment" in version "extensions/v1beta1"
This PR updates call to k8s API server and apiKind in chart files to avoid fission failure in 1.16.
Router readiness probe failed due to kubelet cannot find router healthz endpoint.
This is caused that the healthz endpoint is added only when there is any update request
send to the updateRouterRequestChannel, and makes router failed.
This PR sends an update request right before the router is started to avoid the problem.
The router prints error no matter what error type it is.
It's useful for troubleshooting, however, it also prints
the context canceled error, which means that users abort
request before reply and its really normal nowadays. Also,
the router returns 502 if error is not nil and may confused
client if it's a timeout error.
To solve these problems, this PR adds an error handler to
reverse proxy to examine the return error and change the
status code when needed.
Previously, the spec doesn't update the package status if nothing
in the spec file changed. Due to this, the failed package will
always stay in the failed state.
This PR adds a check to see whether a package is in the failed state.
If yes, then changes the state to pending for builder manager to rebuild it.
Previously, a user has to delete HTTP triggers point to
the same function one by one. This PR adds new flag --function
to list & delete commands. So that user can delete or list all
triggers with the same function. However, this feature is not
working for triggers with canary deployment setup.
For each ingress controller, the format of ingress host,
path and annotations are different. To support different
kinds of controller, this PR adds new ingress config field
to http trigger spec. A user can set annotations, host and
path based on the type of underlying ingress controller
with CLI.
Command example:
fission route create --name foo \
--url /foo/{bar} --function foofn --createingress \
--ingressannotation "nginx.ingress.kubernetes.io/ssl-redirect=false" \
--ingressannotation "nginx.ingress.kubernetes.io/use-regex=true" \
--ingressrule "*=/foo/*"
* Add specializationtimeout flag to function create/update
* Set specialization timeout of 120 seconds if not present
* Add default newdeploy timeout for rest of the test cases
* Comment out specialization timeout in validations for compatibility
* Add warning if specializationtimeout is lower than default value
This fix allows the empty repository tag in values.yaml file. The changes are done in the templates of deployment and fluentbit where repository tag will be added in the docker image name only when specified with some value in case of empty repository tag image name contain the name of image and tag only.
* Config and secret change to invoke function update
* Added recycle function for pool manager as well - where it recycles the specialized pods
* Switched to rolling update of pods and env variable based change instead of deleting pods in new deploy executor
This PR removes not so useful logs and changes most of Info level
log to Debug/Error level in hot path while preserving some of them
that is helpful for troubleshooting.
Currently, only one CI build can be started at the same time due to we only have only one testing cluster. This PR aims to enable multiple concurrent builds can be triggered to reduce the waiting time for CI builds. The major changes listed below:
1. Each build generate unique images for testing and debugging
2. Check whether the testing cluster is being used by another build.
3. Cache docker image & go mod after build finished
4. Improve dockerfiles for reusing docker build cache
* Use terraform to manage ci cluster. Upgrade ci cluster to 1.13
* Upgrade helm version
* Do not break the test if no previous resources
* Do not skip install helm if helm is old version
* Add support for setting nodeSelector/affinity/tolerations on all deployments of fission components.
* Add extraCoreComponmentPodConfig helm value to allow setting of affinty/tolerations/nodeSelectors/etc on core pods
The prometheus server may not come up in a short time or when a cluster doesn’t have a storage class config. Because of this, the connectivity test will increase controller initialization time and so the controller will fail at readiness probe detection. Since we don’t want to delay the time that controller comes up also canary config manager prints error log when it failed at connecting to prometheus server, remove the connectivity test here.
Consolidates the addition of the fetcher container to the pod into a new type FetcherConfig and takes care of serviceAccountName if not set. Also added PreStop lifecycle handler if pod set TerminationGracePeriodSeconds
Currently the implementation of the CLI (or any Fission component) specifies any timeout on
the fission fn test functionality. If a user function or environment does not specify any
timeout the command will stall. Nor is there any parameter to set a timeout if the user
wanted to.
This change sets the sensible default for the timeout to 30 seconds, to avoid new
users having to figure this out the hard way. The alternative could be to set the timeout to
0 (no timeout).
Future work: extend this timeout parameter to a global flag, and support
timeouts on all CLI commands.
From go release history (https://golang.org/doc/devel/release.html#go1.11)
and travis ci build history, looks like 1.11.7 and 1.11.8 contains unknown
bugs that breaks build. To avoid this, fix go version to 1.11.6 for now.
* Added support for --plugin parameter in Fission CLI
This fixes the issue that renaming fission to fission-XYZ would cause a fork bomb, due to the CLI recursively calling itself to discover plugins.
As of K8s 1.8, RBAC mode is stable and backed by the rbac.authorization.k8s.io/v1
API. Using v1beta1 API version for RBAC based objects fails helm deployment
of fission on minikube
It should be possible to keep zero pool size when the environment is created so that there are no idle resources for a newdeploy function. Fixes: https://github.com/fission/fission/issues/1104
This removes fluentd in favor of using fluentbit, which is lighter (in
memory usage) and seems to be more actively maintained.
Fluentbit's config file format is different from fluentd's. It also
doesn't support the same record modification stuff that fluentd
supports, so we have to change the influxdb query slightly. This
means that after an upgrade, the new CLI may won't work for querying older
logs. Hopefully, this slight breakage is acceptable; if users
really need older logs they can use the older CLI.
The root cause of the issue was introduced by PR https://github.com/fission/fission/pull/1009/files .
To be short, even the CRD of environment was delete, it still takes time for executor (poolmgr) to destroy env pool. In our cases, the previous test creates an env and delete it when test finished, then the next one creates the same name env, but failed to create pool due to the deploy name conflict. So the executor selects the pod from the first created env pool. Then, executor starts to delete the env pool, and makes the pod state became Termination state. To fix this problem, a unique name of deployment will be returned after this PR to prevent the name conflict.
* Send the error message to user when enabling canary feature fails.
* Fix controller doesn’t reply canary error when promClient tries connect to invalid prometheus server
* Changed Kafka topic name validation so that capital characters are also allowed
* Removing support for \ in Kafka topic name
* Using raw string some specific sequences
Added support for Kafka headers to be passed and retrieved from functions. The headers and supported and work only for Kafka version 0.11.0.0 and higher.
* Refactor specialization process
* Separate readiness and liveness probe to different routes
For newdeploy, readiness probe should check whether a fetcher specializes env container successfully or not. In this commit, fetcher returns the actual state of current specialization status instead of returning 200ok directly.
Archives with multiple files should have all files in the inputs.
Also clean up some inconsistent variable naming -- archive name,
archive file path, and archive input files were all stored in
incorrectly-named variables.
* Remove version from release name since it contains illegal chars for names
* Put version in release name for uniqueness, but remove invalid chars
* Remove prometheus=false yaml
* Optimize router response time by adding update lock
In 0.9.2, the router sends multiple GetServiceForFunction requests to executor
to get the service URL. However, the response time of executor will increase
under high-concurrency situation due to too many requests are waiting for
processing.
To solve the problem, an update lock was added to the router. All of goroutines belongs
to the same function need to grab the update lock before sending the request.
Only the first goroutine which gets the update lock is allowed to send request.
In this way, we reduce the burden of executor and lower the failure rate.
A simple test for Kafka integration which runs locally provided cluster has Kafka and Fission install has Kafka integratiom enabled (mqtrigger-kafka deployment present).
Improved some of warnings/messages shown to the user when using `fission spec` command. The `spec validate` command now shows appropriate warnings instead
This PR introduces extensibility to the Fission CLI. The principle behind the design of this extensibility is taken from the approach git uses: the main binary (fission) calls other binaries (fission-workflows) - which it discovers using simple prefix-rules - based on specific commands (fission workflow <command>).
* Fix CLI failed to set up port-forwarding when multiple controller pods exist in the same namespace
* Check pod healthy state before establishing the connection
Java environment builder with support for Maven builds. Default build comman runs `mvn clean package` and picks up target/*with-dependencies.jar for function.
This changes image pull policy from Always to IfNotPresent. The previous
policy forbid customer to use the exact image existing on K8S Node.
And it should be OK to use fixed policy here. Customer need not set policy by
their own.
The Round Tripper parameters of timeout, keep alive time and Max retry
times is configured in router system environment.
And they are set to Round Tripper when router service initializing.
This setup new nested structure `tsRoundTripperParams` to transfer them.
This line warns the user that dependent environment doesn’t exist, and presents the command line
for creating an environment.
When the variable `envNamespace` is specified, the `envns` option should be available as well.
* Python Environment: add gevent based WSGI server framework
This supply gevent based WSGI server framework to Python Environment, to
allow customer to run their functions under multi-threads.
* classify the flask based application
This set up the flask based application as individual class, to make
sure the consistent and integrity.
* Add instruction of WSGI server selection applied in Python Environment
* Python Environment starting logger: provide framework name
For certain environments, the archive should not be extracted and kept as it is (For ex. for Java the jar file should not be extracted). This change enables an environment level flag to control this behaviour.
For archives bigger than 256K, the Storage service was called from the client side, this needed few environment variables to be set. This change uses port forwarding to achieve the same and does not need environment variables to be set.
This use constant to instead hard code for ArchiveLiteralSizeLimit. That ensure if the limitation
value is changed in cli, the controller server can follow it. Otherwise, the condition will return
error, and make error HTTP response.
Later, we should delete it in fission-cli, and make it configurable it chart, to work in fission-
servers only.
It is mandatory (from this release onwards) for function to refer to secrets, config-maps and packages in its own namespace to ensure isolation of users sharing the same cluster.
This change runs a pre-upgrade job to verify function references for all functions created prior to this release and fails the upgrade by printing a list of functions that violate this restriction.
Ingress integration to allow the optional creation of ingress for a given route. The ingress controller needs to be set up by the user separately so that ingress path is accessible outside the cluster.
This feature allows creation of fission objects in different namespaces, in addition to retaining the existing behavior of creating fission objects in default namespace if user doesnt provide one.
It also removes cluster admin roles for fission-fetcher and fission-builder Service Accounts and grants them only those privileges that they need.
* Fixes the issue with fn test and adds relevant test cases, fixes#650
* Checking log size for invalid function
* Testing the invalid function logs a few time beforr quitting
* Disabling the test for now, linked to issue #653
The transport in RoundTripper is the default transport and there's just one object shared between different http requests and releasing the idle Connections ended up releasing the idle connections that were established with the executor service address too.
So every time a http request was made to get a service for function, the transport layer ended up having to dial a new connection (that was timing out due to various reasons).
By removing the CloseIdleConnections, we minimize the need for transport to dial a new connection to executor for every request, thereby minimizing the occurrence of dial timeouts.
The router's cache entry for a function might become stale if the pod that had the function specialized gets deleted somehow. In such a case, we'd retry getting a new service for the function from executor and retry forwarding the user request to the newly created service.
Adds a validator, invoked through `fission spec validate` and before `fission spec apply`. The validator checks for duplicate names, dangling references, and unused resources.
Also attempts to fix a race between CLI and builder. Avoids replacing a package while it's building, instead waiting for the build status to change from pending before updating it.
The cli spec dir flag name should be `specdir`, not `specs`, that should match to:
```go
specDirFlag := cli.StringFlag{Name: "specdir", Usage: "Directory to store specs, defaults to ./specs"}
```
in fission/main.go
Currently it cause the specific specs dir not to work.
```
$ fission spec apply --specdir <my spec dir>
Everything up to date.
```
Tests for updates to a function of new deployment executor type. Tests check for changes in the environment, scale, secrets etc. Also checks conversion of function executor type from new deployment to pool manager and vice versa
The Deployment definition for executor, controller, router contains readines/liveness probes. The port definition for those probes should be numbers, instead of string, i.e. port: "8888" should be port 8888.
Environment Specs so far had only an image URL to specify a container image.
This was fine for public images but fell short in a few of cases:
(a) Using private image registries
(b) Specifying environment variables (this is needed for workflows helm install)
(c) Setting a SecurityContext for the container
This change adds the Container object to both build and runtime Environments.
Compatibility is preserved -- the existing ImageURL field is still used. See the comments in types.go for the overriding rules in the case that both Container and ImageURL are specified.
Newdeploy manager now checks the existence of function service cache by function UID before trying to create a new deployment. And return the cached fsvc directly if the cache exists.
Remove the hard coded default "fission" namespace that was added in the last PR, replacing it with some logic to figure out where fission is installed and connecting to it if there's exactly one installation. If there is more than one fission installation, error out with a useful message.
Also improve the error message for a missing kubeconfig file.
This makes set up easier for new users. Users can still set
FISSION_NAMESPACE but it will default to "fission". Users can also
still set KUBECONFIG, but it will default to $HOME/.kube/config.
Also, update the post-install chart notes.txt and the install guide to
use "fission function test" as the first step after install. This
means that if there's anything wrong with the setup, the user will see
useful errors instead of "internal server error". Also, they can test
their setup without worrying about nodeports or ingresses or whatever.
All existing functionality of FISSION_URL and FISSION_ROUTER continues
to work.
This the very first step for fission to integrate with Istio, which is an open platform to connect, manage, and secure microservices. With Istio, users are able to monitor functions usage and trace requests latency through dashboards. For more information, please visit http://fission.io/docs/
setting serviceType to ClusterIP as default for fission controller and corresponding changes in fission cli to be able to port-forward the controller pod.
The fetcher needs a relatively smaller set of resources and does not have to be same as the function container/defaults. This change adds defaults for fetcher containers in function pods.
* prevent leaking packages; add new clis to list and delete orphan packages.
* Fixes after testing.
* Address review comment.
* Removing deleting orphan packages from fnUpdate and fnDelete.
* goFmt and removing unwanted code.
These commits implement support for consuming messages from an Azure storage queue to trigger Fission functions.
* Add stubbed Azure message queue implementation and modify Helm charts.
This commit stubs an implementation for an Azure storage message queue trigger
that will be completed by future commits.
It also modifies the Helm chart to add support for deploying Fission with an
mqtrigger configured for Azure storage queue triggers.
* Add Azure Go SDK to glide.
This commit adds the Azure Go SDK to glide for the upcoming work to support
Azure storage queue triggers.
* Implement Azure message queue trigger.
This commit implements a message queue trigger based on Azure storage queues.
Required message queue trigger manager environment variables:
* AZURE_STORAGE_ACCOUNT_NAME - the Azure storage account to use.
* AZURE_STORAGE_ACCOUNT_KEY - the Azure storage account key.
When creating a message queue trigger, the topic will be the Azure storage
queue to receive messages from.
* Add CA certificates to fission-bundle.
This commit adds the root CA certificates to the fission-bundle image. This
allows Fission to contact third-party APIs that use HTTPS with root CA
signed certificates.
* Add Makefile to build and test.
This commit adds a simple Makefile for building the client and bundle, running
tests, creating the Docker image, and pushing the Docker image.
This change enables users to have declarative specifications for Fission resources. Users can specify their "app" in a set of spec files, and use a new fission CLI command to "apply" these specs to a running cluster.
The new "spec" CLI also includes archiving of local source files, and a file watcher that re-builds archives and uploads them on file changes, and a package build watcher that waits for package builds on the CLI.
A "--spec" option is also added to "function create", and will be added to other resources in future changes. This option causes a YAML to be outputted to the specs directory instead of the resource being created on the cluster.
The CLI "fission spec --help" outputs usage information.
This commit solves part of the issue #52 , functions are able to access secrets/configmaps specified by the user. For now, CLI only accept one secret/configmap. For advanced users, it will be able to use YAML to declare multiple secrets/configmaps in later changes.
A newdeploy backend which uses new deployment to serve requests. This is the second phase of #193 and builds on top of changes in #384 .
* Executor layer added on top of pool manager
* Removed the external server for executor
* Minor changes to keep existing semantics as much possible
* Separating the executor vs. poolmgr backend functionality and associated data members
* Executor logic separated from Poolmgr backend completely, placeholder for new backend
* Changed references to poolmgr in tests
* Moved poolmgr to it's package, as a side effect moved Cache to its's package (was causing cyclical dependency) and had to make some data structures exposed outside package
* Rebased from master and changed references to tpr -> crd
* Executor layer added on top of pool manager
* Executor logic separated from Poolmgr backend completely, placeholder for new backend
* Changed podName to a generic objectReference in fscache (#391)
Changed podName to a generic objectReference in function service cache implementation.
* Moved poolmgr to it's package, as a side effect moved Cache to its's package (was causing cyclical dependency) and had to make some data structures exposed outside package
* Rebased from master and changed references to tpr -> crd
* Merged from master with latest changes
* Executor layer added on top of pool manager
* Removed the external server for executor
* Minor changes to keep existing semantics as much possible
* Separating the executor vs. poolmgr backend functionality and associated data members
* Executor logic separated from Poolmgr backend completely, placeholder for new backend
* Changed references to poolmgr in tests
* update compiling.md to use helm
* Compile instructions: changed pullPolicy to IfNotPresent (#378)
Containers will get stuck in ErrImagePull/ImagePullBackOff state otherwise
* Moved poolmgr to it's package, as a side effect moved Cache to its's package (was causing cyclical dependency) and had to make some data structures exposed outside package
* Fetcher called when pod is created for newDeploy backend but also supports older way, this is WIP and still needs pod specialization and creating & exposing a service so the URL can be hit by end user
* WIP Specializing the POD as part of startup along with fetching
* Working specialization of a new deployment. Needs some work on caching, cleanup etc.
* Switched to service based address instead of POD address
* Minor formating issue fixed
* Added logging to pods and a readiness check, the readiness check is flaky though ATM
* Fixed some rebase issues that were failing build
* Better names for K8S objects and methods
* Switched usage of FuncSvc in backends from pod to api.ObjectReference
* Adding retry to fetcher request, for now just using default retry client which might need tweaking in future
* Switching to plain old retry, some issue in getting retryablehttp with glide import
* Removed stale executor service & deployment from previous merge
* Addressed review comments, still testing some areas
* Added types in FunctionSpec
* Resolved conflicts due to merge from executor_abstraction branch
* Added backend type on EnvironmentSpec along with operations for create/list/update, the pools are created/destroyed based on change in backend type
* Backend from types and a minor err return issue fixed
* Draft version of CPU and memory parameters added to environment
* Added resourceReq to newDeploy, though it has some issues
* Issue with resourceName fixed, now newdeploy pods also pick up resources from the environment config
* Adding scale params, removing validation on CPU params for now
* Fixed a formatting issue
* Checking if slight more delay helps in the test which is currently failing for internal routes
* The resourceList newly added in Env can not be compared by compiler, hence must use breakdown comparison instead
* Added strategy selection on client side
* Added caching, informers, delete operations for newdeploy backend functions
* Deleted a stale directory
* A simple HPA based on scale parameters, testing still WIP
* Fixed a small issue in delete function, added HPA delete too when deleting a function
* Previous merge missed the pkg flag for update fn command somehow, fixed that
* Fixed comments from review
* Changed poolmgr cleanup to be generic cleanup and moved to executor, added instanceID labels to newdeploy so that cleanup works
* Moved instanceIdLabel to types to avoid cyclic dependency
* More review fixes
* Tweaking sleep to see results
* If user does not provide poolsize, then it should not default to zero
* Switched to naming convention for now, fixed default poolsize if not provided
* Changed error return behaviour in delete fn, also changed cleanup to look based on obj type though support for additional type will need more work
* Changed check location so avoid false logging
* Test for newdeploy backend
* Adding tests for poolmgr backend
* Fixed an issue with glide dependency version, already fixed in master
* Added instanceId for NewDeploy, Initial cleanup now cleans older objects of newdeploy backend, removed eagercreate flag and instead using minScale to drive eager creation
* Moved cleanup to executor layer with cleanup for newDeploy backend, changes to use the new Cache impl
* Cleaning up pod & rs along with deployment for newdeploy backend
* Enhanced fn and env listing to show min/maxscale and resuorces respectively
* Added conditional heapster deployment and fixed a small issue with resources for fetcher container in function pod
* Addressed review comments from previous change
* Addressed some more review comments - majorly create only on NotFoundError
* Added TargetCPU as an input for scaling
* Bumped target CPU to be greater than 0 and added a default value
* Min replicas should be 1 even if the minScale is 0 when creating deployment
* Changed name from 'backend' to executorType, added additional test for minscale 0 case, changed TargetCPU to TargetCPUPercent
All functions have a pkg reference. This can be a package with either source and a deploy archives, or, a deploy archive. Everytime a function is updated, a new package is created. With archive pruner, the archives that are pointed to by old pkg reference can be deleted from the storage.
* High level spec for package pruning.
* Skeleton for archive pruning
* Adding meat 1 to skeleton.
* Adding meat #2. Separated storage service into a httpHandler component and
Storage Layer component.
* Adding meat #3. getOrphanedArchives in pruner and getItems on
stowClient.
* Restructured archivePruner methods.
* Commiting the day's work. Ready for testing #1.
* Fixing compile errors.
* Test ready. added a few logs for debugging.
* Adding a filter for getItems in stowClient.
* After testing.
* Added a test for archivePruner.
* Adding helm value pruneInterval for testing.
* Modified test.
* Final test.
* Fixing interval from seconds to minutes.
* Small change.
* Changing debugs to info.
* Removing the WIP design
* Ran gofmt on all these files.
* Fixing prune_interval as string in ENV var.
* Addressing all comments, but one.
* changing getFile method in stowClient to stream it into a response.
* All comments incorporated.
* Introducing a new flag for running archivePruner.
1. This flag is disabled for archivePruner to run in unit test.
2. This flag is enabled for archivePruner to run in production.
3. Also disabling test_archive_pruner.sh in this PR. Follow up with
next PR to enable it.
* Addressing review comments.
* Changing the command to generate a file dynamically.
* Enabling arching_pruner_test
* giving execute permissions to test_archive_pruner.sh
* Making changes of positional parameters after recent commit.
Change test case permission and removing kubectlPortForward.
* Adding debug to see why test_utils.sh passed junk pruneInterval.
* shell needs special handling for positional parameters from 10.
By default, flask turns redirects into absolute URLs. This is
undesired behaviour on Kubernetes, because the Pod's own IP address is
not really useful to any client that's not in the cluster (and even
within the cluster most clients will want to use the address of a
Service, not a Pod).
* Add readiness probe
* Remove builder manager http api interface since we don’t use/need it
* Check environment builder status and block build requests until builder is ready
* Replace deprecated api extension interface
* Add healthy check to python env
Modifies the fsCache to support a list of arbitrary kubernetes objects instead of just one pod. This allows better support for the newdeploy backend.
Also removes the byKubeObject cache index; cleanup is changed to use the byFunction index, and byKubeObject is no longer needed.
A few improvements to the integration test:
* Search for test files recursively in the test dir
* Output a test report at the end for a quick summary of what tests passed/failed/were skipped
* Add support skipping test files; skip the logging test since it's broken (#446 is tracking this)
* Switch to latest helm version
* Cleanup namespace
tested fission on openshift 3.7.0 and installed fine, without having to use openshift variables
those variables were used prior to switching to custom resource definitions, and as such are obsolete
Add package command support: This PR adds package command to CLI. package provides some useful subcommands to use such as packages CRUD and display package detail information. Also it is able to reuse existing package at function creation.
* Check function existence before creating package
* Check package existence
* Add support for downloading archive from given url
The functionality was supported before e238776bf7. Add this functionality back for more flexible usage.
* Add output flag to save archive content in specific file
* Add logs when package create/update
* Fix fnCreate requires environment argument when a pkg is specified
* Fix fnUpdate failed to update function pkg info when a package is specified
* Add package command test
* Fix wrong python test image in test script
* Remove package description fields
* Fix package command not update package but creating a new one
* Set package as pending status only when there is no deploy archive
* Rename function from fetchArchiveFromArbitraryURL to downloadToTempFile
* Use io.Copy to prevent loading all body into memory
* Revise some messages
* Allow user to update package build command
* Allow user to update package content when using function update
* Retrieve pkgName from function packageref if it’s not specified
* Fix function failed to update due to resource conflict
* Fix test case failure due to single quote
* kick ci
* Fix failed to update function packageRef
* kick ci
* kick ci
Before this change, poolmgr tried to hit fetcher just once. However if the pod has just started, fetcher may not be up yet, causing the initial request to fail with a connection refused error. This change adds a similar retry loop that we currently use on the specialize endpoint of the runtime environment container.
Create a container image with the buidler, so we can avoid copying the builder binary into the env container during the env container build.
Gather both stdout and stderr of a build into the logs.
* Add Dockerfile for builder + update release scripts
* Improve logging of buildermgr and builder
* Add support for environment-scoped buildcmd
* Fix fork/exec wd when package is a file
* Update tests to also include builder image
* Add logging to builder's fork/exec
This change adds a layer of abstraction over poolmgr. Poolmgr is now just one of the ways to turn a function into a service; other implementations will be added. The executor abstraction is a uniform API over all these implementations.
* Executor layer added on top of pool manager
* Removed the external server for executor
* Minor changes to keep existing semantics as much possible
* Separating the executor vs. poolmgr backend functionality and associated data members
* Executor logic separated from Poolmgr backend completely, placeholder for new backend
* Changed references to poolmgr in tests
* Moved poolmgr to it's package, as a side effect moved Cache to its's package (was causing cyclical dependency) and had to make some data structures exposed outside package
* Rebased from master and changed references to tpr -> crd
* Executor layer added on top of pool manager
* Executor logic separated from Poolmgr backend completely, placeholder for new backend
* Changed podName to a generic objectReference in fscache (#391)
Changed podName to a generic objectReference in function service cache implementation.
* Moved poolmgr to it's package, as a side effect moved Cache to its's package (was causing cyclical dependency) and had to make some data structures exposed outside package
* Rebased from master and changed references to tpr -> crd
* Merged from master with latest changes
* Removed stale executor service & deployment from previous merge
* Addressed review comments, still testing some areas
Remove the `logger` container from the logging daemonset.
Remove the outgoing call from the poolmgr to the logger. Use Fluentd's Kubernetes filter to add function name and UID to influx metadata.
This means fluentd now figures out when to start collecting function logs on its own, without being informed by poolmgr. This is great for other execution strategies, and for autoscaling, where fission isn't in direct control of function pod creation.
Also adds an integration test to make sure logging keeps working.
This fixes a bug where functionReferenceResolver returned out-of-date function metadata and caused the router to proxy requests to old function pods.
It also uses the go context package to shutdown the controller when the router is shutting down.
This contains a few different builder manager fixes:
* Update the zip file structure to avoid an extra subdirectory
* Annotate packages with what functions are using them
* Only trigger builds when there is no deployment archive
* Python environment loadpath bugfixes
* Other bugfixes
Switch Fission's storage over to the new CustomResourceDefinitions, from the deprecated ThirdPartyResources. This allows us to be compatible with Kubernets 1.8 and onwards.
This also adds a CLI tool for dumping state from an old fission version and restoring state into new CRDs.
The storage service is unaffected by this change.
closes#319
* Make default node-env use alpine, and support separate debian env. List envs in documentation.
* Copy ONBUILD base image commands into node Dockerfiles
* Copy more commands from onbuild image
* Tidy dockerfiles
Add a flag to the environment to control multiple specialization -- the max number of functions per container. This can be set to 1 or infinity.
Add an api proxy to workflow apiserver from the controller.
Add function metadata to FunctionLoadRequest; every v2 environment now knows which function it's loading (but can ignore that information if it wants to).
Add function identity headers to router. This is useful for multiple specialization, so the router can disambiguate between different function calls. (If this turns out to be a non-trivial perf overhead, we could add these headers conditionally, but for now they are always added.)
This change orchestrates function builds.
Environments (in v2) define a builder image, just like they do a runtime image. The builder image contains a build script that's invoked with source and deployment paths (as env vars).
The buildermgr watches for environments with build images defined, and creates build deployments and services.
Functions can define source and deployment. Buildermgr watches for functions with source code (and build status == pending) and invokes the environment's builder when appropriate. It captures logs from the build and sets the build status (success/failure) and build lots into the PackageStatus.
Since `mz` has already bumped for several times without
breaking change, it's better to follow the updates.
```console
$ npm outdated
Package Current Wanted Latest Location
mz 2.1.0 2.1.0 2.7.0 fission-nodejs-runtime
```
Fixes: https://github.com/fission/fission/issues/167
This adds partial support for multiple specialization to the
function service cache. It allows Add() to succeed if the pod and
address maps already contain entries.
But it doesn't deal with DeletePod and TouchByAddress. That's ok for
now, because the workflow engine never deletes the environment pod
(that pod is the workflow engine).
However, if/when we want full support for multiple specialization,
that will require updating both DeletePod and TouchByAddress.
Allows dumping v0.1 state to a json file and restoring it into a new v0.2.1 fission installation.
Usage guide is at: /Documentation/docs-site/content/upgrade-from-v0.1.md
Introduce _Archives_ as a type to reference arbitrary blobs.
_Packages_ are a pair of Source and Deployment archives. Packages have an environment reference.
Functions reference a package. Multiple functions can reference the same package.
Watch functions in the router and use that to trigger functionReferenceResolver cache invalidation.
We might want to rate-limit syncTriggers in a future change, since we might be triggering it more often than needed.
## Web UI (tracked separately in the fission-ui repo)
## Performance and Scalability ([area-perf](https://github.com/fission/fission/labels/area-perf))
- Autoscaling
- Cold-start optimization -- optimistically choose from pool, save about ~20msec
- Cold-start optimization -- preload funcs in fetcher
- Cold-start optimization -- preload libraries in envs (v2) -- mem vs. speed tradeoff
## Function extensibility ([area-ext](https://github.com/fission/fission/labels/area-ext))
- Env v2: easy addition of dependencies etc.
- Integration with Service Broker
## Multi-area stuff
- Execution strategies: cold-start pool vs create-pod-on-cold-start -- one size doesn't fit all, at least with current tech; abstract over execution strategies according to requirements
Save the external IP addresses of controller and router services in
FISSION_URL and FISSION_ROUTER, respectively. Wait for services to
get IP addresses (check this with ```kubectl --namespace fission get
svc```). Then:
```
$ export FISSION_URL=http://$(kubectl --namespace fission get svc controller -o=jsonpath='{..ip}')
$ export FISSION_ROUTER=$(kubectl --namespace fission get svc router -o=jsonpath='{..ip}')
```
### Get and Run Fission: OpenShift
If you're using OpenShift, it's possible to run Fission on it! The
deployment template needs to be deployed as a user with cluster-admin
permissions (like `system:admin`), as it needs to create a
`ClusterRole` for deploying function containers from the `fission`
namespace/project.
Identically as with Kubernetes, you need to set the FISSION_URL and FISSION_ROUTER environment variables. If you're using minishift, use these commands:
Fission is in early alpha. It's not suitable for production use just
yet.
The following companies, organizations, and individuals support Fission's ongoing maintenance and development. If you are using/contributing to Fission, we would be happy to list you here, please raise a Pull request.
_See [helm upgrade](https://helm.sh/docs/helm/helm_upgrade/) for command documentation._
### Upgrading an existing Release to a new major version
A major chart version change (like v1.2.3 -> v2.0.0) indicates that there is an incompatible breaking change needing manual actions.
### Upgrade from 1.18.x to 1.20.x
We have removed controller service from fission-all chart.
### Upgrade from 1.17.x to 1.18.x
With 1.18.x, we have major change in the way we are deploying Fission.
We have added parameters `defaultNamespace`, `additionalFissionNamespaces`, `functionNamespace` and `builderNamespace` to manage the namespaces.
We watch and manage specific namespaces for Fission resources configured via `defaultNamespace` and `additionalFissionNamespaces` parameters.
You dont need to worry about `builderNamespace` and `functionNamespace` parameters, unless you want to consider legacy Fission resources.
Please refer to [core changes](https://fission.io/docs/releases/v1.18.0/#fission-core-changes) for more details.
### Upgrade from 1.16.x to 1.17.x
By default, Fission runs with the default security context. This means that it will be run as root. We have added settings in Helm chart for securityContext across all services in Fission. You can enable recommended securityContext settings during Fission installation.
Please refer to [security context settings](https://fission.io/docs/releases/v1.17.0/#security-context-setting-for-fission-installation) for more details.
### Upgrade from 1.15.x to 1.16.x
If you have been using `prometheus.enabled=true` in your fission-all chart, you will need to deploy the prometheus using prometheus community supported chart.
We have removed prometheus dependency from fission-all chart.
We would recommend [prometheus-community/prometheus](https://artifacthub.io/packages/helm/prometheus-community/prometheus) or [prometheus-community/kube-prometheus-stack](https://artifacthub.io/packages/helm/prometheus-community/kube-prometheus-stack) chart.
### Upgrade from 1.14.x to 1.15.x
With 1.15.x release, following changes are made:
-`fission-core` chart is removed
-`fission-all` chart is made similar `fission-core` chart
- In the `fission-all` chart, the following components are disabled which were enabled by default earlier. If you want to enable them, please use `--set` flag.
- nats - Set `nats.enabled=true` to enable Fission Nats integration
- influxdb - Set `influxdb.enabled=true` to enable Fission InfluxDB and logger component
- prometheus - Set `prometheus.enabled=true` to install Prometheus with Fission
- canaryDeployment - Set `canaryDeployment.enabled=true` to enable Canary Deployment
## Migrating from fission-core chart
With the release of Fission v1.15.x, the fission-core chart was removed.
Fission-all is now exactly similar to fission-core and can be used to migrate from fission-core.
If you are upgrading from the fission-core chart, you can use the following command to migrate with required changes.
See [Customizing the Chart Before Installing](https://helm.sh/docs/intro/using_helm/#customizing-the-chart-before-installing). To see all configurable options with detailed comments:
```console
helm show values fission-charts/fission-all
```
You may also `helm show values` on this chart's [dependencies](#dependencies) for additional options.
### Multiple releases
The same chart can be used to run multiple Fission instances in the same cluster if required.
For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/nightly20170705/fission-cli-windows.exe
For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/{{ .Chart.Version }}/fission-{{ .Chart.Version }}-windows-amd64.exe
2. Set the FISSION_URL and FISSION_ROUTER environment variables.
FISSION_URL is used by the fission CLI to find the server.
FISSION_URL should be prefixed with a http://. (FISSION_ROUTER is only needed for the examples below to work.)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
