Compare commits

...
38 Commits
Author SHA1 Message Date
Sanket SudakeandGitHub 50b5c74c52 Prerelease v1.15.0-rc1 (#2204)
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-09-23 14:19:33 +05:30
Sanket SudakeandGitHub 23d6266335 chore(changelog): Cleanup changelog and exclude non-standard releases (#2203)
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-09-23 12:44:50 +05:30
Sanket SudakeandGitHub 4f8727f9d4 chore: Logo in README compatible with dark themes (#2202) 2021-09-22 16:51:25 +05:30
Sanket SudakeandGitHub 119d674207 enhancement: Add different samplers and propagators support with OpenTelemetry (#2201)
* Samplers: Check PR/helm values for supported types
* Propagators: Check PR/helm values for supported types
* Added tracing support via fission CLI
* Use parentbased_traceidratio as default sampler with 0.1 ratio

Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-09-22 16:29:49 +05:30
Sanket SudakeandGitHub 1347d73b04 fix: Goreleaser for build and release (#2189)
* Add goreleaser for building binaries
* Update CONTRIBUTING.md
* Break release scripts for goreleaser

Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-09-21 15:29:40 +05:30
Ankit ChawlaandGitHub b55dc1fd78 Updated nats version (#2186) 2021-09-21 11:47:30 +05:30
Sanket SudakeandGitHub 1a52f3dce8 chore(dep): update go dependency stow v0.2.7 (#2195)
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-09-17 10:55:56 +05:30
Sanket SudakeandGitHub 4a0bd1aa21 fix: Use standard env variables for opentelemtry (#2194)
Opentelemetry defines standard variables which are supported by multiple observability platforms.

[1] https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/sdk-environment-variables.md
[2] https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/exporter.md

By defining support for standard env vars, Fission open telemetry support can be leveraged with different
platforms such as NewRelic, SigNoz, DataDog etc.

Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-09-16 17:48:26 +05:30
Sanket SudakeandGitHub 33473a4528 feature: Capture important events with span in fission and add trace id in logs (#2180)
* Capture important open telemetry events with span in fission
* Add context to missing HTTP calls
* Add Trace ID in logs
* capture trace id in the proxy handler function
* Always registry tracer to get traceID

Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-09-15 16:41:59 +05:30
Sanket SudakeandGitHub 453debb6e9 ci(Mergify): configuration update (#2191)
Signed-off-by:  Sanket Sudake <sanketsudake@gmail.com>
2021-09-12 10:21:56 +05:30
Sanket SudakeandGitHub 81ccf6359d Add first interaction greetings (#2190)
* Add first interaction greetings

* Update greetings.yml
2021-09-12 10:11:29 +05:30
Ankit ChawlaandGitHub 681e6f02fe Updated versions of connectors (#2185) 2021-09-06 15:43:27 +05:30
Sanket SudakeandGitHub 5e226433b5 Check package resource version from fetch request with fetched package (#2183)
When fetch request is received by fetcher, it fetches
package information. Checking if package resource version mentioned
in fetch req matches with one received. Adding warning for now,
in future we need to ensure we give error if resource version
are not matching.

Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-09-06 13:12:33 +05:30
Sanket SudakeandGitHub f4c56f81f3 Pass podinformer to poolpodcontroller instead of reference to pool manager (#2184)
In upgrade tests, sometimes in race condition pool manager podlister
takes time to sync in which case, poolpodcontrolller get nil reference
for gpm.podLister. Passing podInformer to poolpodcontroller so that
we wait for podInformer cache to sync.

Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-09-06 11:57:33 +05:30
Ankit ChawlaandGitHub ce85b27803 Added redis keda connector (#2176) 2021-09-06 10:01:35 +05:30
Sanket SudakeandGitHub eb31381094 Functions with common prefix path handling with internal routes (#2182)
With addition of prefix based routes in 1.13.1, we have turned
all internal routes prefix based. This is causing wrong responses
in functions names with common prefixes such as `hello-world` and
`hello-world-consumer`. This patch fixes it where we add a fix route
for function name and prefix path for function name with `/`.
We improve overall prefix path handing with fix and prevent breakage
for existing flow.

Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-09-06 08:35:38 +05:30
Sanket SudakeandGitHub a1cbce810e Add kubernetes objects aware logger with zap (#2179)
Using controller-runtime zap integration which is aware of Kubernetesobjects and logs only name+namespace of the object when complete runtime.Object compatible type is given to logger.

Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-08-31 18:38:14 +05:30
nickmancariandGitHub f195975bda Added lables to flags '--url', '--prefix', '--method' to reflect their deprecation for the 'fn create' use case. (#2171) 2021-08-31 18:12:58 +05:30
Sanket SudakeandGitHub 3636bb35ff Fix pod name in port forwarding log (#2178)
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-08-31 10:07:42 +05:30
Sanket SudakeandGitHub 1d5a09699b Use pool pod controller with env informer (#2161)
Signed-off-by: Sanket Sudake sanketsudake@gmail.com

- Use informers and listers in executors
- Passing context properly in pool manager executor
- Environment updates in the pool manager would not cause updates in the deployment
- Environment update minimizing downtime
- Use replicaset controller and environment delete triggers to cleanup specialized pods
2021-08-31 09:31:32 +05:30
Sanket SudakeandGitHub 1df59316e7 Add correct context required in executor (#2175)
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-08-25 13:16:26 +05:30
Sanket SudakeandGitHub d6b47c1a4e Add typed informers instead of generic SharedIndexInformers (#2174)
Generally using typed informers is more standard practise
than using SharedIndexInformer(SII) implicity. SII also lack
listers provided by informer factory and few other high level
abstractions.

Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-08-23 17:41:55 +05:30
Sanket SudakeandGitHub 49fe2cb61f Kind profile with opentelemetry local setup (#2173)
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-08-20 15:04:42 +05:30
Gaurav GahlotandGitHub 5f17b4f3c5 Add context for traces in executor (#2172)
Signed-off-by: Gaurav Gahlot <gauravgahlot0107@gmail.com>
2021-08-20 14:53:06 +05:30
Sanket SudakeandGitHub 1b9d21b5e3 Change default informer resync period to 30 minutes (#2167)
Currently most of informers used in Fission are using resync
period of 30sec which is quite low and causes lot of disturbances
in overall system. Since informers already have watchers which would
be delivering events to the handlers. If any handlers require lower
sync period they should register handler with `AddEventHandlersWithResyncPeriod`
instead of we generaling informer.

Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-08-20 09:03:09 +05:30
c51c6b7f7e fix: Add pathType to create networking/v1 ingress (#2164)
* fix: Add pathType to create networking/v1 ingress

Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>

Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
2021-08-20 08:49:55 +05:30
Sanket SudakeandGitHub 0c8d7c26c5 Use latest env in spec-archive test (#2170)
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-08-19 20:48:37 +05:30
Sanket SudakeandGitHub c4585c8394 Update go version to 1.16 and kubernetes dev dependencies to 1.21 (#2168)
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-08-19 20:30:27 +05:30
Gaurav GahlotandGitHub dfcf961f75 Change default function timeout to 60 seconds and pass context to executor from router (#2169)
Patch changes default function timeout to 60 seconds. Also, we are propagating context from router to executor in a proper way.
2021-08-19 19:13:33 +05:30
0cc3ecc2e9 Fission meets OpenTelemetry (#2157)
* add opentracing section and otelCollectorEndpoint
* initialize OTLP exporter
* pkg/controller: changes for context propagation
* pkg/executor: changes for context propagation
* pkg/fetcher: changes for context propagation
* pkg/router: changes for context propagation
* pkg/storagesvc: changes for context propagation
* set no default value for otel collector endpoint
* update readme and add notes to charts
* move common code to pkg/utils/otel
* adding fn and env as attributes
* don't use otelhttp transport for websocket
* URL ignore with common filter UrlsToIgnore

Note: The web socket example does not work when using OTEL HTTP. Here is an issue related to that on open-telemetry/opentelemetry-js-contrib.

Signed-off-by: Gaurav Gahlot <gauravgahlot0107@gmail.com>
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
2021-08-19 13:00:30 +05:30
Sanket SudakeandGitHub a24934f1a0 Organize pool manager code and few improvements (#2166)
The patch adds few improvements in pool manager and adds better
function composability by reorganizing code.
1. Added created status in get pool call
2. Improved logging in certain areas and having logger per component
3. Separated deployment-specific code in diff file for extensibility

Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-08-18 13:21:54 +05:30
Sanket SudakeandGitHub 9d54f5daac Consider newdeploy functions only for env update in newdeploy manager (#2165)
Currently when we update env, even if we dont have newdeploy related
functions env is getting is getting triggered. Added filter in finding
related functions to avoid this behaviour.

Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-08-17 21:55:38 +05:30
Sanket SudakeandGitHub 0f8b5e51a4 Reduce logs in image pull and system prune for tests (#2163)
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-08-16 12:17:20 +05:30
Sanket SudakeandGitHub 47295309b3 Update alpine base image to 3.14 (#2160)
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-08-12 10:55:49 +05:30
Sanket SudakeandGitHub ba7ee3ed95 Update changelog for 1.14.1 (#2145)
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-08-03 17:33:13 +05:30
Sanket SudakeandGitHub 9b5eb069d6 Update chart version to 1.14.1 (#2143)
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-08-03 15:16:15 +05:30
Sanket SudakeandGitHub 672bdbba6f Fixed service name generation for function name < 17 chars (#2142)
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-08-03 14:19:04 +05:30
Sanket SudakeandGitHub 8271dfae07 Update changelog for 1.14.0 (#2138)
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
2021-08-02 21:35:28 +05:30
112 changed files with 4897 additions and 3007 deletions
+23
View File
@@ -0,0 +1,23 @@
name: Greetings
on: [pull_request, issues]
jobs:
greeting:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/first-interaction@v1
if: env.month != 'Oct'
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
issue-message: 'Thank you for your first issue! ✨😊'
pr-message: 'Thank you for contributing to this project! ✨😊'
- uses: actions/first-interaction@v1
if: env.month == 'Oct'
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
issue-message: 'Thank you for your first issue! Happy Hacktoberfest!!! ✨🎃👕✨'
pr-message: 'Thank you for contributing to this project. Happy Hacktoberfest!!! ✨🎃👕'
+3 -3
View File
@@ -22,10 +22,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.15
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1.15
go-version: 1.16
- name: Check out code
uses: actions/checkout@v2
@@ -42,7 +42,7 @@ jobs:
go mod verify
go mod download
LINT_VERSION=1.41.1
LINT_VERSION=1.42.0
curl -fsSL https://github.com/golangci/golangci-lint/releases/download/v${LINT_VERSION}/golangci-lint-${LINT_VERSION}-linux-amd64.tar.gz | \
tar xz --strip-components 1 --wildcards \*/golangci-lint
mkdir -p bin && mv golangci-lint bin/
+11 -5
View File
@@ -28,13 +28,13 @@ jobs:
strategy:
fail-fast: false
matrix:
kindimage : [ 'kindest/node:v1.19.11','kindest/node:v1.20.7', 'kindest/node:v1.21.2' ]
kindimage : [ 'kindest/node:v1.19.11','kindest/node:v1.20.7', 'kindest/node:v1.21.1' ]
os: [ ubuntu-latest ]
steps:
- name: setup go
uses: actions/setup-go@v2.1.3
uses: actions/setup-go@v2
with:
go-version: "1.15.12"
go-version: 1.16
- name: Checkout sources
uses: actions/checkout@v2.3.4
@@ -83,14 +83,20 @@ jobs:
sudo install skaffold /usr/local/bin/
skaffold version
- name: Install GoReleaser
uses: goreleaser/goreleaser-action@v2
with:
install-only: true
- name: Build and Install Fission
run: |
kubectl create ns fission
kubectl create -k crds/v1
skaffold run -p kind-ci
make create-crds
SKAFFOLD_PROFILE=kind-ci make skaffold-deploy
- name: Build and Install Fission CLI
run: |
make build-fission-cli
sudo make install-fission-cli
sudo chmod +x /usr/local/bin/fission
+7 -2
View File
@@ -31,9 +31,9 @@ jobs:
os: [ ubuntu-latest ]
steps:
- name: Setup go
uses: actions/setup-go@v2.1.3
uses: actions/setup-go@v2
with:
go-version: '1.15.12'
go-version: 1.16
- name: Checkout action sources
uses: actions/checkout@v2.3.4
@@ -56,6 +56,11 @@ jobs:
image: ${{ matrix.kindimage }}
version: v0.11.1
- name: Install GoReleaser
uses: goreleaser/goreleaser-action@v2
with:
install-only: true
- name: Setup kubectl & fetch node information
run: |
kubectl cluster-info --context kind-kind
+2
View File
@@ -31,3 +31,5 @@ vendor/
local/
build/
dist/
manifest/
+245
View File
@@ -0,0 +1,245 @@
project_name: fission
release:
github:
owner: fission
name: fission
prerelease: true
draft: true
header: |
Release Highlights: https://docs.fission.io/docs/releases/{{ .Tag }}/
Install Guide: https://docs.fission.io/installation/
Full Changelog: https://github.com/fission/fission/blob/master/CHANGELOG.md
extra_files:
- glob: ./manifest/charts/*
- glob: ./manifest/yamls/*
before:
hooks:
- go mod tidy
snapshot:
name_template: "{{ .Tag }}"
builds:
- &build-linux
id: builder
ldflags:
- -s -w
- -X github.com/fission/fission/pkg/info.GitCommit={{.ShortCommit}}
- -X github.com/fission/fission/pkg/info.BuildDate={{.Date}}
- -X github.com/fission/fission/pkg/info.Version={{.Tag}}
gcflags:
- all=-trimpath={{.Env.PWD}}
asmflags:
- all=-trimpath={{.Env.PWD}}
env:
- CGO_ENABLED=0
goos:
- linux
goarch:
- amd64
- arm64
- arm
goarm:
- 7
binary: builder
dir: ./cmd/builder
- <<: *build-linux
id: fetcher
binary: fetcher
dir: ./cmd/fetcher
- <<: *build-linux
id: fission-bundle
binary: fission-bundle
dir: ./cmd/fission-bundle
- <<: *build-linux
id: fission-cli
goos:
- linux
- windows
- darwin
binary: fission
dir: ./cmd/fission-cli
ignore:
- goos: windows
goarch: arm64
- goos: darwin
goarch: arm
goarm: 7
- goos: windows
goarch: arm
goarm: 7
- <<: *build-linux
id: pre-upgrade-checks
binary: pre-upgrade-checks
dir: ./cmd/preupgradechecks
- <<: *build-linux
id: reporter
binary: reporter
dir: ./cmd/reporter
dockers:
- &docker-amd64
use: buildx
goos: linux
goarch: amd64
ids:
- builder
image_templates:
- "fission/builder:latest-amd64"
- "fission/builder:{{ .Tag }}-amd64"
dockerfile: cmd/builder/Dockerfile.fission-builder
build_flag_templates:
- "--platform=linux/amd64"
- "--label=org.opencontainers.image.created={{.Date}}"
- "--label=org.opencontainers.image.revision={{.FullCommit}}"
- "--label=org.opencontainers.image.version={{.Tag}}"
- <<: *docker-amd64
ids:
- fetcher
image_templates:
- "fission/fetcher:latest-amd64"
- "fission/fetcher:{{ .Tag }}-amd64"
dockerfile: cmd/fetcher/Dockerfile.fission-fetcher
- <<: *docker-amd64
ids:
- fission-bundle
image_templates:
- "fission/fission-bundle:latest-amd64"
- "fission/fission-bundle:{{ .Tag }}-amd64"
dockerfile: cmd/fission-bundle/Dockerfile.fission-bundle
- <<: *docker-amd64
ids:
- pre-upgrade-checks
image_templates:
- "fission/pre-upgrade-checks:latest-amd64"
- "fission/pre-upgrade-checks:{{ .Tag }}-amd64"
dockerfile: cmd/preupgradechecks/Dockerfile.fission-preupgradechecks
- <<: *docker-amd64
ids:
- reporter
image_templates:
- "fission/reporter:latest-amd64"
- "fission/reporter:{{ .Tag }}-amd64"
dockerfile: cmd/reporter/Dockerfile.reporter
- &docker-arm64
use: buildx
goos: linux
goarch: arm64
ids:
- builder
image_templates:
- "fission/builder:latest-arm64"
- "fission/builder:{{ .Tag }}-arm64"
dockerfile: cmd/builder/Dockerfile.fission-builder
build_flag_templates:
- "--platform=linux/arm64"
- "--label=org.opencontainers.image.created={{.Date}}"
- "--label=org.opencontainers.image.revision={{.FullCommit}}"
- "--label=org.opencontainers.image.version={{.Tag}}"
- <<: *docker-arm64
ids:
- fetcher
image_templates:
- "fission/fetcher:latest-arm64"
- "fission/fetcher:{{ .Tag }}-arm64"
dockerfile: cmd/fetcher/Dockerfile.fission-fetcher
- <<: *docker-arm64
ids:
- fission-bundle
image_templates:
- "fission/fission-bundle:latest-arm64"
- "fission/fission-bundle:{{ .Tag }}-arm64"
dockerfile: cmd/fission-bundle/Dockerfile.fission-bundle
- <<: *docker-arm64
ids:
- pre-upgrade-checks
image_templates:
- "fission/pre-upgrade-checks:latest-arm64"
- "fission/pre-upgrade-checks:{{ .Tag }}-arm64"
dockerfile: cmd/preupgradechecks/Dockerfile.fission-preupgradechecks
- <<: *docker-arm64
ids:
- reporter
image_templates:
- "fission/reporter:latest-arm64"
- "fission/reporter:{{ .Tag }}-arm64"
dockerfile: cmd/reporter/Dockerfile.reporter
- &docker-armv7
use: buildx
goos: linux
goarch: arm
goarm: 7
ids:
- builder
image_templates:
- "fission/builder:latest-armv7"
- "fission/builder:{{ .Tag }}-armv7"
dockerfile: cmd/builder/Dockerfile.fission-builder
build_flag_templates:
- "--platform=linux/arm/v7"
- "--label=org.opencontainers.image.created={{.Date}}"
- "--label=org.opencontainers.image.revision={{.FullCommit}}"
- "--label=org.opencontainers.image.version={{.Tag}}"
- <<: *docker-armv7
ids:
- fetcher
image_templates:
- "fission/fetcher:latest-armv7"
- "fission/fetcher:{{ .Tag }}-armv7"
dockerfile: cmd/fetcher/Dockerfile.fission-fetcher
- <<: *docker-armv7
ids:
- fission-bundle
image_templates:
- "fission/fission-bundle:latest-armv7"
- "fission/fission-bundle:{{ .Tag }}-armv7"
dockerfile: cmd/fission-bundle/Dockerfile.fission-bundle
- <<: *docker-armv7
ids:
- pre-upgrade-checks
image_templates:
- "fission/pre-upgrade-checks:latest-armv7"
- "fission/pre-upgrade-checks:{{ .Tag }}-armv7"
dockerfile: cmd/preupgradechecks/Dockerfile.fission-preupgradechecks
- <<: *docker-armv7
ids:
- reporter
image_templates:
- "fission/reporter:latest-armv7"
- "fission/reporter:{{ .Tag }}-armv7"
dockerfile: cmd/reporter/Dockerfile.reporter
docker_manifests:
- name_template: fission/builder:{{ .Tag }}
image_templates:
- fission/builder:{{ .Tag }}-amd64
- fission/builder:{{ .Tag }}-arm64
- fission/builder:{{ .Tag }}-armv7
- name_template: fission/fetcher:{{ .Tag }}
image_templates:
- fission/fetcher:{{ .Tag }}-amd64
- fission/fetcher:{{ .Tag }}-arm64
- fission/fetcher:{{ .Tag }}-armv7
- name_template: fission/fission-bundle:{{ .Tag }}
image_templates:
- fission/fission-bundle:{{ .Tag }}-amd64
- fission/fission-bundle:{{ .Tag }}-arm64
- fission/fission-bundle:{{ .Tag }}-armv7
- name_template: fission/pre-upgrade-checks:{{ .Tag }}
image_templates:
- fission/pre-upgrade-checks:{{ .Tag }}-amd64
- fission/pre-upgrade-checks:{{ .Tag }}-arm64
- fission/pre-upgrade-checks:{{ .Tag }}-armv7
- name_template: fission/reporter:{{ .Tag }}
image_templates:
- fission/reporter:{{ .Tag }}-amd64
- fission/reporter:{{ .Tag }}-arm64
- fission/reporter:{{ .Tag }}-armv7
changelog:
skip: true
archives:
- id: fission
builds:
- fission-cli
name_template: "{{ .ProjectName }}-{{ .Tag }}-{{ .Os }}-{{ .Arch }}"
format: binary
checksum:
name_template: "checksums.txt"
algorithm: sha256
+13
View File
@@ -0,0 +1,13 @@
pull_request_rules:
- name: Automatic merge on approval
conditions:
- base=master
- "#approved-reviews-by>=1"
- label=ready-to-merge
- label!=hold-off-merging
- status-success=build
actions:
merge:
method: squash
commit_message: title+body
strict: smart
+332 -428
View File
File diff suppressed because it is too large Load Diff
+2 -231
View File
@@ -5,235 +5,6 @@ There are many areas we can use contributions - ranging from code, documentation
First, please read the [code of conduct](CODE_OF_CONDUCT.md). By participating, you're expected to uphold this code.
Table of Contents
=================
Please refer contributing docs for detailed guide.
* [Choose something to work on](#choose-something-to-work-on)
* [Get Help.](#get-help)
* [Contributing - building &amp; deploying](#contributing---building--deploying)
* [Prerequisite](#prerequisite)
* [Getting Started](#getting-started)
* [Use Skaffold with Kind/K8S Cluster to build and deploy](#use-skaffold-with-kindk8s-cluster-to-build-and-deploy)
* [Validating Installation](#validating-installation)
* [Understanding code structure](#understanding-code-structure)
* [cmd](#cmd)
* [pkg](#pkg)
* [Charts](#charts)
* [Environments](#environments)
# Choose something to work on
* The easiest way to start is to look at existing [issues](https://github.com/fission/fission/issues) and see if there's something there that you'd like to work on. You can filter issues with label "[Good first issue](https://github.com/fission/fission/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)" which are relatively self sufficient issues and great for first time contributors.
- If you are going to pick up an issue, it would be good to add a comment stating the intention.
- If the contribution is a big change/new feature, please raise an issue and discuss the needs, design in the issue in detail.
* For contributing a new Fission environment, please check the [environments repo](https://github.com/fission/environments)
* For contributing a new Keda Connector, please check the [Keda Connectors repo](https://github.com/fission/keda-connectors)
### Get Help.
Do reach out on Slack or Twitter and we are happy to help.
* Drop by the [slack channel](http://slack.fission.io).
* Say hi on [twitter](https://twitter.com/fissionio).
# Contributing - building & deploying
## Prerequisite
- You'll need the `go` compiler and tools installed. Currently version 1.12.x of Go is needed.
- You'll also need [docker](https://docs.docker.com/install) for building images locally.
- You will need a Kubernetes cluster and you can use one of options from below.
- [Minikube](https://github.com/kubernetes/minikube)
- [Kind](https://kind.sigs.k8s.io/)
- Cluster in cloud such as GKE (Google Kubernetes Engine cluster)/ EKS (Elastic Kubernetes Service)/ AKS (Azure Kubernetes Service)
- Kubectl and Helm installed.
- [Skaffold](https://skaffold.dev/docs/install/) for local development workflow to make it easier to build and deploy Fission.
- And of course some basic concepts of Fission such as environment, function are good to be aware of!
## Getting Started
Get the code locally and after you have made changes - you can verify formatting and other basic checks.
```sh
# Clone the repo
$ git clone https://github.com/fission/fission.git $GOPATH/src/github.com/fission/fission
$ cd $GOPATH/src/github.com/fission/fission
$ go mod vendor
# Run checks on your changes
$ ./hack/verify-gofmt.sh
$ ./hack/verify-govet.sh
```
### Use Skaffold with Kind/K8S Cluster to build and deploy
You should bring up Kind/Minikube cluster or if using a cloud provider cluster then Kubecontext should be pointing to appropriate cluster.
* For building & deploying to Cloud Provider K8S cluster such as GKE/EKS/AKS:
```
$ skaffold config set default-repo vishalbiyani // (vishalbiyani - should be your registry/Docker Hub handle)
$ skaffold run
```
* For building & deploying to Kind cluster use Kind profile
```
$ kind create cluster
$ kubectl create ns fission
$ skaffold run -p kind
```
## Validating Installation
If you are using Helm, you should see release installed:
```
helm list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
fission fission 1 2020-05-19 16:31:46.947562 +0530 IST success fission-all-1.11.0 1.11.0
```
Also you should see the Fission services deployed and running:
```
$ kubectl get pods -n fission
NAME READY STATUS RESTARTS AGE
buildermgr-6f778d4ff9-dqnq5 1/1 Running 0 6h9m
controller-d44bd4f4d-5q4z5 1/1 Running 0 6h9m
executor-557c68c6fd-dg8ld 1/1 Running 0 6h9m
influxdb-845548c959-2954p 1/1 Running 0 6h9m
kubewatcher-5784c454b8-5mqsk 1/1 Running 0 6h9m
logger-bncqn 2/2 Running 0 6h9m
mqtrigger-kafka-765b674ff-jk5x9 1/1 Running 0 6h9m
mqtrigger-nats-streaming-797498966c-xgxmk 1/1 Running 3 6h9m
nats-streaming-6bf48bccb6-fmmr9 1/1 Running 0 6h9m
router-db76576bd-xxh7r 1/1 Running 0 6h9m
storagesvc-799dcb5bdf-f69k9 1/1 Running 0 6h9m
timer-7d85d9c9fb-knctw 1/1 Running 0 6h9m
```
## Understanding code structure
### cmd
`cmd` package is entry point for all runtime components and also has Dockerfile for each component. The actual logic here will be pretty light and most of logic of each component is in `pkg` (Discussed later)
| Component | Runtime Component |Used in|
| :------------- |:------------- |:-|
| fetcher | Docker Image |Environments|
| fission-bundle | Docker Image |Binary for all components|
| fission-cli | CLI Binary |CLI by user|
| preupgradechecks | Docker Image |Pre-install upgrade|
```
.
cmd
├── fetcher
│   ├── Dockerfile.fission-fetcher
│   ├── app
│   └── main.go
├── fission-bundle
│   ├── Dockerfile.fission-bundle
│   ├── main.go
│   └── mqtrigger
├── fission-cli
│   ├── app
│   ├── fission-cli
│   └── main.go
└── preupgradechecks
├── Dockerfile.fission-preupgradechecks
├── main.go
└── preupgradechecks.go
```
**fetcher** : is a very lightweight component and all of related logic is in fetcher package itself. Fetcher helps in fetching and uploading code and in specializing environments.
**fission-bundle** : is a component which is a single binary for all components. Based on arguments you pass to fission-bundle - it becomes that component. For ex.
```
/fission-bundle --controllerPort "8888" # Runs Controller
/fission-bundle --kubewatcher --routerUrl http://router.fission # Runs Kubewatcher
```
So most server side components running on server side are fission-bundle binary wrapped in container and used with different arguments. Various arguments and environment variables are passed from manifests/helm chart
**fission-cli** : is the cli used by end user to interact Fission
**preupgradechecks** : is again a small independent component to do pre-install upgrade tasks.
### pkg
Pkg is where most of core components and logic reside. The structure is fairly self-explanatory for example all of executor related functionality will be in executor package and so on.
```
.
├── pkg
│   ├── apis
│   ├── builder
│   ├── buildermgr
│   ├── cache
│   ├── canaryconfigmgr
│   ├── controller
│   ├── crd
│   ├── error
│   ├── executor
│   ├── fetcher
│   ├── fission-cli
│   ├── generator
│   ├── info
│   ├── kubewatcher
│   ├── logger
│   ├── mqtrigger
│   ├── plugin
│   ├── publisher
│   ├── router
│   ├── storagesvc
│   ├── throttler
│   ├── timer
│   └── utils
```
### Charts
Fission currently has two charts - and we recommend using fission-all for development.
```
.
├── charts
│   ├── README.md
│   ├── fission-all
│   └── fission-core
```
### Environments
Each of runtime environments is in fission/environments repo and fairly independent. If you are enhancing or creating a new environment - most likely you will end up making changes in that repo.
```
.
├── environments
│   ├── binary
│   ├── dotnet
│   ├── dotnet20
│   ├── go
│   ├── jvm
│   ├── nodejs
│   ├── perl
│   ├── php7
│   ├── python
│   ├── ruby
│   └── tensorflow-serving
```
[https://docs.fission.io/docs/contributing/](https://docs.fission.io/docs/contributing/)
+25 -76
View File
@@ -14,33 +14,23 @@
.DEFAULT_GOAL := check
# Platforms to build in multi-architecture images.
PLATFORMS ?= linux/amd64,linux/arm64,linux/arm/v7
SKAFFOLD_PROFILE ?= kind
# Repository prefix and tag to push multi-architecture images to.
REPO ?= fission
TAG ?= dev
DOCKER_FLAGS ?= --push --progress plain
VERSION ?= master
VERSION ?= v0.0.0
TIMESTAMP ?= $(shell date -u +'%Y-%m-%dT%H:%M:%SZ')
COMMITSHA ?= $(shell git rev-parse HEAD)
GOOS ?= $(shell go env GOOS)
GOARCH ?= $(shell go env GOARCH)
BINDIR ?= build/bin
FISSION-CLI-SUFFIX :=
ifeq ($(GOOS), windows)
FISSION-CLI-SUFFIX := .exe
endif
GO ?= go
GO_LDFLAGS := -X github.com/fission/fission/pkg/info.GitCommit=$(COMMITSHA) $(GO_LDFLAGS)
GO_LDFLAGS := -X github.com/fission/fission/pkg/info.BuildDate=$(TIMESTAMP) $(GO_LDFLAGS)
GO_LDFLAGS := -X github.com/fission/fission/pkg/info.Version=$(VERSION) $(GO_LDFLAGS)
GCFLAGS ?= all=-trimpath=$(CURDIR)
ASMFLAGS ?= all=-trimpath=$(CURDIR)
# Show this help.
help:
@awk '/^#/{c=substr($$0,3);next}c&&/^[[:alpha:]][[:alnum:]_-]+:/{print substr($$1,1,index($$1,":")),c}1{c=0}' $(MAKEFILE_LIST) | column -s: -t
### Static checks
check: test-run build-fission-cli clean
@@ -56,61 +46,11 @@ test-run: code-checks
@rm -f coverage.txt
### Binaries
%-cli:
@mkdir -p $(BINDIR)
GOOS=$(GOOS) GOARCH=$(GOARCH) $(GO) build \
-gcflags '$(GCFLAGS)' \
-asmflags '$(ASMFLAGS)' \
-ldflags "$(GO_LDFLAGS)" \
-o $(BINDIR)/$(subst -cli,,$@)-$(VERSION)-$(GOOS)-$(GOARCH)$(FISSION-CLI-SUFFIX) ./$<
build-fission-cli:
@GOOS=$(GOOS) GOARCH=$(GOARCH) GORELEASER_CURRENT_TAG=$(VERSION) goreleaser build --snapshot --rm-dist --single-target --id fission-cli
fission-cli: cmd/fission-cli
fission-bundle-cli: cmd/fission-bundle
fetcher-cli: cmd/fetcher
builder-cli: cmd/builder
preupgradechecks-cli: cmd/preupgradechecks
reporter-cli: cmd/reporter
local-bins: fission-cli fission-bundle-cli fetcher-cli builder-cli preupgradechecks-cli reporter-cli
all-fission-cli:
$(MAKE) fission-cli GOOS=windows GOARCH=amd64
$(MAKE) fission-cli GOOS=linux GOARCH=amd64
$(MAKE) fission-cli GOOS=linux GOARCH=arm
$(MAKE) fission-cli GOOS=linux GOARCH=arm64
$(MAKE) fission-cli GOOS=darwin GOARCH=amd64
install-fission-cli: fission-cli
mv $(BINDIR)/fission-$(VERSION)-$(GOOS)-$(GOARCH)$(FISSION-CLI-SUFFIX) /usr/local/bin/fission
### Container images
FISSION_IMGS := fission-bundle-multiarch-img \
fetcher-multiarch-img \
builder-multiarch-img\
pre-upgrade-checks-multiarch-img \
reporter-multiarch-img
verify-builder:
@./hack/buildx.sh $(PLATFORMS)
local-images:
PLATFORMS=linux/amd64 $(MAKE) all-images
all-images: verify-builder $(FISSION_IMGS)
fission-bundle-multiarch-img: cmd/fission-bundle/Dockerfile.fission-bundle
fetcher-multiarch-img: cmd/fetcher/Dockerfile.fission-fetcher
builder-multiarch-img: cmd/builder/Dockerfile.fission-builder
pre-upgrade-checks-multiarch-img: cmd/preupgradechecks/Dockerfile.fission-preupgradechecks
reporter-multiarch-img: cmd/reporter/Dockerfile.reporter
%-multiarch-img:
@echo === Building image $(REPO)/$(subst -multiarch-img,,$@):$(TAG) using context $(CURDIR) and dockerfile $<
docker buildx build --platform=$(PLATFORMS) -t $(REPO)/$(subst -multiarch-img,,$@):$(TAG) \
--build-arg GITCOMMIT=$(COMMITSHA) \
--build-arg BUILDDATE=$(TIMESTAMP) \
--build-arg BUILDVERSION=$(VERSION) \
$(DOCKER_FLAGS) -f $< .
install-fission-cli:
mv dist/fission-cli_$(GOOS)_$(GOARCH)/fission$(FISSION-CLI-SUFFIX) /usr/local/bin/fission
### Codegen
codegen:
@@ -133,12 +73,7 @@ delete-crds:
### Cleanup
clean:
@rm -f cmd/fission-bundle/fission-bundle
@rm -f cmd/fission-cli/fission
@rm -f cmd/fetcher/fetcher
@rm -f cmd/fetcher/builder
@rm -f cmd/reporter/reporter
@rm -f pkg/apis/core/v1/types_swagger_doc_generated.go
@rm -f dist/
### Misc
generate-swagger-doc:
@@ -146,6 +81,20 @@ generate-swagger-doc:
all-generators: codegen generate-crds generate-swagger-doc
skaffold-prebuild:
@GOOS=linux GOARCH=amd64 GORELEASER_CURRENT_TAG=$(VERSION) goreleaser build --snapshot --rm-dist --single-target
@cp -v cmd/builder/Dockerfile.fission-builder dist/builder_linux_amd64/Dockerfile
@cp -v cmd/fetcher/Dockerfile.fission-fetcher dist/fetcher_linux_amd64/Dockerfile
@cp -v cmd/fission-bundle/Dockerfile.fission-bundle dist/fission-bundle_linux_amd64/Dockerfile
@cp -v cmd/reporter/Dockerfile.reporter dist/reporter_linux_amd64/Dockerfile
@cp -v cmd/preupgradechecks/Dockerfile.fission-preupgradechecks dist/pre-upgrade-checks_linux_amd64/Dockerfile
skaffold-deploy: skaffold-prebuild
skaffold run -p $(SKAFFOLD_PROFILE)
### Release
release:
@./hack/generate-helm-manifest.sh $(VERSION)
@./hack/release.sh $(VERSION)
@./hack/release-tag.sh $(VERSION)
@./hack/release-tag.sh $(VERSION)
@./hack/changelog.sh
+1 -1
View File
@@ -1,5 +1,5 @@
<p align="center">
<img src="https://docs.fission.io/images/logo.png" width="300" />
<img src="https://docs.fission.io/images/logo-gh.svg" width="300" />
<br>
<h1 align="center">Fission: Serverless Functions for Kubernetes</h1>
</p>
+4
View File
@@ -79,6 +79,10 @@ Parameter | Description | Default
`router.roundTrip.timeout` | HTTP transport request timeout | `50ms`
`router.roundTrip.timeoutExponent` | The length of request timeout will multiply with timeoutExponent after each retry | `2`
`router.roundTrip.maxRetries` | Max retries times of a failed request | `10`
`openTracing.enabled` | If true, OpenTracing is enabled | `false`
`openTracing.collectorEndpoint` | Jaeger collector endpoint | ``
`openTracing.samplingRate` | Probabilistic sampling rate | `0.5`
`otelCollectorEndpoint` | OpenTelemetry collector endpoint | None
### Extra configuration for `fission-all`
+2 -2
View File
@@ -1,6 +1,6 @@
apiVersion: v2
name: fission-all
version: 1.14.0
version: v1.15.0-rc1
description: Fission is a fast serverless framework for Kubernetes.
keywords:
- fission
@@ -12,7 +12,7 @@ maintainers:
- name: Sanket Sudake
email: sanket@infracloud.io
engine: gotpl
appVersion: 1.14.0
appVersion: v1.15.0-rc1
type: application
dependencies:
- name: prometheus
+26
View File
@@ -51,3 +51,29 @@ This template generates the image name for the deployment depending on the value
{{- end }}
{{- end }}
{{- end -}}
{{- define "opentelemtry.envs" }}
- name: OTEL_EXPORTER_OTLP_ENDPOINT
value: "{{ .Values.openTelemetry.otlpCollectorEndpoint }}"
- name: OTEL_EXPORTER_OTLP_INSECURE
value: "{{ .Values.openTelemetry.otlpInsecure }}"
{{- if .Values.openTelemetry.otlpHeaders }}
- name: OTEL_EXPORTER_OTLP_HEADERS
value: "{{ .Values.openTelemetry.otlpHeaders }}"
{{- end }}
- name: OTEL_TRACES_SAMPLER
value: "{{ .Values.openTelemetry.tracesSampler }}"
- name: OTEL_TRACES_SAMPLER_ARG
value: "{{ .Values.openTelemetry.tracesSamplingRate }}"
- name: OTEL_PROPAGATORS
value: "{{ .Values.openTelemetry.propagators }}"
{{- end }}
{{- define "opentracing.envs" }}
- name: OPENTRACING_ENABLED
value: {{ .Values.openTracing.enabled | default false | quote }}
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.openTracing.collectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.openTracing.samplingRate | default "0.5" | quote }}
{{- end }}
+23 -36
View File
@@ -85,6 +85,7 @@ rules:
resources:
- deployments
- deployments/scale
- replicasets
verbs:
- '*'
- apiGroups:
@@ -334,10 +335,6 @@ spec:
env:
- name: FISSION_FUNCTION_NAMESPACE
value: "{{ .Values.functionNamespace }}"
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.traceCollectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.traceSamplingRate | default "0.5" | quote }}
- name: DEBUG_ENV
value: {{ .Values.debugEnv | quote }}
- name: PPROF_ENABLED
@@ -346,6 +343,8 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
readinessProbe:
httpGet:
path: "/healthz"
@@ -424,10 +423,6 @@ spec:
value: {{ .Values.executor.podReadyTimeout | default false | quote }}
- name: ENABLE_ISTIO
value: "{{ .Values.enableIstio }}"
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.traceCollectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.traceSamplingRate | default "0.5" | quote }}
- name: FETCHER_MINCPU
value: {{ .Values.fetcher.resource.cpu.requests | quote }}
- name: FETCHER_MINMEM
@@ -440,6 +435,8 @@ spec:
value: {{ .Values.debugEnv | quote }}
- name: PPROF_ENABLED
value: {{ .Values.pprof.enabled | quote }}
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
readinessProbe:
httpGet:
path: "/healthz"
@@ -504,10 +501,6 @@ spec:
value: "{{ .Values.pullPolicy }}"
- name: ENABLE_ISTIO
value: "{{ .Values.enableIstio }}"
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.traceCollectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.traceSamplingRate | default "0.5" | quote }}
- name: FETCHER_MINCPU
value: {{ .Values.fetcher.resource.cpu.requests | quote }}
- name: FETCHER_MINMEM
@@ -520,6 +513,8 @@ spec:
value: {{ .Values.debugEnv | quote }}
- name: PPROF_ENABLED
value: {{ .Values.pprof.enabled | quote }}
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
serviceAccountName: fission-svc
{{- if .Values.extraCoreComponentPodConfig }}
{{ toYaml .Values.extraCoreComponentPodConfig | indent 6 -}}
@@ -550,14 +545,12 @@ spec:
command: ["/fission-bundle"]
args: ["--kubewatcher", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
env:
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.traceCollectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.traceSamplingRate | default "0.5" | quote }}
- name: DEBUG_ENV
value: {{ .Values.debugEnv | quote }}
- name: PPROF_ENABLED
value: {{ .Values.pprof.enabled | quote }}
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
serviceAccountName: fission-svc
{{- if .Values.extraCoreComponentPodConfig }}
{{ toYaml .Values.extraCoreComponentPodConfig | indent 6 -}}
@@ -694,6 +687,8 @@ spec:
value: {{ .Values.debugEnv | quote }}
- name: PPROF_ENABLED
value: {{ .Values.pprof.enabled | quote }}
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
serviceAccountName: fission-svc
{{- if .Values.extraCoreComponentPodConfig }}
{{ toYaml .Values.extraCoreComponentPodConfig | indent 6 -}}
@@ -827,14 +822,12 @@ spec:
{{- else }}
value: nats://{{ .Values.nats.hostaddress }}
{{- end }}
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.traceCollectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.traceSamplingRate | default "0.5" | quote }}
- name: DEBUG_ENV
value: {{ .Values.debugEnv | quote }}
- name: PPROF_ENABLED
value: {{ .Values.pprof.enabled | quote }}
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
serviceAccountName: fission-svc
{{- if .Values.extraCoreComponentPodConfig }}
{{ toYaml .Values.extraCoreComponentPodConfig | indent 6 -}}
@@ -880,14 +873,12 @@ spec:
value: "{{.Values.kafka.brokers}}"
- name: MESSAGE_QUEUE_KAFKA_VERSION
value: "{{.Values.kafka.version}}"
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.traceCollectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.traceSamplingRate | default "0.5" | quote }}
- name: DEBUG_ENV
value: {{ .Values.debugEnv | quote }}
- name: PPROF_ENABLED
value: {{ .Values.pprof.enabled | quote }}
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
# TLS authentication is TLS with authentication (2 way)
# More info: https://docs.confluent.io/current/kafka/authentication_ssl.html#ssl-overview
{{- if .Values.kafka.authentication.tls.enabled }}
@@ -972,10 +963,6 @@ spec:
command: ["/fission-bundle"]
args: ["--mqt", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
env:
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.traceCollectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.traceSamplingRate | default "0.5" | quote }}
- name: MESSAGE_QUEUE_TYPE
value: azure-storage-queue
- name: AZURE_STORAGE_ACCOUNT_NAME
@@ -989,6 +976,8 @@ spec:
value: {{ .Values.debugEnv | quote }}
- name: PPROF_ENABLED
value: {{ .Values.pprof.enabled | quote }}
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
serviceAccountName: fission-svc
{{- if .Values.extraCoreComponentPodConfig }}
{{ toYaml .Values.extraCoreComponentPodConfig | indent 6 -}}
@@ -1026,10 +1015,6 @@ spec:
args: ["--storageServicePort", "8000", "--storageType", "local"]
{{- end }}
env:
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.traceCollectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.traceSamplingRate | default "0.5" | quote }}
- name: PRUNE_INTERVAL
value: "{{.Values.pruneInterval}}"
- name: DEBUG_ENV
@@ -1050,6 +1035,8 @@ spec:
- name: STORAGE_S3_REGION
value: {{ .Values.persistence.s3.region }}
{{- end }}
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
{{- if ne (.Values.persistence.storageType | default "local") "s3" }}
volumeMounts:
- name: fission-storage
@@ -1119,10 +1106,6 @@ spec:
command: ["/fission-bundle"]
args: ["--mqt_keda", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
env:
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.traceCollectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.traceSamplingRate | default "0.5" | quote }}
- name: DEBUG_ENV
value: {{ .Values.debugEnv | quote }}
- name: CONNECTOR_IMAGE_PULL_POLICY
@@ -1139,6 +1122,10 @@ spec:
value: "{{ .Values.mqt_keda.connector_images.nats_steaming.image }}:{{ .Values.mqt_keda.connector_images.nats_steaming.tag }}"
- name: GCP-PUB-SUB_IMAGE
value: "{{ .Values.mqt_keda.connector_images.gcp_pub_sub.image }}:{{ .Values.mqt_keda.connector_images.gcp_pub_sub.tag }}"
- name: REDIS_IMAGE
value: "{{ .Values.mqt_keda.connector_images.redis.image }}:{{ .Values.mqt_keda.connector_images.redis.tag }}"
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
serviceAccountName: fission-svc
{{- if .Values.extraCoreComponentPodConfig }}
{{ toYaml .Values.extraCoreComponentPodConfig | indent 6 -}}
@@ -117,6 +117,8 @@ spec:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
- name: OPENTRACING_ENABLED
value: {{ .Values.openTracing.enabled | default false | quote }}
command: ["/fission-bundle"]
args: ["--logger"]
volumeMounts:
@@ -158,6 +160,8 @@ spec:
key: password
- name: LOG_PATH
value: /var/log/fission/*.log
- name: OPENTRACING_ENABLED
value: {{ .Values.openTracing.enabled | default false | quote }}
{{- if .Values.logger.enableSecurityContext }}
securityContext:
privileged: true
+30 -32
View File
@@ -35,38 +35,36 @@ spec:
command: ["/fission-bundle"]
args: ["--routerPort", "8888", "--executorUrl", "http://executor.{{ .Release.Namespace }}"]
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: ROUTER_ROUND_TRIP_TIMEOUT
value: {{ .Values.router.roundTrip.timeout | default "50ms" | quote }}
- name: ROUTER_ROUNDTRIP_TIMEOUT_EXPONENT
value: {{ .Values.router.roundTrip.timeoutExponent | default 2 | quote }}
- name: ROUTER_ROUND_TRIP_KEEP_ALIVE_TIME
value: {{ .Values.router.roundTrip.keepAliveTime | default "30s" | quote }}
- name: ROUTER_ROUND_TRIP_DISABLE_KEEP_ALIVE
value: {{ .Values.router.roundTrip.disableKeepAlive | default true | quote }}
- name: ROUTER_ROUND_TRIP_MAX_RETRIES
value: {{ .Values.router.roundTrip.maxRetries | default 10 | quote }}
- name: ROUTER_SVC_ADDRESS_MAX_RETRIES
value: {{ .Values.router.svcAddressMaxRetries | default 5 | quote }}
- name: ROUTER_SVC_ADDRESS_UPDATE_TIMEOUT
value: {{ .Values.router.svcAddressUpdateTimeout | default "30s" | quote }}
- name: ROUTER_UNTAP_SERVICE_TIMEOUT
value: {{ .Values.router.unTapServiceTimeout | default "3600s" | quote }}
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.traceCollectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.router.traceSamplingRate | default "0.5" | quote }}
- name: USE_ENCODED_PATH
value: {{ .Values.router.useEncodedPath | default false | quote }}
- name: DEBUG_ENV
value: {{ .Values.debugEnv | quote }}
- name: PPROF_ENABLED
value: {{ .Values.pprof.enabled | quote }}
- name: DISPLAY_ACCESS_LOG
value: {{ .Values.router.displayAccessLog | default false | quote }}
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: ROUTER_ROUND_TRIP_TIMEOUT
value: {{ .Values.router.roundTrip.timeout | default "50ms" | quote }}
- name: ROUTER_ROUNDTRIP_TIMEOUT_EXPONENT
value: {{ .Values.router.roundTrip.timeoutExponent | default 2 | quote }}
- name: ROUTER_ROUND_TRIP_KEEP_ALIVE_TIME
value: {{ .Values.router.roundTrip.keepAliveTime | default "30s" | quote }}
- name: ROUTER_ROUND_TRIP_DISABLE_KEEP_ALIVE
value: {{ .Values.router.roundTrip.disableKeepAlive | default true | quote }}
- name: ROUTER_ROUND_TRIP_MAX_RETRIES
value: {{ .Values.router.roundTrip.maxRetries | default 10 | quote }}
- name: ROUTER_SVC_ADDRESS_MAX_RETRIES
value: {{ .Values.router.svcAddressMaxRetries | default 5 | quote }}
- name: ROUTER_SVC_ADDRESS_UPDATE_TIMEOUT
value: {{ .Values.router.svcAddressUpdateTimeout | default "30s" | quote }}
- name: ROUTER_UNTAP_SERVICE_TIMEOUT
value: {{ .Values.router.unTapServiceTimeout | default "3600s" | quote }}
- name: USE_ENCODED_PATH
value: {{ .Values.router.useEncodedPath | default false | quote }}
- name: DEBUG_ENV
value: {{ .Values.debugEnv | quote }}
- name: PPROF_ENABLED
value: {{ .Values.pprof.enabled | quote }}
- name: DISPLAY_ACCESS_LOG
value: {{ .Values.router.displayAccessLog | default false | quote }}
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
resources:
{{- toYaml .Values.router.resources | indent 10 }}
readinessProbe:
+60 -13
View File
@@ -20,7 +20,7 @@ image: fission/fission-bundle
pullPolicy: IfNotPresent
## Fission image version
imageTag: 1.14.0
imageTag: v1.15.0-rc1
## Port at which Fission controller service should be exposed
controllerPort: 31313
@@ -50,7 +50,7 @@ fetcher:
## Fetcher repository
image: fission/fetcher
## Fetcher image version
imageTag: 1.14.0
imageTag: v1.15.0-rc1
## Fetcher is only for to downloading or uploading archive.
## Normally, you don't need to change the value here, unless necessary.
@@ -140,7 +140,7 @@ router:
## Sample with a rate per time window (traces/second)
traceSamplingRate: 0.5
## Extend the container specs for the core fission pods.
## Can be used to add things like affinty/tolerations/nodeSelectors/etc.
## For example:
@@ -219,7 +219,6 @@ kafka:
userCert: "" # path to certificate containing public key of the user signed by CA authority
userKey: "" # path to private key of the user
# brokers: 'my-broker.kafka:9092' # or my-bootstrap-server.kafka:9092/9093
# Sample config for authentication
# authentication:
@@ -340,9 +339,54 @@ prometheus:
canaryDeployment:
enabled: true
# Use these flags to enable opentracing, the variable is endpoint of Jaeger collector in the format shown below
#traceCollectorEndpoint: "http://jaeger-collector.jaeger.svc:14268/api/traces?format=jaeger.thrift"
#traceSamplingRate: 0.75
# Use the following flags to enable OpenTracing.
# Note: OpenTracing support will be removed in an upcoming release.
# Please prefer using OpenTelemetry instead.
openTracing:
## set this flag to true if you wish to enable OpenTracing
enabled: false
## if enabled is true, the variable is endpoint of Jaeger collector in the format shown below
#collectorEndpoint: "http://jaeger-collector.jaeger.svc:14268/api/traces?format=jaeger.thrift"
## uniformly sample traces with the given probabilistic sampling rate
#samplingRate: 0.75
# It is an alternate to OpenTracing.
openTelemetry:
# Use this flag to set the collector endpoint for OpenTelemetry.
# The variable is endpoint of the collector in the format shown below.
# otlpCollectorEndpoint: "otel-collector.observability.svc:4317"
otlpCollectorEndpoint: ""
# Set this flag to false if you are using secure endpoint for the collector.
otlpInsecure: true
# Key-value pairs to be used as headers associated with gRPC or HTTP requests
# to the collector.
# Eg. otlpHeaders: "key1=value1,key2=value2"
otlpHeaders: ""
# Supported samplers:
# always_on - Sampler that always samples spans, regardless of the parent span's sampling decision.
# always_off - Sampler that never samples spans, regardless of the parent span's sampling decision.
# traceidratio - Sampler that samples probabalistically based on rate.
# parentbased_always_on - (default if empty) Sampler that respects its parent span's sampling decision, but otherwise always samples.
# parentbased_always_off - Sampler that respects its parent span's sampling decision, but otherwise never samples.
# parentbased_traceidratio - Sampler that respects its parent span's sampling decision, but otherwise samples probabalistically based on rate.
tracesSampler: "parentbased_traceidratio"
# Each Sampler type defines its own expected input, if any.
# Currently we get trace ratio for the case of,
# 1. traceidratio
# 2. parentbased_traceidratio
# Sampling probability, a number in the [0..1] range, e.g. "0.1". Default is 0.1.
tracesSamplingRate: "0.1"
# Supported providers:
# tracecontext - W3C Trace Context
# baggage - W3C Baggage
# b3 - B3 Single
# b3multi - B3 Multi
# jaeger - Jaeger uber-trace-id header
# xray - AWS X-Ray (third party)
# ottrace - OpenTracing Trace (third party)
propagators: "tracecontext,baggage"
## Message Queue Trigger Kind, KEDA: enable and configuration
mqt_keda:
@@ -350,22 +394,25 @@ mqt_keda:
connector_images:
kafka:
image: fission/keda-kafka-http-connector
tag: v0.7
tag: v0.8
rabbitmq:
image: fission/keda-rabbitmq-http-connector
tag: v0.7
tag: v0.8
awskinesis:
image: fission/keda-aws-kinesis-http-connector
tag: v0.7
tag: v0.8
aws_sqs:
image: fission/keda-aws-sqs-http-connector
tag: v0.7
tag: v0.8
nats_steaming:
image: fission/keda-nats-streaming-http-connector
tag: v0.7
tag: v0.9
gcp_pub_sub:
image: fission/keda-gcp-pubsub-http-connector
tag: v0.2
tag: v0.3
redis:
image: fission/keda-redis-http-connector
tag: v0.1
## Enable Pprof based profiling
pprof:
+2 -2
View File
@@ -1,6 +1,6 @@
apiVersion: v2
name: fission-core
version: 1.14.0
version: v1.15.0-rc1
description: Fission is a fast serverless framework for Kubernetes.
keywords:
- fission
@@ -12,7 +12,7 @@ maintainers:
- name: Sanket Sudake
email: sanket@infracloud.io
engine: gotpl
appVersion: 1.14.0
appVersion: v1.15.0-rc1
type: application
dependencies:
- name: prometheus
@@ -42,3 +42,29 @@ This template generates the image name for the deployment depending on the value
{{ .Values.image }}:{{ .Values.imageTag }}
{{- end }}
{{- end -}}
{{- define "opentelemtry.envs" }}
- name: OTEL_EXPORTER_OTLP_ENDPOINT
value: "{{ .Values.openTelemetry.otlpCollectorEndpoint }}"
- name: OTEL_EXPORTER_OTLP_INSECURE
value: "{{ .Values.openTelemetry.otlpInsecure }}"
{{- if .Values.openTelemetry.otlpHeaders }}
- name: OTEL_EXPORTER_OTLP_HEADERS
value: "{{ .Values.openTelemetry.otlpHeaders }}"
{{- end }}
- name: OTEL_TRACES_SAMPLER
value: "{{ .Values.openTelemetry.tracesSampler }}"
- name: OTEL_TRACES_SAMPLER_ARG
value: "{{ .Values.openTelemetry.tracesSamplingRate }}"
- name: OTEL_PROPAGATORS
value: "{{ .Values.openTelemetry.propagators }}"
{{- end }}
{{- define "opentracing.envs" }}
- name: OPENTRACING_ENABLED
value: {{ .Values.openTracing.enabled | default false | quote }}
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.openTracing.collectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.openTracing.samplingRate | default "0.5" | quote }}
{{- end }}
+23 -35
View File
@@ -140,16 +140,14 @@ spec:
command: ["/fission-bundle"]
args: ["--controllerPort", "8888"]
env:
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.traceCollectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.traceSamplingRate | default "0.5" | quote }}
- name: FISSION_FUNCTION_NAMESPACE
value: "{{ .Values.functionNamespace }}"
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: FISSION_FUNCTION_NAMESPACE
value: "{{ .Values.functionNamespace }}"
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
readinessProbe:
httpGet:
path: "/healthz"
@@ -215,10 +213,6 @@ spec:
value: "{{ .Values.pullPolicy }}"
- name: FETCHER_IMAGE_PULL_POLICY
value: "{{ .Values.pullPolicy }}"
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.traceCollectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.traceSamplingRate | default "0.5" | quote }}
- name: ADOPT_EXISTING_RESOURCES
value: {{ .Values.executor.adoptExistingResources | default false | quote }}
- name: POD_READY_TIMEOUT
@@ -233,6 +227,8 @@ spec:
value: {{ .Values.fetcher.resource.cpu.limits | quote }}
- name: FETCHER_MAXMEM
value: {{ .Values.fetcher.resource.mem.limits | quote }}
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
readinessProbe:
httpGet:
path: "/healthz"
@@ -286,11 +282,7 @@ spec:
- name: FETCHER_IMAGE_PULL_POLICY
value: "{{ .Values.pullPolicy }}"
- name: BUILDER_IMAGE_PULL_POLICY
value: "{{ .Values.pullPolicy }}"
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.traceCollectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.traceSamplingRate | default "0.5" | quote }}
value: "{{ .Values.pullPolicy }}"
- name: ENABLE_ISTIO
value: "{{ .Values.enableIstio }}"
- name: FETCHER_MINCPU
@@ -301,6 +293,8 @@ spec:
value: {{ .Values.fetcher.resource.cpu.limits | quote }}
- name: FETCHER_MAXMEM
value: {{ .Values.fetcher.resource.mem.limits | quote }}
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
serviceAccountName: fission-svc
{{- if .Values.extraCoreComponentPodConfig }}
{{ toYaml .Values.extraCoreComponentPodConfig | indent 6 -}}
@@ -331,10 +325,8 @@ spec:
command: ["/fission-bundle"]
args: ["--kubewatcher", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
env:
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.traceCollectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.traceSamplingRate | default "0.5" | quote }}
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
serviceAccountName: fission-svc
{{- if .Values.extraCoreComponentPodConfig }}
{{ toYaml .Values.extraCoreComponentPodConfig | indent 6 -}}
@@ -365,10 +357,8 @@ spec:
command: ["/fission-bundle"]
args: ["--timer", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
env:
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.traceCollectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.traceSamplingRate | default "0.5" | quote }}
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
serviceAccountName: fission-svc
{{- if .Values.extraCoreComponentPodConfig }}
{{ toYaml .Values.extraCoreComponentPodConfig | indent 6 -}}
@@ -408,10 +398,6 @@ spec:
env:
- name: PRUNE_INTERVAL
value: "{{.Values.pruneInterval}}"
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.traceCollectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.traceSamplingRate | default "0.5" | quote }}
{{- if and (.Values.persistence.enabled) (eq (.Values.persistence.storageType | default "local") "s3") }}
- name: STORAGE_S3_ENDPOINT
value: {{ .Values.persistence.s3.endPoint }}
@@ -426,6 +412,8 @@ spec:
- name: STORAGE_S3_REGION
value: {{ .Values.persistence.s3.region }}
{{- end }}
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
{{- if ne (.Values.persistence.storageType | default "local") "s3" }}
volumeMounts:
- name: fission-storage
@@ -478,10 +466,6 @@ spec:
command: ["/fission-bundle"]
args: ["--mqt_keda", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
env:
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.traceCollectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.traceSamplingRate | default "0.5" | quote }}
- name: DEBUG_ENV
value: {{ .Values.debugEnv | quote }}
- name: CONNECTOR_IMAGE_PULL_POLICY
@@ -498,6 +482,10 @@ spec:
value: "{{ .Values.mqt_keda.connector_images.nats_steaming.image }}:{{ .Values.mqt_keda.connector_images.nats_steaming.tag }}"
- name: GCP-PUB-SUB_IMAGE
value: "{{ .Values.mqt_keda.connector_images.gcp_pub_sub.image }}:{{ .Values.mqt_keda.connector_images.gcp_pub_sub.tag }}"
- name: REDIS_IMAGE
value: "{{ .Values.mqt_keda.connector_images.redis.image }}:{{ .Values.mqt_keda.connector_images.redis.tag }}"
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
serviceAccountName: fission-svc
{{- if .Values.extraCoreComponentPodConfig }}
{{ toYaml .Values.extraCoreComponentPodConfig | indent 6 -}}
+28 -30
View File
@@ -35,36 +35,34 @@ spec:
command: ["/fission-bundle"]
args: ["--routerPort", "8888", "--executorUrl", "http://executor.{{ .Release.Namespace }}"]
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: ROUTER_ROUND_TRIP_TIMEOUT
value: {{ .Values.router.roundTrip.timeout | default "50ms" | quote }}
- name: ROUTER_ROUNDTRIP_TIMEOUT_EXPONENT
value: {{ .Values.router.roundTrip.timeoutExponent | default 2 | quote }}
- name: ROUTER_ROUND_TRIP_KEEP_ALIVE_TIME
value: {{ .Values.router.roundTrip.keepAliveTime | default "30s" | quote }}
- name: ROUTER_ROUND_TRIP_DISABLE_KEEP_ALIVE
value: {{ .Values.router.roundTrip.disableKeepAlive | default true | quote }}
- name: ROUTER_ROUND_TRIP_MAX_RETRIES
value: {{ .Values.router.roundTrip.maxRetries | default 10 | quote }}
- name: ROUTER_SVC_ADDRESS_MAX_RETRIES
value: {{ .Values.router.svcAddressMaxRetries | default 5 | quote }}
- name: ROUTER_SVC_ADDRESS_UPDATE_TIMEOUT
value: {{ .Values.router.svcAddressUpdateTimeout | default "30s" | quote }}
- name: ROUTER_UNTAP_SERVICE_TIMEOUT
value: {{ .Values.router.unTapServiceTimeout | default "3600s" | quote }}
- name: TRACE_JAEGER_COLLECTOR_ENDPOINT
value: "{{ .Values.traceCollectorEndpoint }}"
- name: TRACING_SAMPLING_RATE
value: {{ .Values.router.traceSamplingRate | default "0.5" | quote }}
- name: USE_ENCODED_PATH
value: {{ .Values.router.useEncodedPath | default false | quote }}
- name: DEBUG_ENV
value: {{ .Values.debugEnv | quote }}
- name: DISPLAY_ACCESS_LOG
value: {{ .Values.router.displayAccessLog | default false | quote }}
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: ROUTER_ROUND_TRIP_TIMEOUT
value: {{ .Values.router.roundTrip.timeout | default "50ms" | quote }}
- name: ROUTER_ROUNDTRIP_TIMEOUT_EXPONENT
value: {{ .Values.router.roundTrip.timeoutExponent | default 2 | quote }}
- name: ROUTER_ROUND_TRIP_KEEP_ALIVE_TIME
value: {{ .Values.router.roundTrip.keepAliveTime | default "30s" | quote }}
- name: ROUTER_ROUND_TRIP_DISABLE_KEEP_ALIVE
value: {{ .Values.router.roundTrip.disableKeepAlive | default true | quote }}
- name: ROUTER_ROUND_TRIP_MAX_RETRIES
value: {{ .Values.router.roundTrip.maxRetries | default 10 | quote }}
- name: ROUTER_SVC_ADDRESS_MAX_RETRIES
value: {{ .Values.router.svcAddressMaxRetries | default 5 | quote }}
- name: ROUTER_SVC_ADDRESS_UPDATE_TIMEOUT
value: {{ .Values.router.svcAddressUpdateTimeout | default "30s" | quote }}
- name: ROUTER_UNTAP_SERVICE_TIMEOUT
value: {{ .Values.router.unTapServiceTimeout | default "3600s" | quote }}
- name: USE_ENCODED_PATH
value: {{ .Values.router.useEncodedPath | default false | quote }}
- name: DEBUG_ENV
value: {{ .Values.debugEnv | quote }}
- name: DISPLAY_ACCESS_LOG
value: {{ .Values.router.displayAccessLog | default false | quote }}
{{- include "opentracing.envs" . | indent 8 }}
{{- include "opentelemtry.envs" . | indent 8 }}
resources:
{{- toYaml .Values.router.resources | indent 10 }}
readinessProbe:
+60 -12
View File
@@ -17,7 +17,7 @@ repository: index.docker.io
image: fission/fission-bundle
## Fission image version
imageTag: 1.14.0
imageTag: v1.15.0-rc1
## Image pull policy
pullPolicy: IfNotPresent
@@ -43,7 +43,7 @@ fetcher:
## Fetcher repository
image: fission/fetcher
## Fetcher image version
imageTag: 1.14.0
imageTag: v1.15.0-rc1
## Fetcher is only for to downloading or uploading archive.
## Normally, you don't need to change the value here, unless necessary.
@@ -108,7 +108,7 @@ router:
## Sample with a rate per time window (traces/second)
traceSamplingRate: 0.5
## Extend the container specs for the router fission pods.
## Can be used to add things like affinty/tolerations/nodeSelectors/etc.
## For example:
@@ -229,9 +229,54 @@ prometheus:
canaryDeployment:
enabled: false
# Use these flags to enable opentracing, the variable is endpoint of Jaeger collector in the format shown below
#traceCollectorEndpoint: "http://jaeger-collector.jaeger.svc:14268/api/traces?format=jaeger.thrift"
#traceSamplingRate: 0.75
# Use the following flags to enable OpenTracing.
# Note: OpenTracing support will be removed in an upcoming release.
# Please prefer using OpenTelemetry instead.
openTracing:
## set this flag to true if you wish to enable OpenTracing
enabled: false
## if enabled is true, the variable is endpoint of Jaeger collector in the format shown below
#collectorEndpoint: "http://jaeger-collector.jaeger.svc:14268/api/traces?format=jaeger.thrift"
## uniformly sample traces with the given probabilistic sampling rate
#samplingRate: 0.75
# It is an alternate to OpenTracing.
openTelemetry:
# Use this flag to set the collector endpoint for OpenTelemetry.
# The variable is endpoint of the collector in the format shown below.
# otlpCollectorEndpoint: "otel-collector.observability.svc:4317"
otlpCollectorEndpoint: ""
# Set this flag to false if you are using secure endpoint for the collector.
otlpInsecure: true
# Key-value pairs to be used as headers associated with gRPC or HTTP requests
# to the collector.
# Eg. otlpHeaders: "key1=value1,key2=value2"
otlpHeaders: ""
# Supported samplers:
# always_on - Sampler that always samples spans, regardless of the parent span's sampling decision.
# always_off - Sampler that never samples spans, regardless of the parent span's sampling decision.
# traceidratio - Sampler that samples probabalistically based on rate.
# parentbased_always_on - (default if empty) Sampler that respects its parent span's sampling decision, but otherwise always samples.
# parentbased_always_off - Sampler that respects its parent span's sampling decision, but otherwise never samples.
# parentbased_traceidratio - Sampler that respects its parent span's sampling decision, but otherwise samples probabalistically based on rate.
tracesSampler: "parentbased_traceidratio"
# Each Sampler type defines its own expected input, if any.
# Currently we get trace ratio for the case of,
# 1. traceidratio
# 2. parentbased_traceidratio
# Sampling probability, a number in the [0..1] range, e.g. "0.1". Default is 0.1.
tracesSamplingRate: "0.1"
# Supported providers:
# tracecontext - W3C Trace Context
# baggage - W3C Baggage
# b3 - B3 Single
# b3multi - B3 Multi
# jaeger - Jaeger uber-trace-id header
# xray - AWS X-Ray (third party)
# ottrace - OpenTracing Trace (third party)
propagators: "tracecontext,baggage"
## Message Queue Trigger Kind, KEDA: enable and configuration
mqt_keda:
@@ -239,19 +284,22 @@ mqt_keda:
connector_images:
kafka:
image: fission/keda-kafka-http-connector
tag: v0.7
tag: v0.8
rabbitmq:
image: fission/keda-rabbitmq-http-connector
tag: v0.7
tag: v0.8
awskinesis:
image: fission/keda-aws-kinesis-http-connector
tag: v0.7
tag: v0.8
aws_sqs:
image: fission/keda-aws-sqs-http-connector
tag: v0.7
tag: v0.8
nats_steaming:
image: fission/keda-nats-streaming-http-connector
tag: v0.7
tag: v0.9
gcp_pub_sub:
image: fission/keda-gcp-pubsub-http-connector
tag: v0.2
tag: v0.3
redis:
image: fission/keda-redis-http-connector
tag: v0.1
+2 -43
View File
@@ -1,45 +1,4 @@
FROM golang:1.15-alpine as godep
RUN apk add bash ca-certificates git gcc g++ libc-dev
ARG GOPKG=github.com/fission/fission
ENV GO111MODULE=on
WORKDIR /go/src/${GOPKG}
# To reuse build cache, copy go.mod & go.sum and download dependencies first.
COPY go.* ./
RUN go mod download
FROM godep as builder
ARG GOPKG
WORKDIR /go/src/${GOPKG}
# Copy whole fission directory to work dir
COPY ./ ./
WORKDIR /go/src/${GOPKG}/cmd/builder
ARG GITCOMMIT=unknown
# E.g. GITCOMMIT=$(git rev-parse HEAD)
ARG BUILDVERSION=unknown
# E.g. BUILDVERSION=$(git rev-parse HEAD)
ARG BUILDDATE=unknown
# E.g. BUILDDATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
RUN CGO_ENABLED=0 go build \
-o /go/bin/builder \
-gcflags=-trimpath=$GOPATH \
-asmflags=-trimpath=$GOPATH \
-ldflags "-X github.com/fission/fission/pkg/info.GitCommit=${GITCOMMIT} -X github.com/fission/fission/pkg/info.BuildDate=${BUILDDATE} -X github.com/fission/fission/pkg/info.Version=${BUILDVERSION}"
FROM alpine:3.13 as base
FROM alpine:3.14
RUN apk add --update ca-certificates
COPY --from=builder /go/bin/builder /
EXPOSE 8001
COPY builder /builder
ENTRYPOINT ["/builder"]
+5 -11
View File
@@ -17,28 +17,22 @@ limitations under the License.
package main
import (
"log"
"os"
"go.uber.org/zap"
"go.uber.org/zap/zapcore"
"github.com/fission/fission/cmd/builder/app"
"github.com/fission/fission/pkg/utils/loggerfactory"
"github.com/fission/fission/pkg/utils/profile"
)
// Usage: builder <shared volume path>
func main() {
profile.ProfileIfEnabled()
config := zap.NewProductionConfig()
config.EncoderConfig.EncodeTime = zapcore.ISO8601TimeEncoder
logger, err := config.Build()
if err != nil {
log.Fatalf("can't initialize zap logger: %v", err)
}
logger := loggerfactory.GetLogger()
defer logger.Sync()
profile.ProfileIfEnabled(logger)
shareVolume := os.Args[1]
if _, err := os.Stat(shareVolume); err != nil {
if os.IsNotExist(err) {
@@ -49,6 +43,6 @@ func main() {
}
}
err = app.Run(logger, shareVolume)
err := app.Run(logger, shareVolume)
logger.Error("error running builder", zap.Error(err))
}
+2 -43
View File
@@ -1,45 +1,4 @@
FROM golang:1.15-alpine as godep
RUN apk add bash ca-certificates git gcc g++ libc-dev
ARG GOPKG=github.com/fission/fission
ENV GO111MODULE=on
WORKDIR /go/src/${GOPKG}
# To reuse build cache, copy go.mod & go.sum and download dependencies first.
COPY go.* ./
RUN go mod download
FROM godep as builder
ARG GOPKG
WORKDIR /go/src/${GOPKG}
# Copy whole fission directory to work dir
COPY ./ ./
WORKDIR /go/src/${GOPKG}/cmd/fetcher
ARG GITCOMMIT=unknown
# E.g. GITCOMMIT=$(git rev-parse HEAD)
ARG BUILDVERSION=unknown
# E.g. BUILDVERSION=$(git rev-parse HEAD)
ARG BUILDDATE=unknown
# E.g. BUILDDATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
RUN CGO_ENABLED=0 go build \
-o /go/bin/fetcher \
-gcflags=-trimpath=$GOPATH \
-asmflags=-trimpath=$GOPATH \
-ldflags "-X github.com/fission/fission/pkg/info.GitCommit=${GITCOMMIT} -X github.com/fission/fission/pkg/info.BuildDate=${BUILDDATE} -X github.com/fission/fission/pkg/info.Version=${BUILDVERSION}"
FROM alpine:3.13 as base
FROM alpine:3.14
RUN apk add --update ca-certificates
COPY --from=builder /go/bin/fetcher /
EXPOSE 8000
COPY fetcher /
ENTRYPOINT ["/fetcher"]
+31 -34
View File
@@ -26,41 +26,19 @@ import (
"os"
"sync/atomic"
"contrib.go.opencensus.io/exporter/jaeger"
"go.opencensus.io/plugin/ochttp"
"go.opencensus.io/trace"
"go.opentelemetry.io/otel"
"go.uber.org/zap"
"github.com/fission/fission/pkg/fetcher"
otelUtils "github.com/fission/fission/pkg/utils/otel"
"github.com/fission/fission/pkg/utils/tracing"
)
var (
readyToServe uint32
)
func registerTraceExporter(collectorEndpoint string) error {
if collectorEndpoint == "" {
return nil
}
serviceName := "Fission-Fetcher"
exporter, err := jaeger.NewExporter(jaeger.Options{
CollectorEndpoint: collectorEndpoint,
Process: jaeger.Process{
ServiceName: serviceName,
Tags: []jaeger.Tag{
jaeger.BoolTag("fission", true),
},
},
})
if err != nil {
return err
}
trace.RegisterExporter(exporter)
trace.ApplyConfig(trace.Config{DefaultSampler: trace.AlwaysSample()})
return nil
}
func Run(logger *zap.Logger) {
flag.Usage = fetcherUsage
collectorEndpoint := flag.String("jaeger-collector-endpoint", "", "")
@@ -85,11 +63,27 @@ func Run(logger *zap.Logger) {
}
}
go func() {
if err := registerTraceExporter(*collectorEndpoint); err != nil {
logger.Fatal("could not register trace exporter", zap.Error(err), zap.String("collector_endpoint", *collectorEndpoint))
ctx := context.Background()
openTracingEnabled := tracing.TracingEnabled(logger)
if openTracingEnabled {
go func() {
if err := tracing.RegisterTraceExporter(logger, *collectorEndpoint, "Fission-Fetcher"); err != nil {
logger.Fatal("could not register trace exporter", zap.Error(err), zap.String("collector_endpoint", *collectorEndpoint))
}
}()
} else {
shutdown, err := otelUtils.InitProvider(ctx, logger, "Fission-Fetcher")
if err != nil {
logger.Fatal("error initializing provider for OTLP", zap.Error(err))
}
}()
if shutdown != nil {
defer shutdown(ctx)
}
}
tracer := otel.Tracer("fetcher")
ctx, span := tracer.Start(ctx, "fetcher/Run")
defer span.End()
f, err := fetcher.MakeFetcher(logger, dir, *secretDir, *configDir)
if err != nil {
@@ -106,7 +100,6 @@ func Run(logger *zap.Logger) {
logger.Fatal("error decoding specialize request", zap.Error(err))
}
ctx := context.Background()
err = f.SpecializePod(ctx, specializeReq.FetchReq, specializeReq.LoadReq)
if err != nil {
logger.Fatal("error specializing function pod", zap.Error(err))
@@ -137,10 +130,14 @@ func Run(logger *zap.Logger) {
})
logger.Info("fetcher ready to receive requests")
err = http.ListenAndServe(":8000", &ochttp.Handler{
Handler: mux,
})
if err != nil {
var handler http.Handler
if openTracingEnabled {
handler = &ochttp.Handler{Handler: mux}
} else {
handler = otelUtils.GetHandlerWithOTEL(mux, "fission-fetcher", otelUtils.UrlsToIgnore("/healthz", "/readiness-healthz"))
}
if err = http.ListenAndServe(":8000", handler); err != nil {
log.Fatal(err)
}
}
+5 -13
View File
@@ -17,25 +17,17 @@ limitations under the License.
package main
import (
"log"
"go.uber.org/zap"
"go.uber.org/zap/zapcore"
"github.com/fission/fission/cmd/fetcher/app"
"github.com/fission/fission/pkg/utils/loggerfactory"
"github.com/fission/fission/pkg/utils/profile"
)
// Usage: fetcher <shared volume path>
func main() {
profile.ProfileIfEnabled()
config := zap.NewProductionConfig()
config.EncoderConfig.EncodeTime = zapcore.ISO8601TimeEncoder
logger, err := config.Build()
if err != nil {
log.Fatalf("can't initialize zap logger: %v", err)
}
logger := loggerfactory.GetLogger()
defer logger.Sync()
profile.ProfileIfEnabled(logger)
app.Run(logger)
}
+2 -42
View File
@@ -1,44 +1,4 @@
FROM golang:1.15-alpine as godep
RUN apk add bash ca-certificates git gcc g++ libc-dev
ARG GOPKG=github.com/fission/fission
ENV GO111MODULE=on
WORKDIR /go/src/${GOPKG}
# To reuse build cache, copy go.mod & go.sum and download dependencies first.
COPY go.* ./
RUN go mod download
FROM godep as builder
ARG GOPKG
WORKDIR /go/src/${GOPKG}
# Copy whole fission directory to work dir
COPY ./ ./
WORKDIR /go/src/${GOPKG}/cmd/fission-bundle
ARG GITCOMMIT=unknown
# E.g. GITCOMMIT=$(git rev-parse HEAD)
ARG BUILDVERSION=unknown
# E.g. BUILDVERSION=$(git rev-parse HEAD)
ARG BUILDDATE=unknown
# E.g. BUILDDATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
RUN CGO_ENABLED=0 go build \
-o /go/bin/fission-bundle \
-gcflags=-trimpath=$GOPATH \
-asmflags=-trimpath=$GOPATH \
-ldflags "-X github.com/fission/fission/pkg/info.GitCommit=${GITCOMMIT} -X github.com/fission/fission/pkg/info.BuildDate=${BUILDDATE} -X github.com/fission/fission/pkg/info.Version=${BUILDVERSION}"
FROM alpine:3.13 as base
FROM alpine:3.14
RUN apk add --update ca-certificates
COPY --from=builder /go/bin/fission-bundle /
COPY fission-bundle /
ENTRYPOINT ["/fission-bundle"]
+38 -64
View File
@@ -17,17 +17,16 @@ limitations under the License.
package main
import (
"context"
"flag"
"fmt"
"log"
"os"
"strconv"
"contrib.go.opencensus.io/exporter/jaeger"
docopt "github.com/docopt/docopt-go"
"go.opencensus.io/trace"
"go.uber.org/zap"
"go.uber.org/zap/zapcore"
"github.com/fission/fission/cmd/fission-bundle/mqtrigger"
"github.com/fission/fission/pkg/buildermgr"
@@ -40,21 +39,24 @@ import (
"github.com/fission/fission/pkg/router"
"github.com/fission/fission/pkg/storagesvc"
"github.com/fission/fission/pkg/timer"
"github.com/fission/fission/pkg/utils/loggerfactory"
"github.com/fission/fission/pkg/utils/otel"
"github.com/fission/fission/pkg/utils/profile"
"github.com/fission/fission/pkg/utils/tracing"
)
func runController(logger *zap.Logger, port int) {
controller.Start(logger, port, false)
func runController(logger *zap.Logger, port int, openTracingEnabled bool) {
controller.Start(logger, port, false, openTracingEnabled)
logger.Fatal("controller exited")
}
func runRouter(logger *zap.Logger, port int, executorUrl string) {
router.Start(logger, port, executorUrl)
func runRouter(logger *zap.Logger, port int, executorUrl string, openTracingEnabled bool) {
router.Start(logger, port, executorUrl, openTracingEnabled)
logger.Fatal("router exited")
}
func runExecutor(logger *zap.Logger, port int, functionNamespace, envBuilderNamespace string) {
err := executor.StartExecutor(logger, functionNamespace, envBuilderNamespace, port)
func runExecutor(logger *zap.Logger, port int, functionNamespace, envBuilderNamespace string, openTracingEnabled bool) {
err := executor.StartExecutor(logger, functionNamespace, envBuilderNamespace, port, openTracingEnabled)
if err != nil {
logger.Fatal("error starting executor", zap.Error(err))
}
@@ -89,8 +91,8 @@ func runMQManager(logger *zap.Logger, routerURL string) {
}
}
func runStorageSvc(logger *zap.Logger, port int, storage storagesvc.Storage) {
err := storagesvc.Start(logger, storage, port)
func runStorageSvc(logger *zap.Logger, port int, storage storagesvc.Storage, openTracingEnabled bool) {
err := storagesvc.Start(logger, storage, port, openTracingEnabled)
if err != nil {
logger.Fatal("error starting storage service", zap.Error(err))
}
@@ -125,13 +127,7 @@ func getStringArgWithDefault(arg interface{}, defaultValue string) string {
}
}
func registerTraceExporter(logger *zap.Logger, arguments map[string]interface{}) error {
collectorEndpoint := os.Getenv("TRACE_JAEGER_COLLECTOR_ENDPOINT")
if len(collectorEndpoint) == 0 {
logger.Info("skipping trace exporter registration")
return nil
}
func getServiceName(arguments map[string]interface{}) string {
serviceName := "Fission-Unknown"
if arguments["--controllerPort"] != nil {
@@ -154,26 +150,7 @@ func registerTraceExporter(logger *zap.Logger, arguments map[string]interface{})
serviceName = "Fission-Keda-MQTrigger"
}
exporter, err := jaeger.NewExporter(jaeger.Options{
CollectorEndpoint: collectorEndpoint,
Process: jaeger.Process{
ServiceName: serviceName,
Tags: []jaeger.Tag{
jaeger.BoolTag("fission", true),
},
},
})
if err != nil {
return err
}
samplingRate, err := strconv.ParseFloat(os.Getenv("TRACING_SAMPLING_RATE"), 32)
if err != nil {
return err
}
trace.RegisterExporter(exporter)
trace.ApplyConfig(trace.Config{DefaultSampler: trace.ProbabilitySampler(samplingRate)})
return nil
return serviceName
}
func main() {
@@ -238,35 +215,32 @@ Options:
--builderMgr Start builder manager.
--version Print version information
`
profile.ProfileIfEnabled()
var logger *zap.Logger
var config zap.Config
isDebugEnv, _ := strconv.ParseBool(os.Getenv("DEBUG_ENV"))
if isDebugEnv {
config = zap.NewDevelopmentConfig()
config.DisableStacktrace = true
config.EncoderConfig.EncodeTime = zapcore.ISO8601TimeEncoder
} else {
config = zap.NewProductionConfig()
config.DisableStacktrace = true
config.EncoderConfig.EncodeTime = zapcore.ISO8601TimeEncoder
}
logger, err = config.Build()
if err != nil {
log.Fatalf("I can't initialize zap logger: %v", err)
}
logger := loggerfactory.GetLogger()
defer logger.Sync()
profile.ProfileIfEnabled(logger)
version := fmt.Sprintf("Fission Bundle Version: %v", info.BuildInfo().String())
arguments, err := docopt.ParseArgs(usage, nil, version)
if err != nil {
logger.Fatal("Could not parse command line arguments", zap.Error(err))
}
err = registerTraceExporter(logger, arguments)
if err != nil {
logger.Fatal("Could not register trace exporter", zap.Error(err), zap.Any("argument", arguments))
ctx := context.Background()
openTracingEnabled := tracing.TracingEnabled(logger)
if openTracingEnabled {
err = tracing.RegisterTraceExporter(logger, os.Getenv("TRACE_JAEGER_COLLECTOR_ENDPOINT"), getServiceName(arguments))
if err != nil {
logger.Fatal("Could not register trace exporter", zap.Error(err), zap.Any("argument", arguments))
}
} else {
shutdown, err := otel.InitProvider(ctx, logger, getServiceName(arguments))
if err != nil {
logger.Fatal("error initializing provider for OTLP", zap.Error(err), zap.Any("argument", arguments))
}
if shutdown != nil {
defer shutdown(ctx)
}
}
functionNs := getStringArgWithDefault(arguments["--namespace"], "fission-function")
@@ -278,17 +252,17 @@ Options:
if arguments["--controllerPort"] != nil {
port := getPort(logger, arguments["--controllerPort"])
runController(logger, port)
runController(logger, port, openTracingEnabled)
}
if arguments["--routerPort"] != nil {
port := getPort(logger, arguments["--routerPort"])
runRouter(logger, port, executorUrl)
runRouter(logger, port, executorUrl, openTracingEnabled)
}
if arguments["--executorPort"] != nil {
port := getPort(logger, arguments["--executorPort"])
runExecutor(logger, port, functionNs, envBuilderNs)
runExecutor(logger, port, functionNs, envBuilderNs, openTracingEnabled)
}
if arguments["--kubewatcher"] == true {
@@ -325,7 +299,7 @@ Options:
} else if arguments["--storageType"] == string(storagesvc.StorageTypeLocal) {
storage = storagesvc.NewLocalStorage("/fission")
}
runStorageSvc(logger, port, storage)
runStorageSvc(logger, port, storage, openTracingEnabled)
}
select {}
@@ -1,45 +1,4 @@
FROM golang:1.15-alpine as godep
RUN apk add bash ca-certificates git gcc g++ libc-dev
ARG GOPKG=github.com/fission/fission
ENV GO111MODULE=on
WORKDIR /go/src/${GOPKG}
# To reuse build cache, copy go.mod & go.sum and download dependencies first.
COPY go.* ./
RUN go mod download
FROM godep as builder
ARG GOPKG
WORKDIR /go/src/${GOPKG}
# Copy whole fission directory to work dir
COPY ./ ./
WORKDIR /go/src/${GOPKG}/cmd/preupgradechecks
ARG GITCOMMIT=unknown
# E.g. GITCOMMIT=$(git rev-parse HEAD)
ARG BUILDVERSION=unknown
# E.g. BUILDVERSION=$(git rev-parse HEAD)
ARG BUILDDATE=unknown
# E.g. BUILDDATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
RUN CGO_ENABLED=0 go build \
-o /go/bin/pre-upgrade-checks \
-gcflags=-trimpath=$GOPATH \
-asmflags=-trimpath=$GOPATH \
-ldflags "-X github.com/fission/fission/pkg/info.GitCommit=${GITCOMMIT} -X github.com/fission/fission/pkg/info.BuildDate=${BUILDDATE} -X github.com/fission/fission/pkg/info.Version=${BUILDVERSION}"
FROM alpine:3.13 as base
FROM alpine:3.14
RUN apk add --update ca-certificates
COPY --from=builder /go/bin/pre-upgrade-checks /
COPY pre-upgrade-checks /
ENTRYPOINT ["/pre-upgrade-checks"]
EXPOSE 8001
+2 -10
View File
@@ -17,13 +17,11 @@ limitations under the License.
package main
import (
"log"
"github.com/docopt/docopt-go"
"go.uber.org/zap"
"go.uber.org/zap/zapcore"
"github.com/fission/fission/pkg/info"
"github.com/fission/fission/pkg/utils/loggerfactory"
)
func getStringArgWithDefault(arg interface{}, defaultValue string) string {
@@ -35,13 +33,7 @@ func getStringArgWithDefault(arg interface{}, defaultValue string) string {
}
func main() {
config := zap.NewProductionConfig()
config.EncoderConfig.EncodeTime = zapcore.ISO8601TimeEncoder
logger, err := config.Build()
if err != nil {
log.Fatalf("can't initialize zap logger: %v", err)
}
logger := loggerfactory.GetLogger()
defer logger.Sync()
usage := `Package to perform operations needed prior to fission installation
+2 -41
View File
@@ -1,43 +1,4 @@
FROM golang:1.15-alpine as godep
RUN apk add bash ca-certificates git gcc g++ libc-dev
ARG GOPKG=github.com/fission/fission
ENV GO111MODULE=on
WORKDIR /go/src/${GOPKG}
# To reuse build cache, copy go.mod & go.sum and download dependencies first.
COPY go.* ./
RUN go mod download
FROM godep as builder
ARG GOPKG
WORKDIR /go/src/${GOPKG}
# Copy whole fission directory to work dir
COPY ./ ./
WORKDIR /go/src/${GOPKG}/cmd/reporter
ARG GITCOMMIT=unknown
# E.g. GITCOMMIT=$(git rev-parse HEAD)
ARG BUILDVERSION=unknown
# E.g. BUILDVERSION=$(git rev-parse HEAD)
ARG BUILDDATE=unknown
# E.g. BUILDDATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
RUN CGO_ENABLED=0 go build \
-o /go/bin/reporter \
-gcflags=-trimpath=$GOPATH \
-asmflags=-trimpath=$GOPATH \
-ldflags "-X github.com/fission/fission/pkg/info.GitCommit=${GITCOMMIT} -X github.com/fission/fission/pkg/info.BuildDate=${BUILDDATE} -X github.com/fission/fission/pkg/info.Version=${BUILDVERSION}"
FROM alpine:3.13 as base
FROM alpine:3.14
RUN apk add --update ca-certificates
COPY --from=builder /go/bin/reporter /
COPY reporter /
ENTRYPOINT ["/reporter"]
+386 -60
View File
@@ -387,6 +387,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -506,13 +510,17 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
type: integer
type: object
resources:
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
properties:
limits:
additionalProperties:
@@ -521,7 +529,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -530,7 +538,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
securityContext:
@@ -618,7 +626,7 @@ spec:
type: object
type: object
startupProbe:
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. This is a beta feature enabled by the StartupProbe feature flag. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
properties:
exec:
description: One and only one of the following should be specified. Exec specifies the action to take.
@@ -697,6 +705,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -940,8 +952,38 @@ spec:
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaceSelector:
description: A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means "this pod's namespace". An empty selector ({}) matches all namespaces. This field is alpha-level and is only honored when PodAffinityNamespaceSelector feature is enabled.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaces:
description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
description: namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means "this pod's namespace"
items:
type: string
type: array
@@ -995,8 +1037,38 @@ spec:
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaceSelector:
description: A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means "this pod's namespace". An empty selector ({}) matches all namespaces. This field is alpha-level and is only honored when PodAffinityNamespaceSelector feature is enabled.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaces:
description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
description: namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means "this pod's namespace"
items:
type: string
type: array
@@ -1049,8 +1121,38 @@ spec:
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaceSelector:
description: A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means "this pod's namespace". An empty selector ({}) matches all namespaces. This field is alpha-level and is only honored when PodAffinityNamespaceSelector feature is enabled.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaces:
description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
description: namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means "this pod's namespace"
items:
type: string
type: array
@@ -1104,8 +1206,38 @@ spec:
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaceSelector:
description: A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means "this pod's namespace". An empty selector ({}) matches all namespaces. This field is alpha-level and is only honored when PodAffinityNamespaceSelector feature is enabled.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaces:
description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
description: namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means "this pod's namespace"
items:
type: string
type: array
@@ -1466,6 +1598,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -1585,13 +1721,17 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
type: integer
type: object
resources:
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
properties:
limits:
additionalProperties:
@@ -1600,7 +1740,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -1609,7 +1749,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
securityContext:
@@ -1697,7 +1837,7 @@ spec:
type: object
type: object
startupProbe:
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. This is a beta feature enabled by the StartupProbe feature flag. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
properties:
exec:
description: One and only one of the following should be specified. Exec specifies the action to take.
@@ -1776,6 +1916,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -2224,6 +2368,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -2339,6 +2487,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -2354,7 +2506,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -2363,7 +2515,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
securityContext:
@@ -2530,6 +2682,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -2986,6 +3142,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -3105,13 +3265,17 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
type: integer
type: object
resources:
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
properties:
limits:
additionalProperties:
@@ -3120,7 +3284,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -3129,7 +3293,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
securityContext:
@@ -3217,7 +3381,7 @@ spec:
type: object
type: object
startupProbe:
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. This is a beta feature enabled by the StartupProbe feature flag. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
properties:
exec:
description: One and only one of the following should be specified. Exec specifies the action to take.
@@ -3296,6 +3460,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -3423,7 +3591,7 @@ spec:
format: int64
type: integer
fsGroupChangePolicy:
description: 'fsGroupChangePolicy defines behavior of changing ownership and permission of the volume before being exposed inside Pod. This field will only apply to volume types which support fsGroup based ownership(and permissions). It will have no effect on ephemeral volume types such as: secret, configmaps and emptydir. Valid values are "OnRootMismatch" and "Always". If not specified defaults to "Always".'
description: 'fsGroupChangePolicy defines behavior of changing ownership and permission of the volume before being exposed inside Pod. This field will only apply to volume types which support fsGroup based ownership(and permissions). It will have no effect on ephemeral volume types such as: secret, configmaps and emptydir. Valid values are "OnRootMismatch" and "Always". If not specified, "Always" is used.'
type: string
runAsGroup:
description: The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.
@@ -3516,7 +3684,7 @@ spec:
description: If specified, the fully qualified Pod hostname will be "<hostname>.<subdomain>.<pod namespace>.svc.<cluster domain>". If not specified, the pod will not have a domainname at all.
type: string
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully. May be decreased in delete request. Value must be non-negative integer. The value zero indicates delete immediately. If this value is nil, the default grace period will be used instead. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. Defaults to 30 seconds.
description: Optional duration in seconds the pod needs to terminate gracefully. May be decreased in delete request. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). If this value is nil, the default grace period will be used instead. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. Defaults to 30 seconds.
format: int64
type: integer
tolerations:
@@ -3845,11 +4013,8 @@ spec:
x-kubernetes-int-or-string: true
type: object
ephemeral:
description: "Ephemeral represents a volume that is handled by a cluster storage driver (Alpha feature). The volume's lifecycle is tied to the pod that defines it - it will be created before the pod starts, and deleted when the pod is removed. \n Use this if: a) the volume is only needed while the pod runs, b) features of normal volumes like restoring from snapshot or capacity tracking are needed, c) the storage driver is specified through a storage class, and d) the storage driver supports dynamic volume provisioning through a PersistentVolumeClaim (see EphemeralVolumeSource for more information on the connection between this volume type and PersistentVolumeClaim). \n Use PersistentVolumeClaim or one of the vendor-specific APIs for volumes that persist for longer than the lifecycle of an individual pod. \n Use CSI for light-weight local ephemeral volumes if the CSI driver is meant to be used that way - see the documentation of the driver for more information. \n A pod can use both types of ephemeral volumes and persistent volumes at the same time."
description: "Ephemeral represents a volume that is handled by a cluster storage driver. The volume's lifecycle is tied to the pod that defines it - it will be created before the pod starts, and deleted when the pod is removed. \n Use this if: a) the volume is only needed while the pod runs, b) features of normal volumes like restoring from snapshot or capacity tracking are needed, c) the storage driver is specified through a storage class, and d) the storage driver supports dynamic volume provisioning through a PersistentVolumeClaim (see EphemeralVolumeSource for more information on the connection between this volume type and PersistentVolumeClaim). \n Use PersistentVolumeClaim or one of the vendor-specific APIs for volumes that persist for longer than the lifecycle of an individual pod. \n Use CSI for light-weight local ephemeral volumes if the CSI driver is meant to be used that way - see the documentation of the driver for more information. \n A pod can use both types of ephemeral volumes and persistent volumes at the same time. \n This is a beta feature and only available when the GenericEphemeralVolume feature gate is enabled."
properties:
readOnly:
description: Specifies a read-only configuration for the volume. Defaults to false (read/write).
type: boolean
volumeClaimTemplate:
description: "Will be used to create a stand-alone PVC to provision the volume. The pod in which this EphemeralVolumeSource is embedded will be the owner of the PVC, i.e. the PVC will be deleted together with the pod. The name of the PVC will be `<pod name>-<volume name>` where `<volume name>` is the name from the `PodSpec.Volumes` array entry. Pod validation will reject the pod if the concatenated name is not valid for a PVC (for example, too long). \n An existing PVC with that name that is not owned by the pod will *not* be used for the pod to avoid using an unrelated volume by mistake. Starting the pod is then blocked until the unrelated PVC is removed. If such a pre-created PVC is meant to be used by the pod, the PVC has to updated with an owner reference to the pod once the pod exists. Normally this should not be necessary, but it may be useful when manually reconstructing a broken cluster. \n This field is read-only and no changes will be made by Kubernetes to the PVC after it has been created. \n Required, must not be nil."
properties:
@@ -3865,7 +4030,7 @@ spec:
type: string
type: array
dataSource:
description: 'This field can be used to specify either: * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot - Beta) * An existing PVC (PersistentVolumeClaim) * An existing custom resource/object that implements data population (Alpha) In order to use VolumeSnapshot object types, the appropriate feature gate must be enabled (VolumeSnapshotDataSource or AnyVolumeDataSource) If the provisioner or an external controller can support the specified data source, it will create a new volume based on the contents of the specified data source. If the specified data source is not supported, the volume will not be created and the failure will be reported as an event. In the future, we plan to support more data source types and the behavior of the provisioner may change.'
description: 'This field can be used to specify either: * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot) * An existing PVC (PersistentVolumeClaim) * An existing custom resource that implements data population (Alpha) In order to use custom resource types that implement data population, the AnyVolumeDataSource feature gate must be enabled. If the provisioner or an external controller can support the specified data source, it will create a new volume based on the contents of the specified data source.'
properties:
apiGroup:
description: APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.
@@ -3890,7 +4055,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -3899,7 +4064,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
selector:
@@ -4316,8 +4481,6 @@ spec:
type: object
type: object
type: array
required:
- sources
type: object
quobyte:
description: Quobyte represents a Quobyte mount on the host that shares a pod's lifetime
@@ -4525,7 +4688,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -4534,7 +4697,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
runtime:
@@ -4883,6 +5046,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -5002,13 +5169,17 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
type: integer
type: object
resources:
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
properties:
limits:
additionalProperties:
@@ -5017,7 +5188,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -5026,7 +5197,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
securityContext:
@@ -5114,7 +5285,7 @@ spec:
type: object
type: object
startupProbe:
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. This is a beta feature enabled by the StartupProbe feature flag. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
properties:
exec:
description: One and only one of the following should be specified. Exec specifies the action to take.
@@ -5193,6 +5364,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -5436,8 +5611,38 @@ spec:
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaceSelector:
description: A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means "this pod's namespace". An empty selector ({}) matches all namespaces. This field is alpha-level and is only honored when PodAffinityNamespaceSelector feature is enabled.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaces:
description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
description: namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means "this pod's namespace"
items:
type: string
type: array
@@ -5491,8 +5696,38 @@ spec:
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaceSelector:
description: A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means "this pod's namespace". An empty selector ({}) matches all namespaces. This field is alpha-level and is only honored when PodAffinityNamespaceSelector feature is enabled.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaces:
description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
description: namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means "this pod's namespace"
items:
type: string
type: array
@@ -5545,8 +5780,38 @@ spec:
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaceSelector:
description: A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means "this pod's namespace". An empty selector ({}) matches all namespaces. This field is alpha-level and is only honored when PodAffinityNamespaceSelector feature is enabled.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaces:
description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
description: namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means "this pod's namespace"
items:
type: string
type: array
@@ -5600,8 +5865,38 @@ spec:
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaceSelector:
description: A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means "this pod's namespace". An empty selector ({}) matches all namespaces. This field is alpha-level and is only honored when PodAffinityNamespaceSelector feature is enabled.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaces:
description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
description: namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means "this pod's namespace"
items:
type: string
type: array
@@ -5962,6 +6257,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -6081,13 +6380,17 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
type: integer
type: object
resources:
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
properties:
limits:
additionalProperties:
@@ -6096,7 +6399,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -6105,7 +6408,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
securityContext:
@@ -6193,7 +6496,7 @@ spec:
type: object
type: object
startupProbe:
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. This is a beta feature enabled by the StartupProbe feature flag. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
properties:
exec:
description: One and only one of the following should be specified. Exec specifies the action to take.
@@ -6272,6 +6575,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -6720,6 +7027,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -6835,6 +7146,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -6850,7 +7165,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -6859,7 +7174,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
securityContext:
@@ -7026,6 +7341,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -7482,6 +7801,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -7601,13 +7924,17 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
type: integer
type: object
resources:
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
properties:
limits:
additionalProperties:
@@ -7616,7 +7943,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -7625,7 +7952,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
securityContext:
@@ -7713,7 +8040,7 @@ spec:
type: object
type: object
startupProbe:
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. This is a beta feature enabled by the StartupProbe feature flag. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
properties:
exec:
description: One and only one of the following should be specified. Exec specifies the action to take.
@@ -7792,6 +8119,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -7919,7 +8250,7 @@ spec:
format: int64
type: integer
fsGroupChangePolicy:
description: 'fsGroupChangePolicy defines behavior of changing ownership and permission of the volume before being exposed inside Pod. This field will only apply to volume types which support fsGroup based ownership(and permissions). It will have no effect on ephemeral volume types such as: secret, configmaps and emptydir. Valid values are "OnRootMismatch" and "Always". If not specified defaults to "Always".'
description: 'fsGroupChangePolicy defines behavior of changing ownership and permission of the volume before being exposed inside Pod. This field will only apply to volume types which support fsGroup based ownership(and permissions). It will have no effect on ephemeral volume types such as: secret, configmaps and emptydir. Valid values are "OnRootMismatch" and "Always". If not specified, "Always" is used.'
type: string
runAsGroup:
description: The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.
@@ -8012,7 +8343,7 @@ spec:
description: If specified, the fully qualified Pod hostname will be "<hostname>.<subdomain>.<pod namespace>.svc.<cluster domain>". If not specified, the pod will not have a domainname at all.
type: string
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully. May be decreased in delete request. Value must be non-negative integer. The value zero indicates delete immediately. If this value is nil, the default grace period will be used instead. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. Defaults to 30 seconds.
description: Optional duration in seconds the pod needs to terminate gracefully. May be decreased in delete request. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). If this value is nil, the default grace period will be used instead. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. Defaults to 30 seconds.
format: int64
type: integer
tolerations:
@@ -8341,11 +8672,8 @@ spec:
x-kubernetes-int-or-string: true
type: object
ephemeral:
description: "Ephemeral represents a volume that is handled by a cluster storage driver (Alpha feature). The volume's lifecycle is tied to the pod that defines it - it will be created before the pod starts, and deleted when the pod is removed. \n Use this if: a) the volume is only needed while the pod runs, b) features of normal volumes like restoring from snapshot or capacity tracking are needed, c) the storage driver is specified through a storage class, and d) the storage driver supports dynamic volume provisioning through a PersistentVolumeClaim (see EphemeralVolumeSource for more information on the connection between this volume type and PersistentVolumeClaim). \n Use PersistentVolumeClaim or one of the vendor-specific APIs for volumes that persist for longer than the lifecycle of an individual pod. \n Use CSI for light-weight local ephemeral volumes if the CSI driver is meant to be used that way - see the documentation of the driver for more information. \n A pod can use both types of ephemeral volumes and persistent volumes at the same time."
description: "Ephemeral represents a volume that is handled by a cluster storage driver. The volume's lifecycle is tied to the pod that defines it - it will be created before the pod starts, and deleted when the pod is removed. \n Use this if: a) the volume is only needed while the pod runs, b) features of normal volumes like restoring from snapshot or capacity tracking are needed, c) the storage driver is specified through a storage class, and d) the storage driver supports dynamic volume provisioning through a PersistentVolumeClaim (see EphemeralVolumeSource for more information on the connection between this volume type and PersistentVolumeClaim). \n Use PersistentVolumeClaim or one of the vendor-specific APIs for volumes that persist for longer than the lifecycle of an individual pod. \n Use CSI for light-weight local ephemeral volumes if the CSI driver is meant to be used that way - see the documentation of the driver for more information. \n A pod can use both types of ephemeral volumes and persistent volumes at the same time. \n This is a beta feature and only available when the GenericEphemeralVolume feature gate is enabled."
properties:
readOnly:
description: Specifies a read-only configuration for the volume. Defaults to false (read/write).
type: boolean
volumeClaimTemplate:
description: "Will be used to create a stand-alone PVC to provision the volume. The pod in which this EphemeralVolumeSource is embedded will be the owner of the PVC, i.e. the PVC will be deleted together with the pod. The name of the PVC will be `<pod name>-<volume name>` where `<volume name>` is the name from the `PodSpec.Volumes` array entry. Pod validation will reject the pod if the concatenated name is not valid for a PVC (for example, too long). \n An existing PVC with that name that is not owned by the pod will *not* be used for the pod to avoid using an unrelated volume by mistake. Starting the pod is then blocked until the unrelated PVC is removed. If such a pre-created PVC is meant to be used by the pod, the PVC has to updated with an owner reference to the pod once the pod exists. Normally this should not be necessary, but it may be useful when manually reconstructing a broken cluster. \n This field is read-only and no changes will be made by Kubernetes to the PVC after it has been created. \n Required, must not be nil."
properties:
@@ -8361,7 +8689,7 @@ spec:
type: string
type: array
dataSource:
description: 'This field can be used to specify either: * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot - Beta) * An existing PVC (PersistentVolumeClaim) * An existing custom resource/object that implements data population (Alpha) In order to use VolumeSnapshot object types, the appropriate feature gate must be enabled (VolumeSnapshotDataSource or AnyVolumeDataSource) If the provisioner or an external controller can support the specified data source, it will create a new volume based on the contents of the specified data source. If the specified data source is not supported, the volume will not be created and the failure will be reported as an event. In the future, we plan to support more data source types and the behavior of the provisioner may change.'
description: 'This field can be used to specify either: * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot) * An existing PVC (PersistentVolumeClaim) * An existing custom resource that implements data population (Alpha) In order to use custom resource types that implement data population, the AnyVolumeDataSource feature gate must be enabled. If the provisioner or an external controller can support the specified data source, it will create a new volume based on the contents of the specified data source.'
properties:
apiGroup:
description: APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.
@@ -8386,7 +8714,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -8395,7 +8723,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
selector:
@@ -8812,8 +9140,6 @@ spec:
type: object
type: object
type: array
required:
- sources
type: object
quobyte:
description: Quobyte represents a Quobyte mount on the host that shares a pod's lifetime
+178 -27
View File
@@ -286,8 +286,38 @@ spec:
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaceSelector:
description: A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means "this pod's namespace". An empty selector ({}) matches all namespaces. This field is alpha-level and is only honored when PodAffinityNamespaceSelector feature is enabled.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaces:
description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
description: namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means "this pod's namespace"
items:
type: string
type: array
@@ -341,8 +371,38 @@ spec:
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaceSelector:
description: A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means "this pod's namespace". An empty selector ({}) matches all namespaces. This field is alpha-level and is only honored when PodAffinityNamespaceSelector feature is enabled.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaces:
description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
description: namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means "this pod's namespace"
items:
type: string
type: array
@@ -395,8 +455,38 @@ spec:
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaceSelector:
description: A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means "this pod's namespace". An empty selector ({}) matches all namespaces. This field is alpha-level and is only honored when PodAffinityNamespaceSelector feature is enabled.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaces:
description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
description: namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means "this pod's namespace"
items:
type: string
type: array
@@ -450,8 +540,38 @@ spec:
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaceSelector:
description: A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means "this pod's namespace". An empty selector ({}) matches all namespaces. This field is alpha-level and is only honored when PodAffinityNamespaceSelector feature is enabled.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaces:
description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
description: namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means "this pod's namespace"
items:
type: string
type: array
@@ -812,6 +932,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -931,13 +1055,17 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
type: integer
type: object
resources:
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
properties:
limits:
additionalProperties:
@@ -946,7 +1074,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -955,7 +1083,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
securityContext:
@@ -1043,7 +1171,7 @@ spec:
type: object
type: object
startupProbe:
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. This is a beta feature enabled by the StartupProbe feature flag. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
properties:
exec:
description: One and only one of the following should be specified. Exec specifies the action to take.
@@ -1122,6 +1250,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -1570,6 +1702,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -1685,6 +1821,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -1700,7 +1840,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -1709,7 +1849,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
securityContext:
@@ -1876,6 +2016,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -2332,6 +2476,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -2451,13 +2599,17 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
type: integer
type: object
resources:
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
properties:
limits:
additionalProperties:
@@ -2466,7 +2618,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -2475,7 +2627,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
securityContext:
@@ -2563,7 +2715,7 @@ spec:
type: object
type: object
startupProbe:
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. This is a beta feature enabled by the StartupProbe feature flag. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
properties:
exec:
description: One and only one of the following should be specified. Exec specifies the action to take.
@@ -2642,6 +2794,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -2769,7 +2925,7 @@ spec:
format: int64
type: integer
fsGroupChangePolicy:
description: 'fsGroupChangePolicy defines behavior of changing ownership and permission of the volume before being exposed inside Pod. This field will only apply to volume types which support fsGroup based ownership(and permissions). It will have no effect on ephemeral volume types such as: secret, configmaps and emptydir. Valid values are "OnRootMismatch" and "Always". If not specified defaults to "Always".'
description: 'fsGroupChangePolicy defines behavior of changing ownership and permission of the volume before being exposed inside Pod. This field will only apply to volume types which support fsGroup based ownership(and permissions). It will have no effect on ephemeral volume types such as: secret, configmaps and emptydir. Valid values are "OnRootMismatch" and "Always". If not specified, "Always" is used.'
type: string
runAsGroup:
description: The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.
@@ -2862,7 +3018,7 @@ spec:
description: If specified, the fully qualified Pod hostname will be "<hostname>.<subdomain>.<pod namespace>.svc.<cluster domain>". If not specified, the pod will not have a domainname at all.
type: string
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully. May be decreased in delete request. Value must be non-negative integer. The value zero indicates delete immediately. If this value is nil, the default grace period will be used instead. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. Defaults to 30 seconds.
description: Optional duration in seconds the pod needs to terminate gracefully. May be decreased in delete request. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). If this value is nil, the default grace period will be used instead. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. Defaults to 30 seconds.
format: int64
type: integer
tolerations:
@@ -3191,11 +3347,8 @@ spec:
x-kubernetes-int-or-string: true
type: object
ephemeral:
description: "Ephemeral represents a volume that is handled by a cluster storage driver (Alpha feature). The volume's lifecycle is tied to the pod that defines it - it will be created before the pod starts, and deleted when the pod is removed. \n Use this if: a) the volume is only needed while the pod runs, b) features of normal volumes like restoring from snapshot or capacity tracking are needed, c) the storage driver is specified through a storage class, and d) the storage driver supports dynamic volume provisioning through a PersistentVolumeClaim (see EphemeralVolumeSource for more information on the connection between this volume type and PersistentVolumeClaim). \n Use PersistentVolumeClaim or one of the vendor-specific APIs for volumes that persist for longer than the lifecycle of an individual pod. \n Use CSI for light-weight local ephemeral volumes if the CSI driver is meant to be used that way - see the documentation of the driver for more information. \n A pod can use both types of ephemeral volumes and persistent volumes at the same time."
description: "Ephemeral represents a volume that is handled by a cluster storage driver. The volume's lifecycle is tied to the pod that defines it - it will be created before the pod starts, and deleted when the pod is removed. \n Use this if: a) the volume is only needed while the pod runs, b) features of normal volumes like restoring from snapshot or capacity tracking are needed, c) the storage driver is specified through a storage class, and d) the storage driver supports dynamic volume provisioning through a PersistentVolumeClaim (see EphemeralVolumeSource for more information on the connection between this volume type and PersistentVolumeClaim). \n Use PersistentVolumeClaim or one of the vendor-specific APIs for volumes that persist for longer than the lifecycle of an individual pod. \n Use CSI for light-weight local ephemeral volumes if the CSI driver is meant to be used that way - see the documentation of the driver for more information. \n A pod can use both types of ephemeral volumes and persistent volumes at the same time. \n This is a beta feature and only available when the GenericEphemeralVolume feature gate is enabled."
properties:
readOnly:
description: Specifies a read-only configuration for the volume. Defaults to false (read/write).
type: boolean
volumeClaimTemplate:
description: "Will be used to create a stand-alone PVC to provision the volume. The pod in which this EphemeralVolumeSource is embedded will be the owner of the PVC, i.e. the PVC will be deleted together with the pod. The name of the PVC will be `<pod name>-<volume name>` where `<volume name>` is the name from the `PodSpec.Volumes` array entry. Pod validation will reject the pod if the concatenated name is not valid for a PVC (for example, too long). \n An existing PVC with that name that is not owned by the pod will *not* be used for the pod to avoid using an unrelated volume by mistake. Starting the pod is then blocked until the unrelated PVC is removed. If such a pre-created PVC is meant to be used by the pod, the PVC has to updated with an owner reference to the pod once the pod exists. Normally this should not be necessary, but it may be useful when manually reconstructing a broken cluster. \n This field is read-only and no changes will be made by Kubernetes to the PVC after it has been created. \n Required, must not be nil."
properties:
@@ -3211,7 +3364,7 @@ spec:
type: string
type: array
dataSource:
description: 'This field can be used to specify either: * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot - Beta) * An existing PVC (PersistentVolumeClaim) * An existing custom resource/object that implements data population (Alpha) In order to use VolumeSnapshot object types, the appropriate feature gate must be enabled (VolumeSnapshotDataSource or AnyVolumeDataSource) If the provisioner or an external controller can support the specified data source, it will create a new volume based on the contents of the specified data source. If the specified data source is not supported, the volume will not be created and the failure will be reported as an event. In the future, we plan to support more data source types and the behavior of the provisioner may change.'
description: 'This field can be used to specify either: * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot) * An existing PVC (PersistentVolumeClaim) * An existing custom resource that implements data population (Alpha) In order to use custom resource types that implement data population, the AnyVolumeDataSource feature gate must be enabled. If the provisioner or an external controller can support the specified data source, it will create a new volume based on the contents of the specified data source.'
properties:
apiGroup:
description: APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.
@@ -3236,7 +3389,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -3245,7 +3398,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
selector:
@@ -3662,8 +3815,6 @@ spec:
type: object
type: object
type: array
required:
- sources
type: object
quobyte:
description: Quobyte represents a Quobyte mount on the host that shares a pod's lifetime
@@ -3864,7 +4015,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -3873,7 +4024,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
secrets:
+176 -25
View File
@@ -253,8 +253,38 @@ spec:
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaceSelector:
description: A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means "this pod's namespace". An empty selector ({}) matches all namespaces. This field is alpha-level and is only honored when PodAffinityNamespaceSelector feature is enabled.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaces:
description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
description: namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means "this pod's namespace"
items:
type: string
type: array
@@ -308,8 +338,38 @@ spec:
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaceSelector:
description: A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means "this pod's namespace". An empty selector ({}) matches all namespaces. This field is alpha-level and is only honored when PodAffinityNamespaceSelector feature is enabled.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaces:
description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
description: namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means "this pod's namespace"
items:
type: string
type: array
@@ -362,8 +422,38 @@ spec:
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaceSelector:
description: A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means "this pod's namespace". An empty selector ({}) matches all namespaces. This field is alpha-level and is only honored when PodAffinityNamespaceSelector feature is enabled.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaces:
description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
description: namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means "this pod's namespace"
items:
type: string
type: array
@@ -417,8 +507,38 @@ spec:
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaceSelector:
description: A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means "this pod's namespace". An empty selector ({}) matches all namespaces. This field is alpha-level and is only honored when PodAffinityNamespaceSelector feature is enabled.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
namespaces:
description: namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means "this pod's namespace"
description: namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means "this pod's namespace"
items:
type: string
type: array
@@ -779,6 +899,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -898,13 +1022,17 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
type: integer
type: object
resources:
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
properties:
limits:
additionalProperties:
@@ -913,7 +1041,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -922,7 +1050,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
securityContext:
@@ -1010,7 +1138,7 @@ spec:
type: object
type: object
startupProbe:
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. This is a beta feature enabled by the StartupProbe feature flag. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
properties:
exec:
description: One and only one of the following should be specified. Exec specifies the action to take.
@@ -1089,6 +1217,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -1537,6 +1669,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -1652,6 +1788,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -1667,7 +1807,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -1676,7 +1816,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
securityContext:
@@ -1843,6 +1983,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -2299,6 +2443,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -2418,13 +2566,17 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
type: integer
type: object
resources:
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
properties:
limits:
additionalProperties:
@@ -2433,7 +2585,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -2442,7 +2594,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
securityContext:
@@ -2530,7 +2682,7 @@ spec:
type: object
type: object
startupProbe:
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. This is a beta feature enabled by the StartupProbe feature flag. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
description: 'StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod''s lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
properties:
exec:
description: One and only one of the following should be specified. Exec specifies the action to take.
@@ -2609,6 +2761,10 @@ spec:
required:
- port
type: object
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is an alpha field and requires enabling ProbeTerminationGracePeriod feature gate.
format: int64
type: integer
timeoutSeconds:
description: 'Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
format: int32
@@ -2736,7 +2892,7 @@ spec:
format: int64
type: integer
fsGroupChangePolicy:
description: 'fsGroupChangePolicy defines behavior of changing ownership and permission of the volume before being exposed inside Pod. This field will only apply to volume types which support fsGroup based ownership(and permissions). It will have no effect on ephemeral volume types such as: secret, configmaps and emptydir. Valid values are "OnRootMismatch" and "Always". If not specified defaults to "Always".'
description: 'fsGroupChangePolicy defines behavior of changing ownership and permission of the volume before being exposed inside Pod. This field will only apply to volume types which support fsGroup based ownership(and permissions). It will have no effect on ephemeral volume types such as: secret, configmaps and emptydir. Valid values are "OnRootMismatch" and "Always". If not specified, "Always" is used.'
type: string
runAsGroup:
description: The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.
@@ -2829,7 +2985,7 @@ spec:
description: If specified, the fully qualified Pod hostname will be "<hostname>.<subdomain>.<pod namespace>.svc.<cluster domain>". If not specified, the pod will not have a domainname at all.
type: string
terminationGracePeriodSeconds:
description: Optional duration in seconds the pod needs to terminate gracefully. May be decreased in delete request. Value must be non-negative integer. The value zero indicates delete immediately. If this value is nil, the default grace period will be used instead. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. Defaults to 30 seconds.
description: Optional duration in seconds the pod needs to terminate gracefully. May be decreased in delete request. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). If this value is nil, the default grace period will be used instead. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. Defaults to 30 seconds.
format: int64
type: integer
tolerations:
@@ -3158,11 +3314,8 @@ spec:
x-kubernetes-int-or-string: true
type: object
ephemeral:
description: "Ephemeral represents a volume that is handled by a cluster storage driver (Alpha feature). The volume's lifecycle is tied to the pod that defines it - it will be created before the pod starts, and deleted when the pod is removed. \n Use this if: a) the volume is only needed while the pod runs, b) features of normal volumes like restoring from snapshot or capacity tracking are needed, c) the storage driver is specified through a storage class, and d) the storage driver supports dynamic volume provisioning through a PersistentVolumeClaim (see EphemeralVolumeSource for more information on the connection between this volume type and PersistentVolumeClaim). \n Use PersistentVolumeClaim or one of the vendor-specific APIs for volumes that persist for longer than the lifecycle of an individual pod. \n Use CSI for light-weight local ephemeral volumes if the CSI driver is meant to be used that way - see the documentation of the driver for more information. \n A pod can use both types of ephemeral volumes and persistent volumes at the same time."
description: "Ephemeral represents a volume that is handled by a cluster storage driver. The volume's lifecycle is tied to the pod that defines it - it will be created before the pod starts, and deleted when the pod is removed. \n Use this if: a) the volume is only needed while the pod runs, b) features of normal volumes like restoring from snapshot or capacity tracking are needed, c) the storage driver is specified through a storage class, and d) the storage driver supports dynamic volume provisioning through a PersistentVolumeClaim (see EphemeralVolumeSource for more information on the connection between this volume type and PersistentVolumeClaim). \n Use PersistentVolumeClaim or one of the vendor-specific APIs for volumes that persist for longer than the lifecycle of an individual pod. \n Use CSI for light-weight local ephemeral volumes if the CSI driver is meant to be used that way - see the documentation of the driver for more information. \n A pod can use both types of ephemeral volumes and persistent volumes at the same time. \n This is a beta feature and only available when the GenericEphemeralVolume feature gate is enabled."
properties:
readOnly:
description: Specifies a read-only configuration for the volume. Defaults to false (read/write).
type: boolean
volumeClaimTemplate:
description: "Will be used to create a stand-alone PVC to provision the volume. The pod in which this EphemeralVolumeSource is embedded will be the owner of the PVC, i.e. the PVC will be deleted together with the pod. The name of the PVC will be `<pod name>-<volume name>` where `<volume name>` is the name from the `PodSpec.Volumes` array entry. Pod validation will reject the pod if the concatenated name is not valid for a PVC (for example, too long). \n An existing PVC with that name that is not owned by the pod will *not* be used for the pod to avoid using an unrelated volume by mistake. Starting the pod is then blocked until the unrelated PVC is removed. If such a pre-created PVC is meant to be used by the pod, the PVC has to updated with an owner reference to the pod once the pod exists. Normally this should not be necessary, but it may be useful when manually reconstructing a broken cluster. \n This field is read-only and no changes will be made by Kubernetes to the PVC after it has been created. \n Required, must not be nil."
properties:
@@ -3178,7 +3331,7 @@ spec:
type: string
type: array
dataSource:
description: 'This field can be used to specify either: * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot - Beta) * An existing PVC (PersistentVolumeClaim) * An existing custom resource/object that implements data population (Alpha) In order to use VolumeSnapshot object types, the appropriate feature gate must be enabled (VolumeSnapshotDataSource or AnyVolumeDataSource) If the provisioner or an external controller can support the specified data source, it will create a new volume based on the contents of the specified data source. If the specified data source is not supported, the volume will not be created and the failure will be reported as an event. In the future, we plan to support more data source types and the behavior of the provisioner may change.'
description: 'This field can be used to specify either: * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot) * An existing PVC (PersistentVolumeClaim) * An existing custom resource that implements data population (Alpha) In order to use custom resource types that implement data population, the AnyVolumeDataSource feature gate must be enabled. If the provisioner or an external controller can support the specified data source, it will create a new volume based on the contents of the specified data source.'
properties:
apiGroup:
description: APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.
@@ -3203,7 +3356,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
@@ -3212,7 +3365,7 @@ spec:
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
selector:
@@ -3629,8 +3782,6 @@ spec:
type: object
type: object
type: array
required:
- sources
type: object
quobyte:
description: Quobyte represents a Quobyte mount on the host that shares a pod's lifetime
+25 -16
View File
@@ -1,10 +1,10 @@
module github.com/fission/fission
go 1.15
go 1.16
require (
contrib.go.opencensus.io/exporter/jaeger v0.2.1
github.com/Azure/azure-sdk-for-go v12.4.0-beta+incompatible
github.com/Azure/azure-sdk-for-go v32.5.0+incompatible
github.com/Azure/go-autorest/autorest v0.11.18 // indirect
github.com/Microsoft/go-winio v0.4.16 // indirect
github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5 // indirect
@@ -13,10 +13,8 @@ require (
github.com/blend/go-sdk v1.20210116.5 // indirect
github.com/bsm/sarama-cluster v2.1.15+incompatible
github.com/cenkalti/backoff v2.2.1+incompatible // indirect
github.com/cheekybits/is v0.0.0-20150225183255-68e9c0620927 // indirect
github.com/containerd/continuity v0.0.0-20201208142359-180525291bb7 // indirect
github.com/dchest/uniuri v0.0.0-20160212164326-8902c56451e9
github.com/dnaeon/go-vcr v1.1.0 // indirect
github.com/docker/go-connections v0.4.0 // indirect
github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815
github.com/dsnet/compress v0.0.1 // indirect
@@ -28,15 +26,14 @@ require (
github.com/ghodss/yaml v1.0.0
github.com/go-git/go-git/v5 v5.2.0
github.com/go-ini/ini v1.62.0 // indirect
github.com/go-openapi/spec v0.19.3
github.com/go-openapi/spec v0.19.5
github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0 // indirect
github.com/gorilla/mux v1.8.0
github.com/gotestyourself/gotestyourself v2.2.0+incompatible // indirect
github.com/graymeta/stow v0.0.0-20180719215413-7b5498c561bb
github.com/graymeta/stow v0.2.7
github.com/hashicorp/go-multierror v1.1.1
github.com/imdario/mergo v0.3.12
github.com/influxdata/influxdb v1.2.0
github.com/marstr/guid v1.1.0 // indirect
github.com/mholt/archiver v0.0.0-20180417220235-e4ef56d48eb0
github.com/minio/minio-go v6.0.14+incompatible
github.com/nats-io/nats-streaming-server v0.22.0
@@ -47,8 +44,8 @@ require (
github.com/opencontainers/runc v1.0.1 // indirect
github.com/ory/dockertest v3.3.5+incompatible
github.com/pkg/errors v0.9.1
github.com/prometheus/client_golang v1.7.1
github.com/prometheus/common v0.10.0
github.com/prometheus/client_golang v1.11.0
github.com/prometheus/common v0.26.0
github.com/robfig/cron v0.0.0-20180505203441-b41be1df6967
github.com/satori/go.uuid v1.2.0
github.com/spf13/cobra v1.2.1
@@ -57,13 +54,25 @@ require (
github.com/ulikunitz/xz v0.5.9 // indirect
github.com/wcharczuk/go-chart v2.0.1+incompatible
go.opencensus.io v0.23.0
go.uber.org/zap v1.18.1
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.23.0
go.opentelemetry.io/contrib/propagators/aws v0.23.0
go.opentelemetry.io/contrib/propagators/b3 v0.23.0
go.opentelemetry.io/contrib/propagators/jaeger v0.23.0
go.opentelemetry.io/contrib/propagators/ot v0.23.0
go.opentelemetry.io/otel v1.0.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.0.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.0.0
go.opentelemetry.io/otel/sdk v1.0.0
go.opentelemetry.io/otel/trace v1.0.0
go.uber.org/zap v1.19.1
golang.org/x/net v0.0.0-20210614182718-04defd469f4e
golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e // indirect
k8s.io/api v0.19.12
k8s.io/apiextensions-apiserver v0.19.12
k8s.io/apimachinery v0.19.12
k8s.io/client-go v0.19.12
google.golang.org/grpc v1.40.0
gotest.tools v2.2.0+incompatible // indirect
k8s.io/api v0.21.4
k8s.io/apiextensions-apiserver v0.21.4
k8s.io/apimachinery v0.21.4
k8s.io/client-go v0.21.4
k8s.io/klog v1.0.0
k8s.io/metrics v0.19.12
k8s.io/metrics v0.21.4
sigs.k8s.io/controller-runtime v0.9.7
)
+168 -188
View File
File diff suppressed because it is too large Load Diff
+14
View File
@@ -0,0 +1,14 @@
#!/bin/bash
set -e
#set -x
DIR=$(realpath $(dirname "$0"))/../
export GITHUB_TOKEN=$(cat ~/.github-token)
# ignore_tags lists tags which we dont want to include in the changelog
# and non standard releases
exclude_tags=$(tr '\n' ',' < hack/ignore_tags | sed 's/,$/\n/')
github_changelog_generator -u fission -p fission -t "${GITHUB_TOKEN}" \
--no-issues --exclude-labels "no-changelog" -o tmp_CHANGELOG.md \
--exclude-tags $exclude_tags
mv tmp_CHANGELOG.md "${DIR}"/CHANGELOG.md
+122
View File
@@ -0,0 +1,122 @@
#!/bin/bash
set -e
set -x
DIR=$(realpath $(dirname "$0"))/../
MANIFESTDIR=$(realpath "$DIR")/manifest
source $(realpath "${DIR}"/test/init_tools.sh)
doit() {
echo "! $*"
"$@"
}
check_charts_repo() {
local chartsrepo=$1
if [ ! -d "$chartsrepo" ]; then
echo "Error finding chart repo at $chartsrepo"
exit 1
fi
echo "check_charts_repo == PASSED"
}
update_chart_version() {
local version=$1
sed -i "s/^version.*/version\: ${version}/" charts/fission-core/Chart.yaml
sed -i "s/^version.*/version\: ${version}/" charts/fission-all/Chart.yaml
sed -i "s/appVersion.*/appVersion\: ${version}/" charts/fission-core/Chart.yaml
sed -i "s/appVersion.*/appVersion\: ${version}/" charts/fission-all/Chart.yaml
sed -i "s/\bimageTag:.*/imageTag\: ${version}/" charts/fission-core/values.yaml
sed -i "s/\bimageTag:.*/imageTag\: ${version}/" charts/fission-all/values.yaml
}
lint_charts() {
helm lint charts/fission-all charts/fission-core
if [ $? -ne 0 ]; then
echo "helm lint failed"
exit 1
fi
}
build_charts() {
mkdir -p "$MANIFESTDIR"/charts
pushd "$DIR"/charts
find . -iname *.~?~ | xargs -r rm
for c in fission-all fission-core; do
doit helm package -u $c/
mv ./*.tgz "$MANIFESTDIR"/charts/
done
popd
}
build_yamls() {
local version=$1
mkdir -p "${MANIFESTDIR}"/yamls
pushd "${DIR}"/charts
find . -iname *.~?~ | xargs -r rm
releaseName=fission-$(echo "${version}" | sed 's/\./-/g')
for c in fission-all fission-core; do
# fetch dependencies
pushd ${c}
doit helm dependency update
popd
echo "Release name", "$releaseName"
cmdprefix="helm template ${releaseName} ${c} --namespace fission --validate"
# for minikube and other environments that don't support LoadBalancer
command="$cmdprefix --set analytics=false,analyticsNonHelmInstall=true,serviceType=NodePort,routerServiceType=NodePort"
echo "$command"
$command >${c}-"${version}"-minikube.yaml
# for environments that support LoadBalancer
command="$cmdprefix --set analytics=false,analyticsNonHelmInstall=true"
echo "$command"
$command >${c}-"${version}".yaml
# for OpenShift
command="$cmdprefix --set analytics=false,analyticsNonHelmInstall=true,logger.enableSecurityContext=true,prometheus.enabled=false"
echo "$command"
$command >${c}-"${version}"-openshift.yaml
# copy yaml files to build directory
mv ./*.yaml "${MANIFESTDIR}"/yamls/
done
popd
}
update_github_charts_repo() {
local version=$1
local chartsrepo=$2
pushd "$chartsrepo"
cp "$MANIFESTDIR"/charts/fission-all-"${version}".tgz .
cp "$MANIFESTDIR"/charts/fission-core-"${version}".tgz .
./index.sh
popd
}
version=$1
if [ -z "$version" ]; then
echo "Release version not mentioned"
exit 1
fi
echo "Current version for release: $version"
chartsrepo="$DIR../fission-charts"
check_charts_repo "$chartsrepo"
# Build manifests and charts
lint_charts
update_chart_version "$version"
lint_charts
build_yamls "$version"
build_charts
update_github_charts_repo "$version" "$chartsrepo"
+33
View File
@@ -0,0 +1,33 @@
v1.14.1
v1.14.0
v1.13.1
v1.13.0
v1.12.0
v1.11.2
v1.11.1
v1.11.0
v1.10.0
v1.9.0
v1.8.0
v1.7.1
v1.7.0
1.7.0-rc.2
v1.7.0-rc.2
1.7.0-rc.1
v1.7.0-rc.1
v1.6.0
1.0-rc2
1.0-rc1
latest
0.4.0rc
0.3.0-rc
buildmgr-preview-20170922
buildmgr-preview-20170921
v0.2.1
v0.2.1-rc2
v0.2.1-rc
v0.2.0-20170901
nightly20170705
nightly20170621
alpha20170124
kubecon
Regular → Executable
+2 -13
View File
@@ -9,18 +9,7 @@ if [ -z $version ]; then
exit 1
fi
gittag=$version
prefix="v"
gopkgtag=${version/#/${prefix}}
if [[ ${version} == v* ]]; then # if version starts with "v", don't append prefix.
gopkgtag=${version}
fi
# tag the release
doit git tag $gittag
doit git tag -a $gopkgtag -m "Fission $gopkgtag"
doit git tag $version
# push tag
doit git push origin $gittag
doit git push origin $gopkgtag
doit git push origin $version
+27 -249
View File
@@ -4,24 +4,29 @@ set -e
#set -x
DIR=$(realpath $(dirname "$0"))/../
BUILDDIR=$(realpath "$DIR")/build
artifacts=()
source $(realpath "${DIR}"/test/init_tools.sh)
MANIFESTDIR=$(realpath "$DIR")/manifest
doit() {
echo "! $*"
"$@"
}
# Ensure we're on the master branch
check_branch() {
local version=$1
curr_branch=$(git rev-parse --abbrev-ref HEAD)
if [ "$curr_branch" != "release-${version}" ]; then
echo "Not on release-${version} branch."
check_commands() {
if ! command -v goreleaser >/dev/null; then
echo "Goreleaser CLI not found. Please get from https://goreleaser.com/install/"
exit 1
fi
echo "check_commands == PASSED"
}
# Ensure we're on the master branch
check_branch() {
curr_branch=$(git rev-parse --abbrev-ref HEAD)
if [ "$curr_branch" != "master" ]; then
echo "Not on master branch."
exit 1
fi
echo "check_branch == PASSED"
}
# Ensure working dir is clean
@@ -30,232 +35,15 @@ check_clean() {
echo "Unclean tree"
exit 1
fi
echo "check_clean == PASSED"
}
attach_github_release_cli() {
# cli
echo "Artifact for osx amd64 cli"
artifacts+=("$BUILDDIR/bin/fission-$version-darwin-amd64")
echo "Artifact for windows amd64 cli"
artifacts+=("$BUILDDIR/bin/fission-$version-windows-amd64.exe")
echo "Artifact for linux amd64 cli"
artifacts+=("$BUILDDIR/bin/fission-$version-linux-amd64")
echo "Artifact for linux arm cli"
artifacts+=("$BUILDDIR/bin/fission-$version-linux-arm")
echo "Artifact for linux arm64 cli"
artifacts+=("$BUILDDIR/bin/fission-$version-linux-arm64")
}
attach_github_release_charts() {
local version=$1
echo "fission-all chart"
artifacts+=("$BUILDDIR/charts/fission-all-$version.tgz")
echo "Fission-core chart"
artifacts+=("$BUILDDIR/charts/fission-core-$version.tgz")
}
attach_github_release_yamls() {
local version=$1
for c in fission-all fission-core; do
# YAML
artifacts+=("$BUILDDIR/yamls/${c}-${version}-minikube.yaml")
artifacts+=("$BUILDDIR/yamls/${c}-${version}.yaml")
artifacts+=("$BUILDDIR/yamls/${c}-${version}-openshift.yaml")
done
}
update_github_charts_repo() {
local version=$1
local chartsrepo=$2
pushd "$chartsrepo"
cp "$BUILDDIR"/charts/fission-all-"${version}".tgz .
cp "$BUILDDIR"/charts/fission-core-"${version}".tgz .
./index.sh
popd
}
gh_release() {
local version=$1
cp "${DIR}"/hack/notes.md relnotes.md
create_downloads_table ${version} >> relnotes.md
doit gh release create "$version" --draft --prerelease --title "$version" --notes-file $(realpath "${DIR}"/relnotes.md) --target "$gitcommit" "${artifacts[@]}"
}
generate_changelog() {
local version=$1
echo "# ${version}" >new_CHANGELOG.md
echo
echo "[Documentation](https://docs.fission.io/)" >>new_CHANGELOG.md
echo
# generate changelog from github
github_changelog_generator -u fission -p fission -t "${GITHUB_TOKEN}" --future-release "${version}" --no-issues -o tmp_CHANGELOG.md
sed -i '$ d' tmp_CHANGELOG.md
# concatenate two files
cat tmp_CHANGELOG.md >>new_CHANGELOG.md
mv new_CHANGELOG.md "${DIR}"/CHANGELOG.md
rm tmp_CHANGELOG.md
}
create_downloads_table() {
local release_tag=$1
local url_prefix="https://github.com/fission/fission/releases/download"
echo
echo
echo "#### Downloads for ${version}"
echo
echo
echo "filename | sha256 hash"
echo "-------- | -----------"
for file in ${artifacts[@]}; do
filename=${file##*/}
echo "[${filename}]($url_prefix/$release_tag/${filename}) | \`$(shasum -a 256 ${file} | cut -d' ' -f 1)\`"
done
echo
}
export -f create_downloads_table
release_environment_check() {
local version=$1
local chartsrepo=$2
check_branch "$version"
check_clean
check_github_token() {
if [ ! -f "$HOME"/.github-token ]; then
echo "Error finding github access token at ${HOME}/.github-token"
exit 1
fi
if [ ! -d "$chartsrepo" ]; then
echo "Error finding chart repo at $chartsrepo"
exit 1
fi
}
build_charts() {
local version=$1
mkdir -p "$BUILDDIR"/charts
pushd "$DIR"/charts
find . -iname *.~?~ | xargs -r rm
for c in fission-all fission-core; do
doit helm package -u $c/
mv ./*.tgz "$BUILDDIR"/charts/
done
popd
}
build_yamls() {
local version=$1
mkdir -p "${BUILDDIR}"/yamls
pushd "${DIR}"/charts
find . -iname *.~?~ | xargs -r rm
releaseName=fission-$(echo "${version}" | sed 's/\./-/g')
for c in fission-all fission-core; do
# fetch dependencies
pushd ${c}
doit helm dependency update
popd
echo "Release name", "$releaseName"
cmdprefix="helm template ${releaseName} ${c} --namespace fission --validate"
# for minikube and other environments that don't support LoadBalancer
command="$cmdprefix --set analytics=false,analyticsNonHelmInstall=true,serviceType=NodePort,routerServiceType=NodePort"
echo "$command"
$command >${c}-"${version}"-minikube.yaml
# for environments that support LoadBalancer
command="$cmdprefix --set analytics=false,analyticsNonHelmInstall=true"
echo "$command"
$command >${c}-"${version}".yaml
# for OpenShift
command="$cmdprefix --set analytics=false,analyticsNonHelmInstall=true,logger.enableSecurityContext=true,prometheus.enabled=false"
echo "$command"
$command >${c}-"${version}"-openshift.yaml
# copy yaml files to build directory
mv ./*.yaml "${BUILDDIR}"/yamls/
done
popd
}
build_all() {
local version=$1
if [ -z "$version" ]; then
echo "Version unspecified"
exit 1
fi
local date=$2
if [ -z "$date" ]; then
echo "Build date unspecified"
exit 1
fi
local gitcommit=$3
if [ -z "$gitcommit" ]; then
echo "Git commit unspecified"
exit 1
fi
if [ -e "$BUILDDIR" ]; then
echo "Removing existing build dir ($BUILDDIR)."
rm -rf "$BUILDDIR"
fi
mkdir -p "$BUILDDIR"
# generate swagger (OpenApi 2.0) doc before building bundle image
VERSION=$version TIMESTAMP=$date COMMITSHA=$gitcommit make generate-swagger-doc
# Build CLI for all platforms
VERSION=$version TIMESTAMP=$date COMMITSHA=$gitcommit make all-fission-cli
}
build_images() {
local version=$1
if [ -z "$version" ]; then
echo "Version unspecified"
exit 1
fi
local date=$2
if [ -z "$date" ]; then
echo "Build date unspecified"
exit 1
fi
local gitcommit=$3
if [ -z "$gitcommit" ]; then
echo "Git commit unspecified"
exit 1
fi
# Build and push all images
REPO=fission VERSION=$version TAG=latest TIMESTAMP=$date COMMITSHA=$gitcommit make all-images
REPO=fission VERSION=$version TAG=$version TIMESTAMP=$date COMMITSHA=$gitcommit make all-images
}
check_commands() {
if ! command -v hub >/dev/null; then
echo "Github CLI hub not found. Please get from https://cli.github.com/"
fi
echo "check_github_token == PASSED"
}
export GITHUB_TOKEN=$(cat ~/.github-token)
@@ -266,29 +54,19 @@ if [ -z "$version" ]; then
exit 1
fi
date=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
gitcommit=$(git rev-parse HEAD)
echo "Current version for release: $version"
chartsrepo=$2
if [ -z "$chartsrepo" ]; then
chartsrepo="$DIR../fission-charts"
fi
chartsrepo="$DIR../fission-charts"
# Prechecks
check_clean
check_commands
release_environment_check "$version" "$chartsrepo"
build_all "$version" "$date" "$gitcommit"
build_charts "$version"
build_yamls "$version"
check_branch
check_github_token
attach_github_release_cli "$version"
attach_github_release_charts "$version"
attach_github_release_yamls "$version"
update_github_charts_repo "$version" "$chartsrepo"
gh_release "$version"
generate_changelog "$version"
build_images "$version" "$date" "$gitcommit"
export GORELEASER_CURRENT_TAG=$version
echo "Release version $GORELEASER_CURRENT_TAG "
goreleaser release
echo "############ DONE #############"
echo "Congratulation, ${version} is ready to ship !!"
+26 -10
View File
@@ -18,6 +18,7 @@ package client
import (
"bytes"
"context"
"encoding/json"
"io/ioutil"
"net/http"
@@ -25,27 +26,43 @@ import (
"time"
"github.com/pkg/errors"
"go.opencensus.io/plugin/ochttp"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
"go.uber.org/zap"
"golang.org/x/net/context/ctxhttp"
builder "github.com/fission/fission/pkg/builder"
ferror "github.com/fission/fission/pkg/error"
otelUtils "github.com/fission/fission/pkg/utils/otel"
"github.com/fission/fission/pkg/utils/tracing"
)
type (
Client struct {
logger *zap.Logger
url string
logger *zap.Logger
url string
httpClient *http.Client
}
)
func MakeClient(logger *zap.Logger, builderUrl string) *Client {
var hc *http.Client
if tracing.TracingEnabled(logger) {
hc = &http.Client{Transport: &ochttp.Transport{}}
} else {
hc = &http.Client{Transport: otelhttp.NewTransport(http.DefaultTransport)}
}
return &Client{
logger: logger.Named("builder_client"),
url: strings.TrimSuffix(builderUrl, "/"),
logger: logger.Named("builder_client"),
url: strings.TrimSuffix(builderUrl, "/"),
httpClient: hc,
}
}
func (c *Client) Build(req *builder.PackageBuildRequest) (*builder.PackageBuildResponse, error) {
func (c *Client) Build(ctx context.Context, req *builder.PackageBuildRequest) (*builder.PackageBuildResponse, error) {
logger := otelUtils.LoggerWithTraceID(ctx, c.logger)
body, err := json.Marshal(req)
if err != nil {
return nil, errors.Wrap(err, "error marshaling json")
@@ -55,8 +72,7 @@ func (c *Client) Build(req *builder.PackageBuildRequest) (*builder.PackageBuildR
var resp *http.Response
for i := 0; i < maxRetries; i++ {
resp, err = http.Post(c.url, "application/json", bytes.NewReader(body))
resp, err = ctxhttp.Post(ctx, c.httpClient, c.url, "application/json", bytes.NewReader(body))
if err == nil {
if resp.StatusCode == 200 {
break
@@ -66,7 +82,7 @@ func (c *Client) Build(req *builder.PackageBuildRequest) (*builder.PackageBuildR
if i < maxRetries-1 {
time.Sleep(50 * time.Duration(2*i) * time.Millisecond)
c.logger.Error("error building package, retrying", zap.Error(err))
logger.Error("error building package, retrying", zap.Error(err))
continue
}
@@ -77,14 +93,14 @@ func (c *Client) Build(req *builder.PackageBuildRequest) (*builder.PackageBuildR
rBody, err := ioutil.ReadAll(resp.Body)
if err != nil {
c.logger.Error("error reading resp body", zap.Error(err))
logger.Error("error reading resp body", zap.Error(err))
return nil, err
}
pkgBuildResp := builder.PackageBuildResponse{}
err = json.Unmarshal(rBody, &pkgBuildResp)
if err != nil {
c.logger.Error("error parsing resp body", zap.Error(err))
logger.Error("error parsing resp body", zap.Error(err))
return nil, err
}
+2 -2
View File
@@ -50,8 +50,8 @@ func Start(logger *zap.Logger, storageSvcUrl string, envBuilderNamespace string)
envWatcher := makeEnvironmentWatcher(bmLogger, fissionClient, kubernetesClient, fetcherConfig, envBuilderNamespace)
go envWatcher.watchEnvironments()
k8sInformerFactory := k8sInformers.NewSharedInformerFactory(kubernetesClient, time.Second*30)
informerFactory := genInformer.NewSharedInformerFactory(fissionClient, 60*time.Minute)
k8sInformerFactory := k8sInformers.NewSharedInformerFactory(kubernetesClient, time.Minute*30)
informerFactory := genInformer.NewSharedInformerFactory(fissionClient, time.Minute*30)
podInformer := k8sInformerFactory.Core().V1().Pods().Informer()
pkgInformer := informerFactory.Core().V1().Packages().Informer()
pkgWatcher := makePackageWatcher(bmLogger, fissionClient,
+2 -2
View File
@@ -47,7 +47,7 @@ import (
func buildPackage(ctx context.Context, logger *zap.Logger, fissionClient *crd.FissionClient, envBuilderNamespace string,
storageSvcUrl string, pkg *fv1.Package) (uploadResp *fetcher.ArchiveUploadResponse, buildLogs string, err error) {
env, err := fissionClient.CoreV1().Environments(pkg.Spec.Environment.Namespace).Get(context.TODO(), pkg.Spec.Environment.Name, metav1.GetOptions{})
env, err := fissionClient.CoreV1().Environments(pkg.Spec.Environment.Namespace).Get(ctx, pkg.Spec.Environment.Name, metav1.GetOptions{})
if err != nil {
e := "error getting environment CRD info"
logger.Error(e, zap.Error(err))
@@ -88,7 +88,7 @@ func buildPackage(ctx context.Context, logger *zap.Logger, fissionClient *crd.Fi
logger.Info("started building with source package", zap.String("source_package", srcPkgFilename))
// send build request to builder
buildResp, err := builderC.Build(pkgBuildReq)
buildResp, err := builderC.Build(ctx, pkgBuildReq)
if err != nil {
e := fmt.Sprintf("Error building deployment package: %v", err)
var buildLogs string
+7 -7
View File
@@ -73,7 +73,7 @@ func makePackageWatcher(logger *zap.Logger, fissionClient *crd.FissionClient, k8
// 5. Update package resource in package ref of functions that share the same package
// 6. Update package status to succeed state
// *. Update package status to failed state,if any one of steps above failed/time out
func (pkgw *packageWatcher) build(srcpkg *fv1.Package) {
func (pkgw *packageWatcher) build(ctx context.Context, srcpkg *fv1.Package) {
// Ignore duplicate build requests
key := fmt.Sprintf("%v-%v", srcpkg.ObjectMeta.Name, srcpkg.ObjectMeta.ResourceVersion)
_, err := pkgw.buildCache.Set(key, srcpkg)
@@ -95,7 +95,7 @@ func (pkgw *packageWatcher) build(srcpkg *fv1.Package) {
return
}
env, err := pkgw.fissionClient.CoreV1().Environments(pkg.Spec.Environment.Namespace).Get(context.TODO(), pkg.Spec.Environment.Name, metav1.GetOptions{})
env, err := pkgw.fissionClient.CoreV1().Environments(pkg.Spec.Environment.Namespace).Get(ctx, pkg.Spec.Environment.Name, metav1.GetOptions{})
if k8serrors.IsNotFound(err) {
e := "environment does not exist"
pkgw.logger.Error(e, zap.String("environment", pkg.Spec.Environment.Name))
@@ -167,7 +167,7 @@ func (pkgw *packageWatcher) build(srcpkg *fv1.Package) {
// Add the package getter rolebinding to builder sa
// we continue here if role binding was not setup successfully. this is because without this, the fetcher wont be able to fetch the source pkg into the container and
// the build will fail eventually
err := utils.SetupRoleBinding(pkgw.logger, pkgw.k8sClient, fv1.PackageGetterRB, pkg.ObjectMeta.Namespace, fv1.PackageGetterCR, fv1.ClusterRole, fv1.FissionBuilderSA, builderNs)
err := utils.SetupRoleBinding(ctx, pkgw.logger, pkgw.k8sClient, fv1.PackageGetterRB, pkg.ObjectMeta.Namespace, fv1.PackageGetterCR, fv1.ClusterRole, fv1.FissionBuilderSA, builderNs)
if err != nil {
pkgw.logger.Error("error setting up role binding for package",
zap.Error(err),
@@ -181,7 +181,6 @@ func (pkgw *packageWatcher) build(srcpkg *fv1.Package) {
zap.String("package", fmt.Sprintf("%s.%s", pkg.ObjectMeta.Name, pkg.ObjectMeta.Namespace)))
}
ctx := context.Background()
uploadResp, buildLogs, err := buildPackage(ctx, pkgw.logger, pkgw.fissionClient, builderNs, pkgw.storageSvcUrl, pkg)
if err != nil {
pkgw.logger.Error("error building package", zap.Error(err), zap.String("package_name", pkg.ObjectMeta.Name))
@@ -200,7 +199,7 @@ func (pkgw *packageWatcher) build(srcpkg *fv1.Package) {
pkgw.logger.Info("starting package info update", zap.String("package_name", pkg.ObjectMeta.Name))
fnList, err := pkgw.fissionClient.CoreV1().
Functions(metav1.NamespaceAll).List(context.TODO(), metav1.ListOptions{})
Functions(metav1.NamespaceAll).List(ctx, metav1.ListOptions{})
if err != nil {
e := "error getting function list"
pkgw.logger.Error(e, zap.Error(err))
@@ -224,7 +223,7 @@ func (pkgw *packageWatcher) build(srcpkg *fv1.Package) {
fn.Spec.Package.PackageRef.ResourceVersion != pkg.ObjectMeta.ResourceVersion {
fn.Spec.Package.PackageRef.ResourceVersion = pkg.ObjectMeta.ResourceVersion
// update CRD
_, err = pkgw.fissionClient.CoreV1().Functions(fn.ObjectMeta.Namespace).Update(context.TODO(), &fn, metav1.UpdateOptions{})
_, err = pkgw.fissionClient.CoreV1().Functions(fn.ObjectMeta.Namespace).Update(ctx, &fn, metav1.UpdateOptions{})
if err != nil {
e := "error updating function package resource version"
pkgw.logger.Error(e, zap.Error(err))
@@ -296,7 +295,8 @@ func (pkgw *packageWatcher) packageInformerHandler() k8sCache.ResourceEventHandl
}
// Only build pending state packages.
if pkg.Status.BuildStatus == fv1.BuildStatusPending {
go pkgw.build(pkg)
ctx := context.Background()
go pkgw.build(ctx, pkg)
}
}
return k8sCache.ResourceEventHandlerFuncs{
+1 -1
View File
@@ -93,7 +93,7 @@ func MakeCanaryConfigMgr(logger *zap.Logger, fissionClient *crd.FissionClient, k
canaryCfgCancelFuncMap: makecanaryConfigCancelFuncMap(),
}
informerFactory := genInformer.NewSharedInformerFactory(fissionClient, time.Second*30)
informerFactory := genInformer.NewSharedInformerFactory(fissionClient, time.Minute*30)
informer := informerFactory.Core().V1().CanaryConfigs().Informer()
configMgr.canaryConfigInformer = &informer
configMgr.CanaryConfigEventHandlers()
+11 -2
View File
@@ -24,6 +24,7 @@ import (
"strings"
"github.com/gorilla/mux"
"go.opencensus.io/plugin/ochttp"
"go.uber.org/zap"
apiv1 "k8s.io/api/core/v1"
kerrors "k8s.io/apimachinery/pkg/api/errors"
@@ -34,6 +35,7 @@ import (
ferror "github.com/fission/fission/pkg/error"
"github.com/fission/fission/pkg/fission-cli/logdb"
"github.com/fission/fission/pkg/info"
"github.com/fission/fission/pkg/utils/otel"
)
var podNamespace string
@@ -263,9 +265,16 @@ func (api *API) GetHandler() http.Handler {
return r
}
func (api *API) Serve(port int) {
func (api *API) Serve(port int, openTracingEnabled bool) {
address := fmt.Sprintf(":%v", port)
api.logger.Info("server started", zap.Int("port", port))
err := http.ListenAndServe(address, api.GetHandler())
var handler http.Handler
if openTracingEnabled {
handler = &ochttp.Handler{Handler: api.GetHandler()}
} else {
handler = otel.GetHandlerWithOTEL(api.GetHandler(), "fission-controller", otel.UrlsToIgnore("/healthz"))
}
err := http.ListenAndServe(address, handler)
api.logger.Fatal("done listening", zap.Error(err))
}
+1 -1
View File
@@ -374,7 +374,7 @@ func TestMain(m *testing.M) {
panicIf(err)
go Start(logger, 8888, true)
go Start(logger, 8888, true, true)
time.Sleep(5 * time.Second)
+2 -2
View File
@@ -24,7 +24,7 @@ import (
"github.com/fission/fission/pkg/crd"
)
func Start(logger *zap.Logger, port int, unitTestFlag bool) {
func Start(logger *zap.Logger, port int, unitTestFlag bool, openTracingEnabled bool) {
cLogger := logger.Named("controller")
fc, kc, apiExtClient, _, err := crd.MakeFissionClient()
@@ -53,5 +53,5 @@ func Start(logger *zap.Logger, port int, unitTestFlag bool) {
if err != nil {
cLogger.Fatal("failed to start controller", zap.Error(err))
}
api.Serve(port)
api.Serve(port, openTracingEnabled)
}
+7
View File
@@ -31,3 +31,10 @@ import (
func CacheKey(metadata *metav1.ObjectMeta) string {
return fmt.Sprintf("%v_%v", metadata.UID, metadata.ResourceVersion)
}
// CacheKeyForUID create a key that uniquely identifies the
// of the object. Since resourceVersion changes on every update and
// UIDs are unique, we don't use resource version here
func CacheKeyUID(metadata *metav1.ObjectMeta) string {
return fmt.Sprintf("%v", metadata.UID)
}
+39 -24
View File
@@ -17,6 +17,7 @@ limitations under the License.
package executor
import (
"context"
"encoding/json"
"fmt"
"html"
@@ -33,9 +34,11 @@ import (
fv1 "github.com/fission/fission/pkg/apis/core/v1"
ferror "github.com/fission/fission/pkg/error"
"github.com/fission/fission/pkg/executor/client"
otelUtils "github.com/fission/fission/pkg/utils/otel"
)
func (executor *Executor) getServiceForFunctionAPI(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
body, err := ioutil.ReadAll(r.Body)
if err != nil {
http.Error(w, "Failed to read request", http.StatusInternalServerError)
@@ -52,9 +55,10 @@ func (executor *Executor) getServiceForFunctionAPI(w http.ResponseWriter, r *htt
t := fn.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType
et := executor.executorTypes[t]
logger := otelUtils.LoggerWithTraceID(ctx, executor.logger)
// Check function -> svc cache
executor.logger.Debug("checking for cached function service",
logger.Debug("checking for cached function service",
zap.String("function_name", fn.ObjectMeta.Name),
zap.String("function_namespace", fn.ObjectMeta.Namespace))
if t == fv1.ExecutorTypePoolmgr && !fn.Spec.OnceOnly {
@@ -66,51 +70,51 @@ func (executor *Executor) getServiceForFunctionAPI(w http.ResponseWriter, r *htt
if requestsPerpod == 0 {
requestsPerpod = 1
}
fsvc, active, err := et.GetFuncSvcFromPoolCache(fn, requestsPerpod)
fsvc, active, err := et.GetFuncSvcFromPoolCache(ctx, fn, requestsPerpod)
// check if its a cache hit (check if there is already specialized function pod that can serve another request)
if err == nil {
// if a pod is already serving request then it already exists else validated
executor.logger.Debug("from cache", zap.Int("active", active))
if active > 1 || et.IsValid(fsvc) {
logger.Debug("from cache", zap.Int("active", active))
if active > 1 || et.IsValid(ctx, fsvc) {
// Cached, return svc address
executor.logger.Debug("served from cache", zap.String("name", fsvc.Name), zap.String("address", fsvc.Address))
logger.Debug("served from cache", zap.String("name", fsvc.Name), zap.String("address", fsvc.Address))
executor.writeResponse(w, fsvc.Address, fn.ObjectMeta.Name)
return
}
executor.logger.Debug("deleting cache entry for invalid address",
logger.Debug("deleting cache entry for invalid address",
zap.String("function_name", fn.ObjectMeta.Name),
zap.String("function_namespace", fn.ObjectMeta.Namespace),
zap.String("address", fsvc.Address))
et.DeleteFuncSvcFromCache(fsvc)
et.DeleteFuncSvcFromCache(ctx, fsvc)
active--
}
if active >= concurrency {
errMsg := fmt.Sprintf("max concurrency reached for %v. All %v instance are active", fn.ObjectMeta.Name, concurrency)
executor.logger.Error("error occurred", zap.String("error", errMsg))
logger.Error("error occurred", zap.String("error", errMsg))
http.Error(w, html.EscapeString(errMsg), http.StatusTooManyRequests)
return
}
} else if t == fv1.ExecutorTypeNewdeploy || t == fv1.ExecutorTypeContainer {
fsvc, err := et.GetFuncSvcFromCache(fn)
fsvc, err := et.GetFuncSvcFromCache(ctx, fn)
if err == nil {
if et.IsValid(fsvc) {
if et.IsValid(ctx, fsvc) {
// Cached, return svc address
executor.writeResponse(w, fsvc.Address, fn.ObjectMeta.Name)
return
}
executor.logger.Debug("deleting cache entry for invalid address",
logger.Debug("deleting cache entry for invalid address",
zap.String("function_name", fn.ObjectMeta.Name),
zap.String("function_namespace", fn.ObjectMeta.Namespace),
zap.String("address", fsvc.Address))
et.DeleteFuncSvcFromCache(fsvc)
et.DeleteFuncSvcFromCache(ctx, fsvc)
}
}
serviceName, err := executor.getServiceForFunction(fn)
serviceName, err := executor.getServiceForFunction(ctx, fn)
if err != nil {
code, msg := ferror.GetHTTPError(err)
executor.logger.Error("error getting service for function",
logger.Error("error getting service for function",
zap.Error(err),
zap.String("function", fn.ObjectMeta.Name),
zap.String("fission_http_error", msg))
@@ -140,9 +144,10 @@ func (executor *Executor) writeResponse(w http.ResponseWriter, serviceName strin
// stale addresses are not returned to the router.
// To make it optimal, plan is to add an eager cache invalidator function that watches for pod deletion events and
// invalidates the cache entry if the pod address was cached.
func (executor *Executor) getServiceForFunction(fn *fv1.Function) (string, error) {
func (executor *Executor) getServiceForFunction(ctx context.Context, fn *fv1.Function) (string, error) {
respChan := make(chan *createFuncServiceResponse)
executor.requestChan <- &createFuncServiceRequest{
context: ctx,
function: fn,
respChan: respChan,
}
@@ -162,9 +167,12 @@ func (executor *Executor) tapService(w http.ResponseWriter, r *http.Request) {
// find funcSvc and update its atime
func (executor *Executor) tapServices(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
logger := otelUtils.LoggerWithTraceID(ctx, executor.logger)
body, err := ioutil.ReadAll(r.Body)
if err != nil {
executor.logger.Error("failed to read tap service request", zap.Error(err))
logger.Error("failed to read tap service request", zap.Error(err))
http.Error(w, "Failed to read request", http.StatusInternalServerError)
return
}
@@ -172,7 +180,7 @@ func (executor *Executor) tapServices(w http.ResponseWriter, r *http.Request) {
tapSvcReqs := []client.TapServiceRequest{}
err = json.Unmarshal(body, &tapSvcReqs)
if err != nil {
executor.logger.Error("failed to decode tap service request",
logger.Error("failed to decode tap service request",
zap.Error(err),
zap.String("request-payload", string(body)))
http.Error(w, "Failed to decode tap service request", http.StatusBadRequest)
@@ -191,7 +199,7 @@ func (executor *Executor) tapServices(w http.ResponseWriter, r *http.Request) {
continue
}
err = et.TapService(svcHost)
err = et.TapService(ctx, svcHost)
if err != nil {
errs = multierror.Append(errs,
errors.Wrapf(err, "'%v' failed to tap function '%v' in '%v' with service url '%v'",
@@ -200,7 +208,7 @@ func (executor *Executor) tapServices(w http.ResponseWriter, r *http.Request) {
}
if errs.ErrorOrNil() != nil {
executor.logger.Error("error tapping function service", zap.Error(errs))
logger.Error("error tapping function service", zap.Error(errs))
http.Error(w, "Not found", http.StatusNotFound)
return
}
@@ -213,6 +221,7 @@ func (executor *Executor) healthHandler(w http.ResponseWriter, r *http.Request)
}
func (executor *Executor) unTapService(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
body, err := ioutil.ReadAll(r.Body)
if err != nil {
http.Error(w, "Failed to read request", http.StatusInternalServerError)
@@ -234,7 +243,7 @@ func (executor *Executor) unTapService(w http.ResponseWriter, r *http.Request) {
et := executor.executorTypes[t]
et.UnTapService(key, tapSvcReq.ServiceURL)
et.UnTapService(ctx, key, tapSvcReq.ServiceURL)
w.WriteHeader(http.StatusOK)
}
@@ -251,11 +260,17 @@ func (executor *Executor) GetHandler() http.Handler {
}
// Serve starts an HTTP server.
func (executor *Executor) Serve(port int) {
func (executor *Executor) Serve(port int, openTracingEnabled bool) {
executor.logger.Info("starting executor API", zap.Int("port", port))
address := fmt.Sprintf(":%v", port)
err := http.ListenAndServe(address, &ochttp.Handler{
Handler: executor.GetHandler(),
})
var handler http.Handler
if openTracingEnabled {
handler = &ochttp.Handler{Handler: executor.GetHandler()}
} else {
handler = otelUtils.GetHandlerWithOTEL(executor.GetHandler(), "fission-executor", otelUtils.UrlsToIgnore("/healthz"))
}
err := http.ListenAndServe(address, handler)
executor.logger.Fatal("done listening", zap.Error(err))
}
+13 -7
View File
@@ -28,12 +28,14 @@ import (
"github.com/pkg/errors"
"go.opencensus.io/plugin/ochttp"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
"go.uber.org/zap"
"golang.org/x/net/context/ctxhttp"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
fv1 "github.com/fission/fission/pkg/apis/core/v1"
ferror "github.com/fission/fission/pkg/error"
"github.com/fission/fission/pkg/utils/tracing"
)
type (
@@ -56,14 +58,19 @@ type (
// MakeClient initializes and returns a Client instance.
func MakeClient(logger *zap.Logger, executorURL string) *Client {
var hc *http.Client
if tracing.TracingEnabled(logger) {
hc = &http.Client{Transport: &ochttp.Transport{}}
} else {
hc = &http.Client{Transport: otelhttp.NewTransport(http.DefaultTransport)}
}
c := &Client{
logger: logger.Named("executor_client"),
executorURL: strings.TrimSuffix(executorURL, "/"),
tappedByURL: make(map[string]TapServiceRequest),
requestChan: make(chan TapServiceRequest, 100),
httpClient: &http.Client{
Transport: &ochttp.Transport{},
},
httpClient: hc,
}
go c.service()
return c
@@ -143,8 +150,7 @@ func (c *Client) service() {
svcReqs = append(svcReqs, req)
}
c.logger.Debug("tapped services in batch", zap.Int("service_count", len(urls)))
err := c._tapService(svcReqs)
err := c._tapService(context.TODO(), svcReqs)
if err != nil {
c.logger.Error("error tapping function service address", zap.Error(err))
}
@@ -169,7 +175,7 @@ func (c *Client) TapService(fnMeta metav1.ObjectMeta, executorType fv1.ExecutorT
}
}
func (c *Client) _tapService(tapSvcReqs []TapServiceRequest) error {
func (c *Client) _tapService(ctx context.Context, tapSvcReqs []TapServiceRequest) error {
executorURL := c.executorURL + "/v2/tapServices"
body, err := json.Marshal(tapSvcReqs)
@@ -177,7 +183,7 @@ func (c *Client) _tapService(tapSvcReqs []TapServiceRequest) error {
return err
}
resp, err := http.Post(executorURL, "application/json", bytes.NewReader(body))
resp, err := ctxhttp.Post(ctx, c.httpClient, executorURL, "application/json", bytes.NewReader(body))
if err != nil {
return err
}
+5 -5
View File
@@ -29,8 +29,8 @@ import (
"github.com/fission/fission/pkg/executor/executortype"
)
func getConfigmapRelatedFuncs(logger *zap.Logger, m *metav1.ObjectMeta, fissionClient *crd.FissionClient) ([]fv1.Function, error) {
funcList, err := fissionClient.CoreV1().Functions(metav1.NamespaceAll).List(context.TODO(), metav1.ListOptions{})
func getConfigmapRelatedFuncs(ctx context.Context, logger *zap.Logger, m *metav1.ObjectMeta, fissionClient *crd.FissionClient) ([]fv1.Function, error) {
funcList, err := fissionClient.CoreV1().Functions(metav1.NamespaceAll).List(ctx, metav1.ListOptions{})
if err != nil {
return nil, err
}
@@ -47,7 +47,7 @@ func getConfigmapRelatedFuncs(logger *zap.Logger, m *metav1.ObjectMeta, fissionC
return relatedFunctions, nil
}
func ConfigMapEventHandlers(logger *zap.Logger, fissionClient *crd.FissionClient,
func ConfigMapEventHandlers(ctx context.Context, logger *zap.Logger, fissionClient *crd.FissionClient,
kubernetesClient *kubernetes.Clientset, types map[fv1.ExecutorType]executortype.ExecutorType) k8sCache.ResourceEventHandlerFuncs {
return k8sCache.ResourceEventHandlerFuncs{
@@ -62,11 +62,11 @@ func ConfigMapEventHandlers(logger *zap.Logger, fissionClient *crd.FissionClient
zap.String("configmap_name", newCm.ObjectMeta.Name),
zap.String("configmap_namespace", newCm.ObjectMeta.Namespace))
}
funcs, err := getConfigmapRelatedFuncs(logger, &newCm.ObjectMeta, fissionClient)
funcs, err := getConfigmapRelatedFuncs(ctx, logger, &newCm.ObjectMeta, fissionClient)
if err != nil {
logger.Error("Failed to get functions related to configmap", zap.String("configmap_name", newCm.ObjectMeta.Name), zap.String("configmap_namespace", newCm.ObjectMeta.Namespace))
}
refreshPods(logger, funcs, types)
refreshPods(ctx, logger, funcs, types)
}
},
}
+12 -15
View File
@@ -17,10 +17,12 @@ limitations under the License.
package cms
import (
"context"
"github.com/pkg/errors"
"go.uber.org/zap"
informerv1 "k8s.io/client-go/informers/core/v1"
"k8s.io/client-go/kubernetes"
k8sCache "k8s.io/client-go/tools/cache"
fv1 "github.com/fission/fission/pkg/apis/core/v1"
"github.com/fission/fission/pkg/crd"
@@ -32,38 +34,33 @@ type (
ConfigSecretController struct {
logger *zap.Logger
configmapInformer *k8sCache.SharedIndexInformer
secretInformer *k8sCache.SharedIndexInformer
fissionClient *crd.FissionClient
}
)
// MakeConfigSecretController makes a controller for configmaps and secrets which changes related functions
func MakeConfigSecretController(logger *zap.Logger, fissionClient *crd.FissionClient,
func MakeConfigSecretController(ctx context.Context, logger *zap.Logger, fissionClient *crd.FissionClient,
kubernetesClient *kubernetes.Clientset, types map[fv1.ExecutorType]executortype.ExecutorType,
configmapInformer *k8sCache.SharedIndexInformer,
secretInformer *k8sCache.SharedIndexInformer) *ConfigSecretController {
configmapInformer informerv1.ConfigMapInformer,
secretInformer informerv1.SecretInformer) *ConfigSecretController {
logger.Debug("Creating ConfigMap & Secret Controller")
cmsController := &ConfigSecretController{
logger: logger,
configmapInformer: configmapInformer,
secretInformer: secretInformer,
fissionClient: fissionClient,
logger: logger,
fissionClient: fissionClient,
}
(*configmapInformer).AddEventHandler(ConfigMapEventHandlers(logger, fissionClient, kubernetesClient, types))
(*secretInformer).AddEventHandler(SecretEventHandlers(logger, fissionClient, kubernetesClient, types))
configmapInformer.Informer().AddEventHandler(ConfigMapEventHandlers(ctx, logger, fissionClient, kubernetesClient, types))
secretInformer.Informer().AddEventHandler(SecretEventHandlers(ctx, logger, fissionClient, kubernetesClient, types))
return cmsController
}
func refreshPods(logger *zap.Logger, funcs []fv1.Function, types map[fv1.ExecutorType]executortype.ExecutorType) {
func refreshPods(ctx context.Context, logger *zap.Logger, funcs []fv1.Function, types map[fv1.ExecutorType]executortype.ExecutorType) {
for _, f := range funcs {
var err error
et, exists := types[f.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType]
if exists {
err = et.RefreshFuncPods(logger, f)
err = et.RefreshFuncPods(ctx, logger, f)
} else {
err = errors.Errorf("Unknown executor type '%v'", f.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType)
}
+5 -5
View File
@@ -29,8 +29,8 @@ import (
"github.com/fission/fission/pkg/executor/executortype"
)
func getSecretRelatedFuncs(logger *zap.Logger, m *metav1.ObjectMeta, fissionClient *crd.FissionClient) ([]fv1.Function, error) {
funcList, err := fissionClient.CoreV1().Functions(metav1.NamespaceAll).List(context.TODO(), metav1.ListOptions{})
func getSecretRelatedFuncs(ctx context.Context, logger *zap.Logger, m *metav1.ObjectMeta, fissionClient *crd.FissionClient) ([]fv1.Function, error) {
funcList, err := fissionClient.CoreV1().Functions(metav1.NamespaceAll).List(ctx, metav1.ListOptions{})
if err != nil {
return nil, err
}
@@ -47,7 +47,7 @@ func getSecretRelatedFuncs(logger *zap.Logger, m *metav1.ObjectMeta, fissionClie
return relatedFunctions, nil
}
func SecretEventHandlers(logger *zap.Logger, fissionClient *crd.FissionClient,
func SecretEventHandlers(ctx context.Context, logger *zap.Logger, fissionClient *crd.FissionClient,
kubernetesClient *kubernetes.Clientset, types map[fv1.ExecutorType]executortype.ExecutorType) k8sCache.ResourceEventHandlerFuncs {
return k8sCache.ResourceEventHandlerFuncs{
AddFunc: func(obj interface{}) {},
@@ -61,11 +61,11 @@ func SecretEventHandlers(logger *zap.Logger, fissionClient *crd.FissionClient,
zap.String("configmap_name", newS.ObjectMeta.Name),
zap.String("configmap_namespace", newS.ObjectMeta.Namespace))
}
funcs, err := getSecretRelatedFuncs(logger, &newS.ObjectMeta, fissionClient)
funcs, err := getSecretRelatedFuncs(ctx, logger, &newS.ObjectMeta, fissionClient)
if err != nil {
logger.Error("Failed to get functions related to secret", zap.String("secret_name", newS.ObjectMeta.Name), zap.String("secret_namespace", newS.ObjectMeta.Namespace))
}
refreshPods(logger, funcs, types)
refreshPods(ctx, logger, funcs, types)
}
},
}
+68 -33
View File
@@ -45,6 +45,8 @@ import (
"github.com/fission/fission/pkg/executor/util"
fetcherConfig "github.com/fission/fission/pkg/fetcher/config"
genInformer "github.com/fission/fission/pkg/generated/informers/externalversions"
"github.com/fission/fission/pkg/utils"
otelUtils "github.com/fission/fission/pkg/utils/otel"
)
type (
@@ -62,6 +64,7 @@ type (
}
createFuncServiceRequest struct {
context context.Context
function *fv1.Function
respChan chan *createFuncServiceResponse
}
@@ -124,7 +127,7 @@ func (executor *Executor) serveCreateFuncServices() {
specializationTimeout = fv1.DefaultSpecializationTimeOut
}
fnSpecializationTimeoutContext, cancel := context.WithTimeout(context.Background(),
fnSpecializationTimeoutContext, cancel := context.WithTimeout(req.context,
time.Duration(specializationTimeout+buffer)*time.Second)
defer cancel()
@@ -167,7 +170,7 @@ func (executor *Executor) serveCreateFuncServices() {
specializationTimeout = fv1.DefaultSpecializationTimeOut
}
fnSpecializationTimeoutContext, cancel := context.WithTimeout(context.Background(),
fnSpecializationTimeoutContext, cancel := context.WithTimeout(req.context,
time.Duration(specializationTimeout+buffer)*time.Second)
defer cancel()
@@ -195,7 +198,7 @@ func (executor *Executor) serveCreateFuncServices() {
wg.Wait()
// get the function service from the cache
fsvc, err := executor.getFunctionServiceFromCache(req.function)
fsvc, err := executor.getFunctionServiceFromCache(req.context, req.function)
// fsCache return error when the entry does not exist/expire.
// It normally happened if there are multiple requests are
@@ -212,7 +215,9 @@ func (executor *Executor) serveCreateFuncServices() {
}
func (executor *Executor) createServiceForFunction(ctx context.Context, fn *fv1.Function) (*fscache.FuncSvc, error) {
executor.logger.Debug("no cached function service found, creating one",
logger := otelUtils.LoggerWithTraceID(ctx, executor.logger)
otelUtils.SpanTrackEvent(ctx, "createServiceForFunction", otelUtils.GetAttributesForFunction(fn)...)
logger.Debug("no cached function service found, creating one",
zap.String("function_name", fn.ObjectMeta.Name),
zap.String("function_namespace", fn.ObjectMeta.Namespace))
@@ -225,7 +230,7 @@ func (executor *Executor) createServiceForFunction(ctx context.Context, fn *fv1.
fsvc, fsvcErr := e.GetFuncSvc(ctx, fn)
if fsvcErr != nil {
e := "error creating service for function"
executor.logger.Error(e,
logger.Error(e,
zap.Error(fsvcErr),
zap.String("function_name", fn.ObjectMeta.Name),
zap.String("function_namespace", fn.ObjectMeta.Namespace))
@@ -235,13 +240,14 @@ func (executor *Executor) createServiceForFunction(ctx context.Context, fn *fv1.
return fsvc, fsvcErr
}
func (executor *Executor) getFunctionServiceFromCache(fn *fv1.Function) (*fscache.FuncSvc, error) {
func (executor *Executor) getFunctionServiceFromCache(ctx context.Context, fn *fv1.Function) (*fscache.FuncSvc, error) {
otelUtils.SpanTrackEvent(ctx, "getFunctionServiceFromCache", otelUtils.GetAttributesForFunction(fn)...)
t := fn.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType
e, ok := executor.executorTypes[t]
if !ok {
return nil, errors.Errorf("Unknown executor type '%v'", t)
}
return e.GetFuncSvcFromCache(fn)
return e.GetFuncSvcFromCache(ctx, fn)
}
func serveMetric(logger *zap.Logger) {
@@ -255,7 +261,7 @@ func serveMetric(logger *zap.Logger) {
// StartExecutor Starts executor and the executor components such as Poolmgr,
// deploymgr and potential future executor types
func StartExecutor(logger *zap.Logger, functionNamespace string, envBuilderNamespace string, port int) error {
func StartExecutor(logger *zap.Logger, functionNamespace string, envBuilderNamespace string, port int, openTracingEnabled bool) error {
fissionClient, kubernetesClient, _, metricsClient, err := crd.MakeFissionClient()
if err != nil {
return errors.Wrap(err, "failed to get kubernetes client")
@@ -275,43 +281,63 @@ func StartExecutor(logger *zap.Logger, functionNamespace string, envBuilderNames
logger.Info("Starting executor", zap.String("instanceID", executorInstanceID))
informerFactory := genInformer.NewSharedInformerFactory(fissionClient, time.Second*30)
funcInformer := informerFactory.Core().V1().Functions().Informer()
pkgInformer := informerFactory.Core().V1().Packages().Informer()
envInformer := informerFactory.Core().V1().Environments().Informer()
informerFactory := genInformer.NewSharedInformerFactory(fissionClient, time.Minute*30)
funcInformer := informerFactory.Core().V1().Functions()
pkgInformer := informerFactory.Core().V1().Packages()
envInformer := informerFactory.Core().V1().Environments()
gpmInformerFactory, err := utils.GetInformerFactoryByExecutor(kubernetesClient, fv1.ExecutorTypePoolmgr, time.Minute*30)
if err != nil {
return err
}
gpmPodInformer := gpmInformerFactory.Core().V1().Pods()
gpmRsInformer := gpmInformerFactory.Apps().V1().ReplicaSets()
gpm, err := poolmgr.MakeGenericPoolManager(
logger,
fissionClient, kubernetesClient, metricsClient,
functionNamespace, fetcherConfig, executorInstanceID,
&funcInformer, &pkgInformer,
)
funcInformer, pkgInformer, envInformer,
gpmPodInformer, gpmRsInformer)
if err != nil {
return errors.Wrap(err, "pool manager creation faied")
}
ndmInformerFactory, err := utils.GetInformerFactoryByExecutor(kubernetesClient, fv1.ExecutorTypeNewdeploy, time.Minute*30)
if err != nil {
return err
}
ndmDeplInformer := ndmInformerFactory.Apps().V1().Deployments()
ndmSvcInformer := ndmInformerFactory.Core().V1().Services()
ndm, err := newdeploy.MakeNewDeploy(
logger,
fissionClient, kubernetesClient,
functionNamespace, fetcherConfig, executorInstanceID,
&funcInformer, &envInformer,
)
funcInformer, envInformer,
ndmDeplInformer, ndmSvcInformer)
if err != nil {
return errors.Wrap(err, "new deploy manager creation faied")
}
cnmInformerFactory, err := utils.GetInformerFactoryByExecutor(kubernetesClient, fv1.ExecutorTypeContainer, time.Minute*30)
if err != nil {
return err
}
cnmDeplInformer := cnmInformerFactory.Apps().V1().Deployments()
cnmSvcInformer := cnmInformerFactory.Core().V1().Services()
ctx := context.Background()
cnm, err := container.MakeContainer(
logger,
ctx, logger,
fissionClient, kubernetesClient,
functionNamespace, executorInstanceID, &funcInformer)
functionNamespace, executorInstanceID, funcInformer,
cnmDeplInformer, cnmSvcInformer)
if err != nil {
return errors.Wrap(err, "container manager creation faied")
}
executorTypes := make(map[fv1.ExecutorType]executortype.ExecutorType)
executorTypes[gpm.GetTypeName()] = gpm
executorTypes[ndm.GetTypeName()] = ndm
executorTypes[cnm.GetTypeName()] = cnm
executorTypes[gpm.GetTypeName(ctx)] = gpm
executorTypes[ndm.GetTypeName(ctx)] = ndm
executorTypes[cnm.GetTypeName(ctx)] = cnm
adoptExistingResources, _ := strconv.ParseBool(os.Getenv("ADOPT_EXISTING_RESOURCES"))
@@ -321,31 +347,40 @@ func StartExecutor(logger *zap.Logger, functionNamespace string, envBuilderNames
go func(et executortype.ExecutorType) {
defer wg.Done()
if adoptExistingResources {
et.AdoptExistingResources()
et.AdoptExistingResources(ctx)
}
et.CleanupOldExecutorObjects()
et.CleanupOldExecutorObjects(ctx)
}(et)
}
// set hard timeout for resource adoption
// TODO: use context to control the waiting time once kubernetes client supports it.
util.WaitTimeout(wg, 30*time.Second)
k8sInformerFactory := k8sInformers.NewSharedInformerFactory(kubernetesClient, time.Second*30)
configmapInformer := k8sInformerFactory.Core().V1().ConfigMaps().Informer()
secretInformer := k8sInformerFactory.Core().V1().Secrets().Informer()
k8sInformerFactory := k8sInformers.NewSharedInformerFactory(kubernetesClient, time.Minute*30)
configmapInformer := k8sInformerFactory.Core().V1().ConfigMaps()
secretInformer := k8sInformerFactory.Core().V1().Secrets()
cms := cms.MakeConfigSecretController(logger, fissionClient, kubernetesClient, executorTypes, &configmapInformer, &secretInformer)
cms := cms.MakeConfigSecretController(ctx, logger, fissionClient, kubernetesClient, executorTypes, configmapInformer, secretInformer)
ctx := context.Background()
api, err := MakeExecutor(ctx, logger, cms, fissionClient, executorTypes, []k8sCache.SharedIndexInformer{
funcInformer, pkgInformer, envInformer, configmapInformer, secretInformer,
})
api, err := MakeExecutor(ctx, logger, cms, fissionClient, executorTypes,
[]k8sCache.SharedIndexInformer{
funcInformer.Informer(),
pkgInformer.Informer(),
envInformer.Informer(),
configmapInformer.Informer(),
secretInformer.Informer(),
gpmPodInformer.Informer(),
gpmRsInformer.Informer(),
ndmDeplInformer.Informer(),
ndmSvcInformer.Informer(),
cnmDeplInformer.Informer(),
cnmSvcInformer.Informer(),
})
if err != nil {
return err
}
go reaper.CleanupRoleBindings(logger, kubernetesClient, fissionClient, functionNamespace, envBuilderNamespace, time.Minute*30)
go api.Serve(port)
go api.Serve(port, openTracingEnabled)
go serveMetric(logger)
return nil
+1 -1
View File
@@ -173,7 +173,7 @@ func TestExecutor(t *testing.T) {
// create poolmgr
port := 9999
err = StartExecutor(logger, functionNs, "fission-builder", port)
err = StartExecutor(logger, functionNs, "fission-builder", port, true)
if err != nil {
log.Panicf("failed to start poolmgr: %v", err)
}
@@ -65,10 +65,10 @@ func (cn *Container) getResources(fn *fv1.Function) apiv1.ResourceRequirements {
}
// cleanupContainer cleans all kubernetes objects related to function
func (cn *Container) cleanupContainer(ns string, name string) error {
func (cn *Container) cleanupContainer(ctx context.Context, ns string, name string) error {
result := &multierror.Error{}
err := cn.deleteSvc(ns, name)
err := cn.deleteSvc(ctx, ns, name)
if err != nil && !k8s_err.IsNotFound(err) {
cn.logger.Error("error deleting service for Container function",
zap.Error(err),
@@ -77,7 +77,7 @@ func (cn *Container) cleanupContainer(ns string, name string) error {
result = multierror.Append(result, err)
}
err = cn.deleteHpa(ns, name)
err = cn.deleteHpa(ctx, ns, name)
if err != nil && !k8s_err.IsNotFound(err) {
cn.logger.Error("error deleting HPA for Container function",
zap.Error(err),
@@ -86,7 +86,7 @@ func (cn *Container) cleanupContainer(ns string, name string) error {
result = multierror.Append(result, err)
}
err = cn.deleteDeployment(ns, name)
err = cn.deleteDeployment(ctx, ns, name)
if err != nil && !k8s_err.IsNotFound(err) {
cn.logger.Error("error deleting deployment for Container function",
zap.Error(err),
@@ -106,11 +106,11 @@ func (cn *Container) cleanupContainer(ns string, name string) error {
// identical way to get a value that can reflect resources changed without affecting by the time.
// To achieve this goal, the sum of the resource version of all referenced resources is a good fit for our
// scenario since the sum of the resource version is always the same as long as no resources changed.
func referencedResourcesRVSum(client *kubernetes.Clientset, namespace string, secrets []fv1.SecretReference, cfgmaps []fv1.ConfigMapReference) (int, error) {
func referencedResourcesRVSum(ctx context.Context, client *kubernetes.Clientset, namespace string, secrets []fv1.SecretReference, cfgmaps []fv1.ConfigMapReference) (int, error) {
rvCount := 0
if len(secrets) > 0 {
list, err := client.CoreV1().Secrets(namespace).List(context.TODO(), metav1.ListOptions{})
list, err := client.CoreV1().Secrets(namespace).List(ctx, metav1.ListOptions{})
if err != nil {
return 0, err
}
@@ -130,7 +130,7 @@ func referencedResourcesRVSum(client *kubernetes.Clientset, namespace string, se
}
if len(cfgmaps) > 0 {
list, err := client.CoreV1().ConfigMaps(namespace).List(context.TODO(), metav1.ListOptions{})
list, err := client.CoreV1().ConfigMaps(namespace).List(ctx, metav1.ListOptions{})
if err != nil {
return 0, err
}
@@ -29,13 +29,16 @@ import (
multierror "github.com/hashicorp/go-multierror"
"github.com/pkg/errors"
"go.uber.org/zap"
appsv1 "k8s.io/api/apps/v1"
apiv1 "k8s.io/api/core/v1"
k8sErrs "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/labels"
k8sTypes "k8s.io/apimachinery/pkg/types"
appsinformers "k8s.io/client-go/informers/apps/v1"
coreinformers "k8s.io/client-go/informers/core/v1"
"k8s.io/client-go/kubernetes"
appslisters "k8s.io/client-go/listers/apps/v1"
corelisters "k8s.io/client-go/listers/core/v1"
k8sCache "k8s.io/client-go/tools/cache"
fv1 "github.com/fission/fission/pkg/apis/core/v1"
@@ -43,9 +46,11 @@ import (
"github.com/fission/fission/pkg/executor/executortype"
"github.com/fission/fission/pkg/executor/fscache"
"github.com/fission/fission/pkg/executor/reaper"
finformerv1 "github.com/fission/fission/pkg/generated/informers/externalversions/core/v1"
"github.com/fission/fission/pkg/throttler"
"github.com/fission/fission/pkg/utils"
"github.com/fission/fission/pkg/utils/maps"
otelUtils "github.com/fission/fission/pkg/utils/otel"
)
var _ executortype.ExecutorType = &Container{}
@@ -68,22 +73,28 @@ type (
throttler *throttler.Throttler
funcInformer *k8sCache.SharedIndexInformer
serviceInformer k8sCache.SharedIndexInformer
deploymentInformer k8sCache.SharedIndexInformer
defaultIdlePodReapTime time.Duration
deplLister appslisters.DeploymentLister
svcLister corelisters.ServiceLister
deplListerSynced k8sCache.InformerSynced
svcListerSynced k8sCache.InformerSynced
}
)
// MakeContainer initializes and returns an instance of CaaF
func MakeContainer(
ctx context.Context,
logger *zap.Logger,
fissionClient *crd.FissionClient,
kubernetesClient *kubernetes.Clientset,
namespace string,
instanceID string,
funcInformer *k8sCache.SharedIndexInformer) (executortype.ExecutorType, error) {
funcInformer finformerv1.FunctionInformer,
deplInformer appsinformers.DeploymentInformer,
svcInformer coreinformers.ServiceInformer,
) (executortype.ExecutorType, error) {
enableIstio := false
if len(os.Getenv("ENABLE_ISTIO")) > 0 {
istio, err := strconv.ParseBool(os.Getenv("ENABLE_ISTIO"))
@@ -104,33 +115,31 @@ func MakeContainer(
fsCache: fscache.MakeFunctionServiceCache(logger),
throttler: throttler.MakeThrottler(1 * time.Minute),
funcInformer: funcInformer,
runtimeImagePullPolicy: utils.GetImagePullPolicy(os.Getenv("RUNTIME_IMAGE_PULL_POLICY")),
useIstio: enableIstio,
// Time is set slightly higher than NewDeploy as cold starts are longer for CaaF
defaultIdlePodReapTime: 1 * time.Minute,
}
caaf.deplLister = deplInformer.Lister()
caaf.deplListerSynced = deplInformer.Informer().HasSynced
(*caaf.funcInformer).AddEventHandler(caaf.FuncInformerHandler())
informerFactory, err := utils.GetInformerFactoryByExecutor(caaf.kubernetesClient, fv1.ExecutorTypeContainer)
if err != nil {
return nil, err
}
caaf.serviceInformer = informerFactory.Core().V1().Services().Informer()
caaf.deploymentInformer = informerFactory.Apps().V1().Deployments().Informer()
caaf.svcLister = svcInformer.Lister()
caaf.svcListerSynced = svcInformer.Informer().HasSynced
funcInformer.Informer().AddEventHandler(caaf.FuncInformerHandler(ctx))
return caaf, nil
}
// Run start the function along with an object reaper.
func (caaf *Container) Run(ctx context.Context) {
if ok := k8sCache.WaitForCacheSync(ctx.Done(), caaf.deplListerSynced, caaf.svcListerSynced); !ok {
caaf.logger.Fatal("failed to wait for caches to sync")
}
go caaf.idleObjectReaper()
}
// GetTypeName returns the executor type name.
func (caaf *Container) GetTypeName() fv1.ExecutorType {
func (caaf *Container) GetTypeName(ctx context.Context) fv1.ExecutorType {
return fv1.ExecutorTypeContainer
}
@@ -141,33 +150,34 @@ func (caaf *Container) GetTotalAvailable(fn *fv1.Function) int {
}
// UnTapService has not been implemented for CaaF.
func (caaf *Container) UnTapService(key string, svcHost string) {
func (caaf *Container) UnTapService(ctx context.Context, key string, svcHost string) {
// Not Implemented for CaaF.
}
// GetFuncSvc returns a function service; error otherwise.
func (caaf *Container) GetFuncSvc(ctx context.Context, fn *fv1.Function) (*fscache.FuncSvc, error) {
return caaf.createFunction(fn)
return caaf.createFunction(ctx, fn)
}
// GetFuncSvcFromCache returns a function service from cache; error otherwise.
func (caaf *Container) GetFuncSvcFromCache(fn *fv1.Function) (*fscache.FuncSvc, error) {
func (caaf *Container) GetFuncSvcFromCache(ctx context.Context, fn *fv1.Function) (*fscache.FuncSvc, error) {
otelUtils.SpanTrackEvent(ctx, "GetFuncSvcFromCache", otelUtils.GetAttributesForFunction(fn)...)
return caaf.fsCache.GetByFunction(&fn.ObjectMeta)
}
// DeleteFuncSvcFromCache deletes a function service from cache.
func (caaf *Container) DeleteFuncSvcFromCache(fsvc *fscache.FuncSvc) {
func (caaf *Container) DeleteFuncSvcFromCache(ctx context.Context, fsvc *fscache.FuncSvc) {
caaf.fsCache.DeleteEntry(fsvc)
}
// GetFuncSvcFromPoolCache has not been implemented for Container Functions
func (caaf *Container) GetFuncSvcFromPoolCache(fn *fv1.Function, requestsPerPod int) (*fscache.FuncSvc, int, error) {
func (caaf *Container) GetFuncSvcFromPoolCache(ctx context.Context, fn *fv1.Function, requestsPerPod int) (*fscache.FuncSvc, int, error) {
// Not Implemented for NewDeployment. Will be used when support of concurrent specialization of same function is added.
return nil, 0, nil
}
// TapService makes a TouchByAddress request to the cache.
func (caaf *Container) TapService(svcHost string) error {
func (caaf *Container) TapService(ctx context.Context, svcHost string) error {
err := caaf.fsCache.TouchByAddress(svcHost)
if err != nil {
return err
@@ -175,67 +185,34 @@ func (caaf *Container) TapService(svcHost string) error {
return nil
}
func (caaf *Container) getServiceInfo(obj apiv1.ObjectReference) (*apiv1.Service, error) {
item, exists, err := utils.GetCachedItem(obj, caaf.serviceInformer)
if err != nil || !exists {
caaf.logger.Debug(
"Falling back to getting service info from k8s API -- this may cause performance issues for your function.",
zap.Bool("exists", exists),
zap.Error(err),
)
service, err := caaf.kubernetesClient.CoreV1().Services(obj.Namespace).Get(context.TODO(), obj.Name, metav1.GetOptions{})
return service, err
}
service := item.(*apiv1.Service)
return service, nil
}
func (caaf *Container) getDeploymentInfo(obj apiv1.ObjectReference) (*appsv1.Deployment, error) {
item, exists, err := utils.GetCachedItem(obj, caaf.deploymentInformer)
if err != nil || !exists {
caaf.logger.Debug(
"Falling back to getting deployment info from k8s API -- this may cause performance issues for your function.",
zap.Bool("exists", exists),
zap.Error(err),
)
deployment, err := caaf.kubernetesClient.AppsV1().Deployments(obj.Namespace).Get(context.TODO(), obj.Name, metav1.GetOptions{})
return deployment, err
}
deployment := item.(*appsv1.Deployment)
return deployment, nil
}
// IsValid does a get on the service address to ensure it's a valid service, then
// scale deployment to 1 replica if there are no available replicas for function.
// Return true if no error occurs, return false otherwise.
func (caaf *Container) IsValid(fsvc *fscache.FuncSvc) bool {
func (caaf *Container) IsValid(ctx context.Context, fsvc *fscache.FuncSvc) bool {
logger := otelUtils.LoggerWithTraceID(ctx, caaf.logger)
otelUtils.SpanTrackEvent(ctx, "IsValid", otelUtils.GetAttributesForFuncSvc(fsvc)...)
if len(strings.Split(fsvc.Address, ".")) == 0 {
caaf.logger.Error("address not found in function service")
logger.Error("address not found in function service")
return false
}
if len(fsvc.KubernetesObjects) == 0 {
caaf.logger.Error("no kubernetes object related to function", zap.String("function", fsvc.Function.Name))
logger.Error("no kubernetes object related to function", zap.String("function", fsvc.Function.Name))
return false
}
for _, obj := range fsvc.KubernetesObjects {
if strings.ToLower(obj.Kind) == "service" {
_, err := caaf.getServiceInfo(obj)
_, err := caaf.svcLister.Services(obj.Namespace).Get(obj.Name)
if err != nil {
if !k8sErrs.IsNotFound(err) {
caaf.logger.Error("error validating function service", zap.String("function", fsvc.Function.Name), zap.Error(err))
logger.Error("error validating function service", zap.String("function", fsvc.Function.Name), zap.Error(err))
}
return false
}
} else if strings.ToLower(obj.Kind) == "deployment" {
currentDeploy, err := caaf.getDeploymentInfo(obj)
currentDeploy, err := caaf.deplLister.Deployments(obj.Namespace).Get(obj.Name)
if err != nil {
if !k8sErrs.IsNotFound(err) {
caaf.logger.Error("error validating function deployment", zap.String("function", fsvc.Function.Name), zap.Error(err))
logger.Error("error validating function deployment", zap.String("function", fsvc.Function.Name), zap.Error(err))
}
return false
}
@@ -248,11 +225,11 @@ func (caaf *Container) IsValid(fsvc *fscache.FuncSvc) bool {
}
// RefreshFuncPods deletes pods related to the function so that new pods are replenished
func (caaf *Container) RefreshFuncPods(logger *zap.Logger, f fv1.Function) error {
func (caaf *Container) RefreshFuncPods(ctx context.Context, logger *zap.Logger, f fv1.Function) error {
funcLabels := caaf.getDeployLabels(f.ObjectMeta)
dep, err := caaf.kubernetesClient.AppsV1().Deployments(metav1.NamespaceAll).List(context.TODO(), metav1.ListOptions{
dep, err := caaf.kubernetesClient.AppsV1().Deployments(metav1.NamespaceAll).List(ctx, metav1.ListOptions{
LabelSelector: labels.Set(funcLabels).AsSelector().String(),
})
if err != nil {
@@ -261,7 +238,7 @@ func (caaf *Container) RefreshFuncPods(logger *zap.Logger, f fv1.Function) error
// Ideally there should be only one deployment but for now we rely on label/selector to ensure that condition
for _, deployment := range dep.Items {
rvCount, err := referencedResourcesRVSum(caaf.kubernetesClient, deployment.Namespace, f.Spec.Secrets, f.Spec.ConfigMaps)
rvCount, err := referencedResourcesRVSum(ctx, caaf.kubernetesClient, deployment.Namespace, f.Spec.Secrets, f.Spec.ConfigMaps)
if err != nil {
return err
}
@@ -269,7 +246,7 @@ func (caaf *Container) RefreshFuncPods(logger *zap.Logger, f fv1.Function) error
patch := fmt.Sprintf(`{"spec" : {"template": {"spec":{"containers":[{"name": "%s", "env":[{"name": "%s", "value": "%v"}]}]}}}}`,
f.ObjectMeta.Name, fv1.ResourceVersionCount, rvCount)
_, err = caaf.kubernetesClient.AppsV1().Deployments(deployment.ObjectMeta.Namespace).Patch(context.TODO(), deployment.ObjectMeta.Name,
_, err = caaf.kubernetesClient.AppsV1().Deployments(deployment.ObjectMeta.Namespace).Patch(ctx, deployment.ObjectMeta.Name,
k8sTypes.StrategicMergePatchType,
[]byte(patch), metav1.PatchOptions{})
if err != nil {
@@ -280,8 +257,8 @@ func (caaf *Container) RefreshFuncPods(logger *zap.Logger, f fv1.Function) error
}
// AdoptExistingResources attempts to adopt resources for functions in all namespaces.
func (caaf *Container) AdoptExistingResources() {
fnList, err := caaf.fissionClient.CoreV1().Functions(metav1.NamespaceAll).List(context.TODO(), metav1.ListOptions{})
func (caaf *Container) AdoptExistingResources(ctx context.Context) {
fnList, err := caaf.fissionClient.CoreV1().Functions(metav1.NamespaceAll).List(ctx, metav1.ListOptions{})
if err != nil {
caaf.logger.Error("error getting function list", zap.Error(err))
return
@@ -296,7 +273,7 @@ func (caaf *Container) AdoptExistingResources() {
go func() {
defer wg.Done()
_, err = caaf.fnCreate(fn)
_, err = caaf.fnCreate(ctx, fn)
if err != nil {
caaf.logger.Warn("failed to adopt resources for function", zap.Error(err))
return
@@ -310,7 +287,7 @@ func (caaf *Container) AdoptExistingResources() {
}
// CleanupOldExecutorObjects cleans orphaned resources.
func (caaf *Container) CleanupOldExecutorObjects() {
func (caaf *Container) CleanupOldExecutorObjects(ctx context.Context) {
caaf.logger.Info("CaaF starts to clean orphaned resources", zap.String("instanceID", caaf.instanceID))
errs := &multierror.Error{}
@@ -318,17 +295,17 @@ func (caaf *Container) CleanupOldExecutorObjects() {
LabelSelector: labels.Set(map[string]string{fv1.EXECUTOR_TYPE: string(fv1.ExecutorTypeContainer)}).AsSelector().String(),
}
err := reaper.CleanupHpa(caaf.logger, caaf.kubernetesClient, caaf.instanceID, listOpts)
err := reaper.CleanupHpa(ctx, caaf.logger, caaf.kubernetesClient, caaf.instanceID, listOpts)
if err != nil {
errs = multierror.Append(errs, err)
}
err = reaper.CleanupDeployments(caaf.logger, caaf.kubernetesClient, caaf.instanceID, listOpts)
err = reaper.CleanupDeployments(ctx, caaf.logger, caaf.kubernetesClient, caaf.instanceID, listOpts)
if err != nil {
errs = multierror.Append(errs, err)
}
err = reaper.CleanupServices(caaf.logger, caaf.kubernetesClient, caaf.instanceID, listOpts)
err = reaper.CleanupServices(ctx, caaf.logger, caaf.kubernetesClient, caaf.instanceID, listOpts)
if err != nil {
errs = multierror.Append(errs, err)
}
@@ -339,14 +316,14 @@ func (caaf *Container) CleanupOldExecutorObjects() {
}
}
func (caaf *Container) createFunction(fn *fv1.Function) (*fscache.FuncSvc, error) {
func (caaf *Container) createFunction(ctx context.Context, fn *fv1.Function) (*fscache.FuncSvc, error) {
if fn.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType != fv1.ExecutorTypeContainer {
return nil, nil
}
fsvcObj, err := caaf.throttler.RunOnce(string(fn.ObjectMeta.UID), func(ableToCreate bool) (interface{}, error) {
if ableToCreate {
return caaf.fnCreate(fn)
return caaf.fnCreate(ctx, fn)
}
return caaf.fsCache.GetByFunctionUID(fn.ObjectMeta.UID)
})
@@ -367,20 +344,20 @@ func (caaf *Container) createFunction(fn *fv1.Function) (*fscache.FuncSvc, error
return fsvc, err
}
func (caaf *Container) deleteFunction(fn *fv1.Function) error {
func (caaf *Container) deleteFunction(ctx context.Context, fn *fv1.Function) error {
if fn.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType != fv1.ExecutorTypeContainer {
return nil
}
err := caaf.fnDelete(fn)
err := caaf.fnDelete(ctx, fn)
if err != nil {
err = errors.Wrapf(err, "error deleting kubernetes objects of function %v", fn.ObjectMeta)
}
return err
}
func (caaf *Container) fnCreate(fn *fv1.Function) (*fscache.FuncSvc, error) {
func (caaf *Container) fnCreate(ctx context.Context, fn *fv1.Function) (*fscache.FuncSvc, error) {
cleanupFunc := func(ns string, name string) {
err := caaf.cleanupContainer(ns, name)
err := caaf.cleanupContainer(ctx, ns, name)
if err != nil {
caaf.logger.Error("received error while cleaning function resources",
zap.String("namespace", ns), zap.String("name", name))
@@ -402,7 +379,7 @@ func (caaf *Container) fnCreate(fn *fv1.Function) (*fscache.FuncSvc, error) {
// Since Container waits for pods of deployment to be ready,
// change the order of kubeObject creation (create service first,
// then deployment) to take advantage of waiting time.
svc, err := caaf.createOrGetSvc(fn, deployLabels, deployAnnotations, objName, ns)
svc, err := caaf.createOrGetSvc(ctx, fn, deployLabels, deployAnnotations, objName, ns)
if err != nil {
caaf.logger.Error("error creating service", zap.Error(err), zap.String("service", objName))
go cleanupFunc(ns, objName)
@@ -410,14 +387,14 @@ func (caaf *Container) fnCreate(fn *fv1.Function) (*fscache.FuncSvc, error) {
}
svcAddress := fmt.Sprintf("%v.%v", svc.Name, svc.Namespace)
depl, err := caaf.createOrGetDeployment(fn, objName, deployLabels, deployAnnotations, ns)
depl, err := caaf.createOrGetDeployment(ctx, fn, objName, deployLabels, deployAnnotations, ns)
if err != nil {
caaf.logger.Error("error creating deployment", zap.Error(err), zap.String("deployment", objName))
go cleanupFunc(ns, objName)
return nil, errors.Wrapf(err, "error creating deployment %v", objName)
}
hpa, err := caaf.createOrGetHpa(objName, &fn.Spec.InvokeStrategy.ExecutionStrategy, depl, deployLabels, deployAnnotations)
hpa, err := caaf.createOrGetHpa(ctx, objName, &fn.Spec.InvokeStrategy.ExecutionStrategy, depl, deployLabels, deployAnnotations)
if err != nil {
caaf.logger.Error("error creating HPA", zap.Error(err), zap.String("hpa", objName))
go cleanupFunc(ns, objName)
@@ -471,7 +448,7 @@ func (caaf *Container) fnCreate(fn *fv1.Function) (*fscache.FuncSvc, error) {
return fsvc, nil
}
func (caaf *Container) updateFunction(oldFn *fv1.Function, newFn *fv1.Function) error {
func (caaf *Container) updateFunction(ctx context.Context, oldFn *fv1.Function, newFn *fv1.Function) error {
if oldFn.ObjectMeta.ResourceVersion == newFn.ObjectMeta.ResourceVersion {
return nil
@@ -489,7 +466,7 @@ func (caaf *Container) updateFunction(oldFn *fv1.Function, newFn *fv1.Function)
caaf.logger.Info("function does not use new deployment executor anymore, deleting resources",
zap.Any("function", newFn))
// IMP - pass the oldFn, as the new/modified function is not in cache
return caaf.deleteFunction(oldFn)
return caaf.deleteFunction(ctx, oldFn)
}
// Executor type changed to Container from something else
@@ -498,7 +475,7 @@ func (caaf *Container) updateFunction(oldFn *fv1.Function, newFn *fv1.Function)
caaf.logger.Info("function type changed to Container, creating resources",
zap.Any("old_function", oldFn.ObjectMeta),
zap.Any("new_function", newFn.ObjectMeta))
_, err := caaf.createFunction(newFn)
_, err := caaf.createFunction(ctx, newFn)
if err != nil {
caaf.updateStatus(oldFn, err, "error changing the function's type to Container")
}
@@ -520,7 +497,7 @@ func (caaf *Container) updateFunction(oldFn *fv1.Function, newFn *fv1.Function)
return err
}
hpa, err := caaf.getHpa(ns, fsvc.Name)
hpa, err := caaf.getHpa(ctx, ns, fsvc.Name)
if err != nil {
caaf.updateStatus(oldFn, err, "error getting HPA while updating function")
return err
@@ -546,7 +523,7 @@ func (caaf *Container) updateFunction(oldFn *fv1.Function, newFn *fv1.Function)
}
if hpaChanged {
err := caaf.updateHpa(hpa)
err := caaf.updateHpa(ctx, hpa)
if err != nil {
caaf.updateStatus(oldFn, err, "error updating HPA while updating function")
return err
@@ -583,13 +560,13 @@ func (caaf *Container) updateFunction(oldFn *fv1.Function, newFn *fv1.Function)
}
if deployChanged {
return caaf.updateFuncDeployment(newFn)
return caaf.updateFuncDeployment(ctx, newFn)
}
return nil
}
func (caaf *Container) updateFuncDeployment(fn *fv1.Function) error {
func (caaf *Container) updateFuncDeployment(ctx context.Context, fn *fv1.Function) error {
fsvc, err := caaf.fsCache.GetByFunctionUID(fn.ObjectMeta.UID)
if err != nil {
@@ -609,7 +586,7 @@ func (caaf *Container) updateFuncDeployment(fn *fv1.Function) error {
ns = fn.ObjectMeta.Namespace
}
existingDepl, err := caaf.kubernetesClient.AppsV1().Deployments(ns).Get(context.TODO(), fnObjName, metav1.GetOptions{})
existingDepl, err := caaf.kubernetesClient.AppsV1().Deployments(ns).Get(ctx, fnObjName, metav1.GetOptions{})
if err != nil {
return err
}
@@ -617,14 +594,14 @@ func (caaf *Container) updateFuncDeployment(fn *fv1.Function) error {
// the resource version inside function packageRef is changed,
// so the content of fetchRequest in deployment cmd is different.
// Therefore, the deployment update will trigger a rolling update.
newDeployment, err := caaf.getDeploymentSpec(fn, existingDepl.Spec.Replicas, // use current replicas instead of minscale in the ExecutionStrategy.
newDeployment, err := caaf.getDeploymentSpec(ctx, fn, existingDepl.Spec.Replicas, // use current replicas instead of minscale in the ExecutionStrategy.
fnObjName, ns, deployLabels, caaf.getDeployAnnotations(fn.ObjectMeta))
if err != nil {
caaf.updateStatus(fn, err, "failed to get new deployment spec while updating function")
return err
}
err = caaf.updateDeployment(newDeployment, ns)
err = caaf.updateDeployment(ctx, newDeployment, ns)
if err != nil {
caaf.updateStatus(fn, err, "failed to update deployment while updating function")
return err
@@ -633,7 +610,7 @@ func (caaf *Container) updateFuncDeployment(fn *fv1.Function) error {
return nil
}
func (caaf *Container) fnDelete(fn *fv1.Function) error {
func (caaf *Container) fnDelete(ctx context.Context, fn *fv1.Function) error {
multierr := &multierror.Error{}
// GetByFunction uses resource version as part of cache key, however,
@@ -662,7 +639,7 @@ func (caaf *Container) fnDelete(fn *fv1.Function) error {
ns = fn.ObjectMeta.Namespace
}
err = caaf.cleanupContainer(ns, objName)
err = caaf.cleanupContainer(ctx, ns, objName)
multierr = multierror.Append(multierr, err)
return multierr.ErrorOrNil()
@@ -676,7 +653,16 @@ func (caaf *Container) getObjName(fn *fv1.Function) string {
if len(fn.ObjectMeta.Name)+len(fn.ObjectMeta.Namespace) < 35 {
functionMetadata = fn.ObjectMeta.Name + "-" + fn.ObjectMeta.Namespace
} else {
functionMetadata = fn.ObjectMeta.Name[:17] + "-" + fn.ObjectMeta.Namespace[:17]
if len(fn.ObjectMeta.Name) > 17 {
functionMetadata = fn.ObjectMeta.Name[:17]
} else {
functionMetadata = fn.ObjectMeta.Name
}
if len(fn.ObjectMeta.Namespace) > 17 {
functionMetadata = functionMetadata + "-" + fn.ObjectMeta.Namespace[:17]
} else {
functionMetadata = functionMetadata + "-" + fn.ObjectMeta.Namespace
}
}
// contructed name should be 63 characters long, as it is a valid k8s name
// functionMetadata should be 35 characters long, as we take 17 characters from functionUid
@@ -708,6 +694,7 @@ func (caaf *Container) updateStatus(fn *fv1.Function, err error, message string)
// idleObjectReaper reaps objects after certain idle time
func (caaf *Container) idleObjectReaper() {
ctx := context.Background()
pollSleep := 5 * time.Second
for {
@@ -726,7 +713,7 @@ func (caaf *Container) idleObjectReaper() {
continue
}
fn, err := caaf.fissionClient.CoreV1().Functions(fsvc.Function.Namespace).Get(context.TODO(), fsvc.Function.Name, metav1.GetOptions{})
fn, err := caaf.fissionClient.CoreV1().Functions(fsvc.Function.Namespace).Get(ctx, fsvc.Function.Name, metav1.GetOptions{})
if err != nil {
// CaaF manager handles the function delete event and clean cache/kubeobjs itself,
// so we ignore the not found error for functions with CaaF executor type here.
@@ -754,7 +741,7 @@ func (caaf *Container) idleObjectReaper() {
}
currentDeploy, err := caaf.kubernetesClient.AppsV1().
Deployments(deployObj.Namespace).Get(context.TODO(), deployObj.Name, metav1.GetOptions{})
Deployments(deployObj.Namespace).Get(ctx, deployObj.Name, metav1.GetOptions{})
if err != nil {
caaf.logger.Error("error getting function deployment", zap.Error(err), zap.String("function", fsvc.Function.Name))
return
@@ -767,7 +754,7 @@ func (caaf *Container) idleObjectReaper() {
return
}
err = caaf.scaleDeployment(deployObj.Namespace, deployObj.Name, minScale)
err = caaf.scaleDeployment(ctx, deployObj.Namespace, deployObj.Name, minScale)
if err != nil {
caaf.logger.Error("error scaling down function deployment", zap.Error(err), zap.String("function", fsvc.Function.Name))
}
@@ -31,9 +31,11 @@ import (
fv1 "github.com/fission/fission/pkg/apis/core/v1"
"github.com/fission/fission/pkg/executor/util"
otelUtils "github.com/fission/fission/pkg/utils/otel"
)
func (cn *Container) createOrGetDeployment(fn *fv1.Function, deployName string, deployLabels map[string]string, deployAnnotations map[string]string, deployNamespace string) (*appsv1.Deployment, error) {
func (cn *Container) createOrGetDeployment(ctx context.Context, fn *fv1.Function, deployName string, deployLabels map[string]string, deployAnnotations map[string]string, deployNamespace string) (*appsv1.Deployment, error) {
logger := otelUtils.LoggerWithTraceID(ctx, cn.logger)
// The specializationTimeout here refers to the creation of the pod and not the loading of function
// as in other executors.
@@ -47,12 +49,12 @@ func (cn *Container) createOrGetDeployment(fn *fv1.Function, deployName string,
minScale = 1
}
deployment, err := cn.getDeploymentSpec(fn, &minScale, deployName, deployNamespace, deployLabels, deployAnnotations)
deployment, err := cn.getDeploymentSpec(ctx, fn, &minScale, deployName, deployNamespace, deployLabels, deployAnnotations)
if err != nil {
return nil, err
}
existingDepl, err := cn.kubernetesClient.AppsV1().Deployments(deployNamespace).Get(context.TODO(), deployName, metav1.GetOptions{})
existingDepl, err := cn.kubernetesClient.AppsV1().Deployments(deployNamespace).Get(ctx, deployName, metav1.GetOptions{})
if err == nil {
// Try to adopt orphan deployment created by the old executor.
if existingDepl.Annotations[fv1.EXECUTOR_INSTANCEID_LABEL] != cn.instanceID {
@@ -64,9 +66,9 @@ func (cn *Container) createOrGetDeployment(fn *fv1.Function, deployName string,
// Update with the latest deployment spec. Kubernetes will trigger
// rolling update if spec is different from the one in the cluster.
existingDepl, err = cn.kubernetesClient.AppsV1().Deployments(deployNamespace).Update(context.TODO(), existingDepl, metav1.UpdateOptions{})
existingDepl, err = cn.kubernetesClient.AppsV1().Deployments(deployNamespace).Update(ctx, existingDepl, metav1.UpdateOptions{})
if err != nil {
cn.logger.Warn("error adopting cn", zap.Error(err),
logger.Warn("error adopting cn", zap.Error(err),
zap.String("cn", deployName), zap.String("ns", deployNamespace))
return nil, err
}
@@ -75,25 +77,25 @@ func (cn *Container) createOrGetDeployment(fn *fv1.Function, deployName string,
}
if *existingDepl.Spec.Replicas < minScale {
err = cn.scaleDeployment(existingDepl.Namespace, existingDepl.Name, minScale)
err = cn.scaleDeployment(ctx, existingDepl.Namespace, existingDepl.Name, minScale)
if err != nil {
cn.logger.Error("error scaling up function deployment", zap.Error(err), zap.String("function", fn.ObjectMeta.Name))
logger.Error("error scaling up function deployment", zap.Error(err), zap.String("function", fn.ObjectMeta.Name))
return nil, err
}
}
if existingDepl.Status.AvailableReplicas < minScale {
existingDepl, err = cn.waitForDeploy(existingDepl, minScale, specializationTimeout)
existingDepl, err = cn.waitForDeploy(ctx, existingDepl, minScale, specializationTimeout)
}
return existingDepl, err
} else if k8s_err.IsNotFound(err) {
depl, err := cn.kubernetesClient.AppsV1().Deployments(deployNamespace).Create(context.TODO(), deployment, metav1.CreateOptions{})
depl, err := cn.kubernetesClient.AppsV1().Deployments(deployNamespace).Create(ctx, deployment, metav1.CreateOptions{})
if err != nil {
if k8s_err.IsAlreadyExists(err) {
depl, err = cn.kubernetesClient.AppsV1().Deployments(deployNamespace).Get(context.TODO(), deployName, metav1.GetOptions{})
depl, err = cn.kubernetesClient.AppsV1().Deployments(deployNamespace).Get(ctx, deployName, metav1.GetOptions{})
}
if err != nil {
cn.logger.Error("error while creating function deployment",
logger.Error("error while creating function deployment",
zap.Error(err),
zap.String("function", fn.ObjectMeta.Name),
zap.String("deployment_name", deployName),
@@ -101,38 +103,39 @@ func (cn *Container) createOrGetDeployment(fn *fv1.Function, deployName string,
return nil, err
}
}
otelUtils.SpanTrackEvent(ctx, "deploymentCreated", otelUtils.GetAttributesForDeployment(depl)...)
if minScale > 0 {
depl, err = cn.waitForDeploy(depl, minScale, specializationTimeout)
depl, err = cn.waitForDeploy(ctx, depl, minScale, specializationTimeout)
}
return depl, err
}
return nil, err
}
func (cn *Container) updateDeployment(deployment *appsv1.Deployment, ns string) error {
_, err := cn.kubernetesClient.AppsV1().Deployments(ns).Update(context.TODO(), deployment, metav1.UpdateOptions{})
func (cn *Container) updateDeployment(ctx context.Context, deployment *appsv1.Deployment, ns string) error {
_, err := cn.kubernetesClient.AppsV1().Deployments(ns).Update(ctx, deployment, metav1.UpdateOptions{})
return err
}
func (cn *Container) deleteDeployment(ns string, name string) error {
func (cn *Container) deleteDeployment(ctx context.Context, ns string, name string) error {
// DeletePropagationBackground deletes the object immediately and dependent are deleted later
// DeletePropagationForeground not advisable; it marks for deleteion and API can still serve those objects
deletePropagation := metav1.DeletePropagationBackground
return cn.kubernetesClient.AppsV1().Deployments(ns).Delete(context.TODO(), name, metav1.DeleteOptions{
return cn.kubernetesClient.AppsV1().Deployments(ns).Delete(ctx, name, metav1.DeleteOptions{
PropagationPolicy: &deletePropagation,
})
}
func (cn *Container) waitForDeploy(depl *appsv1.Deployment, replicas int32, specializationTimeout int) (latestDepl *appsv1.Deployment, err error) {
func (cn *Container) waitForDeploy(ctx context.Context, depl *appsv1.Deployment, replicas int32, specializationTimeout int) (latestDepl *appsv1.Deployment, err error) {
oldStatus := depl.Status
otelUtils.SpanTrackEvent(ctx, "waitForDeployment", otelUtils.GetAttributesForDeployment(depl)...)
// if no specializationTimeout is set, use default value
if specializationTimeout < fv1.DefaultSpecializationTimeOut {
specializationTimeout = fv1.DefaultSpecializationTimeOut
}
for i := 0; i < specializationTimeout; i++ {
latestDepl, err := cn.kubernetesClient.AppsV1().Deployments(depl.ObjectMeta.Namespace).Get(context.TODO(), depl.Name, metav1.GetOptions{})
latestDepl, err := cn.kubernetesClient.AppsV1().Deployments(depl.ObjectMeta.Namespace).Get(ctx, depl.Name, metav1.GetOptions{})
if err != nil {
return nil, err
}
@@ -140,13 +143,15 @@ func (cn *Container) waitForDeploy(depl *appsv1.Deployment, replicas int32, spec
// use AvailableReplicas here is better than ReadyReplicas
// since the pods may not be able to serve network traffic yet.
if latestDepl.Status.AvailableReplicas >= replicas {
otelUtils.SpanTrackEvent(ctx, "deploymentAvailable", otelUtils.GetAttributesForDeployment(latestDepl)...)
return latestDepl, err
}
time.Sleep(time.Second)
}
cn.logger.Error("Deployment provision failed within timeout window",
zap.String("name", latestDepl.ObjectMeta.Name), zap.Any("old_status", oldStatus),
logger := otelUtils.LoggerWithTraceID(ctx, cn.logger)
logger.Error("Deployment provision failed within timeout window",
zap.String("name", latestDepl.Name), zap.Any("old_status", oldStatus),
zap.Any("current_status", latestDepl.Status), zap.Int("timeout", specializationTimeout))
// this error appears in the executor pod logs
@@ -154,7 +159,7 @@ func (cn *Container) waitForDeploy(depl *appsv1.Deployment, replicas int32, spec
return nil, timeoutError
}
func (cn *Container) getDeploymentSpec(fn *fv1.Function, targetReplicas *int32,
func (cn *Container) getDeploymentSpec(ctx context.Context, fn *fv1.Function, targetReplicas *int32,
deployName string, deployNamespace string, deployLabels map[string]string, deployAnnotations map[string]string) (*appsv1.Deployment, error) {
replicas := int32(fn.Spec.InvokeStrategy.ExecutionStrategy.MinScale)
@@ -196,12 +201,12 @@ func (cn *Container) getDeploymentSpec(fn *fv1.Function, targetReplicas *int32,
resources := cn.getResources(fn)
// Other executor types rely on Environments to add configmaps and secrets
envFromSources, err := util.ConvertConfigSecrets(fn, cn.kubernetesClient)
envFromSources, err := util.ConvertConfigSecrets(ctx, fn, cn.kubernetesClient)
if err != nil {
return nil, err
}
rvCount, err := referencedResourcesRVSum(cn.kubernetesClient, fn.ObjectMeta.Namespace, fn.Spec.Secrets, fn.Spec.ConfigMaps)
rvCount, err := referencedResourcesRVSum(ctx, cn.kubernetesClient, fn.ObjectMeta.Namespace, fn.Spec.Secrets, fn.Spec.ConfigMaps)
if err != nil {
return nil, err
}
@@ -277,12 +282,12 @@ func (cn *Container) getDeploymentSpec(fn *fv1.Function, targetReplicas *int32,
return deployment, nil
}
func (caaf *Container) scaleDeployment(deplNS string, deplName string, replicas int32) error {
func (caaf *Container) scaleDeployment(ctx context.Context, deplNS string, deplName string, replicas int32) error {
caaf.logger.Info("scaling deployment",
zap.String("deployment", deplName),
zap.String("namespace", deplNS),
zap.Int32("replicas", replicas))
_, err := caaf.kubernetesClient.AppsV1().Deployments(deplNS).UpdateScale(context.TODO(), deplName, &autoscalingv1.Scale{
_, err := caaf.kubernetesClient.AppsV1().Deployments(deplNS).UpdateScale(ctx, deplName, &autoscalingv1.Scale{
ObjectMeta: metav1.ObjectMeta{
Name: deplName,
Namespace: deplNS,
@@ -17,13 +17,15 @@ limitations under the License.
package container
import (
"context"
"go.uber.org/zap"
k8sCache "k8s.io/client-go/tools/cache"
fv1 "github.com/fission/fission/pkg/apis/core/v1"
)
func (caaf *Container) FuncInformerHandler() k8sCache.ResourceEventHandlerFuncs {
func (caaf *Container) FuncInformerHandler(ctx context.Context) k8sCache.ResourceEventHandlerFuncs {
return k8sCache.ResourceEventHandlerFuncs{
AddFunc: func(obj interface{}) {
fn := obj.(*fv1.Function)
@@ -37,7 +39,7 @@ func (caaf *Container) FuncInformerHandler() k8sCache.ResourceEventHandlerFuncs
go func() {
log := caaf.logger.With(zap.String("function_name", fn.ObjectMeta.Name), zap.String("function_namespace", fn.ObjectMeta.Namespace))
log.Debug("start function create handler")
_, err := caaf.createFunction(fn)
_, err := caaf.createFunction(ctx, fn)
if err != nil {
log.Error("error eager creating function", zap.Error(err))
}
@@ -53,7 +55,7 @@ func (caaf *Container) FuncInformerHandler() k8sCache.ResourceEventHandlerFuncs
go func() {
log := caaf.logger.With(zap.String("function_name", fn.ObjectMeta.Name), zap.String("function_namespace", fn.ObjectMeta.Namespace))
log.Debug("start function delete handler")
err := caaf.deleteFunction(fn)
err := caaf.deleteFunction(ctx, fn)
if err != nil {
log.Error("error deleting function", zap.Error(err))
}
@@ -72,7 +74,7 @@ func (caaf *Container) FuncInformerHandler() k8sCache.ResourceEventHandlerFuncs
zap.String("function_namespace", newFn.ObjectMeta.Namespace),
zap.String("old_function_name", oldFn.ObjectMeta.Name))
log.Debug("start function update handler")
err := caaf.updateFunction(oldFn, newFn)
err := caaf.updateFunction(ctx, oldFn, newFn)
if err != nil {
log.Error("error updating function",
zap.Error(err))
+15 -12
View File
@@ -27,6 +27,7 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
fv1 "github.com/fission/fission/pkg/apis/core/v1"
otelUtils "github.com/fission/fission/pkg/utils/otel"
)
const (
@@ -34,13 +35,14 @@ const (
DeploymentVersion = "apps/v1"
)
func (cn *Container) createOrGetHpa(hpaName string, execStrategy *fv1.ExecutionStrategy,
func (cn *Container) createOrGetHpa(ctx context.Context, hpaName string, execStrategy *fv1.ExecutionStrategy,
depl *appsv1.Deployment, deployLabels map[string]string, deployAnnotations map[string]string) (*asv1.HorizontalPodAutoscaler, error) {
if depl == nil {
return nil, errors.New("failed to create HPA, found empty deployment")
}
logger := otelUtils.LoggerWithTraceID(ctx, cn.logger)
minRepl := int32(execStrategy.MinScale)
if minRepl == 0 {
minRepl = 1
@@ -69,45 +71,46 @@ func (cn *Container) createOrGetHpa(hpaName string, execStrategy *fv1.ExecutionS
},
}
existingHpa, err := cn.getHpa(depl.ObjectMeta.Namespace, hpaName)
existingHpa, err := cn.getHpa(ctx, depl.ObjectMeta.Namespace, hpaName)
if err == nil {
// to adopt orphan service
if existingHpa.Annotations[fv1.EXECUTOR_INSTANCEID_LABEL] != cn.instanceID {
existingHpa.Annotations = hpa.Annotations
existingHpa.Labels = hpa.Labels
existingHpa.Spec = hpa.Spec
existingHpa, err = cn.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(depl.ObjectMeta.Namespace).Update(context.TODO(), existingHpa, metav1.UpdateOptions{})
existingHpa, err = cn.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(depl.ObjectMeta.Namespace).Update(ctx, existingHpa, metav1.UpdateOptions{})
if err != nil {
cn.logger.Warn("error adopting HPA", zap.Error(err),
logger.Warn("error adopting HPA", zap.Error(err),
zap.String("HPA", hpaName), zap.String("ns", depl.ObjectMeta.Namespace))
return nil, err
}
}
return existingHpa, err
} else if k8s_err.IsNotFound(err) {
cHpa, err := cn.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(depl.ObjectMeta.Namespace).Create(context.TODO(), hpa, metav1.CreateOptions{})
cHpa, err := cn.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(depl.ObjectMeta.Namespace).Create(ctx, hpa, metav1.CreateOptions{})
if err != nil {
if k8s_err.IsAlreadyExists(err) {
cHpa, err = cn.getHpa(depl.ObjectMeta.Namespace, hpaName)
cHpa, err = cn.getHpa(ctx, depl.ObjectMeta.Namespace, hpaName)
}
if err != nil {
return nil, err
}
}
otelUtils.SpanTrackEvent(ctx, "hpaCreated", otelUtils.GetAttributesForHPA(cHpa)...)
return cHpa, nil
}
return nil, err
}
func (cn *Container) getHpa(ns, name string) (*asv1.HorizontalPodAutoscaler, error) {
return cn.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(ns).Get(context.TODO(), name, metav1.GetOptions{})
func (cn *Container) getHpa(ctx context.Context, ns, name string) (*asv1.HorizontalPodAutoscaler, error) {
return cn.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(ns).Get(ctx, name, metav1.GetOptions{})
}
func (cn *Container) updateHpa(hpa *asv1.HorizontalPodAutoscaler) error {
_, err := cn.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(hpa.ObjectMeta.Namespace).Update(context.TODO(), hpa, metav1.UpdateOptions{})
func (cn *Container) updateHpa(ctx context.Context, hpa *asv1.HorizontalPodAutoscaler) error {
_, err := cn.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(hpa.ObjectMeta.Namespace).Update(ctx, hpa, metav1.UpdateOptions{})
return err
}
func (cn *Container) deleteHpa(ns string, name string) error {
return cn.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(ns).Delete(context.TODO(), name, metav1.DeleteOptions{})
func (cn *Container) deleteHpa(ctx context.Context, ns string, name string) error {
return cn.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(ns).Delete(ctx, name, metav1.DeleteOptions{})
}
+11 -8
View File
@@ -27,6 +27,7 @@ import (
"k8s.io/apimachinery/pkg/util/intstr"
fv1 "github.com/fission/fission/pkg/apis/core/v1"
otelUtils "github.com/fission/fission/pkg/utils/otel"
)
func (cn *Container) getSvPort(fn *fv1.Function) (port int32, err error) {
@@ -42,11 +43,12 @@ func (cn *Container) getSvPort(fn *fv1.Function) (port int32, err error) {
return fn.Spec.PodSpec.Containers[0].Ports[0].ContainerPort, nil
}
func (cn *Container) createOrGetSvc(fn *fv1.Function, deployLabels map[string]string, deployAnnotations map[string]string, svcName string, svcNamespace string) (*apiv1.Service, error) {
func (cn *Container) createOrGetSvc(ctx context.Context, fn *fv1.Function, deployLabels map[string]string, deployAnnotations map[string]string, svcName string, svcNamespace string) (*apiv1.Service, error) {
targetPort, err := cn.getSvPort(fn)
if err != nil {
return nil, err
}
logger := otelUtils.LoggerWithTraceID(ctx, cn.logger)
service := &apiv1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: svcName,
@@ -66,7 +68,7 @@ func (cn *Container) createOrGetSvc(fn *fv1.Function, deployLabels map[string]st
},
}
existingSvc, err := cn.kubernetesClient.CoreV1().Services(svcNamespace).Get(context.TODO(), svcName, metav1.GetOptions{})
existingSvc, err := cn.kubernetesClient.CoreV1().Services(svcNamespace).Get(ctx, svcName, metav1.GetOptions{})
if err == nil {
// to adopt orphan service
if existingSvc.Annotations[fv1.EXECUTOR_INSTANCEID_LABEL] != cn.instanceID {
@@ -75,29 +77,30 @@ func (cn *Container) createOrGetSvc(fn *fv1.Function, deployLabels map[string]st
existingSvc.Spec.Ports = service.Spec.Ports
existingSvc.Spec.Selector = service.Spec.Selector
existingSvc.Spec.Type = service.Spec.Type
existingSvc, err = cn.kubernetesClient.CoreV1().Services(svcNamespace).Update(context.TODO(), existingSvc, metav1.UpdateOptions{})
existingSvc, err = cn.kubernetesClient.CoreV1().Services(svcNamespace).Update(ctx, existingSvc, metav1.UpdateOptions{})
if err != nil {
cn.logger.Warn("error adopting service", zap.Error(err),
logger.Warn("error adopting service", zap.Error(err),
zap.String("service", svcName), zap.String("ns", svcNamespace))
return nil, err
}
}
return existingSvc, err
} else if k8s_err.IsNotFound(err) {
svc, err := cn.kubernetesClient.CoreV1().Services(svcNamespace).Create(context.TODO(), service, metav1.CreateOptions{})
svc, err := cn.kubernetesClient.CoreV1().Services(svcNamespace).Create(ctx, service, metav1.CreateOptions{})
if err != nil {
if k8s_err.IsAlreadyExists(err) {
svc, err = cn.kubernetesClient.CoreV1().Services(svcNamespace).Get(context.TODO(), svcName, metav1.GetOptions{})
svc, err = cn.kubernetesClient.CoreV1().Services(svcNamespace).Get(ctx, svcName, metav1.GetOptions{})
}
if err != nil {
return nil, err
}
}
otelUtils.SpanTrackEvent(ctx, "svcCreated", otelUtils.GetAttributesForSvc(svc)...)
return svc, nil
}
return nil, err
}
func (cn *Container) deleteSvc(ns string, name string) error {
return cn.kubernetesClient.CoreV1().Services(ns).Delete(context.TODO(), name, metav1.DeleteOptions{})
func (cn *Container) deleteSvc(ctx context.Context, ns string, name string) error {
return cn.kubernetesClient.CoreV1().Services(ns).Delete(ctx, name, metav1.DeleteOptions{})
}
+10 -10
View File
@@ -30,37 +30,37 @@ type ExecutorType interface {
Run(context.Context)
// GetTypeName returns the name of executor type
GetTypeName() fv1.ExecutorType
GetTypeName(context.Context) fv1.ExecutorType
// GetFuncSvc specializes function pod(s) and returns a service URL for the function.
GetFuncSvc(context.Context, *fv1.Function) (*fscache.FuncSvc, error)
// GetFuncSvcFromCache retrieves function service from cache.
GetFuncSvcFromCache(*fv1.Function) (*fscache.FuncSvc, error)
GetFuncSvcFromCache(context.Context, *fv1.Function) (*fscache.FuncSvc, error)
// GetFuncSvcFromPoolCache retrieves function service and number of active instances after filtering on requestsPerPod and CPULimit
GetFuncSvcFromPoolCache(fn *fv1.Function, requestsPerPod int) (*fscache.FuncSvc, int, error)
GetFuncSvcFromPoolCache(ctx context.Context, fn *fv1.Function, requestsPerPod int) (*fscache.FuncSvc, int, error)
// DeleteFuncSvcFromCache deletes function service entry in cache.
DeleteFuncSvcFromCache(*fscache.FuncSvc)
DeleteFuncSvcFromCache(context.Context, *fscache.FuncSvc)
// TapService updates the access time of function service entry to
// avoid idle pod reaper recycles pods.
TapService(serviceUrl string) error
TapService(ctx context.Context, serviceUrl string) error
// UnTapService updates the isActive to false
UnTapService(key string, svcHost string)
UnTapService(ctx context.Context, key string, svcHost string)
// IsValid returns true if a function service is valid. Different executor types
// use distinct ways to examine the function service.
IsValid(*fscache.FuncSvc) bool
IsValid(context.Context, *fscache.FuncSvc) bool
// RefreshFuncPods refreshes function pods if the secrets/configmaps pods reference to get updated.
RefreshFuncPods(*zap.Logger, fv1.Function) error
RefreshFuncPods(context.Context, *zap.Logger, fv1.Function) error
// AdoptOrphanResources adopts existing resources created by the deleted executor.
AdoptExistingResources()
AdoptExistingResources(context.Context)
// CleanupOldExecutorObjects cleans up resources created by old executor instances
CleanupOldExecutorObjects()
CleanupOldExecutorObjects(context.Context)
}
@@ -32,17 +32,18 @@ func (deploy *NewDeploy) EnvEventHandlers() k8sCache.ResourceEventHandlerFuncs {
UpdateFunc: func(oldObj interface{}, newObj interface{}) {
newEnv := newObj.(*fv1.Environment)
oldEnv := oldObj.(*fv1.Environment)
ctx := context.Background()
// Currently only an image update in environment calls for function's deployment recreation. In future there might be more attributes which would want to do it
if oldEnv.Spec.Runtime.Image != newEnv.Spec.Runtime.Image {
deploy.logger.Debug("Updating all function of the environment that changed, old env:", zap.Any("environment", oldEnv))
funcs := deploy.getEnvFunctions(&newEnv.ObjectMeta)
funcs := deploy.getEnvFunctions(ctx, &newEnv.ObjectMeta)
for _, f := range funcs {
function, err := deploy.fissionClient.CoreV1().Functions(f.ObjectMeta.Namespace).Get(context.TODO(), f.ObjectMeta.Name, metav1.GetOptions{})
function, err := deploy.fissionClient.CoreV1().Functions(f.ObjectMeta.Namespace).Get(ctx, f.ObjectMeta.Name, metav1.GetOptions{})
if err != nil {
deploy.logger.Error("Error getting function", zap.Error(err), zap.Any("function", function))
continue
}
err = deploy.updateFuncDeployment(function, newEnv)
err = deploy.updateFuncDeployment(ctx, function, newEnv)
if err != nil {
deploy.logger.Error("Error updating function", zap.Error(err), zap.Any("function", function))
continue
@@ -16,6 +16,8 @@ limitations under the License.
package newdeploy
import (
"context"
"go.uber.org/zap"
k8sCache "k8s.io/client-go/tools/cache"
@@ -29,9 +31,10 @@ func (deploy *NewDeploy) FunctionEventHandlers() k8sCache.ResourceEventHandlerFu
// and worker pattern to process items instead of moving process to another goroutine.
// example: https://github.com/kubernetes/kubernetes/blob/master/pkg/controller/job/job_controller.go
go func() {
ctx := context.Background()
fn := obj.(*fv1.Function)
deploy.logger.Debug("create deployment for function", zap.Any("fn", fn.ObjectMeta), zap.Any("fnspec", fn.Spec))
_, err := deploy.createFunction(fn)
_, err := deploy.createFunction(ctx, fn)
if err != nil {
deploy.logger.Error("error eager creating function",
zap.Error(err),
@@ -43,7 +46,8 @@ func (deploy *NewDeploy) FunctionEventHandlers() k8sCache.ResourceEventHandlerFu
DeleteFunc: func(obj interface{}) {
fn := obj.(*fv1.Function)
go func() {
err := deploy.deleteFunction(fn)
ctx := context.Background()
err := deploy.deleteFunction(ctx, fn)
if err != nil {
deploy.logger.Error("error deleting function",
zap.Error(err),
@@ -55,7 +59,8 @@ func (deploy *NewDeploy) FunctionEventHandlers() k8sCache.ResourceEventHandlerFu
oldFn := oldObj.(*fv1.Function)
newFn := newObj.(*fv1.Function)
go func() {
err := deploy.updateFunction(oldFn, newFn)
ctx := context.Background()
err := deploy.updateFunction(ctx, oldFn, newFn)
if err != nil {
deploy.logger.Error("error updating function",
zap.Error(err),
@@ -37,6 +37,7 @@ import (
fv1 "github.com/fission/fission/pkg/apis/core/v1"
"github.com/fission/fission/pkg/executor/util"
"github.com/fission/fission/pkg/utils"
otelUtils "github.com/fission/fission/pkg/utils/otel"
)
// Deployment Constants
@@ -45,7 +46,7 @@ const (
DeploymentVersion = "apps/v1"
)
func (deploy *NewDeploy) createOrGetDeployment(fn *fv1.Function, env *fv1.Environment,
func (deploy *NewDeploy) createOrGetDeployment(ctx context.Context, fn *fv1.Function, env *fv1.Environment,
deployName string, deployLabels map[string]string, deployAnnotations map[string]string, deployNamespace string) (*appsv1.Deployment, error) {
specializationTimeout := fn.Spec.InvokeStrategy.ExecutionStrategy.SpecializationTimeout
@@ -58,12 +59,12 @@ func (deploy *NewDeploy) createOrGetDeployment(fn *fv1.Function, env *fv1.Enviro
minScale = 1
}
deployment, err := deploy.getDeploymentSpec(fn, env, &minScale, deployName, deployNamespace, deployLabels, deployAnnotations)
deployment, err := deploy.getDeploymentSpec(ctx, fn, env, &minScale, deployName, deployNamespace, deployLabels, deployAnnotations)
if err != nil {
return nil, err
}
existingDepl, err := deploy.kubernetesClient.AppsV1().Deployments(deployNamespace).Get(context.TODO(), deployName, metav1.GetOptions{})
existingDepl, err := deploy.kubernetesClient.AppsV1().Deployments(deployNamespace).Get(ctx, deployName, metav1.GetOptions{})
if err == nil {
// Try to adopt orphan deployment created by the old executor.
if existingDepl.Annotations[fv1.EXECUTOR_INSTANCEID_LABEL] != deploy.instanceID {
@@ -75,7 +76,7 @@ func (deploy *NewDeploy) createOrGetDeployment(fn *fv1.Function, env *fv1.Enviro
// Update with the latest deployment spec. Kubernetes will trigger
// rolling update if spec is different from the one in the cluster.
existingDepl, err = deploy.kubernetesClient.AppsV1().Deployments(deployNamespace).Update(context.TODO(), existingDepl, metav1.UpdateOptions{})
existingDepl, err = deploy.kubernetesClient.AppsV1().Deployments(deployNamespace).Update(ctx, existingDepl, metav1.UpdateOptions{})
if err != nil {
deploy.logger.Warn("error adopting deploy", zap.Error(err),
zap.String("deploy", deployName), zap.String("ns", deployNamespace))
@@ -86,27 +87,27 @@ func (deploy *NewDeploy) createOrGetDeployment(fn *fv1.Function, env *fv1.Enviro
}
if *existingDepl.Spec.Replicas < minScale {
err = deploy.scaleDeployment(existingDepl.Namespace, existingDepl.Name, minScale)
err = deploy.scaleDeployment(ctx, existingDepl.Namespace, existingDepl.Name, minScale)
if err != nil {
deploy.logger.Error("error scaling up function deployment", zap.Error(err), zap.String("function", fn.ObjectMeta.Name))
return nil, err
}
}
if existingDepl.Status.AvailableReplicas < minScale {
existingDepl, err = deploy.waitForDeploy(existingDepl, minScale, specializationTimeout)
existingDepl, err = deploy.waitForDeploy(ctx, existingDepl, minScale, specializationTimeout)
}
return existingDepl, err
} else if k8s_err.IsNotFound(err) {
err := deploy.setupRBACObjs(deployNamespace, fn)
err := deploy.setupRBACObjs(ctx, deployNamespace, fn)
if err != nil {
return nil, err
}
depl, err := deploy.kubernetesClient.AppsV1().Deployments(deployNamespace).Create(context.TODO(), deployment, metav1.CreateOptions{})
depl, err := deploy.kubernetesClient.AppsV1().Deployments(deployNamespace).Create(ctx, deployment, metav1.CreateOptions{})
if err != nil {
if k8s_err.IsAlreadyExists(err) {
depl, err = deploy.kubernetesClient.AppsV1().Deployments(deployNamespace).Get(context.TODO(), deployName, metav1.GetOptions{})
depl, err = deploy.kubernetesClient.AppsV1().Deployments(deployNamespace).Get(ctx, deployName, metav1.GetOptions{})
}
if err != nil {
deploy.logger.Error("error while creating function deployment",
@@ -118,14 +119,14 @@ func (deploy *NewDeploy) createOrGetDeployment(fn *fv1.Function, env *fv1.Enviro
}
}
if minScale > 0 {
depl, err = deploy.waitForDeploy(depl, minScale, specializationTimeout)
depl, err = deploy.waitForDeploy(ctx, depl, minScale, specializationTimeout)
}
return depl, err
}
return nil, err
}
func (deploy *NewDeploy) setupRBACObjs(deployNamespace string, fn *fv1.Function) error {
func (deploy *NewDeploy) setupRBACObjs(ctx context.Context, deployNamespace string, fn *fv1.Function) error {
// create fetcher SA in this ns, if not already created
err := deploy.fetcherConfig.SetupServiceAccount(deploy.kubernetesClient, deployNamespace, fn.ObjectMeta)
if err != nil {
@@ -139,7 +140,7 @@ func (deploy *NewDeploy) setupRBACObjs(deployNamespace string, fn *fv1.Function)
}
// create a cluster role binding for the fetcher SA, if not already created, granting access to do a get on packages in any ns
err = utils.SetupRoleBinding(deploy.logger, deploy.kubernetesClient, fv1.PackageGetterRB, fn.Spec.Package.PackageRef.Namespace, fv1.PackageGetterCR, fv1.ClusterRole, fv1.FissionFetcherSA, deployNamespace)
err = utils.SetupRoleBinding(ctx, deploy.logger, deploy.kubernetesClient, fv1.PackageGetterRB, fn.Spec.Package.PackageRef.Namespace, fv1.PackageGetterCR, fv1.ClusterRole, fv1.FissionFetcherSA, deployNamespace)
if err != nil {
deploy.logger.Error("error creating role binding for function",
zap.Error(err),
@@ -150,7 +151,7 @@ func (deploy *NewDeploy) setupRBACObjs(deployNamespace string, fn *fv1.Function)
}
// create rolebinding in function namespace for fetcherSA.envNamespace to be able to get secrets and configmaps
err = utils.SetupRoleBinding(deploy.logger, deploy.kubernetesClient, fv1.SecretConfigMapGetterRB, fn.ObjectMeta.Namespace, fv1.SecretConfigMapGetterCR, fv1.ClusterRole, fv1.FissionFetcherSA, deployNamespace)
err = utils.SetupRoleBinding(ctx, deploy.logger, deploy.kubernetesClient, fv1.SecretConfigMapGetterRB, fn.ObjectMeta.Namespace, fv1.SecretConfigMapGetterCR, fv1.ClusterRole, fv1.FissionFetcherSA, deployNamespace)
if err != nil {
deploy.logger.Error("error creating role binding for function",
zap.Error(err),
@@ -166,21 +167,21 @@ func (deploy *NewDeploy) setupRBACObjs(deployNamespace string, fn *fv1.Function)
return nil
}
func (deploy *NewDeploy) updateDeployment(deployment *appsv1.Deployment, ns string) error {
_, err := deploy.kubernetesClient.AppsV1().Deployments(ns).Update(context.TODO(), deployment, metav1.UpdateOptions{})
func (deploy *NewDeploy) updateDeployment(ctx context.Context, deployment *appsv1.Deployment, ns string) error {
_, err := deploy.kubernetesClient.AppsV1().Deployments(ns).Update(ctx, deployment, metav1.UpdateOptions{})
return err
}
func (deploy *NewDeploy) deleteDeployment(ns string, name string) error {
func (deploy *NewDeploy) deleteDeployment(ctx context.Context, ns string, name string) error {
// DeletePropagationBackground deletes the object immediately and dependent are deleted later
// DeletePropagationForeground not advisable; it marks for deleteion and API can still serve those objects
deletePropagation := metav1.DeletePropagationBackground
return deploy.kubernetesClient.AppsV1().Deployments(ns).Delete(context.TODO(), name, metav1.DeleteOptions{
return deploy.kubernetesClient.AppsV1().Deployments(ns).Delete(ctx, name, metav1.DeleteOptions{
PropagationPolicy: &deletePropagation,
})
}
func (deploy *NewDeploy) getDeploymentSpec(fn *fv1.Function, env *fv1.Environment, targetReplicas *int32,
func (deploy *NewDeploy) getDeploymentSpec(ctx context.Context, fn *fv1.Function, env *fv1.Environment, targetReplicas *int32,
deployName string, deployNamespace string, deployLabels map[string]string, deployAnnotations map[string]string) (*appsv1.Deployment, error) {
replicas := int32(fn.Spec.InvokeStrategy.ExecutionStrategy.MinScale)
@@ -233,7 +234,7 @@ func (deploy *NewDeploy) getDeploymentSpec(fn *fv1.Function, env *fv1.Environmen
// rollback, set RevisionHistoryLimit to 0 to disable this feature.
revisionHistoryLimit := int32(0)
rvCount, err := referencedResourcesRVSum(deploy.kubernetesClient, fn.ObjectMeta.Namespace, fn.Spec.Secrets, fn.Spec.ConfigMaps)
rvCount, err := referencedResourcesRVSum(ctx, deploy.kubernetesClient, fn.ObjectMeta.Namespace, fn.Spec.Secrets, fn.Spec.ConfigMaps)
if err != nil {
return nil, err
}
@@ -366,12 +367,13 @@ func (deploy *NewDeploy) getResources(env *fv1.Environment, fn *fv1.Function) ap
return resources
}
func (deploy *NewDeploy) createOrGetHpa(hpaName string, execStrategy *fv1.ExecutionStrategy,
func (deploy *NewDeploy) createOrGetHpa(ctx context.Context, hpaName string, execStrategy *fv1.ExecutionStrategy,
depl *appsv1.Deployment, deployLabels map[string]string, deployAnnotations map[string]string) (*asv1.HorizontalPodAutoscaler, error) {
if depl == nil {
return nil, errors.New("failed to create HPA, found empty deployment")
}
logger := otelUtils.LoggerWithTraceID(ctx, deploy.logger)
minRepl := int32(execStrategy.MinScale)
if minRepl == 0 {
@@ -401,50 +403,52 @@ func (deploy *NewDeploy) createOrGetHpa(hpaName string, execStrategy *fv1.Execut
},
}
existingHpa, err := deploy.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(depl.ObjectMeta.Namespace).Get(context.TODO(), hpaName, metav1.GetOptions{})
existingHpa, err := deploy.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(depl.ObjectMeta.Namespace).Get(ctx, hpaName, metav1.GetOptions{})
if err == nil {
// to adopt orphan service
if existingHpa.Annotations[fv1.EXECUTOR_INSTANCEID_LABEL] != deploy.instanceID {
existingHpa.Annotations = hpa.Annotations
existingHpa.Labels = hpa.Labels
existingHpa.Spec = hpa.Spec
existingHpa, err = deploy.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(depl.ObjectMeta.Namespace).Update(context.TODO(), existingHpa, metav1.UpdateOptions{})
existingHpa, err = deploy.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(depl.ObjectMeta.Namespace).Update(ctx, existingHpa, metav1.UpdateOptions{})
if err != nil {
deploy.logger.Warn("error adopting HPA", zap.Error(err),
logger.Warn("error adopting HPA", zap.Error(err),
zap.String("HPA", hpaName), zap.String("ns", depl.ObjectMeta.Namespace))
return nil, err
}
}
return existingHpa, err
} else if k8s_err.IsNotFound(err) {
cHpa, err := deploy.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(depl.ObjectMeta.Namespace).Create(context.TODO(), hpa, metav1.CreateOptions{})
cHpa, err := deploy.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(depl.ObjectMeta.Namespace).Create(ctx, hpa, metav1.CreateOptions{})
if err != nil {
if k8s_err.IsAlreadyExists(err) {
cHpa, err = deploy.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(depl.ObjectMeta.Namespace).Get(context.TODO(), hpaName, metav1.GetOptions{})
cHpa, err = deploy.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(depl.ObjectMeta.Namespace).Get(ctx, hpaName, metav1.GetOptions{})
}
if err != nil {
return nil, err
}
}
otelUtils.SpanTrackEvent(ctx, "createdService", otelUtils.GetAttributesForHPA(cHpa)...)
return cHpa, nil
}
return nil, err
}
func (deploy *NewDeploy) getHpa(ns, name string) (*asv1.HorizontalPodAutoscaler, error) {
return deploy.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(ns).Get(context.TODO(), name, metav1.GetOptions{})
func (deploy *NewDeploy) getHpa(ctx context.Context, ns, name string) (*asv1.HorizontalPodAutoscaler, error) {
return deploy.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(ns).Get(ctx, name, metav1.GetOptions{})
}
func (deploy *NewDeploy) updateHpa(hpa *asv1.HorizontalPodAutoscaler) error {
_, err := deploy.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(hpa.ObjectMeta.Namespace).Update(context.TODO(), hpa, metav1.UpdateOptions{})
func (deploy *NewDeploy) updateHpa(ctx context.Context, hpa *asv1.HorizontalPodAutoscaler) error {
_, err := deploy.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(hpa.ObjectMeta.Namespace).Update(ctx, hpa, metav1.UpdateOptions{})
return err
}
func (deploy *NewDeploy) deleteHpa(ns string, name string) error {
return deploy.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(ns).Delete(context.TODO(), name, metav1.DeleteOptions{})
func (deploy *NewDeploy) deleteHpa(ctx context.Context, ns string, name string) error {
return deploy.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(ns).Delete(ctx, name, metav1.DeleteOptions{})
}
func (deploy *NewDeploy) createOrGetSvc(deployLabels map[string]string, deployAnnotations map[string]string, svcName string, svcNamespace string) (*apiv1.Service, error) {
func (deploy *NewDeploy) createOrGetSvc(ctx context.Context, deployLabels map[string]string, deployAnnotations map[string]string, svcName string, svcNamespace string) (*apiv1.Service, error) {
logger := otelUtils.LoggerWithTraceID(ctx, deploy.logger)
service := &apiv1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: svcName,
@@ -465,7 +469,7 @@ func (deploy *NewDeploy) createOrGetSvc(deployLabels map[string]string, deployAn
},
}
existingSvc, err := deploy.kubernetesClient.CoreV1().Services(svcNamespace).Get(context.TODO(), svcName, metav1.GetOptions{})
existingSvc, err := deploy.kubernetesClient.CoreV1().Services(svcNamespace).Get(ctx, svcName, metav1.GetOptions{})
if err == nil {
// to adopt orphan service
if existingSvc.Annotations[fv1.EXECUTOR_INSTANCEID_LABEL] != deploy.instanceID {
@@ -474,43 +478,47 @@ func (deploy *NewDeploy) createOrGetSvc(deployLabels map[string]string, deployAn
existingSvc.Spec.Ports = service.Spec.Ports
existingSvc.Spec.Selector = service.Spec.Selector
existingSvc.Spec.Type = service.Spec.Type
existingSvc, err = deploy.kubernetesClient.CoreV1().Services(svcNamespace).Update(context.TODO(), existingSvc, metav1.UpdateOptions{})
existingSvc, err = deploy.kubernetesClient.CoreV1().Services(svcNamespace).Update(ctx, existingSvc, metav1.UpdateOptions{})
if err != nil {
deploy.logger.Warn("error adopting service", zap.Error(err),
logger.Warn("error adopting service", zap.Error(err),
zap.String("service", svcName), zap.String("ns", svcNamespace))
return nil, err
}
}
return existingSvc, err
} else if k8s_err.IsNotFound(err) {
svc, err := deploy.kubernetesClient.CoreV1().Services(svcNamespace).Create(context.TODO(), service, metav1.CreateOptions{})
svc, err := deploy.kubernetesClient.CoreV1().Services(svcNamespace).Create(ctx, service, metav1.CreateOptions{})
if err != nil {
if k8s_err.IsAlreadyExists(err) {
svc, err = deploy.kubernetesClient.CoreV1().Services(svcNamespace).Get(context.TODO(), svcName, metav1.GetOptions{})
svc, err = deploy.kubernetesClient.CoreV1().Services(svcNamespace).Get(ctx, svcName, metav1.GetOptions{})
}
if err != nil {
return nil, err
}
}
otelUtils.SpanTrackEvent(ctx, "createdService", otelUtils.GetAttributesForSvc(svc)...)
return svc, nil
}
return nil, err
}
func (deploy *NewDeploy) deleteSvc(ns string, name string) error {
return deploy.kubernetesClient.CoreV1().Services(ns).Delete(context.TODO(), name, metav1.DeleteOptions{})
func (deploy *NewDeploy) deleteSvc(ctx context.Context, ns string, name string) error {
return deploy.kubernetesClient.CoreV1().Services(ns).Delete(ctx, name, metav1.DeleteOptions{})
}
func (deploy *NewDeploy) waitForDeploy(depl *appsv1.Deployment, replicas int32, specializationTimeout int) (latestDepl *appsv1.Deployment, err error) {
func (deploy *NewDeploy) waitForDeploy(ctx context.Context, depl *appsv1.Deployment, replicas int32, specializationTimeout int) (latestDepl *appsv1.Deployment, err error) {
oldStatus := depl.Status
otelUtils.SpanTrackEvent(ctx, "waitingForDeployment", otelUtils.GetAttributesForDeployment(depl)...)
// if no specializationTimeout is set, use default value
if specializationTimeout < fv1.DefaultSpecializationTimeOut {
specializationTimeout = fv1.DefaultSpecializationTimeOut
}
logger := otelUtils.LoggerWithTraceID(ctx, deploy.logger)
for i := 0; i < specializationTimeout; i++ {
latestDepl, err = deploy.kubernetesClient.AppsV1().Deployments(depl.ObjectMeta.Namespace).Get(context.TODO(), depl.Name, metav1.GetOptions{})
latestDepl, err = deploy.kubernetesClient.AppsV1().Deployments(depl.ObjectMeta.Namespace).Get(ctx, depl.Name, metav1.GetOptions{})
if err != nil {
return nil, err
}
@@ -518,12 +526,13 @@ func (deploy *NewDeploy) waitForDeploy(depl *appsv1.Deployment, replicas int32,
// use AvailableReplicas here is better than ReadyReplicas
// since the pods may not be able to serve network traffic yet.
if latestDepl.Status.AvailableReplicas >= replicas {
otelUtils.SpanTrackEvent(ctx, "deploymentAvailable", otelUtils.GetAttributesForDeployment(latestDepl)...)
return latestDepl, err
}
time.Sleep(time.Second)
}
deploy.logger.Error("Deployment provision failed within timeout window",
logger.Error("Deployment provision failed within timeout window",
zap.String("name", latestDepl.ObjectMeta.Name), zap.Any("old_status", oldStatus),
zap.Any("current_status", latestDepl.Status), zap.Int("timeout", specializationTimeout))
@@ -533,10 +542,10 @@ func (deploy *NewDeploy) waitForDeploy(depl *appsv1.Deployment, replicas int32,
}
// cleanupNewdeploy cleans all kubernetes objects related to function
func (deploy *NewDeploy) cleanupNewdeploy(ns string, name string) error {
func (deploy *NewDeploy) cleanupNewdeploy(ctx context.Context, ns string, name string) error {
result := &multierror.Error{}
err := deploy.deleteSvc(ns, name)
err := deploy.deleteSvc(ctx, ns, name)
if err != nil && !k8s_err.IsNotFound(err) {
deploy.logger.Error("error deleting service for newdeploy function",
zap.Error(err),
@@ -545,7 +554,7 @@ func (deploy *NewDeploy) cleanupNewdeploy(ns string, name string) error {
result = multierror.Append(result, err)
}
err = deploy.deleteHpa(ns, name)
err = deploy.deleteHpa(ctx, ns, name)
if err != nil && !k8s_err.IsNotFound(err) {
deploy.logger.Error("error deleting HPA for newdeploy function",
zap.Error(err),
@@ -554,7 +563,7 @@ func (deploy *NewDeploy) cleanupNewdeploy(ns string, name string) error {
result = multierror.Append(result, err)
}
err = deploy.deleteDeployment(ns, name)
err = deploy.deleteDeployment(ctx, ns, name)
if err != nil && !k8s_err.IsNotFound(err) {
deploy.logger.Error("error deleting deployment for newdeploy function",
zap.Error(err),
@@ -574,11 +583,11 @@ func (deploy *NewDeploy) cleanupNewdeploy(ns string, name string) error {
// identical way to get a value that can reflect resources changed without affecting by the time.
// To achieve this goal, the sum of the resource version of all referenced resources is a good fit for our
// scenario since the sum of the resource version is always the same as long as no resources changed.
func referencedResourcesRVSum(client *kubernetes.Clientset, namespace string, secrets []fv1.SecretReference, cfgmaps []fv1.ConfigMapReference) (int, error) {
func referencedResourcesRVSum(ctx context.Context, client *kubernetes.Clientset, namespace string, secrets []fv1.SecretReference, cfgmaps []fv1.ConfigMapReference) (int, error) {
rvCount := 0
if len(secrets) > 0 {
list, err := client.CoreV1().Secrets(namespace).List(context.TODO(), metav1.ListOptions{})
list, err := client.CoreV1().Secrets(namespace).List(ctx, metav1.ListOptions{})
if err != nil {
return 0, err
}
@@ -598,7 +607,7 @@ func referencedResourcesRVSum(client *kubernetes.Clientset, namespace string, se
}
if len(cfgmaps) > 0 {
list, err := client.CoreV1().ConfigMaps(namespace).List(context.TODO(), metav1.ListOptions{})
list, err := client.CoreV1().ConfigMaps(namespace).List(ctx, metav1.ListOptions{})
if err != nil {
return 0, err
}
@@ -28,14 +28,17 @@ import (
multierror "github.com/hashicorp/go-multierror"
"github.com/pkg/errors"
"go.uber.org/zap"
appsv1 "k8s.io/api/apps/v1"
autoscalingv1 "k8s.io/api/autoscaling/v1"
apiv1 "k8s.io/api/core/v1"
k8sErrs "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/labels"
k8sTypes "k8s.io/apimachinery/pkg/types"
appsinformers "k8s.io/client-go/informers/apps/v1"
coreinformers "k8s.io/client-go/informers/core/v1"
"k8s.io/client-go/kubernetes"
appslisters "k8s.io/client-go/listers/apps/v1"
corelisters "k8s.io/client-go/listers/core/v1"
k8sCache "k8s.io/client-go/tools/cache"
fv1 "github.com/fission/fission/pkg/apis/core/v1"
@@ -44,9 +47,11 @@ import (
"github.com/fission/fission/pkg/executor/fscache"
"github.com/fission/fission/pkg/executor/reaper"
fetcherConfig "github.com/fission/fission/pkg/fetcher/config"
finformerv1 "github.com/fission/fission/pkg/generated/informers/externalversions/core/v1"
"github.com/fission/fission/pkg/throttler"
"github.com/fission/fission/pkg/utils"
"github.com/fission/fission/pkg/utils/maps"
otelUtils "github.com/fission/fission/pkg/utils/otel"
)
var _ executortype.ExecutorType = &NewDeploy{}
@@ -69,13 +74,13 @@ type (
throttler *throttler.Throttler
funcInformer *k8sCache.SharedIndexInformer
envInformer *k8sCache.SharedIndexInformer
serviceInformer k8sCache.SharedIndexInformer
deploymentInformer k8sCache.SharedIndexInformer
defaultIdlePodReapTime time.Duration
deplLister appslisters.DeploymentLister
svcLister corelisters.ServiceLister
deplListerSynced k8sCache.InformerSynced
svcListerSynced k8sCache.InformerSynced
}
)
@@ -87,8 +92,10 @@ func MakeNewDeploy(
namespace string,
fetcherConfig *fetcherConfig.Config,
instanceID string,
funcInformer *k8sCache.SharedIndexInformer,
envInformer *k8sCache.SharedIndexInformer,
funcInformer finformerv1.FunctionInformer,
envInformer finformerv1.EnvironmentInformer,
deplInformer appsinformers.DeploymentInformer,
svcInformer coreinformers.ServiceInformer,
) (executortype.ExecutorType, error) {
enableIstio := false
if len(os.Getenv("ENABLE_ISTIO")) > 0 {
@@ -115,65 +122,64 @@ func MakeNewDeploy(
useIstio: enableIstio,
defaultIdlePodReapTime: 2 * time.Minute,
funcInformer: funcInformer,
envInformer: envInformer,
}
(*nd.funcInformer).AddEventHandler(nd.FunctionEventHandlers())
(*nd.envInformer).AddEventHandler(nd.EnvEventHandlers())
nd.deplLister = deplInformer.Lister()
nd.deplListerSynced = deplInformer.Informer().HasSynced
nd.svcLister = svcInformer.Lister()
nd.svcListerSynced = svcInformer.Informer().HasSynced
funcInformer.Informer().AddEventHandler(nd.FunctionEventHandlers())
envInformer.Informer().AddEventHandler(nd.EnvEventHandlers())
informerFactory, err := utils.GetInformerFactoryByExecutor(nd.kubernetesClient, fv1.ExecutorTypePoolmgr)
if err != nil {
return nil, err
}
nd.serviceInformer = informerFactory.Core().V1().Services().Informer()
nd.deploymentInformer = informerFactory.Apps().V1().Deployments().Informer()
return nd, nil
}
// Run start the function and environment controller along with an object reaper.
func (deploy *NewDeploy) Run(ctx context.Context) {
go deploy.serviceInformer.Run(ctx.Done())
go deploy.deploymentInformer.Run(ctx.Done())
if ok := k8sCache.WaitForCacheSync(ctx.Done(), deploy.deplListerSynced, deploy.svcListerSynced); !ok {
deploy.logger.Fatal("failed to wait for caches to sync")
}
go deploy.idleObjectReaper()
}
// GetTypeName returns the executor type name.
func (deploy *NewDeploy) GetTypeName() fv1.ExecutorType {
func (deploy *NewDeploy) GetTypeName(ctx context.Context) fv1.ExecutorType {
return fv1.ExecutorTypeNewdeploy
}
// GetFuncSvc returns a function service; error otherwise.
func (deploy *NewDeploy) GetFuncSvc(ctx context.Context, fn *fv1.Function) (*fscache.FuncSvc, error) {
// TODO: client-go doesn't support to pass in context.
// Once it supports context, we should change the signature of method.
// https://github.com/kubernetes/kubernetes/issues/46503
return deploy.createFunction(fn)
return deploy.createFunction(ctx, fn)
}
// GetFuncSvcFromCache returns a function service from cache; error otherwise.
func (deploy *NewDeploy) GetFuncSvcFromCache(fn *fv1.Function) (*fscache.FuncSvc, error) {
func (deploy *NewDeploy) GetFuncSvcFromCache(ctx context.Context, fn *fv1.Function) (*fscache.FuncSvc, error) {
otelUtils.SpanTrackEvent(ctx, "GetFuncSvcFromCache")
return deploy.fsCache.GetByFunction(&fn.ObjectMeta)
}
// DeleteFuncSvcFromCache deletes a function service from cache.
func (deploy *NewDeploy) DeleteFuncSvcFromCache(fsvc *fscache.FuncSvc) {
func (deploy *NewDeploy) DeleteFuncSvcFromCache(ctx context.Context, fsvc *fscache.FuncSvc) {
otelUtils.SpanTrackEvent(ctx, "DeleteFuncSvcFromCache")
deploy.fsCache.DeleteEntry(fsvc)
}
// UnTapService has not been implemented for NewDeployment.
func (deploy *NewDeploy) UnTapService(key string, svcHost string) {
func (deploy *NewDeploy) UnTapService(ctx context.Context, key string, svcHost string) {
// Not Implemented for NewDeployment. Will be used when support of concurrent specialization of same function is added.
}
// GetFuncSvcFromPoolCache has not been implemented for NewDeployment
func (deploy *NewDeploy) GetFuncSvcFromPoolCache(fn *fv1.Function, requestsPerPod int) (*fscache.FuncSvc, int, error) {
func (deploy *NewDeploy) GetFuncSvcFromPoolCache(ctx context.Context, fn *fv1.Function, requestsPerPod int) (*fscache.FuncSvc, int, error) {
// Not Implemented for NewDeployment. Will be used when support of concurrent specialization of same function is added.
return nil, 0, nil
}
// TapService makes a TouchByAddress request to the cache.
func (deploy *NewDeploy) TapService(svcHost string) error {
func (deploy *NewDeploy) TapService(ctx context.Context, svcHost string) error {
otelUtils.SpanTrackEvent(ctx, "TapService")
err := deploy.fsCache.TouchByAddress(svcHost)
if err != nil {
return err
@@ -181,67 +187,35 @@ func (deploy *NewDeploy) TapService(svcHost string) error {
return nil
}
func (deploy *NewDeploy) getServiceInfo(obj apiv1.ObjectReference) (*apiv1.Service, error) {
item, exists, err := utils.GetCachedItem(obj, deploy.serviceInformer)
if err != nil || !exists {
deploy.logger.Debug(
"Falling back to getting service info from k8s API -- this may cause performance issues for your function.",
zap.Bool("exists", exists),
zap.Error(err),
)
service, err := deploy.kubernetesClient.CoreV1().Services(obj.Namespace).Get(context.TODO(), obj.Name, metav1.GetOptions{})
return service, err
}
service := item.(*apiv1.Service)
return service, nil
}
func (deploy *NewDeploy) getDeploymentInfo(obj apiv1.ObjectReference) (*appsv1.Deployment, error) {
item, exists, err := utils.GetCachedItem(obj, deploy.deploymentInformer)
if err != nil || !exists {
deploy.logger.Debug(
"Falling back to getting deployment info from k8s API -- this may cause performance issues for your function.",
zap.Bool("exists", exists),
zap.Error(err),
)
deployment, err := deploy.kubernetesClient.AppsV1().Deployments(obj.Namespace).Get(context.TODO(), obj.Name, metav1.GetOptions{})
return deployment, err
}
deployment := item.(*appsv1.Deployment)
return deployment, nil
}
// IsValid does a get on the service address to ensure it's a valid service, then
// scale deployment to 1 replica if there are no available replicas for function.
// Return true if no error occurs, return false otherwise.
func (deploy *NewDeploy) IsValid(fsvc *fscache.FuncSvc) bool {
func (deploy *NewDeploy) IsValid(ctx context.Context, fsvc *fscache.FuncSvc) bool {
logger := otelUtils.LoggerWithTraceID(ctx, deploy.logger)
otelUtils.SpanTrackEvent(ctx, "IsValid", otelUtils.GetAttributesForFuncSvc(fsvc)...)
if len(strings.Split(fsvc.Address, ".")) == 0 {
deploy.logger.Error("address not found in function service")
logger.Error("address not found in function service")
return false
}
if len(fsvc.KubernetesObjects) == 0 {
deploy.logger.Error("no kubernetes object related to function", zap.String("function", fsvc.Function.Name))
logger.Error("no kubernetes object related to function", zap.String("function", fsvc.Function.Name))
return false
}
for _, obj := range fsvc.KubernetesObjects {
if strings.ToLower(obj.Kind) == "service" {
_, err := deploy.getServiceInfo(obj)
_, err := deploy.svcLister.Services(obj.Namespace).Get(obj.Name)
if err != nil {
if !k8sErrs.IsNotFound(err) {
deploy.logger.Error("error validating function service", zap.String("function", fsvc.Function.Name), zap.Error(err))
logger.Error("error validating function service", zap.String("function", fsvc.Function.Name), zap.Error(err))
}
return false
}
} else if strings.ToLower(obj.Kind) == "deployment" {
currentDeploy, err := deploy.getDeploymentInfo(obj)
currentDeploy, err := deploy.deplLister.Deployments(obj.Namespace).Get(obj.Name)
if err != nil {
if !k8sErrs.IsNotFound(err) {
deploy.logger.Error("error validating function deployment", zap.String("function", fsvc.Function.Name), zap.Error(err))
logger.Error("error validating function deployment", zap.String("function", fsvc.Function.Name), zap.Error(err))
}
return false
}
@@ -254,9 +228,9 @@ func (deploy *NewDeploy) IsValid(fsvc *fscache.FuncSvc) bool {
}
// RefreshFuncPods deletes pods related to the function so that new pods are replenished
func (deploy *NewDeploy) RefreshFuncPods(logger *zap.Logger, f fv1.Function) error {
func (deploy *NewDeploy) RefreshFuncPods(ctx context.Context, logger *zap.Logger, f fv1.Function) error {
env, err := deploy.fissionClient.CoreV1().Environments(f.Spec.Environment.Namespace).Get(context.TODO(), f.Spec.Environment.Name, metav1.GetOptions{})
env, err := deploy.fissionClient.CoreV1().Environments(f.Spec.Environment.Namespace).Get(ctx, f.Spec.Environment.Name, metav1.GetOptions{})
if err != nil {
return err
}
@@ -267,7 +241,7 @@ func (deploy *NewDeploy) RefreshFuncPods(logger *zap.Logger, f fv1.Function) err
UID: env.ObjectMeta.UID,
})
dep, err := deploy.kubernetesClient.AppsV1().Deployments(metav1.NamespaceAll).List(context.TODO(), metav1.ListOptions{
dep, err := deploy.kubernetesClient.AppsV1().Deployments(metav1.NamespaceAll).List(ctx, metav1.ListOptions{
LabelSelector: labels.Set(funcLabels).AsSelector().String(),
})
@@ -277,7 +251,7 @@ func (deploy *NewDeploy) RefreshFuncPods(logger *zap.Logger, f fv1.Function) err
// Ideally there should be only one deployment but for now we rely on label/selector to ensure that condition
for _, deployment := range dep.Items {
rvCount, err := referencedResourcesRVSum(deploy.kubernetesClient, deployment.Namespace, f.Spec.Secrets, f.Spec.ConfigMaps)
rvCount, err := referencedResourcesRVSum(ctx, deploy.kubernetesClient, deployment.Namespace, f.Spec.Secrets, f.Spec.ConfigMaps)
if err != nil {
return err
}
@@ -285,7 +259,7 @@ func (deploy *NewDeploy) RefreshFuncPods(logger *zap.Logger, f fv1.Function) err
patch := fmt.Sprintf(`{"spec" : {"template": {"spec":{"containers":[{"name": "%s", "env":[{"name": "%s", "value": "%v"}]}]}}}}`,
f.ObjectMeta.Name, fv1.ResourceVersionCount, rvCount)
_, err = deploy.kubernetesClient.AppsV1().Deployments(deployment.ObjectMeta.Namespace).Patch(context.TODO(), deployment.ObjectMeta.Name,
_, err = deploy.kubernetesClient.AppsV1().Deployments(deployment.ObjectMeta.Namespace).Patch(ctx, deployment.ObjectMeta.Name,
k8sTypes.StrategicMergePatchType,
[]byte(patch), metav1.PatchOptions{})
if err != nil {
@@ -296,8 +270,8 @@ func (deploy *NewDeploy) RefreshFuncPods(logger *zap.Logger, f fv1.Function) err
}
// AdoptExistingResources attempts to adopt resources for functions in all namespaces.
func (deploy *NewDeploy) AdoptExistingResources() {
fnList, err := deploy.fissionClient.CoreV1().Functions(metav1.NamespaceAll).List(context.TODO(), metav1.ListOptions{})
func (deploy *NewDeploy) AdoptExistingResources(ctx context.Context) {
fnList, err := deploy.fissionClient.CoreV1().Functions(metav1.NamespaceAll).List(ctx, metav1.ListOptions{})
if err != nil {
deploy.logger.Error("error getting function list", zap.Error(err))
return
@@ -312,7 +286,7 @@ func (deploy *NewDeploy) AdoptExistingResources() {
go func() {
defer wg.Done()
_, err = deploy.fnCreate(fn)
_, err = deploy.fnCreate(ctx, fn)
if err != nil {
deploy.logger.Warn("failed to adopt resources for function", zap.Error(err))
return
@@ -326,7 +300,7 @@ func (deploy *NewDeploy) AdoptExistingResources() {
}
// CleanupOldExecutorObjects cleans orphaned resources.
func (deploy *NewDeploy) CleanupOldExecutorObjects() {
func (deploy *NewDeploy) CleanupOldExecutorObjects(ctx context.Context) {
deploy.logger.Info("Newdeploy starts to clean orphaned resources", zap.String("instanceID", deploy.instanceID))
errs := &multierror.Error{}
@@ -334,17 +308,17 @@ func (deploy *NewDeploy) CleanupOldExecutorObjects() {
LabelSelector: labels.Set(map[string]string{fv1.EXECUTOR_TYPE: string(fv1.ExecutorTypeNewdeploy)}).AsSelector().String(),
}
err := reaper.CleanupHpa(deploy.logger, deploy.kubernetesClient, deploy.instanceID, listOpts)
err := reaper.CleanupHpa(ctx, deploy.logger, deploy.kubernetesClient, deploy.instanceID, listOpts)
if err != nil {
errs = multierror.Append(errs, err)
}
err = reaper.CleanupDeployments(deploy.logger, deploy.kubernetesClient, deploy.instanceID, listOpts)
err = reaper.CleanupDeployments(ctx, deploy.logger, deploy.kubernetesClient, deploy.instanceID, listOpts)
if err != nil {
errs = multierror.Append(errs, err)
}
err = reaper.CleanupServices(deploy.logger, deploy.kubernetesClient, deploy.instanceID, listOpts)
err = reaper.CleanupServices(ctx, deploy.logger, deploy.kubernetesClient, deploy.instanceID, listOpts)
if err != nil {
errs = multierror.Append(errs, err)
}
@@ -355,35 +329,37 @@ func (deploy *NewDeploy) CleanupOldExecutorObjects() {
}
}
func (deploy *NewDeploy) getEnvFunctions(m *metav1.ObjectMeta) []fv1.Function {
funcList, err := deploy.fissionClient.CoreV1().Functions(metav1.NamespaceAll).List(context.TODO(), metav1.ListOptions{})
func (deploy *NewDeploy) getEnvFunctions(ctx context.Context, m *metav1.ObjectMeta) []fv1.Function {
funcList, err := deploy.fissionClient.CoreV1().Functions(metav1.NamespaceAll).List(ctx, metav1.ListOptions{})
if err != nil {
deploy.logger.Error("Error getting functions for env", zap.Error(err), zap.Any("environment", m))
}
relatedFunctions := make([]fv1.Function, 0)
for _, f := range funcList.Items {
if (f.Spec.Environment.Name == m.Name) && (f.Spec.Environment.Namespace == m.Namespace) {
if (f.Spec.Environment.Name == m.Name) && (f.Spec.Environment.Namespace == m.Namespace) && f.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType == fv1.ExecutorTypeNewdeploy {
relatedFunctions = append(relatedFunctions, f)
}
}
return relatedFunctions
}
func (deploy *NewDeploy) createFunction(fn *fv1.Function) (*fscache.FuncSvc, error) {
func (deploy *NewDeploy) createFunction(ctx context.Context, fn *fv1.Function) (*fscache.FuncSvc, error) {
if fn.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType != fv1.ExecutorTypeNewdeploy {
return nil, nil
}
logger := otelUtils.LoggerWithTraceID(ctx, deploy.logger)
fsvcObj, err := deploy.throttler.RunOnce(string(fn.ObjectMeta.UID), func(ableToCreate bool) (interface{}, error) {
if ableToCreate {
return deploy.fnCreate(fn)
return deploy.fnCreate(ctx, fn)
}
return deploy.fsCache.GetByFunctionUID(fn.ObjectMeta.UID)
})
if err != nil {
e := "error creating k8s resources for function"
deploy.logger.Error(e,
logger.Error(e,
zap.Error(err),
zap.String("function_name", fn.ObjectMeta.Name),
zap.String("function_namespace", fn.ObjectMeta.Namespace))
@@ -392,26 +368,28 @@ func (deploy *NewDeploy) createFunction(fn *fv1.Function) (*fscache.FuncSvc, err
fsvc, ok := fsvcObj.(*fscache.FuncSvc)
if !ok {
deploy.logger.Panic("receive unknown object while creating function - expected pointer of function service object")
logger.Panic("receive unknown object while creating function - expected pointer of function service object")
}
otelUtils.SpanTrackEvent(ctx, "fnSvcResponse", otelUtils.GetAttributesForFuncSvc(fsvc)...)
return fsvc, err
}
func (deploy *NewDeploy) deleteFunction(fn *fv1.Function) error {
func (deploy *NewDeploy) deleteFunction(ctx context.Context, fn *fv1.Function) error {
if fn.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType != fv1.ExecutorTypeNewdeploy {
return nil
}
err := deploy.fnDelete(fn)
err := deploy.fnDelete(ctx, fn)
if err != nil {
err = errors.Wrapf(err, "error deleting kubernetes objects of function %v", fn.ObjectMeta)
}
return err
}
func (deploy *NewDeploy) fnCreate(fn *fv1.Function) (*fscache.FuncSvc, error) {
func (deploy *NewDeploy) fnCreate(ctx context.Context, fn *fv1.Function) (*fscache.FuncSvc, error) {
cleanupFunc := func(ns string, name string) {
err := deploy.cleanupNewdeploy(ns, name)
ctx := context.Background()
err := deploy.cleanupNewdeploy(ctx, ns, name)
if err != nil {
deploy.logger.Error("received error while cleaning function resources",
zap.String("namespace", ns), zap.String("name", name))
@@ -419,7 +397,7 @@ func (deploy *NewDeploy) fnCreate(fn *fv1.Function) (*fscache.FuncSvc, error) {
}
env, err := deploy.fissionClient.CoreV1().
Environments(fn.Spec.Environment.Namespace).
Get(context.TODO(), fn.Spec.Environment.Name, metav1.GetOptions{})
Get(ctx, fn.Spec.Environment.Name, metav1.GetOptions{})
if err != nil {
return nil, err
}
@@ -440,7 +418,7 @@ func (deploy *NewDeploy) fnCreate(fn *fv1.Function) (*fscache.FuncSvc, error) {
// Since newdeploy waits for pods of deployment to be ready,
// change the order of kubeObject creation (create service first,
// then deployment) to take advantage of waiting time.
svc, err := deploy.createOrGetSvc(deployLabels, deployAnnotations, objName, ns)
svc, err := deploy.createOrGetSvc(ctx, deployLabels, deployAnnotations, objName, ns)
if err != nil {
deploy.logger.Error("error creating service", zap.Error(err), zap.String("service", objName))
go cleanupFunc(ns, objName)
@@ -448,14 +426,14 @@ func (deploy *NewDeploy) fnCreate(fn *fv1.Function) (*fscache.FuncSvc, error) {
}
svcAddress := fmt.Sprintf("%v.%v", svc.Name, svc.Namespace)
depl, err := deploy.createOrGetDeployment(fn, env, objName, deployLabels, deployAnnotations, ns)
depl, err := deploy.createOrGetDeployment(ctx, fn, env, objName, deployLabels, deployAnnotations, ns)
if err != nil {
deploy.logger.Error("error creating deployment", zap.Error(err), zap.String("deployment", objName))
go cleanupFunc(ns, objName)
return nil, errors.Wrapf(err, "error creating deployment %v", objName)
}
hpa, err := deploy.createOrGetHpa(objName, &fn.Spec.InvokeStrategy.ExecutionStrategy, depl, deployLabels, deployAnnotations)
hpa, err := deploy.createOrGetHpa(ctx, objName, &fn.Spec.InvokeStrategy.ExecutionStrategy, depl, deployLabels, deployAnnotations)
if err != nil {
deploy.logger.Error("error creating HPA", zap.Error(err), zap.String("hpa", objName))
go cleanupFunc(ns, objName)
@@ -510,7 +488,7 @@ func (deploy *NewDeploy) fnCreate(fn *fv1.Function) (*fscache.FuncSvc, error) {
return fsvc, nil
}
func (deploy *NewDeploy) updateFunction(oldFn *fv1.Function, newFn *fv1.Function) error {
func (deploy *NewDeploy) updateFunction(ctx context.Context, oldFn *fv1.Function, newFn *fv1.Function) error {
if oldFn.ObjectMeta.ResourceVersion == newFn.ObjectMeta.ResourceVersion {
return nil
@@ -528,7 +506,7 @@ func (deploy *NewDeploy) updateFunction(oldFn *fv1.Function, newFn *fv1.Function
deploy.logger.Info("function does not use new deployment executor anymore, deleting resources",
zap.Any("function", newFn))
// IMP - pass the oldFn, as the new/modified function is not in cache
return deploy.deleteFunction(oldFn)
return deploy.deleteFunction(ctx, oldFn)
}
// Executor type changed to New Deployment from something else
@@ -537,7 +515,7 @@ func (deploy *NewDeploy) updateFunction(oldFn *fv1.Function, newFn *fv1.Function
deploy.logger.Info("function type changed to new deployment, creating resources",
zap.Any("old_function", oldFn.ObjectMeta),
zap.Any("new_function", newFn.ObjectMeta))
_, err := deploy.createFunction(newFn)
_, err := deploy.createFunction(ctx, newFn)
if err != nil {
deploy.updateStatus(oldFn, err, "error changing the function's type to newdeploy")
}
@@ -561,7 +539,7 @@ func (deploy *NewDeploy) updateFunction(oldFn *fv1.Function, newFn *fv1.Function
return err
}
hpa, err := deploy.getHpa(ns, fsvc.Name)
hpa, err := deploy.getHpa(ctx, ns, fsvc.Name)
if err != nil {
deploy.updateStatus(oldFn, err, "error getting HPA while updating function")
return err
@@ -587,7 +565,7 @@ func (deploy *NewDeploy) updateFunction(oldFn *fv1.Function, newFn *fv1.Function
}
if hpaChanged {
err := deploy.updateHpa(hpa)
err := deploy.updateHpa(ctx, hpa)
if err != nil {
deploy.updateStatus(oldFn, err, "error updating HPA while updating function")
return err
@@ -625,19 +603,18 @@ func (deploy *NewDeploy) updateFunction(oldFn *fv1.Function, newFn *fv1.Function
if deployChanged {
env, err := deploy.fissionClient.CoreV1().Environments(newFn.Spec.Environment.Namespace).
Get(context.TODO(), newFn.Spec.Environment.Name, metav1.GetOptions{})
Get(ctx, newFn.Spec.Environment.Name, metav1.GetOptions{})
if err != nil {
deploy.updateStatus(oldFn, err, "failed to get environment while updating function")
return err
}
return deploy.updateFuncDeployment(newFn, env)
return deploy.updateFuncDeployment(ctx, newFn, env)
}
return nil
}
func (deploy *NewDeploy) updateFuncDeployment(fn *fv1.Function, env *fv1.Environment) error {
func (deploy *NewDeploy) updateFuncDeployment(ctx context.Context, fn *fv1.Function, env *fv1.Environment) error {
fsvc, err := deploy.fsCache.GetByFunctionUID(fn.ObjectMeta.UID)
if err != nil {
err = errors.Wrapf(err, "error updating function due to unable to find function service cache: %v", fn)
@@ -656,7 +633,7 @@ func (deploy *NewDeploy) updateFuncDeployment(fn *fv1.Function, env *fv1.Environ
ns = fn.ObjectMeta.Namespace
}
existingDepl, err := deploy.kubernetesClient.AppsV1().Deployments(ns).Get(context.TODO(), fnObjName, metav1.GetOptions{})
existingDepl, err := deploy.kubernetesClient.AppsV1().Deployments(ns).Get(ctx, fnObjName, metav1.GetOptions{})
if err != nil {
return err
}
@@ -664,7 +641,7 @@ func (deploy *NewDeploy) updateFuncDeployment(fn *fv1.Function, env *fv1.Environ
// the resource version inside function packageRef is changed,
// so the content of fetchRequest in deployment cmd is different.
// Therefore, the deployment update will trigger a rolling update.
newDeployment, err := deploy.getDeploymentSpec(fn, env,
newDeployment, err := deploy.getDeploymentSpec(ctx, fn, env,
existingDepl.Spec.Replicas, // use current replicas instead of minscale in the ExecutionStrategy.
fnObjName, ns, deployLabels, deploy.getDeployAnnotations(fn.ObjectMeta, env.ObjectMeta))
if err != nil {
@@ -672,7 +649,7 @@ func (deploy *NewDeploy) updateFuncDeployment(fn *fv1.Function, env *fv1.Environ
return err
}
err = deploy.updateDeployment(newDeployment, ns)
err = deploy.updateDeployment(ctx, newDeployment, ns)
if err != nil {
deploy.updateStatus(fn, err, "failed to update deployment while updating function")
return err
@@ -681,7 +658,7 @@ func (deploy *NewDeploy) updateFuncDeployment(fn *fv1.Function, env *fv1.Environ
return nil
}
func (deploy *NewDeploy) fnDelete(fn *fv1.Function) error {
func (deploy *NewDeploy) fnDelete(ctx context.Context, fn *fv1.Function) error {
multierr := &multierror.Error{}
// GetByFunction uses resource version as part of cache key, however,
@@ -710,7 +687,7 @@ func (deploy *NewDeploy) fnDelete(fn *fv1.Function) error {
ns = fn.ObjectMeta.Namespace
}
err = deploy.cleanupNewdeploy(ns, objName)
err = deploy.cleanupNewdeploy(ctx, ns, objName)
multierr = multierror.Append(multierr, err)
return multierr.ErrorOrNil()
@@ -724,7 +701,16 @@ func (deploy *NewDeploy) getObjName(fn *fv1.Function) string {
if len(fn.ObjectMeta.Name)+len(fn.ObjectMeta.Namespace) < 35 {
functionMetadata = fn.ObjectMeta.Name + "-" + fn.ObjectMeta.Namespace
} else {
functionMetadata = fn.ObjectMeta.Name[:17] + "-" + fn.ObjectMeta.Namespace[:17]
if len(fn.ObjectMeta.Name) > 17 {
functionMetadata = fn.ObjectMeta.Name[:17]
} else {
functionMetadata = fn.ObjectMeta.Name
}
if len(fn.ObjectMeta.Namespace) > 17 {
functionMetadata = functionMetadata + "-" + fn.ObjectMeta.Namespace[:17]
} else {
functionMetadata = functionMetadata + "-" + fn.ObjectMeta.Namespace
}
}
// contructed name should be 63 characters long, as it is a valid k8s name
// functionMetadata should be 35 characters long, as we take 17 characters from functionUid
@@ -763,12 +749,12 @@ func (deploy *NewDeploy) updateStatus(fn *fv1.Function, err error, message strin
// idleObjectReaper reaps objects after certain idle time
func (deploy *NewDeploy) idleObjectReaper() {
ctx := context.Background()
pollSleep := 5 * time.Second
for {
time.Sleep(pollSleep)
envs, err := deploy.fissionClient.CoreV1().Environments(metav1.NamespaceAll).List(context.TODO(), metav1.ListOptions{})
envs, err := deploy.fissionClient.CoreV1().Environments(metav1.NamespaceAll).List(ctx, metav1.ListOptions{})
if err != nil {
deploy.logger.Fatal("failed to get environment list", zap.Error(err))
}
@@ -799,7 +785,7 @@ func (deploy *NewDeploy) idleObjectReaper() {
zap.String("function", fsvc.Name))
}
fn, err := deploy.fissionClient.CoreV1().Functions(fsvc.Function.Namespace).Get(context.TODO(), fsvc.Function.Name, metav1.GetOptions{})
fn, err := deploy.fissionClient.CoreV1().Functions(fsvc.Function.Namespace).Get(ctx, fsvc.Function.Name, metav1.GetOptions{})
if err != nil {
// Newdeploy manager handles the function delete event and clean cache/kubeobjs itself,
// so we ignore the not found error for functions with newdeploy executor type here.
@@ -830,7 +816,7 @@ func (deploy *NewDeploy) idleObjectReaper() {
}
currentDeploy, err := deploy.kubernetesClient.AppsV1().
Deployments(deployObj.Namespace).Get(context.TODO(), deployObj.Name, metav1.GetOptions{})
Deployments(deployObj.Namespace).Get(ctx, deployObj.Name, metav1.GetOptions{})
if err != nil {
deploy.logger.Error("error getting function deployment", zap.Error(err), zap.String("function", fsvc.Function.Name))
return
@@ -843,7 +829,7 @@ func (deploy *NewDeploy) idleObjectReaper() {
return
}
err = deploy.scaleDeployment(deployObj.Namespace, deployObj.Name, minScale)
err = deploy.scaleDeployment(ctx, deployObj.Namespace, deployObj.Name, minScale)
if err != nil {
deploy.logger.Error("error scaling down function deployment", zap.Error(err), zap.String("function", fsvc.Function.Name))
}
@@ -863,12 +849,18 @@ func getDeploymentObj(kubeobjs []apiv1.ObjectReference) *apiv1.ObjectReference {
return nil
}
func (deploy *NewDeploy) scaleDeployment(deplNS string, deplName string, replicas int32) error {
deploy.logger.Info("scaling deployment",
func (deploy *NewDeploy) scaleDeployment(ctx context.Context, deplNS string, deplName string, replicas int32) error {
otelUtils.SpanTrackEvent(ctx, "scaleDeployment", otelUtils.MapToAttributes(map[string]string{
"deployment-name": deplName,
"deployment-namespace": deplNS,
"replicas": fmt.Sprintf("%d", replicas),
})...)
logger := otelUtils.LoggerWithTraceID(ctx, deploy.logger)
logger.Info("scaling deployment",
zap.String("deployment", deplName),
zap.String("namespace", deplNS),
zap.Int32("replicas", replicas))
_, err := deploy.kubernetesClient.AppsV1().Deployments(deplNS).UpdateScale(context.TODO(), deplName, &autoscalingv1.Scale{
_, err := deploy.kubernetesClient.AppsV1().Deployments(deplNS).UpdateScale(ctx, deplName, &autoscalingv1.Scale{
ObjectMeta: metav1.ObjectMeta{
Name: deplName,
Namespace: deplNS,
@@ -0,0 +1,39 @@
/*
Copyright 2018 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package poolmgr
import fv1 "github.com/fission/fission/pkg/apis/core/v1"
func getEnvPoolSize(env *fv1.Environment) int32 {
var poolsize int32
if env.Spec.Version < 3 {
poolsize = 3
} else {
poolsize = int32(env.Spec.Poolsize)
}
return poolsize
}
func getSpecializedPodLabels(env *fv1.Environment) map[string]string {
specialPodLabels := make(map[string]string)
specialPodLabels[fv1.EXECUTOR_TYPE] = string(fv1.ExecutorTypePoolmgr)
specialPodLabels[fv1.ENVIRONMENT_NAME] = env.ObjectMeta.Name
specialPodLabels[fv1.ENVIRONMENT_NAMESPACE] = env.ObjectMeta.Namespace
specialPodLabels[fv1.ENVIRONMENT_UID] = string(env.ObjectMeta.UID)
specialPodLabels["managed"] = "false"
return specialPodLabels
}
@@ -37,9 +37,14 @@ func getIstioServiceLabels(fnName string) map[string]string {
}
}
func (gpm *GenericPoolManager) FunctionEventHandlers(kubernetesClient *kubernetes.Clientset, fissionfnNamespace string, istioEnabled bool) k8sCache.ResourceEventHandlerFuncs {
// FunctionEventHandlers provides handlers for function resource events.
// Based on function create/update/delete event, we create role binding
// for the secret/configmap access which is used by fetcher component.
// If istio is enabled, we create a service for the function.
func FunctionEventHandlers(logger *zap.Logger, kubernetesClient *kubernetes.Clientset, fissionfnNamespace string, istioEnabled bool) k8sCache.ResourceEventHandlerFuncs {
return k8sCache.ResourceEventHandlerFuncs{
AddFunc: func(obj interface{}) {
ctx := context.Background()
fn := obj.(*fv1.Function)
// Since istio only allows accessing pod through k8s service,
@@ -65,11 +70,11 @@ func (gpm *GenericPoolManager) FunctionEventHandlers(kubernetesClient *kubernete
// setup rolebinding is tried, if it fails, we don't return. we just log an error and move on, because :
// 1. not all functions have secrets and/or configmaps, so things will work without this rolebinding in that case.
// 2. on the contrary, when the route is tried, the env fetcher logs will show a 403 forbidden message and same will be relayed to executor.
err := utils.SetupRoleBinding(gpm.logger, kubernetesClient, fv1.SecretConfigMapGetterRB, fn.ObjectMeta.Namespace, fv1.SecretConfigMapGetterCR, fv1.ClusterRole, fv1.FissionFetcherSA, envNs)
err := utils.SetupRoleBinding(ctx, logger, kubernetesClient, fv1.SecretConfigMapGetterRB, fn.ObjectMeta.Namespace, fv1.SecretConfigMapGetterCR, fv1.ClusterRole, fv1.FissionFetcherSA, envNs)
if err != nil {
gpm.logger.Error("error creating rolebinding", zap.Error(err), zap.String("role_binding", fv1.SecretConfigMapGetterRB))
logger.Error("error creating rolebinding", zap.Error(err), zap.String("role_binding", fv1.SecretConfigMapGetterRB))
} else {
gpm.logger.Debug("successfully set up rolebinding for fetcher service account for function",
logger.Debug("successfully set up rolebinding for fetcher service account for function",
zap.String("service_account", fv1.FissionFetcherSA),
zap.String("service_account_namepsace", envNs),
zap.String("function_name", fn.ObjectMeta.Name),
@@ -116,9 +121,9 @@ func (gpm *GenericPoolManager) FunctionEventHandlers(kubernetesClient *kubernete
}
// create function istio service if it does not exist
_, err = kubernetesClient.CoreV1().Services(envNs).Create(context.TODO(), &svc, metav1.CreateOptions{})
_, err = kubernetesClient.CoreV1().Services(envNs).Create(ctx, &svc, metav1.CreateOptions{})
if err != nil && !kerrors.IsAlreadyExists(err) {
gpm.logger.Error("error creating istio service for function",
logger.Error("error creating istio service for function",
zap.Error(err),
zap.String("service_name", svcName),
zap.String("function_name", fn.ObjectMeta.Name),
@@ -128,6 +133,7 @@ func (gpm *GenericPoolManager) FunctionEventHandlers(kubernetesClient *kubernete
},
DeleteFunc: func(obj interface{}) {
ctx := context.Background()
fn := obj.(*fv1.Function)
fnExecutorType := fn.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType
@@ -143,9 +149,9 @@ func (gpm *GenericPoolManager) FunctionEventHandlers(kubernetesClient *kubernete
if istioEnabled {
svcName := utils.GetFunctionIstioServiceName(fn.ObjectMeta.Name, fn.ObjectMeta.Namespace)
// delete function istio service
err := kubernetesClient.CoreV1().Services(envNs).Delete(context.TODO(), svcName, metav1.DeleteOptions{})
err := kubernetesClient.CoreV1().Services(envNs).Delete(ctx, svcName, metav1.DeleteOptions{})
if err != nil && !kerrors.IsNotFound(err) {
gpm.logger.Error("error deleting istio service for function",
logger.Error("error deleting istio service for function",
zap.Error(err),
zap.String("service_name", svcName),
zap.String("function_name", fn.ObjectMeta.Name))
@@ -177,15 +183,15 @@ func (gpm *GenericPoolManager) FunctionEventHandlers(kubernetesClient *kubernete
if newFunc.Spec.Environment.Namespace != metav1.NamespaceDefault {
envNs = newFunc.Spec.Environment.Namespace
}
err := utils.SetupRoleBinding(gpm.logger, kubernetesClient, fv1.SecretConfigMapGetterRB,
ctx := context.Background()
err := utils.SetupRoleBinding(ctx, logger, kubernetesClient, fv1.SecretConfigMapGetterRB,
newFunc.ObjectMeta.Namespace, fv1.SecretConfigMapGetterCR, fv1.ClusterRole,
fv1.FissionFetcherSA, envNs)
if err != nil {
gpm.logger.Error("error creating rolebinding", zap.Error(err), zap.String("role_binding", fv1.SecretConfigMapGetterRB))
logger.Error("error creating rolebinding", zap.Error(err), zap.String("role_binding", fv1.SecretConfigMapGetterRB))
} else {
gpm.logger.Debug("successfully set up rolebinding for fetcher service account for function",
logger.Debug("successfully set up rolebinding for fetcher service account for function",
zap.String("service_account", fv1.FissionFetcherSA),
zap.String("service_account_namepsace", envNs),
zap.String("function_name", newFunc.ObjectMeta.Name),
+71 -212
View File
@@ -32,7 +32,6 @@ import (
"go.uber.org/zap"
appsv1 "k8s.io/api/apps/v1"
apiv1 "k8s.io/api/core/v1"
k8sErrs "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/api/resource"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/labels"
@@ -46,11 +45,11 @@ import (
fv1 "github.com/fission/fission/pkg/apis/core/v1"
"github.com/fission/fission/pkg/crd"
"github.com/fission/fission/pkg/executor/fscache"
"github.com/fission/fission/pkg/executor/util"
fetcherClient "github.com/fission/fission/pkg/fetcher/client"
fetcherConfig "github.com/fission/fission/pkg/fetcher/config"
"github.com/fission/fission/pkg/utils"
"github.com/fission/fission/pkg/utils/maps"
otelUtils "github.com/fission/fission/pkg/utils/otel"
)
type (
@@ -58,7 +57,6 @@ type (
GenericPool struct {
logger *zap.Logger
env *fv1.Environment
replicas int32 // num idle pods
deployment *appsv1.Deployment // kubernetes deployment
namespace string // namespace to keep our resources
functionNamespace string // fallback namespace for fission functions
@@ -88,13 +86,12 @@ func MakeGenericPool(
kubernetesClient *kubernetes.Clientset,
metricsClient *metricsclient.Clientset,
env *fv1.Environment,
initialReplicas int32,
namespace string,
functionNamespace string,
fsCache *fscache.FunctionServiceCache,
fetcherConfig *fetcherConfig.Config,
instanceID string,
enableIstio bool) (*GenericPool, error) {
enableIstio bool) *GenericPool {
gpLogger := logger.Named("generic_pool")
@@ -108,14 +105,13 @@ func MakeGenericPool(
zap.Duration("default", podReadyTimeout))
}
gpLogger.Info("creating pool", zap.Any("environment", env.ObjectMeta))
gpLogger.Info("creating pool", zap.Any("environment", env))
// TODO: in general we need to provide the user a way to configure pools. Initial
// replicas, autoscaling params, various timeouts, etc.
gp := &GenericPool{
logger: gpLogger,
env: env,
replicas: initialReplicas, // TODO make this an env param instead?
fissionClient: fissionClient,
kubernetesClient: kubernetesClient,
metricsClient: metricsClient,
@@ -134,39 +130,39 @@ func MakeGenericPool(
gp.runtimeImagePullPolicy = utils.GetImagePullPolicy(os.Getenv("RUNTIME_IMAGE_PULL_POLICY"))
// create fetcher SA in this ns, if not already created
err = fetcherConfig.SetupServiceAccount(gp.kubernetesClient, gp.namespace, nil)
if err != nil {
return nil, errors.Wrapf(err, "error creating fetcher service account in namespace %q", gp.namespace)
}
return gp
}
// Labels for generic deployment/RS/pods.
//gp.labelsForPool = gp.getDeployLabels()
func (gp *GenericPool) setup(ctx context.Context) error {
// create fetcher SA in this ns, if not already created
err := gp.fetcherConfig.SetupServiceAccount(gp.kubernetesClient, gp.namespace, nil)
if err != nil {
return errors.Wrapf(err, "error creating fetcher service account in namespace %q", gp.namespace)
}
// create the pool
err = gp.createPool()
err = gp.createPoolDeployment(ctx, gp.env)
if err != nil {
return nil, err
return err
}
gpLogger.Info("deployment created", zap.Any("environment", env.ObjectMeta))
go gp.startReadyPodController()
go gp.updateCPUUtilizationSvc()
return gp, nil
return nil
}
func (gp *GenericPool) getEnvironmentPoolLabels() map[string]string {
envLabels := maps.CopyStringMap(gp.env.ObjectMeta.Labels)
func (gp *GenericPool) getEnvironmentPoolLabels(env *fv1.Environment) map[string]string {
envLabels := maps.CopyStringMap(env.ObjectMeta.Labels)
envLabels[fv1.EXECUTOR_TYPE] = string(fv1.ExecutorTypePoolmgr)
envLabels[fv1.ENVIRONMENT_NAME] = gp.env.ObjectMeta.Name
envLabels[fv1.ENVIRONMENT_NAMESPACE] = gp.env.ObjectMeta.Namespace
envLabels[fv1.ENVIRONMENT_UID] = string(gp.env.ObjectMeta.UID)
envLabels[fv1.ENVIRONMENT_NAME] = env.ObjectMeta.Name
envLabels[fv1.ENVIRONMENT_NAMESPACE] = env.ObjectMeta.Namespace
envLabels[fv1.ENVIRONMENT_UID] = string(env.ObjectMeta.UID)
envLabels["managed"] = "true" // this allows us to easily find pods managed by the deployment
return envLabels
}
func (gp *GenericPool) getDeployAnnotations() map[string]string {
deployAnnotations := maps.CopyStringMap(gp.env.Annotations)
func (gp *GenericPool) getDeployAnnotations(env *fv1.Environment) map[string]string {
deployAnnotations := maps.CopyStringMap(env.Annotations)
deployAnnotations[fv1.EXECUTOR_INSTANCEID_LABEL] = gp.instanceID
return deployAnnotations
}
@@ -228,46 +224,49 @@ func (gp *GenericPool) updateCPUUtilizationSvc() {
// choosePod picks a ready pod from the pool and relabels it, waiting if necessary.
// returns the key and pod API object.
func (gp *GenericPool) choosePod(newLabels map[string]string) (string, *apiv1.Pod, error) {
func (gp *GenericPool) choosePod(ctx context.Context, newLabels map[string]string) (string, *apiv1.Pod, error) {
startTime := time.Now()
expoDelay := 100 * time.Millisecond
logger := otelUtils.LoggerWithTraceID(ctx, gp.logger)
for {
// Retries took too long, error out.
if time.Since(startTime) > gp.podReadyTimeout {
gp.logger.Error("timed out waiting for pod", zap.Any("labels", newLabels), zap.Duration("timeout", gp.podReadyTimeout))
logger.Error("timed out waiting for pod", zap.Any("labels", newLabels), zap.Duration("timeout", gp.podReadyTimeout))
return "", nil, errors.New("timeout: waited too long to get a ready pod")
}
var chosenPod *apiv1.Pod
var key string
otelUtils.SpanTrackEvent(ctx, "waitForPod", otelUtils.MapToAttributes(newLabels)...)
item, quit := gp.readyPodQueue.Get()
if quit {
gp.logger.Error("readypod controller is not running")
logger.Error("readypod controller is not running")
return "", nil, errors.New("readypod controller is not running")
}
key = item.(string)
gp.logger.Debug("got key from the queue", zap.String("key", key))
logger.Debug("got key from the queue", zap.String("key", key))
obj, exists, err := gp.readyPodInformer.GetIndexer().GetByKey(key)
if err != nil {
gp.logger.Error("fetching object from store failed", zap.String("key", key), zap.Error(err))
logger.Error("fetching object from store failed", zap.String("key", key), zap.Error(err))
return "", nil, err
}
if !exists {
gp.logger.Warn("pod deleted from store", zap.String("pod", key))
logger.Warn("pod deleted from store", zap.String("pod", key))
continue
}
if !utils.IsReadyPod(obj.(*apiv1.Pod)) {
gp.logger.Warn("pod not ready, pod will be checked again", zap.String("key", key), zap.Duration("delay", expoDelay))
logger.Warn("pod not ready, pod will be checked again", zap.String("key", key), zap.Duration("delay", expoDelay))
gp.readyPodQueue.Done(key)
gp.readyPodQueue.AddAfter(key, expoDelay)
expoDelay *= 2
continue
}
chosenPod = obj.(*apiv1.Pod).DeepCopy()
otelUtils.SpanTrackEvent(ctx, "foundPod", otelUtils.GetAttributesForPod(chosenPod)...)
if gp.env.Spec.AllowedFunctionsPerContainer != fv1.AllowedFunctionsPerContainerInfinite {
// Relabel. If the pod already got picked and
@@ -277,19 +276,20 @@ func (gp *GenericPool) choosePod(newLabels map[string]string) (string, *apiv1.Po
// Append executor instance id to pod annotations to
// indicate this pod is managed by this executor.
annotations := gp.getDeployAnnotations()
annotations := gp.getDeployAnnotations(gp.env)
annotationPatch, _ := json.Marshal(annotations)
patch := fmt.Sprintf(`{"metadata":{"annotations":%v, "labels":%v}}`, string(annotationPatch), string(labelPatch))
gp.logger.Info("relabel pod", zap.String("pod", patch))
newPod, err := gp.kubernetesClient.CoreV1().Pods(chosenPod.Namespace).Patch(context.TODO(), chosenPod.Name, k8sTypes.StrategicMergePatchType, []byte(patch), metav1.PatchOptions{})
logger.Info("relabel pod", zap.String("pod", patch))
newPod, err := gp.kubernetesClient.CoreV1().Pods(chosenPod.Namespace).Patch(ctx, chosenPod.Name, k8sTypes.StrategicMergePatchType, []byte(patch), metav1.PatchOptions{})
if err != nil {
gp.logger.Error("failed to relabel pod", zap.Error(err), zap.String("pod", chosenPod.Name), zap.Duration("delay", expoDelay))
logger.Error("failed to relabel pod", zap.Error(err), zap.String("pod", chosenPod.Name), zap.Duration("delay", expoDelay))
gp.readyPodQueue.Done(key)
gp.readyPodQueue.AddAfter(key, expoDelay)
expoDelay *= 2
continue
}
otelUtils.SpanTrackEvent(ctx, "podRelabel", otelUtils.GetAttributesForPod(chosenPod)...)
// With StrategicMergePatchType, the client-go sometimes return
// nil error and the labels & annotations remain the same.
@@ -308,7 +308,7 @@ func (gp *GenericPool) choosePod(newLabels map[string]string) (string, *apiv1.Po
}
}
gp.logger.Info("chose pod", zap.Any("labels", newLabels),
logger.Info("chose pod", zap.Any("labels", newLabels),
zap.String("pod", chosenPod.Name), zap.Duration("elapsed_time", time.Since(startTime)))
return key, chosenPod, nil
@@ -316,7 +316,7 @@ func (gp *GenericPool) choosePod(newLabels map[string]string) (string, *apiv1.Po
}
func (gp *GenericPool) labelsForFunction(metadata *metav1.ObjectMeta) map[string]string {
label := gp.getEnvironmentPoolLabels()
label := gp.getEnvironmentPoolLabels(gp.env)
label[fv1.FUNCTION_NAME] = metadata.Name
label[fv1.FUNCTION_UID] = string(metadata.UID)
label[fv1.FUNCTION_NAMESPACE] = metadata.Namespace // function CRD must stay within same namespace of environment CRD
@@ -373,6 +373,8 @@ func (gp *GenericPool) getFetcherURL(podIP string) string {
// (via fetcher), and calls the function-run container to load it, resulting in a
// specialized pod.
func (gp *GenericPool) specializePod(ctx context.Context, pod *apiv1.Pod, fn *fv1.Function) error {
logger := otelUtils.LoggerWithTraceID(ctx, gp.logger)
// for fetcher we don't need to create a service, just talk to the pod directly
podIP := pod.Status.PodIP
if len(podIP) == 0 {
@@ -386,11 +388,11 @@ func (gp *GenericPool) specializePod(ctx context.Context, pod *apiv1.Pod, fn *fv
// tell fetcher to get the function.
fetcherURL := gp.getFetcherURL(podIP)
gp.logger.Info("calling fetcher to copy function", zap.String("function", fn.ObjectMeta.Name), zap.String("url", fetcherURL))
logger.Info("calling fetcher to copy function", zap.String("function", fn.ObjectMeta.Name), zap.String("url", fetcherURL))
specializeReq := gp.fetcherConfig.NewSpecializeRequest(fn, gp.env)
gp.logger.Info("specializing pod", zap.String("function", fn.ObjectMeta.Name))
logger.Info("specializing pod", zap.String("function", fn.ObjectMeta.Name))
// Fetcher will download user function to share volume of pod, and
// invoke environment specialize api for pod specialization.
@@ -398,161 +400,14 @@ func (gp *GenericPool) specializePod(ctx context.Context, pod *apiv1.Pod, fn *fv
if err != nil {
return err
}
otelUtils.SpanTrackEvent(ctx, "specializedPod", otelUtils.GetAttributesForPod(pod)...)
return nil
}
// getPoolName returns a unique name of an environment
func (gp *GenericPool) getPoolName() string {
return strings.ToLower(fmt.Sprintf("poolmgr-%v-%v-%v", gp.env.ObjectMeta.Name, gp.env.ObjectMeta.Namespace, gp.env.ObjectMeta.ResourceVersion))
}
// A pool is a deployment of generic containers for an env. This
// creates the pool but doesn't wait for any pods to be ready.
func (gp *GenericPool) createPool() error {
deployLabels := gp.getEnvironmentPoolLabels()
deployAnnotations := gp.getDeployAnnotations()
// Use long terminationGracePeriodSeconds for connection draining in case that
// pod still runs user functions.
gracePeriodSeconds := int64(6 * 60)
if gp.env.Spec.TerminationGracePeriod > 0 {
gracePeriodSeconds = gp.env.Spec.TerminationGracePeriod
}
podAnnotations := gp.env.ObjectMeta.Annotations
if podAnnotations == nil {
podAnnotations = make(map[string]string)
}
// Here, we don't append executor instance-id to pod annotations
// to prevent unwanted rolling updates occur. Pool manager will
// append executor instance-id to pod annotations when a pod is chosen
// for function specialization.
if gp.useIstio && gp.env.Spec.AllowAccessToExternalNetwork {
podAnnotations["sidecar.istio.io/inject"] = "false"
}
podLabels := gp.env.ObjectMeta.Labels
if podLabels == nil {
podLabels = make(map[string]string)
}
for k, v := range deployLabels {
podLabels[k] = v
}
container, err := util.MergeContainer(&apiv1.Container{
Name: gp.env.ObjectMeta.Name,
Image: gp.env.Spec.Runtime.Image,
ImagePullPolicy: gp.runtimeImagePullPolicy,
TerminationMessagePath: "/dev/termination-log",
Resources: gp.env.Spec.Resources,
// Pod is removed from endpoints list for service when it's
// state became "Termination". We used preStop hook as the
// workaround for connection draining since pod maybe shutdown
// before grace period expires.
// https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods
// https://github.com/kubernetes/kubernetes/issues/47576#issuecomment-308900172
Lifecycle: &apiv1.Lifecycle{
PreStop: &apiv1.Handler{
Exec: &apiv1.ExecAction{
Command: []string{
"/bin/sleep",
fmt.Sprintf("%v", gracePeriodSeconds),
},
},
},
},
// https://istio.io/docs/setup/kubernetes/additional-setup/requirements/
Ports: []apiv1.ContainerPort{
{
Name: "http-fetcher",
ContainerPort: int32(8000),
},
{
Name: "http-env",
ContainerPort: int32(8888),
},
},
}, gp.env.Spec.Runtime.Container)
if err != nil {
return err
}
pod := apiv1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: podLabels,
Annotations: podAnnotations,
},
Spec: apiv1.PodSpec{
Containers: []apiv1.Container{*container},
ServiceAccountName: "fission-fetcher",
// TerminationGracePeriodSeconds should be equal to the
// sleep time of preStop to make sure that SIGTERM is sent
// to pod after 6 mins.
TerminationGracePeriodSeconds: &gracePeriodSeconds,
},
}
pod.Spec = *(util.ApplyImagePullSecret(gp.env.Spec.ImagePullSecret, pod.Spec))
deployment := &appsv1.Deployment{
ObjectMeta: metav1.ObjectMeta{
Name: gp.getPoolName(),
Labels: deployLabels,
Annotations: deployAnnotations,
},
Spec: appsv1.DeploymentSpec{
Replicas: &gp.replicas,
Selector: &metav1.LabelSelector{
MatchLabels: deployLabels,
},
Template: pod,
},
}
// Order of merging is important here - first fetcher, then containers and lastly pod spec
err = gp.fetcherConfig.AddFetcherToPodSpec(&deployment.Spec.Template.Spec, gp.env.ObjectMeta.Name)
if err != nil {
return err
}
if gp.env.Spec.Runtime.PodSpec != nil {
newPodSpec, err := util.MergePodSpec(&deployment.Spec.Template.Spec, gp.env.Spec.Runtime.PodSpec)
if err != nil {
return err
}
deployment.Spec.Template.Spec = *newPodSpec
}
depl, err := gp.kubernetesClient.AppsV1().Deployments(gp.namespace).Get(context.TODO(), deployment.Name, metav1.GetOptions{})
if err == nil {
if depl.Annotations[fv1.EXECUTOR_INSTANCEID_LABEL] != gp.instanceID {
deployment.Annotations[fv1.EXECUTOR_INSTANCEID_LABEL] = gp.instanceID
// Update with the latest deployment spec. Kubernetes will trigger
// rolling update if spec is different from the one in the cluster.
depl, err = gp.kubernetesClient.AppsV1().Deployments(gp.namespace).Update(context.TODO(), deployment, metav1.UpdateOptions{})
}
gp.deployment = depl
return err
} else if !k8sErrs.IsNotFound(err) {
gp.logger.Error("error getting deployment in kubernetes", zap.Error(err), zap.String("deployment", deployment.Name))
return err
}
depl, err = gp.kubernetesClient.AppsV1().Deployments(gp.namespace).Create(context.TODO(), deployment, metav1.CreateOptions{})
if err != nil {
gp.logger.Error("error creating deployment in kubernetes", zap.Error(err), zap.String("deployment", deployment.Name))
return err
}
gp.deployment = depl
return nil
}
func (gp *GenericPool) createSvc(name string, labels map[string]string) (*apiv1.Service, error) {
func (gp *GenericPool) createSvc(ctx context.Context, name string, labels map[string]string) (*apiv1.Service, error) {
otelUtils.SpanTrackEvent(ctx, "createSvc", otelUtils.MapToAttributes(map[string]string{
"name": name,
})...)
service := apiv1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: name,
@@ -570,12 +425,15 @@ func (gp *GenericPool) createSvc(name string, labels map[string]string) (*apiv1.
Selector: labels,
},
}
svc, err := gp.kubernetesClient.CoreV1().Services(gp.namespace).Create(context.TODO(), &service, metav1.CreateOptions{})
svc, err := gp.kubernetesClient.CoreV1().Services(gp.namespace).Create(ctx, &service, metav1.CreateOptions{})
return svc, err
}
func (gp *GenericPool) getFuncSvc(ctx context.Context, fn *fv1.Function) (*fscache.FuncSvc, error) {
gp.logger.Info("choosing pod from pool", zap.Any("function", fn.ObjectMeta))
logger := otelUtils.LoggerWithTraceID(ctx, gp.logger).With(zap.String("function", fn.ObjectMeta.Name), zap.String("functionNamespace", fn.ObjectMeta.Namespace),
zap.String("env", fn.Spec.Environment.Name), zap.String("envNamespace", fn.Spec.Environment.Namespace))
logger.Info("choosing pod from pool")
funcLabels := gp.labelsForFunction(&fn.ObjectMeta)
if gp.useIstio {
@@ -605,7 +463,7 @@ func (gp *GenericPool) getFuncSvc(ctx context.Context, fn *fv1.Function) (*fscac
"functionName": fn.ObjectMeta.Name,
"functionUid": string(fn.ObjectMeta.UID),
}
podList, err := gp.kubernetesClient.CoreV1().Pods(gp.namespace).List(context.TODO(), metav1.ListOptions{
podList, err := gp.kubernetesClient.CoreV1().Pods(gp.namespace).List(ctx, metav1.ListOptions{
LabelSelector: labels.Set(sel).AsSelector().String(),
})
if err != nil {
@@ -615,11 +473,11 @@ func (gp *GenericPool) getFuncSvc(ctx context.Context, fn *fv1.Function) (*fscac
// Remove old versions function pods
for _, pod := range podList.Items {
// Delete pod no matter what status it is
gp.kubernetesClient.CoreV1().Pods(gp.namespace).Delete(context.TODO(), pod.ObjectMeta.Name, metav1.DeleteOptions{}) //nolint errcheck
gp.kubernetesClient.CoreV1().Pods(gp.namespace).Delete(ctx, pod.ObjectMeta.Name, metav1.DeleteOptions{}) //nolint errcheck
}
}
key, pod, err := gp.choosePod(funcLabels)
key, pod, err := gp.choosePod(ctx, funcLabels)
if err != nil {
return nil, err
}
@@ -629,7 +487,7 @@ func (gp *GenericPool) getFuncSvc(ctx context.Context, fn *fv1.Function) (*fscac
gp.scheduleDeletePod(pod.ObjectMeta.Name)
return nil, err
}
gp.logger.Info("specialized pod", zap.String("pod", pod.ObjectMeta.Name), zap.Any("function", fn.ObjectMeta))
logger.Info("specialized pod", zap.String("pod", pod.ObjectMeta.Name), zap.String("podNamespace", pod.ObjectMeta.Namespace), zap.String("podIP", pod.Status.PodIP))
var svcHost string
if gp.useSvc && !gp.useIstio {
@@ -638,7 +496,7 @@ func (gp *GenericPool) getFuncSvc(ctx context.Context, fn *fv1.Function) (*fscac
svcName = fmt.Sprintf("%s-%v", svcName, fn.ObjectMeta.UID)
}
svc, err := gp.createSvc(svcName, funcLabels)
svc, err := gp.createSvc(ctx, svcName, funcLabels)
if err != nil {
gp.scheduleDeletePod(pod.ObjectMeta.Name)
return nil, err
@@ -658,25 +516,19 @@ func (gp *GenericPool) getFuncSvc(ctx context.Context, fn *fv1.Function) (*fscac
svcHost = fmt.Sprintf("%v:8888", pod.Status.PodIP)
}
otelUtils.SpanTrackEvent(ctx, "addFunctionLabel", otelUtils.GetAttributesForPod(pod)...)
// patch svc-host and resource version to the pod annotations for new executor to adopt the pod
patch := fmt.Sprintf(`{"metadata":{"annotations":{"%v":"%v","%v":"%v"}}}`,
fv1.ANNOTATION_SVC_HOST, svcHost, fv1.FUNCTION_RESOURCE_VERSION, fn.ObjectMeta.ResourceVersion)
p, err := gp.kubernetesClient.CoreV1().Pods(pod.Namespace).Patch(context.TODO(), pod.Name, k8sTypes.StrategicMergePatchType, []byte(patch), metav1.PatchOptions{})
p, err := gp.kubernetesClient.CoreV1().Pods(pod.Namespace).Patch(ctx, pod.Name, k8sTypes.StrategicMergePatchType, []byte(patch), metav1.PatchOptions{})
if err != nil {
// just log the error since it won't affect the function serving
gp.logger.Warn("error patching svc-host to pod", zap.Error(err),
logger.Warn("error patching svc-host to pod", zap.Error(err),
zap.String("pod", pod.Name), zap.String("ns", pod.Namespace))
} else {
pod = p
}
gp.logger.Info("specialized pod",
zap.String("pod", pod.ObjectMeta.Name),
zap.String("podNamespace", pod.ObjectMeta.Namespace),
zap.String("function", fn.ObjectMeta.Name),
zap.String("functionNamespace", fn.ObjectMeta.Namespace),
zap.String("specialization_host", svcHost))
kubeObjRefs := []apiv1.ObjectReference{
{
Kind: "pod",
@@ -696,10 +548,10 @@ func (gp *GenericPool) getFuncSvc(ctx context.Context, fn *fv1.Function) (*fscac
// set cpuLimit to 85th percentage of the cpuUsage
cpuLimit, err := gp.getPercent(cpuUsage, 0.85)
if err != nil {
gp.logger.Error("failed to get 85 of CPU usage", zap.Error(err))
logger.Error("failed to get 85 of CPU usage", zap.Error(err))
cpuLimit = cpuUsage
}
gp.logger.Debug("cpuLimit set to", zap.Any("cpulimit", cpuLimit))
logger.Debug("cpuLimit set to", zap.Any("cpulimit", cpuLimit))
m := fn.ObjectMeta // only cache necessary part
fsvc := &fscache.FuncSvc{
@@ -720,6 +572,13 @@ func (gp *GenericPool) getFuncSvc(ctx context.Context, fn *fv1.Function) (*fscac
gp.fsCache.IncreaseColdStarts(fn.ObjectMeta.Name, string(fn.ObjectMeta.UID))
logger.Info("added function service",
zap.String("pod", pod.ObjectMeta.Name),
zap.String("podNamespace", pod.ObjectMeta.Namespace),
zap.String("serviceHost", svcHost),
zap.String("podIP", pod.Status.PodIP))
otelUtils.SpanTrackEvent(ctx, "getFuncSvcComplete", otelUtils.GetAttributesForFuncSvc(fsvc)...)
return fsvc, nil
}
@@ -730,7 +589,7 @@ func (gp *GenericPool) getPercent(cpuUsage resource.Quantity, percentage float64
}
// destroys the pool -- the deployment, replicaset and pods
func (gp *GenericPool) destroy() error {
func (gp *GenericPool) destroy(ctx context.Context) error {
close(gp.stopReadyPodControllerCh)
deletePropagation := metav1.DeletePropagationBackground
@@ -739,7 +598,7 @@ func (gp *GenericPool) destroy() error {
}
err := gp.kubernetesClient.AppsV1().
Deployments(gp.namespace).Delete(context.TODO(), gp.deployment.ObjectMeta.Name, delOpt)
Deployments(gp.namespace).Delete(ctx, gp.deployment.ObjectMeta.Name, delOpt)
if err != nil {
gp.logger.Error("error destroying deployment",
zap.Error(err),
@@ -0,0 +1,243 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package poolmgr
import (
"context"
"fmt"
"strings"
"go.uber.org/zap"
appsv1 "k8s.io/api/apps/v1"
apiv1 "k8s.io/api/core/v1"
k8sErrs "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
fv1 "github.com/fission/fission/pkg/apis/core/v1"
"github.com/fission/fission/pkg/executor/util"
)
// getPoolName returns a unique name of an environment
func (gp *GenericPool) getPoolName(env *fv1.Environment) string {
// TODO: get rid of resource version here
return strings.ToLower(fmt.Sprintf("poolmgr-%v-%v-%v", env.ObjectMeta.Name, env.ObjectMeta.Namespace, env.ObjectMeta.ResourceVersion))
}
func (gp *GenericPool) genDeploymentMeta(env *fv1.Environment) metav1.ObjectMeta {
deployLabels := gp.getEnvironmentPoolLabels(env)
deployAnnotations := gp.getDeployAnnotations(env)
return metav1.ObjectMeta{
Name: gp.getPoolName(env),
Labels: deployLabels,
Annotations: deployAnnotations,
}
}
func (gp *GenericPool) genDeploymentSpec(env *fv1.Environment) (*appsv1.DeploymentSpec, error) {
deployLabels := gp.getEnvironmentPoolLabels(env)
// Use long terminationGracePeriodSeconds for connection draining in case that
// pod still runs user functions.
gracePeriodSeconds := int64(6 * 60)
if env.Spec.TerminationGracePeriod > 0 {
gracePeriodSeconds = env.Spec.TerminationGracePeriod
}
podAnnotations := env.ObjectMeta.Annotations
if podAnnotations == nil {
podAnnotations = make(map[string]string)
}
// Here, we don't append executor instance-id to pod annotations
// to prevent unwanted rolling updates occur. Pool manager will
// append executor instance-id to pod annotations when a pod is chosen
// for function specialization.
if gp.useIstio && env.Spec.AllowAccessToExternalNetwork {
podAnnotations["sidecar.istio.io/inject"] = "false"
}
podLabels := env.ObjectMeta.Labels
if podLabels == nil {
podLabels = make(map[string]string)
}
for k, v := range deployLabels {
podLabels[k] = v
}
container, err := util.MergeContainer(&apiv1.Container{
Name: env.ObjectMeta.Name,
Image: env.Spec.Runtime.Image,
ImagePullPolicy: gp.runtimeImagePullPolicy,
TerminationMessagePath: "/dev/termination-log",
Resources: env.Spec.Resources,
// Pod is removed from endpoints list for service when it's
// state became "Termination". We used preStop hook as the
// workaround for connection draining since pod maybe shutdown
// before grace period expires.
// https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods
// https://github.com/kubernetes/kubernetes/issues/47576#issuecomment-308900172
Lifecycle: &apiv1.Lifecycle{
PreStop: &apiv1.Handler{
Exec: &apiv1.ExecAction{
Command: []string{
"/bin/sleep",
fmt.Sprintf("%v", gracePeriodSeconds),
},
},
},
},
// https://istio.io/docs/setup/kubernetes/additional-setup/requirements/
Ports: []apiv1.ContainerPort{
{
Name: "http-fetcher",
ContainerPort: int32(8000),
},
{
Name: "http-env",
ContainerPort: int32(8888),
},
},
}, env.Spec.Runtime.Container)
if err != nil {
return nil, err
}
pod := apiv1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: podLabels,
Annotations: podAnnotations,
},
Spec: apiv1.PodSpec{
Containers: []apiv1.Container{*container},
ServiceAccountName: "fission-fetcher",
// TerminationGracePeriodSeconds should be equal to the
// sleep time of preStop to make sure that SIGTERM is sent
// to pod after 6 mins.
TerminationGracePeriodSeconds: &gracePeriodSeconds,
},
}
pod.Spec = *(util.ApplyImagePullSecret(env.Spec.ImagePullSecret, pod.Spec))
poolsize := getEnvPoolSize(env)
switch env.Spec.AllowedFunctionsPerContainer {
case fv1.AllowedFunctionsPerContainerInfinite:
poolsize = 1
}
deploymentSpec := appsv1.DeploymentSpec{
// TODO: fix this hardcoded value
Replicas: &poolsize,
Selector: &metav1.LabelSelector{
MatchLabels: deployLabels,
},
Template: pod,
}
// Order of merging is important here - first fetcher, then containers and lastly pod spec
err = gp.fetcherConfig.AddFetcherToPodSpec(&deploymentSpec.Template.Spec, env.ObjectMeta.Name)
if err != nil {
return nil, err
}
if env.Spec.Runtime.PodSpec != nil {
newPodSpec, err := util.MergePodSpec(&deploymentSpec.Template.Spec, env.Spec.Runtime.PodSpec)
if err != nil {
return nil, err
}
deploymentSpec.Template.Spec = *newPodSpec
}
return &deploymentSpec, nil
}
// A pool is a deployment of generic containers for an env. This
// creates the pool but doesn't wait for any pods to be ready.
func (gp *GenericPool) createPoolDeployment(ctx context.Context, env *fv1.Environment) error {
deploymentMeta := gp.genDeploymentMeta(env)
deploymentSpec, err := gp.genDeploymentSpec(env)
if err != nil {
return err
}
deployment := &appsv1.Deployment{
ObjectMeta: deploymentMeta,
Spec: *deploymentSpec,
}
depl, err := gp.kubernetesClient.AppsV1().Deployments(gp.namespace).Get(ctx, deployment.Name, metav1.GetOptions{})
if err == nil {
if depl.Annotations[fv1.EXECUTOR_INSTANCEID_LABEL] != gp.instanceID {
deployment.Annotations[fv1.EXECUTOR_INSTANCEID_LABEL] = gp.instanceID
// Update with the latest deployment spec. Kubernetes will trigger
// rolling update if spec is different from the one in the cluster.
depl, err = gp.kubernetesClient.AppsV1().Deployments(gp.namespace).Update(ctx, deployment, metav1.UpdateOptions{})
}
gp.deployment = depl
return err
} else if !k8sErrs.IsNotFound(err) {
gp.logger.Error("error getting deployment in kubernetes", zap.Error(err), zap.String("deployment", deployment.Name))
return err
}
depl, err = gp.kubernetesClient.AppsV1().Deployments(gp.namespace).Create(ctx, deployment, metav1.CreateOptions{})
if err != nil {
gp.logger.Error("error creating deployment in kubernetes", zap.Error(err), zap.String("deployment", deployment.Name))
return err
}
gp.deployment = depl
gp.logger.Info("deployment created", zap.String("deployment", depl.Name), zap.String("ns", depl.Namespace), zap.Any("environment", env))
return nil
}
func (gp *GenericPool) updatePoolDeployment(ctx context.Context, env *fv1.Environment) error {
logger := gp.logger.With(zap.String("env", env.Name), zap.String("namespace", env.Namespace))
if gp.env.ObjectMeta.ResourceVersion == env.ObjectMeta.ResourceVersion {
logger.Debug("env resource version matching with pool env")
return nil
}
newDeployment := gp.deployment.DeepCopy()
spec, err := gp.genDeploymentSpec(env)
if err != nil {
logger.Error("error generating deployment spec", zap.Error(err))
return err
}
newDeployment.Spec = *spec
deployMeta := gp.genDeploymentMeta(env)
deployMeta.Name = gp.deployment.Name
newDeployment.ObjectMeta = deployMeta
poolsize := getEnvPoolSize(env)
switch env.Spec.AllowedFunctionsPerContainer {
case fv1.AllowedFunctionsPerContainerInfinite:
poolsize = 1
}
newDeployment.Spec.Replicas = &poolsize
depl, err := gp.kubernetesClient.AppsV1().Deployments(gp.namespace).Update(ctx, newDeployment, metav1.UpdateOptions{})
if err != nil {
logger.Error("error updating deployment in kubernetes", zap.Error(err), zap.String("deployment", depl.Name))
return err
}
// possible concurrency issue here as
// gp.env and gp.deployment referenced at few places
// we can move update pool to gpm.service if required
gp.env = env
gp.deployment = depl
logger.Info("Updated deployment for pool", zap.String("deployment", depl.Name))
return nil
}
+137 -174
View File
@@ -27,6 +27,7 @@ import (
"time"
"github.com/hashicorp/go-multierror"
"go.opentelemetry.io/otel/attribute"
"go.uber.org/zap"
apiv1 "k8s.io/api/core/v1"
k8serrors "k8s.io/apimachinery/pkg/api/errors"
@@ -35,7 +36,10 @@ import (
"k8s.io/apimachinery/pkg/runtime"
k8sTypes "k8s.io/apimachinery/pkg/types"
"k8s.io/apimachinery/pkg/watch"
appsinformers "k8s.io/client-go/informers/apps/v1"
coreinformers "k8s.io/client-go/informers/core/v1"
"k8s.io/client-go/kubernetes"
corelisters "k8s.io/client-go/listers/core/v1"
k8sCache "k8s.io/client-go/tools/cache"
metricsclient "k8s.io/metrics/pkg/client/clientset/versioned"
@@ -46,7 +50,9 @@ import (
"github.com/fission/fission/pkg/executor/fscache"
"github.com/fission/fission/pkg/executor/reaper"
fetcherConfig "github.com/fission/fission/pkg/fetcher/config"
finformerv1 "github.com/fission/fission/pkg/generated/informers/externalversions/core/v1"
"github.com/fission/fission/pkg/utils"
otelUtils "github.com/fission/fission/pkg/utils/otel"
)
var _ executortype.ExecutorType = &GenericPoolManager{}
@@ -55,7 +61,7 @@ type requestType int
const (
GET_POOL requestType = iota
CLEANUP_POOLS
CLEANUP_POOL
)
type (
@@ -76,22 +82,26 @@ type (
enableIstio bool
fetcherConfig *fetcherConfig.Config
funcInformer *k8sCache.SharedIndexInformer
pkgInformer *k8sCache.SharedIndexInformer
// podLister can list/get pods from the shared informer's store
podLister corelisters.PodLister
podInformer k8sCache.SharedIndexInformer
// podListerSynced returns true if the pod store has been synced at least once.
podListerSynced k8sCache.InformerSynced
defaultIdlePodReapTime time.Duration
poolPodC *PoolPodController
}
request struct {
requestType
ctx context.Context
env *fv1.Environment
envList []fv1.Environment
responseChannel chan *response
}
response struct {
error
pool *GenericPool
pool *GenericPool
created bool
}
)
@@ -103,12 +113,27 @@ func MakeGenericPoolManager(
functionNamespace string,
fetcherConfig *fetcherConfig.Config,
instanceID string,
funcInformer *k8sCache.SharedIndexInformer,
pkgInformer *k8sCache.SharedIndexInformer,
funcInformer finformerv1.FunctionInformer,
pkgInformer finformerv1.PackageInformer,
envInformer finformerv1.EnvironmentInformer,
podInformer coreinformers.PodInformer,
rsInformer appsinformers.ReplicaSetInformer,
) (executortype.ExecutorType, error) {
gpmLogger := logger.Named("generic_pool_manager")
enableIstio := false
if len(os.Getenv("ENABLE_ISTIO")) > 0 {
istio, err := strconv.ParseBool(os.Getenv("ENABLE_ISTIO"))
if err != nil {
gpmLogger.Error("failed to parse 'ENABLE_ISTIO', set to false", zap.Error(err))
}
enableIstio = istio
}
poolPodC := NewPoolPodController(gpmLogger, kubernetesClient, functionNamespace,
enableIstio, funcInformer, pkgInformer, envInformer, rsInformer, podInformer)
gpm := &GenericPoolManager{
logger: gpmLogger,
pools: make(map[string]*GenericPool),
@@ -122,79 +147,80 @@ func MakeGenericPoolManager(
requestChannel: make(chan *request),
defaultIdlePodReapTime: 2 * time.Minute,
fetcherConfig: fetcherConfig,
funcInformer: funcInformer,
pkgInformer: pkgInformer,
enableIstio: enableIstio,
poolPodC: poolPodC,
}
gpm.podLister = podInformer.Lister()
gpm.podListerSynced = podInformer.Informer().HasSynced
go gpm.service()
if len(os.Getenv("ENABLE_ISTIO")) > 0 {
istio, err := strconv.ParseBool(os.Getenv("ENABLE_ISTIO"))
if err != nil {
gpmLogger.Error("failed to parse 'ENABLE_ISTIO', set to false", zap.Error(err))
}
gpm.enableIstio = istio
}
(*gpm.funcInformer).AddEventHandler(gpm.FunctionEventHandlers(gpm.kubernetesClient, gpm.namespace, gpm.enableIstio))
(*gpm.pkgInformer).AddEventHandler(gpm.PackageEventHandlers(gpm.kubernetesClient, gpm.namespace))
kubeInformerFactory, err := utils.GetInformerFactoryByExecutor(gpm.kubernetesClient, fv1.ExecutorTypePoolmgr)
if err != nil {
return nil, err
}
gpm.podInformer = kubeInformerFactory.Core().V1().Pods().Informer()
return gpm, nil
}
func (gpm *GenericPoolManager) Run(ctx context.Context) {
// eagerPoolCreator must run after CleanupOldExecutorObjects.
// Otherwise, the poolmanager may wrongly delete the deployment.
go gpm.eagerPoolCreator()
go gpm.podInformer.Run(ctx.Done())
if ok := k8sCache.WaitForCacheSync(ctx.Done(), gpm.podListerSynced); !ok {
gpm.logger.Fatal("failed to wait for caches to sync")
}
go gpm.service()
gpm.poolPodC.InjectGpm(gpm)
go gpm.WebsocketStartEventChecker(gpm.kubernetesClient)
go gpm.NoActiveConnectionEventChecker(gpm.kubernetesClient)
go gpm.idleObjectReaper()
go gpm.poolPodC.Run(ctx.Done())
}
func (gpm *GenericPoolManager) GetTypeName() fv1.ExecutorType {
func (gpm *GenericPoolManager) GetTypeName(ctx context.Context) fv1.ExecutorType {
return fv1.ExecutorTypePoolmgr
}
func (gpm *GenericPoolManager) GetFuncSvc(ctx context.Context, fn *fv1.Function) (*fscache.FuncSvc, error) {
otelUtils.SpanTrackEvent(ctx, "GetFuncSvc", otelUtils.GetAttributesForFunction(fn)...)
logger := otelUtils.LoggerWithTraceID(ctx, gpm.logger)
// from Func -> get Env
gpm.logger.Debug("getting environment for function", zap.String("function", fn.ObjectMeta.Name))
env, err := gpm.getFunctionEnv(fn)
logger.Debug("getting environment for function", zap.String("function", fn.ObjectMeta.Name))
env, err := gpm.getFunctionEnv(ctx, fn)
if err != nil {
return nil, err
}
pool, err := gpm.getPool(env)
pool, created, err := gpm.getPool(ctx, env)
if err != nil {
return nil, err
}
if created {
logger.Info("created pool for the environment", zap.String("env", env.ObjectMeta.Name), zap.String("namespace", gpm.namespace))
}
// from GenericPool -> get one function container
// (this also adds to the cache)
gpm.logger.Debug("getting function service from pool", zap.String("function", fn.ObjectMeta.Name))
logger.Debug("getting function service from pool", zap.String("function", fn.ObjectMeta.Name))
return pool.getFuncSvc(ctx, fn)
}
func (gpm *GenericPoolManager) GetFuncSvcFromCache(fn *fv1.Function) (*fscache.FuncSvc, error) {
func (gpm *GenericPoolManager) GetFuncSvcFromCache(ctx context.Context, fn *fv1.Function) (*fscache.FuncSvc, error) {
return nil, nil
}
func (gpm *GenericPoolManager) GetFuncSvcFromPoolCache(fn *fv1.Function, requestsPerPod int) (*fscache.FuncSvc, int, error) {
func (gpm *GenericPoolManager) GetFuncSvcFromPoolCache(ctx context.Context, fn *fv1.Function, requestsPerPod int) (*fscache.FuncSvc, int, error) {
otelUtils.SpanTrackEvent(ctx, "GetFuncSvcFromPoolCache", otelUtils.GetAttributesForFunction(fn)...)
return gpm.fsCache.GetFuncSvc(&fn.ObjectMeta, requestsPerPod)
}
func (gpm *GenericPoolManager) DeleteFuncSvcFromCache(fsvc *fscache.FuncSvc) {
func (gpm *GenericPoolManager) DeleteFuncSvcFromCache(ctx context.Context, fsvc *fscache.FuncSvc) {
otelUtils.SpanTrackEvent(ctx, "DeleteFuncSvcFromCache", otelUtils.GetAttributesForFuncSvc(fsvc)...)
gpm.fsCache.DeleteFunctionSvc(fsvc)
}
func (gpm *GenericPoolManager) UnTapService(key string, svcHost string) {
func (gpm *GenericPoolManager) UnTapService(ctx context.Context, key string, svcHost string) {
otelUtils.SpanTrackEvent(ctx, "UnTapService",
attribute.KeyValue{Key: "key", Value: attribute.StringValue(key)},
attribute.KeyValue{Key: "svcHost", Value: attribute.StringValue(svcHost)})
gpm.fsCache.MarkAvailable(key, svcHost)
}
func (gpm *GenericPoolManager) TapService(svcHost string) error {
func (gpm *GenericPoolManager) TapService(ctx context.Context, svcHost string) error {
otelUtils.SpanTrackEvent(ctx, "UnTapService",
attribute.KeyValue{Key: "svcHost", Value: attribute.StringValue(svcHost)})
err := gpm.fsCache.TouchByAddress(svcHost)
if err != nil {
return err
@@ -202,30 +228,13 @@ func (gpm *GenericPoolManager) TapService(svcHost string) error {
return nil
}
func (gpm *GenericPoolManager) getPodInfo(obj apiv1.ObjectReference) (*apiv1.Pod, error) {
store := gpm.podInformer.GetStore()
item, exists, err := store.Get(obj)
if err != nil || !exists {
item, exists, err = store.GetByKey(fmt.Sprintf("%s/%s", obj.Namespace, obj.Name))
}
if err != nil || !exists {
gpm.logger.Debug("Falling back to getting pod info from k8s API -- this may cause performance issues for your function.")
pod, err := gpm.kubernetesClient.CoreV1().Pods(obj.Namespace).Get(context.TODO(), obj.Name, metav1.GetOptions{})
return pod, err
}
pod := item.(*apiv1.Pod)
return pod, nil
}
// IsValid checks if pod is not deleted and that it has the address passed as the argument. Also checks that all the
// containers in it are reporting a ready status for the healthCheck.
func (gpm *GenericPoolManager) IsValid(fsvc *fscache.FuncSvc) bool {
func (gpm *GenericPoolManager) IsValid(ctx context.Context, fsvc *fscache.FuncSvc) bool {
otelUtils.SpanTrackEvent(ctx, "IsValid", otelUtils.GetAttributesForFuncSvc(fsvc)...)
for _, obj := range fsvc.KubernetesObjects {
if strings.ToLower(obj.Kind) == "pod" {
pod, err := gpm.getPodInfo(obj)
pod, err := gpm.podLister.Pods(obj.Namespace).Get(obj.Name)
if err == nil && utils.IsReadyPod(pod) {
// Normally, the address format is http://[pod-ip]:[port], however, if the
// Istio is enabled the address format changes to http://[svc-name]:[port].
@@ -246,18 +255,22 @@ func (gpm *GenericPoolManager) IsValid(fsvc *fscache.FuncSvc) bool {
return false
}
func (gpm *GenericPoolManager) RefreshFuncPods(logger *zap.Logger, f fv1.Function) error {
func (gpm *GenericPoolManager) RefreshFuncPods(ctx context.Context, logger *zap.Logger, f fv1.Function) error {
env, err := gpm.fissionClient.CoreV1().Environments(f.Spec.Environment.Namespace).Get(context.TODO(), f.Spec.Environment.Name, metav1.GetOptions{})
env, err := gpm.fissionClient.CoreV1().Environments(f.Spec.Environment.Namespace).Get(ctx, f.Spec.Environment.Name, metav1.GetOptions{})
if err != nil {
return err
}
gp, err := gpm.getPool(env)
gp, created, err := gpm.getPool(ctx, env)
if err != nil {
return err
}
if created {
gpm.logger.Info("created pool for the environment", zap.String("env", env.ObjectMeta.Name), zap.String("namespace", gpm.namespace))
}
funcSvc, err := gp.fsCache.GetByFunction(&f.ObjectMeta)
if err != nil {
return err
@@ -267,7 +280,7 @@ func (gpm *GenericPoolManager) RefreshFuncPods(logger *zap.Logger, f fv1.Functio
funcLabels := gp.labelsForFunction(&f.ObjectMeta)
podList, err := gpm.kubernetesClient.CoreV1().Pods(metav1.NamespaceAll).List(context.TODO(), metav1.ListOptions{
podList, err := gpm.kubernetesClient.CoreV1().Pods(metav1.NamespaceAll).List(ctx, metav1.ListOptions{
LabelSelector: labels.Set(funcLabels).AsSelector().String(),
})
@@ -276,7 +289,7 @@ func (gpm *GenericPoolManager) RefreshFuncPods(logger *zap.Logger, f fv1.Functio
}
for _, po := range podList.Items {
err := gpm.kubernetesClient.CoreV1().Pods(po.ObjectMeta.Namespace).Delete(context.TODO(), po.ObjectMeta.Name, metav1.DeleteOptions{})
err := gpm.kubernetesClient.CoreV1().Pods(po.ObjectMeta.Namespace).Delete(ctx, po.ObjectMeta.Name, metav1.DeleteOptions{})
if k8serrors.IsNotFound(err) {
return nil
}
@@ -288,8 +301,8 @@ func (gpm *GenericPoolManager) RefreshFuncPods(logger *zap.Logger, f fv1.Functio
return nil
}
func (gpm *GenericPoolManager) AdoptExistingResources() {
envs, err := gpm.fissionClient.CoreV1().Environments(metav1.NamespaceAll).List(context.TODO(), metav1.ListOptions{})
func (gpm *GenericPoolManager) AdoptExistingResources(ctx context.Context) {
envs, err := gpm.fissionClient.CoreV1().Environments(metav1.NamespaceAll).List(ctx, metav1.ListOptions{})
if err != nil {
gpm.logger.Error("error getting environment list", zap.Error(err))
return
@@ -301,14 +314,17 @@ func (gpm *GenericPoolManager) AdoptExistingResources() {
for i := range envs.Items {
env := envs.Items[i]
if gpm.getEnvPoolsize(&env) > 0 {
if getEnvPoolSize(&env) > 0 {
wg.Add(1)
go func() {
defer wg.Done()
_, err := gpm.getPool(&env)
_, created, err := gpm.getPool(ctx, &env)
if err != nil {
gpm.logger.Error("adopt pool failed", zap.Error(err))
}
if created {
gpm.logger.Info("created pool for the environment", zap.String("env", env.ObjectMeta.Name), zap.String("namespace", gpm.namespace))
}
}()
}
@@ -321,7 +337,7 @@ func (gpm *GenericPoolManager) AdoptExistingResources() {
fv1.EXECUTOR_TYPE: string(fv1.ExecutorTypePoolmgr),
}
podList, err := gpm.kubernetesClient.CoreV1().Pods(metav1.NamespaceAll).List(context.TODO(), metav1.ListOptions{
podList, err := gpm.kubernetesClient.CoreV1().Pods(metav1.NamespaceAll).List(ctx, metav1.ListOptions{
LabelSelector: labels.Set(l).AsSelector().String(),
})
@@ -344,7 +360,7 @@ func (gpm *GenericPoolManager) AdoptExistingResources() {
time.Sleep(time.Duration(rand.Intn(30)) * time.Millisecond)
patch := fmt.Sprintf(`{"metadata":{"annotations":{"%v":"%v"}}}`, fv1.EXECUTOR_INSTANCEID_LABEL, gpm.instanceID)
pod, err = gpm.kubernetesClient.CoreV1().Pods(pod.Namespace).Patch(context.TODO(), pod.Name, k8sTypes.StrategicMergePatchType, []byte(patch), metav1.PatchOptions{})
pod, err = gpm.kubernetesClient.CoreV1().Pods(pod.Namespace).Patch(ctx, pod.Name, k8sTypes.StrategicMergePatchType, []byte(patch), metav1.PatchOptions{})
if err != nil {
// just log the error since it won't affect the function serving
gpm.logger.Warn("error patching executor instance ID of pod", zap.Error(err),
@@ -417,7 +433,7 @@ func (gpm *GenericPoolManager) AdoptExistingResources() {
wg.Wait()
}
func (gpm *GenericPoolManager) CleanupOldExecutorObjects() {
func (gpm *GenericPoolManager) CleanupOldExecutorObjects(ctx context.Context) {
gpm.logger.Info("Poolmanager starts to clean orphaned resources", zap.String("instanceID", gpm.instanceID))
errs := &multierror.Error{}
@@ -425,12 +441,12 @@ func (gpm *GenericPoolManager) CleanupOldExecutorObjects() {
LabelSelector: labels.Set(map[string]string{fv1.EXECUTOR_TYPE: string(fv1.ExecutorTypePoolmgr)}).AsSelector().String(),
}
err := reaper.CleanupDeployments(gpm.logger, gpm.kubernetesClient, gpm.instanceID, listOpts)
err := reaper.CleanupDeployments(ctx, gpm.logger, gpm.kubernetesClient, gpm.instanceID, listOpts)
if err != nil {
errs = multierror.Append(errs, err)
}
err = reaper.CleanupPods(gpm.logger, gpm.kubernetesClient, gpm.instanceID, listOpts)
err = reaper.CleanupPods(ctx, gpm.logger, gpm.kubernetesClient, gpm.instanceID, listOpts)
if err != nil {
errs = multierror.Append(errs, err)
}
@@ -448,73 +464,77 @@ func (gpm *GenericPoolManager) service() {
case GET_POOL:
// just because they are missing in the cache, we end up creating another duplicate pool.
var err error
pool, ok := gpm.pools[crd.CacheKey(&req.env.ObjectMeta)]
created := false
pool, ok := gpm.pools[crd.CacheKeyUID(&req.env.ObjectMeta)]
if !ok {
poolsize := gpm.getEnvPoolsize(req.env)
switch req.env.Spec.AllowedFunctionsPerContainer {
case fv1.AllowedFunctionsPerContainerInfinite:
poolsize = 1
}
// To support backward compatibility, if envs are created in default ns, we go ahead
// and create pools in fission-function ns as earlier.
ns := gpm.namespace
if req.env.ObjectMeta.Namespace != metav1.NamespaceDefault {
ns = req.env.ObjectMeta.Namespace
}
pool, err = MakeGenericPool(gpm.logger,
gpm.fissionClient, gpm.kubernetesClient, gpm.metricsClient, req.env, poolsize,
ns, gpm.namespace, gpm.fsCache, gpm.fetcherConfig, gpm.instanceID, gpm.enableIstio)
pool = MakeGenericPool(gpm.logger, gpm.fissionClient, gpm.kubernetesClient,
gpm.metricsClient, req.env, ns, gpm.namespace, gpm.fsCache,
gpm.fetcherConfig, gpm.instanceID, gpm.enableIstio)
err = pool.setup(req.ctx)
if err != nil {
req.responseChannel <- &response{error: err}
continue
}
gpm.pools[crd.CacheKey(&req.env.ObjectMeta)] = pool
gpm.pools[crd.CacheKeyUID(&req.env.ObjectMeta)] = pool
created = true
}
req.responseChannel <- &response{pool: pool}
case CLEANUP_POOLS:
latestEnvPoolsize := make(map[string]int)
for _, env := range req.envList {
latestEnvPoolsize[crd.CacheKey(&env.ObjectMeta)] = int(gpm.getEnvPoolsize(&env))
req.responseChannel <- &response{pool: pool, created: created}
case CLEANUP_POOL:
env := *req.env
gpm.logger.Info("destroying pool",
zap.String("environment", env.ObjectMeta.Name),
zap.String("namespace", env.ObjectMeta.Namespace))
key := crd.CacheKeyUID(&req.env.ObjectMeta)
pool, ok := gpm.pools[key]
if !ok {
gpm.logger.Error("Could not find pool", zap.String("environment", env.ObjectMeta.Name), zap.String("namespace", env.ObjectMeta.Namespace))
return
}
for key, pool := range gpm.pools {
poolsize, ok := latestEnvPoolsize[key]
if !ok || poolsize == 0 {
// Env no longer exists or pool size changed to zero
gpm.logger.Info("destroying generic pool", zap.Any("environment", pool.env.ObjectMeta))
delete(gpm.pools, key)
// and delete the pool asynchronously.
go pool.destroy() //nolint errcheck
}
delete(gpm.pools, key)
err := pool.destroy(req.ctx)
if err != nil {
gpm.logger.Error("failed to destroy pool",
zap.String("environment", env.ObjectMeta.Name),
zap.String("namespace", env.ObjectMeta.Namespace),
zap.Error(err))
}
// no response, caller doesn't wait
}
}
}
func (gpm *GenericPoolManager) getPool(env *fv1.Environment) (*GenericPool, error) {
func (gpm *GenericPoolManager) getPool(ctx context.Context, env *fv1.Environment) (*GenericPool, bool, error) {
otelUtils.SpanTrackEvent(ctx, "getPool", otelUtils.GetAttributesForEnv(env)...)
c := make(chan *response)
gpm.requestChannel <- &request{
ctx: ctx,
requestType: GET_POOL,
env: env,
responseChannel: c,
}
resp := <-c
return resp.pool, resp.error
return resp.pool, resp.created, resp.error
}
func (gpm *GenericPoolManager) cleanupPools(envs []fv1.Environment) {
func (gpm *GenericPoolManager) cleanupPool(ctx context.Context, env *fv1.Environment) {
otelUtils.SpanTrackEvent(ctx, "cleanupPool", otelUtils.GetAttributesForEnv(env)...)
gpm.requestChannel <- &request{
requestType: CLEANUP_POOLS,
envList: envs,
ctx: ctx,
requestType: CLEANUP_POOL,
env: env,
}
}
func (gpm *GenericPoolManager) getFunctionEnv(fn *fv1.Function) (*fv1.Environment, error) {
func (gpm *GenericPoolManager) getFunctionEnv(ctx context.Context, fn *fv1.Function) (*fv1.Environment, error) {
var env *fv1.Environment
otelUtils.SpanTrackEvent(ctx, "getFunctionEnv", otelUtils.GetAttributesForFunction(fn)...)
// Cached ?
// TODO: the cache should be able to search by <env name, fn namespace> instead of function metadata.
@@ -525,7 +545,7 @@ func (gpm *GenericPoolManager) getFunctionEnv(fn *fv1.Function) (*fv1.Environmen
}
// Get env from controller
env, err = gpm.fissionClient.CoreV1().Environments(fn.Spec.Environment.Namespace).Get(context.TODO(), fn.Spec.Environment.Name, metav1.GetOptions{})
env, err = gpm.poolPodC.envLister.Environments(fn.Spec.Environment.Namespace).Get(fn.Spec.Environment.Name)
if err != nil {
return nil, err
}
@@ -543,73 +563,15 @@ func (gpm *GenericPoolManager) getFunctionEnv(fn *fv1.Function) (*fv1.Environmen
return env, nil
}
func (gpm *GenericPoolManager) eagerPoolCreator() {
pollSleep := 2 * time.Second
for {
// get list of envs from controller
envs, err := gpm.fissionClient.CoreV1().Environments(metav1.NamespaceAll).List(context.TODO(), metav1.ListOptions{})
if err != nil {
if utils.IsNetworkError(err) {
gpm.logger.Error("encountered network error, retrying", zap.Error(err))
} else {
gpm.logger.Error("failed to get environment list", zap.Error(err))
}
time.Sleep(5 * time.Second)
continue
}
// Create pools for all envs. TODO: we should make this a bit less eager, only
// creating pools for envs that are actually used by functions. Also we might want
// to keep these eagerly created pools smaller than the ones created when there are
// actual function calls.
wg := &sync.WaitGroup{}
for i := range envs.Items {
env := envs.Items[i]
// Create pool only if poolsize greater than zero
if gpm.getEnvPoolsize(&env) > 0 {
wg.Add(1)
go func() {
defer wg.Done()
_, err := gpm.getPool(&env)
if err != nil {
gpm.logger.Error("eager-create pool failed", zap.Error(err))
}
}()
}
}
// Clean up pools whose env was deleted
gpm.cleanupPools(envs.Items)
wg.Wait()
time.Sleep(pollSleep)
}
}
func (gpm *GenericPoolManager) getEnvPoolsize(env *fv1.Environment) int32 {
var poolsize int32
if env.Spec.Version < 3 {
poolsize = 3
} else {
poolsize = int32(env.Spec.Poolsize)
}
return poolsize
}
// idleObjectReaper reaps objects after certain idle time
func (gpm *GenericPoolManager) idleObjectReaper() {
ctx := context.Background()
pollSleep := 5 * time.Second
go gpm.WebsocketStartEventChecker(gpm.kubernetesClient)
go gpm.NoActiveConnectionEventChecker(gpm.kubernetesClient)
for {
time.Sleep(pollSleep)
envs, err := gpm.fissionClient.CoreV1().Environments(metav1.NamespaceAll).List(context.TODO(), metav1.ListOptions{})
envs, err := gpm.fissionClient.CoreV1().Environments(metav1.NamespaceAll).List(ctx, metav1.ListOptions{})
if err != nil {
gpm.logger.Error("failed to get environment list", zap.Error(err))
continue
@@ -620,7 +582,7 @@ func (gpm *GenericPoolManager) idleObjectReaper() {
envList[env.ObjectMeta.UID] = struct{}{}
}
fns, err := gpm.fissionClient.CoreV1().Functions(metav1.NamespaceAll).List(context.TODO(), metav1.ListOptions{})
fns, err := gpm.fissionClient.CoreV1().Functions(metav1.NamespaceAll).List(ctx, metav1.ListOptions{})
if err != nil {
gpm.logger.Error("failed to get environment list", zap.Error(err))
continue
@@ -688,7 +650,7 @@ func (gpm *GenericPoolManager) idleObjectReaper() {
zap.String("executor", string(fsvc.Executor)),
zap.String("pod", fsvc.Name),
)
reaper.CleanupKubeObject(gpm.logger, gpm.kubernetesClient, &fsvc.KubernetesObjects[i])
reaper.CleanupKubeObject(ctx, gpm.logger, gpm.kubernetesClient, &fsvc.KubernetesObjects[i])
time.Sleep(50 * time.Millisecond)
gpm.fsCache.ReapTime(fsvc.Function.Name, fsvc.Address, time.Since(startTime).Seconds())
}
@@ -780,7 +742,8 @@ func (gpm *GenericPoolManager) NoActiveConnectionEventChecker(kubeClient *kubern
zap.String("executor", string(fsvc.Executor)),
zap.String("pod", fsvc.Name),
)
reaper.CleanupKubeObject(gpm.logger, gpm.kubernetesClient, &fsvc.KubernetesObjects[i])
ctx := context.Background()
reaper.CleanupKubeObject(ctx, gpm.logger, gpm.kubernetesClient, &fsvc.KubernetesObjects[i])
time.Sleep(50 * time.Millisecond)
}
}
@@ -17,6 +17,8 @@ limitations under the License.
package poolmgr
import (
"context"
"go.uber.org/zap"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
@@ -26,11 +28,14 @@ import (
"github.com/fission/fission/pkg/utils"
)
func (gpm *GenericPoolManager) PackageEventHandlers(kubernetesClient *kubernetes.Clientset, fissionfnNamespace string) k8sCache.ResourceEventHandlerFuncs {
// PackageEventHandlers provides handlers for package events.
// Based on package create/update event, we create role binding
// for the package which is used by fetcher component.
func PackageEventHandlers(logger *zap.Logger, kubernetesClient *kubernetes.Clientset, fissionfnNamespace string) k8sCache.ResourceEventHandlerFuncs {
return k8sCache.ResourceEventHandlerFuncs{
AddFunc: func(obj interface{}) {
pkg := obj.(*fv1.Package)
gpm.logger.Debug("list watch for package reported a new package addition",
logger.Debug("list watch for package reported a new package addition",
zap.String("package_name", pkg.ObjectMeta.Name),
zap.String("package_namespace", pkg.ObjectMeta.Namespace))
@@ -39,12 +44,12 @@ func (gpm *GenericPoolManager) PackageEventHandlers(kubernetesClient *kubernetes
if pkg.Spec.Environment.Namespace != metav1.NamespaceDefault {
envNs = pkg.Spec.Environment.Namespace
}
ctx := context.Background()
// here, we return if we hit an error during rolebinding setup. this is because this rolebinding is mandatory for
// every function's package to be loaded into its env. without that, there's no point to move forward.
err := utils.SetupRoleBinding(gpm.logger, kubernetesClient, fv1.PackageGetterRB, pkg.ObjectMeta.Namespace, fv1.PackageGetterCR, fv1.ClusterRole, fv1.FissionFetcherSA, envNs)
err := utils.SetupRoleBinding(ctx, logger, kubernetesClient, fv1.PackageGetterRB, pkg.ObjectMeta.Namespace, fv1.PackageGetterCR, fv1.ClusterRole, fv1.FissionFetcherSA, envNs)
if err != nil {
gpm.logger.Error("error creating rolebinding for package",
logger.Error("error creating rolebinding for package",
zap.Error(err),
zap.String("role_binding", fv1.PackageGetterRB),
zap.String("package_name", pkg.ObjectMeta.Name),
@@ -52,7 +57,7 @@ func (gpm *GenericPoolManager) PackageEventHandlers(kubernetesClient *kubernetes
return
}
gpm.logger.Debug("successfully set up rolebinding for fetcher service account",
logger.Debug("successfully set up rolebinding for fetcher service account",
zap.String("service_account", fv1.FissionFetcherSA),
zap.String("service_account_namespace", envNs),
zap.String("package_name", pkg.ObjectMeta.Name),
@@ -76,11 +81,12 @@ func (gpm *GenericPoolManager) PackageEventHandlers(kubernetesClient *kubernetes
envNs = newPkg.Spec.Environment.Namespace
}
err := utils.SetupRoleBinding(gpm.logger, kubernetesClient, fv1.PackageGetterRB,
ctx := context.Background()
err := utils.SetupRoleBinding(ctx, logger, kubernetesClient, fv1.PackageGetterRB,
newPkg.ObjectMeta.Namespace, fv1.PackageGetterCR, fv1.ClusterRole,
fv1.FissionFetcherSA, envNs)
if err != nil {
gpm.logger.Error("error updating rolebinding for package",
logger.Error("error updating rolebinding for package",
zap.Error(err),
zap.String("role_binding", fv1.PackageGetterRB),
zap.String("package_name", newPkg.ObjectMeta.Name),
@@ -88,7 +94,7 @@ func (gpm *GenericPoolManager) PackageEventHandlers(kubernetesClient *kubernetes
return
}
gpm.logger.Debug("successfully updated rolebinding for fetcher service account",
logger.Debug("successfully updated rolebinding for fetcher service account",
zap.String("service_account", fv1.FissionFetcherSA),
zap.String("service_account_namespace", envNs),
zap.String("package_name", newPkg.ObjectMeta.Name),
@@ -0,0 +1,426 @@
/*
Copyright 2021 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package poolmgr
import (
"context"
"strings"
"time"
"go.uber.org/zap"
apps "k8s.io/api/apps/v1"
v1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/labels"
utilruntime "k8s.io/apimachinery/pkg/util/runtime"
"k8s.io/apimachinery/pkg/util/wait"
appsinformers "k8s.io/client-go/informers/apps/v1"
coreinformers "k8s.io/client-go/informers/core/v1"
"k8s.io/client-go/kubernetes"
corelisters "k8s.io/client-go/listers/core/v1"
k8sCache "k8s.io/client-go/tools/cache"
"k8s.io/client-go/util/workqueue"
fv1 "github.com/fission/fission/pkg/apis/core/v1"
"github.com/fission/fission/pkg/executor/fscache"
finformerv1 "github.com/fission/fission/pkg/generated/informers/externalversions/core/v1"
flisterv1 "github.com/fission/fission/pkg/generated/listers/core/v1"
)
type (
PoolPodController struct {
logger *zap.Logger
kubernetesClient *kubernetes.Clientset
namespace string
enableIstio bool
envLister flisterv1.EnvironmentLister
envListerSynced k8sCache.InformerSynced
// podLister can list/get pods from the shared informer's store
podLister corelisters.PodLister
// podListerSynced returns true if the pod store has been synced at least once.
podListerSynced k8sCache.InformerSynced
envCreateUpdateQueue workqueue.RateLimitingInterface
envDeleteQueue workqueue.RateLimitingInterface
spCleanupPodQueue workqueue.RateLimitingInterface
gpm *GenericPoolManager
}
)
func NewPoolPodController(logger *zap.Logger,
kubernetesClient *kubernetes.Clientset,
namespace string,
enableIstio bool,
funcInformer finformerv1.FunctionInformer,
pkgInformer finformerv1.PackageInformer,
envInformer finformerv1.EnvironmentInformer,
rsInformer appsinformers.ReplicaSetInformer,
podInformer coreinformers.PodInformer) *PoolPodController {
logger = logger.Named("pool_pod_controller")
p := &PoolPodController{
logger: logger,
kubernetesClient: kubernetesClient,
namespace: namespace,
enableIstio: enableIstio,
envCreateUpdateQueue: workqueue.NewNamedRateLimitingQueue(workqueue.DefaultControllerRateLimiter(), "EnvAddUpdateQueue"),
envDeleteQueue: workqueue.NewNamedRateLimitingQueue(workqueue.DefaultControllerRateLimiter(), "EnvDeleteQueue"),
spCleanupPodQueue: workqueue.NewNamedRateLimitingQueue(workqueue.DefaultControllerRateLimiter(), "SpecializedPodCleanupQueue"),
}
funcInformer.Informer().AddEventHandler(FunctionEventHandlers(p.logger, p.kubernetesClient, p.namespace, p.enableIstio))
pkgInformer.Informer().AddEventHandler(PackageEventHandlers(p.logger, p.kubernetesClient, p.namespace))
envInformer.Informer().AddEventHandler(k8sCache.ResourceEventHandlerFuncs{
AddFunc: p.enqueueEnvAdd,
UpdateFunc: p.enqueueEnvUpdate,
DeleteFunc: p.enqueueEnvDelete,
})
rsInformer.Informer().AddEventHandler(k8sCache.ResourceEventHandlerFuncs{
AddFunc: p.handleRSAdd,
UpdateFunc: p.handleRSUpdate,
DeleteFunc: p.handleRSDelete,
})
p.envLister = envInformer.Lister()
p.envListerSynced = envInformer.Informer().HasSynced
p.podLister = podInformer.Lister()
p.podListerSynced = podInformer.Informer().HasSynced
p.logger.Info("pool pod controller handlers registered")
return p
}
func (p *PoolPodController) InjectGpm(gpm *GenericPoolManager) {
p.gpm = gpm
}
func IsPodActive(p *v1.Pod) bool {
return v1.PodSucceeded != p.Status.Phase &&
v1.PodFailed != p.Status.Phase &&
p.DeletionTimestamp == nil
}
func (p *PoolPodController) processRS(rs *apps.ReplicaSet) {
if *(rs.Spec.Replicas) != 0 {
return
}
logger := p.logger.With(zap.String("rs", rs.Name), zap.String("namespace", rs.Namespace))
logger.Debug("replica set has zero replica count")
// List all specialized pods and schedule for cleanup
rsLabelMap, err := metav1.LabelSelectorAsMap(rs.Spec.Selector)
if err != nil {
p.logger.Error("Failed to parse label selector", zap.Error(err))
return
}
rsLabelMap["managed"] = "false"
specializedPods, err := p.podLister.Pods(rs.Namespace).List(labels.SelectorFromSet(rsLabelMap))
if err != nil {
logger.Error("Failed to list specialized pods", zap.Error(err))
}
if len(specializedPods) == 0 {
return
}
logger.Info("specialized pods identified for cleanup with RS", zap.Int("numPods", len(specializedPods)))
for _, pod := range specializedPods {
if !IsPodActive(pod) {
continue
}
key, err := k8sCache.MetaNamespaceKeyFunc(pod)
if err != nil {
logger.Error("Failed to get key for pod", zap.Error(err))
continue
}
p.spCleanupPodQueue.Add(key)
}
}
func (p *PoolPodController) handleRSAdd(obj interface{}) {
rs, ok := obj.(*apps.ReplicaSet)
if !ok {
p.logger.Error("unexpected type when adding rs to pool pod controller", zap.Any("obj", obj))
return
}
p.processRS(rs)
}
func (p *PoolPodController) handleRSUpdate(oldObj interface{}, newObj interface{}) {
rs, ok := newObj.(*apps.ReplicaSet)
if !ok {
p.logger.Error("unexpected type when updating rs to pool pod controller", zap.Any("obj", newObj))
return
}
p.processRS(rs)
}
func (p *PoolPodController) handleRSDelete(obj interface{}) {
rs, ok := obj.(*apps.ReplicaSet)
if !ok {
tombstone, ok := obj.(k8sCache.DeletedFinalStateUnknown)
if !ok {
p.logger.Error("couldnt get object from tombstone", zap.Any("obj", obj))
return
}
rs, ok = tombstone.Obj.(*apps.ReplicaSet)
if !ok {
p.logger.Error("tombstone contained object that is not a replicaset", zap.Any("obj", obj))
return
}
}
p.processRS(rs)
}
func (p *PoolPodController) enqueueEnvAdd(obj interface{}) {
key, err := k8sCache.MetaNamespaceKeyFunc(obj)
if err != nil {
p.logger.Error("error retrieving key from object in poolPodController", zap.Any("obj", obj))
return
}
p.logger.Debug("enqueue env add", zap.String("key", key))
p.envCreateUpdateQueue.Add(key)
}
func (p *PoolPodController) enqueueEnvUpdate(oldObj, newObj interface{}) {
key, err := k8sCache.MetaNamespaceKeyFunc(newObj)
if err != nil {
p.logger.Error("error retrieving key from object in poolPodController", zap.Any("obj", key))
return
}
p.logger.Debug("enqueue env update", zap.String("key", key))
p.envCreateUpdateQueue.Add(key)
}
func (p *PoolPodController) enqueueEnvDelete(obj interface{}) {
env, ok := obj.(*fv1.Environment)
if !ok {
p.logger.Error("unexpected type when deleting env to pool pod controller", zap.Any("obj", obj))
return
}
p.logger.Debug("enqueue env delete", zap.Any("env", env))
p.envDeleteQueue.Add(env)
}
func (p *PoolPodController) Run(stopCh <-chan struct{}) {
defer utilruntime.HandleCrash()
defer p.envCreateUpdateQueue.ShutDown()
defer p.envDeleteQueue.ShutDown()
defer p.spCleanupPodQueue.ShutDown()
// Wait for the caches to be synced before starting workers
p.logger.Info("Waiting for informer caches to sync")
if ok := k8sCache.WaitForCacheSync(stopCh, p.envListerSynced, p.podListerSynced); !ok {
p.logger.Fatal("failed to wait for caches to sync")
}
for i := 0; i < 4; i++ {
go wait.Until(p.workerRun("envCreateUpdate", p.envCreateUpdateQueueProcessFunc), time.Second, stopCh)
}
go wait.Until(p.workerRun("envDeleteQueue", p.envDeleteQueueProcessFunc), time.Second, stopCh)
go wait.Until(p.workerRun("spCleanupPodQueue", p.spCleanupPodQueueProcessFunc), time.Second, stopCh)
p.logger.Info("Started workers for poolPodController")
<-stopCh
p.logger.Info("Shutting down workers for poolPodController")
}
func (p *PoolPodController) workerRun(name string, processFunc func() bool) func() {
return func() {
p.logger.Debug("Starting worker with func", zap.String("name", name))
for {
if quit := processFunc(); quit {
p.logger.Info("Shutting down worker", zap.String("name", name))
return
}
}
}
}
func (p *PoolPodController) envCreateUpdateQueueProcessFunc() bool {
maxRetries := 3
handleEnv := func(ctx context.Context, env *fv1.Environment) error {
log := p.logger.With(zap.String("env", env.ObjectMeta.Name), zap.String("namespace", env.ObjectMeta.Namespace))
log.Debug("env reconsile request processing")
pool, created, err := p.gpm.getPool(ctx, env)
if err != nil {
log.Error("error getting pool", zap.Error(err))
return err
}
if created {
log.Info("created pool for the environment")
return nil
}
poolsize := getEnvPoolSize(env)
if poolsize == 0 {
log.Info("pool size is zero")
p.gpm.cleanupPool(ctx, env)
return nil
}
err = pool.updatePoolDeployment(ctx, env)
if err != nil {
log.Error("error updating pool", zap.Error(err))
return err
}
// If any specialized pods are running, those would be
// deleted by replicaSet controller.
return nil
}
obj, quit := p.envCreateUpdateQueue.Get()
if quit {
return true
}
key := obj.(string)
defer p.envCreateUpdateQueue.Done(key)
namespace, name, err := k8sCache.SplitMetaNamespaceKey(key)
if err != nil {
p.logger.Error("error splitting key", zap.Error(err))
p.envCreateUpdateQueue.Forget(key)
return false
}
env, err := p.envLister.Environments(namespace).Get(name)
if apierrors.IsNotFound(err) {
p.logger.Info("env not found", zap.String("key", key))
p.envCreateUpdateQueue.Forget(key)
return false
}
if err != nil {
if p.envCreateUpdateQueue.NumRequeues(key) < maxRetries {
p.envCreateUpdateQueue.AddRateLimited(key)
p.logger.Error("error getting env, retrying", zap.Error(err))
} else {
p.envCreateUpdateQueue.Forget(key)
p.logger.Error("error getting env, retrying, max retries reached", zap.Error(err))
}
return false
}
ctx := context.Background()
err = handleEnv(ctx, env)
if err != nil {
if p.envCreateUpdateQueue.NumRequeues(key) < maxRetries {
p.envCreateUpdateQueue.AddRateLimited(key)
p.logger.Error("error handling env from envInformer, retrying", zap.String("key", key), zap.Error(err))
} else {
p.envCreateUpdateQueue.Forget(key)
p.logger.Error("error handling env from envInformer, max retries reached", zap.String("key", key), zap.Error(err))
}
return false
}
p.envCreateUpdateQueue.Forget(key)
return false
}
func (p *PoolPodController) envDeleteQueueProcessFunc() bool {
obj, quit := p.envDeleteQueue.Get()
if quit {
return true
}
defer p.envDeleteQueue.Done(obj)
env, ok := obj.(*fv1.Environment)
if !ok {
p.logger.Error("unexpected type when deleting env to pool pod controller", zap.Any("obj", obj))
p.envDeleteQueue.Forget(obj)
return false
}
ctx := context.Background()
p.logger.Debug("env delete request processing")
p.gpm.cleanupPool(ctx, env)
specializePodLables := getSpecializedPodLabels(env)
specializedPods, err := p.podLister.Pods(p.gpm.namespace).List(labels.SelectorFromSet(specializePodLables))
if err != nil {
p.logger.Error("failed to list specialized pods", zap.Error(err))
p.envDeleteQueue.Forget(obj)
return false
}
if len(specializedPods) == 0 {
p.envDeleteQueue.Forget(obj)
return false
}
p.logger.Info("specialized pods identified for cleanup after env delete", zap.String("env", env.ObjectMeta.Name), zap.String("namespace", env.ObjectMeta.Namespace), zap.Int("count", len(specializedPods)))
for _, pod := range specializedPods {
if !IsPodActive(pod) {
continue
}
key, err := k8sCache.MetaNamespaceKeyFunc(pod)
if err != nil {
p.logger.Error("Failed to get key for pod", zap.Error(err))
continue
}
p.spCleanupPodQueue.Add(key)
}
p.envDeleteQueue.Forget(obj)
return false
}
func (p *PoolPodController) spCleanupPodQueueProcessFunc() bool {
maxRetries := 3
obj, quit := p.spCleanupPodQueue.Get()
if quit {
return true
}
key := obj.(string)
defer p.spCleanupPodQueue.Done(key)
namespace, name, err := k8sCache.SplitMetaNamespaceKey(key)
if err != nil {
p.logger.Error("error splitting key", zap.Error(err))
p.spCleanupPodQueue.Forget(key)
return false
}
pod, err := p.podLister.Pods(namespace).Get(name)
if apierrors.IsNotFound(err) {
p.logger.Info("pod not found", zap.String("key", key))
p.spCleanupPodQueue.Forget(key)
return false
}
if !IsPodActive(pod) {
p.logger.Info("pod not active", zap.String("key", key))
p.spCleanupPodQueue.Forget(key)
return false
}
if err != nil {
if p.spCleanupPodQueue.NumRequeues(key) < maxRetries {
p.spCleanupPodQueue.AddRateLimited(key)
p.logger.Error("error getting pod, retrying", zap.Error(err))
} else {
p.spCleanupPodQueue.Forget(key)
p.logger.Error("error getting pod, max retries reached", zap.Error(err))
}
return false
}
podName := strings.SplitAfter(pod.GetName(), ".")
if fsvc, ok := p.gpm.fsCache.PodToFsvc.Load(strings.TrimSuffix(podName[0], ".")); ok {
fsvc, ok := fsvc.(*fscache.FuncSvc)
if ok {
p.gpm.fsCache.DeleteFunctionSvc(fsvc)
p.gpm.fsCache.DeleteEntry(fsvc)
} else {
p.logger.Error("could not covert item from PodToFsvc", zap.String("key", key))
}
}
err = p.kubernetesClient.CoreV1().Pods(p.namespace).Delete(context.TODO(), pod.Name, metav1.DeleteOptions{})
if err != nil {
p.logger.Error("failed to delete pod", zap.Error(err), zap.String("pod", pod.ObjectMeta.Name), zap.String("pod_namespace", pod.ObjectMeta.Namespace))
return false
}
p.logger.Info("cleaned specialized pod as environment update/deleted",
zap.String("pod", pod.ObjectMeta.Name), zap.String("pod_namespace", pod.ObjectMeta.Namespace),
zap.String("address", pod.Status.PodIP))
p.spCleanupPodQueue.Forget(key)
return false
}
+22 -21
View File
@@ -37,28 +37,28 @@ var (
)
// CleanupKubeObject deletes given kubernetes object
func CleanupKubeObject(logger *zap.Logger, kubeClient *kubernetes.Clientset, kubeobj *apiv1.ObjectReference) {
func CleanupKubeObject(ctx context.Context, logger *zap.Logger, kubeClient *kubernetes.Clientset, kubeobj *apiv1.ObjectReference) {
switch strings.ToLower(kubeobj.Kind) {
case "pod":
err := kubeClient.CoreV1().Pods(kubeobj.Namespace).Delete(context.TODO(), kubeobj.Name, meta_v1.DeleteOptions{})
err := kubeClient.CoreV1().Pods(kubeobj.Namespace).Delete(ctx, kubeobj.Name, meta_v1.DeleteOptions{})
if err != nil {
logger.Error("error cleaning up pod", zap.Error(err), zap.String("pod", kubeobj.Name))
}
case "service":
err := kubeClient.CoreV1().Services(kubeobj.Namespace).Delete(context.TODO(), kubeobj.Name, meta_v1.DeleteOptions{})
err := kubeClient.CoreV1().Services(kubeobj.Namespace).Delete(ctx, kubeobj.Name, meta_v1.DeleteOptions{})
if err != nil {
logger.Error("error cleaning up service", zap.Error(err), zap.String("service", kubeobj.Name))
}
case "deployment":
err := kubeClient.AppsV1().Deployments(kubeobj.Namespace).Delete(context.TODO(), kubeobj.Name, delOpt)
err := kubeClient.AppsV1().Deployments(kubeobj.Namespace).Delete(ctx, kubeobj.Name, delOpt)
if err != nil {
logger.Error("error cleaning up deployment", zap.Error(err), zap.String("deployment", kubeobj.Name))
}
case "horizontalpodautoscaler":
err := kubeClient.AutoscalingV1().HorizontalPodAutoscalers(kubeobj.Namespace).Delete(context.TODO(), kubeobj.Name, meta_v1.DeleteOptions{})
err := kubeClient.AutoscalingV1().HorizontalPodAutoscalers(kubeobj.Namespace).Delete(ctx, kubeobj.Name, meta_v1.DeleteOptions{})
if err != nil {
logger.Error("error cleaning up horizontalpodautoscaler", zap.Error(err), zap.String("horizontalpodautoscaler", kubeobj.Name))
}
@@ -70,8 +70,8 @@ func CleanupKubeObject(logger *zap.Logger, kubeClient *kubernetes.Clientset, kub
}
// CleanupDeployments deletes deployment(s) for a given instanceID
func CleanupDeployments(logger *zap.Logger, client *kubernetes.Clientset, instanceID string, listOps meta_v1.ListOptions) error {
deploymentList, err := client.AppsV1().Deployments(meta_v1.NamespaceAll).List(context.TODO(), listOps)
func CleanupDeployments(ctx context.Context, logger *zap.Logger, client *kubernetes.Clientset, instanceID string, listOps meta_v1.ListOptions) error {
deploymentList, err := client.AppsV1().Deployments(meta_v1.NamespaceAll).List(ctx, listOps)
if err != nil {
return err
}
@@ -83,7 +83,7 @@ func CleanupDeployments(logger *zap.Logger, client *kubernetes.Clientset, instan
}
if ok && id != instanceID {
logger.Info("cleaning up deployment", zap.String("deployment", dep.ObjectMeta.Name))
err := client.AppsV1().Deployments(dep.ObjectMeta.Namespace).Delete(context.TODO(), dep.ObjectMeta.Name, delOpt)
err := client.AppsV1().Deployments(dep.ObjectMeta.Namespace).Delete(ctx, dep.ObjectMeta.Name, delOpt)
if err != nil {
logger.Error("error cleaning up deployment",
zap.Error(err),
@@ -97,8 +97,8 @@ func CleanupDeployments(logger *zap.Logger, client *kubernetes.Clientset, instan
}
// CleanupPods deletes pod(s) for a given instanceID
func CleanupPods(logger *zap.Logger, client *kubernetes.Clientset, instanceID string, listOps meta_v1.ListOptions) error {
podList, err := client.CoreV1().Pods(meta_v1.NamespaceAll).List(context.TODO(), listOps)
func CleanupPods(ctx context.Context, logger *zap.Logger, client *kubernetes.Clientset, instanceID string, listOps meta_v1.ListOptions) error {
podList, err := client.CoreV1().Pods(meta_v1.NamespaceAll).List(ctx, listOps)
if err != nil {
return err
}
@@ -110,7 +110,7 @@ func CleanupPods(logger *zap.Logger, client *kubernetes.Clientset, instanceID st
}
if ok && id != instanceID {
logger.Info("cleaning up pod", zap.String("pod", pod.ObjectMeta.Name))
err := client.CoreV1().Pods(pod.ObjectMeta.Namespace).Delete(context.TODO(), pod.ObjectMeta.Name, meta_v1.DeleteOptions{})
err := client.CoreV1().Pods(pod.ObjectMeta.Namespace).Delete(ctx, pod.ObjectMeta.Name, meta_v1.DeleteOptions{})
if err != nil {
logger.Error("error cleaning up pod",
zap.Error(err),
@@ -124,8 +124,8 @@ func CleanupPods(logger *zap.Logger, client *kubernetes.Clientset, instanceID st
}
// CleanupServices deletes service(s) for a given instanceID
func CleanupServices(logger *zap.Logger, client *kubernetes.Clientset, instanceID string, listOps meta_v1.ListOptions) error {
svcList, err := client.CoreV1().Services(meta_v1.NamespaceAll).List(context.TODO(), listOps)
func CleanupServices(ctx context.Context, logger *zap.Logger, client *kubernetes.Clientset, instanceID string, listOps meta_v1.ListOptions) error {
svcList, err := client.CoreV1().Services(meta_v1.NamespaceAll).List(ctx, listOps)
if err != nil {
return err
}
@@ -137,7 +137,7 @@ func CleanupServices(logger *zap.Logger, client *kubernetes.Clientset, instanceI
}
if ok && id != instanceID {
logger.Info("cleaning up service", zap.String("service", svc.ObjectMeta.Name))
err := client.CoreV1().Services(svc.ObjectMeta.Namespace).Delete(context.TODO(), svc.ObjectMeta.Name, meta_v1.DeleteOptions{})
err := client.CoreV1().Services(svc.ObjectMeta.Namespace).Delete(ctx, svc.ObjectMeta.Name, meta_v1.DeleteOptions{})
if err != nil {
logger.Error("error cleaning up service",
zap.Error(err),
@@ -151,8 +151,8 @@ func CleanupServices(logger *zap.Logger, client *kubernetes.Clientset, instanceI
}
// CleanupHpa deletes horizontal pod autoscaler(s) for a given instanceID
func CleanupHpa(logger *zap.Logger, client *kubernetes.Clientset, instanceID string, listOps meta_v1.ListOptions) error {
hpaList, err := client.AutoscalingV1().HorizontalPodAutoscalers(meta_v1.NamespaceAll).List(context.TODO(), listOps)
func CleanupHpa(ctx context.Context, logger *zap.Logger, client *kubernetes.Clientset, instanceID string, listOps meta_v1.ListOptions) error {
hpaList, err := client.AutoscalingV1().HorizontalPodAutoscalers(meta_v1.NamespaceAll).List(ctx, listOps)
if err != nil {
return err
}
@@ -165,7 +165,7 @@ func CleanupHpa(logger *zap.Logger, client *kubernetes.Clientset, instanceID str
}
if ok && id != instanceID {
logger.Info("cleaning up HPA", zap.String("hpa", hpa.ObjectMeta.Name))
err := client.AutoscalingV1().HorizontalPodAutoscalers(hpa.ObjectMeta.Namespace).Delete(context.TODO(), hpa.ObjectMeta.Name, meta_v1.DeleteOptions{})
err := client.AutoscalingV1().HorizontalPodAutoscalers(hpa.ObjectMeta.Namespace).Delete(ctx, hpa.ObjectMeta.Name, meta_v1.DeleteOptions{})
if err != nil {
logger.Error("error cleaning up HPA",
zap.Error(err),
@@ -181,13 +181,14 @@ func CleanupHpa(logger *zap.Logger, client *kubernetes.Clientset, instanceID str
// CleanupRoleBindings periodically lists rolebindings across all namespaces and removes Service Accounts from them or
// deletes the rolebindings completely if there are no Service Accounts in a rolebinding object.
func CleanupRoleBindings(logger *zap.Logger, client *kubernetes.Clientset, fissionClient *crd.FissionClient, functionNs, envBuilderNs string, cleanupRoleBindingInterval time.Duration) {
ctx := context.Background()
for {
// some sleep before the next reaper iteration
time.Sleep(cleanupRoleBindingInterval)
logger.Debug("starting cleanupRoleBindings cycle")
// get all rolebindings ( just to be efficient, one call to kubernetes )
rbList, err := client.RbacV1().RoleBindings(meta_v1.NamespaceAll).List(context.TODO(), meta_v1.ListOptions{})
rbList, err := client.RbacV1().RoleBindings(meta_v1.NamespaceAll).List(ctx, meta_v1.ListOptions{})
if err != nil {
// something wrong, but next iteration hopefully succeeds
logger.Error("error listing role bindings in all namespaces", zap.Error(err))
@@ -208,7 +209,7 @@ func CleanupRoleBindings(logger *zap.Logger, client *kubernetes.Clientset, fissi
// in order to find out if there are any functions that need this role-binding in role-binding namespace,
// we can list the functions once per role-binding.
funcList, err := fissionClient.CoreV1().Functions(roleBinding.Namespace).List(context.TODO(), meta_v1.ListOptions{})
funcList, err := fissionClient.CoreV1().Functions(roleBinding.Namespace).List(ctx, meta_v1.ListOptions{})
if err != nil {
logger.Error("error fetching function list in namespace", zap.Error(err), zap.String("namespace", roleBinding.Namespace))
continue
@@ -259,7 +260,7 @@ func CleanupRoleBindings(logger *zap.Logger, client *kubernetes.Clientset, fissi
// else if its a secret-configmap-rb, we have only one SA which is fission-fetcher
if roleBinding.Name == fv1.PackageGetterRB {
// check if there is an env obj in saNs
envList, err := fissionClient.CoreV1().Environments(saNs).List(context.TODO(), meta_v1.ListOptions{})
envList, err := fissionClient.CoreV1().Environments(saNs).List(ctx, meta_v1.ListOptions{})
if err != nil {
logger.Error("error fetching environment list in service account namespace", zap.Error(err), zap.String("namespace", saNs))
continue
@@ -305,7 +306,7 @@ func CleanupRoleBindings(logger *zap.Logger, client *kubernetes.Clientset, fissi
zap.String("role_binding_namespace", roleBinding.Namespace))
// call this once in the end for each role-binding
err = utils.RemoveSAFromRoleBindingWithRetries(logger, client, roleBinding.Name, roleBinding.Namespace, saToRemove)
err = utils.RemoveSAFromRoleBindingWithRetries(ctx, logger, client, roleBinding.Name, roleBinding.Namespace, saToRemove)
if err != nil {
// if there's an error, we just log it and proceed with the next role-binding, hoping that this role-binding
// will be processed in next iteration.
+5 -5
View File
@@ -57,7 +57,7 @@ func WaitTimeout(wg *sync.WaitGroup, timeout time.Duration) {
}
// ConvertConfigSecrets returns envFromSource which can be passed directly into the pod spec
func ConvertConfigSecrets(fn *fv1.Function, kc *kubernetes.Clientset) ([]apiv1.EnvFromSource, error) {
func ConvertConfigSecrets(ctx context.Context, fn *fv1.Function, kc *kubernetes.Clientset) ([]apiv1.EnvFromSource, error) {
cmList := fn.Spec.ConfigMaps
secList := fn.Spec.Secrets
@@ -65,9 +65,9 @@ func ConvertConfigSecrets(fn *fv1.Function, kc *kubernetes.Clientset) ([]apiv1.E
secEnvSources := make([]*apiv1.SecretEnvSource, 0)
for _, cm := range cmList {
if cm.Namespace != fn.Namespace {
return nil, errors.New("Function should not reference config map of different namespace")
return nil, errors.New("function should not reference config map of different namespace")
}
_, err := kc.CoreV1().ConfigMaps(cm.Namespace).Get(context.TODO(), cm.Name, metav1.GetOptions{})
_, err := kc.CoreV1().ConfigMaps(cm.Namespace).Get(ctx, cm.Name, metav1.GetOptions{})
if err != nil {
return nil, err
}
@@ -81,9 +81,9 @@ func ConvertConfigSecrets(fn *fv1.Function, kc *kubernetes.Clientset) ([]apiv1.E
for _, sec := range secList {
if sec.Namespace != fn.Namespace {
return nil, errors.New("Function should not reference secret of different namespace")
return nil, errors.New("function should not reference secret of different namespace")
}
_, err := kc.CoreV1().Secrets(sec.Namespace).Get(context.TODO(), sec.Name, metav1.GetOptions{})
_, err := kc.CoreV1().Secrets(sec.Namespace).Get(ctx, sec.Name, metav1.GetOptions{})
if err != nil {
return nil, err
}
+12 -5
View File
@@ -11,11 +11,13 @@ import (
"github.com/pkg/errors"
"go.opencensus.io/plugin/ochttp"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
"go.uber.org/zap"
"golang.org/x/net/context/ctxhttp"
ferror "github.com/fission/fission/pkg/error"
"github.com/fission/fission/pkg/fetcher"
"github.com/fission/fission/pkg/utils/tracing"
)
type (
@@ -27,12 +29,17 @@ type (
)
func MakeClient(logger *zap.Logger, fetcherUrl string) *Client {
var hc *http.Client
if tracing.TracingEnabled(logger) {
hc = &http.Client{Transport: &ochttp.Transport{}}
} else {
hc = &http.Client{Transport: otelhttp.NewTransport(http.DefaultTransport)}
}
return &Client{
logger: logger.Named("fetcher_client"),
url: strings.TrimSuffix(fetcherUrl, "/"),
httpClient: &http.Client{
Transport: &ochttp.Transport{},
},
logger: logger.Named("fetcher_client"),
url: strings.TrimSuffix(fetcherUrl, "/"),
httpClient: hc,
}
}
+5 -2
View File
@@ -18,6 +18,7 @@ import (
fv1 "github.com/fission/fission/pkg/apis/core/v1"
"github.com/fission/fission/pkg/fetcher"
"github.com/fission/fission/pkg/utils"
"github.com/fission/fission/pkg/utils/otel"
)
type Config struct {
@@ -125,8 +126,9 @@ func (cfg *Config) NewSpecializeRequest(fn *fv1.Function, env *fv1.Environment)
FetchReq: fetcher.FunctionFetchRequest{
FetchType: fv1.FETCH_DEPLOYMENT,
Package: metav1.ObjectMeta{
Namespace: fn.Spec.Package.PackageRef.Namespace,
Name: fn.Spec.Package.PackageRef.Name,
Namespace: fn.Spec.Package.PackageRef.Namespace,
Name: fn.Spec.Package.PackageRef.Name,
ResourceVersion: fn.Spec.Package.PackageRef.ResourceVersion,
},
Filename: targetFilename,
Secrets: fn.Spec.Secrets,
@@ -282,6 +284,7 @@ func (cfg *Config) addFetcherToPodSpecWithCommand(podSpec *apiv1.PodSpec, mainCo
},
},
},
Env: otel.OtelEnvForContainer(),
}
// Pod is removed from endpoints list for service when it's
+133 -68
View File
@@ -25,13 +25,16 @@ import (
"net/http"
"os"
"path/filepath"
"strconv"
"time"
"github.com/mholt/archiver"
"github.com/pkg/errors"
uuid "github.com/satori/go.uuid"
"go.opencensus.io/plugin/ochttp"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
"go.uber.org/zap"
"golang.org/x/net/context/ctxhttp"
corev1 "k8s.io/api/core/v1"
k8serr "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -49,6 +52,8 @@ import (
"github.com/fission/fission/pkg/info"
storageSvcClient "github.com/fission/fission/pkg/storagesvc/client"
"github.com/fission/fission/pkg/utils"
otelUtils "github.com/fission/fission/pkg/utils/otel"
"github.com/fission/fission/pkg/utils/tracing"
)
type (
@@ -102,6 +107,13 @@ func MakeFetcher(logger *zap.Logger, sharedVolumePath string, sharedSecretPath s
return nil, errors.Wrap(err, "error reading pod namespace from downward volume")
}
var hc *http.Client
if tracing.TracingEnabled(logger) {
hc = &http.Client{Transport: &ochttp.Transport{}}
} else {
hc = &http.Client{Transport: otelhttp.NewTransport(http.DefaultTransport)}
}
return &Fetcher{
logger: fLogger,
sharedVolumePath: sharedVolumePath,
@@ -113,9 +125,7 @@ func MakeFetcher(logger *zap.Logger, sharedVolumePath string, sharedSecretPath s
Name: string(name),
Namespace: string(namespace),
},
httpClient: &http.Client{
Transport: &ochttp.Transport{},
},
httpClient: hc,
}, nil
}
@@ -149,83 +159,90 @@ func (fetcher *Fetcher) VersionHandler(w http.ResponseWriter, r *http.Request) {
}
func (fetcher *Fetcher) FetchHandler(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
if r.Method != "POST" {
http.Error(w, "only POST is supported on this endpoint", http.StatusMethodNotAllowed)
return
}
logger := otelUtils.LoggerWithTraceID(ctx, fetcher.logger)
startTime := time.Now()
defer func() {
elapsed := time.Since(startTime)
fetcher.logger.Info("fetch request done", zap.Duration("elapsed_time", elapsed))
logger.Info("fetch request done", zap.Duration("elapsed_time", elapsed))
}()
// parse request
body, err := ioutil.ReadAll(r.Body)
if err != nil {
fetcher.logger.Error("error reading request body", zap.Error(err))
logger.Error("error reading request body", zap.Error(err))
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
var req FunctionFetchRequest
err = json.Unmarshal(body, &req)
if err != nil {
fetcher.logger.Error("error parsing request body", zap.Error(err))
logger.Error("error parsing request body", zap.Error(err))
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
pkg, err := fetcher.getPkgInformation(req)
pkg, err := fetcher.getPkgInformation(ctx, req)
if err != nil {
fetcher.logger.Error("error getting package information", zap.Error(err))
logger.Error("error getting package information", zap.Error(err))
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
code, err := fetcher.Fetch(r.Context(), pkg, req)
code, err := fetcher.Fetch(ctx, pkg, req)
if err != nil {
fetcher.logger.Error("error fetching", zap.Error(err))
logger.Error("error fetching", zap.Error(err))
http.Error(w, err.Error(), code)
return
}
fetcher.logger.Info("checking secrets/cfgmaps")
code, err = fetcher.FetchSecretsAndCfgMaps(req.Secrets, req.ConfigMaps)
logger.Info("checking secrets/cfgmaps")
code, err = fetcher.FetchSecretsAndCfgMaps(ctx, req.Secrets, req.ConfigMaps)
if err != nil {
fetcher.logger.Error("error fetching secrets and config maps", zap.Error(err))
logger.Error("error fetching secrets and config maps", zap.Error(err))
http.Error(w, err.Error(), code)
return
}
fetcher.logger.Info("completed fetch request")
logger.Info("completed fetch request")
// all done
w.WriteHeader(http.StatusOK)
}
func (fetcher *Fetcher) SpecializeHandler(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
if r.Method != "POST" {
http.Error(w, fmt.Sprintf("only POST is supported on this endpoint, %v received", r.Method), http.StatusMethodNotAllowed)
return
}
logger := otelUtils.LoggerWithTraceID(ctx, fetcher.logger)
// parse request
body, err := ioutil.ReadAll(r.Body)
if err != nil {
fetcher.logger.Error("error reading request body", zap.Error(err))
logger.Error("error reading request body", zap.Error(err))
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
var req FunctionSpecializeRequest
err = json.Unmarshal(body, &req)
if err != nil {
fetcher.logger.Error("error parsing request body", zap.Error(err))
logger.Error("error parsing request body", zap.Error(err))
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
err = fetcher.SpecializePod(r.Context(), req.FetchReq, req.LoadReq)
err = fetcher.SpecializePod(ctx, req.FetchReq, req.LoadReq)
if err != nil {
fetcher.logger.Error("error specializing pod", zap.Error(err))
logger.Error("error specializing pod", zap.Error(err))
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
@@ -237,18 +254,21 @@ func (fetcher *Fetcher) SpecializeHandler(w http.ResponseWriter, r *http.Request
// Fetch takes FetchRequest and makes the fetch call
// It returns the HTTP code and error if any
func (fetcher *Fetcher) Fetch(ctx context.Context, pkg *fv1.Package, req FunctionFetchRequest) (int, error) {
logger := otelUtils.LoggerWithTraceID(ctx, fetcher.logger)
// check that the requested filename is not an empty string and error out if so
if len(req.Filename) == 0 {
e := "fetch request received for an empty file name"
fetcher.logger.Error(e, zap.Any("request", req))
logger.Error(e, zap.Any("request", req))
return http.StatusBadRequest, errors.New(fmt.Sprintf("%s, request: %v", e, req))
}
// verify first if the file already exists.
if _, err := os.Stat(filepath.Join(fetcher.sharedVolumePath, req.Filename)); err == nil {
fetcher.logger.Info("requested file already exists at shared volume - skipping fetch",
logger.Info("requested file already exists at shared volume - skipping fetch",
zap.String("requested_file", req.Filename),
zap.String("shared_volume_path", fetcher.sharedVolumePath))
otelUtils.SpanTrackEvent(ctx, "packageAlreadyExists", otelUtils.GetAttributesForPackage(pkg)...)
return http.StatusOK, nil
}
@@ -256,11 +276,16 @@ func (fetcher *Fetcher) Fetch(ctx context.Context, pkg *fv1.Package, req Functio
tmpPath := filepath.Join(fetcher.sharedVolumePath, tmpFile)
if req.FetchType == fv1.FETCH_URL {
otelUtils.SpanTrackEvent(ctx, "fetch_url", otelUtils.MapToAttributes(map[string]string{
"package-name": pkg.Name,
"package-namespace": pkg.Namespace,
"fetch-url": req.Url,
})...)
// fetch the file and save it to the tmp path
err := utils.DownloadUrl(ctx, fetcher.httpClient, req.Url, tmpPath)
if err != nil {
e := "failed to download url"
fetcher.logger.Error(e, zap.Error(err), zap.String("url", req.Url))
logger.Error(e, zap.Error(err), zap.String("url", req.Url))
return http.StatusBadRequest, errors.Wrapf(err, "%s: %s", e, req.Url)
}
} else {
@@ -274,7 +299,7 @@ func (fetcher *Fetcher) Fetch(ctx context.Context, pkg *fv1.Package, req Functio
// it may be useful to the user if we can send a more meaningful error in such a scenario.
if pkg.Status.BuildStatus != fv1.BuildStatusSucceeded && pkg.Status.BuildStatus != fv1.BuildStatusNone {
e := fmt.Sprintf("cannot fetch deployment: package build status was not %q", fv1.BuildStatusSucceeded)
fetcher.logger.Error(e,
logger.Error(e,
zap.String("package_name", pkg.ObjectMeta.Name),
zap.String("package_namespace", pkg.ObjectMeta.Namespace),
zap.Any("package_build_status", pkg.Status.BuildStatus))
@@ -291,15 +316,21 @@ func (fetcher *Fetcher) Fetch(ctx context.Context, pkg *fv1.Package, req Functio
err := ioutil.WriteFile(tmpPath, archive.Literal, 0600)
if err != nil {
e := "failed to write file"
fetcher.logger.Error(e, zap.Error(err), zap.String("location", tmpPath))
logger.Error(e, zap.Error(err), zap.String("location", tmpPath))
return http.StatusInternalServerError, errors.Wrapf(err, "%s %s", e, tmpPath)
}
otelUtils.SpanTrackEvent(ctx, "archiveLiteral", otelUtils.GetAttributesForPackage(pkg)...)
} else {
// download and verify
otelUtils.SpanTrackEvent(ctx, "dowloadArchieveLiteral", otelUtils.MapToAttributes(map[string]string{
"package-name": pkg.Name,
"package-namespace": pkg.Namespace,
"archive-url": archive.URL,
})...)
err := utils.DownloadUrl(ctx, fetcher.httpClient, archive.URL, tmpPath)
if err != nil {
e := "failed to download url"
fetcher.logger.Error(e, zap.Error(err), zap.String("url", req.Url))
logger.Error(e, zap.Error(err), zap.String("url", req.Url))
return http.StatusBadRequest, errors.Wrapf(err, "%s %s", e, req.Url)
}
@@ -308,13 +339,13 @@ func (fetcher *Fetcher) Fetch(ctx context.Context, pkg *fv1.Package, req Functio
checksum, err := utils.GetFileChecksum(tmpPath)
if err != nil {
e := "failed to get checksum"
fetcher.logger.Error(e, zap.Error(err))
logger.Error(e, zap.Error(err))
return http.StatusBadRequest, errors.Wrap(err, e)
}
err = verifyChecksum(checksum, &archive.Checksum)
if err != nil {
e := "failed to verify checksum"
fetcher.logger.Error(e, zap.Error(err))
logger.Error(e, zap.Error(err))
return http.StatusBadRequest, errors.Wrap(err, e)
}
}
@@ -326,7 +357,7 @@ func (fetcher *Fetcher) Fetch(ctx context.Context, pkg *fv1.Package, req Functio
tmpUnarchivePath := filepath.Join(fetcher.sharedVolumePath, uuid.NewV4().String())
err := fetcher.unarchive(tmpPath, tmpUnarchivePath)
if err != nil {
fetcher.logger.Error("error unarchive",
logger.Error("error unarchive",
zap.Error(err),
zap.String("archive_location", tmpPath),
zap.String("target_location", tmpUnarchivePath))
@@ -340,23 +371,26 @@ func (fetcher *Fetcher) Fetch(ctx context.Context, pkg *fv1.Package, req Functio
renamePath := filepath.Join(fetcher.sharedVolumePath, req.Filename)
err := fetcher.rename(tmpPath, renamePath)
if err != nil {
fetcher.logger.Error("error renaming file",
logger.Error("error renaming file",
zap.Error(err),
zap.String("original_path", tmpPath),
zap.String("rename_path", renamePath))
return http.StatusInternalServerError, err
}
fetcher.logger.Info("successfully placed", zap.String("location", renamePath))
otelUtils.SpanTrackEvent(ctx, "packageFetched", otelUtils.GetAttributesForPackage(pkg)...)
logger.Info("successfully placed", zap.String("location", renamePath))
return http.StatusOK, nil
}
// FetchSecretsAndCfgMaps fetches secrets and configmaps specified by user
// It returns the HTTP code and error if any
func (fetcher *Fetcher) FetchSecretsAndCfgMaps(secrets []fv1.SecretReference, cfgmaps []fv1.ConfigMapReference) (int, error) {
func (fetcher *Fetcher) FetchSecretsAndCfgMaps(ctx context.Context, secrets []fv1.SecretReference, cfgmaps []fv1.ConfigMapReference) (int, error) {
logger := otelUtils.LoggerWithTraceID(ctx, fetcher.logger)
if len(secrets) > 0 {
for _, secret := range secrets {
data, err := fetcher.kubeClient.CoreV1().Secrets(secret.Namespace).Get(context.TODO(), secret.Name, metav1.GetOptions{})
data, err := fetcher.kubeClient.CoreV1().Secrets(secret.Namespace).Get(ctx, secret.Name, metav1.GetOptions{})
if err != nil {
e := "error getting secret from kubeapi"
@@ -366,7 +400,7 @@ func (fetcher *Fetcher) FetchSecretsAndCfgMaps(secrets []fv1.SecretReference, cf
httpCode = http.StatusNotFound
e = "secret was not found in kubeapi"
}
fetcher.logger.Error(e,
logger.Error(e,
zap.Error(err),
zap.String("secret_name", secret.Name),
zap.String("secret_namespace", secret.Namespace))
@@ -379,7 +413,7 @@ func (fetcher *Fetcher) FetchSecretsAndCfgMaps(secrets []fv1.SecretReference, cf
err = os.MkdirAll(secretDir, os.ModeDir|0750)
if err != nil {
e := "failed to create directory for secret"
fetcher.logger.Error(e,
logger.Error(e,
zap.Error(err),
zap.String("directory", secretDir),
zap.String("secret_name", secret.Name),
@@ -388,19 +422,23 @@ func (fetcher *Fetcher) FetchSecretsAndCfgMaps(secrets []fv1.SecretReference, cf
}
err = writeSecretOrConfigMap(data.Data, secretDir)
if err != nil {
fetcher.logger.Error("failed to write secret to file location",
logger.Error("failed to write secret to file location",
zap.Error(err),
zap.String("location", secretDir),
zap.String("secret_name", secret.Name),
zap.String("secret_namespace", secret.Namespace))
return http.StatusInternalServerError, err
}
otelUtils.SpanTrackEvent(ctx, "storedSecret", otelUtils.MapToAttributes(map[string]string{
"secret-name": secret.Name,
"secret-namespace": secret.Namespace,
})...)
}
}
if len(cfgmaps) > 0 {
for _, config := range cfgmaps {
data, err := fetcher.kubeClient.CoreV1().ConfigMaps(config.Namespace).Get(context.TODO(), config.Name, metav1.GetOptions{})
data, err := fetcher.kubeClient.CoreV1().ConfigMaps(config.Namespace).Get(ctx, config.Name, metav1.GetOptions{})
if err != nil {
e := "error getting configmap from kubeapi"
@@ -410,7 +448,7 @@ func (fetcher *Fetcher) FetchSecretsAndCfgMaps(secrets []fv1.SecretReference, cf
httpCode = http.StatusNotFound
e = "configmap was not found in kubeapi"
}
fetcher.logger.Error(e,
logger.Error(e,
zap.Error(err),
zap.String("config_map_name", config.Name),
zap.String("config_map_namespace", config.Namespace))
@@ -423,7 +461,7 @@ func (fetcher *Fetcher) FetchSecretsAndCfgMaps(secrets []fv1.SecretReference, cf
err = os.MkdirAll(configDir, os.ModeDir|0750)
if err != nil {
e := "failed to create directory for configmap"
fetcher.logger.Error(e,
logger.Error(e,
zap.Error(err),
zap.String("directory", configDir),
zap.String("config_map_name", config.Name),
@@ -436,13 +474,17 @@ func (fetcher *Fetcher) FetchSecretsAndCfgMaps(secrets []fv1.SecretReference, cf
}
err = writeSecretOrConfigMap(configMap, configDir)
if err != nil {
fetcher.logger.Error("failed to write configmap to file location",
logger.Error("failed to write configmap to file location",
zap.Error(err),
zap.String("location", configDir),
zap.String("config_map_name", config.Name),
zap.String("config_map_namespace", config.Namespace))
return http.StatusInternalServerError, err
}
otelUtils.SpanTrackEvent(ctx, "storedConfigmap", otelUtils.MapToAttributes(map[string]string{
"configmap-name": config.Name,
"configmap-namespace": config.Namespace,
})...)
}
}
@@ -450,6 +492,9 @@ func (fetcher *Fetcher) FetchSecretsAndCfgMaps(secrets []fv1.SecretReference, cf
}
func (fetcher *Fetcher) UploadHandler(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
logger := otelUtils.LoggerWithTraceID(ctx, fetcher.logger)
if r.Method != "POST" {
http.Error(w, "only POST is supported on this endpoint", http.StatusMethodNotAllowed)
return
@@ -458,13 +503,13 @@ func (fetcher *Fetcher) UploadHandler(w http.ResponseWriter, r *http.Request) {
startTime := time.Now()
defer func() {
elapsed := time.Since(startTime)
fetcher.logger.Info("upload request done", zap.Duration("elapsed_time", elapsed))
logger.Info("upload request done", zap.Duration("elapsed_time", elapsed))
}()
// parse request
body, err := ioutil.ReadAll(r.Body)
if err != nil {
fetcher.logger.Error("error reading request body", zap.Error(err))
logger.Error("error reading request body", zap.Error(err))
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
@@ -472,11 +517,11 @@ func (fetcher *Fetcher) UploadHandler(w http.ResponseWriter, r *http.Request) {
var req ArchiveUploadRequest
err = json.Unmarshal(body, &req)
if err != nil {
fetcher.logger.Error("error parsing request body", zap.Error(err))
logger.Error("error parsing request body", zap.Error(err))
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
fetcher.logger.Info("fetcher received upload request", zap.Any("request", req))
logger.Info("fetcher received upload request", zap.Any("request", req))
zipFilename := req.Filename + ".zip"
srcFilepath := filepath.Join(fetcher.sharedVolumePath, req.Filename)
@@ -486,7 +531,7 @@ func (fetcher *Fetcher) UploadHandler(w http.ResponseWriter, r *http.Request) {
err = fetcher.archive(srcFilepath, dstFilepath)
if err != nil {
e := "error archiving zip file"
fetcher.logger.Error(e, zap.Error(err), zap.String("source", srcFilepath), zap.String("destination", dstFilepath))
logger.Error(e, zap.Error(err), zap.String("source", srcFilepath), zap.String("destination", dstFilepath))
http.Error(w, fmt.Sprintf("%s: %v", e, err), http.StatusInternalServerError)
return
}
@@ -494,19 +539,19 @@ func (fetcher *Fetcher) UploadHandler(w http.ResponseWriter, r *http.Request) {
err = os.Rename(srcFilepath, dstFilepath)
if err != nil {
e := "error renaming the archive"
fetcher.logger.Error(e, zap.Error(err), zap.String("source", srcFilepath), zap.String("destination", dstFilepath))
logger.Error(e, zap.Error(err), zap.String("source", srcFilepath), zap.String("destination", dstFilepath))
http.Error(w, fmt.Sprintf("%s: %v", e, err), http.StatusInternalServerError)
return
}
}
fetcher.logger.Info("starting upload...")
logger.Info("starting upload...")
ssClient := storageSvcClient.MakeClient(req.StorageSvcUrl)
fileID, err := ssClient.Upload(r.Context(), dstFilepath, nil)
fileID, err := ssClient.Upload(ctx, dstFilepath, nil)
if err != nil {
e := "error uploading zip file"
fetcher.logger.Error(e, zap.Error(err), zap.String("file", dstFilepath))
logger.Error(e, zap.Error(err), zap.String("file", dstFilepath))
http.Error(w, fmt.Sprintf("%s: %v", e, err), http.StatusInternalServerError)
return
}
@@ -514,7 +559,7 @@ func (fetcher *Fetcher) UploadHandler(w http.ResponseWriter, r *http.Request) {
sum, err := utils.GetFileChecksum(dstFilepath)
if err != nil {
e := "error calculating checksum of zip file"
fetcher.logger.Error(e, zap.Error(err), zap.String("file", dstFilepath))
logger.Error(e, zap.Error(err), zap.String("file", dstFilepath))
http.Error(w, fmt.Sprintf("%s: %v", e, err), http.StatusInternalServerError)
return
}
@@ -527,18 +572,18 @@ func (fetcher *Fetcher) UploadHandler(w http.ResponseWriter, r *http.Request) {
rBody, err := json.Marshal(resp)
if err != nil {
e := "error encoding upload response"
fetcher.logger.Error(e, zap.Error(err))
logger.Error(e, zap.Error(err))
http.Error(w, fmt.Sprintf("%s: %v", e, err), http.StatusInternalServerError)
return
}
fetcher.logger.Info("completed upload request")
logger.Info("completed upload request")
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
_, err = w.Write(rBody)
if err != nil {
e := "error writing response"
fetcher.logger.Error(e, zap.Error(err))
logger.Error(e, zap.Error(err))
http.Error(w, fmt.Sprintf("%s: %v", e, err), http.StatusInternalServerError)
}
}
@@ -581,11 +626,21 @@ func (fetcher *Fetcher) unarchive(src string, dst string) error {
}
// getPkgInformation gets package information from k8s api server.
func (fetcher *Fetcher) getPkgInformation(req FunctionFetchRequest) (pkg *fv1.Package, err error) {
func (fetcher *Fetcher) getPkgInformation(ctx context.Context, req FunctionFetchRequest) (pkg *fv1.Package, err error) {
logger := otelUtils.LoggerWithTraceID(ctx, fetcher.logger)
maxRetries := 5
for i := 0; i < maxRetries; i++ {
pkg, err = fetcher.fissionClient.CoreV1().Packages(req.Package.Namespace).Get(context.TODO(), req.Package.Name, metav1.GetOptions{})
otelUtils.SpanTrackEvent(ctx, "fetchPkgInfo", otelUtils.MapToAttributes(map[string]string{
"package_name": req.Package.Name,
"package_namespace": req.Package.Namespace,
"retry_count": strconv.Itoa(i),
})...)
// TODO: pass resource version in the GetOptions, added warning for now
pkg, err = fetcher.fissionClient.CoreV1().Packages(req.Package.Namespace).Get(ctx, req.Package.Name, metav1.GetOptions{})
if err == nil {
if req.Package.ResourceVersion != pkg.ResourceVersion {
logger.Warn("package resource version mismatch", zap.String("pkgName", req.Package.Name), zap.String("pkgNamespace", req.Package.Namespace), zap.String("pkgResourceVersion", req.Package.ResourceVersion), zap.String("fetchedResourceVersion", pkg.ResourceVersion))
}
return pkg, nil
}
if i < maxRetries-1 {
@@ -612,13 +667,14 @@ func (fetcher *Fetcher) getPkgInformation(req FunctionFetchRequest) (pkg *fv1.Pa
}
func (fetcher *Fetcher) SpecializePod(ctx context.Context, fetchReq FunctionFetchRequest, loadReq FunctionLoadRequest) error {
logger := otelUtils.LoggerWithTraceID(ctx, fetcher.logger)
startTime := time.Now()
defer func() {
elapsed := time.Since(startTime)
fetcher.logger.Info("specialize request done", zap.Duration("elapsed_time", elapsed))
logger.Info("specialize request done", zap.Duration("elapsed_time", elapsed))
}()
pkg, err := fetcher.getPkgInformation(fetchReq)
pkg, err := fetcher.getPkgInformation(ctx, fetchReq)
if err != nil {
return errors.Wrap(err, "error getting package information")
}
@@ -628,7 +684,7 @@ func (fetcher *Fetcher) SpecializePod(ctx context.Context, fetchReq FunctionFetc
return errors.Wrap(err, "error fetching deploy package")
}
_, err = fetcher.FetchSecretsAndCfgMaps(fetchReq.Secrets, fetchReq.ConfigMaps)
_, err = fetcher.FetchSecretsAndCfgMaps(ctx, fetchReq.Secrets, fetchReq.ConfigMaps)
if err != nil {
return errors.Wrap(err, "error fetching secrets/configs")
}
@@ -652,16 +708,19 @@ func (fetcher *Fetcher) SpecializePod(ctx context.Context, fetchReq FunctionFetc
contentType = "application/json"
specializeURL = "http://127.0.0.1:8888/v2/specialize"
reader = bytes.NewReader(loadPayload)
fetcher.logger.Info("calling environment v2 specialization endpoint")
logger.Info("calling environment v2 specialization endpoint")
} else {
contentType = "text/plain"
specializeURL = "http://127.0.0.1:8888/specialize"
reader = bytes.NewReader([]byte{})
fetcher.logger.Info("calling environment v1 specialization endpoint")
logger.Info("calling environment v1 specialization endpoint")
}
for i := 0; i < maxRetries; i++ {
resp, err := http.Post(specializeURL, contentType, reader)
otelUtils.SpanTrackEvent(ctx, "specializeCall", otelUtils.MapToAttributes(map[string]string{
"url": specializeURL,
})...)
resp, err := ctxhttp.Post(ctx, fetcher.httpClient, specializeURL, contentType, reader)
if err == nil && resp.StatusCode < 300 {
// Success
resp.Body.Close()
@@ -673,7 +732,7 @@ func (fetcher *Fetcher) SpecializePod(ctx context.Context, fetchReq FunctionFetc
if netErr != nil && (netErr.IsConnRefusedError() || netErr.IsDialError()) {
if i < maxRetries-1 {
time.Sleep(500 * time.Duration(2*i) * time.Millisecond)
fetcher.logger.Error("error connecting to function environment pod for specialization request, retrying", zap.Error(netErr))
logger.Error("error connecting to function environment pod for specialization request, retrying", zap.Error(netErr))
continue
}
}
@@ -691,6 +750,8 @@ func (fetcher *Fetcher) SpecializePod(ctx context.Context, fetchReq FunctionFetc
// WsStartHandler is used to generate websocket events in Kubernetes
func (fetcher *Fetcher) WsStartHandler(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
logger := otelUtils.LoggerWithTraceID(ctx, fetcher.logger)
if r.Method != "GET" {
http.Error(w, "only GET is supported on this endpoint", http.StatusMethodNotAllowed)
return
@@ -700,27 +761,31 @@ func (fetcher *Fetcher) WsStartHandler(w http.ResponseWriter, r *http.Request) {
klog.Errorf("Error creating recorder %s", err)
http.Error(w, err.Error(), http.StatusInternalServerError)
}
pods, err := fetcher.kubeClient.CoreV1().Pods(fetcher.Info.Namespace).List(context.TODO(), metav1.ListOptions{
pods, err := fetcher.kubeClient.CoreV1().Pods(fetcher.Info.Namespace).List(ctx, metav1.ListOptions{
FieldSelector: "metadata.name=" + fetcher.Info.Name,
})
if err != nil {
fetcher.logger.Error("Failed to get the pod", zap.Error(err))
logger.Error("Failed to get the pod", zap.Error(err))
http.Error(w, err.Error(), http.StatusInternalServerError)
}
for _, pod := range pods.Items {
ref, err := reference.GetReference(scheme.Scheme, &pod)
if err != nil {
fetcher.logger.Error("Could not get reference for pod", zap.Error(err))
logger.Error("Could not get reference for pod", zap.Error(err))
http.Error(w, err.Error(), http.StatusInternalServerError)
}
rec.Event(ref, corev1.EventTypeNormal, "WsConnectionStarted", "Websocket connection has been formed on this pod")
fetcher.logger.Info("Sent websocket initiation event")
logger.Info("Sent websocket initiation event")
}
w.WriteHeader(http.StatusOK)
}
// WsEndHandler is used to generate inactive events in Kubernetes
func (fetcher *Fetcher) WsEndHandler(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
logger := otelUtils.LoggerWithTraceID(ctx, fetcher.logger)
if r.Method != "GET" {
http.Error(w, "only GET is supported on this endpoint", http.StatusMethodNotAllowed)
return
@@ -730,23 +795,23 @@ func (fetcher *Fetcher) WsEndHandler(w http.ResponseWriter, r *http.Request) {
klog.Errorf("Error creating recorder %s", err)
http.Error(w, err.Error(), http.StatusInternalServerError)
}
pods, err := fetcher.kubeClient.CoreV1().Pods(fetcher.Info.Namespace).List(context.TODO(), metav1.ListOptions{
pods, err := fetcher.kubeClient.CoreV1().Pods(fetcher.Info.Namespace).List(ctx, metav1.ListOptions{
FieldSelector: "metadata.name=" + fetcher.Info.Name,
})
if err != nil {
fetcher.logger.Error("Failed to get the pod", zap.Error(err))
logger.Error("Failed to get the pod", zap.Error(err))
http.Error(w, err.Error(), http.StatusInternalServerError)
}
for _, pod := range pods.Items {
// There will only be one time since we've used field selector
ref, err := reference.GetReference(scheme.Scheme, &pod)
if err != nil {
fetcher.logger.Error("Could not get reference for pod", zap.Error(err))
logger.Error("Could not get reference for pod", zap.Error(err))
http.Error(w, err.Error(), http.StatusInternalServerError)
}
// We could use Eventf and supply the amount of time the connection was inactive although, in case of multiple connections, it doesn't make sense
rec.Event(ref, corev1.EventTypeNormal, "NoActiveConnections", "Connection has been inactive")
fetcher.logger.Info("Sent no active connections event")
logger.Info("Sent no active connections event")
}
w.WriteHeader(http.StatusOK)
}
+18 -2
View File
@@ -26,6 +26,8 @@ import (
"time"
"github.com/pkg/errors"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
"go.opentelemetry.io/otel"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission/pkg/controller/client"
@@ -35,6 +37,7 @@ import (
"github.com/fission/fission/pkg/fission-cli/console"
flagkey "github.com/fission/fission/pkg/fission-cli/flag/key"
"github.com/fission/fission/pkg/fission-cli/util"
otelUtils "github.com/fission/fission/pkg/utils/otel"
)
type TestSubCommand struct {
@@ -153,7 +156,19 @@ func (opts *TestSubCommand) do(input cli.Input) error {
}
func doHTTPRequest(ctx context.Context, url string, headers []string, method, body string) (*http.Response, error) {
method, err := httptrigger.GetMethod(method)
shutdown, err := otelUtils.InitProvider(ctx, nil, "fission-cli")
if err != nil {
return nil, err
}
if shutdown != nil {
defer shutdown(ctx)
}
tracer := otel.Tracer("fission-cli")
ctx, span := tracer.Start(ctx, "httpRequest")
defer span.End()
method, err = httptrigger.GetMethod(method)
if err != nil {
return nil, err
}
@@ -170,7 +185,8 @@ func doHTTPRequest(ctx context.Context, url string, headers []string, method, bo
}
req.Header.Set(headerKeyValue[0], headerKeyValue[1])
}
resp, err := http.DefaultClient.Do(req.WithContext(ctx))
hc := &http.Client{Transport: otelhttp.NewTransport(http.DefaultTransport)}
resp, err := hc.Do(req.WithContext(ctx))
if err != nil {
return nil, errors.Wrap(err, "error executing HTTP request")
}
+4 -4
View File
@@ -113,7 +113,7 @@ var (
FnLogReverseQuery = Flag{Type: Bool, Name: flagkey.FnLogReverseQuery, Short: "r", Usage: "Specify the log reverse query base on time, it will be invalid if the 'follow' flag is specified"}
FnLogCount = Flag{Type: Int, Name: flagkey.FnLogCount, Usage: "Get N most recent log records", DefaultValue: 20}
FnTestBody = Flag{Type: String, Name: flagkey.FnTestBody, Short: "b", Usage: "Request body"}
FnTestTimeout = Flag{Type: Duration, Name: flagkey.FnTestTimeout, Short: "t", Usage: "Length of time to wait for the response. If set to zero or negative number, no timeout is set", DefaultValue: 30 * time.Second}
FnTestTimeout = Flag{Type: Duration, Name: flagkey.FnTestTimeout, Short: "t", Usage: "Length of time to wait for the response. If set to zero or negative number, no timeout is set", DefaultValue: 60 * time.Second}
FnTestHeader = Flag{Type: StringSlice, Name: flagkey.FnTestHeader, Short: "H", Usage: "Request headers"}
FnTestQuery = Flag{Type: StringSlice, Name: flagkey.FnTestQuery, Short: "q", Usage: "Request query parameters: -q key1=value1 -q key2=value2"}
FnIdleTimeout = Flag{Type: Int, Name: flagkey.FnIdleTimeout, Usage: "The length of time (in seconds) that a function is idle before pod(s) are eligible for recycling", DefaultValue: 120}
@@ -123,8 +123,8 @@ var (
FnSubPath = Flag{Type: String, Name: flagkey.FnSubPath, Usage: "Sub Path to check if function internally supports routing"}
HtName = Flag{Type: String, Name: flagkey.HtName, Usage: "HTTP trigger name"}
HtMethod = Flag{Type: StringSlice, Name: flagkey.HtMethod, Usage: "HTTP Methods: GET,POST,PUT,DELETE,HEAD. To mention single method: --method GET and for multiple methods --method GET --method POST.", DefaultValue: []string{http.MethodGet}}
HtUrl = Flag{Type: String, Name: flagkey.HtUrl, Usage: "URL pattern (See gorilla/mux supported patterns)"}
HtMethod = Flag{Type: StringSlice, Name: flagkey.HtMethod, Usage: "HTTP Methods: GET,POST,PUT,DELETE,HEAD. To mention single method: --method GET and for multiple methods --method GET --method POST. [DEPRECATED for 'fn create', use 'route create' instead]", DefaultValue: []string{http.MethodGet}}
HtUrl = Flag{Type: String, Name: flagkey.HtUrl, Usage: "URL pattern (See gorilla/mux supported patterns) [DEPRECATED for 'fn create', use 'route create' instead]"}
HtHost = Flag{Type: String, Name: flagkey.HtHost, Usage: "Use --ingressrule instead", Deprecated: true, Substitute: flagkey.HtIngressRule}
HtIngress = Flag{Type: Bool, Name: flagkey.HtIngress, Usage: "Creates ingress with same URL"}
HtIngressRule = Flag{Type: String, Name: flagkey.HtIngressRule, Usage: "Host for Ingress rule: --ingressrule host=path (the format of host/path depends on what ingress controller you used)"}
@@ -133,7 +133,7 @@ var (
HtFnName = Flag{Type: StringSlice, Name: flagkey.HtFnName, Usage: "Name(s) of the function for this trigger. (If 2 functions are supplied with this flag, traffic gets routed to them based on weights supplied with --weight flag.)"}
HtFnWeight = Flag{Type: IntSlice, Name: flagkey.HtFnWeight, Usage: "Weight for each function supplied with --function flag, in the same order. Used for canary deployment"}
HtFnFilter = Flag{Type: String, Name: flagkey.HtFilter, Usage: "Name of the function for trigger(s)"}
HtPrefix = Flag{Type: String, Name: flagkey.HtPrefix, Usage: "Prefix with which functions are exposed. NOTE: Prefix takes precedence over URL/RelativeURL"}
HtPrefix = Flag{Type: String, Name: flagkey.HtPrefix, Usage: "Prefix with which functions are exposed. NOTE: Prefix takes precedence over URL/RelativeURL [DEPRECATED for 'fn create', use 'route create' instead]"}
HtKeepPrefix = Flag{Type: Bool, Name: flagkey.HtKeepPrefix, Usage: "Keep the prefix in the URL while forwarding request to the function"}
TtName = Flag{Type: String, Name: flagkey.TtName, Usage: "Time Trigger name"}
+1 -1
View File
@@ -178,7 +178,7 @@ func runPortForward(labelSelector string, localPort string, ns string, kubeConte
for _, servicePort := range service.Spec.Ports {
targetPort = servicePort.TargetPort.String()
}
console.Verbose(2, "Connecting to port %v on pod %v/%v", targetPort, podNameSpace, podNameSpace)
console.Verbose(2, "Connecting to port %v on pod %v/%v", targetPort, podNameSpace, podName)
stopChannel := make(chan struct{}, 1)
readyChannel := make(chan struct{})
+9 -20
View File
@@ -25,7 +25,6 @@ import (
"time"
"go.uber.org/zap"
"go.uber.org/zap/zapcore"
corev1 "k8s.io/api/core/v1"
k8sInformers "k8s.io/client-go/informers"
k8sCache "k8s.io/client-go/tools/cache"
@@ -33,6 +32,7 @@ import (
fv1 "github.com/fission/fission/pkg/apis/core/v1"
"github.com/fission/fission/pkg/crd"
"github.com/fission/fission/pkg/utils"
"github.com/fission/fission/pkg/utils/loggerfactory"
)
var nodeName = os.Getenv("NODE_NAME")
@@ -158,36 +158,25 @@ func symlinkReaper(zapLogger *zap.Logger) {
}
func Start() {
config := zap.NewProductionConfig()
config.EncoderConfig.EncodeTime = zapcore.ISO8601TimeEncoder
zapLogger, err := config.Build()
if err != nil {
log.Fatalf("can't initialize zap logger: %v", err)
}
defer func() {
err := zapLogger.Sync()
if err != nil {
log.Fatalf("failed to sync zap logger: %v", err)
}
}()
logger := loggerfactory.GetLogger()
defer logger.Sync()
if _, err := os.Stat(fissionSymlinkPath); os.IsNotExist(err) {
zapLogger.Info("symlink path not exist, create it",
logger.Info("symlink path not exist, create it",
zap.String("fissionSymlinkPath", fissionSymlinkPath))
err = os.Mkdir(fissionSymlinkPath, 0755)
if err != nil {
zapLogger.Fatal("error creating fissionSymlinkPath", zap.Error(err))
logger.Fatal("error creating fissionSymlinkPath", zap.Error(err))
}
}
go symlinkReaper(zapLogger)
go symlinkReaper(logger)
_, kubernetesClient, _, _, err := crd.MakeFissionClient()
if err != nil {
log.Fatalf("Error starting pod watcher: %v", err)
}
informerFactory := k8sInformers.NewSharedInformerFactory(kubernetesClient, 30*time.Second)
informerFactory := k8sInformers.NewSharedInformerFactory(kubernetesClient, time.Minute*30)
podInformer := informerFactory.Core().V1().Pods().Informer()
podInformer.AddEventHandler(podInformerHandlers(zapLogger))
podInformer.AddEventHandler(podInformerHandlers(logger))
podInformer.Run(make(chan struct{}))
zapLogger.Fatal("Stop watching pod changes")
logger.Fatal("Stop watching pod changes")
}
+1 -1
View File
@@ -158,7 +158,7 @@ func StartScalerManager(logger *zap.Logger, routerURL string) error {
if err != nil {
return errors.Wrap(err, "error waiting for CRDs")
}
informerFactory := genInformer.NewSharedInformerFactory(fissionClient, 30*time.Second)
informerFactory := genInformer.NewSharedInformerFactory(fissionClient, time.Minute*30)
mqTriggerInformer := informerFactory.Core().V1().MessageQueueTriggers().Informer()
mqTriggerInformer.AddEventHandler(mqTriggerEventHandlers(logger, kubeClient, routerURL))
mqTriggerInformer.Run(context.Background().Done())
+62 -21
View File
@@ -32,6 +32,7 @@ import (
"github.com/pkg/errors"
"go.opencensus.io/plugin/ochttp"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
"go.uber.org/zap"
k8stypes "k8s.io/apimachinery/pkg/types"
@@ -40,8 +41,10 @@ import (
ferror "github.com/fission/fission/pkg/error"
"github.com/fission/fission/pkg/error/network"
executorClient "github.com/fission/fission/pkg/executor/client"
"github.com/fission/fission/pkg/router/util"
"github.com/fission/fission/pkg/throttler"
"github.com/fission/fission/pkg/utils"
otelUtils "github.com/fission/fission/pkg/utils/otel"
)
const (
@@ -66,6 +69,7 @@ type (
svcAddrUpdateThrottler *throttler.Throttler
functionTimeoutMap map[k8stypes.UID]int
unTapServiceTimeout time.Duration
openTracingEnabled bool
}
tsRoundTripperParams struct {
@@ -154,10 +158,11 @@ func (w *fakeCloseReadCloser) RealClose() error {
// Earlier, GetServiceForFunction was called inside handler function and fission explicitly set http status code to 500
// if it returned an error.
func (roundTripper *RetryingRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
ctx := req.Context()
// set the timeout for transport context
roundTripper.addForwardedHostHeader(req)
transport := roundTripper.getDefaultTransport()
ocRoundTripper := &ochttp.Transport{Base: transport}
executingTimeout := roundTripper.funcHandler.tsRoundTripperParams.timeout
@@ -190,7 +195,7 @@ func (roundTripper *RetryingRoundTripper) RoundTrip(req *http.Request) (*http.Re
var err error
var fnMeta = &roundTripper.funcHandler.function.ObjectMeta
logger := roundTripper.logger.With(zap.String("function", fnMeta.Name), zap.String("namespace", fnMeta.Namespace))
logger := otelUtils.LoggerWithTraceID(ctx, roundTripper.logger).With(zap.String("function", fnMeta.Name), zap.String("namespace", fnMeta.Namespace))
dumpReqFunc := func(request *http.Request) {
if request == nil {
@@ -219,8 +224,11 @@ func (roundTripper *RetryingRoundTripper) RoundTrip(req *http.Request) (*http.Re
// set service url of target service of request only when
// trying to get new service url from cache/executor.
if retryCounter == 0 {
otelUtils.SpanTrackEvent(ctx, "getServiceEntry", otelUtils.MapToAttributes(map[string]string{
"function-name": fnMeta.Name,
"function-namespace": fnMeta.Namespace})...)
// get function service url from cache or executor
roundTripper.serviceURL, roundTripper.urlFromCache, err = roundTripper.funcHandler.getServiceEntry()
roundTripper.serviceURL, roundTripper.urlFromCache, err = roundTripper.funcHandler.getServiceEntry(ctx)
if err != nil {
// We might want a specific error code or header for fission failures as opposed to
// user function bugs.
@@ -248,10 +256,14 @@ func (roundTripper *RetryingRoundTripper) RoundTrip(req *http.Request) (*http.Re
executingTimeout = executingTimeout * time.Duration(roundTripper.funcHandler.tsRoundTripperParams.timeoutExponent)
continue
}
otelUtils.SpanTrackEvent(ctx, "serviceEntryReceived", otelUtils.MapToAttributes(map[string]string{
"function-name": fnMeta.Name,
"function-namespace": fnMeta.Namespace,
"service-entry": roundTripper.serviceURL.String()})...)
if roundTripper.funcHandler.function.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType == fv1.ExecutorTypePoolmgr {
defer func(fn *fv1.Function, serviceURL *url.URL) {
defer func(ctx context.Context, fn *fv1.Function, serviceURL *url.URL) {
go roundTripper.funcHandler.unTapService(fn, serviceURL) //nolint errcheck
}(roundTripper.funcHandler.function, roundTripper.serviceURL)
}(ctx, roundTripper.funcHandler.function, roundTripper.serviceURL)
}
// modify the request to reflect the service url
@@ -309,8 +321,28 @@ func (roundTripper *RetryingRoundTripper) RoundTrip(req *http.Request) (*http.Re
dumpReqFunc(newReq)
}
// The otelhttp.NewTransport() does not work with WebSocket.
// This is probably because it modifies the response body.
// Until we find a better solution to handle websocket requests, we will continue to
// use ochttp.Transport(). We check if the request isWebsocketRequest() and use the
// ochttp.Transport() irrespective of open telemetry is enabled or not.
// Related issue: https://github.com/open-telemetry/opentelemetry-js-contrib/issues/12
// forward the request to the function service
resp, err := ocRoundTripper.RoundTrip(newReq)
var resp *http.Response
if roundTripper.funcHandler.openTracingEnabled || util.IsWebsocketRequest(newReq) {
ocRoundTripper := &ochttp.Transport{Base: transport}
resp, err = ocRoundTripper.RoundTrip(newReq)
} else {
otelUtils.SpanTrackEvent(ctx, "roundtrip", otelUtils.MapToAttributes(map[string]string{
"function-name": fnMeta.Name,
"function-namespace": fnMeta.Namespace,
"function-url": newReq.URL.String(),
"retryCounter": fmt.Sprintf("%d", retryCounter)})...)
otelRoundTripper := otelhttp.NewTransport(transport)
resp, err = otelRoundTripper.RoundTrip(newReq)
}
if err == nil {
// return response back to user
if roundTripper.funcHandler.isDebugEnv {
@@ -494,6 +526,7 @@ func (fh functionHandler) handler(responseWriter http.ResponseWriter, request *h
rrt.closeContext()
}()
otelUtils.SpanTrackEvent(request.Context(), "functionRequestProxy", otelUtils.GetAttributesForFunction(fh.function)...)
proxy.ServeHTTP(responseWriter, request)
}
@@ -626,19 +659,25 @@ func (fh functionHandler) removeServiceEntryFromCache() {
}
}
func (fh functionHandler) getServiceEntryFromExecutor() (serviceUrl *url.URL, err error) {
func (fh functionHandler) getServiceEntryFromExecutor(ctx context.Context) (serviceUrl *url.URL, err error) {
logger := otelUtils.LoggerWithTraceID(ctx, fh.logger)
// send a request to executor to specialize a new pod
fh.logger.Debug("function timeout specified", zap.Int("timeout", fh.function.Spec.FunctionTimeout))
timeout := 30 * time.Second
var fContext context.Context
if fh.function.Spec.FunctionTimeout > 0 {
timeout = time.Second * time.Duration(fh.function.Spec.FunctionTimeout)
timeout := time.Second * time.Duration(fh.function.Spec.FunctionTimeout)
f, cancel := context.WithTimeout(ctx, timeout)
fContext = f
defer cancel()
} else {
fContext = ctx
}
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
service, err := fh.executor.GetServiceForFunction(ctx, fh.function)
service, err := fh.executor.GetServiceForFunction(fContext, fh.function)
if err != nil {
statusCode, errMsg := ferror.GetHTTPError(err)
fh.logger.Error("error from GetServiceForFunction",
logger.Error("error from GetServiceForFunction",
zap.Error(err),
zap.String("error_message", errMsg),
zap.Any("function", fh.function),
@@ -648,7 +687,7 @@ func (fh functionHandler) getServiceEntryFromExecutor() (serviceUrl *url.URL, er
// parse the address into url
svcURL, err := url.Parse(fmt.Sprintf("http://%v", service))
if err != nil {
fh.logger.Error("error parsing service url",
logger.Error("error parsing service url",
zap.Error(err),
zap.String("service_url", svcURL.String()))
return nil, err
@@ -657,9 +696,9 @@ func (fh functionHandler) getServiceEntryFromExecutor() (serviceUrl *url.URL, er
}
// getServiceEntryFromExecutor returns service url entry returns from executor
func (fh functionHandler) getServiceEntry() (svcURL *url.URL, cacheHit bool, err error) {
func (fh functionHandler) getServiceEntry(ctx context.Context) (svcURL *url.URL, cacheHit bool, err error) {
if fh.function.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType == fv1.ExecutorTypePoolmgr {
svcURL, err = fh.getServiceEntryFromExecutor()
svcURL, err = fh.getServiceEntryFromExecutor(ctx)
return svcURL, false, err
}
// Check if service URL present in cache
@@ -681,7 +720,7 @@ func (fh functionHandler) getServiceEntry() (svcURL *url.URL, cacheHit bool, err
}
return svcEntryRecord{svcURL: svcURL, cacheHit: true}, err
}
svcURL, err = fh.getServiceEntryFromExecutor()
svcURL, err = fh.getServiceEntryFromExecutor(ctx)
if err != nil {
return nil, err
}
@@ -705,6 +744,8 @@ func (fh functionHandler) getProxyErrorHandler(start time.Time, rrt *RetryingRou
return func(rw http.ResponseWriter, req *http.Request, err error) {
var status int
var msg string
ctx := req.Context()
logger := otelUtils.LoggerWithTraceID(ctx, fh.logger)
switch err {
case context.Canceled:
// 499 CLIENT CLOSED REQUEST
@@ -713,16 +754,16 @@ func (fh functionHandler) getProxyErrorHandler(start time.Time, rrt *RetryingRou
// Reference: https://httpstatuses.com/499
status = 499
msg = "client closes the connection"
fh.logger.Debug(msg, zap.Any("function", fh.function), zap.String("status", "Client Closed Request"))
logger.Debug(msg, zap.Any("function", fh.function), zap.String("status", "Client Closed Request"))
case context.DeadlineExceeded:
status = http.StatusGatewayTimeout
msg := "no response from function before timeout"
fh.logger.Error(msg, zap.Any("function", fh.function), zap.String("status", http.StatusText(status)))
logger.Error(msg, zap.Any("function", fh.function), zap.String("status", http.StatusText(status)))
default:
code, _ := ferror.GetHTTPError(err)
status = code
msg = "error sending request to function"
fh.logger.Error(msg, zap.Error(err), zap.Any("function", fh.function),
logger.Error(msg, zap.Error(err), zap.Any("function", fh.function),
zap.Any("status", http.StatusText(status)), zap.Int("code", code))
}
@@ -735,7 +776,7 @@ func (fh functionHandler) getProxyErrorHandler(start time.Time, rrt *RetryingRou
rw.WriteHeader(status)
_, err = rw.Write([]byte(msg))
if err != nil {
fh.logger.Error(
logger.Error(
"error writing HTTP response",
zap.Error(err),
zap.Any("function", fh.function),
+60 -11
View File
@@ -19,6 +19,7 @@ package router
import (
"context"
"net/http"
"strings"
"time"
"github.com/gorilla/mux"
@@ -33,6 +34,8 @@ import (
genInformer "github.com/fission/fission/pkg/generated/informers/externalversions"
"github.com/fission/fission/pkg/throttler"
"github.com/fission/fission/pkg/utils"
"github.com/fission/fission/pkg/utils/otel"
"github.com/fission/fission/pkg/utils/tracing"
)
// HTTPTriggerSet represents an HTTP trigger set
@@ -73,7 +76,7 @@ func makeHTTPTriggerSet(logger *zap.Logger, fmap *functionServiceMap, fissionCli
unTapServiceTimeout: unTapServiceTimeout,
}
informerFactory := genInformer.NewSharedInformerFactory(fissionClient, time.Second*30)
informerFactory := genInformer.NewSharedInformerFactory(fissionClient, time.Minute*30)
httpTriggerSet.triggerInformer = informerFactory.Core().V1().HTTPTriggers().Informer()
httpTriggerSet.funcInformer = informerFactory.Core().V1().Functions().Informer()
@@ -110,6 +113,8 @@ func routerHealthHandler(w http.ResponseWriter, r *http.Request) {
func (ts *HTTPTriggerSet) getRouter(fnTimeoutMap map[types.UID]int) *mux.Router {
muxRouter := mux.NewRouter()
openTracingEnabled := tracing.TracingEnabled(ts.logger)
// HTTP triggers setup by the user
homeHandled := false
for i := range ts.triggers {
@@ -143,6 +148,7 @@ func (ts *HTTPTriggerSet) getRouter(fnTimeoutMap map[types.UID]int) *mux.Router
svcAddrUpdateThrottler: ts.svcAddrUpdateThrottler,
functionTimeoutMap: fnTimeoutMap,
unTapServiceTimeout: ts.unTapServiceTimeout,
openTracingEnabled: openTracingEnabled,
}
// The functionHandler for HTTP trigger with fn reference type "FunctionReferenceTypeFunctionName",
@@ -157,13 +163,7 @@ func (ts *HTTPTriggerSet) getRouter(fnTimeoutMap map[types.UID]int) *mux.Router
fh.function = fn
}
}
var ht *mux.Route
if trigger.Spec.Prefix != nil && *trigger.Spec.Prefix != "" {
ht = muxRouter.PathPrefix(*trigger.Spec.Prefix).HandlerFunc(fh.handler)
} else {
ht = muxRouter.HandleFunc(trigger.Spec.RelativeURL, fh.handler)
}
methods := trigger.Spec.Methods
if len(trigger.Spec.Method) > 0 {
present := false
@@ -177,9 +177,47 @@ func (ts *HTTPTriggerSet) getRouter(fnTimeoutMap map[types.UID]int) *mux.Router
methods = append(methods, trigger.Spec.Method)
}
}
ht.Methods(methods...)
if trigger.Spec.Host != "" {
ht.Host(trigger.Spec.Host)
var handler http.Handler
if openTracingEnabled {
handler = http.HandlerFunc(fh.handler)
} else {
if trigger.Spec.Prefix != nil && *trigger.Spec.Prefix != "" {
handler = otel.GetHandlerWithOTEL(http.HandlerFunc(fh.handler), *trigger.Spec.Prefix)
} else {
handler = otel.GetHandlerWithOTEL(http.HandlerFunc(fh.handler), trigger.Spec.RelativeURL)
}
}
if trigger.Spec.Prefix != nil && *trigger.Spec.Prefix != "" {
prefix := *trigger.Spec.Prefix
if strings.HasSuffix(prefix, "/") {
ht := muxRouter.PathPrefix(prefix).Handler(handler)
ht.Methods(methods...)
if trigger.Spec.Host != "" {
ht.Host(trigger.Spec.Host)
}
ts.logger.Debug("add prefix route for function", zap.String("route", prefix), zap.Any("function", fh.function), zap.Strings("methods", methods))
} else {
ht1 := muxRouter.Handle(prefix, handler)
ht1.Methods(methods...)
if trigger.Spec.Host != "" {
ht1.Host(trigger.Spec.Host)
}
ht2 := muxRouter.PathPrefix(prefix + "/").Handler(handler)
ht2.Methods(methods...)
if trigger.Spec.Host != "" {
ht2.Host(trigger.Spec.Host)
}
ts.logger.Debug("add prefix and handler route for function", zap.String("route", prefix), zap.Any("function", fh.function), zap.Strings("methods", methods))
}
} else {
ht := muxRouter.Handle(trigger.Spec.RelativeURL, handler)
ht.Methods(methods...)
if trigger.Spec.Host != "" {
ht.Host(trigger.Spec.Host)
}
ts.logger.Debug("add handler route for function", zap.String("router", trigger.Spec.RelativeURL), zap.Any("function", fh.function), zap.Strings("methods", methods))
}
if trigger.Spec.Prefix == nil && trigger.Spec.RelativeURL == "/" && len(methods) == 1 && methods[0] == http.MethodGet {
@@ -212,7 +250,18 @@ func (ts *HTTPTriggerSet) getRouter(fnTimeoutMap map[types.UID]int) *mux.Router
functionTimeoutMap: fnTimeoutMap,
unTapServiceTimeout: ts.unTapServiceTimeout,
}
muxRouter.PathPrefix(utils.UrlForFunction(fn.ObjectMeta.Name, fn.ObjectMeta.Namespace)).HandlerFunc(fh.handler)
var handler http.Handler
internalRoute := utils.UrlForFunction(fn.ObjectMeta.Name, fn.ObjectMeta.Namespace)
internalPrefixRoute := internalRoute + "/"
if openTracingEnabled {
handler = http.HandlerFunc(fh.handler)
} else {
handler = otel.GetHandlerWithOTEL(http.HandlerFunc(fh.handler), internalRoute)
}
muxRouter.Handle(internalRoute, handler)
muxRouter.PathPrefix(internalPrefixRoute).Handler(handler)
ts.logger.Debug("add internal handler and prefix route for function", zap.String("router", internalRoute), zap.Any("function", fn))
}
// Healthz endpoint for the router.
+36 -27
View File
@@ -53,11 +53,13 @@ import (
"github.com/prometheus/client_golang/prometheus/promhttp"
"go.opencensus.io/plugin/ochttp"
"go.opencensus.io/trace"
"go.opentelemetry.io/otel"
"go.uber.org/zap"
"github.com/fission/fission/pkg/crd"
executorClient "github.com/fission/fission/pkg/executor/client"
"github.com/fission/fission/pkg/throttler"
otelUtils "github.com/fission/fission/pkg/utils/otel"
)
// request url ---[mux]---> Function(name,uid) ----[fmap]----> k8s service url
@@ -80,36 +82,38 @@ func router(ctx context.Context, logger *zap.Logger, httpTriggerSet *HTTPTrigger
}
func serve(ctx context.Context, logger *zap.Logger, port int, tracingSamplingRate float64,
httpTriggerSet *HTTPTriggerSet, displayAccessLog bool) {
httpTriggerSet *HTTPTriggerSet, displayAccessLog bool, openTracingEnabled bool) {
mr := router(ctx, logger, httpTriggerSet)
url := fmt.Sprintf(":%v", port)
err := http.ListenAndServe(url, &ochttp.Handler{
Handler: mr,
GetStartOptions: func(r *http.Request) trace.StartOptions {
// do not trace router healthz endpoint
if strings.Compare(r.URL.Path, "/router-healthz") == 0 {
var err error
if openTracingEnabled {
err = http.ListenAndServe(url, &ochttp.Handler{
Handler: mr,
GetStartOptions: func(r *http.Request) trace.StartOptions {
// do not trace router healthz endpoint
if strings.Compare(r.URL.Path, "/router-healthz") == 0 {
return trace.StartOptions{
Sampler: trace.NeverSample(),
}
}
if displayAccessLog {
reqMsg, err := httputil.DumpRequest(r, false)
if err != nil {
logger.Error("error dumping request", zap.Error(err))
}
logger.Info("request dump", zap.String("request", string(reqMsg)))
}
return trace.StartOptions{
Sampler: trace.NeverSample(),
Sampler: trace.ProbabilitySampler(tracingSamplingRate),
}
}
if displayAccessLog {
reqMsg, err := httputil.DumpRequest(r, false)
if err != nil {
logger.Error("error dumping request", zap.Error(err))
}
logger.Info("request dump", zap.String("request", string(reqMsg)))
}
return trace.StartOptions{
Sampler: trace.ProbabilitySampler(tracingSamplingRate),
}
},
})
},
})
} else {
err = http.ListenAndServe(url, otelUtils.GetHandlerWithOTEL(mr, "fission-router", otelUtils.UrlsToIgnore("/router-healthz")))
}
if err != nil {
logger.Error(
"HTTP server error",
zap.Error(err),
)
logger.Error("HTTP server error", zap.Error(err))
}
}
@@ -122,7 +126,7 @@ func serveMetric(logger *zap.Logger) {
}
// Start starts a router
func Start(logger *zap.Logger, port int, executorURL string) {
func Start(logger *zap.Logger, port int, executorURL string, openTracingEnabled bool) {
fmap := makeFunctionServiceMap(logger, time.Minute)
fissionClient, kubeClient, _, _, err := crd.MakeFissionClient()
@@ -252,7 +256,12 @@ func Start(logger *zap.Logger, port int, executorURL string) {
go serveMetric(logger)
logger.Info("starting router", zap.Int("port", port))
ctx, cancel := context.WithCancel(context.Background())
tracer := otel.Tracer("router")
ctx, span := tracer.Start(context.Background(), "router/Start")
defer span.End()
ctxWithCancel, cancel := context.WithCancel(ctx)
defer cancel()
serve(ctx, logger, port, tracingSamplingRate, triggers, displayAccessLog)
serve(ctxWithCancel, logger, port, tracingSamplingRate, triggers, displayAccessLog, openTracingEnabled)
}
+10 -1
View File
@@ -17,6 +17,8 @@ limitations under the License.
package util
import (
"net/http"
v1 "k8s.io/api/networking/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -53,6 +55,7 @@ func GetIngressSpec(namespace string, trigger *fv1.HTTPTrigger) *v1.Ingress {
}
}
var pathType v1.PathType = v1.PathTypeImplementationSpecific
ing := &v1.Ingress{
ObjectMeta: metav1.ObjectMeta{
Labels: GetDeployLabels(trigger),
@@ -80,7 +83,8 @@ func GetIngressSpec(namespace string, trigger *fv1.HTTPTrigger) *v1.Ingress {
},
},
},
Path: path,
Path: path,
PathType: &pathType,
},
},
},
@@ -100,3 +104,8 @@ func GetDeployLabels(trigger *fv1.HTTPTrigger) map[string]string {
"triggerNamespace": trigger.ObjectMeta.Namespace,
}
}
func IsWebsocketRequest(request *http.Request) bool {
return request.Header.Get("Upgrade") == "websocket" &&
request.Header.Get("Connection") == "Upgrade"
}
+17 -8
View File
@@ -31,6 +31,7 @@ func TestGetIngressSpec(t *testing.T) {
ingressNS string
trigger *fv1.HTTPTrigger
}
var pathType v1.PathType = v1.PathTypeImplementationSpecific
tests := []struct {
name string
args args
@@ -84,7 +85,8 @@ func TestGetIngressSpec(t *testing.T) {
},
},
},
Path: "/foo/bar",
Path: "/foo/bar",
PathType: &pathType,
},
},
},
@@ -145,7 +147,8 @@ func TestGetIngressSpec(t *testing.T) {
},
},
},
Path: "/foo/bar",
Path: "/foo/bar",
PathType: &pathType,
},
},
},
@@ -204,7 +207,8 @@ func TestGetIngressSpec(t *testing.T) {
},
},
},
Path: "/foo/bar",
Path: "/foo/bar",
PathType: &pathType,
},
},
},
@@ -263,7 +267,8 @@ func TestGetIngressSpec(t *testing.T) {
},
},
},
Path: "/foo/{bar}",
Path: "/foo/{bar}",
PathType: &pathType,
},
},
},
@@ -322,7 +327,8 @@ func TestGetIngressSpec(t *testing.T) {
},
},
},
Path: "/foo/{bar}",
Path: "/foo/{bar}",
PathType: &pathType,
},
},
},
@@ -382,7 +388,8 @@ func TestGetIngressSpec(t *testing.T) {
},
},
},
Path: "/foo/bar",
Path: "/foo/bar",
PathType: &pathType,
},
},
},
@@ -441,7 +448,8 @@ func TestGetIngressSpec(t *testing.T) {
},
},
},
Path: "/foo/bar",
Path: "/foo/bar",
PathType: &pathType,
},
},
},
@@ -512,7 +520,8 @@ func TestGetIngressSpec(t *testing.T) {
},
},
},
Path: "/foo/bar",
Path: "/foo/bar",
PathType: &pathType,
},
},
},
+10 -4
View File
@@ -31,9 +31,11 @@ import (
"github.com/pkg/errors"
"go.opencensus.io/plugin/ochttp"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
"golang.org/x/net/context/ctxhttp"
"github.com/fission/fission/pkg/storagesvc"
"github.com/fission/fission/pkg/utils/tracing"
)
type (
@@ -45,11 +47,15 @@ type (
// Client creates a storage service client.
func MakeClient(url string) *Client {
var hc *http.Client
if tracing.TracingEnabled(nil) {
hc = &http.Client{Transport: &ochttp.Transport{}}
} else {
hc = &http.Client{Transport: otelhttp.NewTransport(http.DefaultTransport)}
}
return &Client{
url: strings.TrimSuffix(url, "/") + "/v1",
httpClient: &http.Client{
Transport: &ochttp.Transport{},
},
url: strings.TrimSuffix(url, "/") + "/v1",
httpClient: hc,
}
}
+2 -3
View File
@@ -95,8 +95,7 @@ func startS3StorageService(endpoint, bucketName, subDir string) {
os.Setenv("STORAGE_S3_REGION", minioRegion)
storage := storagesvc.NewS3Storage()
_ = storagesvc.Start(logger, storage, port)
_ = storagesvc.Start(logger, storage, port, true)
}
func TestS3StorageService(t *testing.T) {
@@ -213,7 +212,7 @@ func TestLocalStorageService(t *testing.T) {
localPath := fmt.Sprintf("/tmp/%v", testID)
_ = os.Mkdir(localPath, os.ModePerm)
storage := storagesvc.NewLocalStorage(localPath)
_ = storagesvc.Start(logger, storage, port)
_ = storagesvc.Start(logger, storage, port, true)
time.Sleep(time.Second)
client := MakeClient(fmt.Sprintf("http://localhost:%v/", port))
+13 -8
View File
@@ -29,6 +29,8 @@ import (
"github.com/pkg/errors"
"go.opencensus.io/plugin/ochttp"
"go.uber.org/zap"
"github.com/fission/fission/pkg/utils/otel"
)
type (
@@ -132,7 +134,6 @@ func (ss *StorageService) uploadHandler(w http.ResponseWriter, r *http.Request)
zap.String("filename", handler.Filename),
)
}
}
func (ss *StorageService) getIdFromRequest(r *http.Request) (string, error) {
@@ -199,7 +200,7 @@ func MakeStorageService(logger *zap.Logger, storageClient *StowClient, port int)
}
}
func (ss *StorageService) Start(port int) {
func (ss *StorageService) Start(port int, openTracingEnabled bool) {
r := mux.NewRouter()
r.HandleFunc("/v1/archive", ss.uploadHandler).Methods("POST")
r.HandleFunc("/v1/archive", ss.downloadHandler).Methods("GET")
@@ -208,15 +209,19 @@ func (ss *StorageService) Start(port int) {
address := fmt.Sprintf(":%v", port)
err := http.ListenAndServe(address, &ochttp.Handler{
Handler: r,
})
var err error
if openTracingEnabled {
err = http.ListenAndServe(address, &ochttp.Handler{
Handler: r,
})
} else {
err = http.ListenAndServe(address, otel.GetHandlerWithOTEL(r, "fission-storagesvc", otel.UrlsToIgnore("/healthz")))
}
ss.logger.Fatal("done listening", zap.Error(err))
}
// Start runs storage service
func Start(logger *zap.Logger, storage Storage, port int) error {
func Start(logger *zap.Logger, storage Storage, port int, openTracingEnabled bool) error {
enablePruner := true
// create a storage client
storageClient, err := MakeStowClient(logger, storage)
@@ -226,7 +231,7 @@ func Start(logger *zap.Logger, storage Storage, port int) error {
// create http handlers
storageService := MakeStorageService(logger, storageClient, port)
go storageService.Start(port)
go storageService.Start(port, openTracingEnabled)
// enablePruner prevents storagesvc unit test from needing to talk to kubernetes
if enablePruner {
+3 -16
View File
@@ -1,28 +1,26 @@
package utils
import (
"fmt"
"time"
apiv1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/labels"
"k8s.io/apimachinery/pkg/selection"
k8sInformers "k8s.io/client-go/informers"
"k8s.io/client-go/kubernetes"
k8sCache "k8s.io/client-go/tools/cache"
metricsapi "k8s.io/metrics/pkg/apis/metrics"
v1 "github.com/fission/fission/pkg/apis/core/v1"
)
func GetInformerFactoryByExecutor(client *kubernetes.Clientset, executorType v1.ExecutorType) (k8sInformers.SharedInformerFactory, error) {
func GetInformerFactoryByExecutor(client *kubernetes.Clientset, executorType v1.ExecutorType, defaultResync time.Duration) (k8sInformers.SharedInformerFactory, error) {
executorLabel, err := labels.NewRequirement(v1.EXECUTOR_TYPE, selection.DoubleEquals, []string{string(executorType)})
if err != nil {
return nil, err
}
labelSelector := labels.NewSelector()
labelSelector.Add(*executorLabel)
informerFactory := k8sInformers.NewSharedInformerFactoryWithOptions(client, 0,
informerFactory := k8sInformers.NewSharedInformerFactoryWithOptions(client, defaultResync,
k8sInformers.WithTweakListOptions(func(options *metav1.ListOptions) {
options.LabelSelector = labelSelector.String()
}))
@@ -47,14 +45,3 @@ func SupportedMetricsAPIVersionAvailable(discoveredAPIGroups *metav1.APIGroupLis
}
return false
}
func GetCachedItem(obj apiv1.ObjectReference, informer k8sCache.SharedIndexInformer) (item interface{}, exists bool, err error) {
store := informer.GetStore()
item, exists, err = store.Get(obj)
if err != nil || !exists {
item, exists, err = store.GetByKey(fmt.Sprintf("%s/%s", obj.Namespace, obj.Name))
}
return item, exists, err
}
+38
View File
@@ -0,0 +1,38 @@
/*
Copyright 2021 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package loggerfactory
import (
"os"
"strconv"
"go.uber.org/zap"
"go.uber.org/zap/zapcore"
kzap "sigs.k8s.io/controller-runtime/pkg/log/zap"
)
func GetLogger() *zap.Logger {
encConfOpt := func(o *kzap.Options) {
encTimeFunc := func(encConfig *zapcore.EncoderConfig) {
encConfig.EncodeTime = zapcore.ISO8601TimeEncoder
}
o.EncoderConfigOptions = append(o.EncoderConfigOptions, encTimeFunc)
o.ZapOpts = append(o.ZapOpts, zap.AddCaller())
}
isDebugEnv, _ := strconv.ParseBool(os.Getenv("DEBUG_ENV"))
return kzap.NewRaw(kzap.UseDevMode(isDebugEnv), encConfOpt)
}
+129
View File
@@ -0,0 +1,129 @@
package otel
import (
"context"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/trace"
appsv1 "k8s.io/api/apps/v1"
asv1 "k8s.io/api/autoscaling/v1"
apiv1 "k8s.io/api/core/v1"
fv1 "github.com/fission/fission/pkg/apis/core/v1"
"github.com/fission/fission/pkg/executor/fscache"
)
/* GetAttributesForFunction returns a set of attributes for a function. Attributes returned:
function-name
function-namespace
environment-name
environment-namespace
These attributes are tags that can be used to filter traces.
*/
func GetAttributesForFunction(fn *fv1.Function) []attribute.KeyValue {
if fn == nil {
return []attribute.KeyValue{}
}
attrs := []attribute.KeyValue{
{Key: "function-name", Value: attribute.StringValue(fn.Name)},
{Key: "function-namespace", Value: attribute.StringValue(fn.Namespace)},
}
if fn.Spec.Environment.Name != "" {
attrs = append(attrs,
attribute.KeyValue{Key: "environment-name", Value: attribute.StringValue(fn.Spec.Environment.Name)},
attribute.KeyValue{Key: "environment-namespace", Value: attribute.StringValue(fn.Spec.Environment.Namespace)})
}
return attrs
}
func GetAttributesForEnv(env *fv1.Environment) []attribute.KeyValue {
if env == nil {
return []attribute.KeyValue{}
}
return []attribute.KeyValue{
{Key: "environment-name", Value: attribute.StringValue(env.Name)},
{Key: "environment-namespace", Value: attribute.StringValue(env.Namespace)}}
}
func GetAttributesForPackage(pkg *fv1.Package) []attribute.KeyValue {
if pkg == nil {
return []attribute.KeyValue{}
}
return []attribute.KeyValue{
{Key: "package-name", Value: attribute.StringValue(pkg.Name)},
{Key: "package-namespace", Value: attribute.StringValue(pkg.Namespace)}}
}
func GetAttributesForFuncSvc(fsvc *fscache.FuncSvc) []attribute.KeyValue {
if fsvc == nil {
return []attribute.KeyValue{}
}
var attrs []attribute.KeyValue
if fsvc.Function != nil {
attrs = append(attrs,
attribute.KeyValue{Key: "function-name", Value: attribute.StringValue(fsvc.Function.Name)},
attribute.KeyValue{Key: "function-namespace", Value: attribute.StringValue(fsvc.Function.Namespace)})
}
if fsvc.Environment != nil {
attrs = append(attrs,
attribute.KeyValue{Key: "environment-name", Value: attribute.StringValue(fsvc.Environment.Name)},
attribute.KeyValue{Key: "environment-namespace", Value: attribute.StringValue(fsvc.Environment.Namespace)})
}
if fsvc.Address != "" {
attrs = append(attrs, attribute.KeyValue{Key: "address", Value: attribute.StringValue(fsvc.Address)})
}
return attrs
}
func GetAttributesForPod(pod *apiv1.Pod) []attribute.KeyValue {
if pod == nil {
return []attribute.KeyValue{}
}
return []attribute.KeyValue{
{Key: "pod-name", Value: attribute.StringValue(pod.Name)},
{Key: "pod-namespace", Value: attribute.StringValue(pod.Namespace)},
{Key: "pod-ip", Value: attribute.StringValue(pod.Status.PodIP)},
}
}
func GetAttributesForDeployment(deployment *appsv1.Deployment) []attribute.KeyValue {
if deployment == nil {
return []attribute.KeyValue{}
}
return []attribute.KeyValue{
{Key: "deployment-name", Value: attribute.StringValue(deployment.Name)},
{Key: "deployment-namespace", Value: attribute.StringValue(deployment.Namespace)},
}
}
func GetAttributesForHPA(hpa *asv1.HorizontalPodAutoscaler) []attribute.KeyValue {
if hpa == nil {
return []attribute.KeyValue{}
}
return []attribute.KeyValue{
{Key: "hpa-name", Value: attribute.StringValue(hpa.Name)},
{Key: "hpa-namespace", Value: attribute.StringValue(hpa.Namespace)},
}
}
func GetAttributesForSvc(svc *apiv1.Service) []attribute.KeyValue {
if svc == nil {
return []attribute.KeyValue{}
}
return []attribute.KeyValue{
{Key: "svc-name", Value: attribute.StringValue(svc.Name)},
{Key: "svc-namespace", Value: attribute.StringValue(svc.Namespace)},
}
}
func MapToAttributes(m map[string]string) []attribute.KeyValue {
attrs := make([]attribute.KeyValue, 0, len(m))
for k, v := range m {
attrs = append(attrs, attribute.KeyValue{Key: attribute.Key(k), Value: attribute.StringValue(v)})
}
return attrs
}
func SpanTrackEvent(ctx context.Context, event string, attributes ...attribute.KeyValue) {
trace.SpanFromContext(ctx).AddEvent(event, trace.WithAttributes(attributes...))
}
+31
View File
@@ -0,0 +1,31 @@
package otel
import (
"net/http"
"strings"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
)
func UrlsToIgnore(ignoreEndpoints ...string) func(r *http.Request) bool {
return func(r *http.Request) bool {
for _, ignore := range ignoreEndpoints {
if strings.HasPrefix(r.URL.Path, ignore) {
return false
}
}
return true
}
}
func GetHandlerWithOTEL(h http.Handler, name string, filter ...otelhttp.Filter) http.Handler {
opts := []otelhttp.Option{
otelhttp.WithMessageEvents(otelhttp.ReadEvents, otelhttp.WriteEvents),
}
for _, f := range filter {
opts = append(opts, otelhttp.WithFilter(f))
}
return otelhttp.NewHandler(h, name, opts...)
}
+30
View File
@@ -0,0 +1,30 @@
/*
Copyright 2021 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package otel
import (
"context"
"go.opentelemetry.io/otel/trace"
"go.uber.org/zap"
)
func LoggerWithTraceID(context context.Context, logger *zap.Logger) *zap.Logger {
if span := trace.SpanContextFromContext(context); span.TraceID().IsValid() {
return logger.With(zap.String("trace_id", span.TraceID().String()))
}
return logger
}

Some files were not shown because too many files have changed in this diff Show More