Compare commits

...
58 Commits
Author SHA1 Message Date
Soam VasaniandTa-Ching Chen 2699aa6069 Handle duplicate archive and package specs; handle multifile archives better (#1018)
Archives with multiple files should have all files in the inputs.
Also clean up some inconsistent variable naming -- archive name,
archive file path, and archive input files were all stored in
incorrectly-named variables.
2018-12-11 16:35:31 +08:00
Ta-Ching ChenandGitHub f0c0936b87 Validate command flag input by adding cli hook (#1017) 2018-12-10 23:48:33 +08:00
Ta-Ching ChenandVishal 314e16a88e Pre-create kubernetes resources for function with minScale=0 (#976)
Pre-create Kubernetes resources if functions has MinScale=0 and then scale when first function is called.
2018-12-03 15:52:07 +05:30
Ta-Ching ChenandGitHub 72faa927de Use executor type as a delimiter to prevent deploy name conflict (#1009) 2018-11-30 13:04:44 +08:00
VishalandTa-Ching Chen 19b52938c3 Improve archive package user experience (#927) 2018-11-29 23:00:38 +08:00
Ta-Ching ChenandGitHub 7cedf8d580 Fix issues when specifying resources/scales during updating/creation process (#970) 2018-11-29 20:46:48 +08:00
Ta-Ching ChenandGitHub 17cbc5baad Upgrade environment dependencies for security alert (#1006) 2018-11-29 16:38:38 +08:00
Ta-Ching ChenandGitHub c2bcbb06a9 Rename canary flag name from funcN/funcN-1 to newfunction/oldfunction (#1003) 2018-11-28 16:27:29 +08:00
VishalandTa-Ching Chen 152c899e8b Added warning when package env and environment env mismatched (#996)
Fix issue #946
2018-11-26 19:05:00 +08:00
Ta-Ching ChenandGitHub c476f54a6c Fix MQ trigger (NATS) wrongly sends error message to response topic (#1002) 2018-11-25 10:07:46 +08:00
Ta-Ching ChenandGitHub 4b5ab0ad93 Shorten executor kubernetes objects name (#975) 2018-11-24 20:41:38 +08:00
smruthi2187andTa-Ching Chen 82af0e554a Fix Read on Closed body error (#963) 2018-11-21 14:27:52 +08:00
Harald NordgrenandTa-Ching Chen b456dec138 Update formatting directive logic to unbreak tests 2018-11-20 17:24:41 +08:00
Ta-Ching Chen 4a8c200c37 Fix helm command not found issue when kubeconfig is not provided. 2018-11-20 17:24:37 +08:00
VishalandTa-Ching Chen 52b736c892 Package info error should warn user (#995) 2018-11-20 00:44:20 +08:00
Ta-Ching ChenandVishal d9eee058ce Newdeploy - change only in minscale should not restart deployment 2018-11-15 15:31:28 +05:30
Soam VasaniandVishal 3676602fa2 Specs for JVM example (#825) 2018-11-15 00:00:26 +05:30
Ta-Ching ChenandGitHub 4f3b7de04a Fix release script failed to generate yaml for nonhelm installation (#978) 2018-11-14 15:02:43 +08:00
VishalandTa-Ching Chen cdf5838420 Upgrade OpenJDK8 to latest available version (#985) 2018-11-13 18:00:38 +08:00
Soam VasaniandTa-Ching Chen a714983c28 Fix the analytics jobs in the YAMLs (remove duplicates) (#977) 2018-11-08 14:39:37 +08:00
Steven DakeandSoam Vasani b9547160b5 Properly render Helm charts (#969)
* Properly render Helm charts

Fixes: https://github.com/fission/fission/issues/968

* Addres reviewier comments
2018-11-06 13:27:17 -08:00
Ta-Ching ChenandGitHub bfa8e6a45f Show builder image when list all envs (#971) 2018-11-05 23:18:37 +08:00
Ta-Ching ChenandGitHub 9764eff05d Fix CLI not shows package name when creating a function (#966) 2018-11-05 17:19:05 +08:00
Ganesh GuttikondaandTa-Ching Chen 253e44fdbe Updating the compile documentation link (#965)
* Updating the compile documentation link
* Changing the protocol to https
2018-11-05 15:41:22 +08:00
smruthi2187andGitHub 585d86448f V0.12.0 (#967) 2018-11-01 15:57:25 -07:00
smruthi2187andGitHub dad7b6632e Keep prometheus and canary deploy set to false in fission-core (#964) 2018-11-01 13:02:46 -07:00
Soam VasaniandGitHub 4e9e025ca4 Remove version from release name since it contains illegal chars for names (#939)
* Remove version from release name since it contains illegal chars for names

* Put version in release name for uniqueness, but remove invalid chars

* Remove prometheus=false yaml
2018-10-31 14:54:25 -07:00
Ta-Ching ChenandGitHub b6ea85e353 Fix failed to pull influxdb image from dockerhub (#957)
* Fix failed to pull influxdb image from dockerhub
* Disable jvm test temporarily (issue 958)
2018-10-31 16:32:45 +08:00
Ta-Ching ChenandGitHub 29aabaabda Optimize function latency when cache expired/invalid under high concurrency (#856)
* Optimize router response time by adding update lock

In 0.9.2, the router sends multiple GetServiceForFunction requests to executor
to get the service URL. However, the response time of executor will increase
under high-concurrency situation due to too many requests are waiting for
processing.

To solve the problem, an update lock was added to the router. All of goroutines belongs
to the same function need to grab the update lock before sending the request.
Only the first goroutine which gets the update lock is allowed to send request.
In this way, we reduce the burden of executor and lower the failure rate.
2018-10-27 23:35:00 +08:00
David McGillicuddyandSoam Vasani 394c5b13f4 Update readme to include JVM (#953) 2018-10-24 09:03:27 -07:00
smruthi2187andGitHub 0a8c6e97a6 Feature flag to enable/disable canary + optional prometheus install (#937) 2018-10-22 15:24:43 -07:00
VishalandGitHub e7f1d4564a Kafka tests (#944)
A simple test for Kafka integration which runs locally provided cluster has Kafka and Fission install has Kafka integratiom enabled (mqtrigger-kafka deployment present).
2018-10-22 20:07:51 +05:30
Ta-Ching ChenandGitHub 619b390af3 Support for full url (base on aalubin changes) (#941) 2018-10-21 04:04:07 +08:00
smruthi2187andGitHub 1488d0ca2d fix a few canary deployment issues (#943) 2018-10-19 14:56:40 -07:00
smruthi2187andGitHub dd35750482 Return the error on failed specializations with fn test --debug (#917) 2018-10-19 11:17:58 -07:00
Ta-Ching ChenandGitHub 5aeba59b75 Bump flask version (#942) 2018-10-18 22:56:04 +08:00
Soam VasaniandGitHub 8329ef36b3 demo script updates (#934)
Demo script updates.  Record-replay, a bugfix in the canary demo script, etc.
2018-10-15 14:48:01 -07:00
VishalandGitHub 4cc42fb216 Adding JVM heap options to environment (#936) 2018-10-15 23:01:43 +05:30
Ta-Ching ChenandGitHub d23b210f4e Add X-Forwarded-Host to request header (#890) 2018-10-15 14:17:18 +08:00
VishalandGitHub 803dbd6d39 Added build and push procedures for Nodejs builder environment (#916) 2018-10-03 08:07:23 +05:30
Ta-Ching ChenandGitHub e4213f6eb6 Fix flag not found problem when running canary demo scripts (#914) 2018-10-02 22:07:03 +08:00
VishalandGitHub 2d2576cf2d V0.11.0 (#913)
Release v0.11.0
2018-10-02 00:08:29 +05:30
VishalandGitHub 96cbee185d Kafka integration (#831)
Kafka integration with Fission enables invoking a function when a message arrives in a Kafka topic
2018-10-01 18:03:31 +05:30
VishalandGitHub 9f83d70b63 Fixes #758, uses v2 specialize for env versions 2 or higher (#911) 2018-10-01 10:07:49 +05:30
VishalandGitHub 7deb45bed2 Making the Maven build quite for Java env test, right now it is too verbose for CI (#902) 2018-10-01 08:00:45 +05:30
smruthi2187andVishal 12d3a04bf9 Print status with the get option. (#907) 2018-09-28 22:19:24 +05:30
smruthi2187andGitHub fa565b75ae Canary deployments for fission functions. (#892) 2018-09-25 18:26:26 -07:00
Michael GaidaandVishal 437d4dc04d Configurable namespace creation (#855)
Configurable namespace creation defaulting to true (backward compatible)
2018-09-25 13:49:08 +05:30
VishalandGitHub 188138ccd8 Fixed the spec validation UX issue (#898)
Improved some of warnings/messages shown to the user when using `fission spec` command. The `spec validate` command now shows appropriate warnings instead
2018-09-21 14:52:49 +05:30
Ta-Ching ChenandGitHub 6b49c194ca Fission supportability: Add dump command to dump information for debugging (#754)
* Add v1 support tool proposal
* Fission supportability: Add dump command to dump information for debugging
2018-09-19 14:35:51 +08:00
Ta-Ching ChenandGitHub a86c9431d3 Check CRD creation error instead of doing return directly (#897) 2018-09-18 16:30:56 +08:00
Gary YeapandTa-Ching Chen 6f60f3eef8 Add v2 interface support for nodes env (#836) 2018-09-14 14:28:04 +08:00
VishalandGitHub d18777bed4 Used hidden flag instead of removing command completely (#886)
The `fission spec helm` is now a hidden command since it is not implemented yet.
2018-09-13 15:27:22 +05:30
Ta-Ching ChenandGitHub 82d23b9bf8 Fix fetcher not close file descriptor correctly (#889)
* Fix fetcher not close file descriptor correctly
* Call Sync() before chmod
2018-09-11 14:15:56 +08:00
Ta-Ching ChenandGitHub a4f58fbf10 Check pod container ready state (#861)
Ensure pod is ready by checking pod’s containers state and deletionTimestamp
2018-09-07 22:17:07 +08:00
Adam BouhenguelandSoam Vasani 2dbae32f20 FIX CleanupOldExecutorObjects in all namespaces (#879)
Don't just look in functionNamespace anymore
2018-08-29 15:12:31 -07:00
Ta-Ching ChenandGitHub 21b0a1d852 Fix failed to find release-builder dockerfile & push specific tag (#870) 2018-08-26 23:02:07 +08:00
Ta-Ching ChenandGitHub f8535be7da V0.10.0 (#868) 2018-08-19 19:05:19 +08:00
199 changed files with 10635 additions and 1498 deletions
+2 -1
View File
@@ -9,7 +9,7 @@ branches:
language: go
go:
- 1.10.2
- 1.11.x
cache:
directories:
@@ -22,6 +22,7 @@ services:
before_install:
- sudo apt-get update
- sudo apt-get -y -o Dpkg::Options::="--force-confnew" install docker-ce
- sudo apt-get -y install apache2-utils
- sudo sysctl net.ipv6.conf.all.disable_ipv6=0
install:
+110 -31
View File
@@ -1,13 +1,102 @@
# 0.9.2
[Documentation](https://docs.fission.io/0.9.2/)
## Downloads for 0.9.2
# 0.12.0
[Documentation](https://docs.fission.io/0.12.0/)
## Downloads for 0.12.0
filename | sha256 hash
-------- | -----------
[fission-all-0.12.0.yaml](https://github.com/fission/fission/releases/download/0.12.0/fission-all-0.12.0.yaml) | `1cee07f922849fb41830e2bf32d02f3999ef8c60b49fce8c244c870274c732e3`
[fission-core-0.12.0.yaml](https://github.com/fission/fission/releases/download/0.12.0/fission-core-0.12.0.yaml) | `c8d41e62b939f39a69280360015a71d9bab0301c50ea6c3755f16bb5dbc7aa1d`
[fission-core-0.12.0-minikube.yaml](https://github.com/fission/fission/releases/download/0.12.0/fission-core-0.12.0-minikube.yaml) | `54b64b6363ba267d768bff4f68379d835467efd99ceab6410ff6900839b6185f`
[fission-all-0.12.0-minikube.yaml](https://github.com/fission/fission/releases/download/0.12.0/fission-all-0.12.0-minikube.yaml) | `0044d5d2ecbebae2c8252d96949de9bcd348c8de76d013fbdf7fee5c9fe757af`
[fission-core-0.12.0.tgz](https://github.com/fission/fission/releases/download/0.12.0/fission-core-0.12.0.tgz) | `c93949fe0428951ffe5b2fef7133f48d23b82d7b045129bb57ee2de2bd9d9257`
[fission-all-0.12.0.tgz](https://github.com/fission/fission/releases/download/0.12.0/fission-all-0.12.0.tgz) | `7b9b739abde387bd96689e709ed25ebae879e30575036b0e4fb54544b1c9f10b`
[fission-cli-osx](https://github.com/fission/fission/releases/download/0.12.0/fission-cli-osx) | `54f3b0bf64d57b1996d3871415f540506c5aa9f65227d27be086dfe568a945ce`
[fission-cli-linux](https://github.com/fission/fission/releases/download/0.12.0/fission-cli-linux) | `c04e077c79b9211f116bb16341028f0bae2d6a8d89ddb59bb3ce1b4a2ad57364`
[fission-cli-windows.exe](https://github.com/fission/fission/releases/download/0.12.0/fission-cli-windows.exe) | `987c62899fb5c26599b14bf08d713d036c18f12e74abb903665e52c41a14f9a6`
# Change Log
## [0.12.0](https://github.com/fission/fission/tree/0.12.0) (2018-11-01)
[Full Changelog](https://github.com/fission/fission/compare/0.11.0...0.12.0)
**Merged pull requests:**
- Keep prometheus and canary deploy set to false in fission-core [\#964](https://github.com/fission/fission/pull/964) ([smruthi2187](https://github.com/smruthi2187))
- Update readme to include JVM [\#953](https://github.com/fission/fission/pull/953) ([david-mcgillicuddy-ovo](https://github.com/david-mcgillicuddy-ovo))
- Bump flask version [\#942](https://github.com/fission/fission/pull/942) ([life1347](https://github.com/life1347))
- Adding JVM heap options to environment [\#936](https://github.com/fission/fission/pull/936) ([vishal-biyani](https://github.com/vishal-biyani))
- Demo script updates [\#934](https://github.com/fission/fission/pull/934) ([soamvasani](https://github.com/soamvasani))
- Fix flag not found problem when running canary demo scripts [\#914](https://github.com/fission/fission/pull/914) ([life1347](https://github.com/life1347))
- V0.11.0 [\#913](https://github.com/fission/fission/pull/913) ([vishal-biyani](https://github.com/vishal-biyani))
- Fix failed to pull influxdb image from dockerhub [\#957](https://github.com/fission/fission/pull/957) ([life1347](https://github.com/life1347))
- Kafka tests [\#944](https://github.com/fission/fission/pull/944) ([vishal-biyani](https://github.com/vishal-biyani))
- fix a few canary deployment issues [\#943](https://github.com/fission/fission/pull/943) ([smruthi2187](https://github.com/smruthi2187))
- Support for full url \(base on aalubin 882 changes\) [\#941](https://github.com/fission/fission/pull/941) ([life1347](https://github.com/life1347))
- Remove version from release name since it contains illegal chars for names [\#939](https://github.com/fission/fission/pull/939) ([soamvasani](https://github.com/soamvasani))
- Feature flag to enable/disable canary + optional prometheus install [\#937](https://github.com/fission/fission/pull/937) ([smruthi2187](https://github.com/smruthi2187))
- Return the error on failed specializations with `fn test --debug` [\#917](https://github.com/fission/fission/pull/917) ([smruthi2187](https://github.com/smruthi2187))
- Added build and push procedures for Nodejs builder environment [\#916](https://github.com/fission/fission/pull/916) ([vishal-biyani](https://github.com/vishal-biyani))
- Add X-Forwarded-Host to request header [\#890](https://github.com/fission/fission/pull/890) ([life1347](https://github.com/life1347))
- Optimize function latency when cache expired/invalid under high concurrency [\#856](https://github.com/fission/fission/pull/856) ([life1347](https://github.com/life1347))
## [0.11.0](https://github.com/fission/fission/tree/0.11.0) (2018-10-01)
[Full Changelog](https://github.com/fission/fission/compare/0.10.0...0.11.0)
**Merged pull requests:**
- Print status with the get option. [\#907](https://github.com/fission/fission/pull/907) ([smruthi2187](https://github.com/smruthi2187))
- Fixed the spec validation UX issue [\#898](https://github.com/fission/fission/pull/898) ([vishal-biyani](https://github.com/vishal-biyani))
- Check CRD creation error instead of doing return directly [\#897](https://github.com/fission/fission/pull/897) ([life1347](https://github.com/life1347))
- Fix failed to find release-builder dockerfile & push specific tag [\#870](https://github.com/fission/fission/pull/870) ([life1347](https://github.com/life1347))
- V0.10.0 [\#868](https://github.com/fission/fission/pull/868) ([life1347](https://github.com/life1347))
- Fixes \#758, uses v2 specialize for env versions 2 or higher [\#911](https://github.com/fission/fission/pull/911) ([vishal-biyani](https://github.com/vishal-biyani))
- Java env test - Maven verbosity reduction [\#902](https://github.com/fission/fission/pull/902) ([vishal-biyani](https://github.com/vishal-biyani))
- Canary deployments for fission functions. [\#892](https://github.com/fission/fission/pull/892) ([smruthi2187](https://github.com/smruthi2187))
- Fix fetcher not close file descriptor correctly [\#889](https://github.com/fission/fission/pull/889) ([life1347](https://github.com/life1347))
- Removes the spec helm command for now to fix \#881 [\#886](https://github.com/fission/fission/pull/886) ([vishal-biyani](https://github.com/vishal-biyani))
- FIX CleanupOldExecutorObjects in all namespaces [\#879](https://github.com/fission/fission/pull/879) ([ajbouh](https://github.com/ajbouh))
- Check pod container ready state [\#861](https://github.com/fission/fission/pull/861) ([life1347](https://github.com/life1347))
- Configurable namespace creation [\#855](https://github.com/fission/fission/pull/855) ([michaelgaida](https://github.com/michaelgaida))
- Add v2 interface support for nodes env [\#836](https://github.com/fission/fission/pull/836) ([garyyeap](https://github.com/garyyeap))
- Kafka integration [\#831](https://github.com/fission/fission/pull/831) ([vishal-biyani](https://github.com/vishal-biyani))
- Fission supportability: Add dump command to dump information for debugging [\#754](https://github.com/fission/fission/pull/754) ([life1347](https://github.com/life1347))
## [0.10.0](https://github.com/fission/fission/tree/0.10.0) (2018-08-17)
[Full Changelog](https://github.com/fission/fission/compare/0.9.2...0.10.0)
**Merged pull requests:**
- Fix CLI failed to setup port-forward caused by \#712 [\#867](https://github.com/fission/fission/pull/867) ([life1347](https://github.com/life1347))
- Replay recorded requests by ReqUID [\#864](https://github.com/fission/fission/pull/864) ([Amusement](https://github.com/Amusement))
- Add cleanup function to test scripts [\#863](https://github.com/fission/fission/pull/863) ([life1347](https://github.com/life1347))
- Fix newdeploy fail to update HPA, deployment of a function after function update [\#862](https://github.com/fission/fission/pull/862) ([life1347](https://github.com/life1347))
- Fix router not taps function services [\#860](https://github.com/fission/fission/pull/860) ([life1347](https://github.com/life1347))
- Do resources validation when validate spec files [\#840](https://github.com/fission/fission/pull/840) ([life1347](https://github.com/life1347))
- Fixed the name of JVM builder image name [\#824](https://github.com/fission/fission/pull/824) ([vishal-biyani](https://github.com/vishal-biyani))
- V0.9.2 [\#817](https://github.com/fission/fission/pull/817) ([vishal-biyani](https://github.com/vishal-biyani))
- Add retry subcommand to pkg command [\#808](https://github.com/fission/fission/pull/808) ([life1347](https://github.com/life1347))
- add gevent based Python server to benchmark test cases [\#794](https://github.com/fission/fission/pull/794) ([xiekeyang](https://github.com/xiekeyang))
- Add more meaningful error messages to executor when getServiceForFunction [\#752](https://github.com/fission/fission/pull/752) ([life1347](https://github.com/life1347))
- Fix for \#662: avoid unnecessary builds [\#866](https://github.com/fission/fission/pull/866) ([smruthi2187](https://github.com/smruthi2187))
- Fix newdeploy not updates deployment after function's entrypoint changed [\#838](https://github.com/fission/fission/pull/838) ([life1347](https://github.com/life1347))
- Fix spec failed to archive single directory [\#837](https://github.com/fission/fission/pull/837) ([life1347](https://github.com/life1347))
- Uses a real go project to showcase vendor example so glide works [\#828](https://github.com/fission/fission/pull/828) ([vishal-biyani](https://github.com/vishal-biyani))
- Recorder CRD, Records API, Redis deployment [\#818](https://github.com/fission/fission/pull/818) ([Amusement](https://github.com/Amusement))
- Fix router panic when trying to update route [\#811](https://github.com/fission/fission/pull/811) ([life1347](https://github.com/life1347))
- Add query options to `fission function test` [\#782](https://github.com/fission/fission/pull/782) ([erwinvaneyk](https://github.com/erwinvaneyk))
- Add go environment vendor directory support [\#781](https://github.com/fission/fission/pull/781) ([life1347](https://github.com/life1347))
- Scale deployment to zero when function is in idle state [\#775](https://github.com/fission/fission/pull/775) ([life1347](https://github.com/life1347))
- Update binary environment readme [\#773](https://github.com/fission/fission/pull/773) ([erwinvaneyk](https://github.com/erwinvaneyk))
- Added readme for JVM environment [\#768](https://github.com/fission/fission/pull/768) ([vishal-biyani](https://github.com/vishal-biyani))
- Fix spec command overrides existing archive's url of a package [\#764](https://github.com/fission/fission/pull/764) ([life1347](https://github.com/life1347))
- Fixed typos from from goreportcard [\#760](https://github.com/fission/fission/pull/760) ([vishal-biyani](https://github.com/vishal-biyani))
- Extensible Fission CLI [\#743](https://github.com/fission/fission/pull/743) ([erwinvaneyk](https://github.com/erwinvaneyk))
- Updating releasing notes with details and structure [\#738](https://github.com/fission/fission/pull/738) ([vishal-biyani](https://github.com/vishal-biyani))
- Update route without providing function reference [\#718](https://github.com/fission/fission/pull/718) ([vishal-biyani](https://github.com/vishal-biyani))
- Allow router round-trip to be configurable [\#713](https://github.com/fission/fission/pull/713) ([xiekeyang](https://github.com/xiekeyang))
- Fix CLI failed to set up port-forwarding when multiple controller pods exist in the same namespace [\#712](https://github.com/fission/fission/pull/712) ([life1347](https://github.com/life1347))
## [0.9.2](https://github.com/fission/fission/tree/0.9.2) (2018-07-25)
[Full Changelog](https://github.com/fission/fission/compare/0.9.1...0.9.2)
@@ -21,13 +110,6 @@ filename | sha256 hash
- Add changelog. [\#789](https://github.com/fission/fission/pull/789) ([smruthi2187](https://github.com/smruthi2187))
- changes needed for 0.9.1 [\#788](https://github.com/fission/fission/pull/788) ([smruthi2187](https://github.com/smruthi2187))
- Working version of Java builder with Maven [\#783](https://github.com/fission/fission/pull/783) ([vishal-biyani](https://github.com/vishal-biyani))
- Add query options to `fission function test` [\#782](https://github.com/fission/fission/pull/782) ([erwinvaneyk](https://github.com/erwinvaneyk))
- Add go environment vendor directory support [\#781](https://github.com/fission/fission/pull/781) ([life1347](https://github.com/life1347))
- Added readme for JVM environment [\#768](https://github.com/fission/fission/pull/768) ([vishal-biyani](https://github.com/vishal-biyani))
- Fix spec command overrides existing archive's url of a package [\#764](https://github.com/fission/fission/pull/764) ([life1347](https://github.com/life1347))
- Fixed typos from from goreportcard [\#760](https://github.com/fission/fission/pull/760) ([vishal-biyani](https://github.com/vishal-biyani))
- Updating releasing notes with details and structure [\#738](https://github.com/fission/fission/pull/738) ([vishal-biyani](https://github.com/vishal-biyani))
- Allow router round-trip to be configurable [\#713](https://github.com/fission/fission/pull/713) ([xiekeyang](https://github.com/xiekeyang))
## [0.9.1](https://github.com/fission/fission/tree/0.9.1) (2018-07-07)
[Full Changelog](https://github.com/fission/fission/compare/0.9.0...0.9.1)
@@ -152,7 +234,7 @@ filename | sha256 hash
- Show fission deployment version with cli [\#538](https://github.com/fission/fission/pull/538) ([life1347](https://github.com/life1347))
## [0.6.1](https://github.com/fission/fission/tree/0.6.1) (2018-03-22)
[Full Changelog](https://github.com/fission/fission/compare/0.6.0...0.6.1)
[Full Changelog](https://github.com/fission/fission/compare/latest...0.6.1)
**Merged pull requests:**
@@ -168,27 +250,11 @@ filename | sha256 hash
- Fix executor tries to create a new deployment when a function is updated [\#524](https://github.com/fission/fission/pull/524) ([life1347](https://github.com/life1347))
- Add container spec config options to \(build\) environments [\#413](https://github.com/fission/fission/pull/413) ([erwinvaneyk](https://github.com/erwinvaneyk))
## [0.6.0](https://github.com/fission/fission/tree/0.6.0) (2018-03-01)
[Full Changelog](https://github.com/fission/fission/compare/latest...0.6.0)
**Merged pull requests:**
- Detect fission namespace in cli [\#519](https://github.com/fission/fission/pull/519) ([soamvasani](https://github.com/soamvasani))
- Default values for FISSION\_\* env vars [\#518](https://github.com/fission/fission/pull/518) ([soamvasani](https://github.com/soamvasani))
- Add chart version to job name [\#516](https://github.com/fission/fission/pull/516) ([soamvasani](https://github.com/soamvasani))
- Fix CLI not update function's secret/configmap correctly [\#512](https://github.com/fission/fission/pull/512) ([life1347](https://github.com/life1347))
- Adds latest tags and pushes to dockerhub for fetcher and fission-bundle [\#509](https://github.com/fission/fission/pull/509) ([vishal-biyani](https://github.com/vishal-biyani))
- Fixes the backward compatibility with older environment versions [\#508](https://github.com/fission/fission/pull/508) ([vishal-biyani](https://github.com/vishal-biyani))
- Update Fn: Executor New Deployment [\#504](https://github.com/fission/fission/pull/504) ([vishal-biyani](https://github.com/vishal-biyani))
- Adds default resources for fetcher pod [\#500](https://github.com/fission/fission/pull/500) ([vishal-biyani](https://github.com/vishal-biyani))
- Documentation Revamp [\#496](https://github.com/fission/fission/pull/496) ([vishal-biyani](https://github.com/vishal-biyani))
- Delete and list orphan pkgs [\#468](https://github.com/fission/fission/pull/468) ([smruthi2187](https://github.com/smruthi2187))
- Service type ClusterIP - Controller port forward through CLI [\#431](https://github.com/fission/fission/pull/431) ([prithviramesh](https://github.com/prithviramesh))
- Istio integration [\#421](https://github.com/fission/fission/pull/421) ([life1347](https://github.com/life1347))
- Implement support for Azure storage message queue triggers [\#371](https://github.com/fission/fission/pull/371) ([peterhuene](https://github.com/peterhuene))
## [latest](https://github.com/fission/fission/tree/latest) (2018-03-01)
[Full Changelog](https://github.com/fission/fission/compare/0.5.0...latest)
[Full Changelog](https://github.com/fission/fission/compare/0.6.0...latest)
## [0.6.0](https://github.com/fission/fission/tree/0.6.0) (2018-03-01)
[Full Changelog](https://github.com/fission/fission/compare/0.5.0...0.6.0)
**Merged pull requests:**
@@ -203,6 +269,19 @@ filename | sha256 hash
- CI modifications [\#491](https://github.com/fission/fission/pull/491) ([smruthi2187](https://github.com/smruthi2187))
- Add upgrade guide from 0.4.x to 0.5.0 [\#490](https://github.com/fission/fission/pull/490) ([life1347](https://github.com/life1347))
- Version -\> 0.5.0 [\#489](https://github.com/fission/fission/pull/489) ([life1347](https://github.com/life1347))
- Detect fission namespace in cli [\#519](https://github.com/fission/fission/pull/519) ([soamvasani](https://github.com/soamvasani))
- Default values for FISSION\_\* env vars [\#518](https://github.com/fission/fission/pull/518) ([soamvasani](https://github.com/soamvasani))
- Add chart version to job name [\#516](https://github.com/fission/fission/pull/516) ([soamvasani](https://github.com/soamvasani))
- Fix CLI not update function's secret/configmap correctly [\#512](https://github.com/fission/fission/pull/512) ([life1347](https://github.com/life1347))
- Adds latest tags and pushes to dockerhub for fetcher and fission-bundle [\#509](https://github.com/fission/fission/pull/509) ([vishal-biyani](https://github.com/vishal-biyani))
- Fixes the backward compatibility with older environment versions [\#508](https://github.com/fission/fission/pull/508) ([vishal-biyani](https://github.com/vishal-biyani))
- Update Fn: Executor New Deployment [\#504](https://github.com/fission/fission/pull/504) ([vishal-biyani](https://github.com/vishal-biyani))
- Adds default resources for fetcher pod [\#500](https://github.com/fission/fission/pull/500) ([vishal-biyani](https://github.com/vishal-biyani))
- Documentation Revamp [\#496](https://github.com/fission/fission/pull/496) ([vishal-biyani](https://github.com/vishal-biyani))
- Delete and list orphan pkgs [\#468](https://github.com/fission/fission/pull/468) ([smruthi2187](https://github.com/smruthi2187))
- Service type ClusterIP - Controller port forward through CLI [\#431](https://github.com/fission/fission/pull/431) ([prithviramesh](https://github.com/prithviramesh))
- Istio integration [\#421](https://github.com/fission/fission/pull/421) ([life1347](https://github.com/life1347))
- Implement support for Azure storage message queue triggers [\#371](https://github.com/fission/fission/pull/371) ([peterhuene](https://github.com/peterhuene))
## [0.5.0](https://github.com/fission/fission/tree/0.5.0) (2018-02-07)
[Full Changelog](https://github.com/fission/fission/compare/0.4.1...0.5.0)
+62
View File
@@ -0,0 +1,62 @@
# Fission Support Tool
Fission now has rich functionality supported by multiple services, however, it brings the complexity of troubleshooting.
This proposal tends to give a picture of fission support tool that can help both user and developer to locate the problem in short time.
To achieve this, the support tool will dump related kubernetes objects, fission resources and pod logs from the given cluster.
# Functionality
## Environment Information Collection
Before troubleshooting, some of the basic information is needed to give others an overview of kubernetes/fission user test with so that we can locate the problem in short time.
* Fission version
* Client/Server version
* Kubernetes cluster version
* Cluster version (i.e v1.9.7-gke.0)
* Running environment (i.e GKE, AKS and minikube)
* Nodes version and other information
## Service Logs Collection
The component logs and the logs of interaction between components are important for people to understand what really happened in cluster. Following are components need to collect logs from.
* All fission component pods
* Function pods
* Builder pods
* Environment pods
## Object dumping
Fission is deeply coupled with Kubernetes, most of the objects are created and maintained by it. There is two major type of objects need to be dumped from kubernetes:
* K8S objects
* CRD resources
All objects should be dumped into a readable file format. It will be great if people can reproduce similar environment with these files.
## Information upload
Upload dump files to the specific backend server for support channel to analysis
# CLI Interface
```
$ fission support collect
NAME:
fission support collect - Collect pod logs, fission resources and related kubernetes objects for troubleshooting
USAGE:
fission support collect [command options] [arguments...]
OPTIONS:
--dumpdir value Directory to save dump kubernetes objects and fission resources (default: "fission-dump")
--fissionns value Namespace of fission installation (default: "fission")
--builderns value Namespace of fission package builder (default: "fission-builder")
--funcns value Namespace of fission function pod (default: "fission-function")
```
# Thoughts?
1. What to do with sensitive objects like secrets and configmap? Ignore the dump for such objects?
2. The functionality is necessary but not listed above?
+3 -2
View File
@@ -53,6 +53,7 @@ The following environments are currently available:
| ------------------------------------ | ------------------------- |
| Binary (for executables or scripts) | `fission/binary-env` |
| Go | `fission/go-env` |
| JVM | `fission/jvm-env` |
| .NET | `fission/dotnet-env` |
| .NET 2.0 | `fission/dotnet20-env` |
| NodeJS (Alpine) | `fission/node-env` |
@@ -97,12 +98,12 @@ See the [examples](examples) directory for more.
Running Fission on your Cluster
===============================
See the [installation guide](http://docs.fission.io/latest/installation).
See the [installation guide](https://docs.fission.io/latest/installation).
Compiling Fission
=================
See the [compilation guide](http://docs.fission.io/latest/contributing/compiling/).
See the [compilation guide](https://docs.fission.io/latest/contributing/).
Status
======
+78
View File
@@ -0,0 +1,78 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package canaryconfigmgr
import (
"context"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission/cache"
"time"
)
type (
canaryConfigCancelFuncMap struct {
cache *cache.Cache // map[metadataKey]*context.Context
}
// metav1.ObjectMeta is not hashable, so we make a hashable copy
// of the subset of its fields that are identifiable.
metadataKey struct {
Name string
Namespace string
}
CanaryProcessingInfo struct {
CancelFunc *context.CancelFunc
Ticker *time.Ticker
}
)
func makecanaryConfigCancelFuncMap() *canaryConfigCancelFuncMap {
return &canaryConfigCancelFuncMap{
cache: cache.MakeCache(0, 0),
}
}
func keyFromMetadata(m *metav1.ObjectMeta) metadataKey {
return metadataKey{
Name: m.Name,
Namespace: m.Namespace,
}
}
func (cancelFuncMap *canaryConfigCancelFuncMap) lookup(f *metav1.ObjectMeta) (*CanaryProcessingInfo, error) {
mk := keyFromMetadata(f)
item, err := cancelFuncMap.cache.Get(mk)
if err != nil {
return nil, err
}
value := item.(*CanaryProcessingInfo)
return value, nil
}
func (cancelFuncMap *canaryConfigCancelFuncMap) assign(f *metav1.ObjectMeta, value *CanaryProcessingInfo) error {
mk := keyFromMetadata(f)
err, _ := cancelFuncMap.cache.Set(mk, value)
return err
}
func (cancelFuncMap *canaryConfigCancelFuncMap) remove(f *metav1.ObjectMeta) error {
mk := keyFromMetadata(f)
return cancelFuncMap.cache.Delete(mk)
}
+387
View File
@@ -0,0 +1,387 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package canaryconfigmgr
import (
"context"
"fmt"
"os"
"strings"
"time"
log "github.com/sirupsen/logrus"
k8serrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/fields"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
k8sCache "k8s.io/client-go/tools/cache"
"github.com/fission/fission"
"github.com/fission/fission/crd"
)
type canaryConfigMgr struct {
fissionClient *crd.FissionClient
kubeClient *kubernetes.Clientset
canaryConfigStore k8sCache.Store
canaryConfigController k8sCache.Controller
promClient *PrometheusApiClient
crdClient *rest.RESTClient
canaryCfgCancelFuncMap *canaryConfigCancelFuncMap
}
func MakeCanaryConfigMgr(fissionClient *crd.FissionClient, kubeClient *kubernetes.Clientset, crdClient *rest.RESTClient, prometheusSvc string) (*canaryConfigMgr, error) {
if prometheusSvc == "" {
// handle a case where there is a prometheus server is already installed, try to find the service from env variable
envVars := os.Environ()
for _, envVar := range envVars {
if strings.Contains(envVar, "PROMETHEUS_SERVER_SERVICE_HOST") {
envVarSplit := strings.Split(envVar, "=")
prometheusSvc = envVarSplit[1]
break
}
}
if prometheusSvc == "" {
return nil, fmt.Errorf("prometheus service not found, cant create canary config manager")
}
}
configMgr := &canaryConfigMgr{
fissionClient: fissionClient,
kubeClient: kubeClient,
crdClient: crdClient,
promClient: MakePrometheusClient(prometheusSvc),
canaryCfgCancelFuncMap: makecanaryConfigCancelFuncMap(),
}
store, controller := configMgr.initCanaryConfigController()
configMgr.canaryConfigStore = store
configMgr.canaryConfigController = controller
return configMgr, nil
}
func (canaryCfgMgr *canaryConfigMgr) initCanaryConfigController() (k8sCache.Store, k8sCache.Controller) {
resyncPeriod := 30 * time.Second
listWatch := k8sCache.NewListWatchFromClient(canaryCfgMgr.crdClient, "canaryconfigs", metav1.NamespaceAll, fields.Everything())
store, controller := k8sCache.NewInformer(listWatch, &crd.CanaryConfig{}, resyncPeriod,
k8sCache.ResourceEventHandlerFuncs{
AddFunc: func(obj interface{}) {
canaryConfig := obj.(*crd.CanaryConfig)
if canaryConfig.Status.Status == fission.CanaryConfigStatusPending {
go canaryCfgMgr.addCanaryConfig(canaryConfig)
}
},
DeleteFunc: func(obj interface{}) {
canaryConfig := obj.(*crd.CanaryConfig)
go canaryCfgMgr.deleteCanaryConfig(canaryConfig)
},
UpdateFunc: func(oldObj interface{}, newObj interface{}) {
oldConfig := oldObj.(*crd.CanaryConfig)
newConfig := newObj.(*crd.CanaryConfig)
if oldConfig.Metadata.ResourceVersion != newConfig.Metadata.ResourceVersion &&
newConfig.Status.Status == fission.CanaryConfigStatusPending {
log.Printf("update canary config invoked for : %s.%s, newConfig.Status.Status=%s", newConfig.Metadata.Name, newConfig.Metadata.Namespace, newConfig.Status.Status)
go canaryCfgMgr.updateCanaryConfig(oldConfig, newConfig)
}
go canaryCfgMgr.reSyncCanaryConfigs()
},
})
return store, controller
}
func (canaryCfgMgr *canaryConfigMgr) Run(ctx context.Context) {
go canaryCfgMgr.canaryConfigController.Run(ctx.Done())
log.Printf("started Canary configmgr controller")
}
func (canaryCfgMgr *canaryConfigMgr) addCanaryConfig(canaryConfig *crd.CanaryConfig) {
log.Printf("addCanaryConfig called for %s", canaryConfig.Metadata.Name)
// for each canary config, create a ticker with increment interval
interval, err := time.ParseDuration(canaryConfig.Spec.WeightIncrementDuration)
if err != nil {
log.Printf("Error parsing duration: %v, cant proceed with this canaryConfig : %v.%v", err,
canaryConfig.Metadata.Name, canaryConfig.Metadata.Namespace)
return
}
ticker := time.NewTicker(interval)
// create a context cancel func for each canary config. this will be used to cancel the processing of this canary
// config in the event that it's deleted
ctx, cancel := context.WithCancel(context.Background())
cacheValue := &CanaryProcessingInfo{
CancelFunc: &cancel,
Ticker: ticker,
}
err = canaryCfgMgr.canaryCfgCancelFuncMap.assign(&canaryConfig.Metadata, cacheValue)
if err != nil {
log.Printf("Error caching canary config : %s.%s. err : %v", canaryConfig.Metadata.Name, canaryConfig.Metadata.Namespace, err)
return
}
canaryCfgMgr.processCanaryConfig(&ctx, canaryConfig, ticker)
}
func (canaryCfgMgr *canaryConfigMgr) processCanaryConfig(ctx *context.Context, canaryConfig *crd.CanaryConfig, ticker *time.Ticker) {
quit := make(chan struct{})
for {
select {
case <-(*ctx).Done():
// this case when someone deleted their canary config in the middle of it being processed
log.Printf("Cancel Func called for canary config : %s", canaryConfig.Metadata.Name)
err := canaryCfgMgr.canaryCfgCancelFuncMap.remove(&canaryConfig.Metadata)
if err != nil {
log.Printf("error removing canary config: %s from map, err : %v", canaryConfig.Metadata.Name, err)
}
return
case <-ticker.C:
// every weightIncrementDuration, check if failureThreshold has reached.
// if yes, rollback.
// else, increment the weight of new function and decrement old function by `weightIncrement`
log.Printf("Processing canary config : %s.%s", canaryConfig.Metadata.Name, canaryConfig.Metadata.Namespace)
canaryCfgMgr.RollForwardOrBack(canaryConfig, quit, ticker)
case <-quit:
// we're done processing this canary config either because the new function receives 100% of the traffic
// or we rolled back to send all 100% traffic to old function
log.Printf("Quit processing canaryConfig : %s", canaryConfig.Metadata.Name)
err := canaryCfgMgr.canaryCfgCancelFuncMap.remove(&canaryConfig.Metadata)
if err != nil {
log.Printf("error removing canary config: %s from map, err : %v", canaryConfig.Metadata.Name, err)
}
return
}
}
}
func (canaryCfgMgr *canaryConfigMgr) RollForwardOrBack(canaryConfig *crd.CanaryConfig, quit chan struct{}, ticker *time.Ticker) {
// handle race between delete event and notification on ticker.C
_, err := canaryCfgMgr.canaryCfgCancelFuncMap.lookup(&canaryConfig.Metadata)
if err != nil {
log.Printf("No need of processing the config, not in cache anymore")
return
}
// get the http trigger object associated with this canary config
triggerObj, err := canaryCfgMgr.fissionClient.HTTPTriggers(canaryConfig.Metadata.Namespace).Get(canaryConfig.Spec.Trigger)
if err != nil {
// if the http trigger is not found, then give up processing this config.
if k8serrors.IsNotFound(err) {
log.Printf("Http trigger object : %v.%v missing", canaryConfig.Spec.Trigger, canaryConfig.Metadata.Namespace)
close(quit)
return
}
// just silently ignore. wait for next window to increment weight
log.Printf("Error fetching http trigger object, err : %v", err)
return
}
// handle a race between ticker.Stop and receiving a notification on ticker.C
if canaryConfig.Status.Status != fission.CanaryConfigStatusPending {
log.Printf("No need of processing the config, not pending anymore")
return
}
if triggerObj.Spec.FunctionReference.Type == fission.FunctionReferenceTypeFunctionWeights &&
triggerObj.Spec.FunctionReference.FunctionWeights[canaryConfig.Spec.NewFunction] != 0 {
failurePercent, err := canaryCfgMgr.promClient.GetFunctionFailurePercentage(triggerObj.Spec.RelativeURL, triggerObj.Spec.Method,
canaryConfig.Spec.NewFunction, canaryConfig.Metadata.Namespace, canaryConfig.Spec.WeightIncrementDuration)
if err != nil {
// silently ignore. wait for next window to increment weight
log.Printf("Error calculating failure percentage, err : %v", err)
return
}
log.Printf("Failure percentage calculated : %v for canaryConfig %s", failurePercent, canaryConfig.Metadata.Name)
if failurePercent == -1 {
// this means there were no requests triggered to this url during this window. return here and check back
// during next iteration
log.Printf("Total requests received for url : %v is 0", triggerObj.Spec.RelativeURL)
return
}
if int(failurePercent) > canaryConfig.Spec.FailureThreshold {
log.Printf("Failure percent %v crossed the threshold %v, so rolling back", failurePercent, canaryConfig.Spec.FailureThreshold)
ticker.Stop()
canaryCfgMgr.rollback(canaryConfig, triggerObj)
close(quit)
return
}
}
doneProcessingCanaryConfig, err := canaryCfgMgr.rollForward(canaryConfig, triggerObj)
if err != nil {
// just log the error and hope that next iteration will succeed
log.Printf("Error incrementing weights for triggerObj : %v, err : %v", triggerObj.Metadata.Name, err)
return
}
if doneProcessingCanaryConfig {
ticker.Stop()
// update the status of canary config as done processing, we dont care if we arent able to update because
// resync takes care of the update
err = canaryCfgMgr.updateCanaryConfigStatusWithRetries(canaryConfig.Metadata.Name, canaryConfig.Metadata.Namespace,
fission.CanaryConfigStatusSucceeded)
if err != nil {
// cant do much after max retries other than logging it.
log.Printf("Error updating canary config : %s.%s after max retries, err :%v", canaryConfig.Metadata.Name, canaryConfig.Metadata.Namespace,
err)
}
log.Printf("We're done processing canary config : %s. The new function is receiving all the traffic", canaryConfig.Metadata.Name)
close(quit)
return
}
}
func (canaryCfgMgr *canaryConfigMgr) updateHttpTriggerWithRetries(triggerName, triggerNamespace string, fnWeights map[string]int) (err error) {
for i := 0; i < fission.MaxRetries; i++ {
triggerObj, err := canaryCfgMgr.fissionClient.HTTPTriggers(triggerNamespace).Get(triggerName)
if err != nil {
log.Printf("Error getting http trigger object : %v", err)
return err
}
triggerObj.Spec.FunctionReference.FunctionWeights = fnWeights
_, err = canaryCfgMgr.fissionClient.HTTPTriggers(triggerNamespace).Update(triggerObj)
switch {
case err == nil:
log.Printf("Updated Http trigger : %s.%s", triggerName, triggerNamespace)
return nil
case k8serrors.IsConflict(err):
log.Printf("Conflict in updating http trigger : %s.%s, retrying", triggerName, triggerNamespace)
continue
default:
log.Printf("Error updating trigger : %s.%s = %v", triggerName, triggerNamespace, err)
return err
}
}
return err
}
func (canaryCfgMgr *canaryConfigMgr) updateCanaryConfigStatusWithRetries(cfgName, cfgNamespace string, status string) (err error) {
for i := 0; i < fission.MaxRetries; i++ {
canaryCfgObj, err := canaryCfgMgr.fissionClient.CanaryConfigs(cfgNamespace).Get(cfgName)
if err != nil {
log.Printf("Error getting http Canary Config object : %v", err)
return err
}
log.Printf("Updating status of canaryCfg : %s.%s to %s", cfgName, cfgNamespace, status)
canaryCfgObj.Status.Status = status
_, err = canaryCfgMgr.fissionClient.CanaryConfigs(cfgNamespace).Update(canaryCfgObj)
switch {
case err == nil:
log.Printf("Updated Canary Config : %s.%s", cfgName, cfgNamespace)
return nil
case k8serrors.IsConflict(err):
log.Printf("Conflict in updating Canary Config : %s.%s, retrying", cfgName, cfgNamespace)
continue
default:
log.Printf("Error updating Canary Config : %s.%s = %v", cfgName, cfgNamespace, err)
return err
}
}
return err
}
func (canaryCfgMgr *canaryConfigMgr) rollback(canaryConfig *crd.CanaryConfig, trigger *crd.HTTPTrigger) error {
functionWeights := trigger.Spec.FunctionReference.FunctionWeights
functionWeights[canaryConfig.Spec.NewFunction] = 0
functionWeights[canaryConfig.Spec.OldFunction] = 100
err := canaryCfgMgr.updateHttpTriggerWithRetries(trigger.Metadata.Name, trigger.Metadata.Namespace, functionWeights)
err = canaryCfgMgr.updateCanaryConfigStatusWithRetries(canaryConfig.Metadata.Name, canaryConfig.Metadata.Namespace,
fission.CanaryConfigStatusFailed)
return err
}
func (canaryCfgMgr *canaryConfigMgr) rollForward(canaryConfig *crd.CanaryConfig, trigger *crd.HTTPTrigger) (bool, error) {
doneProcessingCanaryConfig := false
functionWeights := trigger.Spec.FunctionReference.FunctionWeights
if functionWeights[canaryConfig.Spec.NewFunction]+canaryConfig.Spec.WeightIncrement >= 100 {
doneProcessingCanaryConfig = true
functionWeights[canaryConfig.Spec.NewFunction] = 100
functionWeights[canaryConfig.Spec.OldFunction] = 0
} else {
functionWeights[canaryConfig.Spec.NewFunction] += canaryConfig.Spec.WeightIncrement
if functionWeights[canaryConfig.Spec.OldFunction]-canaryConfig.Spec.WeightIncrement < 0 {
functionWeights[canaryConfig.Spec.OldFunction] = 0
} else {
functionWeights[canaryConfig.Spec.OldFunction] -= canaryConfig.Spec.WeightIncrement
}
}
log.Printf("Incremented functionWeights : %v", functionWeights)
err := canaryCfgMgr.updateHttpTriggerWithRetries(trigger.Metadata.Name, trigger.Metadata.Namespace, functionWeights)
return doneProcessingCanaryConfig, err
}
func (canaryCfgMgr *canaryConfigMgr) reSyncCanaryConfigs() {
for _, obj := range canaryCfgMgr.canaryConfigStore.List() {
canaryConfig := obj.(*crd.CanaryConfig)
_, err := canaryCfgMgr.canaryCfgCancelFuncMap.lookup(&canaryConfig.Metadata)
if err != nil && canaryConfig.Status.Status == fission.CanaryConfigStatusPending {
log.Printf("Adding canary config : %s.%s from resync loop", canaryConfig.Metadata.Name, canaryConfig.Metadata.Namespace)
// new canaryConfig detected, add it to our cache and start processing it
go canaryCfgMgr.addCanaryConfig(canaryConfig)
}
}
}
func (canaryCfgMgr *canaryConfigMgr) deleteCanaryConfig(canaryConfig *crd.CanaryConfig) {
log.Printf("Delete event received for canary config : %v, %v, %v", canaryConfig.Metadata.Name, canaryConfig.Metadata.Namespace, canaryConfig.Metadata.ResourceVersion)
canaryProcessingInfo, err := canaryCfgMgr.canaryCfgCancelFuncMap.lookup(&canaryConfig.Metadata)
if err != nil {
log.Printf("lookup of canaryConfig failed, err : %v", err)
return
}
// first stop the ticker
canaryProcessingInfo.Ticker.Stop()
// call cancel func so that the ctx.Done returns inside processCanaryConfig function and processing gets stopped
(*canaryProcessingInfo.CancelFunc)()
}
func (canaryCfgMgr *canaryConfigMgr) updateCanaryConfig(oldCanaryConfig *crd.CanaryConfig, newCanaryConfig *crd.CanaryConfig) {
// before removing the object from cache, we need to get it's cancel func and cancel it
canaryCfgMgr.deleteCanaryConfig(oldCanaryConfig)
err := canaryCfgMgr.canaryCfgCancelFuncMap.remove(&oldCanaryConfig.Metadata)
if err != nil {
log.Printf("error removing canary config: %s from map, err : %v", oldCanaryConfig.Metadata.Name, err)
return
}
canaryCfgMgr.addCanaryConfig(newCanaryConfig)
}
+163
View File
@@ -0,0 +1,163 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package canaryconfigmgr
import (
"fmt"
"golang.org/x/net/context"
"time"
promClient "github.com/prometheus/client_golang/api/prometheus"
"github.com/prometheus/common/model"
log "github.com/sirupsen/logrus"
)
type PrometheusApiClient struct {
client promClient.QueryAPI
}
func MakePrometheusClient(prometheusSvc string) *PrometheusApiClient {
promApiConfig := promClient.Config{
Address: prometheusSvc,
}
promApiClient, err := promClient.New(promApiConfig)
if err != nil {
log.Errorf("Error creating prometheus api client for svc : %s, err : %v", prometheusSvc, err)
}
apiQueryClient := promClient.NewQueryAPI(promApiClient)
log.Printf("Successfully made prometheus client with service : %s", prometheusSvc)
return &PrometheusApiClient{
client: apiQueryClient,
}
}
func (promApiClient *PrometheusApiClient) GetFunctionFailurePercentage(path, method, funcName, funcNs string, window string) (float64, error) {
// first get a total count of requests to this url in a time window
reqs, err := promApiClient.GetRequestsToFuncInWindow(path, method, funcName, funcNs, window)
if err != nil {
return 0, err
}
if reqs <= 0 {
return -1, fmt.Errorf("no requests to this url %v and method %v in the window : %v", path, method, window)
}
// next, get a total count of errored out requests to this function in the same window
failedReqs, err := promApiClient.GetTotalFailedRequestsToFuncInWindow(funcName, funcNs, path, method, window)
if err != nil {
return 0, err
}
// calculate the failure percentage of the function
failurePercentForFunc := (failedReqs / reqs) * 100
return failurePercentForFunc, nil
}
func (promApiClient *PrometheusApiClient) GetRequestsToFuncInWindow(path string, method string, funcName string, funcNs string, window string) (float64, error) {
queryString := fmt.Sprintf("fission_function_calls_total{path=\"%s\",method=\"%s\",name=\"%s\",namespace=\"%s\"}[%v]", path, method, funcName, funcNs, window)
reqs, err := promApiClient.executeQuery(queryString)
if err != nil {
log.Printf("Error executing query : %s, err : %v", queryString, err)
return 0, err
}
queryString = fmt.Sprintf("fission_function_calls_total{path=\"%s\",method=\"%s\",name=\"%s\",namespace=\"%s\"} offset %v", path, method, funcName, funcNs, window)
reqsInPrevWindow, err := promApiClient.executeQuery(queryString)
if err != nil {
log.Printf("Error executing query : %s, err : %v", queryString, err)
return 0, err
}
reqsInCurrentWindow := reqs - reqsInPrevWindow
log.Printf("reqs : %v, reqsInPrevWindow : %v, reqsInCurrentWindow : %v to function %v", reqs, reqsInPrevWindow, reqsInCurrentWindow, funcName)
return reqsInCurrentWindow, nil
}
func (promApiClient *PrometheusApiClient) GetTotalFailedRequestsToFuncInWindow(funcName string, funcNs string, path string, method string, window string) (float64, error) {
queryString := fmt.Sprintf("fission_function_errors_total{name=\"%s\",namespace=\"%s\",path=\"%s\", method=\"%s\"}[%v]", funcName, funcNs, path, method, window)
failedRequests, err := promApiClient.executeQuery(queryString)
if err != nil {
log.Printf("Error executing query : %s, err : %v", queryString, err)
return 0, err
}
queryString = fmt.Sprintf("fission_function_errors_total{name=\"%s\",namespace=\"%s\",path=\"%s\", method=\"%s\"} offset %v", funcName, funcNs, path, method, window)
failedReqsInPrevWindow, err := promApiClient.executeQuery(queryString)
if err != nil {
log.Printf("Error executing query : %s, err : %v", queryString, err)
return 0, err
}
failedReqsInCurrentWindow := failedRequests - failedReqsInPrevWindow
log.Printf("failedReqs : %v, failedReqsInPrevWindow : %v, failedReqsInCurrentWindow : %v to function : %v", failedRequests, failedReqsInPrevWindow, failedReqsInCurrentWindow, funcName)
return failedReqsInCurrentWindow, nil
}
func (promApiClient *PrometheusApiClient) executeQuery(queryString string) (float64, error) {
val, err := promApiClient.client.Query(context.Background(), queryString, time.Now())
if err != nil {
log.Errorf("Error querying prometheus qs : %v, err : %v", queryString, err)
return 0, err
}
switch {
case val.Type() == model.ValScalar:
scalarVal := val.(*model.Scalar)
return float64(scalarVal.Value), nil
case val.Type() == model.ValVector:
vectorVal := val.(model.Vector)
total := float64(0)
for _, elem := range vectorVal {
total = total + float64(elem.Value)
}
return total, nil
case val.Type() == model.ValMatrix:
matrixVal := val.(model.Matrix)
total := float64(0)
for _, elem := range matrixVal {
//log.Printf("Only one value, so taking the 0th elem")
total += float64(elem.Values[len(elem.Values)-1].Value)
}
return total, nil
default:
log.Printf("type unrecognized")
return 0, nil
}
}
func addInterval(window string) string {
timeDuration, _ := time.ParseDuration(window)
log.Println("window in seconds", int64(timeDuration/time.Second))
timeInStr := fmt.Sprintf("%ds", int64((timeDuration+timeDuration)/time.Second))
fmt.Println(timeInStr)
return timeInStr
}
+10 -5
View File
@@ -56,11 +56,16 @@ The following table lists the configurable parameters of the Fission chart and t
| `logger.influxdbAdmin` | Log database admin username | `admin` |
| `logger.fluentdImage` | Logger fluentd image | `fission/fluentd` |
| `fissionUiImage` | Fission ui image | `fission/fission-ui:0.1.0` |
| `messageQueue` | Message queue type | `nats-streaming` |
| `nats.authToken` | Nats streaming auth token | `defaultFissionAuthToken` |
| `nats.clusterID` | Nats streaming clusterID | `fissionMQTrigger` |
| `azureStorageQueue.accountName` | Azure storage account name | None (required if `messageQueue` is `azure-storage-queue`) |
| `azureStorageQueue.key` | Azure storage access key | None (required if `messageQueue` is `azure-storage-queue`) |
| `nats.enabled` | Nats streaming enabled | `true` |
| `nats.authToken` | Nats streaming auth token | `defaultFissionAuthToken`(required if `nats.enabled` is `true`) |
| `nats.clusterID` | Nats streaming clusterID | `fissionMQTrigger`(required if `nats.enabled` is `true`) |
| `azureStorageQueue.enabled` * | Azure storage account name | false |
| `azureStorageQueue.accountName` | Azure storage account name | None (required if `azureStorageQueue.enabled` is `true`) |
| `azureStorageQueue.key` | Azure storage access key | None (required if `azureStorageQueue.enabled` is `true`) |
| `kafka.enabled` * | Kafka trigger enabled | `false` |
| `kafka.brokers` | Kafka brokers uri | `broker.kafka:9092` (required if `kafka.enabled` is `true`) |
* - Please note that deploying of Azure Storage Queue or Kafka is not done by Fission chart and you will have to explicitly deploy them.
Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example,
+2 -2
View File
@@ -1,6 +1,6 @@
apiVersion: v1
name: fission-all
version: 0.9.2
version: 0.12.0
description: Fission is a fast serverless framework for Kubernetes.
keywords:
- fission
@@ -12,4 +12,4 @@ maintainers:
- name: Ta Ching Chen
email: contact@tachingchen.com
engine: gotpl
appVersion: 0.9.2
appVersion: 0.12.0
+5
View File
@@ -0,0 +1,5 @@
dependencies:
- name: prometheus
version: 7.1.0
repository: https://kubernetes-charts.storage.googleapis.com
condition: prometheusDeploy
+3 -3
View File
@@ -1,13 +1,13 @@
1. Install the client CLI.
Mac:
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.9.2/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.12.0/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
Linux:
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.9.2/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.12.0/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
Windows:
For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/0.9.2/fission-cli-windows.exe
For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/0.12.0/fission-cli-windows.exe
2. You're ready to use Fission!
+17
View File
@@ -14,3 +14,20 @@ We truncate at 24 chars because some Kubernetes name fields are limited to this
{{- $name := default .Chart.Name .Values.nameOverride -}}
{{- printf "%s-%s" .Release.Name $name | trunc 24 | trimSuffix "-" -}}
{{- end -}}
{{/*
This is a template with config parameters for optional features in fission. This gets mounted on to the controller pod
as a config map.
To add new features with config parameters, create a yaml block below with the feature name and define a corresponding struct in
controller/config.go
*/}}
{{- define "config" -}}
canary:
enabled: {{ .Values.canaryDeployment.enabled }}
{{- if .Values.prometheusDeploy }}
prometheusSvc: "http://{{ .Release.Name }}-prometheus-server.{{ .Release.Namespace }}"
{{- end }}
{{- printf "\n" -}}
{{- end -}}
+104 -27
View File
@@ -1,4 +1,5 @@
---
{{- if .Values.createNamespace }}
apiVersion: v1
kind: Namespace
metadata:
@@ -21,7 +22,7 @@ metadata:
{{- if .Values.enableIstio }}
istio-injection: enabled
{{- end }}
{{- end }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
@@ -102,6 +103,15 @@ metadata:
name: fission-builder
namespace: {{ .Values.builderNamespace }}
---
apiVersion: v1
kind: ConfigMap
metadata:
name: feature-config
namespace: {{ .Release.Namespace }}
data:
"config.yaml": {{ include "config" . | b64enc }}
---
apiVersion: extensions/v1beta1
kind: Deployment
@@ -119,7 +129,7 @@ spec:
spec:
containers:
- name: controller
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
image: "{{ .Values.repository }}/{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--controllerPort", "8888"]
@@ -143,8 +153,15 @@ spec:
port: 8888
initialDelaySeconds: 35
periodSeconds: 5
volumeMounts:
- name: config-volume
mountPath: /etc/config/config.yaml
subPath: config.yaml
serviceAccount: fission-svc
volumes:
- name: config-volume
configMap:
name: feature-config
---
apiVersion: extensions/v1beta1
kind: Deployment
@@ -166,7 +183,7 @@ spec:
spec:
containers:
- name: router
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
image: "{{ .Values.repository }}/{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--routerPort", "8888", "--executorUrl", "http://executor.{{ .Release.Namespace }}"]
@@ -183,6 +200,12 @@ spec:
value: {{ .Values.routerRoundTripKeepAliveTime | default "30s" | quote }}
- name: ROUTER_ROUND_TRIP_MAX_RETRIES
value: {{ .Values.routerRoundTripMaxRetries | default 10 | quote }}
- name: ROUTER_ROUND_TRIP_SVC_ADDRESS_MAX_RETRIES
value: {{ .Values.routerRoundTripSvcAddressMaxRetries | default 5 | quote }}
- name: ROUTER_ROUND_TRIP_SVC_ADDRESS_UPDATE_TIMEOUT
value: {{ .Values.routerRoundTripSvcAddressUpdateTimeout | default 30 | quote }}
- name: DEBUG_ENV
value: {{ .Values.debugEnv | quote }}
readinessProbe:
httpGet:
path: "/router-healthz"
@@ -239,7 +262,7 @@ spec:
spec:
containers:
- name: executor
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
image: "{{ .Values.repository }}/{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--executorPort", "8888", "--namespace", "{{ .Values.functionNamespace }}", "--fission-namespace", "{{ .Release.Namespace }}"]
@@ -296,7 +319,7 @@ spec:
spec:
containers:
- name: buildermgr
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
image: "{{ .Values.repository }}/{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--builderMgr", "--storageSvcUrl", "http://storagesvc.{{ .Release.Namespace }}", "--envbuilder-namespace", "{{ .Values.builderNamespace }}"]
@@ -325,7 +348,7 @@ spec:
spec:
containers:
- name: kubewatcher
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
image: "{{ .Values.repository }}/{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--kubewatcher", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
@@ -340,7 +363,7 @@ metadata:
svc: influxdb
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
spec:
type: ClusterIP
type: ClusterIP
ports:
- port: 8086
targetPort: 8086
@@ -363,7 +386,7 @@ spec:
spec:
containers:
- name: influxdb
image: tutum/influxdb
image: fission/influxdb
env:
- name: PRE_CREATE_DB
value: fissionFunctionLog
@@ -419,8 +442,9 @@ spec:
- /heapster
- --source=kubernetes:https://kubernetes.default
serviceAccount: {{ .Release.Namespace }}/fission-svc
---
{{- end -}}
---
{{- end }}
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
@@ -435,7 +459,7 @@ spec:
spec:
containers:
- name: fluentd
image: "{{ .Values.logger.fluentdImage }}:{{ .Values.logger.fluentdImageTag }}"
image: "{{ .Values.repository }}/{{ .Values.logger.fluentdImage }}:{{ .Values.logger.fluentdImageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
env:
- name: INFLUXDB_ADDRESS
@@ -495,7 +519,7 @@ spec:
spec:
containers:
- name: timer
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
image: "{{ .Values.repository }}/{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--timer", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
@@ -522,14 +546,14 @@ spec:
# spec:
# containers:
# - name: nginx
# image: {{ .Values.fissionUiImage }}
# image: {{ .Values.repository }}/{{ .Values.fissionUiImage }}
# - name: kubectl-proxy
# image: lachlanevenson/k8s-kubectl
# image: {{ .Values.repository }}/lachlanevenson/k8s-kubectl
# args: ["proxy", "--port", "8001", "--address", "127.0.0.1"]
# serviceAccount: fission-svc
---
{{- if eq .Values.messageQueue.type "nats-streaming" }}
{{- if .Values.nats.enabled }}
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
@@ -551,17 +575,16 @@ spec:
"--auth", "{{ .Values.nats.authToken }}",
"--max_channels", "0"
]
ports:
- containerPort: 4222
hostPort: 4222
protocol: TCP
{{- end }}
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: mqtrigger
name: mqtrigger-nats-streaming
labels:
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
spec:
@@ -570,6 +593,65 @@ spec:
metadata:
labels:
svc: mqtrigger
messagequeue: nats-streaming
spec:
containers:
- name: mqtrigger
image: "{{ .Values.repository }}/{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--mqt", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
env:
- name: MESSAGE_QUEUE_TYPE
value: nats-streaming
- name: MESSAGE_QUEUE_URL
value: nats://{{ .Values.nats.authToken }}@nats-streaming:4222
serviceAccount: fission-svc
{{- end }}
---
{{- if .Values.kafka.enabled }}
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: mqtrigger-kafka
labels:
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
spec:
replicas: 1
template:
metadata:
labels:
svc: mqtrigger
messagequeue: kafka
spec:
containers:
- name: mqtrigger
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--mqt"]
env:
- name: MESSAGE_QUEUE_TYPE
value: kafka
- name: MESSAGE_QUEUE_URL
value: "{{.Values.kafka.brokers}}"
serviceAccount: fission-svc
{{- end }}
---
{{- if .Values.azureStorageQueue.enabled }}
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: mqtrigger-azure-storage-queue
labels:
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
spec:
replicas: 1
template:
metadata:
labels:
svc: mqtrigger
messagequeue: azure-storage-queue
spec:
containers:
- name: mqtrigger
@@ -579,11 +661,7 @@ spec:
args: ["--mqt", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
env:
- name: MESSAGE_QUEUE_TYPE
value: {{ .Values.messageQueue.type }}
{{- if eq .Values.messageQueue.type "nats-streaming" }}
- name: MESSAGE_QUEUE_URL
value: nats://{{ .Values.nats.authToken }}@nats-streaming:4222
{{- else if eq .Values.messageQueue.type "azure-storage-queue" }}
value: azure-storage-queue
- name: AZURE_STORAGE_ACCOUNT_NAME
value: {{ required "An Azure storage account name is required." .Values.azureStorageQueue.accountName }}
- name: AZURE_STORAGE_ACCOUNT_KEY
@@ -591,9 +669,8 @@ spec:
secretKeyRef:
name: azure-storage-account-key
key: key
{{- end }}
serviceAccount: fission-svc
{{- end }}
---
apiVersion: extensions/v1beta1
kind: Deployment
@@ -611,7 +688,7 @@ spec:
spec:
containers:
- name: storagesvc
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
image: "{{ .Values.repository }}/{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--storageServicePort", "8000", "--filePath", "/fission"]
@@ -0,0 +1,31 @@
{{ if .Values.analyticsNonHelmInstall }}
apiVersion: batch/v1
kind: Job
metadata:
name: {{ template "fullname" . }}-{{ .Chart.Version }}-{{ randNumeric 3 }}
labels:
# The "release" convention makes it easy to tie a release to all of the
# Kubernetes resources that were created as part of that release.
release: {{ .Release.Name }}
# This makes it easy to audit chart usage.
chart: {{ .Chart.Name }}-{{ .Chart.Version }}
app: {{ template "name" . }}
spec:
#ttlSecondsAfterFinished: 120
template:
metadata:
name: {{ template "fullname" . }}
labels:
release: {{ .Release.Name }}
app: {{ template "name" . }}
annotations:
{{- if .Values.enableIstio }}
"sidecar.istio.io/inject": "false"
{{- end }}
spec:
restartPolicy: Never
containers:
- name: post-install-job
image: "fission/alpinecurl"
command: ["sh", "-c", "/usr/bin/curl -m 5 -H \"Content-Type: application/json\" -X POST -d '{\"type\": \"yaml-post-install\", \"chartName\": \"{{ .Chart.Name }}\", \"chartVersion\": \"{{ .Chart.Version }}\"}' https://g.fission.sh/analytics || true"]
{{ end }}
+1 -1
View File
@@ -10,7 +10,7 @@ data:
password: {{ randAlphaNum 20 | b64enc | quote }}
---
{{- if eq .Values.messageQueue.type "azure-storage-queue" }}
{{- if .Values.azureStorageQueue.enabled }}
apiVersion: v1
kind: Secret
metadata:
+2 -3
View File
@@ -38,7 +38,7 @@ spec:
svc: controller
---
{{- if eq .Values.messageQueue.type "nats-streaming" }}
{{- if .Values.nats.enabled }}
apiVersion: v1
kind: Service
metadata:
@@ -57,7 +57,6 @@ spec:
selector:
svc: nats-streaming
{{- end }}
---
apiVersion: v1
kind: Service
@@ -73,4 +72,4 @@ spec:
- port: 80
targetPort: 8000
selector:
svc: storagesvc
svc: storagesvc
+32 -11
View File
@@ -9,6 +9,9 @@ serviceType: ClusterIP
## For minikube, set this to NodePort, elsewhere use LoadBalancer or ClusterIP.
routerServiceType: LoadBalancer
## Image base repository
repository: index.docker.io
## Fission image repository
image: fission/fission-bundle
@@ -16,13 +19,13 @@ image: fission/fission-bundle
pullPolicy: IfNotPresent
## Fission image version
imageTag: 0.9.2
imageTag: 0.12.0
## Fission fetcher repository
fetcherImage: fission/fetcher
## Fission fetcher image version
fetcherImageTag: 0.9.2
fetcherImageTag: 0.12.0
## Port at which Fission controller service should be exposed
controllerPort: 31313
@@ -33,6 +36,9 @@ routerPort: 31314
## Port at which NATS streaming service should be exposed
natsStreamingPort: 31316
## Set to false if you create the namespaces manually
createNamespace: true
## Namespace in which to run fission functions (this is different from
## the release namespace)
functionNamespace: fission-function
@@ -48,23 +54,25 @@ enableIstio: false
logger:
influxdbAdmin: "admin"
fluentdImage: fission/fluentd
fluentdImageTag: 0.9.2
## Type of Queue you would like to use
## currently supports nats-streaming, azure-storage-queue
messageQueue:
type: nats-streaming
fluentdImageTag: 0.12.0
## Message queue trigger config
### NATS Streaming
### NATS Streaming, enabled by default
nats:
enabled: true
authToken: "defaultFissionAuthToken"
clusterID: "fissionMQTrigger"
## Required if messageQueue type is azure-storage-queue
## Azure-storage-queue: enable and configure the details
azureStorageQueue:
enabled: false
key: ""
accountName: ""
## Kafka: enable and configure the details
kafka:
enabled: false
brokers: 'broker.kafka:9092'
## Persist data to a persistent volume.
persistence:
@@ -83,6 +91,8 @@ persistence:
## false to disable analytics.
analytics: true
## Internally used for generating an analytics job for non-helm installs
analyticsNonHelmInstall: false
## Enable Heapster only in clusters where heapster does not exist already
heapster: false
@@ -93,4 +103,15 @@ heapster: false
pruneInterval: 60
## Fission pre-install/pre-upgrade checks live in this image
preUpgradeChecksImage: fission/pre-upgrade-checks
preUpgradeChecksImage: fission/pre-upgrade-checks
## if there are any pod specialization errors when a function is triggered and this flag is set to true, the error
## summary is returned as part of http response
debugEnv: true
## set this flag to true if prometheus needs to be deployed along with fission
prometheusDeploy: true
## set this flag to false if you dont need canary deployment feature
canaryDeployment:
enabled: true
+2 -2
View File
@@ -1,6 +1,6 @@
apiVersion: v1
name: fission-core
version: 0.9.2
version: 0.12.0
description: Fission is a fast serverless framework for Kubernetes.
keywords:
- fission
@@ -12,4 +12,4 @@ maintainers:
- name: Ta Ching Chen
email: contact@tachingchen.com
engine: gotpl
appVersion: 0.9.2
appVersion: 0.12.0
+5
View File
@@ -0,0 +1,5 @@
dependencies:
- name: prometheus
version: 7.1.0
repository: https://kubernetes-charts.storage.googleapis.com
condition: prometheusDeploy
+3 -3
View File
@@ -1,13 +1,13 @@
1. Install the client CLI.
Mac:
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.9.2/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.12.0/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
Linux:
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.9.2/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.12.0/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
Windows:
For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/0.9.2/fission-cli-windows.exe
For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/0.12.0/fission-cli-windows.exe
2. You're ready to use Fission!
@@ -14,3 +14,18 @@ We truncate at 24 chars because some Kubernetes name fields are limited to this
{{- $name := default .Chart.Name .Values.nameOverride -}}
{{- printf "%s-%s" .Release.Name $name | trunc 24 | trimSuffix "-" -}}
{{- end -}}
{{/*
This is a template with config parameters for optional features in fission. This gets mounted on to the controller pod
as a config map.
To add new features with config parameters, create a yaml block below with the feature name and define a corresponding struct in
controller/config.go
*/}}
{{- define "config" -}}
canary:
enabled: {{ .Values.canaryDeployment.enabled }}
{{- if .Values.prometheusDeploy }}
prometheusSvc: "http://{{ .Release.Name }}-prometheus-server.{{ .Release.Namespace }}"
{{- end }}
{{- printf "\n" -}}
{{- end -}}
+30 -7
View File
@@ -103,6 +103,15 @@ metadata:
name: fission-builder
namespace: {{ .Values.builderNamespace }}
---
apiVersion: v1
kind: ConfigMap
metadata:
name: feature-config
namespace: {{ .Release.Namespace }}
data:
"config.yaml": {{ include "config" . | b64enc }}
---
apiVersion: extensions/v1beta1
kind: Deployment
@@ -120,7 +129,7 @@ spec:
spec:
containers:
- name: controller
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
image: "{{ .Values.repository }}/{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--controllerPort", "8888"]
@@ -144,7 +153,15 @@ spec:
port: 8888
initialDelaySeconds: 35
periodSeconds: 5
volumeMounts:
- name: config-volume
mountPath: /etc/config/config.yaml
subPath: config.yaml
serviceAccount: fission-svc
volumes:
- name: config-volume
configMap:
name: feature-config
---
apiVersion: extensions/v1beta1
@@ -163,7 +180,7 @@ spec:
spec:
containers:
- name: router
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
image: "{{ .Values.repository }}/{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--routerPort", "8888", "--executorUrl", "http://executor.{{ .Release.Namespace }}"]
@@ -180,6 +197,12 @@ spec:
value: {{ .Values.routerRoundTripKeepAliveTime | default "30s" | quote }}
- name: ROUTER_ROUND_TRIP_MAX_RETRIES
value: {{ .Values.routerRoundTripMaxRetries | default 10 | quote }}
- name: ROUTER_ROUND_TRIP_SVC_ADDRESS_MAX_RETRIES
value: {{ .Values.routerRoundTripSvcAddressMaxRetries | default 5 | quote }}
- name: ROUTER_ROUND_TRIP_SVC_ADDRESS_UPDATE_TIMEOUT
value: {{ .Values.routerRoundTripSvcAddressUpdateTimeout | default 30 | quote }}
- name: DEBUG_ENV
value: {{ .Values.debugEnv | quote }}
readinessProbe:
httpGet:
path: "/router-healthz"
@@ -227,7 +250,7 @@ spec:
spec:
containers:
- name: executor
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
image: "{{ .Values.repository }}/{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--executorPort", "8888", "--namespace", "{{ .Values.functionNamespace }}", "--fission-namespace", "{{ .Release.Namespace }}"]
@@ -277,7 +300,7 @@ spec:
spec:
containers:
- name: buildermgr
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
image: "{{ .Values.repository }}/{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--builderMgr", "--storageSvcUrl", "http://storagesvc.{{ .Release.Namespace }}", "--envbuilder-namespace", "{{ .Values.builderNamespace }}"]
@@ -306,7 +329,7 @@ spec:
spec:
containers:
- name: kubewatcher
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
image: "{{ .Values.repository }}/{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--kubewatcher", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
@@ -328,7 +351,7 @@ spec:
spec:
containers:
- name: timer
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
image: "{{ .Values.repository }}/{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--timer", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
@@ -351,7 +374,7 @@ spec:
spec:
containers:
- name: storagesvc
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
image: "{{ .Values.repository }}/{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--storageServicePort", "8000", "--filePath", "/fission"]
@@ -0,0 +1,31 @@
{{ if .Values.analyticsNonHelmInstall }}
apiVersion: batch/v1
kind: Job
metadata:
name: {{ template "fullname" . }}-{{ .Chart.Version }}-{{ randNumeric 3 }}
labels:
# The "release" convention makes it easy to tie a release to all of the
# Kubernetes resources that were created as part of that release.
release: {{ .Release.Name }}
# This makes it easy to audit chart usage.
chart: {{ .Chart.Name }}-{{ .Chart.Version }}
app: {{ template "name" . }}
spec:
#ttlSecondsAfterFinished: 120
template:
metadata:
name: {{ template "fullname" . }}
labels:
release: {{ .Release.Name }}
app: {{ template "name" . }}
annotations:
{{- if .Values.enableIstio }}
"sidecar.istio.io/inject": "false"
{{- end }}
spec:
restartPolicy: Never
containers:
- name: post-install-job
image: "fission/alpinecurl"
command: ["sh", "-c", "/usr/bin/curl -m 5 -H \"Content-Type: application/json\" -X POST -d '{\"type\": \"yaml-post-install\", \"chartName\": \"{{ .Chart.Name }}\", \"chartVersion\": \"{{ .Chart.Version }}\"}' https://g.fission.sh/analytics || true"]
{{ end }}
+20 -3
View File
@@ -9,11 +9,14 @@ serviceType: ClusterIP
## For minikube, set this to NodePort, elsewhere use LoadBalancer or ClusterIP.
routerServiceType: LoadBalancer
## Image base repository
repository: index.docker.io
## Fission image repository
image: fission/fission-bundle
## Fission image version
imageTag: 0.9.2
imageTag: 0.12.0
## Image pull policy
pullPolicy: IfNotPresent
@@ -22,7 +25,7 @@ pullPolicy: IfNotPresent
fetcherImage: fission/fetcher
## Fission fetcher image version
fetcherImageTag: 0.9.2
fetcherImageTag: 0.12.0
## Port at which Fission controller service should be exposed
controllerPort: 31313
@@ -58,10 +61,24 @@ persistence:
## false to disable analytics.
analytics: true
## Internally used for generating an analytics job for non-helm installs
analyticsNonHelmInstall: false
## Archive pruner is a garbage collector for archives on the fission storage service.
## This interval configures the frequency at which it runs inside the storagesvc pod.
## The value is in minutes.
pruneInterval: 60
## Fission pre-install/pre-upgrade checks live in this image
preUpgradeChecksImage: fission/pre-upgrade-checks
preUpgradeChecksImage: fission/pre-upgrade-checks
## if there are any pod specialization errors when a function is triggered and this flag is set to true, the error
## summary is returned as part of http response
debugEnv: true
## set this flag to true if prometheus needs to be deployed along with fission
prometheusDeploy: false
## set this flag to false if you dont need canary deployment feature
canaryDeployment:
enabled: false
+62 -1
View File
@@ -18,16 +18,20 @@ package fission
import (
"fmt"
"io/ioutil"
"net"
"net/http"
"os"
"os/signal"
"path/filepath"
"runtime/debug"
"strings"
"syscall"
"github.com/gorilla/handlers"
"github.com/imdario/mergo"
"github.com/mholt/archiver"
"github.com/satori/go.uuid"
apiv1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
@@ -108,13 +112,20 @@ func IsNetworkDialError(err error) bool {
return false
}
// IsReadyPod checks that all containers in a pod are ready and returns true if so
// IsReadyPod checks both all containers in a pod are ready and whether
// the .metadata.DeletionTimestamp is nil.
func IsReadyPod(pod *apiv1.Pod) bool {
// since its a utility function, just ensuring there is no nil pointer exception
if pod == nil {
return false
}
// pod is in "Terminating" status if deletionTimestamp is not nil
// https://github.com/kubernetes/kubernetes/issues/61376
if pod.ObjectMeta.DeletionTimestamp != nil {
return false
}
for _, cStatus := range pod.Status.ContainerStatuses {
if !cStatus.Ready {
return false
@@ -123,3 +134,53 @@ func IsReadyPod(pod *apiv1.Pod) bool {
return true
}
// GetTempDir creates and return a temporary directory
func GetTempDir() (string, error) {
tmpDir := uuid.NewV4().String()
dir, err := ioutil.TempDir("", tmpDir)
return dir, err
}
// FindAllGlobs returns a list of globs of input list.
func FindAllGlobs(inputList []string) ([]string, error) {
files := make([]string, 0)
for _, glob := range inputList {
f, err := filepath.Glob(glob)
if err != nil {
return nil, fmt.Errorf("Invalid glob %v: %v", glob, err)
}
files = append(files, f...)
}
return files, nil
}
func MakeArchive(targetName string, globs ...string) (string, error) {
files, err := FindAllGlobs(globs)
if err != nil {
return "", err
}
// zip up the file list
err = archiver.Zip.Make(targetName, files)
if err != nil {
return "", err
}
return filepath.Abs(targetName)
}
// RemoveZeroBytes remove empty byte(\x00) from input byte slice and return a new byte slice
// This function is trying to fix the problem that empty byte will fail os.Openfile
// For more information, please visit:
// 1. https://github.com/golang/go/issues/24195
// 2. https://play.golang.org/p/5F9ykC2tlbc
func RemoveZeroBytes(src []byte) []byte {
var bs []byte
for _, v := range src {
if v != 0 {
bs = append(bs, v)
}
}
return bs
}
+11 -1
View File
@@ -32,6 +32,7 @@ import (
"github.com/fission/fission"
"github.com/fission/fission/crd"
config "github.com/fission/fission/featureconfig"
"github.com/fission/fission/fission/logdb"
)
@@ -53,6 +54,7 @@ type (
workflowApiUrl string
functionNamespace string
useIstio bool
featureConfig *config.FeatureConfig
}
logDBConfig struct {
@@ -62,7 +64,7 @@ type (
}
)
func MakeAPI() (*API, error) {
func MakeAPI(featureConfig *config.FeatureConfig) (*API, error) {
api, err := makeCRDBackedAPI()
u := os.Getenv("STORAGE_SERVICE_URL")
@@ -93,6 +95,8 @@ func MakeAPI() (*API, error) {
api.functionNamespace = "fission-function"
}
api.featureConfig = featureConfig
return api, err
}
@@ -253,6 +257,12 @@ func (api *API) Serve(port int) {
r.HandleFunc("/v2/secrets/{secret}", api.SecretGet).Methods("GET")
r.HandleFunc("/v2/configmaps/{configmap}", api.ConfigMapGet).Methods("GET")
r.HandleFunc("/v2/canaryconfigs", api.CanaryConfigApiCreate).Methods("POST")
r.HandleFunc("/v2/canaryconfigs/{canaryConfig}", api.CanaryConfigApiGet).Methods("GET")
r.HandleFunc("/v2/canaryconfigs/{canaryConfig}", api.CanaryConfigApiUpdate).Methods("PUT")
r.HandleFunc("/v2/canaryconfigs/{canaryConfig}", api.CanaryConfigApiDelete).Methods("DELETE")
r.HandleFunc("/v2/canaryconfigs", api.CanaryConfigApiList).Methods("GET")
r.HandleFunc("/proxy/{dbType}", api.FunctionLogsApiPost).Methods("POST")
r.HandleFunc("/proxy/storage/v1/archive", api.StorageServiceProxy)
r.HandleFunc("/proxy/logs/{function}", api.FunctionPodLogs).Methods("POST")
+10 -4
View File
@@ -171,7 +171,10 @@ func TestHTTPTriggerApi(t *testing.T) {
tr, err := g.client.HTTPTriggerGet(m)
panicIf(err)
assert(testTrigger.Spec == tr.Spec, "trigger should match after reading")
assert(testTrigger.Spec.Method == tr.Spec.Method &&
testTrigger.Spec.RelativeURL == tr.Spec.RelativeURL &&
testTrigger.Spec.FunctionReference.Type == tr.Spec.FunctionReference.Type &&
testTrigger.Spec.FunctionReference.Name == tr.Spec.FunctionReference.Name, "trigger should match after reading")
testTrigger.Metadata.ResourceVersion = m.ResourceVersion
testTrigger.Spec.RelativeURL = "/hi"
@@ -272,7 +275,8 @@ func TestWatchApi(t *testing.T) {
panicIf(err)
assert(testWatch.Spec.Namespace == w.Spec.Namespace &&
testWatch.Spec.Type == w.Spec.Type &&
testWatch.Spec.FunctionReference == w.Spec.FunctionReference, "watch should match after reading")
testWatch.Spec.FunctionReference.Type == w.Spec.FunctionReference.Type &&
testWatch.Spec.FunctionReference.Name == w.Spec.FunctionReference.Name, "watch should match after reading")
testWatch.Metadata.Name = "yyy"
m2, err := g.client.WatchCreate(testWatch)
@@ -310,7 +314,9 @@ func TestTimeTriggerApi(t *testing.T) {
tr, err := g.client.TimeTriggerGet(m)
panicIf(err)
assert(testTrigger.Spec == tr.Spec, "trigger should match after reading")
assert(testTrigger.Spec.Cron == tr.Spec.Cron &&
testTrigger.Spec.FunctionReference.Type == tr.Spec.FunctionReference.Type &&
testTrigger.Spec.FunctionReference.Name == tr.Spec.FunctionReference.Name, "trigger should match after reading")
testTrigger.Metadata.ResourceVersion = m.ResourceVersion
testTrigger.Spec.Cron = "@hourly"
@@ -338,7 +344,7 @@ func TestMain(m *testing.M) {
return
}
go Start(8888)
go Start(8888, true)
time.Sleep(5 * time.Second)
g.client = client.MakeClient("http://localhost:8888")
+177
View File
@@ -0,0 +1,177 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package controller
import (
"encoding/json"
"io/ioutil"
"net/http"
"github.com/gorilla/mux"
log "github.com/sirupsen/logrus"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission"
"github.com/fission/fission/crd"
)
func (a *API) CanaryConfigApiCreate(w http.ResponseWriter, r *http.Request) {
if !a.featureConfig.CanaryConfig.IsEnabled {
a.respondWithError(w, fission.MakeError(http.StatusBadRequest, "Please enable canary feature while installing fission"))
return
}
body, err := ioutil.ReadAll(r.Body)
if err != nil {
a.respondWithError(w, err)
return
}
var canaryCfg crd.CanaryConfig
err = json.Unmarshal(body, &canaryCfg)
if err != nil {
log.Printf("Failed to unmarshal request body: [%v]", body)
a.respondWithError(w, err)
return
}
canaryCfgNew, err := a.fissionClient.CanaryConfigs(canaryCfg.Metadata.Namespace).Create(&canaryCfg)
if err != nil {
a.respondWithError(w, err)
return
}
resp, err := json.Marshal(canaryCfgNew.Metadata)
if err != nil {
a.respondWithError(w, err)
return
}
w.WriteHeader(http.StatusCreated)
a.respondWithSuccess(w, resp)
}
func (a *API) CanaryConfigApiGet(w http.ResponseWriter, r *http.Request) {
if !a.featureConfig.CanaryConfig.IsEnabled {
a.respondWithError(w, fission.MakeError(http.StatusBadRequest, "Please enable canary feature while installing fission"))
return
}
vars := mux.Vars(r)
name := vars["canaryConfig"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
canaryCfg, err := a.fissionClient.CanaryConfigs(ns).Get(name)
if err != nil {
a.respondWithError(w, err)
return
}
resp, err := json.Marshal(canaryCfg)
if err != nil {
a.respondWithError(w, err)
return
}
a.respondWithSuccess(w, resp)
}
func (a *API) CanaryConfigApiList(w http.ResponseWriter, r *http.Request) {
if !a.featureConfig.CanaryConfig.IsEnabled {
a.respondWithError(w, fission.MakeError(http.StatusBadRequest, "Please enable canary feature while installing fission"))
return
}
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
canaryCfgs, err := a.fissionClient.CanaryConfigs(ns).List(metav1.ListOptions{})
if err != nil {
a.respondWithError(w, err)
return
}
resp, err := json.Marshal(canaryCfgs.Items)
if err != nil {
a.respondWithError(w, err)
return
}
a.respondWithSuccess(w, resp)
}
func (a *API) CanaryConfigApiUpdate(w http.ResponseWriter, r *http.Request) {
if !a.featureConfig.CanaryConfig.IsEnabled {
a.respondWithError(w, fission.MakeError(http.StatusBadRequest, "Please enable canary feature while installing fission"))
return
}
body, err := ioutil.ReadAll(r.Body)
if err != nil {
a.respondWithError(w, err)
return
}
var c crd.CanaryConfig
err = json.Unmarshal(body, &c)
if err != nil {
a.respondWithError(w, err)
return
}
canayCfgNew, err := a.fissionClient.CanaryConfigs(c.Metadata.Namespace).Update(&c)
if err != nil {
a.respondWithError(w, err)
return
}
resp, err := json.Marshal(canayCfgNew.Metadata)
if err != nil {
a.respondWithError(w, err)
return
}
a.respondWithSuccess(w, resp)
}
func (a *API) CanaryConfigApiDelete(w http.ResponseWriter, r *http.Request) {
if !a.featureConfig.CanaryConfig.IsEnabled {
a.respondWithError(w, fission.MakeError(http.StatusBadRequest, "Please enable canary feature while installing fission"))
return
}
vars := mux.Vars(r)
name := vars["canaryConfig"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
err := a.fissionClient.CanaryConfigs(ns).Delete(name, &metav1.DeleteOptions{})
if err != nil {
a.respondWithError(w, err)
return
}
a.respondWithSuccess(w, []byte(""))
}
+133
View File
@@ -0,0 +1,133 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
import (
"bytes"
"encoding/json"
"fmt"
"net/http"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission/crd"
)
func (c *Client) CanaryConfigCreate(canaryConf *crd.CanaryConfig) (*metav1.ObjectMeta, error) {
reqbody, err := json.Marshal(canaryConf)
if err != nil {
return nil, err
}
resp, err := http.Post(c.url("canaryconfigs"), "application/json", bytes.NewReader(reqbody))
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := c.handleCreateResponse(resp)
if err != nil {
return nil, err
}
var m metav1.ObjectMeta
err = json.Unmarshal(body, &m)
if err != nil {
return nil, err
}
return &m, nil
}
func (c *Client) CanaryConfigGet(m *metav1.ObjectMeta) (*crd.CanaryConfig, error) {
relativeUrl := fmt.Sprintf("canaryconfigs/%v", m.Name)
relativeUrl += fmt.Sprintf("?namespace=%v", m.Namespace)
resp, err := http.Get(c.url(relativeUrl))
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := c.handleResponse(resp)
if err != nil {
return nil, err
}
var canaryCfg crd.CanaryConfig
err = json.Unmarshal(body, &canaryCfg)
if err != nil {
return nil, err
}
return &canaryCfg, nil
}
func (c *Client) CanaryConfigUpdate(canaryConf *crd.CanaryConfig) (*metav1.ObjectMeta, error) {
reqbody, err := json.Marshal(canaryConf)
if err != nil {
return nil, err
}
relativeUrl := fmt.Sprintf("canaryconfigs/%v", canaryConf.Metadata.Name)
resp, err := c.put(relativeUrl, "application/json", reqbody)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := c.handleResponse(resp)
if err != nil {
return nil, err
}
var m metav1.ObjectMeta
err = json.Unmarshal(body, &m)
if err != nil {
return nil, err
}
return &m, nil
}
func (c *Client) CanaryConfigDelete(m *metav1.ObjectMeta) error {
relativeUrl := fmt.Sprintf("canaryconfigs/%v", m.Name)
relativeUrl += fmt.Sprintf("?namespace=%v", m.Namespace)
return c.delete(relativeUrl)
}
func (c *Client) CanaryConfigList(ns string) ([]crd.CanaryConfig, error) {
relativeUrl := fmt.Sprintf("canaryconfigs?namespace=%v", ns)
resp, err := http.Get(c.url(relativeUrl))
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := c.handleResponse(resp)
if err != nil {
return nil, err
}
canaryCfgs := make([]crd.CanaryConfig, 0)
err = json.Unmarshal(body, &canaryCfgs)
if err != nil {
return nil, err
}
return canaryCfgs, nil
}
+65
View File
@@ -0,0 +1,65 @@
/*
Copyright 2018 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package controller
import (
"context"
"fmt"
log "github.com/sirupsen/logrus"
"k8s.io/client-go/kubernetes"
"github.com/fission/fission/canaryconfigmgr"
"github.com/fission/fission/crd"
config "github.com/fission/fission/featureconfig"
)
func ConfigCanaryFeature(context context.Context, fissionClient *crd.FissionClient, kubeClient *kubernetes.Clientset, featureConfig *config.FeatureConfig) error {
// start the appropriate controller
if featureConfig.CanaryConfig.IsEnabled {
canaryCfgMgr, err := canaryconfigmgr.MakeCanaryConfigMgr(fissionClient, kubeClient, fissionClient.GetCrdClient(),
featureConfig.CanaryConfig.PrometheusSvc)
if err != nil {
return fmt.Errorf("failed to start canary config manager: %v", err)
}
canaryCfgMgr.Run(context)
log.Printf("Started canary config manager")
}
return nil
}
// ConfigureFeatures gets the feature config and configures the features that are enabled
func ConfigureFeatures(context context.Context, unitTestMode bool, fissionClient *crd.FissionClient, kubeClient *kubernetes.Clientset) (*config.FeatureConfig, error) {
// set feature enabled to false if unitTestMode
if unitTestMode {
featureConfig := &config.FeatureConfig{}
return featureConfig, nil
}
// get the featureConfig from config map mounted onto the file system
featureConfig, err := config.GetFeatureConfig()
if err != nil {
log.Printf("Error getting feature config : %v", err)
return featureConfig, err
}
// configure respective features
// in the future when new optional features are added, we need to add corresponding feature handlers and invoke them here
err = ConfigCanaryFeature(context, fissionClient, kubeClient, featureConfig)
return featureConfig, err
}
+13 -3
View File
@@ -17,17 +17,18 @@ limitations under the License.
package controller
import (
"context"
"log"
"github.com/fission/fission"
"github.com/fission/fission/crd"
)
func Start(port int) {
func Start(port int, unitTestFlag bool) {
// setup a signal handler for SIGTERM
fission.SetupStackTraceHandler()
fc, _, apiExtClient, err := crd.MakeFissionClient()
fc, kc, apiExtClient, err := crd.MakeFissionClient()
if err != nil {
log.Fatalf("Failed to connect to K8s API: %v", err)
}
@@ -42,7 +43,16 @@ func Start(port int) {
log.Fatalf("Error waiting for CRDs: %v", err)
}
api, err := MakeAPI()
ctx, cancel := context.WithCancel(context.Background())
featureConfig, err := ConfigureFeatures(ctx, unitTestFlag, fc, kc)
if err != nil {
log.Printf("Error configuring features : %v. Proceeding without optional features", err.Error())
// set all features to false for the MakeApi call below
featureConfig.CanaryConfig.IsEnabled = false
}
defer cancel()
api, err := MakeAPI(featureConfig)
if err != nil {
log.Fatalf("Failed to start controller: %v", err)
}
+120
View File
@@ -0,0 +1,120 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package crd
import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/watch"
"k8s.io/client-go/kubernetes/scheme"
"k8s.io/client-go/rest"
)
type (
CanaryConfigInterface interface {
Create(*CanaryConfig) (*CanaryConfig, error)
Get(name string) (*CanaryConfig, error)
Update(*CanaryConfig) (*CanaryConfig, error)
Delete(name string, options *metav1.DeleteOptions) error
List(opts metav1.ListOptions) (*CanaryConfigList, error)
Watch(opts metav1.ListOptions) (watch.Interface, error)
}
canaryConfigClient struct {
client *rest.RESTClient
namespace string
}
)
func MakeCanaryConfigInterface(crdClient *rest.RESTClient, namespace string) CanaryConfigInterface {
return &canaryConfigClient{
client: crdClient,
namespace: namespace,
}
}
func (c *canaryConfigClient) Create(f *CanaryConfig) (*CanaryConfig, error) {
var result CanaryConfig
err := c.client.Post().
Resource("canaryconfigs").
Namespace(c.namespace).
Body(f).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (c *canaryConfigClient) Get(name string) (*CanaryConfig, error) {
var result CanaryConfig
err := c.client.Get().
Resource("canaryconfigs").
Namespace(c.namespace).
Name(name).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (c *canaryConfigClient) Update(f *CanaryConfig) (*CanaryConfig, error) {
var result CanaryConfig
err := c.client.Put().
Resource("canaryconfigs").
Namespace(c.namespace).
Name(f.Metadata.Name).
Body(f).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (c *canaryConfigClient) Delete(name string, opts *metav1.DeleteOptions) error {
return c.client.Delete().
Namespace(c.namespace).
Resource("canaryconfigs").
Name(name).
Body(opts).
Do().
Error()
}
func (c *canaryConfigClient) List(opts metav1.ListOptions) (*CanaryConfigList, error) {
var result CanaryConfigList
err := c.client.Get().
Namespace(c.namespace).
Resource("canaryconfigs").
VersionedParams(&opts, scheme.ParameterCodec).
Do().
Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (c *canaryConfigClient) Watch(opts metav1.ListOptions) (watch.Interface, error) {
return c.client.Get().
Prefix("watch").
Namespace(c.namespace).
Resource("canaryconfigs").
VersionedParams(&opts, scheme.ParameterCodec).
Watch()
}
+10
View File
@@ -160,6 +160,13 @@ func configureClient(config *rest.Config) {
&metav1.ListOptions{},
&metav1.DeleteOptions{},
)
scheme.AddKnownTypes(
groupversion,
&CanaryConfig{},
&CanaryConfigList{},
&metav1.ListOptions{},
&metav1.DeleteOptions{},
)
return nil
})
schemeBuilder.AddToScheme(scheme.Scheme)
@@ -221,6 +228,9 @@ func (fc *FissionClient) Recorders(ns string) RecorderInterface {
func (fc *FissionClient) Packages(ns string) PackageInterface {
return MakePackageInterface(fc.crdClient, ns)
}
func (fc *FissionClient) CanaryConfigs(ns string) CanaryConfigInterface {
return MakeCanaryConfigInterface(fc.crdClient, ns)
}
func (fc *FissionClient) WaitForCRDs() error {
return waitForCRDs(fc.crdClient)
}
+20 -7
View File
@@ -38,17 +38,14 @@ func ensureCRD(clientset *apiextensionsclient.Clientset, crd *apiextensionsv1bet
maxRetries := 5
for i := 0; i < maxRetries; i++ {
_, err = clientset.ApiextensionsV1beta1().CustomResourceDefinitions().Get(crd.ObjectMeta.Name, metav1.GetOptions{})
_, err = clientset.ApiextensionsV1beta1().CustomResourceDefinitions().Create(crd)
if err == nil {
return nil
}
if errors.IsNotFound(err) {
// crd resource not found error
_, err = clientset.ApiextensionsV1beta1().CustomResourceDefinitions().Create(crd)
if err != nil {
return err
}
// return if the resource already exists
if errors.IsAlreadyExists(err) {
return nil
} else {
// The requests fail to connect to k8s api server before
// istio-prxoy is ready to serve traffic. Retry again.
@@ -192,6 +189,22 @@ func EnsureFissionCRDs(clientset *apiextensionsclient.Clientset) error {
},
},
},
// CanaryConfig: configuration for canary deployment of functions
{
ObjectMeta: metav1.ObjectMeta{
Name: "canaryconfigs.fission.io",
},
Spec: apiextensionsv1beta1.CustomResourceDefinitionSpec{
Group: crdGroupName,
Version: crdVersion,
Scope: apiextensionsv1beta1.NamespaceScoped,
Names: apiextensionsv1beta1.CustomResourceDefinitionNames{
Kind: "CanaryConfig",
Plural: "canaryconfigs",
Singular: "canaryconfig",
},
},
},
}
for _, crd := range crds {
err := ensureCRD(clientset, &crd)
+2
View File
@@ -37,4 +37,6 @@ type (
MessageQueueTriggerList = fv1.MessageQueueTriggerList
Recorder = fv1.Recorder
RecorderList = fv1.RecorderList
CanaryConfig = fv1.CanaryConfig
CanaryConfigList = fv1.CanaryConfigList
)
+4
View File
@@ -0,0 +1,4 @@
#!/bin/bash
kubectl delete canaryconfig canary-2
kubectl delete httptrigger route-fail
+23
View File
@@ -0,0 +1,23 @@
#!/bin/bash
# this script is useful to demo canary deployment when the latest function starts receiving 100% of the traffic
DEMO_RUN_FAST=1
ROOT_DIR=$(dirname $0)/..
. $ROOT_DIR/util.sh
desc "Function version-1"
run "fission function get --name fn1-v6"
desc "Function version-2"
run "fission function get --name fn1-v7"
desc "Create a route \(HTTP trigger\) the version-1 of the function with weight 100% and version-2 with weight 0%"
run "fission route create --name route-fail --method GET --url /fail --function fn1-v6 --weight 100 --function fn1-v7 --weight 0"
desc "Create a canary config to gradually increment the weight of version-2 by a step of 20 every 1 minute"
run "fission canary-config create --name canary-2 --newfunction fn1-v7 --oldfunction fn1-v6 --httptrigger route-fail --increment-step 30 --increment-interval 30s --failure-threshold 10"
desc "Fire requests to the route"
run "ab -n 10000 -c 1 http://$FISSION_ROUTER/fail"
+11
View File
@@ -0,0 +1,11 @@
#!/bin/bash
kubectl delete canaryconfig canary-1
kubectl delete httptrigger route-canary
fission fn delete --name func-v1
fission fn delete --name func-v2
fission package delete --orphan
# bug in canary config cache when you re-create a config with the same name, restart controller
kubectl -n fission get pod -l application=fission-api -o name | xargs -n1 kubectl -n fission delete
@@ -0,0 +1,7 @@
module.exports = async function(context) {
return {
status: 200,
body: "This is Version One!\n"
};
}
@@ -0,0 +1,7 @@
module.exports = async function(context) {
return {
status: 200,
body: "This is Version Two!\n"
};
}
+37
View File
@@ -0,0 +1,37 @@
#!/bin/bash
# this script is useful to demo canary deployment when the latest function starts receiving 100% of the traffic
DEMO_RUN_FAST=1
ROOT_DIR=$(dirname $0)/..
. $ROOT_DIR/util.sh
desc "Kubernetes cluster"
run "kubectl get nodes"
desc "Fission installed"
run "kubectl --namespace fission get deployment"
clear
desc "NodeJS environment pods"
run "kubectl --namespace fission-function get pod -l environmentName=nodejs"
# set up functions
fission fn create --name func-v1 --env nodejs --code func-v1.js
desc "Function version 1"
run "fission function get --name func-v1"
fission fn create --name func-v2 --env nodejs --code func-v2.js
desc "Function version 2"
run "fission function get --name func-v2"
desc "Create a route \(HTTP trigger\) the version-1 of the function with weight 100% and version-2 with weight 0%"
run "fission route create --name route-canary --method GET --url /canary --function func-v2 --weight 0 --function func-v1 --weight 100"
desc "Start sending requests to the route"
run_bg "hey -n 100000 -c 1 http://$FISSION_ROUTER/canary"
desc "Create a canary config: with an increment of 10 percent, every 1 minute, rolling back if 10% of requests fail"
run "fission canary-config create --name canary-1 --newfunction func-v2 --oldfunction func-v1 --httptrigger route-canary --increment-step 10 --increment-interval 30s --failure-threshold 10"
+4
View File
@@ -0,0 +1,4 @@
#!/bin/bash
fission fn delete --name hello
fission route delete --name $(fission route list|grep hello|cut -f1 -d' ')
+7
View File
@@ -0,0 +1,7 @@
module.exports = async function(context) {
return {
status: 200,
body: "This is Version One!\n"
};
}
+7
View File
@@ -0,0 +1,7 @@
module.exports = async function(context) {
return {
status: 200,
body: "This is Version Two!\n"
};
}
+7
View File
@@ -0,0 +1,7 @@
module.exports = async function(context) {
return {
status: 200,
body: "Hello, world!\n"
};
}
+113
View File
@@ -0,0 +1,113 @@
#!/bin/bash
DEMO_RUN_FAST=1
ROOT_DIR=$(dirname $0)/..
. $ROOT_DIR/util.sh
#
# General setup intro
#
desc "Our Kubernetes cluster"
run "kubectl get nodes"
desc "Fission is installed"
run "kubectl --namespace fission get deployment"
clear
#
# Hello world, environments, cold starts
#
desc "Hello world function"
run "cat hello.js"
desc "Add the NodeJS environment to fission"
run "fission env create --name nodejs --image fission/node-env"
desc "NodeJS environment pods"
run "kubectl --namespace fission-function get pod -l environmentName=nodejs"
desc "Upload our function to fission"
run "fission function create --name hello --env nodejs --code hello.js"
desc "Set up a route (aka HTTP Trigger) for the function"
run "fission route create --method GET --url /hello --function hello"
sleep 2
desc "Finally, run the function"
run "time -p curl http://$FISSION_ROUTER/hello"
desc "Run the function again"
run "time -p curl http://$FISSION_ROUTER/hello"
run "time -p curl http://$FISSION_ROUTER/hello"
desc "The pod is now labeled by the function name"
run "kubectl --namespace fission-function get pod -l functionName=hello"
#
# Declarative Specs.
#
# Use go for this one to show off the fact that we can do builds.
#
clear
pushd declarative
desc "Declaratively specified app"
run "ls"
desc "Declaratively specified app"
run "ls specs"
desc "Validate configs"
run "fission spec validate"
desc "Deploy application, wait for build to finish"
run "fission spec apply --wait"
desc "Invoke our new function"
run "fission function test --name hello-go"
#
# Live Reload (still in the same app)
#
clear
popd
#
# Record replay
#
clear
#
# Canary Deployments
#
clear
# set up functions
fission fn create --name func-v1 --env nodejs --code func-v1.js
desc "Function version 1"
run "fission function get --name func-v1"
fission fn create --name func-v2 --env nodejs --code func-v2.js
desc "Function version 2"
run "fission function get --name func-v2"
desc "Create a route \(HTTP trigger\) the version-1 of the function with weight 100% and version-2 with weight 0%"
run "fission route create --name route-canary --method GET --url /canary --function func-v2 --weight 0 --function func-v1 --weight 100"
desc "Start sending requests to the route"
run_bg "hey -n 100000 -c 1 http://$FISSION_ROUTER/canary"
desc "Create a canary config: with an increment of 10 percent, every 1 minute, rolling back if 10% of requests fail"
run "fission canary-config create --name canary-1 --newfunction func-v2 --oldfunction func-v1 --httptrigger route-canary --increment-step 10 --increment-interval 30s --failure-threshold 10"
+12
View File
@@ -0,0 +1,12 @@
#!/bin/bash
DEMO_RUN_FAST=1
ROOT_DIR=$(dirname $0)/..
. $ROOT_DIR/util.sh
desc "one"
run "sleep 30"
desc "two"
run "echo hi"
+9
View File
@@ -0,0 +1,9 @@
#!/bin/bash
fission spec destroy
# this one is generated in run.sh
rm specs/function-hello-go.yaml
rm *.~?~
+12
View File
@@ -0,0 +1,12 @@
package main
import (
"net/http"
)
// Handler is the entry point for this fission function
func Handler(w http.ResponseWriter, r *http.Request) {
msg := "Hello, CNCF Webinar!\n"
w.Write([]byte(msg))
}
+24
View File
@@ -0,0 +1,24 @@
#!/bin/bash
DEMO_RUN_FAST=1
ROOT_DIR=$(dirname $0)/..
. $ROOT_DIR/util.sh
desc "Declaratively specified app"
run "ls specs"
desc "Generate initial YAML, so we don't have to write it by hand"
run "fission function create --spec --name hello-go --env go --src hello.go --entrypoint Handler"
desc "Generated function YAML"
run "tail -25 specs/function-hello-go.yaml"
desc "Deploy on cluster, and wait for build result"
run "fission spec apply --wait"
desc "Invoke the function"
run "fission function test --name hello-go"
desc "Live-reload: auto build + deploy on save"
run "fission spec apply --watch"
+42
View File
@@ -0,0 +1,42 @@
Fission Specs
=============
This is a set of specifications for a Fission app. This includes functions,
environments, and triggers; we collectively call these things "resources".
How to use these specs
----------------------
These specs are handled with the 'fission spec' command. See 'fission spec --help'.
'fission spec apply' will "apply" all resources specified in this directory to your
cluster. That means it checks what resources exist on your cluster, what resources are
specified in the specs directory, and reconciles the difference by creating, updating or
deleting resources on the cluster.
'fission spec apply' will also package up your source code (or compiled binaries) and
upload the archives to the cluster if needed. It uses 'ArchiveUploadSpec' resources in
this directory to figure out which files to archive.
You can use 'fission spec apply --watch' to watch for file changes and continuously keep
the cluster updated.
You can add YAMLs to this directory by writing them manually, but it's easier to generate
them. Use 'fission function create --spec' to generate a function spec,
'fission environment create --spec' to generate an environment spec, and so on.
You can edit any of the files in this directory, except 'fission-deployment-config.yaml',
which contains a UID that you should never change. To apply your changes simply use
'fission spec apply'.
fission-deployment-config.yaml
------------------------------
fission-deployment-config.yaml contains a UID. This UID is what fission uses to correlate
resources on the cluster to resources in this directory.
All resources created by 'fission spec apply' are annotated with this UID. Resources on
the cluster that are _not_ annotated with this UID are never modified or deleted by
fission.
@@ -0,0 +1,7 @@
# This file is generated by the 'fission spec init' command.
# See the README in this directory for background and usage information.
# Do not edit the UID below: that will break 'fission spec apply'
apiVersion: fission.io/v1
kind: DeploymentConfig
name: declarative-specs
uid: e20fe7b4-b012-4ae0-98bd-0f968d21d397
@@ -0,0 +1,51 @@
include:
- hello.go
kind: ArchiveUploadSpec
name: hello-go
---
apiVersion: fission.io/v1
kind: Package
metadata:
creationTimestamp: null
name: hello-go-xfxy
namespace: default
spec:
deployment:
checksum: {}
environment:
name: go
namespace: default
source:
checksum: {}
type: url
url: archive://hello-go
status:
buildstatus: pending
---
apiVersion: fission.io/v1
kind: Function
metadata:
creationTimestamp: null
name: hello-go
namespace: default
spec:
InvokeStrategy:
ExecutionStrategy:
ExecutorType: poolmgr
MaxScale: 1
MinScale: 0
TargetCPUPercent: 80
StrategyType: execution
configmaps: null
environment:
name: go
namespace: default
package:
functionName: Handler
packageref:
name: hello-go-xfxy
namespace: default
resources: {}
secrets: null
+6
View File
@@ -0,0 +1,6 @@
#!/bin/bash
fission fn delete --name hi-py
fission route delete --name $(fission route list|grep hi|cut -f1 -d' ')
fission recorder delete --name my-recorder
kubectl -n fission delete pod redis-0
+7
View File
@@ -0,0 +1,7 @@
from flask import request
from flask import current_app
def main():
name=request.args["name"]
return "Hello, %s" % name
+28
View File
@@ -0,0 +1,28 @@
#!/bin/bash
DEMO_RUN_FAST=1
ROOT_DIR=$(dirname $0)/..
. $ROOT_DIR/util.sh
desc "A simple function that takes inputs"
run "cat hi.py"
desc "Set up function"
run "fission function create --name hi-py --env python --code hi.py --entrypoint hi.main"
desc "Set up route"
run "fission route create --function hi-py --url /hi --method GET"
desc "Set up recorder"
run "fission recorder create --name my-recorder --function hi-py"
desc "Run function"
run "curl http://$FISSION_ROUTER/hi?name=world"
desc "View recording"
run "fission records view -v"
requid=$(fission records view -v|tail -1|cut -f1 -d' ')
desc "Replay recording"
run "fission replay --reqUID $requid"
+18
View File
@@ -54,5 +54,23 @@ function run() {
echo -e "\b "
}
function run_bg() {
maybe_first_prompt
rate=25
if [ -n "$DEMO_RUN_FAST" ]; then
rate=1000
fi
echo "$green$1$reset" | pv -qL $rate
if [ -n "$DEMO_RUN_FAST" ]; then
sleep 0.5
fi
$1 &
echo
echo -n ">"
read -s
echo -e "\b "
}
SSH_NODE=$(kubectl get nodes | tail -1 | cut -f1 -d' ')
+1 -1
View File
@@ -114,7 +114,7 @@ func specializePod(f *fetcher.Fetcher, fetchPayload *string, loadPayload *string
var specializeURL string
var reader *bytes.Reader
if envVersion == 2 {
if envVersion >= 2 {
contentType = "application/json"
specializeURL = "http://localhost:8888/v2/specialize"
reader = bytes.NewReader([]byte(*loadPayload))
+9
View File
@@ -105,10 +105,19 @@ func downloadUrl(url string, localPath string) error {
if err != nil {
return err
}
defer w.Close()
_, err = io.Copy(w, resp.Body)
if err != nil {
return err
}
// flushing write buffer to file
err = w.Sync()
if err != nil {
return err
}
err = os.Chmod(localPath, 0600)
if err != nil {
return err
+1 -1
View File
@@ -6,5 +6,5 @@ RUN mvn -f /usr/src/myapp/pom.xml clean package
FROM openjdk:8-jdk-alpine
VOLUME /tmp
COPY --from=BUILD /usr/src/myapp/target/env-java-0.0.1-SNAPSHOT.jar /app.jar
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar","--server.port=8888"]
ENTRYPOINT java ${JVM_OPTS} -Djava.security.egd=file:/dev/./urandom -jar /app.jar --server.port=8888
EXPOSE 8888
+3 -3
View File
@@ -2,7 +2,7 @@
ARG BUILDER_IMAGE=fission/builder:latest
FROM ${BUILDER_IMAGE}
## Section copied from the OpenJDK 8 Dockerfile
## Section copied from the openjdk:8-jdk-alpine Dockerfile - (https://github.com/docker-library/openjdk/blob/47a6539cd18023dafb45db9013455136cc0bca07/8/jdk/alpine/Dockerfile)
ENV LANG C.UTF-8
RUN { \
@@ -15,8 +15,8 @@ RUN { \
ENV JAVA_HOME /usr/lib/jvm/java-1.8-openjdk
ENV PATH $PATH:/usr/lib/jvm/java-1.8-openjdk/jre/bin:/usr/lib/jvm/java-1.8-openjdk/bin
ENV JAVA_VERSION 8u171
ENV JAVA_ALPINE_VERSION 8.171.11-r0
ENV JAVA_VERSION 8u181
ENV JAVA_ALPINE_VERSION 8.181.13-r0
RUN set -x \
&& apk add --no-cache \
+11
View File
@@ -0,0 +1,11 @@
ARG BUILDER_IMAGE=fission/builder
FROM ${BUILDER_IMAGE}
# default variant is the official alpine node image (much smaller than the standard image)
FROM node:8-alpine
ARG NODE_ENV
ENV NODE_ENV $NODE_ENV
COPY --from=0 /builder /builder
ADD build.sh /usr/local/bin/build
RUN chmod +x /usr/local/bin/build
+3
View File
@@ -0,0 +1,3 @@
#!/bin/sh
cd ${SRC_PKG}
npm install && cp -r ${SRC_PKG} ${DEPLOY_PKG}
+6 -2
View File
@@ -6,6 +6,10 @@
{
"name": "Soam Vasani",
"email": "soamvasani@platform9.com"
},
{
"name": "Gary Yeap",
"email": "contact@garyyeap.com"
}
],
"description": "NodeJS run container for the fission framework",
@@ -20,7 +24,7 @@
"morgan": "*",
"mz": "~2.7.0",
"request": "^2.81.0",
"request-promise-native": "^1.0.3",
"underscore": ">=1.8.3"
"underscore": ">=1.8.3",
"request-promise-native": "^1.0.3"
}
}
+68 -37
View File
@@ -7,56 +7,86 @@ const express = require('express');
const app = express();
const bodyParser = require('body-parser');
const morgan = require('morgan');
const argv = require('minimist')(process.argv.slice(1));// Command line opts
// Command line opts
const argv = require('minimist')(process.argv.slice(1));
if (!argv.codepath) {
argv.codepath = "/userfunc/user";
console.log("Codepath defaulting to ", argv.codepath);
}
if (!argv.port) {
console.log("Port defaulting to 8888");
argv.port = 8888;
}
// Node resolves module paths according to a file's location. We load
// the file from argv.codepath, but tell users to put dependencies in
// the server's package.json; this means the function's dependencies
// are in /usr/src/app/node_modules. We could be smarter and have the
// function deps in the right place in argv.codepath; but for now we
// just symlink the function's node_modules to the server's
// node_modules.
fs.symlinkSync('/usr/src/app/node_modules', `${path.dirname(argv.codepath)}/node_modules`);
// User function. Starts out undefined.
let userFunction;
//
// Specialize this server to a given user function. The user function
// is read from argv.codepath; it's expected to be placed there by the
// fission runtime.
//
function specialize(req, res) {
// Make sure we're a generic container. (No reuse of containers.
// Once specialized, the container remains specialized.)
if (userFunction) {
res.status(400).send("Not a generic container");
return;
}
function loadFunction(modulepath, funcname) {
// Read and load the code. It's placed there securely by the fission runtime.
try {
var startTime = process.hrtime();
userFunction = require(argv.codepath);
var elapsed = process.hrtime(startTime);
let startTime = process.hrtime();
// support v1 codepath and v2 entrypoint like 'foo', '', 'index.hello'
let userFunction = funcname ? require(modulepath)[funcname] : require(modulepath);
let elapsed = process.hrtime(startTime);
console.log(`user code loaded in ${elapsed[0]}sec ${elapsed[1]/1000000}ms`);
return userFunction;
} catch(e) {
console.error(`user code load error: ${e}`);
res.status(500).send(JSON.stringify(e));
return;
return e;
}
}
function withEnsureGeneric(func) {
return function(req, res) {
// Make sure we're a generic container. (No reuse of containers.
// Once specialized, the container remains specialized.)
if (userFunction) {
res.status(400).send("Not a generic container");
return;
}
func(req, res);
}
}
function isFunction(func) {
return func && func.constructor && func.call && func.apply;
}
function specializeV2(req, res) {
// for V2 entrypoint, 'filename.funcname' => ['filename', 'funcname']
const entrypoint = req.body.functionName ? req.body.functionName.split('.') : [];
// for V2, filepath is dynamic path
const modulepath = path.join(req.body.filepath, entrypoint[0] || '');
const result = loadFunction(modulepath, entrypoint[1]);
if(isFunction(result)){
userFunction = result;
res.status(202).send();
} else {
res.status(500).send(JSON.stringify(result));
}
}
function specialize(req, res) {
// Specialize this server to a given user function. The user function
// is read from argv.codepath; it's expected to be placed there by the
// fission runtime.
//
const modulepath = argv.codepath || '/userfunc/user';
// Node resolves module paths according to a file's location. We load
// the file from argv.codepath, but tell users to put dependencies in
// the server's package.json; this means the function's dependencies
// are in /usr/src/app/node_modules. We could be smarter and have the
// function deps in the right place in argv.codepath; b ut for now we
// just symlink the function's node_modules to the server's
// node_modules.
fs.symlinkSync('/usr/src/app/node_modules', `${path.dirname(modulepath)}/node_modules`);
const result = loadFunction(modulepath);
if(isFunction(result)){
userFunction = result;
res.status(202).send();
} else {
res.status(500).send(JSON.stringify(result));
}
res.status(202).send();
}
@@ -68,7 +98,8 @@ app.use(bodyParser.json());
app.use(bodyParser.raw());
app.use(bodyParser.text({ type : "text/*" }));
app.post('/specialize', specialize);
app.post('/specialize', withEnsureGeneric(specialize));
app.post('/v2/specialize', withEnsureGeneric(specializeV2));
// Generic route -- all http requests go to the user function.
app.all('/', function (req, res) {
+2 -2
View File
@@ -1,8 +1,8 @@
Flask===0.11.1
flask ~= 0.12.3
bjoern
httplib2
python-dateutil
requests==2.7.0
requests==2.20.0
redis
hiredis
gevent
+1 -1
View File
@@ -1,7 +1,7 @@
GEM
remote: https://rubygems.org/
specs:
rack (2.0.3)
rack (2.0.6)
PLATFORMS
ruby
+42
View File
@@ -0,0 +1,42 @@
Fission Specs
=============
This is a set of specifications for a Fission app. This includes functions,
environments, and triggers; we collectively call these things "resources".
How to use these specs
----------------------
These specs are handled with the 'fission spec' command. See 'fission spec --help'.
'fission spec apply' will "apply" all resources specified in this directory to your
cluster. That means it checks what resources exist on your cluster, what resources are
specified in the specs directory, and reconciles the difference by creating, updating or
deleting resources on the cluster.
'fission spec apply' will also package up your source code (or compiled binaries) and
upload the archives to the cluster if needed. It uses 'ArchiveUploadSpec' resources in
this directory to figure out which files to archive.
You can use 'fission spec apply --watch' to watch for file changes and continuously keep
the cluster updated.
You can add YAMLs to this directory by writing them manually, but it's easier to generate
them. Use 'fission function create --spec' to generate a function spec,
'fission environment create --spec' to generate an environment spec, and so on.
You can edit any of the files in this directory, except 'fission-deployment-config.yaml',
which contains a UID that you should never change. To apply your changes simply use
'fission spec apply'.
fission-deployment-config.yaml
------------------------------
fission-deployment-config.yaml contains a UID. This UID is what fission uses to correlate
resources on the cluster to resources in this directory.
All resources created by 'fission spec apply' are annotated with this UID. Resources on
the cluster that are _not_ annotated with this UID are never modified or deleted by
fission.
@@ -0,0 +1,7 @@
# This file is generated by the 'fission spec init' command.
# See the README in this directory for background and usage information.
# Do not edit the UID below: that will break 'fission spec apply'
apiVersion: fission.io/v1
kind: DeploymentConfig
name: java
uid: 3305cef5-21e6-4a55-b977-5c76ee62b0a6
@@ -0,0 +1,52 @@
include:
- src/
- pom.xml
kind: ArchiveUploadSpec
name: helloworld-java
---
apiVersion: fission.io/v1
kind: Package
metadata:
creationTimestamp: null
name: helloworld-java-127w
namespace: default
spec:
deployment:
checksum: {}
environment:
name: java
namespace: default
source:
checksum: {}
type: url
url: archive://helloworld-java
status:
buildstatus: pending
---
apiVersion: fission.io/v1
kind: Function
metadata:
creationTimestamp: null
name: hello-java
namespace: default
spec:
InvokeStrategy:
ExecutionStrategy:
ExecutorType: poolmgr
MaxScale: 1
MinScale: 0
TargetCPUPercent: 80
StrategyType: execution
configmaps: null
environment:
name: java
namespace: default
package:
functionName: io.fission.HelloWorld
packageref:
name: helloworld-java-127w
namespace: default
resources: {}
secrets: null
+51 -44
View File
@@ -1,19 +1,19 @@
# Fission Node.js Examples
This is V2 example, check [here](README_V1.md) for V1.
This directory contains several examples to get you started using Node.js with Fission.
## Environment
Before running any of these functions, make sure you have created a `nodejs` Fission environment:
```bash
# Create an environment with default nodejs images
$ fission env create --name nodeenv --image fission/node-env:latest --builder fission/node-builder:latest
# Create zip file from our example
$ zip -jr nodejs.zip nodejs/
# Create a package with the zip file
$ fission pkg create --sourcearchive nodejs.zip --env nodeenv
```
$ fission env create --name nodejs --image fission/node-env
```
Note: The default `fission/node-env` image is based on Alpine, which is much smaller than the main Debian Node image (65MB vs 680MB) while still being suitable for most use cases.
If you need to use the full Debian image use the `fission/node-env-debian` image instead.
See the [official Node docker hub repo](https://hub.docker.com/_/node/) for considerations
relating to this choice.
## Function signature
@@ -27,28 +27,54 @@ module.exports = async function(context) {
headers: {
'Foo': 'Bar'
}
}
}
}
```
Since it is an `async` function, you can `await` `Promise`s, as demonstrated in the `stock.js` function.
## hello.js
This is a basic "Hello, World!" example. It simply returns a status of `200` and text body.
### Usage
Since it is an `async` function, you can `await` `Promise`s, as demonstrated in the `weather.js` function.
```bash
# Upload your function code to fission
$ fission function create --name hello --env nodejs --code hello.js
# Create a function
$ fission fn create --name hello --pkg [pkgname] --entrypoint "hello"
# Map GET /hello to your new function
$ fission route create --method GET --url /hello --function hello
# Test the function
$ fission fn test --name hello
```
# Run the function.
$ curl http://$FISSION_ROUTER/hello
Hello, world!
## index.js
This file does nothing but for demonstrating `require` feature.
### Usage
```bash
# Create a function, you can skip `--entrypoint` as node will look for `index.js` by default
$ fission fn create --name index --pkg [pkgname]
# Test the function
$ fission fn test --name index
```
## multi-entry.js
This is a multiple exports example. There are two exports: entry1 and entry2
### Usage
```bash
# Create a function for entry1
$ fission fn create --name entry1 --pkg [pkgname] --entrypoint "multi-entry.entry1"
# Test the function
$ fission fn test --name entry1
# Create a function for entry2
$ fission fn create --name entry2 --pkg [pkgname] --entrypoint "multi-entry.entry2"
# Test the function
$ fission fn test --name entry2
```
## hello-callback.js
@@ -60,8 +86,8 @@ This is a basic "Hello, World!" example implemented with the legacy callback imp
### Usage
```bash
# Upload your function code to fission
$ fission function create --name hello-callback --env nodejs --code hello-callback.js
# Create a function
$ fission fn create --name hello-callback --pkg [pkgname] --entrypoint "hello-callback"
# Map GET /hello-callback to your new function
$ fission route create --method GET --url /hello-callback --function hello-callback
@@ -71,25 +97,6 @@ $ curl http://$FISSION_ROUTER/hello-callback
Hello, world!
```
## stock.js
This is a basic example of how you can easily use asynchronous requests in your functions. By default, the Node.js environment makes the [`request-promise-native`](https://github.com/request/request-promise-native) library available. In this example, the Google Finance API is used to determine when a stock was last traded.
### Usage
```bash
# Upload your function code to fission
$ fission function create --name stock --env nodejs --code stock.js
# Map GET /stock to your new function
$ fission route create --method POST --url /stock --function stock
# Run the function.
$ curl -H "Content-Type: application/json" -X POST -d '{"symbol":"AAPL"}' http://$FISSION_ROUTER/stock
{"text":"AAPL last traded at 138.99"}
```
## kubeEventsSlack.js
This example watches Kubernetes events and sends them to a Slack channel. To use this, create an incoming webhook for your Slack channel, and replace the `slackWebhookPath` in the example code.
@@ -98,7 +105,7 @@ This example watches Kubernetes events and sends them to a Slack channel. To use
```bash
# Upload your function code to fission
$ fission fn create --name kubeEventsSlack --env nodejs --code kubeEventsSlack.js
$ fission fn create --name kubeEventsSlack --pkg [pkgname] --entrypoint "hello-callback"
# Watch all services in the default namespace:
$ fission watch create --function kubeEventsSlack --type service --ns default
@@ -112,7 +119,7 @@ In this example, the Yahoo Weather API is used to current weather at a given loc
```bash
# Upload your function code to fission
$ fission function create --name weather --env nodejs --code weather.js
$ fission function create --name weather --pkg [pkgname] --entrypoint "weather"
# Map GET /stock to your new function
$ fission route create --method POST --url /weather --function weather
@@ -121,4 +128,4 @@ $ fission route create --method POST --url /weather --function weather
$ curl -H "Content-Type: application/json" -X POST -d '{"location":"Sieteiglesias, Spain"}' http://$FISSION_ROUTER/weather
{"text":"It is 2 celsius degrees in Sieteiglesias, Spain and Mostly Clear"}
```
```
+105
View File
@@ -0,0 +1,105 @@
# Fission Node.js Examples
This directory contains several examples to get you started using Node.js with Fission.
## Environment
Before running any of these functions, make sure you have created a `nodejs` Fission environment:
```
$ fission env create --name nodejs --image fission/node-env
```
Note: The default `fission/node-env` image is based on Alpine, which is much smaller than the main Debian Node image (65MB vs 680MB) while still being suitable for most use cases.
If you need to use the full Debian image use the `fission/node-env-debian` image instead.
See the [official Node docker hub repo](https://hub.docker.com/_/node/) for considerations
relating to this choice.
## Function signature
Every Node.js function has the same basic form:
```javascript
module.exports = async function(context) {
return {
status: 200,
body: 'Your body here',
headers: {
'Foo': 'Bar'
}
}
}
```
Since it is an `async` function, you can `await` `Promise`s, as demonstrated in the `weather.js` function.
## hello.js
This is a basic "Hello, World!" example. It simply returns a status of `200` and text body.
### Usage
```bash
# Upload your function code to fission
$ fission function create --name hello --env nodejs --code hello.js
# Map GET /hello to your new function
$ fission route create --method GET --url /hello --function hello
# Run the function.
$ curl http://$FISSION_ROUTER/hello
Hello, world!
```
## hello-callback.js
This is a basic "Hello, World!" example implemented with the legacy callback implementation. If you declare your function with two arguments (`context`, `callback`), a callback taking three arguments (`status`, `body`, `headers`) is provided.
⚠️ Callback support is only provided for backwards compatibility! We recommend that you use `async` functions instead.
### Usage
```bash
# Upload your function code to fission
$ fission function create --name hello-callback --env nodejs --code hello-callback.js
# Map GET /hello-callback to your new function
$ fission route create --method GET --url /hello-callback --function hello-callback
# Run the function.
$ curl http://$FISSION_ROUTER/hello-callback
Hello, world!
```
## kubeEventsSlack.js
This example watches Kubernetes events and sends them to a Slack channel. To use this, create an incoming webhook for your Slack channel, and replace the `slackWebhookPath` in the example code.
### Usage
```bash
# Upload your function code to fission
$ fission fn create --name kubeEventsSlack --env nodejs --code kubeEventsSlack.js
# Watch all services in the default namespace:
$ fission watch create --function kubeEventsSlack --type service --ns default
```
## weather.js
In this example, the Yahoo Weather API is used to current weather at a given location.
### Usage
```bash
# Upload your function code to fission
$ fission function create --name weather --env nodejs --code weather.js
# Map GET /stock to your new function
$ fission route create --method POST --url /weather --function weather
# Run the function.
$ curl -H "Content-Type: application/json" -X POST -d '{"location":"Sieteiglesias, Spain"}' http://$FISSION_ROUTER/weather
{"text":"It is 2 celsius degrees in Sieteiglesias, Spain and Mostly Clear"}
```
+1 -1
View File
@@ -1,4 +1,4 @@
module.exports = function(context, callback) {
callback(200, "Hello, world!\n");
callback(200, "Hello, world callback!\n");
}
+1 -1
View File
@@ -2,6 +2,6 @@
module.exports = async function(context) {
return {
status: 200,
body: "Hello, world!\n"
body: "hello, world!\n"
};
}
+1
View File
@@ -0,0 +1 @@
module.exports = require('./hello');
+13
View File
@@ -0,0 +1,13 @@
module.exports.entry1 = async function(context) {
return {
status: 200,
body: "Hello, entry 1!\n"
};
}
module.exports.entry2 = async function(context) {
return {
status: 200,
body: "Hello, entry 2!\n"
};
}
+30
View File
@@ -0,0 +1,30 @@
{
"name": "fission-nodejs-example",
"version": "0.1.0",
"author": "Soam Vasani",
"contributors": [
{
"name": "Soam Vasani",
"email": "soamvasani@platform9.com"
},
{
"name": "Gary Yeap",
"email": "contact@garyyeap.com"
}
],
"description": "Nodejs example for Fission framework",
"engines": {
"node": ">=7.6.0"
},
"dependencies": {
"body-parser": "*",
"co": "~4.6.0",
"express": "*",
"minimist": "*",
"morgan": "*",
"mz": "~2.7.0",
"request": "^2.81.0",
"request-promise-native": "^1.0.3",
"underscore": ">=1.8.3"
}
}
-41
View File
@@ -1,41 +0,0 @@
'use strict';
const rp = require('request-promise-native');
module.exports = async function (context) {
const body = context.request.body;
console.log(`body = ${body}`);
const symbol = body['text'].split(' ')[1];
console.log(`Got symbol: ${symbol}`);
if (!symbol) {
return {
status: 400,
body: {
text: 'You must provide a stock symbol.'
}
};
}
try {
const response = await rp(`http://finance.google.com/finance/info?q=NYSE:${symbol}`);
const parsed = JSON.parse(response.slice(3));
const lastTrade = parsed[0]['l_cur'];
return {
status: 200,
body: {
text: `${symbol} last traded at ${lastTrade}`
},
headers: {
'Content-Type': 'application/json'
}
};
} catch (e) {
console.error(e);
return {
status: 500,
body: e
};
}
}
+1
View File
@@ -67,6 +67,7 @@ func (c *Client) GetServiceForFunction(metadata *metav1.ObjectMeta) (string, err
svcName, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Printf("Returning from ioutil read body")
return "", err
}
+8 -8
View File
@@ -142,13 +142,6 @@ func (executor *Executor) getFunctionExecutorType(meta *metav1.ObjectMeta) (fiss
func (executor *Executor) createServiceForFunction(meta *metav1.ObjectMeta) (*fscache.FuncSvc, error) {
log.Printf("[%v] No cached function service found, creating one", meta.Name)
// from Func -> get Env
log.Printf("[%v] getting environment for function", meta.Name)
env, err := executor.getFunctionEnv(meta)
if err != nil {
return nil, err
}
executorType, err := executor.getFunctionExecutorType(meta)
if err != nil {
return nil, err
@@ -161,6 +154,13 @@ func (executor *Executor) createServiceForFunction(meta *metav1.ObjectMeta) (*fs
case fission.ExecutorTypeNewdeploy:
fsvc, fsvcErr = executor.ndm.GetFuncSvc(meta)
default:
// from Func -> get Env
log.Printf("[%v] getting environment for function", meta.Name)
env, err := executor.getFunctionEnv(meta)
if err != nil {
return nil, err
}
pool, err := executor.gpm.GetPool(env)
if err != nil {
return nil, err
@@ -250,7 +250,7 @@ func StartExecutor(fissionNamespace string, functionNamespace string, envBuilder
fsCache := fscache.MakeFunctionServiceCache()
poolID := strings.ToLower(uniuri.NewLen(8))
reaper.CleanupOldExecutorObjects(kubernetesClient, functionNamespace, poolID)
reaper.CleanupOldExecutorObjects(kubernetesClient, poolID)
go reaper.CleanupRoleBindings(kubernetesClient, fissionClient, functionNamespace, envBuilderNamespace, time.Minute*30)
gpm := poolmgr.MakeGenericPoolManager(
+36 -26
View File
@@ -49,24 +49,31 @@ const (
)
func (deploy *NewDeploy) createOrGetDeployment(fn *crd.Function, env *crd.Environment,
deployName string, deployLabels map[string]string, deployNamespace string) (*v1beta1.Deployment, error) {
deployName string, deployLabels map[string]string, deployNamespace string, firstcreate bool) (*v1beta1.Deployment, error) {
minScale := int32(fn.Spec.InvokeStrategy.ExecutionStrategy.MinScale)
if minScale == 0 {
// If it's not the first time creation and minscale is 0 means that all pods for function were recycled,
// in such cases we need set minscale to 1 for router to serve requests.
if !firstcreate && minScale <= 0 {
minScale = 1
}
waitForDeploy := minScale > 0
existingDepl, err := deploy.kubernetesClient.ExtensionsV1beta1().Deployments(deployNamespace).Get(deployName, metav1.GetOptions{})
if err == nil {
err = scaleDeployment(deploy.kubernetesClient,
existingDepl.Namespace, existingDepl.Name, minScale)
if err != nil {
log.Printf("Error scaling up deployment for function %v: %v", fn.Metadata.Name, err)
return nil, err
}
if waitForDeploy {
err = scaleDeployment(deploy.kubernetesClient,
existingDepl.Namespace, existingDepl.Name, minScale)
if err != nil {
log.Printf("Error scaling up deployment for function %v: %v", fn.Metadata.Name, err)
return nil, err
}
if existingDepl.Status.AvailableReplicas < minScale {
existingDepl, err = deploy.waitForDeploy(existingDepl, minScale)
if existingDepl.Status.AvailableReplicas < minScale {
existingDepl, err = deploy.waitForDeploy(existingDepl, minScale)
}
}
return existingDepl, err
}
@@ -88,7 +95,11 @@ func (deploy *NewDeploy) createOrGetDeployment(fn *crd.Function, env *crd.Enviro
return nil, err
}
return deploy.waitForDeploy(depl, minScale)
if waitForDeploy {
depl, err = deploy.waitForDeploy(depl, minScale)
}
return depl, err
}
return nil, err
@@ -148,9 +159,6 @@ func (deploy *NewDeploy) getDeploymentSpec(fn *crd.Function, env *crd.Environmen
deployName string, deployLabels map[string]string) (*v1beta1.Deployment, error) {
replicas := int32(fn.Spec.InvokeStrategy.ExecutionStrategy.MinScale)
if replicas == 0 {
replicas = 1
}
targetFilename := "user"
@@ -203,8 +211,8 @@ func (deploy *NewDeploy) getDeploymentSpec(fn *crd.Function, env *crd.Environmen
deployment := &v1beta1.Deployment{
ObjectMeta: metav1.ObjectMeta{
Labels: deployLabels,
Name: deployName,
Labels: deployLabels,
},
Spec: v1beta1.DeploymentSpec{
Replicas: &replicas,
@@ -361,23 +369,23 @@ func (deploy *NewDeploy) getResources(env *crd.Environment, fn *crd.Function) ap
resources.Limits = make(map[apiv1.ResourceName]resource.Quantity)
}
// Only override the once specified at function, rest default to values from env.
_, ok := fn.Spec.Resources.Requests[apiv1.ResourceCPU]
if ok {
val, ok := fn.Spec.Resources.Requests[apiv1.ResourceCPU]
if ok && !val.IsZero() {
resources.Requests[apiv1.ResourceCPU] = fn.Spec.Resources.Requests[apiv1.ResourceCPU]
}
_, ok = fn.Spec.Resources.Requests[apiv1.ResourceMemory]
if ok {
val, ok = fn.Spec.Resources.Requests[apiv1.ResourceMemory]
if ok && !val.IsZero() {
resources.Requests[apiv1.ResourceMemory] = fn.Spec.Resources.Requests[apiv1.ResourceMemory]
}
_, ok = fn.Spec.Resources.Limits[apiv1.ResourceCPU]
if ok {
val, ok = fn.Spec.Resources.Limits[apiv1.ResourceCPU]
if ok && !val.IsZero() {
resources.Limits[apiv1.ResourceCPU] = fn.Spec.Resources.Limits[apiv1.ResourceCPU]
}
_, ok = fn.Spec.Resources.Limits[apiv1.ResourceMemory]
if ok {
val, ok = fn.Spec.Resources.Limits[apiv1.ResourceMemory]
if ok && !val.IsZero() {
resources.Limits[apiv1.ResourceMemory] = fn.Spec.Resources.Limits[apiv1.ResourceMemory]
}
@@ -391,6 +399,9 @@ func (deploy *NewDeploy) createOrGetHpa(hpaName string, execStrategy *fission.Ex
minRepl = 1
}
maxRepl := int32(execStrategy.MaxScale)
if maxRepl == 0 {
maxRepl = minRepl
}
targetCPU := int32(execStrategy.TargetCPUPercent)
existingHpa, err := deploy.kubernetesClient.AutoscalingV1().HorizontalPodAutoscalers(depl.ObjectMeta.Namespace).Get(hpaName, metav1.GetOptions{})
@@ -405,9 +416,8 @@ func (deploy *NewDeploy) createOrGetHpa(hpaName string, execStrategy *fission.Ex
if err != nil && k8s_err.IsNotFound(err) {
hpa := asv1.HorizontalPodAutoscaler{
ObjectMeta: metav1.ObjectMeta{
Name: hpaName,
Namespace: depl.ObjectMeta.Namespace,
Labels: depl.Labels,
Name: hpaName,
Labels: depl.Labels,
},
Spec: asv1.HorizontalPodAutoscalerSpec{
ScaleTargetRef: asv1.CrossVersionObjectReference{
+22 -19
View File
@@ -72,6 +72,7 @@ type (
reqType requestType
fn *crd.Function
responseChannel chan *fnResponse
firstcreate bool
}
fnResponse struct {
@@ -156,7 +157,7 @@ func (deploy *NewDeploy) initFuncController() (k8sCache.Store, k8sCache.Controll
store, controller := k8sCache.NewInformer(listWatch, &crd.Function{}, resyncPeriod, k8sCache.ResourceEventHandlerFuncs{
AddFunc: func(obj interface{}) {
fn := obj.(*crd.Function)
deploy.createFunction(fn)
deploy.createFunction(fn, true)
},
DeleteFunc: func(obj interface{}) {
fn := obj.(*crd.Function)
@@ -176,7 +177,7 @@ func (deploy *NewDeploy) service() {
req := <-deploy.requestChannel
switch req.reqType {
case FnCreate:
fsvc, err := deploy.fnCreate(req.fn)
fsvc, err := deploy.fnCreate(req.fn, req.firstcreate)
req.responseChannel <- &fnResponse{
error: err,
fSvc: fsvc,
@@ -205,6 +206,7 @@ func (deploy *NewDeploy) GetFuncSvc(metadata *metav1.ObjectMeta) (*fscache.FuncS
fn: fn,
reqType: FnCreate,
responseChannel: c,
firstcreate: false,
}
resp := <-c
@@ -214,13 +216,11 @@ func (deploy *NewDeploy) GetFuncSvc(metadata *metav1.ObjectMeta) (*fscache.FuncS
return resp.fSvc, nil
}
func (deploy *NewDeploy) createFunction(fn *crd.Function) {
func (deploy *NewDeploy) createFunction(fn *crd.Function, firstcreate bool) {
if fn.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType != fission.ExecutorTypeNewdeploy {
return
}
if fn.Spec.InvokeStrategy.ExecutionStrategy.MinScale <= 0 {
return
}
// Eager creation of function if minScale is greater than 0
log.Printf("Eagerly creating newDeploy objects for function")
c := make(chan *fnResponse)
@@ -228,6 +228,7 @@ func (deploy *NewDeploy) createFunction(fn *crd.Function) {
fn: fn,
reqType: FnCreate,
responseChannel: c,
firstcreate: firstcreate,
}
resp := <-c
if resp.error != nil {
@@ -263,7 +264,7 @@ func (deploy *NewDeploy) deleteFunction(fn *crd.Function) {
}
}
func (deploy *NewDeploy) fnCreate(fn *crd.Function) (*fscache.FuncSvc, error) {
func (deploy *NewDeploy) fnCreate(fn *crd.Function, firstcreate bool) (*fscache.FuncSvc, error) {
fsvc, err := deploy.fsCache.GetByFunction(&fn.Metadata)
if err == nil {
return fsvc, err
@@ -298,8 +299,7 @@ func (deploy *NewDeploy) fnCreate(fn *crd.Function) (*fscache.FuncSvc, error) {
return fsvc, err
}
svcAddress := fmt.Sprintf("%v.%v", svc.Name, svc.Namespace)
depl, err := deploy.createOrGetDeployment(fn, env, objName, deployLabels, ns)
depl, err := deploy.createOrGetDeployment(fn, env, objName, deployLabels, ns, firstcreate)
if err != nil {
log.Printf("Error creating the deployment %v: %v", objName, err)
return fsvc, err
@@ -384,7 +384,7 @@ func (deploy *NewDeploy) fnUpdate(oldFn *crd.Function, newFn *crd.Function) {
if oldFn.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType != fission.ExecutorTypeNewdeploy &&
newFn.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType == fission.ExecutorTypeNewdeploy {
log.Printf("function type changed to new deployment, creating resources: %v", newFn)
_, err := deploy.fnCreate(newFn)
_, err := deploy.fnCreate(newFn, true)
if err != nil {
updateStatus(oldFn, err, "error changing the function's type to newdeploy")
}
@@ -409,7 +409,6 @@ func (deploy *NewDeploy) fnUpdate(oldFn *crd.Function, newFn *crd.Function) {
if newFn.Spec.InvokeStrategy.ExecutionStrategy.MinScale != oldFn.Spec.InvokeStrategy.ExecutionStrategy.MinScale {
replicas := int32(newFn.Spec.InvokeStrategy.ExecutionStrategy.MinScale)
hpa.Spec.MinReplicas = &replicas
deployChanged = true
hpaChanged = true
}
@@ -548,19 +547,23 @@ func (deploy *NewDeploy) fnDelete(fn *crd.Function) (*fscache.FuncSvc, error) {
}
func (deploy *NewDeploy) getObjName(fn *crd.Function) string {
return fmt.Sprintf("%v-%v",
fn.Metadata.Name,
deploy.instanceID)
// Use executor type as delimiter between function name and namespace to prevent deployment name conflict.
// For example:
// 1. fn-name: a-b fn-namespace: c => a-b-newdeploy-c
// 2. fn-name: a fn-namespace: b-c => a-newdeploy-b-c
return strings.ToLower(fmt.Sprintf("%v-newdeploy-%v", fn.Metadata.Name, fn.Metadata.Namespace))
}
func (deploy *NewDeploy) getDeployLabels(fn *crd.Function, env *crd.Environment) map[string]string {
return map[string]string{
"environmentName": env.Metadata.Name,
"environmentUid": string(env.Metadata.UID),
"functionName": fn.Metadata.Name,
"functionUid": string(fn.Metadata.UID),
fission.EXECUTOR_INSTANCEID_LABEL: deploy.instanceID,
"executorType": fission.ExecutorTypeNewdeploy,
fission.EXECUTOR_TYPE: fission.ExecutorTypeNewdeploy,
fission.ENVIRONMENT_NAME: env.Metadata.Name,
fission.ENVIRONMENT_NAMESPACE: env.Metadata.Namespace,
fission.ENVIRONMENT_UID: string(env.Metadata.UID),
fission.FUNCTION_NAME: fn.Metadata.Name,
fission.FUNCTION_NAMESPACE: fn.Metadata.Namespace,
fission.FUNCTION_UID: string(fn.Metadata.UID),
}
}
+23 -23
View File
@@ -47,8 +47,6 @@ import (
"github.com/fission/fission/executor/util"
)
const POD_PHASE_RUNNING string = "Running"
type (
GenericPool struct {
env *crd.Environment
@@ -159,12 +157,7 @@ func MakeGenericPool(
}
// Labels for generic deployment/RS/pods.
gp.labelsForPool = map[string]string{
"environmentName": gp.env.Metadata.Name,
"environmentUid": string(gp.env.Metadata.UID),
fission.EXECUTOR_INSTANCEID_LABEL: gp.instanceId,
"executorType": fission.ExecutorTypePoolmgr,
}
gp.labelsForPool = gp.getDeployLabels()
// create the pool
err = gp.createPool()
@@ -178,6 +171,16 @@ func MakeGenericPool(
return gp, nil
}
func (gp *GenericPool) getDeployLabels() map[string]string {
return map[string]string{
fission.EXECUTOR_INSTANCEID_LABEL: gp.instanceId,
fission.EXECUTOR_TYPE: fission.ExecutorTypePoolmgr,
fission.ENVIRONMENT_NAME: gp.env.Metadata.Name,
fission.ENVIRONMENT_NAMESPACE: gp.env.Metadata.Namespace,
fission.ENVIRONMENT_UID: string(gp.env.Metadata.UID),
}
}
// choosePodService serializes the choosing of pods
func (gp *GenericPool) choosePodService() {
for {
@@ -228,21 +231,13 @@ func (gp *GenericPool) _choosePod(newLabels map[string]string) (*apiv1.Pod, erro
for i := range podList.Items {
pod := podList.Items[i]
// If a pod has no IP it's not ready
if len(pod.Status.PodIP) == 0 || string(pod.Status.Phase) != POD_PHASE_RUNNING {
// Ignore not ready pod here
if !fission.IsReadyPod(&pod) {
continue
}
// Wait for all containers in the pod to be ready
podReady := true
for _, cs := range pod.Status.ContainerStatuses {
podReady = podReady && cs.Ready
}
// add it to the list of ready pods
if podReady {
readyPods = append(readyPods, &pod)
}
readyPods = append(readyPods, &pod)
}
log.Printf("[%v] found %v ready pods of %v total", newLabels, len(readyPods), len(podList.Items))
@@ -466,12 +461,17 @@ func (gp *GenericPool) specializePod(pod *apiv1.Pod, metadata *metav1.ObjectMeta
return nil
}
func (gp *GenericPool) getPoolName() string {
// Use executor type as delimiter between function name and namespace to prevent deployment name conflict.
// For example:
// 1. fn-name: a-b fn-namespace: c => a-b-poolmgr-c
// 2. fn-name: a fn-namespace: b-c => a-poolmgr-b-c
return strings.ToLower(fmt.Sprintf("%v-poolmgr-%v", gp.env.Metadata.Name, gp.env.Metadata.Namespace))
}
// A pool is a deployment of generic containers for an env. This
// creates the pool but doesn't wait for any pods to be ready.
func (gp *GenericPool) createPool() error {
poolDeploymentName := fmt.Sprintf("%v-%v-%v",
gp.env.Metadata.Name, gp.env.Metadata.UID, strings.ToLower(gp.poolInstanceId))
fetcherResources, err := util.GetFetcherResources()
if err != nil {
return err
@@ -494,7 +494,7 @@ func (gp *GenericPool) createPool() error {
deployment := &v1beta1.Deployment{
ObjectMeta: metav1.ObjectMeta{
Name: poolDeploymentName,
Name: gp.getPoolName(),
Labels: gp.labelsForPool,
},
Spec: v1beta1.DeploymentSpec{
+21 -23
View File
@@ -36,11 +36,9 @@ var (
)
// CleanupOldExecutorObjects cleans up resources created by old executor instances
func CleanupOldExecutorObjects(kubernetesClient *kubernetes.Clientset,
namespace string,
instanceId string) {
func CleanupOldExecutorObjects(kubernetesClient *kubernetes.Clientset, instanceId string) {
go func() {
err := cleanup(kubernetesClient, namespace, instanceId)
err := cleanup(kubernetesClient, instanceId)
if err != nil {
// TODO retry reaper; logged and ignored for now
log.Printf("Failed to reaper: %v", err)
@@ -48,14 +46,14 @@ func CleanupOldExecutorObjects(kubernetesClient *kubernetes.Clientset,
}()
}
func cleanup(client *kubernetes.Clientset, namespace string, instanceId string) error {
func cleanup(client *kubernetes.Clientset, instanceId string) error {
err := cleanupServices(client, namespace, instanceId)
err := cleanupServices(client, instanceId)
if err != nil {
return err
}
err = cleanupHpa(client, namespace, instanceId)
err = cleanupHpa(client, instanceId)
if err != nil {
return err
}
@@ -63,7 +61,7 @@ func cleanup(client *kubernetes.Clientset, namespace string, instanceId string)
// Deployments are used for idle pools and can be cleaned up
// immediately. (We should "adopt" these instead of creating
// a new pool.)
err = cleanupDeployments(client, namespace, instanceId)
err = cleanupDeployments(client, instanceId)
if err != nil {
return err
}
@@ -75,7 +73,7 @@ func cleanup(client *kubernetes.Clientset, namespace string, instanceId string)
// time.
time.Sleep(6 * time.Minute)
err = cleanupPods(client, namespace, instanceId)
err = cleanupPods(client, instanceId)
if err != nil {
return err
}
@@ -108,8 +106,8 @@ func CleanupKubeObject(kubeClient *kubernetes.Clientset, kubeobj *apiv1.ObjectRe
}
}
func cleanupDeployments(client *kubernetes.Clientset, namespace string, instanceId string) error {
deploymentList, err := client.ExtensionsV1beta1().Deployments(namespace).List(meta_v1.ListOptions{})
func cleanupDeployments(client *kubernetes.Clientset, instanceId string) error {
deploymentList, err := client.ExtensionsV1beta1().Deployments(meta_v1.NamespaceAll).List(meta_v1.ListOptions{})
if err != nil {
return err
}
@@ -117,7 +115,7 @@ func cleanupDeployments(client *kubernetes.Clientset, namespace string, instance
id, ok := dep.ObjectMeta.Labels[fission.EXECUTOR_INSTANCEID_LABEL]
if ok && id != instanceId {
log.Printf("Cleaning up deployment %v", dep.ObjectMeta.Name)
err := client.ExtensionsV1beta1().Deployments(namespace).Delete(dep.ObjectMeta.Name, &delOpt)
err := client.ExtensionsV1beta1().Deployments(dep.ObjectMeta.Namespace).Delete(dep.ObjectMeta.Name, &delOpt)
logErr("cleaning up deployment", err)
// ignore err
}
@@ -125,7 +123,7 @@ func cleanupDeployments(client *kubernetes.Clientset, namespace string, instance
pid, pok := dep.ObjectMeta.Labels[fission.POOLMGR_INSTANCEID_LABEL]
if pok && pid != instanceId {
log.Printf("Cleaning up deployment %v", dep.ObjectMeta.Name)
err := client.ExtensionsV1beta1().Deployments(namespace).Delete(dep.ObjectMeta.Name, &delOpt)
err := client.ExtensionsV1beta1().Deployments(dep.ObjectMeta.Namespace).Delete(dep.ObjectMeta.Name, &delOpt)
logErr("cleaning up deployment", err)
// ignore err
}
@@ -133,8 +131,8 @@ func cleanupDeployments(client *kubernetes.Clientset, namespace string, instance
return nil
}
func cleanupPods(client *kubernetes.Clientset, namespace string, instanceId string) error {
podList, err := client.CoreV1().Pods(namespace).List(meta_v1.ListOptions{})
func cleanupPods(client *kubernetes.Clientset, instanceId string) error {
podList, err := client.CoreV1().Pods(meta_v1.NamespaceAll).List(meta_v1.ListOptions{})
if err != nil {
return err
}
@@ -143,7 +141,7 @@ func cleanupPods(client *kubernetes.Clientset, namespace string, instanceId stri
id, ok := pod.ObjectMeta.Labels[fission.EXECUTOR_INSTANCEID_LABEL]
if ok && id != instanceId {
log.Printf("Cleaning up pod %v", pod.ObjectMeta.Name)
err := client.CoreV1().Pods(namespace).Delete(pod.ObjectMeta.Name, nil)
err := client.CoreV1().Pods(pod.ObjectMeta.Namespace).Delete(pod.ObjectMeta.Name, nil)
logErr("cleaning up pod", err)
// ignore err
}
@@ -151,7 +149,7 @@ func cleanupPods(client *kubernetes.Clientset, namespace string, instanceId stri
pid, pok := pod.ObjectMeta.Labels[fission.POOLMGR_INSTANCEID_LABEL]
if pok && pid != instanceId {
log.Printf("Cleaning up pod %v", pod.ObjectMeta.Name)
err := client.CoreV1().Pods(namespace).Delete(pod.ObjectMeta.Name, nil)
err := client.CoreV1().Pods(pod.ObjectMeta.Namespace).Delete(pod.ObjectMeta.Name, nil)
logErr("cleaning up pod", err)
// ignore err
}
@@ -160,8 +158,8 @@ func cleanupPods(client *kubernetes.Clientset, namespace string, instanceId stri
return nil
}
func cleanupServices(client *kubernetes.Clientset, namespace string, instanceId string) error {
svcList, err := client.CoreV1().Services(namespace).List(meta_v1.ListOptions{})
func cleanupServices(client *kubernetes.Clientset, instanceId string) error {
svcList, err := client.CoreV1().Services(meta_v1.NamespaceAll).List(meta_v1.ListOptions{})
if err != nil {
return err
}
@@ -169,7 +167,7 @@ func cleanupServices(client *kubernetes.Clientset, namespace string, instanceId
id, ok := svc.ObjectMeta.Labels[fission.EXECUTOR_INSTANCEID_LABEL]
if ok && id != instanceId {
log.Printf("Cleaning up svc %v", svc.ObjectMeta.Name)
err := client.CoreV1().Services(namespace).Delete(svc.ObjectMeta.Name, nil)
err := client.CoreV1().Services(svc.ObjectMeta.Namespace).Delete(svc.ObjectMeta.Name, nil)
logErr("cleaning up service", err)
// ignore err
}
@@ -177,8 +175,8 @@ func cleanupServices(client *kubernetes.Clientset, namespace string, instanceId
return nil
}
func cleanupHpa(client *kubernetes.Clientset, namespace string, instanceId string) error {
hpaList, err := client.AutoscalingV1().HorizontalPodAutoscalers(namespace).List(meta_v1.ListOptions{})
func cleanupHpa(client *kubernetes.Clientset, instanceId string) error {
hpaList, err := client.AutoscalingV1().HorizontalPodAutoscalers(meta_v1.NamespaceAll).List(meta_v1.ListOptions{})
if err != nil {
return err
}
@@ -187,7 +185,7 @@ func cleanupHpa(client *kubernetes.Clientset, namespace string, instanceId strin
id, ok := hpa.ObjectMeta.Labels[fission.EXECUTOR_INSTANCEID_LABEL]
if ok && id != instanceId {
log.Printf("Cleaning up HPA %v", hpa.ObjectMeta.Name)
err := client.AutoscalingV1().HorizontalPodAutoscalers(namespace).Delete(hpa.ObjectMeta.Name, nil)
err := client.AutoscalingV1().HorizontalPodAutoscalers(hpa.ObjectMeta.Namespace).Delete(hpa.ObjectMeta.Name, nil)
logErr("cleaning up HPA", err)
}
+49
View File
@@ -0,0 +1,49 @@
/*
Copyright 2018 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package featureconfig
import (
"encoding/base64"
"fmt"
"io/ioutil"
"github.com/ghodss/yaml"
)
// GetFeatureConfig reads the configMap file and unmarshals the config into a feature config struct
func GetFeatureConfig() (*FeatureConfig, error) {
// read the file
b64EncodedContent, err := ioutil.ReadFile(FeatureConfigFile)
if err != nil {
return nil, fmt.Errorf("error reading YAML file %s: %v", FeatureConfigFile, err)
}
// b64 decode file
yamlContent, err := base64.StdEncoding.DecodeString(string(b64EncodedContent))
if err != nil {
return nil, fmt.Errorf("error b64 decoding the config : %v", err)
}
// unmarshal into feature config
featureConfig := &FeatureConfig{}
err = yaml.Unmarshal(yamlContent, featureConfig)
if err != nil {
return nil, fmt.Errorf("error unmarshalling YAML config %v", err)
}
return featureConfig, err
}
+40
View File
@@ -0,0 +1,40 @@
/*
Copyright 2018 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package featureconfig
const (
FeatureConfigFile = "/etc/config/config.yaml"
)
type (
// config.yaml contains config parameters for optional features
// To add new features with config parameters:
// 1. create a yaml block with feature name in charts/_helpers.tpl
// 2. define a corresponding struct with the feature config for the yaml unmarshal below
// 3. start the appropriate controllers needed for this feature
FeatureConfig struct {
// In the future more such feature configs can be added here for each optional feature
CanaryConfig CanaryFeatureConfig `json:"canary"`
}
// specific feature config
CanaryFeatureConfig struct {
IsEnabled bool `json:"enabled"`
PrometheusSvc string `json:"prometheusSvc"`
}
)
+1 -1
View File
@@ -20,7 +20,7 @@ import (
)
func runController(port int) {
controller.Start(port)
controller.Start(port, false)
log.Fatalf("Error: Controller exited.")
}
+2 -1
View File
@@ -26,6 +26,7 @@ import (
"github.com/fission/fission"
"github.com/fission/fission/controller/client"
"github.com/fission/fission/crd"
"github.com/fission/fission/fission/util"
)
type (
@@ -67,7 +68,7 @@ func (w *packageBuildWatcher) watch(ctx context.Context) {
// poll list of packages (TODO: convert to watch)
pkgs, err := w.fclient.PackageList(metav1.NamespaceAll)
checkErr(err, "Getting list of packages")
util.CheckErr(err, "Getting list of packages")
// find packages that (a) are in the app spec and (b) have an interesting
// build status (either succeeded or failed; not "none")
+235
View File
@@ -0,0 +1,235 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"fmt"
"os"
"text/tabwriter"
"time"
"github.com/urfave/cli"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission"
"github.com/fission/fission/crd"
"github.com/fission/fission/fission/log"
"github.com/fission/fission/fission/util"
)
func canaryConfigCreate(c *cli.Context) error {
client := util.GetApiClient(c.GlobalString("server"))
canaryConfigName := c.String("name")
// canary configs can be created for functions in the same namespace
if len(canaryConfigName) == 0 {
log.Fatal("Need a name, use --name.")
}
trigger := c.String("httptrigger")
newFunc := c.String("newfunction")
oldFunc := c.String("oldfunction")
ns := c.String("fnNamespace")
incrementStep := c.Int("increment-step")
failureThreshold := c.Int("failure-threshold")
incrementInterval := c.String("increment-interval")
// check for time parsing
_, err := time.ParseDuration(incrementInterval)
util.CheckErr(err, "parsing time duration.")
// check that the trigger exists in the same namespace.
m := &metav1.ObjectMeta{
Name: trigger,
Namespace: ns,
}
htTrigger, err := client.HTTPTriggerGet(m)
if err != nil {
util.CheckErr(err, "Trigger referenced in the canary config is not created")
}
// check that the trigger has function reference type function weights
if htTrigger.Spec.FunctionReference.Type != fission.FunctionReferenceTypeFunctionWeights {
log.Fatal("Canary config cannot be created for http triggers that do not reference functions by weights")
}
// check that the trigger references same functions in the function weights
_, ok := htTrigger.Spec.FunctionReference.FunctionWeights[newFunc]
if !ok {
log.Fatal(fmt.Sprintf("HTTP Trigger doesn't reference the function %s in Canary Config", newFunc))
}
_, ok = htTrigger.Spec.FunctionReference.FunctionWeights[oldFunc]
if !ok {
log.Fatal(fmt.Sprintf("HTTP Trigger doesn't reference the function %s in Canary Config", oldFunc))
}
// check that the functions exist in the same namespace
fnList := []string{newFunc, oldFunc}
err = util.CheckFunctionExistence(client, fnList, ns)
if err != nil {
log.Fatal(fmt.Sprintf("checkFunctionExistence err : %v", err))
}
// finally create canaryCfg in the same namespace as the functions referenced
canaryCfg := &crd.CanaryConfig{
Metadata: metav1.ObjectMeta{
Name: canaryConfigName,
Namespace: ns,
},
Spec: fission.CanaryConfigSpec{
Trigger: trigger,
NewFunction: newFunc,
OldFunction: oldFunc,
WeightIncrement: incrementStep,
WeightIncrementDuration: incrementInterval,
FailureThreshold: failureThreshold,
FailureType: fission.FailureTypeStatusCode,
},
Status: fission.CanaryConfigStatus{
Status: fission.CanaryConfigStatusPending,
},
}
_, err = client.CanaryConfigCreate(canaryCfg)
util.CheckErr(err, "create canary config")
fmt.Printf("canary config '%v' created\n", canaryConfigName)
return err
}
func canaryConfigGet(c *cli.Context) error {
client := util.GetApiClient(c.GlobalString("server"))
name := c.String("name")
if len(name) == 0 {
log.Fatal("Need a name, use --name.")
}
ns := c.String("canaryNamespace")
m := &metav1.ObjectMeta{
Name: name,
Namespace: ns,
}
canaryCfg, err := client.CanaryConfigGet(m)
util.CheckErr(err, "get canary config")
w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\n", "NAME", "TRIGGER", "FUNCTION-N", "FUNCTION-N-1", "WEIGHT-INCREMENT", "INTERVAL", "FAILURE-THRESHOLD", "FAILURE-TYPE", "STATUS")
fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\n",
canaryCfg.Metadata.Name, canaryCfg.Spec.Trigger, canaryCfg.Spec.NewFunction, canaryCfg.Spec.OldFunction, canaryCfg.Spec.WeightIncrement, canaryCfg.Spec.WeightIncrementDuration,
canaryCfg.Spec.FailureThreshold, canaryCfg.Spec.FailureType, canaryCfg.Status.Status)
w.Flush()
return nil
}
func canaryConfigUpdate(c *cli.Context) error {
client := util.GetApiClient(c.GlobalString("server"))
canaryConfigName := c.String("name")
ns := c.String("canaryNamespace")
if len(canaryConfigName) == 0 {
log.Fatal("Need a name, use --name.")
}
incrementStep := c.Int("increment-step")
failureThreshold := c.Int("failure-threshold")
incrementInterval := c.String("increment-interval")
// check for time parsing
_, err := time.ParseDuration(incrementInterval)
util.CheckErr(err, "parsing time duration.")
// get the current config
m := &metav1.ObjectMeta{
Name: canaryConfigName,
Namespace: ns,
}
var updateNeeded bool
canaryCfg, err := client.CanaryConfigGet(m)
util.CheckErr(err, "get canary config")
if incrementStep != canaryCfg.Spec.WeightIncrement {
canaryCfg.Spec.WeightIncrement = incrementStep
updateNeeded = true
}
if failureThreshold != canaryCfg.Spec.FailureThreshold {
canaryCfg.Spec.FailureThreshold = failureThreshold
updateNeeded = true
}
if incrementInterval != canaryCfg.Spec.WeightIncrementDuration {
canaryCfg.Spec.WeightIncrementDuration = incrementInterval
updateNeeded = true
}
if updateNeeded {
canaryCfg.Status.Status = fission.CanaryConfigStatusPending
_, err = client.CanaryConfigUpdate(canaryCfg)
util.CheckErr(err, "update canary config")
}
return nil
}
func canaryConfigDelete(c *cli.Context) error {
client := util.GetApiClient(c.GlobalString("server"))
canaryConfigName := c.String("name")
ns := c.String("canaryNamespace")
if len(canaryConfigName) == 0 {
log.Fatal("Need a name, use --name.")
}
// get the current config
m := &metav1.ObjectMeta{
Name: canaryConfigName,
Namespace: ns,
}
err := client.CanaryConfigDelete(m)
util.CheckErr(err, fmt.Sprintf("delete function '%v.%v'", canaryConfigName, ns))
fmt.Printf("canaryconfig '%v.%v' deleted\n", canaryConfigName, ns)
return err
}
func canaryConfigList(c *cli.Context) error {
client := util.GetApiClient(c.GlobalString("server"))
ns := c.String("canaryNamespace")
canaryCfgs, err := client.CanaryConfigList(ns)
util.CheckErr(err, "list canary config")
w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\n", "NAME", "TRIGGER", "FUNCTION-N", "FUNCTION-N-1", "WEIGHT-INCREMENT", "INTERVAL", "FAILURE-THRESHOLD", "FAILURE-TYPE", "STATUS")
for _, canaryCfg := range canaryCfgs {
fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\n",
canaryCfg.Metadata.Name, canaryCfg.Spec.Trigger, canaryCfg.Spec.NewFunction, canaryCfg.Spec.OldFunction, canaryCfg.Spec.WeightIncrement, canaryCfg.Spec.WeightIncrementDuration,
canaryCfg.Spec.FailureThreshold, canaryCfg.Spec.FailureType, canaryCfg.Status.Status)
}
w.Flush()
return nil
}
-343
View File
@@ -1,343 +0,0 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"io"
"io/ioutil"
"net/http"
"os"
"path"
"path/filepath"
"regexp"
"strings"
"github.com/dchest/uniuri"
uuid "github.com/satori/go.uuid"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission"
"github.com/fission/fission/controller/client"
"github.com/fission/fission/crd"
"github.com/fission/fission/fission/log"
storageSvcClient "github.com/fission/fission/storagesvc/client"
)
func getClient(serverUrl string) *client.Client {
if len(serverUrl) == 0 {
// starts local portforwarder etc.
serverUrl = getServerUrl()
}
isHTTPS := strings.Index(serverUrl, "https://") == 0
isHTTP := strings.Index(serverUrl, "http://") == 0
if !(isHTTP || isHTTPS) {
serverUrl = "http://" + serverUrl
}
return client.MakeClient(serverUrl)
}
func checkErr(err error, msg string) {
if err != nil {
log.Fatal(fmt.Sprintf("Failed to %v: %v", msg, err))
}
}
func httpRequest(method, url, body string, headers []string) *http.Response {
if method == "" {
method = "GET"
}
if method != http.MethodGet &&
method != http.MethodDelete &&
method != http.MethodPost &&
method != http.MethodPut {
log.Fatal(fmt.Sprintf("Invalid HTTP method '%s'.", method))
}
req, err := http.NewRequest(method, url, strings.NewReader(body))
checkErr(err, "create HTTP request")
for _, header := range headers {
headerKeyValue := strings.SplitN(header, ":", 2)
if len(headerKeyValue) != 2 {
checkErr(errors.New(""), "create request without appropriate headers")
}
req.Header.Set(headerKeyValue[0], headerKeyValue[1])
}
client := &http.Client{}
resp, err := client.Do(req)
checkErr(err, "execute HTTP request")
return resp
}
func fileSize(filePath string) int64 {
info, err := os.Stat(filePath)
checkErr(err, fmt.Sprintf("stat %v", filePath))
return info.Size()
}
func fileChecksum(fileName string) (*fission.Checksum, error) {
f, err := os.Open(fileName)
if err != nil {
return nil, fmt.Errorf("failed to open file %v: %v", fileName, err)
}
defer f.Close()
h := sha256.New()
_, err = io.Copy(h, f)
if err != nil {
return nil, fmt.Errorf("failed to calculate checksum for %v", fileName)
}
return &fission.Checksum{
Type: fission.ChecksumTypeSHA256,
Sum: hex.EncodeToString(h.Sum(nil)),
}, nil
}
// upload a file and return a fission.Archive
func createArchive(client *client.Client, fileName string, specFile string) *fission.Archive {
var archive fission.Archive
// fetch archive from arbitrary url if fileName is a url
if strings.HasPrefix(fileName, "http://") || strings.HasPrefix(fileName, "https://") {
fileName = downloadToTempFile(fileName)
}
if len(specFile) > 0 {
// create an ArchiveUploadSpec and reference it from the archive
aus := &ArchiveUploadSpec{
Name: kubifyName(path.Base(fileName)),
IncludeGlobs: []string{fileName},
}
// save the uploadspec
err := specSave(*aus, specFile)
checkErr(err, fmt.Sprintf("write spec file %v", specFile))
// create the archive
ar := &fission.Archive{
Type: fission.ArchiveTypeUrl,
URL: fmt.Sprintf("%v%v", ARCHIVE_URL_PREFIX, aus.Name),
}
return ar
}
if fileSize(fileName) < fission.ArchiveLiteralSizeLimit {
contents := getContents(fileName)
archive.Type = fission.ArchiveTypeLiteral
archive.Literal = contents
} else {
u := strings.TrimSuffix(client.Url, "/") + "/proxy/storage"
ssClient := storageSvcClient.MakeClient(u)
// TODO add a progress bar
id, err := ssClient.Upload(fileName, nil)
checkErr(err, fmt.Sprintf("upload file %v", fileName))
storageSvc, err := client.GetSvcURL("application=fission-storage")
storageSvcURL := "http://" + storageSvc
checkErr(err, "get fission storage service name")
// We make a new client with actual URL of Storage service so that the URL is not
// pointing to 127.0.0.1 i.e. proxy. DON'T reuse previous ssClient
pkgClient := storageSvcClient.MakeClient(storageSvcURL)
archiveURL := pkgClient.GetUrl(id)
archive.Type = fission.ArchiveTypeUrl
archive.URL = archiveURL
csum, err := fileChecksum(fileName)
checkErr(err, fmt.Sprintf("calculate checksum for file %v", fileName))
archive.Checksum = *csum
}
return &archive
}
func createPackage(client *client.Client, pkgNamespace, envName, envNamespace, srcArchiveName, deployArchiveName, buildcmd string, specFile string) *metav1.ObjectMeta {
pkgSpec := fission.PackageSpec{
Environment: fission.EnvironmentReference{
Namespace: envNamespace,
Name: envName,
},
}
var pkgStatus fission.BuildStatus = fission.BuildStatusSucceeded
var pkgName string
if len(deployArchiveName) > 0 {
if len(specFile) > 0 { // we should do this in all cases, i think
pkgStatus = fission.BuildStatusNone
}
pkgSpec.Deployment = *createArchive(client, deployArchiveName, specFile)
pkgName = kubifyName(fmt.Sprintf("%v-%v", path.Base(deployArchiveName), uniuri.NewLen(4)))
}
if len(srcArchiveName) > 0 {
pkgSpec.Source = *createArchive(client, srcArchiveName, specFile)
// set pending status to package
pkgStatus = fission.BuildStatusPending
pkgName = kubifyName(fmt.Sprintf("%v-%v", path.Base(srcArchiveName), uniuri.NewLen(4)))
}
if len(buildcmd) > 0 {
pkgSpec.BuildCommand = buildcmd
}
if len(pkgName) == 0 {
pkgName = strings.ToLower(uuid.NewV4().String())
}
pkg := &crd.Package{
Metadata: metav1.ObjectMeta{
Name: pkgName,
Namespace: pkgNamespace,
},
Spec: pkgSpec,
Status: fission.PackageStatus{
BuildStatus: pkgStatus,
},
}
if len(specFile) > 0 {
err := specSave(*pkg, specFile)
checkErr(err, "save package spec")
return &pkg.Metadata
} else {
pkgMetadata, err := client.PackageCreate(pkg)
checkErr(err, "create package")
return pkgMetadata
}
}
func getContents(filePath string) []byte {
var code []byte
var err error
code, err = ioutil.ReadFile(filePath)
checkErr(err, fmt.Sprintf("read %v", filePath))
return code
}
func getTempDir() (string, error) {
tmpDir := uuid.NewV4().String()
tmpPath := filepath.Join(os.TempDir(), tmpDir)
err := os.Mkdir(tmpPath, 0744)
return tmpPath, err
}
func writeArchiveToFile(fileName string, reader io.Reader) error {
tmpDir, err := getTempDir()
if err != nil {
return err
}
path := filepath.Join(tmpDir, fileName+".tmp")
w, err := os.Create(path)
if err != nil {
return err
}
_, err = io.Copy(w, reader)
if err != nil {
return err
}
err = os.Chmod(path, 0644)
if err != nil {
return err
}
err = os.Rename(path, fileName)
if err != nil {
return err
}
return nil
}
// downloadToTempFile fetches archive file from arbitrary url
// and write it to temp file for further usage
func downloadToTempFile(fileUrl string) string {
reader, err := downloadURL(fileUrl)
defer reader.Close()
checkErr(err, fmt.Sprintf("download from url: %v", fileUrl))
tmpDir, err := getTempDir()
checkErr(err, "create temp directory")
tmpFilename := uuid.NewV4().String()
destination := filepath.Join(tmpDir, tmpFilename)
err = os.Mkdir(tmpDir, 0744)
checkErr(err, "create temp directory")
err = writeArchiveToFile(destination, reader)
checkErr(err, "write archive to file")
return destination
}
// downloadURL downloads file from given url
func downloadURL(fileUrl string) (io.ReadCloser, error) {
resp, err := http.Get(fileUrl)
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("%v - HTTP response returned non 200 status", resp.StatusCode)
}
return resp.Body, nil
}
// make a kubernetes compliant name out of an arbitrary string
func kubifyName(old string) string {
// Kubernetes maximum name length (for some names; others can be 253 chars)
maxLen := 63
newName := strings.ToLower(old)
// replace disallowed chars with '-'
inv, err := regexp.Compile("[^-a-z0-9]")
checkErr(err, "compile regexp")
newName = string(inv.ReplaceAll([]byte(newName), []byte("-")))
// trim leading non-alphabetic
leadingnonalpha, err := regexp.Compile("^[^a-z]+")
checkErr(err, "compile regexp")
newName = string(leadingnonalpha.ReplaceAll([]byte(newName), []byte{}))
// trim trailing
trailing, err := regexp.Compile("[^a-z0-9]+$")
checkErr(err, "compile regexp")
newName = string(trailing.ReplaceAll([]byte(newName), []byte{}))
// truncate to length
if len(newName) > maxLen {
newName = newName[0:maxLen]
}
// if we removed everything, call this thing "default". maybe
// we should generate a unique name...
if len(newName) == 0 {
newName = "default"
}
return newName
}
+40 -15
View File
@@ -22,6 +22,7 @@ import (
"strconv"
"text/tabwriter"
"github.com/fission/fission/fission/util"
"github.com/urfave/cli"
"k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"
@@ -48,7 +49,7 @@ func getFunctionsByEnvironment(client *client.Client, envName, envNamespace stri
}
func envCreate(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
envName := c.String("name")
if len(envName) == 0 {
@@ -129,19 +130,19 @@ func envCreate(c *cli.Context) error {
if c.Bool("spec") {
specFile := fmt.Sprintf("env-%v.yaml", envName)
err := specSave(*env, specFile)
checkErr(err, "create environment spec")
util.CheckErr(err, "create environment spec")
return nil
}
_, err = client.EnvironmentCreate(env)
checkErr(err, "create environment")
util.CheckErr(err, "create environment")
fmt.Printf("environment '%v' created\n", envName)
return err
}
func envGet(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
envName := c.String("name")
if len(envName) == 0 {
@@ -154,7 +155,7 @@ func envGet(c *cli.Context) error {
Namespace: envNamespace,
}
env, err := client.EnvironmentGet(m)
checkErr(err, "get environment")
util.CheckErr(err, "get environment")
w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
fmt.Fprintf(w, "%v\t%v\t%v\n", "NAME", "UID", "IMAGE")
@@ -165,7 +166,7 @@ func envGet(c *cli.Context) error {
}
func envUpdate(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
envName := c.String("name")
if len(envName) == 0 {
@@ -186,7 +187,7 @@ func envUpdate(c *cli.Context) error {
Name: envName,
Namespace: envNamespace,
})
checkErr(err, "find environment")
util.CheckErr(err, "find environment")
if len(envImg) > 0 {
env.Spec.Runtime.Image = envImg
@@ -217,15 +218,19 @@ func envUpdate(c *cli.Context) error {
env.Spec.AllowAccessToExternalNetwork = envExternalNetwork
if c.IsSet("mincpu") || c.IsSet("maxcpu") || c.IsSet("minmemory") || c.IsSet("maxmemory") || c.IsSet("minscale") || c.IsSet("maxscale") {
log.Fatal("Updating resource limits/requests for existing environments is currently unsupported; re-create the environment instead.")
}
_, err = client.EnvironmentUpdate(env)
checkErr(err, "update environment")
util.CheckErr(err, "update environment")
fmt.Printf("environment '%v' updated\n", envName)
return nil
}
func envDelete(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
envName := c.String("name")
if len(envName) == 0 {
@@ -238,24 +243,24 @@ func envDelete(c *cli.Context) error {
Namespace: envNamespace,
}
err := client.EnvironmentDelete(m)
checkErr(err, "delete environment")
util.CheckErr(err, "delete environment")
fmt.Printf("environment '%v' deleted\n", envName)
return nil
}
func envList(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
envNamespace := c.String("envNamespace")
envs, err := client.EnvironmentList(envNamespace)
checkErr(err, "list environments")
util.CheckErr(err, "list environments")
w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\n", "NAME", "UID", "IMAGE", "POOLSIZE", "MINCPU", "MAXCPU", "MINMEMORY", "MAXMEMORY", "EXTNET", "GRACETIME")
fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\n", "NAME", "UID", "IMAGE", "BUILDER_IMAGE", "POOLSIZE", "MINCPU", "MAXCPU", "MINMEMORY", "MAXMEMORY", "EXTNET", "GRACETIME")
for _, env := range envs {
fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\n",
env.Metadata.Name, env.Metadata.UID, env.Spec.Runtime.Image, env.Spec.Poolsize,
fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\n",
env.Metadata.Name, env.Metadata.UID, env.Spec.Runtime.Image, env.Spec.Builder.Image, env.Spec.Poolsize,
env.Spec.Resources.Requests.Cpu(), env.Spec.Resources.Limits.Cpu(),
env.Spec.Resources.Requests.Memory(), env.Spec.Resources.Limits.Memory(),
env.Spec.AllowAccessToExternalNetwork, env.Spec.TerminationGracePeriod)
@@ -294,6 +299,7 @@ func getResourceReq(c *cli.Context, resources v1.ResourceRequirements) v1.Resour
}
var limitResources map[v1.ResourceName]resource.Quantity
if len(resources.Limits) == 0 {
limitResources = make(map[v1.ResourceName]resource.Quantity)
} else {
@@ -318,9 +324,28 @@ func getResourceReq(c *cli.Context, resources v1.ResourceRequirements) v1.Resour
limitResources[v1.ResourceMemory] = memLimit
}
limitCPU := limitResources[v1.ResourceCPU]
requestCPU := requestResources[v1.ResourceCPU]
if limitCPU.IsZero() && !requestCPU.IsZero() {
limitResources[v1.ResourceCPU] = requestCPU
} else if limitCPU.Cmp(requestCPU) < 0 {
log.Fatal(fmt.Sprintf("MinCPU (%v) cannot be greater than MaxCPU (%v)", requestCPU.String(), limitCPU.String()))
}
limitMem := limitResources[v1.ResourceMemory]
requestMem := requestResources[v1.ResourceMemory]
if limitMem.IsZero() && !requestMem.IsZero() {
limitResources[v1.ResourceMemory] = requestMem
} else if limitMem.Cmp(requestMem) < 0 {
log.Fatal(fmt.Sprintf("MinMemory (%v) cannot be greater than MaxMemory (%v)", requestMem.String(), limitMem.String()))
}
resources = v1.ResourceRequirements{
Requests: requestResources,
Limits: limitResources,
}
return resources
}
+214 -143
View File
@@ -38,7 +38,12 @@ import (
"github.com/fission/fission/crd"
"github.com/fission/fission/fission/log"
"github.com/fission/fission/fission/logdb"
"github.com/fission/fission/fission/portforward"
"github.com/fission/fission/fission/util"
)
const (
DEFAULT_MIN_SCALE = 1
DEFAULT_TARGET_CPU_PERCENTAGE = 80
)
func printPodLogs(c *cli.Context) error {
@@ -47,16 +52,16 @@ func printPodLogs(c *cli.Context) error {
log.Fatal("Need --name argument.")
}
queryURL, err := url.Parse(getServerUrl())
checkErr(err, "parse the base URL")
queryURL, err := url.Parse(util.GetServerUrl())
util.CheckErr(err, "parse the base URL")
queryURL.Path = fmt.Sprintf("/proxy/logs/%s", fnName)
req, err := http.NewRequest("POST", queryURL.String(), nil)
checkErr(err, "create logs request")
util.CheckErr(err, "create logs request")
httpClient := http.Client{}
resp, err := httpClient.Do(req)
checkErr(err, "execute get logs request")
util.CheckErr(err, "execute get logs request")
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
@@ -64,45 +69,96 @@ func printPodLogs(c *cli.Context) error {
}
body, err := ioutil.ReadAll(resp.Body)
checkErr(err, "read the response body")
util.CheckErr(err, "read the response body")
fmt.Println(string(body))
return nil
}
func getInvokeStrategy(minScale int, maxScale int, executorType string, targetcpu int) fission.InvokeStrategy {
func getInvokeStrategy(c *cli.Context, existingInvokeStrategy *fission.InvokeStrategy) (strategy *fission.InvokeStrategy, err error) {
if maxScale == 0 {
maxScale = 1
}
var fnExecutor, newFnExecutor fission.ExecutorType
if minScale > maxScale {
log.Fatal("Maxscale must be higher than or equal to minscale")
}
var fnExecutor fission.ExecutorType
switch executorType {
switch c.String("executortype") {
case "":
fnExecutor = fission.ExecutorTypePoolmgr
fallthrough
case fission.ExecutorTypePoolmgr:
fnExecutor = fission.ExecutorTypePoolmgr
newFnExecutor = fission.ExecutorTypePoolmgr
case fission.ExecutorTypeNewdeploy:
fnExecutor = fission.ExecutorTypeNewdeploy
newFnExecutor = fission.ExecutorTypeNewdeploy
default:
log.Fatal("Executor type must be one of 'poolmgr' or 'newdeploy', defaults to 'poolmgr'")
return nil, errors.New("Executor type must be one of 'poolmgr' or 'newdeploy', defaults to 'poolmgr'")
}
// Right now a simple single case strategy implementation
// This will potentially get more sophisticated once we have more strategies in place
strategy := fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fnExecutor,
MinScale: minScale,
MaxScale: maxScale,
TargetCPUPercent: targetcpu,
},
if existingInvokeStrategy != nil {
fnExecutor = existingInvokeStrategy.ExecutionStrategy.ExecutorType
// override the executor type if user specified a new executor type
if c.IsSet("executortype") {
fnExecutor = newFnExecutor
}
} else {
fnExecutor = newFnExecutor
}
return strategy
if fnExecutor == fission.ExecutorTypePoolmgr {
if c.IsSet("targetcpu") || c.IsSet("minscale") || c.IsSet("maxscale") {
log.Fatal("To set target CPU or min/max scale for function, please specify \"--executortype newdeploy\"")
}
if c.IsSet("mincpu") || c.IsSet("maxcpu") || c.IsSet("minmemory") || c.IsSet("maxmemory") {
log.Warn("To limit CPU/Memory for function with executor type \"poolmgr\", please specify resources limits when creating environment")
}
strategy = &fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fission.ExecutorTypePoolmgr,
},
}
} else {
// set default value
targetCPU := DEFAULT_TARGET_CPU_PERCENTAGE
minScale := DEFAULT_MIN_SCALE
maxScale := minScale
if existingInvokeStrategy != nil && existingInvokeStrategy.ExecutionStrategy.ExecutorType == fission.ExecutorTypeNewdeploy {
minScale = existingInvokeStrategy.ExecutionStrategy.MinScale
maxScale = existingInvokeStrategy.ExecutionStrategy.MaxScale
targetCPU = existingInvokeStrategy.ExecutionStrategy.TargetCPUPercent
}
if c.IsSet("targetcpu") {
targetCPU = getTargetCPU(c)
}
if c.IsSet("minscale") {
minScale = c.Int("minscale")
}
if c.IsSet("maxscale") {
maxScale = c.Int("maxscale")
if maxScale <= 0 {
return nil, errors.New("Maxscale must be greater than 0")
}
}
if minScale > maxScale {
return nil, errors.New(fmt.Sprintf("Minscale provided: %v can not be greater than maxscale value %v", minScale, maxScale))
}
// Right now a simple single case strategy implementation
// This will potentially get more sophisticated once we have more strategies in place
strategy = &fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fnExecutor,
MinScale: minScale,
MaxScale: maxScale,
TargetCPUPercent: targetCPU,
},
}
}
return strategy, nil
}
func getTargetCPU(c *cli.Context) int {
@@ -113,14 +169,14 @@ func getTargetCPU(c *cli.Context) int {
log.Fatal("TargetCPU must be a value between 1 - 100")
}
} else {
targetCPU = 80
targetCPU = DEFAULT_TARGET_CPU_PERCENTAGE
}
return targetCPU
}
// From this change onwards, we mandate that a function should reference a secret, config map and package in its own ns
func fnCreate(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
fnNamespace := c.String("fnNamespace")
envNamespace := c.String("envNamespace")
@@ -137,10 +193,11 @@ func fnCreate(c *cli.Context) error {
spec = true
specFile = fmt.Sprintf("function-%v.yaml", fnName)
}
specDir := getSpecDir(c)
// check for unique function names within a namespace
fnList, err := client.FunctionList(fnNamespace)
checkErr(err, "get function list")
util.CheckErr(err, "get function list")
// check function existence before creating package
for _, fn := range fnList {
if fn.Metadata.Name == fnName {
@@ -150,21 +207,29 @@ func fnCreate(c *cli.Context) error {
entrypoint := c.String("entrypoint")
pkgName := c.String("pkg")
var pkgMetadata *metav1.ObjectMeta
var envName string
secretName := c.String("secret")
cfgMapName := c.String("configmap")
invokeStrategy, err := getInvokeStrategy(c, nil)
if err != nil {
log.Fatal(err)
}
resourceReq := getResourceReq(c, apiv1.ResourceRequirements{})
var pkgMetadata *metav1.ObjectMeta
var envName string
if len(pkgName) > 0 {
// use existing package
pkg, err := client.PackageGet(&metav1.ObjectMeta{
Namespace: fnNamespace,
Name: pkgName,
})
checkErr(err, fmt.Sprintf("read package in '%v' in Namespace: %s. Package needs to be present in the same namespace as function", pkgName, fnNamespace))
util.CheckErr(err, fmt.Sprintf("read package in '%v' in Namespace: %s. Package needs to be present in the same namespace as function", pkgName, fnNamespace))
pkgMetadata = &pkg.Metadata
envName = pkg.Spec.Environment.Name
if envName != c.String("env") {
log.Warn("Function's environment is different than package's environment, package's environment will be used for creating function")
}
envNamespace = pkg.Spec.Environment.Namespace
} else {
// need to specify environment for creating new package
@@ -173,40 +238,42 @@ func fnCreate(c *cli.Context) error {
log.Fatal("Need --env argument.")
}
// examine existence of given environment
_, err := client.EnvironmentGet(&metav1.ObjectMeta{
Namespace: envNamespace,
Name: envName,
})
if err != nil {
if e, ok := err.(fission.Error); ok && e.Code == fission.ErrorNotFound {
fmt.Printf("Environment \"%v\" does not exist. Please create the environment before executing the function. \nFor example: `fission env create --name %v --envns %v --image <image>`\n", envName, envName, envNamespace)
} else {
checkErr(err, "retrieve environment information")
// examine existence of given environment. If specs - then spec validate will do it, don't check here.
if !spec {
_, err := client.EnvironmentGet(&metav1.ObjectMeta{
Namespace: envNamespace,
Name: envName,
})
if err != nil {
if e, ok := err.(fission.Error); ok && e.Code == fission.ErrorNotFound {
log.Warn(fmt.Sprintf("Environment \"%v\" does not exist. Please create the environment before executing the function. \nFor example: `fission env create --name %v --envns %v --image <image>`\n", envName, envName, envNamespace))
} else {
util.CheckErr(err, "retrieve environment information")
}
}
}
srcArchiveName := c.String("src")
deployArchiveName := c.String("code")
if len(deployArchiveName) == 0 {
deployArchiveName = c.String("deploy")
srcArchiveFiles := c.StringSlice("src")
var deployArchiveFiles []string
noZip := false
code := c.String("code")
if len(code) == 0 {
deployArchiveFiles = c.StringSlice("deploy")
} else {
deployArchiveFiles = append(deployArchiveFiles, c.String("code"))
noZip = true
}
// fatal when both src & deploy archive are empty
if len(srcArchiveName) == 0 && len(deployArchiveName) == 0 {
if len(srcArchiveFiles) == 0 && len(deployArchiveFiles) == 0 {
log.Fatal("Need --deploy or --src argument.")
}
buildcmd := c.String("buildcmd")
// create new package in the same namespace as the function.
pkgMetadata = createPackage(client, fnNamespace, envName, envNamespace, srcArchiveName, deployArchiveName, buildcmd, specFile)
}
pkgMetadata = createPackage(client, fnNamespace, envName, envNamespace, srcArchiveFiles, deployArchiveFiles, buildcmd, specDir, specFile, noZip)
invokeStrategy := getInvokeStrategy(c.Int("minscale"), c.Int("maxscale"), c.String("executortype"), getTargetCPU(c))
resourceReq := getResourceReq(c, apiv1.ResourceRequirements{})
if (c.IsSet("mincpu") || c.IsSet("maxcpu") || c.IsSet("minmemory") || c.IsSet("maxmemory")) &&
invokeStrategy.ExecutionStrategy.ExecutorType == fission.ExecutorTypePoolmgr {
log.Warn("CPU/Memory specified for function with pool manager executor will be ignored in favor of resources specified at environment")
fmt.Printf("package '%v' created\n", pkgMetadata.Name)
}
var secrets []fission.SecretReference
@@ -267,20 +334,20 @@ func fnCreate(c *cli.Context) error {
Secrets: secrets,
ConfigMaps: cfgmaps,
Resources: resourceReq,
InvokeStrategy: invokeStrategy,
InvokeStrategy: *invokeStrategy,
},
}
// if we're writing a spec, don't create the function
if spec {
err = specSave(*function, specFile)
checkErr(err, "create function spec")
util.CheckErr(err, "create function spec")
return nil
}
_, err = client.FunctionCreate(function)
checkErr(err, "create function")
util.CheckErr(err, "create function")
fmt.Printf("function '%v' created\n", fnName)
@@ -295,7 +362,7 @@ func fnCreate(c *cli.Context) error {
method := c.String("method")
if len(method) == 0 {
method = "GET"
method = http.MethodGet
}
triggerName := uuid.NewV4().String()
ht := &crd.HTTPTrigger{
@@ -313,14 +380,14 @@ func fnCreate(c *cli.Context) error {
},
}
_, err = client.HTTPTriggerCreate(ht)
checkErr(err, "create HTTP trigger")
util.CheckErr(err, "create HTTP trigger")
fmt.Printf("route created: %v %v -> %v\n", method, triggerUrl, fnName)
return err
}
func fnGet(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
fnName := c.String("name")
if len(fnName) == 0 {
@@ -332,20 +399,20 @@ func fnGet(c *cli.Context) error {
Namespace: fnNamespace,
}
fn, err := client.FunctionGet(m)
checkErr(err, "get function")
util.CheckErr(err, "get function")
pkg, err := client.PackageGet(&metav1.ObjectMeta{
Name: fn.Spec.Package.PackageRef.Name,
Namespace: fn.Spec.Package.PackageRef.Namespace,
})
checkErr(err, "get package")
util.CheckErr(err, "get package")
os.Stdout.Write(pkg.Spec.Deployment.Literal)
return err
}
func fnGetMeta(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
fnName := c.String("name")
if len(fnName) == 0 {
@@ -359,7 +426,7 @@ func fnGetMeta(c *cli.Context) error {
}
f, err := client.FunctionGet(m)
checkErr(err, "get function")
util.CheckErr(err, "get function")
w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
fmt.Fprintf(w, "%v\t%v\t%v\n", "NAME", "UID", "ENV")
@@ -370,7 +437,7 @@ func fnGetMeta(c *cli.Context) error {
}
func fnUpdate(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
if len(c.String("package")) > 0 {
log.Fatal("--package is deprecated, please use --deploy instead.")
@@ -390,7 +457,7 @@ func fnUpdate(c *cli.Context) error {
Name: fnName,
Namespace: fnNamespace,
})
checkErr(err, fmt.Sprintf("read function '%v'", fnName))
util.CheckErr(err, fmt.Sprintf("read function '%v'", fnName))
envName := c.String("env")
envNamespace := c.String("envNamespace")
@@ -405,11 +472,17 @@ func fnUpdate(c *cli.Context) error {
envNamespace = ""
}
deployArchiveName := c.String("code")
if len(deployArchiveName) == 0 {
deployArchiveName = c.String("deploy")
var deployArchiveFiles []string
codeFlag := false
code := c.String("code")
if len(code) == 0 {
deployArchiveFiles = c.StringSlice("deploy")
} else {
deployArchiveFiles = append(deployArchiveFiles, c.String("code"))
codeFlag = true
}
srcArchiveName := c.String("src")
srcArchiveFiles := c.StringSlice("src")
pkgName := c.String("pkg")
entrypoint := c.String("entrypoint")
buildcmd := c.String("buildcmd")
@@ -418,7 +491,7 @@ func fnUpdate(c *cli.Context) error {
secretName := c.String("secret")
cfgMapName := c.String("configmap")
if len(srcArchiveName) > 0 && len(deployArchiveName) > 0 {
if len(srcArchiveFiles) > 0 && len(deployArchiveFiles) > 0 {
log.Fatal("Need either of --src or --deploy and not both arguments.")
}
@@ -479,24 +552,31 @@ func fnUpdate(c *cli.Context) error {
pkgName = function.Spec.Package.PackageRef.Name
}
strategy, err := getInvokeStrategy(c, &function.Spec.InvokeStrategy)
if err != nil {
log.Fatal(err)
}
function.Spec.InvokeStrategy = *strategy
function.Spec.Resources = getResourceReq(c, function.Spec.Resources)
pkg, err := client.PackageGet(&metav1.ObjectMeta{
Namespace: fnNamespace,
Name: pkgName,
})
checkErr(err, fmt.Sprintf("read package '%v.%v'. Pkg should be present in the same ns as the function", pkgName, fnNamespace))
util.CheckErr(err, fmt.Sprintf("read package '%v.%v'. Pkg should be present in the same ns as the function", pkgName, fnNamespace))
pkgMetadata := &pkg.Metadata
if len(deployArchiveName) != 0 || len(srcArchiveName) != 0 || len(buildcmd) != 0 || len(envName) != 0 || len(envNamespace) != 0 {
if len(deployArchiveFiles) != 0 || len(srcArchiveFiles) != 0 || len(buildcmd) != 0 || len(envName) != 0 || len(envNamespace) != 0 {
fnList, err := getFunctionsByPackage(client, pkg.Metadata.Name, pkg.Metadata.Namespace)
checkErr(err, "get function list")
util.CheckErr(err, "get function list")
if !force && len(fnList) > 1 {
log.Fatal("Package is used by multiple functions, use --force to force update")
}
pkgMetadata, err = updatePackage(client, pkg, envName, envNamespace, srcArchiveName, deployArchiveName, buildcmd, false)
checkErr(err, fmt.Sprintf("update package '%v'", pkgName))
pkgMetadata, err = updatePackage(client, pkg, envName, envNamespace, srcArchiveFiles, deployArchiveFiles, buildcmd, false, codeFlag)
util.CheckErr(err, fmt.Sprintf("update package '%v'", pkgName))
fmt.Printf("package '%v' updated\n", pkgMetadata.GetName())
@@ -506,7 +586,7 @@ func fnUpdate(c *cli.Context) error {
if fn.Metadata.Name != fnName {
fn.Spec.Package.PackageRef.ResourceVersion = pkgMetadata.ResourceVersion
_, err := client.FunctionUpdate(&fn)
checkErr(err, "update function")
util.CheckErr(err, "update function")
}
}
}
@@ -521,63 +601,21 @@ func fnUpdate(c *cli.Context) error {
ResourceVersion: pkgMetadata.ResourceVersion,
}
function.Spec.Resources = getResourceReq(c, function.Spec.Resources)
if c.IsSet("targetcpu") {
function.Spec.InvokeStrategy.ExecutionStrategy.TargetCPUPercent = getTargetCPU(c)
}
if c.IsSet("minscale") {
minscale := c.Int("minscale")
maxscale := c.Int("maxscale")
if c.IsSet("maxscale") && minscale > c.Int("maxscale") {
log.Fatal(fmt.Sprintf("Minscale's value %v can not be greater than maxscale value %v", minscale, maxscale))
}
if function.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType != fission.ExecutorTypePoolmgr &&
minscale > function.Spec.InvokeStrategy.ExecutionStrategy.MaxScale {
log.Fatal(fmt.Sprintf("Minscale provided: %v can not be greater than maxscale of existing function: %v", minscale,
function.Spec.InvokeStrategy.ExecutionStrategy.MaxScale))
}
function.Spec.InvokeStrategy.ExecutionStrategy.MinScale = minscale
}
if c.IsSet("maxscale") {
maxscale := c.Int("maxscale")
if maxscale < function.Spec.InvokeStrategy.ExecutionStrategy.MinScale {
log.Fatal(fmt.Sprintf("Function's minscale: %v can not be greater than maxscale provided: %v",
function.Spec.InvokeStrategy.ExecutionStrategy.MinScale, maxscale))
}
function.Spec.InvokeStrategy.ExecutionStrategy.MaxScale = maxscale
}
if c.IsSet("executortype") {
var fnExecutor fission.ExecutorType
switch c.String("executortype") {
case "":
fnExecutor = fission.ExecutorTypePoolmgr
case fission.ExecutorTypePoolmgr:
fnExecutor = fission.ExecutorTypePoolmgr
case fission.ExecutorTypeNewdeploy:
fnExecutor = fission.ExecutorTypeNewdeploy
default:
log.Fatal("Executor type must be one of 'poolmgr' or 'newdeploy', defaults to 'poolmgr'")
}
if (c.IsSet("mincpu") || c.IsSet("maxcpu") || c.IsSet("minmemory") || c.IsSet("maxmemory")) &&
fnExecutor == fission.ExecutorTypePoolmgr {
log.Warn("CPU/Memory specified for function with pool manager executor will be ignored in favor of resources specified at environment")
}
function.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType = fnExecutor
if function.Spec.Environment.Name != pkg.Spec.Environment.Name {
log.Warn("Function's environment is different than package's environment, package's environment will be used for updating function")
function.Spec.Environment.Name = pkg.Spec.Environment.Name
function.Spec.Environment.Namespace = pkg.Spec.Environment.Namespace
}
_, err = client.FunctionUpdate(function)
checkErr(err, "update function")
util.CheckErr(err, "update function")
fmt.Printf("function '%v' updated\n", fnName)
return err
}
func fnDelete(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
fnName := c.String("name")
if len(fnName) == 0 {
@@ -591,18 +629,18 @@ func fnDelete(c *cli.Context) error {
}
err := client.FunctionDelete(m)
checkErr(err, fmt.Sprintf("delete function '%v'", fnName))
util.CheckErr(err, fmt.Sprintf("delete function '%v'", fnName))
fmt.Printf("function '%v' deleted\n", fnName)
return err
}
func fnList(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
ns := c.String("fnNamespace")
fns, err := client.FunctionList(ns)
checkErr(err, "list functions")
util.CheckErr(err, "list functions")
w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
@@ -628,7 +666,7 @@ func fnList(c *cli.Context) error {
func fnLogs(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
fnName := c.String("name")
if len(fnName) == 0 {
@@ -653,10 +691,10 @@ func fnLogs(c *cli.Context) error {
}
f, err := client.FunctionGet(m)
checkErr(err, "get function")
util.CheckErr(err, "get function")
// request the controller to establish a proxy server to the database.
logDB, err := logdb.GetLogDB(dbType, getServerUrl())
logDB, err := logdb.GetLogDB(dbType, util.GetServerUrl())
if err != nil {
log.Fatal("failed to connect log database")
}
@@ -679,7 +717,7 @@ func fnLogs(c *cli.Context) error {
}
logEntries, err := logDB.GetLogs(logFilter)
if err != nil {
log.Fatal("failed to query logs")
log.Fatal(fmt.Sprintf("Error querying logs: %v", err))
}
for _, logEntry := range logEntries {
if c.Bool("d") {
@@ -718,8 +756,8 @@ func fnTest(c *cli.Context) error {
routerURL := os.Getenv("FISSION_ROUTER")
if len(routerURL) == 0 {
// Portforward to the fission router
localRouterPort := portforward.Setup(getKubeConfigPath(),
getFissionNamespace(), "application=fission-router")
localRouterPort := util.SetupPortForward(util.GetKubeConfigPath(),
util.GetFissionNamespace(), "application=fission-router")
routerURL = "127.0.0.1:" + localRouterPort
} else {
routerURL = strings.TrimPrefix(routerURL, "http://")
@@ -754,17 +792,19 @@ func fnTest(c *cli.Context) error {
functionUrl.RawQuery = query.Encode()
}
resp := httpRequest(c.String("method"), functionUrl.String(), c.String("body"), c.StringSlice("header"))
headers := c.StringSlice("header")
resp := httpRequest(c.String("method"), functionUrl.String(), c.String("body"), headers)
if resp.StatusCode < 400 {
body, err := ioutil.ReadAll(resp.Body)
checkErr(err, "Function test")
util.CheckErr(err, "Function test")
fmt.Print(string(body))
defer resp.Body.Close()
return nil
}
body, err := ioutil.ReadAll(resp.Body)
checkErr(err, "read log response from pod")
util.CheckErr(err, "read log response from pod")
fmt.Printf("Error calling function %s: %d %s", fnName, resp.StatusCode, string(body))
defer resp.Body.Close()
err = printPodLogs(c)
@@ -774,3 +814,34 @@ func fnTest(c *cli.Context) error {
return nil
}
func httpRequest(method, url, body string, headers []string) *http.Response {
if method == "" {
method = "GET"
}
if method != http.MethodGet &&
method != http.MethodDelete &&
method != http.MethodPost &&
method != http.MethodPut &&
method != http.MethodOptions {
log.Fatal(fmt.Sprintf("Invalid HTTP method '%s'.", method))
}
req, err := http.NewRequest(method, url, strings.NewReader(body))
util.CheckErr(err, "create HTTP request")
for _, header := range headers {
headerKeyValue := strings.SplitN(header, ":", 2)
if len(headerKeyValue) != 2 {
log.Fatal("Failed to create request without appropriate headers")
}
req.Header.Set(headerKeyValue[0], headerKeyValue[1])
}
client := &http.Client{}
resp, err := client.Do(req)
util.CheckErr(err, "execute HTTP request")
return resp
}
+289
View File
@@ -0,0 +1,289 @@
package main
import (
"flag"
"fmt"
"testing"
"github.com/stretchr/testify/assert"
"github.com/urfave/cli"
"github.com/fission/fission"
)
func TestGetInvokeStrategy(t *testing.T) {
cases := []struct {
testArgs map[string]string
existingInvokeStrategy *fission.InvokeStrategy
expectedResult *fission.InvokeStrategy
expectError bool
}{
{
// case: use default executor poolmgr
testArgs: map[string]string{},
existingInvokeStrategy: nil,
expectedResult: &fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fission.ExecutorTypePoolmgr,
},
},
expectError: false,
},
{
// case: executor type set to poolmgr
testArgs: map[string]string{"executortype": fission.ExecutorTypePoolmgr},
existingInvokeStrategy: nil,
expectedResult: &fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fission.ExecutorTypePoolmgr,
},
},
expectError: false,
},
{
// case: executor type set to newdeploy
testArgs: map[string]string{"executortype": fission.ExecutorTypeNewdeploy},
existingInvokeStrategy: nil,
expectedResult: &fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fission.ExecutorTypeNewdeploy,
MinScale: DEFAULT_MIN_SCALE,
MaxScale: DEFAULT_MIN_SCALE,
TargetCPUPercent: DEFAULT_TARGET_CPU_PERCENTAGE,
},
},
expectError: false,
},
{
// case: executor type change from poolmgr to newdeploy
testArgs: map[string]string{"executortype": fission.ExecutorTypeNewdeploy},
existingInvokeStrategy: &fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fission.ExecutorTypePoolmgr,
},
},
expectedResult: &fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fission.ExecutorTypeNewdeploy,
MinScale: DEFAULT_MIN_SCALE,
MaxScale: DEFAULT_MIN_SCALE,
TargetCPUPercent: DEFAULT_TARGET_CPU_PERCENTAGE,
},
},
expectError: false,
},
{
// case: executor type change from newdeploy to poolmgr
testArgs: map[string]string{"executortype": fission.ExecutorTypePoolmgr},
existingInvokeStrategy: &fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fission.ExecutorTypeNewdeploy,
MinScale: DEFAULT_MIN_SCALE,
MaxScale: DEFAULT_MIN_SCALE,
TargetCPUPercent: DEFAULT_TARGET_CPU_PERCENTAGE,
},
},
expectedResult: &fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fission.ExecutorTypePoolmgr,
},
},
expectError: false,
},
{
// case: minscale < maxscale
testArgs: map[string]string{
"executortype": fission.ExecutorTypeNewdeploy,
"minscale": "2",
"maxscale": "3",
},
existingInvokeStrategy: nil,
expectedResult: &fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fission.ExecutorTypeNewdeploy,
MinScale: 2,
MaxScale: 3,
TargetCPUPercent: DEFAULT_TARGET_CPU_PERCENTAGE,
},
},
expectError: false,
},
{
// case: minscale > maxscale
testArgs: map[string]string{
"executortype": fission.ExecutorTypeNewdeploy,
"minscale": "5",
"maxscale": "3",
},
existingInvokeStrategy: nil,
expectedResult: nil,
expectError: true,
},
{
// case: maxscale not specified
testArgs: map[string]string{
"executortype": fission.ExecutorTypeNewdeploy,
"minscale": "5",
},
existingInvokeStrategy: nil,
expectedResult: nil,
expectError: true,
},
{
// case: minscale not specified
testArgs: map[string]string{
"executortype": fission.ExecutorTypeNewdeploy,
"maxscale": "3",
},
existingInvokeStrategy: nil,
expectedResult: &fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fission.ExecutorTypeNewdeploy,
MinScale: DEFAULT_MIN_SCALE,
MaxScale: 3,
TargetCPUPercent: DEFAULT_TARGET_CPU_PERCENTAGE,
},
},
expectError: false,
},
{
// case: maxscale set to 0
testArgs: map[string]string{
"executortype": fission.ExecutorTypeNewdeploy,
"maxscale": "0",
},
existingInvokeStrategy: nil,
expectedResult: nil,
expectError: true,
},
{
// case: maxscale set to 9 when existing is 5
testArgs: map[string]string{
"executortype": fission.ExecutorTypeNewdeploy,
"maxscale": "9",
},
existingInvokeStrategy: &fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fission.ExecutorTypeNewdeploy,
MinScale: 2,
MaxScale: 5,
TargetCPUPercent: DEFAULT_TARGET_CPU_PERCENTAGE,
},
},
expectedResult: &fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fission.ExecutorTypeNewdeploy,
MinScale: 2,
MaxScale: 9,
TargetCPUPercent: DEFAULT_TARGET_CPU_PERCENTAGE,
},
},
expectError: false,
},
{
// case: change nothing for existing strategy
testArgs: map[string]string{
"executortype": fission.ExecutorTypeNewdeploy,
},
existingInvokeStrategy: &fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fission.ExecutorTypeNewdeploy,
MinScale: 2,
MaxScale: 5,
TargetCPUPercent: DEFAULT_TARGET_CPU_PERCENTAGE,
},
},
expectedResult: &fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fission.ExecutorTypeNewdeploy,
MinScale: 2,
MaxScale: 5,
TargetCPUPercent: DEFAULT_TARGET_CPU_PERCENTAGE,
},
},
expectError: false,
},
{
// case: set target cpu percentage
testArgs: map[string]string{
"executortype": fission.ExecutorTypeNewdeploy,
"targetcpu": "50",
},
existingInvokeStrategy: nil,
expectedResult: &fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fission.ExecutorTypeNewdeploy,
MinScale: DEFAULT_MIN_SCALE,
MaxScale: DEFAULT_MIN_SCALE,
TargetCPUPercent: 50,
},
},
expectError: false,
},
{
// case: change target cpu percentage
testArgs: map[string]string{
"executortype": fission.ExecutorTypeNewdeploy,
"targetcpu": "20",
},
existingInvokeStrategy: &fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fission.ExecutorTypeNewdeploy,
MinScale: 2,
MaxScale: 5,
TargetCPUPercent: 88,
},
},
expectedResult: &fission.InvokeStrategy{
StrategyType: fission.StrategyTypeExecution,
ExecutionStrategy: fission.ExecutionStrategy{
ExecutorType: fission.ExecutorTypeNewdeploy,
MinScale: 2,
MaxScale: 5,
TargetCPUPercent: 20,
},
},
expectError: false,
},
}
for i, c := range cases {
fmt.Printf("=== Test Case %v ===\n", i)
app := newCliApp()
set := flag.NewFlagSet("test-cmd", 0)
ctx := cli.NewContext(app, set, nil)
for k, v := range c.testArgs {
set.String(k, v, "")
ctx.Set(k, v)
}
strategy, err := getInvokeStrategy(ctx, c.existingInvokeStrategy)
if c.expectError {
assert.NotNil(t, err)
if err != nil {
fmt.Println(err)
}
} else {
assert.Nil(t, err)
assert.NoError(t, strategy.Validate(), fmt.Sprintf("Failed at test case %v", i))
assert.Equal(t, *c.expectedResult, *strategy)
}
}
}
+131 -36
View File
@@ -23,12 +23,12 @@ import (
"strings"
"text/tabwriter"
"github.com/fission/fission/fission/util"
"github.com/satori/go.uuid"
"github.com/urfave/cli"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission"
"github.com/fission/fission/controller/client"
"github.com/fission/fission/crd"
"github.com/fission/fission/fission/log"
)
@@ -59,26 +59,64 @@ func getMethod(method string) string {
return ""
}
func checkFunctionExistence(fissionClient *client.Client, fnName string, fnNamespace string) {
meta := &metav1.ObjectMeta{
Name: fnName,
Namespace: fnNamespace,
func setHtFunctionRef(functionList []string, functionWeightsList []int) (*fission.FunctionReference, error) {
if len(functionList) == 1 {
return &fission.FunctionReference{
Type: fission.FunctionReferenceTypeFunctionName,
Name: functionList[0],
}, nil
} else if len(functionList) == 2 {
if len(functionWeightsList) != 2 {
return nil, fmt.Errorf("weights of the function need to be specified when 2 functions are supplied")
}
totalWeight := functionWeightsList[0] + functionWeightsList[1]
if totalWeight != 100 {
log.Fatal("The function weights should add up to 100")
}
functionWeights := make(map[string]int, 0)
for index := range functionList {
functionWeights[functionList[index]] = functionWeightsList[index]
}
return &fission.FunctionReference{
Type: fission.FunctionReferenceTypeFunctionWeights,
FunctionWeights: functionWeights,
}, nil
}
_, err := fissionClient.FunctionGet(meta)
if err != nil {
fmt.Printf("function '%v' does not exist, use 'fission function create --name %v ...' to create the function\n", fnName, fnName)
}
return nil, fmt.Errorf("the number of functions in a trigger can be 1 or 2(for canary feature along with their weights)")
}
func htCreate(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
fnName := c.String("function")
if len(fnName) == 0 {
functionList := c.StringSlice("function")
functionWeightsList := c.IntSlice("weight")
if len(functionList) == 0 {
log.Fatal("Need a function name to create a trigger, use --function")
}
functionRef, err := setHtFunctionRef(functionList, functionWeightsList)
if err != nil {
log.Fatal(err.Error())
}
triggerName := c.String("name")
fnNamespace := c.String("fnNamespace")
spec := c.Bool("spec")
m := &metav1.ObjectMeta{
Name: triggerName,
Namespace: fnNamespace,
}
htTrigger, err := client.HTTPTriggerGet(m)
if htTrigger != nil {
util.CheckErr(fmt.Errorf("duplicate trigger exists"), "choose a different name or leave it empty for fission to auto-generate it")
}
triggerUrl := c.String("url")
if len(triggerUrl) == 0 {
@@ -93,7 +131,14 @@ func htCreate(c *cli.Context) error {
method = "GET"
}
checkFunctionExistence(client, fnName, fnNamespace)
// For Specs, the spec validate checks for function reference
if !spec {
err = util.CheckFunctionExistence(client, functionList, fnNamespace)
if err != nil {
log.Warn(err.Error())
}
}
createIngress := false
if c.IsSet("createingress") {
createIngress = c.Bool("createingress")
@@ -102,7 +147,9 @@ func htCreate(c *cli.Context) error {
host := c.String("host")
// just name triggers by uuid.
triggerName := uuid.NewV4().String()
if triggerName == "" {
triggerName = uuid.NewV4().String()
}
ht := &crd.HTTPTrigger{
Metadata: metav1.ObjectMeta{
@@ -110,38 +157,67 @@ func htCreate(c *cli.Context) error {
Namespace: fnNamespace,
},
Spec: fission.HTTPTriggerSpec{
Host: host,
RelativeURL: triggerUrl,
Method: getMethod(method),
FunctionReference: fission.FunctionReference{
Type: fission.FunctionReferenceTypeFunctionName,
Name: fnName,
},
CreateIngress: createIngress,
Host: host,
RelativeURL: triggerUrl,
Method: getMethod(method),
FunctionReference: *functionRef,
CreateIngress: createIngress,
},
}
// if we're writing a spec, don't call the API
if c.Bool("spec") {
if spec {
specFile := fmt.Sprintf("route-%v.yaml", triggerName)
err := specSave(*ht, specFile)
checkErr(err, "create HTTP trigger spec")
util.CheckErr(err, "create HTTP trigger spec")
return nil
}
_, err := client.HTTPTriggerCreate(ht)
checkErr(err, "create HTTP trigger")
_, err = client.HTTPTriggerCreate(ht)
util.CheckErr(err, "create HTTP trigger")
fmt.Printf("trigger '%v' created\n", triggerName)
return err
}
func htGet(c *cli.Context) error {
return nil
cliClient := util.GetApiClient(c.GlobalString("server"))
name := c.String("name")
ns := c.String("fnNamespace")
m := &metav1.ObjectMeta{
Name: name,
Namespace: ns,
}
htTrigger, err := cliClient.HTTPTriggerGet(m)
util.CheckErr(err, "get http trigger")
w := tabwriter.NewWriter(os.Stdout, 0, 1, 1, ' ', 0)
fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\n", "NAME", "UID", "METHOD", "RELATIVE-URL", "FUNCTION-REFERENCE-TYPE", "FUNCTION(s)")
function := ""
if htTrigger.Spec.FunctionReference.Type == fission.FunctionReferenceTypeFunctionName {
function = htTrigger.Spec.FunctionReference.Name
} else {
for k, v := range htTrigger.Spec.FunctionReference.FunctionWeights {
function += fmt.Sprintf("%s:%v ", k, v)
}
}
fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\n",
htTrigger.Metadata.Name, htTrigger.Metadata.UID, htTrigger.Spec.Method, htTrigger.Spec.RelativeURL,
htTrigger.Spec.FunctionReference.Type, function)
w.Flush()
return err
}
func htUpdate(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
htName := c.String("name")
if len(htName) == 0 {
log.Fatal("Need name of trigger, use --name")
@@ -152,12 +228,31 @@ func htUpdate(c *cli.Context) error {
Name: htName,
Namespace: triggerNamespace,
})
checkErr(err, "get HTTP trigger")
util.CheckErr(err, "get HTTP trigger")
if c.IsSet("function") {
ht.Spec.FunctionReference.Name = c.String("function")
// get the functions and their weights if specified
functionList := c.StringSlice("function")
err := util.CheckFunctionExistence(client, functionList, triggerNamespace)
if err != nil {
if err != nil {
log.Warn(err.Error())
}
}
var functionWeightsList []int
if c.IsSet("weight") {
functionWeightsList = c.IntSlice("weight")
}
// set function reference
functionRef, err := setHtFunctionRef(functionList, functionWeightsList)
if err != nil {
log.Fatal(err.Error())
}
ht.Spec.FunctionReference = *functionRef
}
checkFunctionExistence(client, ht.Spec.FunctionReference.Name, triggerNamespace)
if c.IsSet("createingress") {
ht.Spec.CreateIngress = c.Bool("createingress")
@@ -168,14 +263,14 @@ func htUpdate(c *cli.Context) error {
}
_, err = client.HTTPTriggerUpdate(ht)
checkErr(err, "update HTTP trigger")
util.CheckErr(err, "update HTTP trigger")
fmt.Printf("trigger '%v' updated\n", htName)
return nil
}
func htDelete(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
htName := c.String("name")
if len(htName) == 0 {
log.Fatal("Need name of trigger to delete, use --name")
@@ -186,18 +281,18 @@ func htDelete(c *cli.Context) error {
Name: htName,
Namespace: triggerNamespace,
})
checkErr(err, "delete trigger")
util.CheckErr(err, "delete trigger")
fmt.Printf("trigger '%v' deleted\n", htName)
return nil
}
func htList(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
triggerNamespace := c.String("triggerNamespace")
hts, err := client.HTTPTriggerList(triggerNamespace)
checkErr(err, "list HTTP triggers")
util.CheckErr(err, "list HTTP triggers")
w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
+5 -1
View File
@@ -27,11 +27,15 @@ var (
)
func Fatal(msg interface{}) {
os.Stderr.WriteString(fmt.Sprintf("%v\n", msg))
os.Stderr.WriteString(fmt.Sprintf("Fatal error: %v\n", msg))
os.Exit(1)
}
func Warn(msg interface{}) {
os.Stderr.WriteString(fmt.Sprintf("Warning: %v\n", msg))
}
func Info(msg interface{}) {
os.Stderr.WriteString(fmt.Sprintf("%v\n", msg))
}
+99 -84
View File
@@ -19,64 +19,38 @@ package main
import (
"fmt"
"os"
"path/filepath"
"strings"
"github.com/ghodss/yaml"
"github.com/pkg/errors"
"github.com/urfave/cli"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission"
"github.com/fission/fission/fission/log"
"github.com/fission/fission/fission/plugin"
"github.com/fission/fission/fission/portforward"
"github.com/fission/fission/fission/support"
"github.com/fission/fission/fission/util"
)
func getFissionNamespace() string {
fissionNamespace := os.Getenv("FISSION_NAMESPACE")
return fissionNamespace
}
func getKubeConfigPath() string {
kubeConfig := os.Getenv("KUBECONFIG")
if len(kubeConfig) == 0 {
home := os.Getenv("HOME")
kubeConfig = filepath.Join(home, ".kube", "config")
if _, err := os.Stat(kubeConfig); os.IsNotExist(err) {
log.Fatal("Couldn't find kubeconfig file. " +
"Set the KUBECONFIG environment variable to your kubeconfig's path.")
}
}
return kubeConfig
}
func getServerUrl() string {
return getApplicationUrl("application=fission-api")
}
func getApplicationUrl(selector string) string {
var serverUrl string
// Use FISSION_URL env variable if set; otherwise, port-forward to controller.
fissionUrl := os.Getenv("FISSION_URL")
if len(fissionUrl) == 0 {
fissionNamespace := getFissionNamespace()
kubeConfig := getKubeConfigPath()
localPort := portforward.Setup(kubeConfig, fissionNamespace, "application=fission-api")
serverUrl = "http://127.0.0.1:" + localPort
} else {
serverUrl = fissionUrl
}
return serverUrl
}
func cliHook(c *cli.Context) error {
log.Verbosity = c.Int("verbosity")
log.Verbose(2, "Verbosity = 2")
err := flagValueParser(c.Args())
if err != nil {
// The cli package wont't print out error, as a workaround we need to
// fatal here instead of return it.
log.Fatal(err)
}
return nil
}
func main() {
newCliApp().Run(os.Args)
}
func newCliApp() *cli.App {
app := cli.NewApp()
app.Name = "fission"
app.Usage = "Serverless functions for Kubernetes"
@@ -106,26 +80,27 @@ func main() {
pkgNamespaceFlag := cli.StringFlag{Name: "pkgNamespace, pkgns", Value: metav1.NamespaceDefault, Usage: "Namespace for package object"}
triggerNamespaceFlag := cli.StringFlag{Name: "triggerNamespace, triggerns", Value: metav1.NamespaceDefault, Usage: "Namespace for trigger object"}
recorderNamespaceFlag := cli.StringFlag{Name: "recorderNamespace, recorderns", Value: metav1.NamespaceDefault, Usage: "Namespace for recorder object"}
canaryNamespaceFlag := cli.StringFlag{Name: "canaryNamespace, canaryns", Value: metav1.NamespaceDefault, Usage: "Namespace for canary config object"}
// trigger method and url flags (used in function and route CLIs)
htMethodFlag := cli.StringFlag{Name: "method", Value: "GET", Usage: "HTTP Method: GET|POST|PUT|DELETE|HEAD"}
htUrlFlag := cli.StringFlag{Name: "url", Usage: "URL pattern (See gorilla/mux supported patterns)"}
// Resource & scale related flags (Used in env and function)
minCpu := cli.StringFlag{Name: "mincpu", Usage: "Minimum CPU to be assigned to pod (In millicore, minimum 1)"}
maxCpu := cli.StringFlag{Name: "maxcpu", Usage: "Maximum CPU to be assigned to pod (In millicore, minimum 1)"}
minMem := cli.StringFlag{Name: "minmemory", Usage: "Minimum memory to be assigned to pod (In megabyte)"}
maxMem := cli.StringFlag{Name: "maxmemory", Usage: "Maximum memory to be assigned to pod (In megabyte)"}
minScale := cli.StringFlag{Name: "minscale", Usage: "Minimum number of pods (Uses resource inputs to configure HPA)"}
maxScale := cli.StringFlag{Name: "maxscale", Usage: "Maximum number of pods (Uses resource inputs to configure HPA)"}
targetcpu := cli.IntFlag{Name: "targetcpu", Value: 80, Usage: "Target average CPU usage percentage across pods for scaling"}
minCpu := cli.IntFlag{Name: "mincpu", Usage: "Minimum CPU to be assigned to pod (In millicore, minimum 1)"}
maxCpu := cli.IntFlag{Name: "maxcpu", Usage: "Maximum CPU to be assigned to pod (In millicore, minimum 1)"}
minMem := cli.IntFlag{Name: "minmemory", Usage: "Minimum memory to be assigned to pod (In megabyte)"}
maxMem := cli.IntFlag{Name: "maxmemory", Usage: "Maximum memory to be assigned to pod (In megabyte)"}
minScale := cli.IntFlag{Name: "minscale", Usage: "Minimum number of pods (Uses resource inputs to configure HPA)"}
maxScale := cli.IntFlag{Name: "maxscale", Usage: "Maximum number of pods (Uses resource inputs to configure HPA)"}
targetcpu := cli.IntFlag{Name: "targetcpu", Usage: "Target average CPU usage percentage across pods for scaling"}
// functions
fnNameFlag := cli.StringFlag{Name: "name", Usage: "function name"}
fnEnvNameFlag := cli.StringFlag{Name: "env", Usage: "environment name for function"}
fnCodeFlag := cli.StringFlag{Name: "code", Usage: "local path or URL for source code"}
fnDeployArchiveFlag := cli.StringFlag{Name: "deployarchive, deploy", Usage: "local path or URL for deployment archive"}
fnSrcArchiveFlag := cli.StringFlag{Name: "sourcearchive, src", Usage: "local path or URL for source archive"}
fnDeployArchiveFlag := cli.StringSliceFlag{Name: "deployarchive, deploy", Usage: "local path or URL for deployment archive"}
fnSrcArchiveFlag := cli.StringSliceFlag{Name: "sourcearchive, src, source", Usage: "local path or URL for source archive"}
fnPkgNameFlag := cli.StringFlag{Name: "pkgname, pkg", Usage: "Name of the existing package (--deploy and --src and --env will be ignored), should be in the same namespace as the function"}
fnPodFlag := cli.StringFlag{Name: "pod", Usage: "function pod name, optional (use latest if unspecified)"}
fnFollowFlag := cli.BoolFlag{Name: "follow, f", Usage: "specify if the logs should be streamed"}
@@ -140,7 +115,7 @@ func main() {
fnCfgMapFlag := cli.StringFlag{Name: "configmap", Usage: "function access to configmap, should be present in the same namespace as the function"}
fnLogCountFlag := cli.StringFlag{Name: "recordcount", Usage: "the n most recent log records"}
fnForceFlag := cli.BoolFlag{Name: "force", Usage: "Force update a package even if it is used by one or more functions"}
fnExecutorTypeFlag := cli.StringFlag{Name: "executortype", Usage: "Executor type for execution; one of 'poolmgr', 'newdeploy' defaults to 'poolmgr'"}
fnExecutorTypeFlag := cli.StringFlag{Name: "executortype", Value: fission.ExecutorTypePoolmgr, Usage: "Executor type for execution; one of 'poolmgr', 'newdeploy' defaults to 'poolmgr'"}
fnSubcommands := []cli.Command{
{Name: "create", Usage: "Create new function (and optionally, an HTTP route to it)", Flags: []cli.Flag{fnNameFlag, fnNamespaceFlag, fnEnvNameFlag, envNamespaceFlag, specSaveFlag, fnCodeFlag, fnSrcArchiveFlag, fnDeployArchiveFlag, fnEntryPointFlag, fnBuildCmdFlag, fnPkgNameFlag, htUrlFlag, htMethodFlag, minCpu, maxCpu, minMem, maxMem, minScale, maxScale, fnExecutorTypeFlag, targetcpu, fnCfgMapFlag, fnSecretFlag}, Action: fnCreate},
@@ -157,14 +132,16 @@ func main() {
// httptriggers
htNameFlag := cli.StringFlag{Name: "name", Usage: "HTTP Trigger name"}
htFnNameFlag := cli.StringFlag{Name: "function", Usage: "Function name"}
htHostFlag := cli.StringFlag{Name: "host", Usage: "FQDN of the network host for route"}
htIngressFlag := cli.BoolFlag{Name: "createingress", Usage: "Creates ingress with same URL, defaults to false"}
htFnNameFlag := cli.StringSliceFlag{Name: "function", Usage: "Name(s) of the function for this trigger. If 2 functions are supplied with this flag, traffic gets routed to them based on weights supplied with --weight flag."}
htFnWeightFlag := cli.IntSliceFlag{Name: "weight", Usage: "Weight for each function supplied with --function flag, in the same order. Used for canary deployment"}
htSubcommands := []cli.Command{
{Name: "create", Aliases: []string{"add"}, Usage: "Create HTTP trigger", Flags: []cli.Flag{htMethodFlag, htUrlFlag, htFnNameFlag, htHostFlag, htIngressFlag, fnNamespaceFlag, specSaveFlag}, Action: htCreate},
{Name: "get", Usage: "Get HTTP trigger", Flags: []cli.Flag{htMethodFlag, htUrlFlag}, Action: htGet},
{Name: "update", Usage: "Update HTTP trigger", Flags: []cli.Flag{htNameFlag, triggerNamespaceFlag, htFnNameFlag, htHostFlag, htIngressFlag}, Action: htUpdate},
{Name: "create", Aliases: []string{"add"}, Usage: "Create HTTP trigger", Flags: []cli.Flag{htNameFlag, htMethodFlag, htUrlFlag, htFnNameFlag, htHostFlag, htIngressFlag, fnNamespaceFlag, specSaveFlag, htFnWeightFlag}, Action: htCreate},
{Name: "get", Usage: "Get HTTP trigger", Flags: []cli.Flag{htNameFlag}, Action: htGet},
{Name: "update", Usage: "Update HTTP trigger", Flags: []cli.Flag{htNameFlag, triggerNamespaceFlag, htFnNameFlag, htHostFlag, htIngressFlag, htFnWeightFlag}, Action: htUpdate},
{Name: "delete", Usage: "Delete HTTP trigger", Flags: []cli.Flag{htNameFlag, triggerNamespaceFlag}, Action: htDelete},
{Name: "list", Usage: "List HTTP triggers", Flags: []cli.Flag{triggerNamespaceFlag}, Action: htList},
}
@@ -266,8 +243,8 @@ func main() {
pkgNameFlag := cli.StringFlag{Name: "name", Usage: "Package name"}
pkgForceFlag := cli.BoolFlag{Name: "force, f", Usage: "Force update a package even if it is used by one or more functions"}
pkgEnvironmentFlag := cli.StringFlag{Name: "env", Usage: "Environment name"}
pkgSrcArchiveFlag := cli.StringFlag{Name: "sourcearchive, src", Usage: "Local path or URL for source archive"}
pkgDeployArchiveFlag := cli.StringFlag{Name: "deployarchive, deploy", Usage: "Local path or URL for binary archive"}
pkgSrcArchiveFlag := cli.StringSliceFlag{Name: "sourcearchive, src", Usage: "Local path or URL for source archive"}
pkgDeployArchiveFlag := cli.StringSliceFlag{Name: "deployarchive, deploy", Usage: "Local path or URL for binary archive"}
pkgBuildCmdFlag := cli.StringFlag{Name: "buildcmd", Usage: "Build command for builder to run with"}
pkgOutputFlag := cli.StringFlag{Name: "output, o", Usage: "Output filename to save archive content"}
pkgOrphanFlag := cli.BoolFlag{Name: "orphan", Usage: "orphan packages that are not referenced by any function"}
@@ -300,7 +277,30 @@ func main() {
{Name: "validate", Usage: "Validate Fission app specification", Flags: []cli.Flag{specDirFlag}, Action: specValidate},
{Name: "apply", Usage: "Create, update, or delete Fission resources from app specification", Flags: []cli.Flag{specDirFlag, specDeleteFlag, specWaitFlag, specWatchFlag}, Action: specApply},
{Name: "destroy", Usage: "Delete all Fission resources in the app specification", Flags: []cli.Flag{specDirFlag}, Action: specDestroy},
{Name: "helm", Usage: "Create a helm chart from the app specification", Flags: []cli.Flag{specDirFlag}, Action: specHelm},
{Name: "helm", Usage: "Create a helm chart from the app specification", Flags: []cli.Flag{specDirFlag}, Action: specHelm, Hidden: true},
}
// support
supportOutputFlag := cli.StringFlag{Name: "output, o", Value: support.DEFAULT_OUTPUT_DIR, Usage: "Output directory to save dump archive/files"}
supportNoZipFlag := cli.BoolFlag{Name: "nozip", Usage: "Save dump information into multiple files instead of single zip file"}
supportSubCommands := []cli.Command{
{Name: "dump", Usage: "Collect & dump all necessary for troubleshooting", Flags: []cli.Flag{supportOutputFlag, supportNoZipFlag}, Action: support.DumpInfo},
}
// canary configs
canaryConfigNameFlag := cli.StringFlag{Name: "name", Usage: "Name for the canary config"}
triggerNameFlag := cli.StringFlag{Name: "httptrigger", Usage: "Http trigger that this config references"}
newFunc := cli.StringFlag{Name: "newfunction", Usage: "New version of the function"}
oldFunc := cli.StringFlag{Name: "oldfunction", Usage: "Old stable version of the function"}
weightIncrementFlag := cli.IntFlag{Name: "increment-step", Value: 20, Usage: "Weight increment step for function"}
incrementIntervalFlag := cli.StringFlag{Name: "increment-interval", Value: "2m", Usage: "Weight increment interval, string representation of time.Duration, ex : 1m, 2h, 2d"}
failureThresholdFlag := cli.IntFlag{Name: "failure-threshold", Value: 10, Usage: "Threshold in percentage beyond which the new version of the function is considered unstable"}
canarySubCommands := []cli.Command{
{Name: "create", Usage: "Create a canary config", Flags: []cli.Flag{canaryConfigNameFlag, triggerNameFlag, newFunc, oldFunc, fnNamespaceFlag, weightIncrementFlag, incrementIntervalFlag, failureThresholdFlag}, Action: canaryConfigCreate},
{Name: "get", Usage: "View parameters in a canary config", Flags: []cli.Flag{canaryConfigNameFlag, canaryNamespaceFlag}, Action: canaryConfigGet},
{Name: "update", Usage: "Update parameters of a canary config", Flags: []cli.Flag{canaryConfigNameFlag, canaryNamespaceFlag, incrementIntervalFlag, weightIncrementFlag, failureThresholdFlag}, Action: canaryConfigUpdate},
{Name: "delete", Usage: "Delete a canary config", Flags: []cli.Flag{canaryConfigNameFlag, canaryNamespaceFlag}, Action: canaryConfigDelete},
{Name: "list", Usage: "List all canary configs in a namespace", Flags: []cli.Flag{canaryNamespaceFlag}, Action: canaryConfigList},
}
app.Commands = []cli.Command{
@@ -316,11 +316,14 @@ func main() {
{Name: "package", Aliases: []string{"pkg"}, Usage: "Manage packages", Subcommands: pkgSubCommands},
{Name: "spec", Aliases: []string{"specs"}, Usage: "Manage a declarative app specification", Subcommands: specSubCommands},
{Name: "upgrade", Aliases: []string{}, Usage: "Upgrade tool from fission v0.1", Subcommands: upgradeSubCommands},
{Name: "support", Usage: "Collect an archive of diagnostic information for support", Subcommands: supportSubCommands},
cmdPlugin,
{Name: "canary-config", Aliases: []string{}, Usage: "Create, Update and manage Canary Configs", Subcommands: canarySubCommands},
}
app.Before = cliHook
app.CommandNotFound = handleCommandNotFound
app.Run(os.Args)
return app
}
func handleCommandNotFound(ctx *cli.Context, subCommand string) {
@@ -361,40 +364,52 @@ To install it for your local Fission CLI:
}
}
// Versions is a container of versions of the client (and its plugins) and server (and its plugins).
type Versions struct {
Client map[string]fission.BuildMeta `json:"client"`
Server map[string]fission.BuildMeta `json:"server"`
func versionPrinter(_ *cli.Context) {
client := util.GetApiClient(util.GetServerUrl())
ver := util.GetVersion(client)
fmt.Print(string(ver))
}
func versionPrinter(_ *cli.Context) {
serverInfo, err := getClient(getServerUrl()).ServerInfo()
if err != nil {
log.Warn(fmt.Sprintf("Error getting Fission API version: %v", err))
func flagValueParser(args []string) error {
// all input value for flags are properly set
if len(args) == 0 {
return nil
}
// Fetch client versions
versions := Versions{
Client: map[string]fission.BuildMeta{
"fission/core": fission.BuildInfo(),
},
}
for _, pmd := range plugin.FindAll() {
versions.Client[pmd.Name] = fission.BuildMeta{
Version: pmd.Version,
var flagIndexes []int
var errorFlags []string
// find out all flag indexes
for i, v := range args {
// support both flags with "--" and "-"
if strings.HasPrefix(v, "-") {
flagIndexes = append(flagIndexes, i)
}
}
// Fetch server versions
versions.Server = map[string]fission.BuildMeta{
"fission/core": serverInfo.Build,
// add total length of args to indicate the end of args
flagIndexes = append(flagIndexes, len(args))
for i := 0; i < len(flagIndexes)-1; i++ {
// if the difference between the flag index i and i+1
// is bigger then 2 means that CLI receives extra arguments
// for one flag. For example,
// 1. fission fn create --name e1 --code examples/nodejs/* --env nodejs ...
// The wildcard will be extracted to multiple files and cause the difference between `--code` and `--env` large than 2.
// 2. fission fn create --spec --name e1 ...
// The difference between --spec and --name is 1.
if flagIndexes[i+1]-flagIndexes[i] > 2 {
index := flagIndexes[i]
errorFlags = append(errorFlags, args[index])
}
}
// FUTURE: fetch versions of plugins server-side
bs, err := yaml.Marshal(versions)
if err != nil {
log.Fatal("Failed to format versions: " + err.Error())
if len(errorFlags) > 0 {
e := fmt.Sprintf("Unable to parse flags: %v\nThe argument should have only one input value. Please quote the input value if it contains wildcard characters(*).", strings.Join(errorFlags[:], ", "))
return errors.New(e)
}
fmt.Print(string(bs))
return nil
}
var helpTemplate = `NAME:
+17 -12
View File
@@ -21,6 +21,7 @@ import (
"os"
"text/tabwriter"
"github.com/fission/fission/fission/util"
"github.com/satori/go.uuid"
"github.com/urfave/cli"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -32,7 +33,7 @@ import (
)
func mqtCreate(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
mqtName := c.String("name")
if len(mqtName) == 0 {
@@ -52,14 +53,18 @@ func mqtCreate(c *cli.Context) error {
mqType = fission.MessageQueueTypeNats
case fission.MessageQueueTypeASQ:
mqType = fission.MessageQueueTypeASQ
case fission.MessageQueueTypeKafka:
mqType = fission.MessageQueueTypeKafka
default:
log.Fatal("Unknown message queue type, currently only \"nats-streaming, azure-storage-queue \" is supported")
log.Fatal("Unknown message queue type, currently only \"nats-streaming, azure-storage-queue, kafka \" is supported")
}
// TODO: check topic availability
topic := c.String("topic")
if len(topic) == 0 {
log.Fatal("Listen topic cannot be empty")
log.Fatal("Topic cannot be empty")
}
respTopic := c.String("resptopic")
@@ -107,12 +112,12 @@ func mqtCreate(c *cli.Context) error {
if c.Bool("spec") {
specFile := fmt.Sprintf("mqtrigger-%v.yaml", mqtName)
err := specSave(*mqt, specFile)
checkErr(err, "create message queue trigger spec")
util.CheckErr(err, "create message queue trigger spec")
return nil
}
_, err := client.MessageQueueTriggerCreate(mqt)
checkErr(err, "create message queue trigger")
util.CheckErr(err, "create message queue trigger")
fmt.Printf("trigger '%s' created\n", mqtName)
return err
@@ -123,7 +128,7 @@ func mqtGet(c *cli.Context) error {
}
func mqtUpdate(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
mqtName := c.String("name")
if len(mqtName) == 0 {
log.Fatal("Need name of trigger, use --name")
@@ -141,7 +146,7 @@ func mqtUpdate(c *cli.Context) error {
Name: mqtName,
Namespace: mqtNs,
})
checkErr(err, "get Time trigger")
util.CheckErr(err, "get Time trigger")
// TODO : Find out if we can make a call to checkIfFunctionExists, in the same ns more importantly.
@@ -178,14 +183,14 @@ func mqtUpdate(c *cli.Context) error {
}
_, err = client.MessageQueueTriggerUpdate(mqt)
checkErr(err, "update Time trigger")
util.CheckErr(err, "update Time trigger")
fmt.Printf("trigger '%v' updated\n", mqtName)
return nil
}
func mqtDelete(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
mqtName := c.String("name")
if len(mqtName) == 0 {
log.Fatal("Need name of trigger to delete, use --name")
@@ -196,18 +201,18 @@ func mqtDelete(c *cli.Context) error {
Name: mqtName,
Namespace: mqtNs,
})
checkErr(err, "delete trigger")
util.CheckErr(err, "delete trigger")
fmt.Printf("trigger '%v' deleted\n", mqtName)
return nil
}
func mqtList(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
mqtNs := c.String("triggerns")
mqts, err := client.MessageQueueTriggerList(c.String("mqtype"), mqtNs)
checkErr(err, "list message queue triggers")
util.CheckErr(err, "list message queue triggers")
w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
+341 -38
View File
@@ -18,13 +18,24 @@ package main
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"fmt"
"io"
"io/ioutil"
"net/http"
"net/url"
"os"
"path"
"path/filepath"
"strings"
"text/tabwriter"
"github.com/dchest/uniuri"
"github.com/fission/fission/fission/util"
storageSvcClient "github.com/fission/fission/storagesvc/client"
"github.com/mholt/archiver"
"github.com/satori/go.uuid"
"github.com/urfave/cli"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -59,12 +70,12 @@ func downloadStoragesvcURL(client *client.Client, fileUrl string) io.ReadCloser
fileDownloadUrl := strings.TrimSuffix(client.Url, "/") + "/proxy/storage/" + u.RequestURI()
reader, err := downloadURL(fileDownloadUrl)
checkErr(err, fmt.Sprintf("download from storage service url: %v", fileUrl))
util.CheckErr(err, fmt.Sprintf("download from storage service url: %v", fileUrl))
return reader
}
func pkgCreate(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
pkgNamespace := c.String("pkgNamespace")
envName := c.String("env")
@@ -72,22 +83,21 @@ func pkgCreate(c *cli.Context) error {
log.Fatal("Need --env argument.")
}
envNamespace := c.String("envNamespace")
srcArchiveName := c.String("src")
deployArchiveName := c.String("deploy")
srcArchiveFiles := c.StringSlice("src")
deployArchiveFiles := c.StringSlice("deploy")
buildcmd := c.String("buildcmd")
if len(srcArchiveName) == 0 && len(deployArchiveName) == 0 {
if len(srcArchiveFiles) == 0 && len(deployArchiveFiles) == 0 {
log.Fatal("Need --src to specify source archive, or use --deploy to specify deployment archive.")
}
meta := createPackage(client, pkgNamespace, envName, envNamespace, srcArchiveName, deployArchiveName, buildcmd, "")
fmt.Printf("Package '%v' created\n", meta.GetName())
createPackage(client, pkgNamespace, envName, envNamespace, srcArchiveFiles, deployArchiveFiles, buildcmd, "", "", false)
return nil
}
func pkgUpdate(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
pkgName := c.String("name")
if len(pkgName) == 0 {
@@ -98,15 +108,15 @@ func pkgUpdate(c *cli.Context) error {
force := c.Bool("f")
envName := c.String("env")
envNamespace := c.String("envNamespace")
srcArchiveName := c.String("src")
deployArchiveName := c.String("deploy")
srcArchiveFiles := c.StringSlice("src")
deployArchiveFiles := c.StringSlice("deploy")
buildcmd := c.String("buildcmd")
if len(srcArchiveName) > 0 && len(deployArchiveName) > 0 {
if len(srcArchiveFiles) > 0 && len(deployArchiveFiles) > 0 {
log.Fatal("Need either of --src or --deploy and not both arguments.")
}
if len(srcArchiveName) == 0 && len(deployArchiveName) == 0 &&
if len(srcArchiveFiles) == 0 && len(deployArchiveFiles) == 0 &&
len(envName) == 0 && len(buildcmd) == 0 {
log.Fatal("Need --env or --src or --deploy or --buildcmd argument.")
}
@@ -115,7 +125,7 @@ func pkgUpdate(c *cli.Context) error {
Namespace: pkgNamespace,
Name: pkgName,
})
checkErr(err, "get package")
util.CheckErr(err, "get package")
// if the new env specified is the same as the old one, no need to update package
// same is true for all update parameters, but, for now, we dont check all of them - because, its ok to
@@ -129,23 +139,23 @@ func pkgUpdate(c *cli.Context) error {
}
fnList, err := getFunctionsByPackage(client, pkg.Metadata.Name, pkg.Metadata.Namespace)
checkErr(err, "get function list")
util.CheckErr(err, "get function list")
if !force && len(fnList) > 1 {
log.Fatal("Package is used by multiple functions, use --force to force update")
}
newPkgMeta, err := updatePackage(client, pkg,
envName, envNamespace, srcArchiveName, deployArchiveName, buildcmd, false)
envName, envNamespace, srcArchiveFiles, deployArchiveFiles, buildcmd, false, false)
if err != nil {
checkErr(err, "update package")
util.CheckErr(err, "update package")
}
// update resource version of package reference of functions that shared the same package
for _, fn := range fnList {
fn.Spec.Package.PackageRef.ResourceVersion = newPkgMeta.ResourceVersion
_, err := client.FunctionUpdate(&fn)
checkErr(err, "update function")
util.CheckErr(err, "update function")
}
fmt.Printf("Package '%v' updated\n", newPkgMeta.GetName())
@@ -153,8 +163,8 @@ func pkgUpdate(c *cli.Context) error {
return nil
}
func updatePackage(client *client.Client, pkg *crd.Package, envName, envNamespace,
srcArchiveName, deployArchiveName, buildcmd string, forceRebuild bool) (*metav1.ObjectMeta, error) {
func updatePackage(client *client.Client, pkg *crd.Package, envName, envNamespace string,
srcArchiveFiles []string, deployArchiveFiles []string, buildcmd string, forceRebuild bool, noZip bool) (*metav1.ObjectMeta, error) {
var srcArchiveMetadata, deployArchiveMetadata *fission.Archive
needToBuild := false
@@ -174,14 +184,14 @@ func updatePackage(client *client.Client, pkg *crd.Package, envName, envNamespac
needToBuild = true
}
if len(srcArchiveName) > 0 {
srcArchiveMetadata = createArchive(client, srcArchiveName, "")
if len(srcArchiveFiles) > 0 {
srcArchiveMetadata = createArchive(client, srcArchiveFiles, false, "", "")
pkg.Spec.Source = *srcArchiveMetadata
needToBuild = true
}
if len(deployArchiveName) > 0 {
deployArchiveMetadata = createArchive(client, deployArchiveName, "")
if len(deployArchiveFiles) > 0 {
deployArchiveMetadata = createArchive(client, deployArchiveFiles, noZip, "", "")
pkg.Spec.Deployment = *deployArchiveMetadata
// Users may update the env, envNS and deploy archive at the same time,
// but without the source archive. In this case, we should set needToBuild to false
@@ -197,13 +207,13 @@ func updatePackage(client *client.Client, pkg *crd.Package, envName, envNamespac
}
newPkgMeta, err := client.PackageUpdate(pkg)
checkErr(err, "update package")
util.CheckErr(err, "update package")
return newPkgMeta, err
}
func pkgSourceGet(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
pkgName := c.String("name")
if len(pkgName) == 0 {
@@ -240,7 +250,7 @@ func pkgSourceGet(c *cli.Context) error {
}
func pkgDeployGet(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
pkgName := c.String("name")
if len(pkgName) == 0 {
@@ -277,7 +287,7 @@ func pkgDeployGet(c *cli.Context) error {
}
func pkgInfo(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
pkgName := c.String("name")
if len(pkgName) == 0 {
@@ -290,7 +300,7 @@ func pkgInfo(c *cli.Context) error {
Name: pkgName,
})
if err != nil {
return err
util.CheckErr(err, fmt.Sprintf("find package %s", pkgName))
}
w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
@@ -304,7 +314,7 @@ func pkgInfo(c *cli.Context) error {
}
func pkgList(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
// option for the user to list all orphan packages (not referenced by any function)
listOrphans := c.Bool("orphan")
pkgNamespace := c.String("pkgNamespace")
@@ -319,7 +329,7 @@ func pkgList(c *cli.Context) error {
if listOrphans {
for _, pkg := range pkgList {
fnList, err := getFunctionsByPackage(client, pkg.Metadata.Name, pkg.Metadata.Namespace)
checkErr(err, fmt.Sprintf("get functions sharing package %s", pkg.Metadata.Name))
util.CheckErr(err, fmt.Sprintf("get functions sharing package %s", pkg.Metadata.Name))
if len(fnList) == 0 {
fmt.Fprintf(w, "%v\t%v\t%v\n", pkg.Metadata.Name, pkg.Status.BuildStatus, pkg.Spec.Environment.Name)
}
@@ -345,7 +355,7 @@ func deleteOrphanPkgs(client *client.Client, pkgNamespace string) error {
// range through all packages and find out the ones not referenced by any function
for _, pkg := range pkgList {
fnList, err := getFunctionsByPackage(client, pkg.Metadata.Name, pkgNamespace)
checkErr(err, fmt.Sprintf("get functions sharing package %s", pkg.Metadata.Name))
util.CheckErr(err, fmt.Sprintf("get functions sharing package %s", pkg.Metadata.Name))
if len(fnList) == 0 {
err = deletePackage(client, pkg.Metadata.Name, pkgNamespace)
if err != nil {
@@ -364,7 +374,7 @@ func deletePackage(client *client.Client, pkgName string, pkgNamespace string) e
}
func pkgDelete(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
pkgName := c.String("name")
pkgNamespace := c.String("pkgNamespace")
@@ -386,7 +396,7 @@ func pkgDelete(c *cli.Context) error {
Namespace: pkgNamespace,
Name: pkgName,
})
checkErr(err, "find package")
util.CheckErr(err, "find package")
fnList, err := getFunctionsByPackage(client, pkgName, pkgNamespace)
@@ -402,7 +412,7 @@ func pkgDelete(c *cli.Context) error {
fmt.Printf("Package '%v' deleted\n", pkgName)
} else {
err := deleteOrphanPkgs(client, pkgNamespace)
checkErr(err, "error deleting orphan packages")
util.CheckErr(err, "error deleting orphan packages")
fmt.Println("Orphan packages deleted")
}
@@ -410,7 +420,7 @@ func pkgDelete(c *cli.Context) error {
}
func pkgRebuild(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
pkgName := c.String("name")
if len(pkgName) == 0 {
@@ -422,17 +432,310 @@ func pkgRebuild(c *cli.Context) error {
Name: pkgName,
Namespace: pkgNamespace,
})
checkErr(err, "find package")
util.CheckErr(err, "find package")
if pkg.Status.BuildStatus != fission.BuildStatusFailed {
log.Fatal(fmt.Sprintf("Package %v is not in %v state.",
pkg.Metadata.Name, fission.BuildStatusFailed))
}
_, err = updatePackage(client, pkg, "", "", "", "", "", true)
checkErr(err, "update package")
_, err = updatePackage(client, pkg, "", "", nil, nil, "", true, false)
util.CheckErr(err, "update package")
fmt.Printf("Retrying build for pkg %v. Use \"fission pkg info --name %v\" to view status.\n", pkg.Metadata.Name, pkg.Metadata.Name)
return nil
}
func fileSize(filePath string) int64 {
info, err := os.Stat(filePath)
util.CheckErr(err, fmt.Sprintf("stat %v", filePath))
return info.Size()
}
func fileChecksum(fileName string) (*fission.Checksum, error) {
f, err := os.Open(fileName)
if err != nil {
return nil, fmt.Errorf("failed to open file %v: %v", fileName, err)
}
defer f.Close()
h := sha256.New()
_, err = io.Copy(h, f)
if err != nil {
return nil, fmt.Errorf("failed to calculate checksum for %v", fileName)
}
return &fission.Checksum{
Type: fission.ChecksumTypeSHA256,
Sum: hex.EncodeToString(h.Sum(nil)),
}, nil
}
// Return a fission.Archive made from an archive . If specFile, then
// create an archive upload spec in the specs directory; otherwise
// upload the archive using client. noZip avoids zipping the
// includeFiles, but is ignored if there's more than one includeFile.
func createArchive(client *client.Client, includeFiles []string, noZip bool, specDir string, specFile string) *fission.Archive {
if len(specFile) > 0 {
// create an ArchiveUploadSpec and reference it from the archive
aus := &ArchiveUploadSpec{
Name: archiveName("", includeFiles),
IncludeGlobs: includeFiles,
}
// check if this AUS exists in the specs; if so, don't create a new one
fr, err := readSpecs(specDir)
util.CheckErr(err, "read specs")
if m := fr.specExists(aus, false, true); m != nil {
fmt.Printf("Re-using previously created archive %v\n", m.Name)
aus.Name = m.Name
} else {
// save the uploadspec
err := specSave(*aus, specFile)
util.CheckErr(err, fmt.Sprintf("write spec file %v", specFile))
}
// create the archive object
ar := &fission.Archive{
Type: fission.ArchiveTypeUrl,
URL: fmt.Sprintf("%v%v", ARCHIVE_URL_PREFIX, aus.Name),
}
return ar
}
archivePath := makeArchiveFileIfNeeded("", includeFiles, noZip)
return uploadArchive(client, archivePath)
}
func uploadArchive(client *client.Client, fileName string) *fission.Archive {
var archive fission.Archive
// If filename is a URL, download it first
if strings.HasPrefix(fileName, "http://") || strings.HasPrefix(fileName, "https://") {
fileName = downloadToTempFile(fileName)
}
if fileSize(fileName) < fission.ArchiveLiteralSizeLimit {
archive.Type = fission.ArchiveTypeLiteral
archive.Literal = getContents(fileName)
} else {
u := strings.TrimSuffix(client.Url, "/") + "/proxy/storage"
ssClient := storageSvcClient.MakeClient(u)
// TODO add a progress bar
id, err := ssClient.Upload(fileName, nil)
util.CheckErr(err, fmt.Sprintf("upload file %v", fileName))
storageSvc, err := client.GetSvcURL("application=fission-storage")
storageSvcURL := "http://" + storageSvc
util.CheckErr(err, "get fission storage service name")
// We make a new client with actual URL of Storage service so that the URL is not
// pointing to 127.0.0.1 i.e. proxy. DON'T reuse previous ssClient
pkgClient := storageSvcClient.MakeClient(storageSvcURL)
archiveURL := pkgClient.GetUrl(id)
archive.Type = fission.ArchiveTypeUrl
archive.URL = archiveURL
csum, err := fileChecksum(fileName)
util.CheckErr(err, fmt.Sprintf("calculate checksum for file %v", fileName))
archive.Checksum = *csum
}
return &archive
}
func createPackage(client *client.Client, pkgNamespace string, envName string, envNamespace string, srcArchiveFiles []string, deployArchiveFiles []string, buildcmd string, specDir string, specFile string, noZip bool) *metav1.ObjectMeta {
pkgSpec := fission.PackageSpec{
Environment: fission.EnvironmentReference{
Namespace: envNamespace,
Name: envName,
},
}
var pkgStatus fission.BuildStatus = fission.BuildStatusSucceeded
var pkgName string
if len(deployArchiveFiles) > 0 {
if len(specFile) > 0 { // we should do this in all cases, i think
pkgStatus = fission.BuildStatusNone
}
pkgSpec.Deployment = *createArchive(client, deployArchiveFiles, noZip, specDir, specFile)
pkgName = util.KubifyName(fmt.Sprintf("%v-%v", path.Base(deployArchiveFiles[0]), uniuri.NewLen(4)))
}
if len(srcArchiveFiles) > 0 {
pkgSpec.Source = *createArchive(client, srcArchiveFiles, false, specDir, specFile)
pkgStatus = fission.BuildStatusPending // set package build status to pending
pkgName = util.KubifyName(fmt.Sprintf("%v-%v", path.Base(srcArchiveFiles[0]), uniuri.NewLen(4)))
}
if len(buildcmd) > 0 {
pkgSpec.BuildCommand = buildcmd
}
if len(pkgName) == 0 {
pkgName = strings.ToLower(uuid.NewV4().String())
}
pkg := &crd.Package{
Metadata: metav1.ObjectMeta{
Name: pkgName,
Namespace: pkgNamespace,
},
Spec: pkgSpec,
Status: fission.PackageStatus{
BuildStatus: pkgStatus,
},
}
if len(specFile) > 0 {
// if a package sith the same spec exists, don't create a new spec file
fr, err := readSpecs(getSpecDir(nil))
util.CheckErr(err, "read specs")
if m := fr.specExists(pkg, false, true); m != nil {
fmt.Printf("Re-using previously created package %v\n", m.Name)
return m
}
err = specSave(*pkg, specFile)
util.CheckErr(err, "save package spec")
return &pkg.Metadata
} else {
pkgMetadata, err := client.PackageCreate(pkg)
util.CheckErr(err, "create package")
fmt.Printf("Package '%v' created\n", pkgMetadata.GetName())
return pkgMetadata
}
}
func getContents(filePath string) []byte {
var code []byte
var err error
code, err = ioutil.ReadFile(filePath)
util.CheckErr(err, fmt.Sprintf("read %v", filePath))
return code
}
func writeArchiveToFile(fileName string, reader io.Reader) error {
tmpDir, err := fission.GetTempDir()
if err != nil {
return err
}
path := filepath.Join(tmpDir, fileName+".tmp")
w, err := os.Create(path)
if err != nil {
return err
}
_, err = io.Copy(w, reader)
if err != nil {
return err
}
err = os.Chmod(path, 0644)
if err != nil {
return err
}
err = os.Rename(path, fileName)
if err != nil {
return err
}
return nil
}
// downloadToTempFile fetches archive file from arbitrary url
// and write it to temp file for further usage
func downloadToTempFile(fileUrl string) string {
reader, err := downloadURL(fileUrl)
defer reader.Close()
util.CheckErr(err, fmt.Sprintf("download from url: %v", fileUrl))
tmpDir, err := fission.GetTempDir()
util.CheckErr(err, "create temp directory")
tmpFilename := uuid.NewV4().String()
destination := filepath.Join(tmpDir, tmpFilename)
err = os.Mkdir(tmpDir, 0744)
util.CheckErr(err, "create temp directory")
err = writeArchiveToFile(destination, reader)
util.CheckErr(err, "write archive to file")
return destination
}
// downloadURL downloads file from given url
func downloadURL(fileUrl string) (io.ReadCloser, error) {
resp, err := http.Get(fileUrl)
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("%v - HTTP response returned non 200 status", resp.StatusCode)
}
return resp.Body, nil
}
// Create an archive from the given list of input files, unless that
// list has only one item and that item is either a zip file or a URL.
//
// If the inputs have only one file and noZip is true, the file is
// returned as-is with no zipping. (This is used for compatibility
// with v1 envs.) noZip is IGNORED if there is more than one input
// file.
func makeArchiveFileIfNeeded(archiveNameHint string, archiveInput []string, noZip bool) string {
// Unique name for the archive
archiveName := archiveName(archiveNameHint, archiveInput)
// Get files from inputs as number of files decide next steps
files, err := fission.FindAllGlobs(archiveInput)
if err != nil {
util.CheckErr(err, "finding all globs")
}
// We have one file; if it's a zip file or a URL, no need to archive it
if len(files) == 1 {
// make sure it exists
if _, err := os.Stat(files[0]); err != nil {
util.CheckErr(err, fmt.Sprintf("open input file %v", files[0]))
}
// if it's an existing zip file OR we're not supposed to zip it, don't do anything
if archiver.Zip.Match(files[0]) || noZip {
return files[0]
}
// if it's an HTTP URL, just use the URL.
if strings.HasPrefix(files[0], "http://") || strings.HasPrefix(files[0], "https://") {
return files[0]
}
}
// For anything else, create a new archive
tmpDir, err := fission.GetTempDir()
if err != nil {
util.CheckErr(err, "create temporary archive directory")
}
archivePath, err := fission.MakeArchive(filepath.Join(tmpDir, archiveName), archiveInput...)
if err != nil {
util.CheckErr(err, "create archive file")
}
return archivePath
}
// Name an archive
func archiveName(givenNameHint string, includedFiles []string) string {
if len(givenNameHint) > 0 {
return fmt.Sprintf("%v-%v", givenNameHint, uniuri.NewLen(4))
}
if len(includedFiles) == 0 {
return uniuri.NewLen(8)
}
return fmt.Sprintf("%v-%v", util.KubifyName(includedFiles[0]), uniuri.NewLen(4))
}
+12 -11
View File
@@ -29,10 +29,11 @@ import (
"github.com/fission/fission"
"github.com/fission/fission/crd"
"github.com/fission/fission/fission/util"
)
func recorderCreate(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
recName := c.String("name")
if len(recName) == 0 {
@@ -81,19 +82,19 @@ func recorderCreate(c *cli.Context) error {
if c.Bool("spec") {
specFile := fmt.Sprintf("recorder-%v.yaml", recName)
err := specSave(*recorder, specFile)
checkErr(err, "create recorder spec")
util.CheckErr(err, "create recorder spec")
return nil
}
_, err := client.RecorderCreate(recorder)
checkErr(err, "create recorder")
util.CheckErr(err, "create recorder")
fmt.Printf("recorder '%s' created\n", recName)
return err
}
func recorderGet(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
recName := c.String("name")
@@ -102,7 +103,7 @@ func recorderGet(c *cli.Context) error {
Namespace: "default",
})
checkErr(err, "get recorder")
util.CheckErr(err, "get recorder")
w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
@@ -116,7 +117,7 @@ func recorderGet(c *cli.Context) error {
}
func recorderUpdate(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
recName := c.String("name")
enable := c.Bool("enable")
@@ -190,14 +191,14 @@ func recorderUpdate(c *cli.Context) error {
}
_, err = client.RecorderUpdate(recorder)
checkErr(err, "update recorder")
util.CheckErr(err, "update recorder")
fmt.Printf("recorder '%v' updated\n", recName)
return nil
}
func recorderDelete(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
recName := c.String("name")
@@ -212,17 +213,17 @@ func recorderDelete(c *cli.Context) error {
Namespace: recNs,
})
checkErr(err, "delete recorder")
util.CheckErr(err, "delete recorder")
fmt.Printf("recorder '%v' deleted\n", recName)
return nil
}
func recorderList(c *cli.Context) error {
client := getClient(c.GlobalString("server"))
client := util.GetApiClient(c.GlobalString("server"))
recorders, err := client.RecorderList("default")
checkErr(err, "list recorders")
util.CheckErr(err, "list recorders")
w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
+10 -9
View File
@@ -24,6 +24,7 @@ import (
"github.com/urfave/cli"
"github.com/fission/fission/fission/util"
"github.com/fission/fission/redis/build/gen"
)
@@ -59,15 +60,15 @@ func recordsView(c *cli.Context) error {
return recordsByTime(from, to, verbosity, c)
}
err := recordsAll(verbosity, c)
checkErr(err, "view records")
util.CheckErr(err, "view records")
return nil
}
func recordsAll(verbosity int, c *cli.Context) error {
fc := getClient(c.GlobalString("server"))
fc := util.GetApiClient(c.GlobalString("server"))
records, err := fc.RecordsAll()
checkErr(err, "view records")
util.CheckErr(err, "view records")
showRecords(records, verbosity)
@@ -75,10 +76,10 @@ func recordsAll(verbosity int, c *cli.Context) error {
}
func recordsByTrigger(trigger string, verbosity int, c *cli.Context) error {
fc := getClient(c.GlobalString("server"))
fc := util.GetApiClient(c.GlobalString("server"))
records, err := fc.RecordsByTrigger(trigger)
checkErr(err, "view records")
util.CheckErr(err, "view records")
showRecords(records, verbosity)
@@ -87,10 +88,10 @@ func recordsByTrigger(trigger string, verbosity int, c *cli.Context) error {
// TODO: More accurate function name (function filter)
func recordsByFunction(function string, verbosity int, c *cli.Context) error {
fc := getClient(c.GlobalString("server"))
fc := util.GetApiClient(c.GlobalString("server"))
records, err := fc.RecordsByFunction(function)
checkErr(err, "view records")
util.CheckErr(err, "view records")
showRecords(records, verbosity)
@@ -98,10 +99,10 @@ func recordsByFunction(function string, verbosity int, c *cli.Context) error {
}
func recordsByTime(from string, to string, verbosity int, c *cli.Context) error {
fc := getClient(c.GlobalString("server"))
fc := util.GetApiClient(c.GlobalString("server"))
records, err := fc.RecordsByTime(from, to)
checkErr(err, "view records")
util.CheckErr(err, "view records")
showRecords(records, verbosity)

Some files were not shown because too many files have changed in this diff Show More