Compare commits

...
141 Commits
Author SHA1 Message Date
Nafisa Shaziaandsmruthi2187 95eef49cf8 Replay recorded requests by ReqUID (#864) 2018-08-17 13:17:54 -07:00
Ta-Ching ChenandGitHub 2d01382a82 Add cleanup function to test scripts (#863) 2018-08-18 02:11:08 +08:00
Ta-Ching ChenandGitHub d443f8b2ef Scale deployment to zero when function is in idle state (#775) 2018-08-17 23:03:34 +08:00
Ta-Ching ChenandGitHub dffb5cdf8b Fix CLI failed to setup port-forward caused by #712 (#867) 2018-08-17 16:05:13 +08:00
Erwin van EykandTa-Ching Chen 5c1d8bc90c Extensible Fission CLI (#743)
This PR introduces extensibility to the Fission CLI. The principle behind the design of this extensibility is taken from the approach git uses: the main binary (fission) calls other binaries (fission-workflows) - which it discovers using simple prefix-rules - based on specific commands (fission workflow <command>).
2018-08-17 15:20:46 +08:00
smruthi2187andTa-Ching Chen ad28922871 Fix for #662: avoid unnecessary builds (#866) 2018-08-17 13:53:37 +08:00
Ta-Ching ChenandGitHub 9d7355cbaa Fix newdeploy fail to update HPA, deployment of a function after function update (#862) 2018-08-15 19:56:30 +08:00
Nafisa ShaziaandTa-Ching Chen 74a3a54543 Recorder CRD, Records API, Redis deployment (#818) 2018-08-15 05:28:40 +08:00
Ta-Ching ChenandGitHub 07ca5df8d0 Fix router not taps function services (#860) 2018-08-13 19:45:49 +08:00
Ta-Ching ChenandGitHub e730376e3e Add rebuild subcommand to pkg command (#808) 2018-08-12 04:13:25 +08:00
VishalandTa-Ching Chen ee61c8c333 Update route without providing function reference (#718) 2018-08-06 11:31:05 +08:00
Ta-Ching ChenandGitHub 27b74a7cf2 Fix router panic when trying to update route (#811) 2018-08-05 01:06:47 +08:00
Ta-Ching ChenandGitHub 088e665d28 Fix spec failed to archive single directory (#837) 2018-08-03 21:26:19 +08:00
Erwin van EykandTa-Ching Chen 9438f3c4ec Updated binary environment README (#773)
- Added usage instructions for getting started quickly.
- Added instructions for using the builder
2018-08-03 15:07:08 +08:00
Ta-Ching ChenandGitHub b5a906b2f4 Do resources validation when validate spec files (#840) 2018-08-03 13:00:29 +08:00
Ta-Ching ChenandGitHub 0e40cb5f4f Fix newdeploy not updates deployment after function's entrypoint changed (#838) 2018-08-02 21:00:27 +08:00
Ta-Ching ChenandGitHub 1a1ac49ce0 Fix CLI failed to set up port-forwarding when multiple controller pods exist in the same namespace (#712)
* Fix CLI failed to set up port-forwarding when multiple controller pods exist in the same namespace
* Check pod healthy state before establishing the connection
2018-08-02 17:20:53 +08:00
VishalandTa-Ching Chen 4356378301 Uses a real go project to showcase vendor example so glide works (#828) 2018-08-01 23:55:54 +08:00
xiekeyangandTa-Ching Chen 6123873d2a Add gevent based Python server to benchmark test cases (#794)
This add the benchmark test cases, which is using gevent based on wsgi
framework.
2018-08-01 21:22:03 +08:00
Ta-Ching ChenandGitHub 9a7ffc2f5b Add more meaningful error messages to executor when getServiceForFunction (#752) 2018-07-30 09:10:48 +08:00
VishalandGitHub f3857abdf5 Fixed the name of JVM builder image name (#824) 2018-07-27 23:28:11 +05:30
VishalandSoam Vasani e977c7caca V0.9.2 (#817)
* Release 0.9.2

* Added changelog
2018-07-25 12:59:30 -07:00
VishalandGitHub f40688435d Added readme for JVM environment & examples (#768)
Instructions for JVM environment and examples
2018-07-25 09:26:01 +05:30
Erwin van EykandTa-Ching Chen e8b2a26a3b Add query options to function test (#782) 2018-07-24 23:00:08 +08:00
VishalandGitHub 27c3a50d23 Java env builder with Maven (#783)
Java environment builder with support for Maven builds. Default build comman runs `mvn clean package` and picks up target/*with-dependencies.jar for function.
2018-07-24 15:16:21 +05:30
Erwin van EykandTa-Ching Chen ec495fbd1d Added Helm lint check (#799) 2018-07-19 11:49:28 +08:00
WrathZAandVishal 80fadcc7b8 Typo correction (#797) 2018-07-17 18:18:10 +05:30
VishalandGitHub 4b37b9ac03 Fixed typos from from goreportcard (#760) 2018-07-16 16:24:05 +05:30
VishalandGitHub 16d0f6a94f Updating release notes with details and structure (#738) 2018-07-16 14:41:28 +05:30
xiekeyangandTa-Ching Chen cc40d1ed2e change image pull policy of builder manager (#793)
This changes image pull policy from Always to IfNotPresent. The previous
policy forbid customer to use the exact image existing on K8S Node.
And it should be OK to use fixed policy here. Customer need not set policy by
their own.
2018-07-16 10:48:47 +08:00
Ta-Ching ChenandGitHub 425e447e5c Add go environment vendor directory support (#781)
* Add go environment vendor directory support
* Use symbolink instead of copying files to gopath
* Add go vendor example package & test
2018-07-13 13:16:22 +08:00
xiekeyangandSoam Vasani 3707b95edb Round Tripper of Fission Router: parameters be configurable (#713)
The Round Tripper parameters of timeout, keep alive time and Max retry
times is configured in router system environment.

And they are set to Round Tripper when router service initializing.

This setup new nested structure `tsRoundTripperParams` to transfer them.
2018-07-12 12:08:52 -07:00
Ta-Ching ChenandGitHub 2982efebb8 Delete namespace in background to reduce build time (#791)
* Delete namespace in background
* TPR is no longer exists
2018-07-12 11:00:19 +08:00
Ta-Ching ChenandGitHub 14db705891 Break & Stop the build immediately if a non-zero exit code was returned (#790) 2018-07-11 20:15:03 +08:00
Ta-Ching ChenandGitHub 9c885ef6f5 Fix spec command overrides existing archive's url of a package (#764) 2018-07-11 13:03:45 +08:00
smruthi2187andGitHub 34e8d0ee3d Add changelog. (#789) 2018-07-06 22:00:14 -07:00
smruthi2187andGitHub 728979636f Updating charts to version 0.9.1 (#788) 2018-07-06 19:11:35 -07:00
Ta-Ching ChenandGitHub 462db2f00c Fix “rm: missing operand” in release script (#779) 2018-07-06 14:15:20 +08:00
Ta-Ching ChenandGitHub 2504408803 Fix go env panic “plugin: not implemented” (#784) 2018-07-06 12:24:46 +08:00
Ta-Ching ChenandGitHub 61569d1c38 Change flag name to KeepArchive for backward compatibility (#787) 2018-07-06 03:13:07 +08:00
smruthi2187andGitHub 5b6582698c changelog for release 0.9.0. (#780) 2018-07-04 02:08:47 -07:00
smruthi2187andGitHub 81f0f1df6d Changes in charts for release 0.9.0 (#778) 2018-07-04 00:27:47 -07:00
Ta-Ching Chenandsmruthi2187 44be0f6859 Move build process from host to docker container during release process (#682) 2018-07-03 21:14:06 -07:00
smruthi2187andGitHub 9daf97a9ab router tracks function liveness before forwarding request (#701) 2018-07-03 15:51:52 -07:00
Erwin van EykandTa-Ching Chen 032d1a8b9a Support annotations in environment specs (#733)
* Added annotations to runtime and builder environment specs
* Use crd annotations for env pod
2018-06-30 05:38:22 +08:00
VishalandGitHub 65fd1d9fbb Java env alpha (#656)
This change adds support for JVM based environment for running Java functions. This is alpha release of JVM environment and might undergo changes.
2018-06-27 21:39:24 +05:30
John GallagherandSoam Vasani a35cd7f4a4 Convert go-env Dockerfile into a multi-stage build (#683)
Converting to a multi-stage build reduces the size of the final image from ~750MB to ~15MB.
2018-06-26 13:21:36 -07:00
xiekeyangandSoam Vasani 74212f1810 envns should be availabe in message line (#734)
This line warns the user that dependent environment doesn’t exist, and presents the command line
for creating an environment.

When the variable `envNamespace` is specified, the `envns` option should be available as well.
2018-06-26 11:15:41 -07:00
Nafisa ShaziaandTa-Ching Chen 1ed59f0789 Push error messages to error topic when NATS mqtrigger fail (#724) 2018-06-27 00:27:57 +08:00
xiekeyangandTa-Ching Chen 71d8b769d5 Python Environment: add gevent based WSGI server framework (#750)
* Python Environment: add gevent based WSGI server framework

This supply gevent based WSGI server framework to Python Environment, to
allow customer to run their functions under multi-threads.

* classify the flask based application

This set up the flask based application as individual class, to make
sure the consistent and integrity.

* Add instruction of WSGI server selection applied in Python Environment

* Python Environment starting logger: provide framework name
2018-06-26 15:50:06 +08:00
Ta-Ching ChenandGitHub c863616b38 Add release steps to render & upload fission installation YAML (#745) 2018-06-26 13:41:43 +08:00
xiekeyangandsmruthi2187 09b62e514e Replace the hard-coded archive size from error message with actual value (#735) 2018-06-25 17:48:43 -07:00
Ta-Ching ChenandGitHub c705ba355e Fix executor not reaps idle function pods for functions with executortype newdeploy (#744) 2018-06-21 17:54:42 +08:00
VishalandGitHub 7642dcf05f Disable caching of connections from router to function pods (#742)
A temporary fix for #723 which needs to be optimized in future. Disables caching of connections from the router to function pod/service.
2018-06-21 12:45:11 +05:30
Ta-Ching ChenandGitHub 58a5f1cb0b Fix executor wrongly passes loop variable reference to function (#751) 2018-06-19 09:55:41 +08:00
VishalandGitHub 1b86c367a0 Added a flag to control the extraction of archive based on user input (#675)
For certain environments, the archive should not be extracted and kept as it is (For ex. for Java the jar file should not be extracted). This change enables an environment level flag to control this behaviour.
2018-06-14 12:32:40 +05:30
VishalandGitHub 6d08e6e0bc Archives bigger than 256K size need env variable for uploading (#697)
For archives bigger than 256K, the Storage service was called from the client side, this needed few environment variables to be set. This change uses port forwarding to achieve the same and does not need environment variables to be set.
2018-06-14 01:12:06 +05:30
xiekeyangandVishal ef7d4ae385 Make fetcher resource requests and limits configurable (#708)
This change allows fetcher container resource requests and limits to be configurable.
2018-06-13 16:12:55 +05:30
Erwin van EykandTa-Ching Chen cf1f944069 Extract portforward to separate package (#728)
* Extract port-forward from CLI to own package (for reuseability)
* Extract CLI logging to subpackage
2018-06-13 11:26:39 +08:00
VishalandSoam Vasani ddff7100e9 Temporarily disabling the tests so that other PRs can be worked on, will enable after closure on #723 (#737) 2018-06-08 15:07:50 -07:00
xiekeyangandSoam Vasani a5bc2a6d4c add build exe to gitignore (#736)
This is a built binary, should be ignore when git committion.
2018-06-08 10:46:17 -07:00
Soam VasaniandGitHub 952b48bd68 Environment warning message bugfix (#725) 2018-06-07 14:17:27 -07:00
Joe StevensandTa-Ching Chen 2ad07d68b6 Fix for Windows CLI Port Forwarding (#715) 2018-06-07 16:37:37 +08:00
xiekeyangandTa-Ching Chen 8c96d9c6a0 ArchiveLiteralSizeLimit: Use Constant Instead Hard Code (#731)
This use constant to instead hard code for ArchiveLiteralSizeLimit. That ensure if the limitation
value is changed in cli, the controller server can follow it. Otherwise, the condition will return
error, and make error HTTP response.

Later, we should delete it in fission-cli, and make it configurable it chart, to work in fission-
servers only.
2018-06-07 15:45:32 +08:00
VishalandGitHub 46d9808173 V0.8.0 (#722)
Release v0.8.0
2018-06-06 00:01:00 +05:30
smruthi2187andGitHub 2e37611da8 pre-upgrade job to verify function references and restricted privileges for fetcher and builder SA (#717)
It is mandatory (from this release onwards) for function to refer to secrets, config-maps and packages in its own namespace to ensure isolation of users sharing the same cluster.
This change runs a pre-upgrade job to verify function references for all functions created prior to this release and fails the upgrade by printing a list of functions that violate this restriction.
2018-06-04 19:30:27 -07:00
Ta-Ching ChenandGitHub 0057270551 Add time trigger cron spec examination tool (#680) 2018-06-04 19:02:54 +08:00
Ta-Ching ChenandGitHub 7a7d15b50c Update k8s dependencies to 1.10 (#687) 2018-06-01 15:43:19 +08:00
VishalandGitHub 202cc3cb7e Changing the logger daemonset's update strategy (#714)
Changing the logger daemonset's update strategy from OnDelete to RollingUpdate (Ref: https://kubernetes.io/docs/tasks/manage-daemon/update-daemon-set/#daemonset-update-strategy)
2018-05-31 23:42:23 +05:30
VishalandGitHub a761393816 Ingress integration (#688)
Ingress integration to allow the optional creation of ingress for a given route. The ingress controller needs to be set up by the user separately so that ingress path is accessible outside the cluster.
2018-05-31 11:27:55 +05:30
Ta-Ching ChenandGitHub fedf6b6e5b Check spec directory exists before reading spec files (#709)
* Check spec directory exists before reading spec files
* Fix spec example
2018-05-31 03:01:40 +08:00
xiekeyangandSoam Vasani 6eec55223d Formatted specifiers are not compatible with variables (#706)
It missed specifier for namespace
2018-05-29 09:08:38 -07:00
xiekeyangandTa-Ching Chen 940b9adabd Indicate HTTP status code by library const (#703) 2018-05-24 19:10:45 +08:00
smruthi2187andGitHub 8984e4916e Enabling multi-tenancy for fission objects. (#655)
This feature allows creation of fission objects in different namespaces, in addition to retaining the existing behavior of creating fission objects in default namespace if user doesnt provide one. 
It also removes cluster admin roles for fission-fetcher and fission-builder Service Accounts and grants them only those privileges that they need.
2018-05-23 13:22:46 -07:00
Joe StevensandSoam Vasani c9d2757760 docker-distribution version bump for windows compatibility (#691) 2018-05-21 18:41:53 -07:00
Soam VasaniandGitHub d846aed612 Fission metrics integration (#677)
Add prometheus metrics collection endpoints to router and executor. Add prometheus annotations to router and executor pods.
2018-05-21 13:57:18 -07:00
VishalandGitHub d9b5b9b8a8 Java Environment Design proposal
Java environment design decisions and considerations.
2018-05-14 13:45:01 +05:30
Ta-Ching ChenandGitHub 9c91f00cb4 Replace Werkzeug with Bjoern as underlying WSGI server (#672) 2018-05-09 14:00:40 +08:00
Ta-Ching ChenandGitHub 0a648d1d75 Version -> 0.7.2 (#670) 2018-05-05 20:01:23 +08:00
Ta-Ching ChenandGitHub 6a3e326f36 Add benchmark script (#666) 2018-05-05 12:42:16 +08:00
smruthi2187andTa-Ching Chen f0dc2554a1 Meaningful error message when fetch request is received for a package when build is not successful. (#661) 2018-05-03 22:51:48 +08:00
VishalandTa-Ching Chen 2ece0db13f Fixed the issue with update wiping original values & added resources in listing, fixes #654 (#663) 2018-05-02 22:29:08 +08:00
Ta-Ching ChenandGitHub 0e3a270d27 Delete deployment with proper delete propagation policy (#630) 2018-05-01 23:25:50 +08:00
Ta-Ching ChenandGitHub 6a7381c575 Fix newdeploy backend failed to delete deployment due to incorrect resource version (#657) 2018-05-01 16:48:34 +08:00
VishalandTa-Ching Chen 0aff2f214d Fixes the issue with fn test and adds relevant test cases, fixes #650 (#651)
* Fixes the issue with fn test and adds relevant test cases, fixes #650
* Checking log size for invalid function
* Testing the invalid function logs a few time beforr quitting
* Disabling the test for now, linked to issue #653
2018-04-30 14:14:46 +08:00
VishalandTa-Ching Chen dec4a39c15 Function update should be possible without change to code (#652) 2018-04-30 11:35:19 +08:00
Ta-Ching ChenandGitHub 7577339912 Change time precision for fluentd influxdb plugin to nano second (#646) 2018-04-26 20:57:55 +08:00
Ta-Ching ChenandGitHub 6375819195 Fix test cases occasional failure (#647) 2018-04-26 14:39:16 +08:00
VishalandGitHub 0b34abcada Update .travis.yml
Seems a small item is missing, causing notifications not to work. Similar block in fission-workflows does work.
2018-04-25 23:15:32 +05:30
smruthi2187andTa-Ching Chen 772fdfa15c Setting package buildStatus to pending when functions source is updated. (#637) 2018-04-24 15:24:46 +08:00
VishalandGitHub 7198b24028 Fission upgrade tests (#605)
Tests for upgrading Fission from the last version to current branch and testing. Currently test is very simple and more tests can be added in future
2018-04-24 12:51:17 +05:30
VishalandGitHub eb696432d8 Testing proposal: Requirements and frameworks exploration (#581)
Proposal for revamping the testing framework and tests in general based on needs of Fission project
2018-04-24 00:45:09 +05:30
Ta-Ching ChenandGitHub 2d359f879a Fix SEGFAULT issue when buildmgr failed to update package (#635) 2018-04-21 23:17:33 +08:00
smruthi2187andGitHub 94e2f11ea6 Remove port forward in tests for router, controller and nats pods (#611) 2018-04-20 19:00:22 -07:00
VishalandTa-Ching Chen ff01cf2452 Removed the fn pods functionality (#594) 2018-04-19 14:32:16 +08:00
Ta-Ching ChenandGitHub 400e19a48f Fix executor does not reap specialized function pod when env no longer exists (#633) 2018-04-19 07:39:37 +08:00
Ta-Ching ChenandGitHub 0194f53e12 Fix SEGFAULT issue when build manager failed to update package (#626) 2018-04-17 09:36:11 +08:00
John GallagherandSoam Vasani fef1469c61 Update readme to point to the proper link (#628) 2018-04-16 11:30:27 -07:00
Ta-Ching ChenandGitHub cf53e17be5 Remove confusing default value description & add default value to cli flag (#619) 2018-04-16 17:27:10 +08:00
smruthi2187andGitHub 449f49515a Changes needed for release 0.7.1 (#622)
bump up the version to 0.7.1 in charts and adding changelog.
2018-04-10 18:03:08 -07:00
Ta-Ching ChenandGitHub d01b309fc7 Add validate function to crd resource and do validate before creation/update (#580) 2018-04-08 00:56:13 +08:00
smruthi2187andGitHub 12299df811 Prevent releasing idle connections because transport is shared. (#609)
The transport in RoundTripper is the default transport and there's just one object shared between different http requests and releasing the idle Connections ended up releasing the idle connections that were established with the executor service address too.

So every time a http request was made to get a service for function, the transport layer ended up having to dial a new connection (that was timing out due to various reasons).

By removing the CloseIdleConnections, we minimize the need for transport to dial a new connection to executor for every request, thereby minimizing the occurrence of dial timeouts.
2018-04-06 13:23:25 -07:00
Soam VasaniandGitHub 95a50dbbd6 --spec flags for env and trigger creates (#607) 2018-04-06 09:00:01 -07:00
smruthi2187andGitHub 8014c83b02 Invalidate stale router cache entry with podIP's for deleted pods. (#546)
The router's cache entry for a function might become stale if the pod that had the function specialized gets deleted somehow. In such a case, we'd retry getting a new service for the function from executor and retry forwarding the user request to the newly created service.
2018-04-05 13:47:47 -07:00
smruthi2187andGitHub db2f620121 Updating releasing guideliness with a few more details. (#599) 2018-04-05 13:44:17 -07:00
Soam VasaniandGitHub f9ca626940 E2E test for NATS-streaming trigger (#338)
E2E test for NATS-streaming trigger: move ad-hoc shell script into CI tests.
2018-04-05 10:07:00 -07:00
Ta-Ching ChenandGitHub 1dbf8f9243 Fix components crash before crds creation (#602)
* Wait for CRDs creation for 30 sec when component start
* Fix ensureCRD return nil while the error is not empty
2018-04-04 15:40:55 +08:00
Ta-Ching ChenandGitHub 9406720e8a Add deprecated message to subcommand pods (#592) 2018-04-04 01:26:27 +08:00
Ta-Ching ChenandGitHub 2a06705320 Use a separate controller loop to watch functions change and create a service (#544) 2018-04-04 01:17:51 +08:00
smruthi2187andGitHub d39944e04d updates to changelog. (#598) 2018-04-02 18:08:15 -07:00
smruthi2187andGitHub 6f6b52438a changes needed for release 0.7.0 (#597) 2018-04-02 15:32:22 -07:00
Soam VasaniandGitHub 5baf67f143 Spec validator, better errors, apply waits for previous build (#560)
Adds a validator, invoked through `fission spec validate` and before `fission spec apply`. The validator checks for duplicate names, dangling references, and unused resources.

Also attempts to fix a race between CLI and builder.  Avoids replacing a package while it's building, instead waiting for the build status to change from pending before updating it.
2018-04-02 14:33:31 -07:00
xiekeyangandSoam Vasani 6d61f6e5eb bug fix: spec dir flag (#595)
The cli spec dir flag name should be `specdir`, not `specs`, that should match to:

```go
specDirFlag := cli.StringFlag{Name: "specdir", Usage: "Directory to store specs, defaults to ./specs"}
```

in fission/main.go

Currently it cause the specific specs dir not to work.

```
$ fission spec apply --specdir <my spec dir>
Everything up to date.
```
2018-04-02 05:51:46 -07:00
Ta-Ching ChenandGitHub 6ab902e7a3 Add steps for how to set FISSION_ROUTER env variable & update docs (#593) 2018-03-31 01:10:40 +08:00
smruthi2187andTa-Ching Chen ed7d088bbc Check if the requested file already exists in fetcher and skip fetch (#584) 2018-03-30 16:33:06 +08:00
smruthi2187andGitHub c0058153a0 Adding routerUrl parameter for kubewatch, timer, mqueue trigger deployments (#591)
* This is to enable these triggers to be able to talk to fission router if deployed in a namespace other than fission.
2018-03-29 17:38:50 -07:00
VishalandGitHub a8482a8b89 Function logs: Uses proper way to get server URL
After changes related to port forwarding, function logs was getting empty URL, fixed to use the appropriate method instead.
2018-03-29 15:26:47 +05:30
VishalandGitHub a23f8acffc Tests for function update (#550)
Tests for updates to a function of new deployment executor type. Tests check for changes in the environment, scale, secrets etc. Also checks conversion of function executor type from new deployment to pool manager and vice versa
2018-03-28 20:32:04 +05:30
smruthi2187andSoam Vasani 77be3379b2 Prepending a slash to user input url if missing. (#547)
* Prepending a slash to user input url if missing.

* Adding the url slash check for spec.
2018-03-26 12:31:02 -07:00
Ta-Ching ChenandSoam Vasani 03046170dc Add post-upgrade-job to track fission upgrade (#564) 2018-03-26 12:28:35 -07:00
VishalandSoam Vasani c0acaa74c3 Fixes the issue #559 with env versions (#569)
Do not set automatically set env version if user has already specified it.
2018-03-26 12:25:16 -07:00
Soam VasaniandGitHub 8708fdec93 Add verbosity flag and verbose logs for portforwarder (#575)
Also, parse flags before connecting to server.
2018-03-26 12:23:45 -07:00
Chris LeeandSoam Vasani 6b188c053a Add golang example to installation guide (#578)
Use --src instead of --code for Go example.
2018-03-26 10:04:25 -07:00
Soam Vasani 0fd6114218 Update release checklist 2018-03-22 14:05:51 -07:00
Soam Vasani 9ae06f56c8 Update generated changelog 2018-03-22 14:03:02 -07:00
Soam Vasani 704a8094e6 Bump version in charts, forgot one file 2018-03-22 11:33:23 -07:00
Soam Vasani 6f54565195 Bump version in charts 2018-03-22 11:26:58 -07:00
Soam Vasani 90206fbaa0 Release checklist updates 2018-03-22 11:26:41 -07:00
yiqundingandSoam Vasani 8fb4aca231 This change fixes an error in a yaml file in the fission-core chart. (#563)
The Deployment definition for executor, controller, router contains readines/liveness probes. The port definition for those probes should be numbers, instead of string, i.e. port: "8888" should be port 8888.
2018-03-22 11:14:47 -07:00
Erwin van EykandSoam Vasani 70a93a7302 Add Container object to environment build and runtime specs (#413)
Environment Specs so far had only an image URL to specify a container image.

This was fine for public images but fell short in a few of cases:
(a) Using private image registries
(b) Specifying environment variables (this is needed for workflows helm install)
(c) Setting a SecurityContext for the container

This change adds the Container object to both build and runtime Environments. 

Compatibility is preserved -- the existing ImageURL field is still used.  See the comments in types.go for the overriding rules in the case that both Container and ImageURL are specified.
2018-03-22 00:13:02 -07:00
Bo-Yi WuandSoam Vasani c76b6bacc0 [ci skip] update release number (#561) 2018-03-21 16:16:43 -07:00
VishalandGitHub 15bedfb688 Warning when updating function from poolmgr type to newdeploy executor type
When changing the function executor type from pool manager to new deployment - warning should not be given about ignore environment resources
2018-03-20 14:11:30 -07:00
Ta-Ching ChenandGitHub 1447e6949d Show fission deployment version with cli (#538)
Add version information to binary through go ldflags
2018-03-19 14:14:55 +08:00
Ta-Ching ChenandGitHub 048ab6149a Fix executor failed to clean cache & kubeobjs after function deleted (#534) 2018-03-14 02:29:12 +08:00
Soam Vasani 8a0abc59db use specific go env version in example 2018-03-11 11:54:30 -07:00
Ta-Ching ChenandGitHub 039e003af6 Always retry for function pod specialization when istio feature is enabled. (#536) 2018-03-11 23:17:05 +08:00
Soam VasaniandGitHub 1ea665386a Docs update (#542)
* Misc doc updates

* Some theme updates: no all caps headers, remove blue arrows
2018-03-09 11:26:15 -08:00
Soam VasaniandGitHub e7c9a67a68 Release script updates (#541) 2018-03-08 13:48:57 -08:00
Ta-Ching ChenandGitHub afd7f7f436 Show warning when trying to create a route with non-existent function (#539) 2018-03-09 02:45:30 +08:00
Ta-Ching ChenandGitHub b4300feabc Fix executor tries to create a new deployment when a function is updated (#524)
Newdeploy manager now checks the existence of function service cache by function UID before trying to create a new deployment. And return the cached fsvc directly if the cache exists.
2018-03-08 02:52:36 +08:00
smruthi2187andGitHub a3826046a5 Delete healthz log (#525)
Silencing the healthz logs.
2018-03-02 15:50:48 -08:00
312 changed files with 22864 additions and 3530 deletions
+1
View File
@@ -1,5 +1,6 @@
fission-bundle/fission-bundle
fission/fission
environments/fetcher/cmd/fetcher
# Pycharm IDE
.idea
+12 -5
View File
@@ -9,7 +9,7 @@ branches:
language: go
go:
- 1.8
- 1.10.2
cache:
directories:
@@ -31,17 +31,24 @@ install:
env:
- KUBECONFIG=${HOME}/.kube/config PATH=$HOME/k8scli:${PATH}
script:
before_script:
- cd ${TRAVIS_BUILD_DIR}
- glide install -v
- ./fission-bundle/build.sh
- hack/verify-gofmt.sh
- hack/verify-govet.sh
- helm lint charts/fission-all/ charts/fission-core/
- glide install -v
- go build -o fission/fission fission/*.go
- ./fission-bundle/build.sh
- hack/runtests.sh
script:
- cd ${TRAVIS_BUILD_DIR}
- test/build_and_test.sh
- test/upgrade/fission_upgrade_test.sh
notifications:
slack:
secure: YZ34vsfw1TtftJypg1MyP4+ihONI4gaxeS3FghBQli6+EezjzcxOXyj5VD+x0ucXfDeTaDrFfmVN0SAGObOoZUE+ea9KAoTo50tRLaD9kwOTACiekZalC4uuBguH0D1/A6vlbU8dchsr9mvIhbisG6mTncdPtqGYYHtyBQme6ngmmHbVAQFZcIBHmNuDb/HWhSr8KMEuyB6+mBXLYELHnXnf26cOhdGNaagqCOJTiemX85RGwIuOPxyBhKDFMLyHohDT7FJMH/qijveE6YgOYTQC5nYc2Np1KvC7hQkIu4nuyczyYlrNQl/TWv+SVI8PjIs0PYuCuD3gUqoVEi8d94HbrOPzFEpbwDS9P4qL39DGmco1Q56Vqxe6sRI0vDWPb5gCP1lSgs3PMECVn7Wor/pTvcL+C+U2jLwWJUl0vbyWCL7ngl/3iTssV7qBpUrI7Oclwp8LrQo9fPj0DL4gE9rNanpEWjQ6yPGaysIL1zLHtRghhm52A22NJGp71jkS2KEpLi6ZWFYjMeuXw5eOQFhqFlzyRJOmLYa3B607TLWuyo2L2CxAfMmq0FGfemvrkLZIWtlQKK4y9ImpsURwaGT2XCtThFtHl77wEss913nC+T2dX3O5Bl0UmxFd5S3mVM109I8c4lDosxnAjRfS9MheFlrG0gjSJSBCw57x7f0=
rooms:
secure: YZ34vsfw1TtftJypg1MyP4+ihONI4gaxeS3FghBQli6+EezjzcxOXyj5VD+x0ucXfDeTaDrFfmVN0SAGObOoZUE+ea9KAoTo50tRLaD9kwOTACiekZalC4uuBguH0D1/A6vlbU8dchsr9mvIhbisG6mTncdPtqGYYHtyBQme6ngmmHbVAQFZcIBHmNuDb/HWhSr8KMEuyB6+mBXLYELHnXnf26cOhdGNaagqCOJTiemX85RGwIuOPxyBhKDFMLyHohDT7FJMH/qijveE6YgOYTQC5nYc2Np1KvC7hQkIu4nuyczyYlrNQl/TWv+SVI8PjIs0PYuCuD3gUqoVEi8d94HbrOPzFEpbwDS9P4qL39DGmco1Q56Vqxe6sRI0vDWPb5gCP1lSgs3PMECVn7Wor/pTvcL+C+U2jLwWJUl0vbyWCL7ngl/3iTssV7qBpUrI7Oclwp8LrQo9fPj0DL4gE9rNanpEWjQ6yPGaysIL1zLHtRghhm52A22NJGp71jkS2KEpLi6ZWFYjMeuXw5eOQFhqFlzyRJOmLYa3B607TLWuyo2L2CxAfMmq0FGfemvrkLZIWtlQKK4y9ImpsURwaGT2XCtThFtHl77wEss913nC+T2dX3O5Bl0UmxFd5S3mVM109I8c4lDosxnAjRfS9MheFlrG0gjSJSBCw57x7f0=
on_success: change
on_failure: always
+202 -9
View File
@@ -1,17 +1,209 @@
# 0.5.0
[Documentation](http://fission.io/docs/0.5.0/)
## Downloads for 0.5.0
# 0.9.2
[Documentation](https://docs.fission.io/0.9.2/)
## Downloads for 0.9.2
filename | sha256 hash
-------- | -----------
[fission-core-0.5.0.tgz](https://github.com/fission/fission/releases/download/0.5.0/fission-core-0.5.0.tgz) | `6ee668de0870a52d16a051bb5d60c12e523a87c54269b321bdab6dd97f883ee4`
[fission-all-0.5.0.tgz](https://github.com/fission/fission/releases/download/0.5.0/fission-all-0.5.0.tgz) | `2f72fe4ba836d84e60538ecf401a508384bb194c979688be69f1370af0c071bb`
[fission-cli-osx](https://github.com/fission/fission/releases/download/0.5.0/fission-cli-osx) | `1b7de8a5bbf7710ba66997003e7a63f9887ef89d7f7e027c31a78f2f51a11536`
[fission-cli-linux](https://github.com/fission/fission/releases/download/0.5.0/fission-cli-linux) | `7bbff6d166daf16f6e08b82753cbd6bd0d407e12372a0c6612b6f1a11ed39e09`
[fission-cli-windows.exe](https://github.com/fission/fission/releases/download/0.5.0/fission-cli-windows.exe) | `5db18472bd071daf39ad681c6ad3b4fd4e89ccb243f81e7ea67785e0c7b2f209`
# Change Log
## [0.9.2](https://github.com/fission/fission/tree/0.9.2) (2018-07-25)
[Full Changelog](https://github.com/fission/fission/compare/0.9.1...0.9.2)
**Merged pull requests:**
- Helm lint check in Travis CI [\#799](https://github.com/fission/fission/pull/799) ([erwinvaneyk](https://github.com/erwinvaneyk))
- Spelling. [\#797](https://github.com/fission/fission/pull/797) ([WrathZA](https://github.com/WrathZA))
- change image pull policy of builder manager [\#793](https://github.com/fission/fission/pull/793) ([xiekeyang](https://github.com/xiekeyang))
- Delete namespace in background to reduce build time [\#791](https://github.com/fission/fission/pull/791) ([life1347](https://github.com/life1347))
- Break & Stop the build immediately if a non-zero exit code was returned [\#790](https://github.com/fission/fission/pull/790) ([life1347](https://github.com/life1347))
- Add changelog. [\#789](https://github.com/fission/fission/pull/789) ([smruthi2187](https://github.com/smruthi2187))
- changes needed for 0.9.1 [\#788](https://github.com/fission/fission/pull/788) ([smruthi2187](https://github.com/smruthi2187))
- Working version of Java builder with Maven [\#783](https://github.com/fission/fission/pull/783) ([vishal-biyani](https://github.com/vishal-biyani))
- Add query options to `fission function test` [\#782](https://github.com/fission/fission/pull/782) ([erwinvaneyk](https://github.com/erwinvaneyk))
- Add go environment vendor directory support [\#781](https://github.com/fission/fission/pull/781) ([life1347](https://github.com/life1347))
- Added readme for JVM environment [\#768](https://github.com/fission/fission/pull/768) ([vishal-biyani](https://github.com/vishal-biyani))
- Fix spec command overrides existing archive's url of a package [\#764](https://github.com/fission/fission/pull/764) ([life1347](https://github.com/life1347))
- Fixed typos from from goreportcard [\#760](https://github.com/fission/fission/pull/760) ([vishal-biyani](https://github.com/vishal-biyani))
- Updating releasing notes with details and structure [\#738](https://github.com/fission/fission/pull/738) ([vishal-biyani](https://github.com/vishal-biyani))
- Allow router round-trip to be configurable [\#713](https://github.com/fission/fission/pull/713) ([xiekeyang](https://github.com/xiekeyang))
## [0.9.1](https://github.com/fission/fission/tree/0.9.1) (2018-07-07)
[Full Changelog](https://github.com/fission/fission/compare/0.9.0...0.9.1)
**Merged pull requests:**
- Committing changelog. [\#780](https://github.com/fission/fission/pull/780) ([smruthi2187](https://github.com/smruthi2187))
- Changes in charts for release 0.9.0 [\#778](https://github.com/fission/fission/pull/778) ([smruthi2187](https://github.com/smruthi2187))
- Change flag name to KeepArchive for backward compatibility [\#787](https://github.com/fission/fission/pull/787) ([life1347](https://github.com/life1347))
- Fix go env plugin [\#784](https://github.com/fission/fission/pull/784) ([life1347](https://github.com/life1347))
- Fix “rm: missing operand” in release script [\#779](https://github.com/fission/fission/pull/779) ([life1347](https://github.com/life1347))
## [0.9.0](https://github.com/fission/fission/tree/0.9.0) (2018-07-04)
[Full Changelog](https://github.com/fission/fission/compare/0.8.0...0.9.0)
**Merged pull requests:**
- Fix executor wrongly passes loop variable reference to function [\#751](https://github.com/fission/fission/pull/751) ([life1347](https://github.com/life1347))
- Python Environment: add gevent based WSGI server framework [\#750](https://github.com/fission/fission/pull/750) ([xiekeyang](https://github.com/xiekeyang))
- Temporarily disabling the tests so that other PRs can be worked on [\#737](https://github.com/fission/fission/pull/737) ([vishal-biyani](https://github.com/vishal-biyani))
- add build exe to gitignore [\#736](https://github.com/fission/fission/pull/736) ([xiekeyang](https://github.com/xiekeyang))
- ArchiveLiteralSizeLimit: Use Constant Instead Hard Code [\#731](https://github.com/fission/fission/pull/731) ([xiekeyang](https://github.com/xiekeyang))
- Environment warning message bugfix [\#725](https://github.com/fission/fission/pull/725) ([soamvasani](https://github.com/soamvasani))
- V0.8.0 [\#722](https://github.com/fission/fission/pull/722) ([vishal-biyani](https://github.com/vishal-biyani))
- Make fetcher resource requests and limits configurable [\#708](https://github.com/fission/fission/pull/708) ([xiekeyang](https://github.com/xiekeyang))
- Add steps to render & upload fission installation YAML [\#745](https://github.com/fission/fission/pull/745) ([life1347](https://github.com/life1347))
- Fix executor not reaps idle function pods for functions with executortype newdeploy [\#744](https://github.com/fission/fission/pull/744) ([life1347](https://github.com/life1347))
- Testing with keep alive settings for connections [\#742](https://github.com/fission/fission/pull/742) ([vishal-biyani](https://github.com/vishal-biyani))
- instead hard code by variable in error message [\#735](https://github.com/fission/fission/pull/735) ([xiekeyang](https://github.com/xiekeyang))
- envns should be availabe in message line [\#734](https://github.com/fission/fission/pull/734) ([xiekeyang](https://github.com/xiekeyang))
- Support annotations in environment specs [\#733](https://github.com/fission/fission/pull/733) ([erwinvaneyk](https://github.com/erwinvaneyk))
- Extract portforward to separate package [\#728](https://github.com/fission/fission/pull/728) ([erwinvaneyk](https://github.com/erwinvaneyk))
- Push NATS error messages to error queue [\#724](https://github.com/fission/fission/pull/724) ([Amusement](https://github.com/Amusement))
- Fix for Windows CLI Port Forwarding [\#715](https://github.com/fission/fission/pull/715) ([thejosephstevens](https://github.com/thejosephstevens))
- Router liveness [\#701](https://github.com/fission/fission/pull/701) ([smruthi2187](https://github.com/smruthi2187))
- Archives bigger than 256K size need env variable for uploading [\#697](https://github.com/fission/fission/pull/697) ([vishal-biyani](https://github.com/vishal-biyani))
- Convert go-env Dockerfile into a multi-stage build [\#683](https://github.com/fission/fission/pull/683) ([jgall](https://github.com/jgall))
- Move build process from host to docker container during release process [\#682](https://github.com/fission/fission/pull/682) ([life1347](https://github.com/life1347))
- Added a flag to control the extraction of archive based on user input [\#675](https://github.com/fission/fission/pull/675) ([vishal-biyani](https://github.com/vishal-biyani))
- Java env alpha [\#656](https://github.com/fission/fission/pull/656) ([vishal-biyani](https://github.com/vishal-biyani))
## [0.8.0](https://github.com/fission/fission/tree/0.8.0) (2018-06-05)
[Full Changelog](https://github.com/fission/fission/compare/0.7.2...0.8.0)
**Merged pull requests:**
- Pre-install/pre-upgrade hooks to verify func references and assign restricted role bindings [\#717](https://github.com/fission/fission/pull/717) ([smruthi2187](https://github.com/smruthi2187))
- Logger daemonset's update strategy [\#714](https://github.com/fission/fission/pull/714) ([vishal-biyani](https://github.com/vishal-biyani))
- Check spec directory exists before reading spec files [\#709](https://github.com/fission/fission/pull/709) ([life1347](https://github.com/life1347))
- Formatted specifiers are not compatible with variables [\#706](https://github.com/fission/fission/pull/706) ([xiekeyang](https://github.com/xiekeyang))
- Indicate HTTP status code by library const [\#703](https://github.com/fission/fission/pull/703) ([xiekeyang](https://github.com/xiekeyang))
- docker-distribution version bump for windows compatibility [\#691](https://github.com/fission/fission/pull/691) ([thejosephstevens](https://github.com/thejosephstevens))
- Version -\> 0.7.2 [\#670](https://github.com/fission/fission/pull/670) ([life1347](https://github.com/life1347))
- Java environment Design & considerations [\#642](https://github.com/fission/fission/pull/642) ([vishal-biyani](https://github.com/vishal-biyani))
- Working version of Ingress integration [\#688](https://github.com/fission/fission/pull/688) ([vishal-biyani](https://github.com/vishal-biyani))
- Update k8s dependencies to 1.10 [\#687](https://github.com/fission/fission/pull/687) ([life1347](https://github.com/life1347))
- Add time trigger cron spec examination tool [\#680](https://github.com/fission/fission/pull/680) ([life1347](https://github.com/life1347))
- Fission metrics integration [\#677](https://github.com/fission/fission/pull/677) ([soamvasani](https://github.com/soamvasani))
- Replace Werkzeug with Bjoern as underlying WSGI server [\#672](https://github.com/fission/fission/pull/672) ([life1347](https://github.com/life1347))
- Enabling multi-tenancy for fission objects. [\#655](https://github.com/fission/fission/pull/655) ([smruthi2187](https://github.com/smruthi2187))
## [0.7.2](https://github.com/fission/fission/tree/0.7.2) (2018-05-05)
[Full Changelog](https://github.com/fission/fission/compare/0.7.1...0.7.2)
**Merged pull requests:**
- Add benchmark script [\#666](https://github.com/fission/fission/pull/666) ([life1347](https://github.com/life1347))
- Fixed the issue with update wiping values [\#663](https://github.com/fission/fission/pull/663) ([vishal-biyani](https://github.com/vishal-biyani))
- Fix newdeploy backend failed to delete deployment due to incorrect resource version [\#657](https://github.com/fission/fission/pull/657) ([life1347](https://github.com/life1347))
- Function update should be possible without change to code [\#652](https://github.com/fission/fission/pull/652) ([vishal-biyani](https://github.com/vishal-biyani))
- Fixes the issue with fn test and adds relevant test cases, fixes \#650 [\#651](https://github.com/fission/fission/pull/651) ([vishal-biyani](https://github.com/vishal-biyani))
- Fix test cases occasional failure [\#647](https://github.com/fission/fission/pull/647) ([life1347](https://github.com/life1347))
- Change time precision for fluentd influxdb plugin to nano second [\#646](https://github.com/fission/fission/pull/646) ([life1347](https://github.com/life1347))
- Setting buildStatus to pending when function's source archive is updated. [\#637](https://github.com/fission/fission/pull/637) ([smruthi2187](https://github.com/smruthi2187))
- Fix SEGFAULT issue when buildmgr failed to update package [\#635](https://github.com/fission/fission/pull/635) ([life1347](https://github.com/life1347))
- Fix executor does not reap specialized function pod when env no longer exists [\#633](https://github.com/fission/fission/pull/633) ([life1347](https://github.com/life1347))
- Update readme to point to the proper link [\#628](https://github.com/fission/fission/pull/628) ([jgall](https://github.com/jgall))
- Changes needed for release 0.7.1 [\#622](https://github.com/fission/fission/pull/622) ([smruthi2187](https://github.com/smruthi2187))
- Add default value to cli flag [\#619](https://github.com/fission/fission/pull/619) ([life1347](https://github.com/life1347))
- Remove port forward in tests for router, controller and nats pods [\#611](https://github.com/fission/fission/pull/611) ([smruthi2187](https://github.com/smruthi2187))
- meaningful error message when fetch request is received for a package when build is not successful. [\#661](https://github.com/fission/fission/pull/661) ([smruthi2187](https://github.com/smruthi2187))
- Delete deployment with proper delete propagation policy [\#630](https://github.com/fission/fission/pull/630) ([life1347](https://github.com/life1347))
- Fix buildmgr SEGFAULT when it failed to update package [\#626](https://github.com/fission/fission/pull/626) ([life1347](https://github.com/life1347))
- Fission upgrade tests [\#605](https://github.com/fission/fission/pull/605) ([vishal-biyani](https://github.com/vishal-biyani))
- Removed the fn pods functionality [\#594](https://github.com/fission/fission/pull/594) ([vishal-biyani](https://github.com/vishal-biyani))
- Testing proposal: Requirements and frameworks exploration [\#581](https://github.com/fission/fission/pull/581) ([vishal-biyani](https://github.com/vishal-biyani))
## [0.7.1](https://github.com/fission/fission/tree/0.7.1) (2018-04-10)
[Full Changelog](https://github.com/fission/fission/compare/0.7.0...0.7.1)
**Merged pull requests:**
- Prevent releasing idle connections because transport is shared. [\#609](https://github.com/fission/fission/pull/609) ([smruthi2187](https://github.com/smruthi2187))
- Fix components crash before crds creation [\#602](https://github.com/fission/fission/pull/602) ([life1347](https://github.com/life1347))
- updates to changelog. [\#598](https://github.com/fission/fission/pull/598) ([smruthi2187](https://github.com/smruthi2187))
- changes needed for release 0.7.0 [\#597](https://github.com/fission/fission/pull/597) ([smruthi2187](https://github.com/smruthi2187))
- `fission X create --spec` flags for env and trigger create commands [\#607](https://github.com/fission/fission/pull/607) ([soamvasani](https://github.com/soamvasani))
- Updating releasing guideliness with a few more details. [\#599](https://github.com/fission/fission/pull/599) ([smruthi2187](https://github.com/smruthi2187))
- Add deprecated message to subcommand pods [\#592](https://github.com/fission/fission/pull/592) ([life1347](https://github.com/life1347))
- Add validate function to crd resource and do validate before creation/update [\#580](https://github.com/fission/fission/pull/580) ([life1347](https://github.com/life1347))
- Invalidate stale router cache entry with podIP's for deleted pods. [\#546](https://github.com/fission/fission/pull/546) ([smruthi2187](https://github.com/smruthi2187))
- Use a separate controller loop to watch functions change and create a service [\#544](https://github.com/fission/fission/pull/544) ([life1347](https://github.com/life1347))
- E2E test for NATS-streaming trigger [\#338](https://github.com/fission/fission/pull/338) ([soamvasani](https://github.com/soamvasani))
## [0.7.0](https://github.com/fission/fission/tree/0.7.0) (2018-04-02)
[Full Changelog](https://github.com/fission/fission/compare/0.6.1...0.7.0)
**Merged pull requests:**
- bug fix: spec dir flag [\#595](https://github.com/fission/fission/pull/595) ([xiekeyang](https://github.com/xiekeyang))
- Add steps to set FISSION\_ROUTER env variable & update docs [\#593](https://github.com/fission/fission/pull/593) ([life1347](https://github.com/life1347))
- Adding routerUrl parameter for kubewatch, timer, message queue trigge… [\#591](https://github.com/fission/fission/pull/591) ([smruthi2187](https://github.com/smruthi2187))
- Uses proper way to get server URL [\#587](https://github.com/fission/fission/pull/587) ([vishal-biyani](https://github.com/vishal-biyani))
- Check if the requested file already exists in fetcher and skip fetch [\#584](https://github.com/fission/fission/pull/584) ([smruthi2187](https://github.com/smruthi2187))
- Add golang example to installation guide [\#578](https://github.com/fission/fission/pull/578) ([clee](https://github.com/clee))
- Fixes the issue \#559 with env versions [\#569](https://github.com/fission/fission/pull/569) ([vishal-biyani](https://github.com/vishal-biyani))
- Add post-upgrade-job to track fission upgrade [\#564](https://github.com/fission/fission/pull/564) ([life1347](https://github.com/life1347))
- Prepending a slash to user input url if missing. [\#547](https://github.com/fission/fission/pull/547) ([smruthi2187](https://github.com/smruthi2187))
- Add verbosity flag and verbose logs for portforwarder [\#575](https://github.com/fission/fission/pull/575) ([soamvasani](https://github.com/soamvasani))
- Spec validator, better errors, apply waits for previous build [\#560](https://github.com/fission/fission/pull/560) ([soamvasani](https://github.com/soamvasani))
- Tests for function update [\#550](https://github.com/fission/fission/pull/550) ([vishal-biyani](https://github.com/vishal-biyani))
- Show fission deployment version with cli [\#538](https://github.com/fission/fission/pull/538) ([life1347](https://github.com/life1347))
## [0.6.1](https://github.com/fission/fission/tree/0.6.1) (2018-03-22)
[Full Changelog](https://github.com/fission/fission/compare/0.6.0...0.6.1)
**Merged pull requests:**
- This change fixes an error in a yaml file in the fission-core chart. [\#563](https://github.com/fission/fission/pull/563) ([smartding](https://github.com/smartding))
- \[ci skip\] update release number [\#561](https://github.com/fission/fission/pull/561) ([appleboy](https://github.com/appleboy))
- Fixes \#537 - warning should not be given when updating to newdeploy [\#545](https://github.com/fission/fission/pull/545) ([vishal-biyani](https://github.com/vishal-biyani))
- Docs update [\#542](https://github.com/fission/fission/pull/542) ([soamvasani](https://github.com/soamvasani))
- Release script updates [\#541](https://github.com/fission/fission/pull/541) ([soamvasani](https://github.com/soamvasani))
- Show warning when trying to create a route with non-existent function \(\#238\) [\#539](https://github.com/fission/fission/pull/539) ([life1347](https://github.com/life1347))
- Fix executor failed to clean cache & kubeobjs after function deleted \(\#533\) [\#534](https://github.com/fission/fission/pull/534) ([life1347](https://github.com/life1347))
- Delete healthz log [\#525](https://github.com/fission/fission/pull/525) ([smruthi2187](https://github.com/smruthi2187))
- Always retry when istio is enabled. [\#536](https://github.com/fission/fission/pull/536) ([life1347](https://github.com/life1347))
- Fix executor tries to create a new deployment when a function is updated [\#524](https://github.com/fission/fission/pull/524) ([life1347](https://github.com/life1347))
- Add container spec config options to \(build\) environments [\#413](https://github.com/fission/fission/pull/413) ([erwinvaneyk](https://github.com/erwinvaneyk))
## [0.6.0](https://github.com/fission/fission/tree/0.6.0) (2018-03-01)
[Full Changelog](https://github.com/fission/fission/compare/latest...0.6.0)
**Merged pull requests:**
- Detect fission namespace in cli [\#519](https://github.com/fission/fission/pull/519) ([soamvasani](https://github.com/soamvasani))
- Default values for FISSION\_\* env vars [\#518](https://github.com/fission/fission/pull/518) ([soamvasani](https://github.com/soamvasani))
- Add chart version to job name [\#516](https://github.com/fission/fission/pull/516) ([soamvasani](https://github.com/soamvasani))
- Fix CLI not update function's secret/configmap correctly [\#512](https://github.com/fission/fission/pull/512) ([life1347](https://github.com/life1347))
- Adds latest tags and pushes to dockerhub for fetcher and fission-bundle [\#509](https://github.com/fission/fission/pull/509) ([vishal-biyani](https://github.com/vishal-biyani))
- Fixes the backward compatibility with older environment versions [\#508](https://github.com/fission/fission/pull/508) ([vishal-biyani](https://github.com/vishal-biyani))
- Update Fn: Executor New Deployment [\#504](https://github.com/fission/fission/pull/504) ([vishal-biyani](https://github.com/vishal-biyani))
- Adds default resources for fetcher pod [\#500](https://github.com/fission/fission/pull/500) ([vishal-biyani](https://github.com/vishal-biyani))
- Documentation Revamp [\#496](https://github.com/fission/fission/pull/496) ([vishal-biyani](https://github.com/vishal-biyani))
- Delete and list orphan pkgs [\#468](https://github.com/fission/fission/pull/468) ([smruthi2187](https://github.com/smruthi2187))
- Service type ClusterIP - Controller port forward through CLI [\#431](https://github.com/fission/fission/pull/431) ([prithviramesh](https://github.com/prithviramesh))
- Istio integration [\#421](https://github.com/fission/fission/pull/421) ([life1347](https://github.com/life1347))
- Implement support for Azure storage message queue triggers [\#371](https://github.com/fission/fission/pull/371) ([peterhuene](https://github.com/peterhuene))
## [latest](https://github.com/fission/fission/tree/latest) (2018-03-01)
[Full Changelog](https://github.com/fission/fission/compare/0.5.0...latest)
**Merged pull requests:**
- Release checklist [\#522](https://github.com/fission/fission/pull/522) ([soamvasani](https://github.com/soamvasani))
- Fix post-install-job container failure due to command not found [\#514](https://github.com/fission/fission/pull/514) ([life1347](https://github.com/life1347))
- Replace the release with the latest tag. [\#513](https://github.com/fission/fission/pull/513) ([smruthi2187](https://github.com/smruthi2187))
- Go: Set image to right version, update example readme [\#497](https://github.com/fission/fission/pull/497) ([soamvasani](https://github.com/soamvasani))
- Remove a noisy log from router [\#495](https://github.com/fission/fission/pull/495) ([soamvasani](https://github.com/soamvasani))
- Improve release script [\#494](https://github.com/fission/fission/pull/494) ([life1347](https://github.com/life1347))
- Update SHA256 HASH in CHANGELOG.md due to binaries update [\#493](https://github.com/fission/fission/pull/493) ([life1347](https://github.com/life1347))
- Go builder for single file functions [\#492](https://github.com/fission/fission/pull/492) ([soamvasani](https://github.com/soamvasani))
- CI modifications [\#491](https://github.com/fission/fission/pull/491) ([smruthi2187](https://github.com/smruthi2187))
- Add upgrade guide from 0.4.x to 0.5.0 [\#490](https://github.com/fission/fission/pull/490) ([life1347](https://github.com/life1347))
- Version -\> 0.5.0 [\#489](https://github.com/fission/fission/pull/489) ([life1347](https://github.com/life1347))
## [0.5.0](https://github.com/fission/fission/tree/0.5.0) (2018-02-07)
[Full Changelog](https://github.com/fission/fission/compare/0.4.1...0.5.0)
@@ -31,6 +223,7 @@ filename | sha256 hash
- NewDeploy Doc [\#432](https://github.com/fission/fission/pull/432) ([vishal-biyani](https://github.com/vishal-biyani))
- Add go vet check [\#430](https://github.com/fission/fission/pull/430) ([life1347](https://github.com/life1347))
- Fix potential nil pointer problem [\#485](https://github.com/fission/fission/pull/485) ([life1347](https://github.com/life1347))
- Add simple usage doc for accessing secret/configmap in function [\#484](https://github.com/fission/fission/pull/484) ([life1347](https://github.com/life1347))
- Helm hook bugfixes: run on upgrade, delete on completion [\#473](https://github.com/fission/fission/pull/473) ([soamvasani](https://github.com/soamvasani))
- Archive pruner [\#471](https://github.com/fission/fission/pull/471) ([smruthi2187](https://github.com/smruthi2187))
- Build and push fluentd image on release; update chart to use that image [\#462](https://github.com/fission/fission/pull/462) ([soamvasani](https://github.com/soamvasani))
@@ -236,7 +429,7 @@ filename | sha256 hash
- Upgrade node environment to Node.js 7.6.0+ [\#151](https://github.com/fission/fission/pull/151) ([RobertHerhold](https://github.com/RobertHerhold))
- use fmt.Errorf instead of error.New\(\) [\#149](https://github.com/fission/fission/pull/149) ([maxwell92](https://github.com/maxwell92))
- Return 201 for created resources [\#148](https://github.com/fission/fission/pull/148) ([RobertHerhold](https://github.com/RobertHerhold))
- Set correct Content-Type in the http response [\#147](https://github.com/fission/fission/pull/147) ([LingxianKong](https://github.com/LingxianKong))
- Set correct Content-Type in the http response [\#147](https://github.com/fission/fission/pull/147) ([lingxiankong](https://github.com/lingxiankong))
- Make it more clear where to clone this repo [\#145](https://github.com/fission/fission/pull/145) ([RobertHerhold](https://github.com/RobertHerhold))
- Fix function delete with uid [\#142](https://github.com/fission/fission/pull/142) ([yqf3139](https://github.com/yqf3139))
- Fixed pod has no ip \(\#139\) [\#141](https://github.com/fission/fission/pull/141) ([life1347](https://github.com/life1347))
+2 -2
View File
@@ -22,7 +22,7 @@ minikube, you'll need to set the proper environment variables with
```
# Get dependencies
$ glide install
$ glide install -v
# Build fission server and an image
$ pushd fission-bundle
@@ -41,7 +41,7 @@ minikube and its built-in docker daemon:
Next, install fission with this image on your kubernetes cluster using the helm chart:
```
$ helm install --set "image=minikube/fission-bundle,pullPolicy=IfNotPresent,analytics=false" charts/fission-all
$ helm install --set "image=minikube/fission-bundle,imageTag=latest,pullPolicy=IfNotPresent,analytics=false" charts/fission-all
```
And if you're changing the CLI too, you can build it with:
+2 -2
View File
@@ -1,4 +1,4 @@
baseURL = "http://fission.io/docs/0.5.0"
baseURL = "http://fission.io/docs/0.6.1"
languageCode = "en-US"
defaultContentLanguage = "en"
@@ -11,7 +11,7 @@ defaultContentLanguageInSubdir= true
editURL = "https://github.com/fission/fission/edit/master/Documentation/docs-site/content/"
description = "Documentation for Fission"
author = "Fission"
showVisitedLinks = true
showVisitedLinks = false
themeVariant = "fission"
[outputs]
+2 -2
View File
@@ -92,8 +92,8 @@ Usage
$ fission route create --method GET --url /hello --function hello
# Run the function. This takes about 100msec the first time.
$ curl http://$FISSION_ROUTER/hello
$ fission function test --name hello
Hello, world!
```
See the [examples](examples) directory for more.
See the [examples](examples) directory for more.
@@ -7,4 +7,5 @@ weight: 30
# Fission Concepts
#### Understanding Fission terminology and concepts
This is an overview of the few main concepts in Fission: Functions,
Environments, and Triggers.
@@ -1,5 +1,5 @@
---
title: "Environment"
title: "Environments"
draft: false
weight: 32
---
@@ -7,4 +7,4 @@ chapter : true
# Installation
### Installing and upgrading Fission
### Installing and upgrading Fission
@@ -20,19 +20,19 @@ ready. If you already have helm, [skip ahead to the fission install](#install-fi
Ensure you have the Kubernetes CLI.
You can get the Kubernetes CLI for OSX like this:
```
```sh
$ curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/darwin/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin
```
Or, for Linux:
```
```sh
$ curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin
```
Ensure you have access to a cluster; use kubectl to check your
Kubernetes version:
```
```sh
$ kubectl version
```
@@ -47,7 +47,7 @@ the next section](#install-fission).
First, you'll need the helm CLI:
On __OS X__:
```
```sh
$ curl -LO https://storage.googleapis.com/kubernetes-helm/helm-v2.7.0-darwin-amd64.tar.gz
$ tar xzf helm-v2.7.0-darwin-amd64.tar.gz
@@ -56,7 +56,7 @@ $ mv darwin-amd64/helm /usr/local/bin
```
On __Linux__:
```
```sh
$ curl -LO https://storage.googleapis.com/kubernetes-helm/helm-v2.7.0-linux-amd64.tar.gz
$ tar xzf helm-v2.7.0-linux-amd64.tar.gz
@@ -66,7 +66,7 @@ $ mv linux-amd64/helm /usr/local/bin
Next, install the Helm server on your Kubernetes cluster:
```
```sh
$ helm init
```
@@ -74,7 +74,7 @@ $ helm init
#### Minikube
```
```sh
$ helm install --namespace fission --set serviceType=NodePort https://github.com/fission/fission/releases/download/0.6.0/fission-all-0.6.0.tgz
```
@@ -84,7 +84,7 @@ want to expose anything outside the cluster.
#### Cloud hosted clusters (GKE, AWS, Azure etc.)
```
```sh
$ helm install --namespace fission https://github.com/fission/fission/releases/download/0.6.0/fission-all-0.6.0.tgz
```
@@ -94,7 +94,7 @@ The fission-all helm chart installs a full set of services including
the NATS message queue, influxDB for logs, etc. If you want a more
minimal setup, you can install the fission-core chart instead:
```
```sh
$ helm install --namespace fission https://github.com/fission/fission/releases/download/0.6.0/fission-core-0.6.0.tgz
```
@@ -104,13 +104,13 @@ $ helm install --namespace fission https://github.com/fission/fission/releases/d
Get the CLI binary for Mac:
```
```sh
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.6.0/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
```
#### Linux
```
```sh
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.6.0/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
```
@@ -123,7 +123,7 @@ this windows executable: [fission.exe](https://github.com/fission/fission/releas
Finally, you're ready to use Fission!
```
```sh
$ fission env create --name nodejs --image fission/node-env:0.6.0
$ curl -LO https://raw.githubusercontent.com/fission/fission/master/examples/nodejs/hello.js
@@ -134,6 +134,19 @@ $ fission function test --name hello
Hello, world!
```
For a compiled language like Go:
```sh
$ fission env create --name go --image fission/go-env:0.6.0 --builder fission/go-builder:0.6.0
$ curl -LO https://raw.githubusercontent.com/fission/fission/master/examples/go/hello.go
$ fission function create --name gohello --env go --src hello.go --entrypoint Handler
$ fission function test --name gohello
Hello, world!
```
### What's next?
If something went wrong, we'd love to help -- please [drop by the
@@ -1,28 +1,38 @@
---
title: "Accessing Secret/configmap in function"
title: "Accessing Secrets in Functions"
draft: false
weight: 47
---
From fission v0.5.0 and later, functions are able to access [Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) and [ConfigMaps](https://kubernetes.io/docs/concepts/storage/volumes/#configmap) specified by users.
Functions can access Kubernetes
[Secrets](https://kubernetes.io/docs/concepts/configuration/secret/)
and
[ConfigMaps](https://kubernetes.io/docs/concepts/storage/volumes/#configmap).
### Create Secret and ConfigMap
Use secrets for things like API keys, authentication tokens, and so
on.
You can create Secret and ConfigMap with CLI.
Use config maps for any other configuration that doesn't need to be a
secret.
### Create A Secret or a ConfigMap
You can create a Secret or ConfigMap with the Kubernetes CLI:
``` bash
$ kubectl -n default create secret generic foo --from-literal=TEST_KEY="TESTVALUE"
$ kubectl -n default create configmap bar --from-literal=TEST_KEY=TESTVALUE
$ kubectl -n default create secret generic my-secret --from-literal=TEST_KEY="TESTVALUE"
$ kubectl -n default create configmap my-configmap --from-literal=TEST_KEY="TESTVALUE"
```
Or use `kubectl create -f <filename.yaml>` to create these from a YAML file.
Or, use `kubectl create -f <filename.yaml>` to create these from a YAML file.
``` yaml
apiVersion: v1
kind: Secret
metadata:
namespace: default
name: foo
name: my-secret
data:
TEST_KEY: VEVTVFZBTFVF # value after base64 encode
type: Opaque
@@ -32,14 +42,16 @@ apiVersion: v1
kind: ConfigMap
metadata:
namespace: default
name: bar
name: my-configmap
data:
TEST_KEY: TESTVALUE
```
### Access Secret and ConfigMap
### Accessing Secrets and ConfigMaps
Since content of Secret and ConfigMap are key-value pairs, functions can access them with following paths:
Secrets and configmaps are accessed similarly. Each secret or
configmap is a set of key value pairs. Fission sets these up as files
you can read from your function.
``` bash
# Secret path
@@ -52,24 +64,25 @@ Since content of Secret and ConfigMap are key-value pairs, functions can access
From the previous example, the paths are:
``` bash
# secret foo
/secrets/default/foo/TEST_KEY
# secret my-secret
/secrets/default/my-secret/TEST_KEY
# confimap bar
/configs/default/bar/TEST_KEY
# confimap my-configmap
/configs/default/my-configmap/TEST_KEY
```
Now, let's create a simple python function (leaker.py) that return value of Secret `foo` and ConfigMap `bar`.
Now, let's create a simple python function (leaker.py) that returns
the value of Secret `my-secret` and ConfigMap `my-configmap`.
``` python
# leaker.py
def main():
path = "/configs/default/bar/TEST_KEY"
path = "/configs/default/my-configmap/TEST_KEY"
f = open(path, "r")
config = f.read()
path = "/secrets/default/foo/TEST_KEY"
path = "/secrets/default/my-secret/TEST_KEY"
f = open(path, "r")
secret = f.read()
@@ -79,27 +92,29 @@ def main():
```
Create environment, function and http trigger.
Create an environment and a function:
``` bash
# create python env
$ fission env create --name python --image fission/python-env
# create function named "leaker"
$ fission fn create --name leaker --env python --code leaker.py --secret foo --configmap bar
# create route(http trigger)
$ fission route create --function leaker --url /leaker --method GET
$ fission fn create --name leaker --env python --code leaker.py --secret my-secret --configmap my-configmap
```
Try to access the function, the output should look like following.
Run the function, and the output should look like this:
``` bash
$ curl http://$FISSION_ROUTER/leaker
$ fission function test --name leaker
ConfigMap: TESTVALUE
Secret: TESTVALUE
```
Note: If the Secret or ConfigMap value is updated, the function may not get the updated value for some time; it may get a cached older value.
{{% notice note %}}
If the Secret or ConfigMap value is updated, the function may
not get the updated value for some time; it may get a cached older
value.
{{% /notice %}}
@@ -9,7 +9,7 @@ weight: 42
You can create an environment on your cluster from an image for that language. Optionally, you can specify CPU and memory resource limits. You can also specify the number of initially pre-warmed pods, which is called the poolsize.
```
fission env create --name node --image fission/node-env:0.4.0 --mincpu 40 --maxcpu 80 --minmemory 64 --maxmemory 128 --poolsize 4
$ fission env create --name node --image fission/node-env:0.4.0 --mincpu 40 --maxcpu 80 --minmemory 64 --maxmemory 128 --poolsize 4
```
In case of pool based executor, the resources specified for environment are used for function pod as well. In case of new deployment executor, you can override the resources when you create a function.
@@ -19,10 +19,10 @@ In case of pool based executor, the resources specified for environment are used
When you create an environment, you can specify a builder image and builder command which will be used for building from source code. You can override the build command when creating a function. For more details on builder and packages you should check out examples in [Functions](../functions) and [packages](../package)
```
fission env create --name python --image fission/python-env:latest --builder fission/python-builder:latest
$ fission env create --name python --image fission/python-env:latest --builder fission/python-builder:latest
```
### Viweing environment information
### Viewing environment information
You can list the environments or view information of an individual environment:
@@ -30,8 +30,8 @@ You can list the environments or view information of an individual environment:
$ fission env list
NAME UID IMAGE POOLSIZE MINCPU MAXCPU MINMEMORY MAXMEMORY
node ac84d62e-001f-11e8-85c9-42010aa00010 fission/node-env:0.4.0 4 40m 80m 64Mi 128Mi
$
$ fission env get --name node
NAME UID IMAGE
node ac84d62e-001f-11e8-85c9-42010aa00010 fission/node-env:0.4.0
```
```
@@ -62,7 +62,7 @@ Status code distribution:
While the load is being generated, we will watch the HorizontalPodAutoscaler and how it scales over period of time. As you can notice, the number of pods is scaled from 1 to 3 after the load rises from 8 - 103%. After the load generator stops, it takes a few iterations to scale down from 3 to 1 pod.
```
$ k -n fission-function get hpa -w
$ kubectl -n fission-function get hpa -w
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
hello-qoxmothj Deployment/hello-qoxmothj 5% / 50% 1 6 1 3m
hello-qoxmothj Deployment/hello-qoxmothj 8% / 50% 1 6 1 3m
@@ -83,4 +83,4 @@ hello-qoxmothj Deployment/hello-qoxmothj 7% / 50% 1 6 1
hello-qoxmothj Deployment/hello-qoxmothj 6% / 50% 1 6 1 11m
hello-qoxmothj Deployment/hello-qoxmothj 6% / 50% 1 6 1 12m
hello-qoxmothj Deployment/hello-qoxmothj 6% / 50% 1 6 1 12m
```
```
@@ -10,7 +10,7 @@ Before creating a function the environment should be created, we will assume tha
Let's create a simple code snippet in nodejs which will output Hello world:
```
``` js
module.exports = async function(context) {
return {
status: 200,
@@ -19,36 +19,65 @@ module.exports = async function(context) {
}
```
Let's create a function based on pool based executor.
``` bash
$ fission fn create --name hello --code hello.js --env node --executortype poolmgr
```
### Access function
Before accessing function, we need to set the `FISSION_ROUTER` environment variable first. It is needed for the examples below to work
#### Minikube
If you're using minikube, use these commands:
``` bash
$ export FISSION_ROUTER=$(minikube ip):$(kubectl -n fission get svc router -o jsonpath='{...nodePort}')
```
#### Cloud setups
Save the external IP addresses of router services in FISSION_ROUTER.
Wait for services to get IP addresses (check this with `kubectl --namespace fission get svc`). Then:
``` bash
# AWS
$ export FISSION_ROUTER=$(kubectl --namespace fission get svc router -o=jsonpath='{..hostname}')
# GCP
$ export FISSION_ROUTER=$(kubectl --namespace fission get svc router -o=jsonpath='{..ip}')
```
#### Create a HTTP trigger
Let's create a route for the function which can be used for making HTTP requests:
```
``` bash
$ fission route create --function hello --url /hello
trigger '5327e9a7-6d87-4533-a4fb-c67f55b1e492' created
```
Let's create a function based on pool based executor.
```
fission fn create --name hello --code hello.js --env node --executortype poolmgr
```
When you hit this function's URL , you get a response:
```
``` bash
$ curl http://$FISSION_ROUTER/hello
Hello, world!
```
Similarly you can create a new deployment executor type function and provide minmum and maximum scale for the function.
```
fission fn create --name hello --code hello.js --env node --minscale 1 --maxscale 5 --executortype newdeploy
``` bash
$ fission fn create --name hello --code hello.js --env node --minscale 1 --maxscale 5 --executortype newdeploy
```
### View & update function source code
You can look at the source code associated with given function:
```
``` bash
$ fission fn get --name hello
module.exports = async function(context) {
return {
@@ -60,7 +89,7 @@ module.exports = async function(context) {
Let's say you want to update the function to output "Hello Fission" instead of "Hello world", you can update the source file and update the source code for function:
```
``` bash
$ fission fn update --name hello --code ../hello.js
package 'hello-js-ku9s' updated
function 'hello' updated
@@ -68,7 +97,7 @@ function 'hello' updated
Let's verify that the function now respond with a different output than earlier:
```
``` bash
$ curl http://$FISSION_ROUTER/hello
Hello, Fission!
```
@@ -76,13 +105,13 @@ Hello, Fission!
### Test and debug function
You can directly test a function using test command. If the function call succeeds, it will output the function's response.
```
``` bash
$ fission fn test --name hello
Hello, Fission!
```
But if there is an error in function execution then the logs of function execution are displayed:
```
``` bash
$ fission fn test --name hello
Error calling function hello: 500 Internal server error (fission)
@@ -97,7 +126,7 @@ user code load error: SyntaxError: Unexpected token function
```
You can also look at function execution logs explicitly:
```
``` bash
$ fission fn logs --name hello
[2018-02-16 08:41:43 +0000 UTC] 2018/02/16 08:41:43 fetcher received fetch request and started downloading: {1 {hello-js-rqew default 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] nil [] } user [] []}
[2018-02-16 08:41:43 +0000 UTC] 2018/02/16 08:41:43 Successfully placed at /userfunc/user
@@ -119,15 +148,17 @@ You can attach the source/deployment packages to a function or explicitly create
Let's take a simple python function which has dependency on a python pyyaml module. We can specify the dependencies in requirements.txt and a simple command to build from source. The tree structure of directory looks like:
```
``` bash
sourcepkg/
├── __init__.py
├── build.sh
├── requirements.txt
└── user.py
```
And the file contents:
```
``` bash
$ cat user.py
import sys
import yaml
@@ -152,13 +183,14 @@ pip3 install -r ${SRC_PKG}/requirements.txt -t ${SRC_PKG} && cp -r ${SRC_PKG} ${
You first need to create an environment with environment image and python-builder image specified:
``` bash
$ fission env create --name python --image fission/python-env:latest --builder fission/python-builder:latest --mincpu 40 --maxcpu 80 --minmemory 64 --maxmemory 128 --poolsize 2
```
$fission env create --name python --image fission/python-env:latest --builder fission/python-builder:latest --mincpu 40 --maxcpu 80 --minmemory 64 --maxmemory 128 --poolsize 2
```
Now let's zip the directory containing the source files and create a function with source package:
```
$zip -jr demo-src-pkg.zip sourcepkg/
``` bash
$ zip -jr demo-src-pkg.zip sourcepkg/
adding: __init__.py (stored 0%)
adding: build.sh (deflated 24%)
adding: requirements.txt (stored 0%)
@@ -169,10 +201,11 @@ function 'hellopy' created
$ fission route create --function hellopy --url /hellopy
```
Once we create the function, the build process is started. You can check logs of the builder in fission-builder namespace:
```
$ k -n fission-builder logs -f py3-4214348-59555d9bd8-ks7m4 builder
``` bash
$ kubectl -n fission-builder logs -f py3-4214348-59555d9bd8-ks7m4 builder
2018/02/16 11:44:21 Builder received request: {demo-src-pkg-zip-ninf-djtswo ./build.sh}
2018/02/16 11:44:21 Starting build...
@@ -189,8 +222,8 @@ Successfully installed pyyaml-3.12
```
Once the build has succeeded, you can hit the function URL to test the function:
```
$curl http://$FISSION_ROUTER/hellopy
``` bash
$ curl http://$FISSION_ROUTER/hellopy
a: 1
b: {c: 3, d: 4}
```
@@ -201,7 +234,7 @@ In some cases you have a pre-built deployment package which you need to deploy t
We will use a simple python file in a directory and turn it into a deployment package:
```
``` bash
$ cat testDir/hello.py
def main():
return "Hello, world!"
@@ -209,9 +242,10 @@ def main():
$zip -jr demo-deploy-pkg.zip testDir/
```
Let's use the deployment package to create a function and route and then test it.
```
``` bash
$ fission fn create --name hellopy --env python --deploy demo-deploy-pkg.zip --entrypoint "hello.main"
function 'hellopy' created
@@ -225,7 +259,7 @@ Hello, world!
You can retrieve metadata information of a single function or list all functions to look at basic information of functions:
```
``` bash
$ fission fn getmeta --name hello
NAME UID ENV
hello 34234b50-12f5-11e8-85c9-42010aa00010 node
@@ -234,5 +268,4 @@ $ fission fn list
NAME UID ENV EXECUTORTYPE MINSCALE MAXSCALE TARGETCPU
hello 34234b50-12f5-11e8-85c9-42010aa00010 node poolmgr 0 1 80
hello2 e37a46e3-12f4-11e8-85c9-42010aa00010 node newdeploy 1 5 80
```
```
@@ -123,7 +123,7 @@ Build Logs:
Finally you can create a function with the package and test the function:
```
$fission fn create --name deploypy --pkg demo-deploy-pkg-zip-whzl --entrypoint "hello.main"
$ fission fn create --name deploypy --pkg demo-deploy-pkg-zip-whzl --entrypoint "hello.main"
$curl http://$FISSION_ROUTER/deploypy
Hello, world!
@@ -1,5 +1,5 @@
---
title: "Trigger"
title: "Triggers"
draft: false
weight: 44
---
@@ -38,11 +38,16 @@ halfhourly 0 30 * * * hello
minute @every 1m hello
```
### Create a MQ Trigger
### Create a Message Queue Trigger
For creating a MQ based trigger which will invoke the function when a new message arrives in newfile topic, you can use the syntax below. The response of the function execution will be sent to topic newfileresponse.
A message queue trigger invokes a function based on messages from an
message queue. Currently, NATS and Azure Storage Queue are supported
queues. (Kafka support is under development.)
```
$ fission mqt create --name hellomsg --function hello --mqtype nats-streaming --topic newfile --resptopic newfileresponse
trigger 'hellomsg' created
```
```
You can list or update message queue triggers with `fission mqt list`,
or `fission mqt update`.
@@ -39,13 +39,7 @@
{{ end }}
{{ end }}
{{with ($.Scratch.Get "prevPage")}}
<a class="nav nav-prev" href="{{.URL}}" title="{{.Title}}"> <i class="fa fa-chevron-left"></i></a>
{{end}}
{{with ($.Scratch.Get "nextPage")}}
<a class="nav nav-next" href="{{.URL}}" title="{{.Title}}" style="margin-right: 0px;"><i class="fa fa-chevron-right"></i></a>
{{end}}
</div>
</section>
@@ -98,6 +98,7 @@ textarea:focus, input[type="email"]:focus, input[type="number"]:focus, input[typ
box-shadow: inset 0 1px 3px rgba(0,0,0,.06),0 0 5px rgba(0,169,218,.7)
}
#header-wrapper {
/*background: #4a4a4b;*/
background: #1E022D;
color: #fff;
text-align: center;
@@ -445,6 +446,7 @@ textarea:focus, input[type="email"]:focus, input[type="number"]:focus, input[typ
}
body {
font-family: "Work Sans", "Helvetica", "Tahoma", "Geneva", "Arial", sans-serif;
/*font-family: -apple-system,BlinkMacSystemFont,"avenir next",avenir,"helvetica neue",helvetica,ubuntu,roboto,noto,"segoe ui",arial,sans-serif;*/
font-weight: 300;
line-height: 1.6;
font-size: 18px !important;
@@ -457,11 +459,13 @@ h2, h3, h4, h5, h6 {
letter-spacing: -1px;
}
h1 {
font-family: "Novacento Sans Wide", "Helvetica", "Tahoma", "Geneva", "Arial", sans-serif;
text-align: center;
text-transform: uppercase;
color: #222;
font-weight: 200;
font-family: "Work Sans", "Helvetica", "Tahoma", "Geneva", "Arial", sans-serif;
/*font-family: "Novacento Sans Wide", "Helvetica", "Tahoma", "Geneva", "Arial", sans-serif;*/
/*text-align: center;*/
/*text-transform: uppercase;*/
color: #5e5e5e;
font-weight: 400;
letter-spacing: -1px;
}
blockquote {
border-left: 10px solid #F0F2F4;
@@ -1116,8 +1120,10 @@ pre .copy-to-clipboard:hover {
}
#sidebar #shortcuts h3 {
font-family: "Novacento Sans Wide", "Helvetica", "Tahoma", "Geneva", "Arial", sans-serif;
color: white ;
/*font-family: "Novacento Sans Wide", "Helvetica", "Tahoma", "Geneva", "Arial", sans-serif;*/
font-family: "Work Sans", "Helvetica", "Tahoma", "Geneva", "Arial", sans-serif;
color: #eee;
font-weight: 200;
margin-top:1rem;
padding-left: 1rem;
}
@@ -458,8 +458,8 @@ h2, h3, h4, h5, h6 {
}
h1 {
font-family: "Novacento Sans Wide", "Helvetica", "Tahoma", "Geneva", "Arial", sans-serif;
text-align: center;
text-transform: uppercase;
/*text-align: center;*/
/* text-transform: uppercase; */
color: #222;
font-weight: 200;
}
+66
View File
@@ -0,0 +1,66 @@
# Multi-tenancy in Fission
Multi-tenancy in Fission allows users to create Fission objects, i.e functions, packages, environments and triggers in different namespaces.
It mandates that a function reference secrets, configmaps and its package (if explicity referenced during function creation/updation) to be present in the same namespace as the function.
This allows user separation and prevents in-advertent access to sensitive data of other users sharing the same cluster.
However, users are allowed and encouraged to share environments to ensure optimal utilization of cluster resources. To achieve this, users can create all the necessary environments in a ns, say ns1 and then go on to create functions in different namespaces and refer to env in ns1.
Users that prefer complete isolation can create their env, functions in the same ns.
## Roles and privileges
1. Cluster-Admin Role : Fission's services need cluster-admin privileges to monitor, create, update and delete resources across namespaces.
2. Package-getter Role : This role has privileges to do a get, watch and list on fission package objects.
3. Secret-Configmap-getter Role : This role has privileges to do a get, watch and list on secrets and configmaps.
## Service Accounts
1. fission-fetcher
This SA is created in every namespace that a user creates runtime environments in.
Also created in function namespaces where a user creates functions that use NewDeploy executor backend.
2. fission-builder
This SA is created in every namespace that a user creates builder environments in.
## Role-bindings
1. Package-getter-binding
Every time a user creates a package explicity in a namespace, this rolebinding is created in package's namespace (which is also function's namespace). This grants package-getter role to fission-fetcher SA present in the referenced environment's namespace.
If the package is a source package, then, fission-builder SA present in the environment namespace is also added to this rolebinding.
Next when the user creates a function in the same namespace, if the function's executor type is newdeploy, then, the fission-fetcher SA present in function namespace is also added to the same rolebinding.
Note : For functions that have executor type poolmgr, the env pods are created in the namespace that env object is created. Whereas, for those functions that have executor type New deploy mgr, the function pods are created in the namespace that function object is created in.
This is because, poolmgr allows env sharing and optimal resource utilization. so generic env pools are created in a different namespace and all functions that prefer sharing this env pool can reference these pools.
If users require strict isolation, they can either create functions with new deploy backend, or, create env's in different namespaces and not share them across functions.
2. Secret-Configmap-getter-binding
Every time a user creates a function in a namespace, Secret-Configmap-getter-binding is created in the same namespace, granting secret-configmap-getter role to fission-fetcher SA present in the referenced environment's namespace in case the executor type is poolmgr.
If the executor type is newdeploymgr, then the same rolebinding is created in the same namespace as the function, granting the same secret-configmap-getter role to fission-fetcher SA present in the function namespace.
## Examples
1. create a generic python runtime env in ns 1 and function with poolmgr executor type in ns 2 that references it.
```bash
$ fission env create --name python --image fission/python-env --envns ns1
$ fission function create --name func1 --env python --code hello.py --fns ns2
```
2. create a builder and runtime environment in ns3, a source pkg in ns3 and a function referring to this src pkg also in ns3. (for complete isolation, all objs are in ns3)
```bash
$ fission env create --name python-builder-env --builder fission/python-builder --image fission/python-env --ns3
$ fission package create --src src-pkg.zip --env python-builder-env --buildcmd "./build.sh" --pkgns ns3
$ fission fn create --name func3 --fns ns3 --pkg $pkg --entrypoint "user.main"
```
## Note
1. To maintain backward compatibility, fission objects that are created without the ns flags are created in default namespace. Also, the run time env pods in such a case will continue to live in fission-function ns and builder env pods in fission-builder ns
2. Since all envs in a namespace have the same fission-fetcher SA mounted in them, even though multiple envs are created in a namespace and referenced by functions in different namespaces, the SA will have previleges to view those function's secrets if any.
3. Similarly, if there are multiple functions in different namespaces but all sharing an env in one namespace, the fission-fetcher SA in that namespace will have previleges to see all of their secrets.
+128
View File
@@ -0,0 +1,128 @@
# Fission CLI Extensibility
### Approach
To sum up the approach: git-style plugins.
Plugins are named with the fission prefix `fission-*`. When fission is invoked with an undefined/non-core subcommand
is called (like `fission foo`). Fission will look in the PATH
**Installing Fission**
```bash
# The same as before
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.7.2/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
```
**Installing Fission Workflows**
```bash
# The same process as Fission cli itself
$ curl -Lo fission https://github.com/fission/fission-workflows/releases/download/0.4.0/fission-workflows-osx && chmod +x fission-workflows && sudo mv fission-workflows /usr/local/bin/
```
**Invoking Fission Workflows**
```bash
$ fission workflows invocation get b1278e802a
# Which is equivalent to:
$ fission-workflows invocation get b1278e802a
```
General flow:
1. fission does not recognize `workflows` subcommand
2. fission checks the path for a binary called `fission-workflows`
3. fission finds the binary.
4. fission invokes the `fission-workflows`, passing the remainder of the arguments.
**Discoverability: fission --help**
```bash
$ fission --help
USAGE:
fission [global options] command [command options] [arguments...]
VERSION:
0.6.0
COMMANDS:
function, fn Create, update and manage functions
httptrigger, ht, route Manage HTTP triggers (routes) for functions
timetrigger, tt, timer Manage Time triggers (timers) for functions
mqtrigger, mqt, messagequeue Manage message queue triggers for functions
environment, env Manage environments
watch, w Manage watches
package, pkg Manage packages
spec, specs Manage a declarative app specification
upgrade Upgrade tool from fission v0.1
tpr2crd Migrate tool for TPR to CRD
help, h Shows a list of commands or help for one command
PLUGINS:
workflows, wf Inspect and manage workflow executions
ui Start the user interface
GLOBAL OPTIONS:
--server value Fission server URL (default: "http://127.0.0.1:65356")
--help, -h show help
--version, -v print the version
```
Of course Fission needs to be able to find all plugins for this. There are several ways in which we can provide discoverability. The simplest one is for Fission to look in the path for all binaries starting with the `fission-*` prefix. Optionally, fission could invoke a specific command on the subcommand to get info about the plugin (such as version, help text, aliases...)
With Fission Workflows this info would look something like this:
```bash
$ fission-workflows --plugin
name: workflows
version: 0.4.0
help: Inspect and manage workflow executions
```
The idea is that this plugin info is all completely optional.
If it is not available, we simply degrade the results to user.
This way users/we can easily prototype or add plugins without having to worry about adhering to some interface.
**List version**
```bash
$ fission --version
client:
fission: 0.8.0
fission-workflows: 0.4.0
server:
fission: 0.8.1
fission-workflows: 0.3.0
```
Again, versioning info for fission-workflows is taken from the plugin info of the commands.
Note: a related issue is to have some more formalized plugin support/discoverability on the server-side,
but that is out of the scope of this issue.
### Other (optional) extensions and notes
- Like git we could setup a preferred binary path, where fission looks first when searching for the subcommand.
This could optionally be defined with a `FISSION_EXEC_PATH`.
- With the current approach we cannot have aliases for commands---fission will not be able to find fission-workflows
when the user calls `fission wf`. This might be UX issue, with these long path names. One option is let the user fix
it themselves by symlinking `fission-wf` to `fission-workflows`; using the plugin info Fission can recognize and
merge aliases together.
- To help detect versioning conflicts (old version of fission, too new version of fission workflows). We could add
a `requires` field to the fission-workflows plugin info. Then we could throw a warning or error, when two out of sync
versions are being used.
- To avoid unhelpful errors to the user when they have not installed a plugin, we could add a heuristic to check
`https://github.com/fission/SUBCOMMAND` to see if the subcommand might be an uninstalled plugin.
OR, we could lookup a simple text file that contains common plugins `https://github.com/fission/fission/plugins.txt`
and list them as suggestions to the user. OR we could of course just default to a bit help text that says something
like `unknown subcommand 'foo'. If this is a plugin, ensure that it is present on your PATH`.
---
### Motivation
The proposed approach is to use the git-based plugin system for now. Reasons for this approach over a sophisticated,
integrated plugin-based approach:
- It is low effort to implement.
- It is easy to extend with minimal to no required interface.
- The binaries remain standalone, allowing users to separate them if needed and make independent development on the
binaries easy.
Limitations of the proposed approach:
- The user still has to do some work, adding binaries to the PATH; ensuring that permissions are correct; ensuring
that the binary is executable; how to deal with duplicate binaries on the PATH. All this makes this approach assume
basic/intermediate knowledge of the OS from the user.
- I have to admit: I am not entirely sure if this approach requires any changes for Windows. Probably not.
- Upgrading fission with many plugins could be cumbersome, as you would need to upgrade each binary one by one.
Improving this is probably best left to future work.
The more heavyweight solution solves some of these limitations to an extent, but these do not way up to the increased
development and maintainance cost IMO. If needed we could explore this option (or some hybrid option) in the future.
+67
View File
@@ -0,0 +1,67 @@
# Java Environment : Design & considerations
This document documents the design and thoughts that lead to design of Java environment. Before we dive deeper, some important points:
- When we say Java, we really mean JVM. That does not mean that all languages will work seamlessly, so support will be added gradually based on validation. Some of popular languages as of today are:
- Scala
- Groovy
- Kotlin (Server side with Spring)
- In Java there are a few prominent frameworks which have a ecosystem of their own (See list below). How these framework fit in the environment will be detailed later, but it is important to understand their place in ecosystem and design for it.
- A large percentage of entertprise developers use [Spring framework](https://spring.io/) as has been shown by multiple surveys
- Reactive has taken up recently with data intensive operations [Reactive extensions for JVM](https://github.com/ReactiveX/RxJava)
- [Spark](http://sparkjava.com/) is a micro web framework
A draft implementation of the Java environment design is in the branch [java_env_alpha](https://github.com/fission/fission/tree/java_env_alpha). Also a earlier implementation based on Vort.x framework [can be found here](https://github.com/tobias/fission-java-env/)
## Function interface
The goal is here is to minimize the lock in for the user into any framework as much as possible. Java 8 introduced an interface called ```Function``` which could be a great fit here. The user has to implement the ```Function<T, R>``` class and to meet the contract implement the apply method:
```
public class HelloWorld implements Function<T, R> {
public R apply(T str) {
```
Now - the T & R could be different things and we discuss some options below:
### Body in request and response
From the early implementation in the branch mentioned above, the environment extracts the body and send it as a JSON string. The JSON then can be transformed into the appropriate object by the function.
This works well, but has one major limitation: the function does not get access to other things like headers etc. The same thing applies to the reponse: function can send the body but looses control over status code etc.
### HttpServletRequest and HttpServletResponse
It is possible to send the [HttpServeletRequest](https://docs.oracle.com/javaee/6/api/javax/servlet/http/HttpServletRequest.html) request object as it is to the function class but then the interface becomes a bit too low level. For example the function user has to retrieve the body of request using ```getInputStream``` which gives raw input stream and needs additional work.
Also most enterprise applications today use a framework of some sort for web applications instead of dealing with the raw HttpServlet
### Custom/Context Object
A custom object which encapsulates all needed fields etc. can be used to pass the data from environment to the function. But this means the user has to import a Fission object/library for this object in the application code.
This approach has been taken in the implementation done earlier for a Java environment in Fission and [object interface can be found here](https://github.com/tobias/fission-java-env/blob/master/src/main/java/io/fission/api/Context.java). Related discussion is in the [issue](https://github.com/fission/fission/issues/91)
AWS Lambda also uses a context object, but the purpose is very different, [details of context object here](https://docs.aws.amazon.com/lambda/latest/dg/java-context-object.html).
### Spring's HttpEntity
If we have to depend on a class/library, it is probably better to depend on a class which is part of ecosystem. So instead of using the low level interface of Servlet, we can use [HttpEntity's subclasses RequestEntity and ResponseEntity](https://docs.spring.io/spring/docs/5.0.5.RELEASE/javadoc-api/org/springframework/http/HttpEntity.html). This ensures that the function user is not locked in the Fission object contract, but also gets the full access to request/response object.
The Spring cloud function project also discusses the issue of not having access to other things in request and [related issues are here](https://github.com/spring-cloud/spring-cloud-function/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+header)
### Thoughts
- If we only intend to pass request/response object to function - then using HttpEntity might be a good choice
- If there is a need for additional exchange of information between the environment and function execution in future, then a custom/context object is a better option. We can wrap the HttpEntity's fields and additional fields in the custom context object
## Environment Design
JVM environment design is based on Spring boot and Spring MVC frameworks. The details can be found in branch, but here are some key points:
- All classes in the function and dependent classes are loaded into JVM. Which means the user should supply the uber/fat jar for execution.
- The entrypoint class is specified by the user as ```entrypoint``` flag on the class. The method is by convention (```apply``` as per the Function interface contract)
+209
View File
@@ -0,0 +1,209 @@
# Testing Proposal
This proposal was initially started as a upgrade testing proposal but soon problems that were posed resulted in a bigger proposal.
### Kinds of testing
Most of current integration tests are CLI driven. Fission CLI is used to test execute various test cases. In future we would have to also focus on API level testing as a UI is built for Fission.
## Needs & patterns
This section only explains the problems/best practices without going into tooling and language used for implementation.
### Separating the test & data
Seperating the tests from test data has two aspects - one is separation of concerns and second is scaling the tests without touching the test logic. The test data is a simple data structure which holds all information and test can take data and execute the logic.
As an example today we test "Hello world" for nodejs environment with a simple hello.js like this:
```
fission env create --name nodejs --image fission/node-env
fission fn create --name $fn --env nodejs --code $ROOT/examples/nodejs/hello.js
fission route create --function $fn --url /$fn --method GET
response=$(curl http://$FISSION_ROUTER/$fn)
```
The variables here are environment image, function code & route URL.
If tomorrow if we had to scale this test for all environments, we will have to repeat ourselves. (Violate DRY principle). Instead of that if we encapsulate the test setup & test in a simple function:
```
test_hello_env(envImage, codePath, routeURL){
}
```
And feed it with a dictionary which has all possible combination of tests:
```
{
node: {"fission/node-env", "test/hello.js", "/hellonode"},
python: {"fission/python-env", "test/hello.py", "/hellopy"},
golang: {"fission/go-env", "test/hello.go", "/hellogo"},
binray: {"fission/binary-env", "test/hello.sh", "/hellobinary"},
}
```
This would achieve a few things:
- Separate the test execution logic from the data it needs clearly.
- For adding new kind of environments, you just need to add one more entry into data structure.
Testing all environments may not be most apt example for this, but there can be potential use cases like this.
### Separating the test & setup/teardown
When we run a test there are typically three distinct phases:
- Setup (Create env, fn, route)
- Test (Curl the function)
- Cleanup (Delete fn, route & env)
It should be possible to separate the before test and after test parts from actual tests at two levels:
- Each test
- A whole test suite
The ability to have clean and separate before and after blocks, apart from separation of concerns, enables:
- Running a suite of tests for same setup (See tagging for suite of tests)
### Tagging tests & running a selection
Over a period of time as tests grow, there will be unit, smkoe, integration, performance, soak tests and so on. Ability to run a perticular test suite only or a combination of them makes it easy to run for specific purpose.
### Measuring test times
[Good to have, not a must] Measuring time for tests and reporting somewhere helps over time to monitor trends. Although this job is better done by performance/benchmark tests so it is not a strict requirement
### Cleaner Logging
It would be good to have cleaner/relevant logging as part of build & test. For example something that Ginkgo framework does is it shows error logs only for failed tests.
### Tests in Parallel
It would be good to be able to run tests in parallel.
## Evalutaing the tools/alternatives
### BATS
Bash Automated Testing System is like a enhanced version of bash with support for @test tags and before and after steps & ability to skip tests etc. While it enhances the bash to certain extent, the overall improvement is only marginal.
```
#!/usr/bin/env bats
@test "addition using bc" {
result="$(echo 2+2 | bc)"
[ "$result" -eq 4 ]
}
$ bats addition.bats
✓ addition using bc
✓ addition using dc
2 tests, 0 failures
```
#### Links
- Bats repo: https://github.com/sstephenson/bats
- Runc uses Bats https://github.com/opencontainers/runc/tree/master/tests/integration
### Go Test
The testing package of Go also is quite feature rich for most of the use cases we need. Go 1.7 onwards there is support for setup & teardown parts and parallelism etc.
#### Go - Testing package
- Support for setup and teardown based on https://golang.org/pkg/testing/#hdr-Main
- Go testing already supports and has examples of table driven tests (Separating test & data), measuring test times and parallel tests
#### Shell execution: Go's Exec Library
GO provides a built in Exec library for working with CLI commands. The package seems good enough for us to work, though a few working examples will help decide better
https://golang.org/pkg/os/exec
### Using CLI package
Currently we build a CLI and then execute the tests. The tests basically call one of functions from CLI package. If we decide to use a go lang based framework, then we can import the CLI package and then call those functions by providing them context. This is as good as calling the Fission from CLI, with added benefit of programmibility of Go langugage.
```
func TestSomething(t *testing.T) {
// Build the Cli context with flags etc.
ctx := cli.Context{}
// Pass the ctx to create function
fnCreate(ctx)
}
```
Some of benefits of using above pattern are:
- We can build a small framework around above core where we can pass various flag combinations etc. and exercise all flags in great detail
- We can use rest of Go testing library and other libraries to build matchers, looping, parallelism etc.
- It allows us to exercise the logic in CLI as well as validate the API at the same time.
### Ginkgo & Gomega
Ginkgo is a BDD framework which works with Gomega matcher library. I will state relevant portions of these two frameworks which can be utilized:
From Ginkgo:
- Global `BeforeSuite` and after `AfterSuite` can be used to have global setup and tear down phases
- For tests `BeforeEach` and `AfterEach` and more such varients to do before and after test tasks.
From Gomega:
Gomega is a matcher library but the `gexec` library makes it really easy to interact with OS execution environment. Some working examples:
- Build and cleanup the Fission CLI before & after the tests
```
var fissionCli string
BeforeSuite(func() {
var err error
fissionCli, err = gexec.Build("github.com/fission/fission")
Ω(err).ShouldNot(HaveOccurred())
})
AfterSuite(func() {
gexec.CleanupBuildArtifacts()
})
```
- Following will run Fission commands with Fission CLI and print error if there is one (verbosity is configurable)
```
command := exec.Command(fissionCli, "fission env create --name nodejs --image fission/node-env")
session, err := gexec.Start(command, GinkgoWriter, GinkgoWriter)
Ω(err).ShouldNot(HaveOccurred())
```
- Use Fission CLI's output to validate test results
```
Eventually(session.Out).Should(gbytes.Say("hello [A-Za-z], world"))
```
#### Gomega Matchers
- Gomega provides quite a few built in matchers - so you don't have to code those small usual checks, for example:
```
Ω(ACTUAL).Should(BeTrue()) // The output should be true
Ω(ACTUAL).Should(BeAnExistingFile()) // The file should already exist
```
There are many more matchers which cane be found here: http://onsi.github.io/gomega/#provided-matchers
- We can build custom mathers in Go language for reusable logic.
#### Links
Ginkgo: http://onsi.github.io/ginkgo/
Gomega: http://onsi.github.io/gomega/
## Thoughts & Next actions
Based on the discussion with team, here are current thoughts and next action items:
### Thoughts
- As far as possible we should stick to Go's built in testing package
- Ginkgo's cleaner logging feature (Onlu log if there are errors) - is very useful. We can decide to incorporate this in future.
- Gomega's (gexec)[http://onsi.github.io/gomega/#gexec-testing-external-processes] is really neat and some matchers can be used if necessary
### Action items
- How will upgrade test for Fission fit in the framework?
- How will migration of tests happen over time:
- Aim is to keep existing tests around so that enough validation is in place
- May be migrate one test at a time
- How much of current setup etc. will move into framework? For example it is clear that helm commands should be part of test framework as part of setup/teardown. But other sections may or may not be. A RCA needs to be done to analyze and come up with clear demarkation.
## References
- EngineYard uses BATS: https://www.engineyard.com/blog/bats-test-command-line-tools
- AWS CLI Tests, written in Python (CLI itself is also in Python): https://github.com/aws/aws-cli/tree/develop/tests
- Kubernetes uses Ginkgo and Gomega extensively: https://github.com/kubernetes/kubernetes/search?l=Go&q=onsi&type=
- Hashicorp's Mitchell's talk on Advanced testing with go talks about some good patterns to use: https://www.youtube.com/watch?v=8hQG7QlcLBk
+4 -1
View File
@@ -1,2 +1,5 @@
These instructions are replaced by:
http://fission.io/docs/0.4.0/installation/
http://fission.io/docs/0.6.0/installation/
You can change the release number `0.6.0` from [release page](https://github.com/fission/fission/releases).
+3 -3
View File
@@ -15,7 +15,7 @@
.DEFAULT_GOAL := build
IMAGE ?= fission/fission-bundle
VERSION ?= 0.3.0
VERSION ?= latest
ARCH ?= amd64
OS ?= linux
@@ -42,5 +42,5 @@ image-push: image
docker push "$(IMAGE):$(VERSION)"
clean:
@rm -rf fission-bundle/fission-bundle
@rm -rf fission/fission
@rm -f fission-bundle/fission-bundle
@rm -f fission/fission
+3 -3
View File
@@ -88,7 +88,7 @@ Usage
$ fission route create --method GET --url /hello --function hello
# Run the function. This takes about 100msec the first time.
$ curl http://$FISSION_ROUTER/hello
$ fission function test --name hello
Hello, world!
```
@@ -97,12 +97,12 @@ See the [examples](examples) directory for more.
Running Fission on your Cluster
===============================
See the [installation guide](http://fission.io/docs/latest/installation/).
See the [installation guide](http://docs.fission.io/latest/installation).
Compiling Fission
=================
See the [compilation guide](http://fission.io/docs/latest/compilation/).
See the [compilation guide](http://docs.fission.io/latest/contributing/compiling/).
Status
======
+7
View File
@@ -33,6 +33,8 @@ import (
"time"
"github.com/dchest/uniuri"
"github.com/fission/fission"
)
const (
@@ -68,6 +70,11 @@ func MakeBuilder(sharedVolumePath string) *Builder {
}
}
func (builder *Builder) VersionHandler(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
fmt.Fprintf(w, fission.BuildInfo().String())
}
func (builder *Builder) Handler(w http.ResponseWriter, r *http.Request) {
if r.Method != "POST" {
e := fmt.Sprintf("Method not allowed: %v", r.Method)
+16 -1
View File
@@ -1,3 +1,18 @@
#!/bin/bash
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -gcflags=-trimpath=$GOPATH -asmflags=-trimpath=$GOPATH -o builder .
version=$1
if [ -z $version ]; then
version=$(git rev-parse HEAD)
fi
date=$2
if [ -z $date ]; then
date=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
fi
gitcommit=$3
if [ -z $gitcommit ]; then
gitcommit=$(git rev-parse HEAD)
fi
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -gcflags=-trimpath=$GOPATH -asmflags=-trimpath=$GOPATH -ldflags "-X github.com/fission/fission.GitCommit=$gitcommit -X github.com/fission/fission.BuildDate=$date -X github.com/fission/fission.Version=$version" -o builder .
+1
View File
@@ -38,6 +38,7 @@ func main() {
builder := builder.MakeBuilder(dir)
mux := http.NewServeMux()
mux.HandleFunc("/", builder.Handler)
mux.HandleFunc("/version", builder.VersionHandler)
mux.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
})
+7 -2
View File
@@ -31,12 +31,17 @@ func Start(storageSvcUrl string, envBuilderNamespace string) error {
return err
}
err = fissionClient.WaitForCRDs()
if err != nil {
log.Fatalf("Error waiting for CRDs: %v", err)
}
envWatcher := makeEnvironmentWatcher(fissionClient, kubernetesClient, envBuilderNamespace)
go envWatcher.watchEnvironments()
pkgWatcher := makePackageWatcher(fissionClient,
kubernetesClient.CoreV1().RESTClient(), envBuilderNamespace, storageSvcUrl)
go pkgWatcher.watchPackages()
kubernetesClient, envBuilderNamespace, storageSvcUrl)
go pkgWatcher.watchPackages(fissionClient, kubernetesClient, envBuilderNamespace)
select {}
}
+13 -11
View File
@@ -22,8 +22,6 @@ import (
"net/http"
"strings"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/dchest/uniuri"
"github.com/fission/fission"
"github.com/fission/fission/builder"
@@ -40,25 +38,26 @@ import (
// 3. Send upload request to fetcher to upload deployment package.
// 4. Return upload response and build logs.
// *. Return build logs and error if any one of steps above failed.
func buildPackage(fissionClient *crd.FissionClient, builderNamespace string,
func buildPackage(fissionClient *crd.FissionClient, envBuilderNamespace string,
storageSvcUrl string, pkg *crd.Package) (uploadResp *fetcher.UploadResponse, buildLogs string, err error) {
env, err := fissionClient.Environments(metav1.NamespaceDefault).Get(pkg.Spec.Environment.Name)
env, err := fissionClient.Environments(pkg.Spec.Environment.Namespace).Get(pkg.Spec.Environment.Name)
if err != nil {
e := fmt.Sprintf("Error getting environment CRD info: %v", err)
log.Println(e)
return nil, e, fission.MakeError(http.StatusInternalServerError, e)
}
svcName := fmt.Sprintf("%v-%v.%v", env.Metadata.Name, env.Metadata.ResourceVersion, builderNamespace)
svcName := fmt.Sprintf("%v-%v.%v", env.Metadata.Name, env.Metadata.ResourceVersion, envBuilderNamespace)
srcPkgFilename := fmt.Sprintf("%v-%v", pkg.Metadata.Name, strings.ToLower(uniuri.NewLen(6)))
fetcherC := fetcherClient.MakeClient(fmt.Sprintf("http://%v:8000", svcName))
builderC := builderClient.MakeClient(fmt.Sprintf("http://%v:8001", svcName))
fetchReq := &fetcher.FetchRequest{
FetchType: fetcher.FETCH_SOURCE,
Package: pkg.Metadata,
Filename: srcPkgFilename,
FetchType: fetcher.FETCH_SOURCE,
Package: pkg.Metadata,
Filename: srcPkgFilename,
KeepArchive: false,
}
// send fetch request to fetcher
@@ -95,9 +94,12 @@ func buildPackage(fissionClient *crd.FissionClient, builderNamespace string,
log.Printf("Build succeed, source package: %v, deployment package: %v", srcPkgFilename, buildResp.ArtifactFilename)
archivePackage := !env.Spec.KeepArchive
uploadReq := &fetcher.UploadRequest{
Filename: buildResp.ArtifactFilename,
StorageSvcUrl: storageSvcUrl,
Filename: buildResp.ArtifactFilename,
StorageSvcUrl: storageSvcUrl,
ArchivePackage: archivePackage,
}
log.Printf("Start uploading deployment package: %v", buildResp.ArtifactFilename)
@@ -131,7 +133,7 @@ func updatePackage(fissionClient *crd.FissionClient,
}
// update package spec
pkg, err := fissionClient.Packages(metav1.NamespaceDefault).Update(pkg)
pkg, err := fissionClient.Packages(pkg.Metadata.Namespace).Update(pkg)
if err != nil {
log.Printf("Error updating package: %v", err)
return nil, err
+114 -66
View File
@@ -23,13 +23,13 @@ import (
"strconv"
"time"
apiv1 "k8s.io/api/core/v1"
"k8s.io/api/extensions/v1beta1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/labels"
"k8s.io/apimachinery/pkg/util/intstr"
"k8s.io/apimachinery/pkg/watch"
"k8s.io/client-go/kubernetes"
apiv1 "k8s.io/client-go/pkg/api/v1"
"k8s.io/client-go/pkg/apis/extensions/v1beta1"
"github.com/fission/fission"
"github.com/fission/fission/crd"
@@ -42,7 +42,15 @@ const (
CLEANUP_BUILDERS
LABEL_ENV_NAME = "envName"
LABEL_ENV_NAMESPACE = "envNamespace"
LABEL_ENV_RESOURCEVERSION = "envResourceVersion"
LABEL_DEPLOYMENT_OWNER = "owner"
BUILDER_MGR = "buildermgr"
)
var (
deletePropagation = metav1.DeletePropagationBackground
delOpt = metav1.DeleteOptions{PropagationPolicy: &deletePropagation}
)
type (
@@ -125,14 +133,22 @@ func makeEnvironmentWatcher(fissionClient *crd.FissionClient,
return envWatcher
}
func (envw *environmentWatcher) getCacheKey(envName string, envResourceVersion string) string {
return fmt.Sprintf("%v-%v", envName, envResourceVersion)
func (envw *environmentWatcher) getCacheKey(envName string, envNamespace string, envResourceVersion string) string {
return fmt.Sprintf("%v-%v-%v", envName, envNamespace, envResourceVersion)
}
func (envw *environmentWatcher) getLabels(envName string, envResourceVersion string) map[string]string {
func (env *environmentWatcher) getLabelForDeploymentOwner() map[string]string {
return map[string]string{
LABEL_DEPLOYMENT_OWNER: BUILDER_MGR,
}
}
func (envw *environmentWatcher) getLabels(envName string, envNamespace string, envResourceVersion string) map[string]string {
return map[string]string{
LABEL_ENV_NAME: envName,
LABEL_ENV_NAMESPACE: envNamespace,
LABEL_ENV_RESOURCEVERSION: envResourceVersion,
LABEL_DEPLOYMENT_OWNER: BUILDER_MGR,
}
}
@@ -208,10 +224,17 @@ func (envw *environmentWatcher) service() {
req := <-envw.requestChan
switch req.requestType {
case GET_BUILDER:
key := envw.getCacheKey(req.env.Metadata.Name, req.env.Metadata.ResourceVersion)
// In order to support backward compatibility, for all environments with builder image created in default env,
// the pods will be created in fission-builder namespace
ns := envw.builderNamespace
if req.env.Metadata.Namespace != metav1.NamespaceDefault {
ns = req.env.Metadata.Namespace
}
key := envw.getCacheKey(req.env.Metadata.Name, ns, req.env.Metadata.ResourceVersion)
builderInfo, ok := envw.cache[key]
if !ok {
builderInfo, err := envw.createBuilder(req.env)
builderInfo, err := envw.createBuilder(req.env, ns)
if err != nil {
req.respChan <- envwResponse{err: err}
continue
@@ -224,7 +247,13 @@ func (envw *environmentWatcher) service() {
latestEnvList := make(map[string]*crd.Environment)
for i := range req.envList {
env := req.envList[i]
key := envw.getCacheKey(env.Metadata.Name, env.Metadata.ResourceVersion)
// In order to support backward compatibility, for all builder images created in default
// env, the pods are created in fission-builder namespace
ns := envw.builderNamespace
if env.Metadata.Namespace != metav1.NamespaceDefault {
ns = env.Metadata.Namespace
}
key := envw.getCacheKey(env.Metadata.Name, ns, env.Metadata.ResourceVersion)
latestEnvList[key] = &env
}
@@ -234,38 +263,40 @@ func (envw *environmentWatcher) service() {
// cache and CRD. We need to iterate over the services &
// deployments to remove both normal and orphan builders.
svcList, err := envw.getBuilderServiceList(nil)
svcList, err := envw.getBuilderServiceList(envw.getLabelForDeploymentOwner(), metav1.NamespaceAll)
if err != nil {
log.Println(err.Error())
}
for _, svc := range svcList {
envName := svc.ObjectMeta.Labels[LABEL_ENV_NAME]
envNamespace := svc.ObjectMeta.Labels[LABEL_ENV_NAMESPACE]
envResourceVersion := svc.ObjectMeta.Labels[LABEL_ENV_RESOURCEVERSION]
key := envw.getCacheKey(envName, envResourceVersion)
key := envw.getCacheKey(envName, envNamespace, envResourceVersion)
if _, ok := latestEnvList[key]; !ok {
err := envw.deleteBuilderService(svc.ObjectMeta.Labels)
err := envw.deleteBuilderServiceByName(svc.ObjectMeta.Name, svc.ObjectMeta.Namespace)
if err != nil {
log.Printf("Error removing builder service: %v", err)
}
}
delete(envw.cache, svc.ObjectMeta.Name)
delete(envw.cache, key)
}
deployList, err := envw.getBuilderDeploymentList(nil)
deployList, err := envw.getBuilderDeploymentList(envw.getLabelForDeploymentOwner(), metav1.NamespaceAll)
if err != nil {
log.Printf(err.Error())
}
for _, deploy := range deployList {
envName := deploy.ObjectMeta.Labels[LABEL_ENV_NAME]
envNamespace := deploy.ObjectMeta.Labels[LABEL_ENV_NAMESPACE]
envResourceVersion := deploy.ObjectMeta.Labels[LABEL_ENV_RESOURCEVERSION]
key := envw.getCacheKey(envName, envResourceVersion)
key := envw.getCacheKey(envName, envNamespace, envResourceVersion)
if _, ok := latestEnvList[key]; !ok {
err := envw.deleteBuilderDeployment(deploy.ObjectMeta.Labels)
err := envw.deleteBuilderDeploymentByName(deploy.ObjectMeta.Name, deploy.ObjectMeta.Namespace)
if err != nil {
log.Printf("Error removing builder deployment: %v", err)
}
}
delete(envw.cache, deploy.ObjectMeta.Name)
delete(envw.cache, key)
}
}
}
@@ -289,19 +320,19 @@ func (envw *environmentWatcher) cleanupEnvBuilders(envs []crd.Environment) {
}
}
func (envw *environmentWatcher) createBuilder(env *crd.Environment) (*builderInfo, error) {
func (envw *environmentWatcher) createBuilder(env *crd.Environment, ns string) (*builderInfo, error) {
var svc *apiv1.Service
var deploy *v1beta1.Deployment
sel := envw.getLabels(env.Metadata.Name, env.Metadata.ResourceVersion)
sel := envw.getLabels(env.Metadata.Name, ns, env.Metadata.ResourceVersion)
svcList, err := envw.getBuilderServiceList(sel)
svcList, err := envw.getBuilderServiceList(sel, ns)
if err != nil {
return nil, err
}
// there should be only one service in svcList
if len(svcList) == 0 {
svc, err = envw.createBuilderService(env)
svc, err = envw.createBuilderService(env, ns)
if err != nil {
return nil, fmt.Errorf("Error creating builder service: %v", err)
}
@@ -311,13 +342,20 @@ func (envw *environmentWatcher) createBuilder(env *crd.Environment) (*builderInf
return nil, fmt.Errorf("Found more than one builder service for environment %v", env.Metadata.Name)
}
deployList, err := envw.getBuilderDeploymentList(sel)
deployList, err := envw.getBuilderDeploymentList(sel, ns)
if err != nil {
return nil, err
}
// there should be only one deploy in deployList
if len(deployList) == 0 {
deploy, err = envw.createBuilderDeployment(env)
// create builder SA in this ns, if not already created
_, err := fission.SetupSA(envw.kubernetesClient, fission.FissionBuilderSA, ns)
if err != nil {
log.Printf("Error : %v creating %s in ns : %s", err, fission.FissionBuilderSA, ns)
return nil, err
}
deploy, err = envw.createBuilderDeployment(env, ns)
if err != nil {
return nil, fmt.Errorf("Error creating builder deployment: %v", err)
}
@@ -334,24 +372,36 @@ func (envw *environmentWatcher) createBuilder(env *crd.Environment) (*builderInf
}, nil
}
func (envw *environmentWatcher) deleteBuilderService(sel map[string]string) error {
svcList, err := envw.getBuilderServiceList(sel)
func (envw *environmentWatcher) deleteBuilderServiceByName(name, namespace string) error {
err := envw.kubernetesClient.CoreV1().
Services(namespace).
Delete(name, &delOpt)
if err != nil {
return fmt.Errorf("Error deleting builder service %s.%s: %v", name, namespace, err)
}
return nil
}
func (envw *environmentWatcher) deleteBuilderDeploymentByName(name, namespace string) error {
err := envw.kubernetesClient.ExtensionsV1beta1().
Deployments(namespace).
Delete(name, &delOpt)
if err != nil {
return fmt.Errorf("Error deleting builder deployment %s.%s: %v", name, namespace, err)
}
return nil
}
func (envw *environmentWatcher) deleteBuilderService(sel map[string]string, ns string) error {
svcList, err := envw.getBuilderServiceList(sel, ns)
if err != nil {
return err
}
for _, svc := range svcList {
log.Printf("Removing builder service: %v", svc.ObjectMeta.Name)
// cascading deletion
// https://kubernetes.io/docs/concepts/workloads/controllers/garbage-collection/
falseVal := false
delOpt := &metav1.DeleteOptions{
OrphanDependents: &falseVal,
}
err = envw.kubernetesClient.
Services(envw.builderNamespace).
Delete(svc.ObjectMeta.Name, delOpt)
err = envw.kubernetesClient.CoreV1().
Services(ns).
Delete(svc.ObjectMeta.Name, &delOpt)
if err != nil {
return fmt.Errorf("Error deleting builder service: %v", err)
}
@@ -359,22 +409,16 @@ func (envw *environmentWatcher) deleteBuilderService(sel map[string]string) erro
return nil
}
func (envw *environmentWatcher) deleteBuilderDeployment(sel map[string]string) error {
deployList, err := envw.getBuilderDeploymentList(sel)
func (envw *environmentWatcher) deleteBuilderDeployment(sel map[string]string, ns string) error {
deployList, err := envw.getBuilderDeploymentList(sel, ns)
if err != nil {
return err
}
for _, deploy := range deployList {
log.Printf("Removing builder deployment: %v", deploy.ObjectMeta.Name)
falseVal := false
delOpt := &metav1.DeleteOptions{
OrphanDependents: &falseVal,
}
err = envw.kubernetesClient.ExtensionsV1beta1().
Deployments(envw.builderNamespace).
Delete(deploy.ObjectMeta.Name, delOpt)
Deployments(ns).
Delete(deploy.ObjectMeta.Name, &delOpt)
if err != nil {
return fmt.Errorf("Error deleteing builder deployment: %v", err)
}
@@ -382,8 +426,8 @@ func (envw *environmentWatcher) deleteBuilderDeployment(sel map[string]string) e
return nil
}
func (envw *environmentWatcher) getBuilderServiceList(sel map[string]string) ([]apiv1.Service, error) {
svcList, err := envw.kubernetesClient.Services(envw.builderNamespace).List(
func (envw *environmentWatcher) getBuilderServiceList(sel map[string]string, ns string) ([]apiv1.Service, error) {
svcList, err := envw.kubernetesClient.CoreV1().Services(ns).List(
metav1.ListOptions{
LabelSelector: labels.Set(sel).AsSelector().String(),
})
@@ -393,12 +437,12 @@ func (envw *environmentWatcher) getBuilderServiceList(sel map[string]string) ([]
return svcList.Items, nil
}
func (envw *environmentWatcher) createBuilderService(env *crd.Environment) (*apiv1.Service, error) {
name := envw.getCacheKey(env.Metadata.Name, env.Metadata.ResourceVersion)
sel := envw.getLabels(env.Metadata.Name, env.Metadata.ResourceVersion)
func (envw *environmentWatcher) createBuilderService(env *crd.Environment, ns string) (*apiv1.Service, error) {
name := fmt.Sprintf("%v-%v", env.Metadata.Name, env.Metadata.ResourceVersion)
sel := envw.getLabels(env.Metadata.Name, ns, env.Metadata.ResourceVersion)
service := apiv1.Service{
ObjectMeta: metav1.ObjectMeta{
Namespace: envw.builderNamespace,
Namespace: ns,
Name: name,
Labels: sel,
},
@@ -428,15 +472,15 @@ func (envw *environmentWatcher) createBuilderService(env *crd.Environment) (*api
},
}
log.Printf("Creating builder service: %v", name)
_, err := envw.kubernetesClient.Services(envw.builderNamespace).Create(&service)
_, err := envw.kubernetesClient.CoreV1().Services(ns).Create(&service)
if err != nil {
return nil, err
}
return &service, nil
}
func (envw *environmentWatcher) getBuilderDeploymentList(sel map[string]string) ([]v1beta1.Deployment, error) {
deployList, err := envw.kubernetesClient.ExtensionsV1beta1().Deployments(envw.builderNamespace).List(
func (envw *environmentWatcher) getBuilderDeploymentList(sel map[string]string, ns string) ([]v1beta1.Deployment, error) {
deployList, err := envw.kubernetesClient.ExtensionsV1beta1().Deployments(ns).List(
metav1.ListOptions{
LabelSelector: labels.Set(sel).AsSelector().String(),
})
@@ -446,22 +490,25 @@ func (envw *environmentWatcher) getBuilderDeploymentList(sel map[string]string)
return deployList.Items, nil
}
func (envw *environmentWatcher) createBuilderDeployment(env *crd.Environment) (*v1beta1.Deployment, error) {
func (envw *environmentWatcher) createBuilderDeployment(env *crd.Environment, ns string) (*v1beta1.Deployment, error) {
sharedMountPath := "/packages"
sharedCfgMapPath := "/configs"
sharedSecretPath := "/secrets"
name := envw.getCacheKey(env.Metadata.Name, env.Metadata.ResourceVersion)
sel := envw.getLabels(env.Metadata.Name, env.Metadata.ResourceVersion)
name := fmt.Sprintf("%v-%v", env.Metadata.Name, env.Metadata.ResourceVersion)
sel := envw.getLabels(env.Metadata.Name, ns, env.Metadata.ResourceVersion)
var replicas int32 = 1
podAnnotation := make(map[string]string)
podAnnotations := env.Metadata.Annotations
if podAnnotations == nil {
podAnnotations = make(map[string]string)
}
if envw.useIstio && env.Spec.AllowAccessToExternalNetwork {
podAnnotation["sidecar.istio.io/inject"] = "false"
podAnnotations["sidecar.istio.io/inject"] = "false"
}
deployment := &v1beta1.Deployment{
ObjectMeta: metav1.ObjectMeta{
Namespace: envw.builderNamespace,
Namespace: ns,
Name: name,
Labels: sel,
},
@@ -473,7 +520,7 @@ func (envw *environmentWatcher) createBuilderDeployment(env *crd.Environment) (*
Template: apiv1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: sel,
Annotations: podAnnotation,
Annotations: podAnnotations,
},
Spec: apiv1.PodSpec{
Volumes: []apiv1.Volume{
@@ -497,10 +544,10 @@ func (envw *environmentWatcher) createBuilderDeployment(env *crd.Environment) (*
},
},
Containers: []apiv1.Container{
{
fission.MergeContainerSpecs(&apiv1.Container{
Name: "builder",
Image: env.Spec.Builder.Image,
ImagePullPolicy: apiv1.PullAlways,
ImagePullPolicy: apiv1.PullIfNotPresent,
TerminationMessagePath: "/dev/termination-log",
VolumeMounts: []apiv1.VolumeMount{
{
@@ -530,7 +577,7 @@ func (envw *environmentWatcher) createBuilderDeployment(env *crd.Environment) (*
},
},
},
},
}, env.Spec.Builder.Container),
{
Name: "fetcher",
Image: envw.fetcherImage,
@@ -574,10 +621,11 @@ func (envw *environmentWatcher) createBuilderDeployment(env *crd.Environment) (*
},
},
}
log.Printf("Creating builder deployment: %v", envw.getCacheKey(env.Metadata.Name, env.Metadata.ResourceVersion))
_, err := envw.kubernetesClient.ExtensionsV1beta1().Deployments(envw.builderNamespace).Create(deployment)
log.Printf("Creating builder deployment: %v", fmt.Sprintf("%v-%v", env.Metadata.Name, env.Metadata.ResourceVersion))
_, err := envw.kubernetesClient.ExtensionsV1beta1().Deployments(ns).Create(deployment)
if err != nil {
return nil, err
}
return deployment, nil
}
+44 -17
View File
@@ -21,35 +21,38 @@ import (
"log"
"time"
apiv1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/fields"
apiv1 "k8s.io/client-go/pkg/api/v1"
k8sCache "k8s.io/client-go/tools/cache"
"github.com/fission/fission"
"github.com/fission/fission/cache"
"github.com/fission/fission/crd"
"k8s.io/client-go/kubernetes"
)
type (
packageWatcher struct {
fissionClient *crd.FissionClient
k8sClient *kubernetes.Clientset
podStore k8sCache.Store
pkgStore k8sCache.Store
builderNamespace string
storageSvcUrl string
}
)
func makePackageWatcher(fissionClient *crd.FissionClient, getter k8sCache.Getter,
func makePackageWatcher(fissionClient *crd.FissionClient, k8sClientSet *kubernetes.Clientset,
builderNamespace string, storageSvcUrl string) *packageWatcher {
lw := k8sCache.NewListWatchFromClient(getter, "pods", builderNamespace, fields.Everything())
lw := k8sCache.NewListWatchFromClient(k8sClientSet.CoreV1().RESTClient(), "pods", metav1.NamespaceAll, fields.Everything())
store, controller := k8sCache.NewInformer(lw, &apiv1.Pod{}, 30*time.Second, k8sCache.ResourceEventHandlerFuncs{})
go controller.Run(make(chan struct{}))
pkgw := &packageWatcher{
fissionClient: fissionClient,
k8sClient: k8sClientSet,
podStore: store,
builderNamespace: builderNamespace,
storageSvcUrl: storageSvcUrl,
@@ -67,28 +70,28 @@ func makePackageWatcher(fissionClient *crd.FissionClient, getter k8sCache.Getter
// 5. Update package resource in package ref of functions that share the same package
// 6. Update package status to succeed state
// *. Update package status to failed state,if any one of steps above failed/time out
func (pkgw *packageWatcher) build(buildCache *cache.Cache, pkg *crd.Package) {
func (pkgw *packageWatcher) build(buildCache *cache.Cache, srcpkg *crd.Package) {
// Ignore non-pending state packages.
if pkg.Status.BuildStatus != fission.BuildStatusPending {
if srcpkg.Status.BuildStatus != fission.BuildStatusPending {
return
}
// Ignore duplicate build requests
key := fmt.Sprintf("%v-%v", pkg.Metadata.Name, pkg.Metadata.ResourceVersion)
err, _ := buildCache.Set(key, pkg)
key := fmt.Sprintf("%v-%v", srcpkg.Metadata.Name, srcpkg.Metadata.ResourceVersion)
err, _ := buildCache.Set(key, srcpkg)
if err != nil {
return
}
defer buildCache.Delete(key)
log.Printf("Start build for package %v with resource version %v", pkg.Metadata.Name, pkg.Metadata.ResourceVersion)
log.Printf("Start build for package %v with resource version %v", srcpkg.Metadata.Name, srcpkg.Metadata.ResourceVersion)
pkg, err = updatePackage(pkgw.fissionClient, pkg, fission.BuildStatusRunning, "", nil)
pkg, err := updatePackage(pkgw.fissionClient, srcpkg, fission.BuildStatusRunning, "", nil)
if err != nil {
e := fmt.Sprintf("Error setting package pending state: %v", err)
log.Println(e)
updatePackage(pkgw.fissionClient, pkg, fission.BuildStatusFailed, e, nil)
updatePackage(pkgw.fissionClient, srcpkg, fission.BuildStatusFailed, e, nil)
return
}
@@ -118,8 +121,16 @@ func (pkgw *packageWatcher) build(buildCache *cache.Cache, pkg *crd.Package) {
for _, item := range items {
pod := item.(*apiv1.Pod)
// In order to support backward compatibility, for all builder images created in default env,
// the pods will be created in fission-builder namespace
builderNs := pkgw.builderNamespace
if env.Metadata.Namespace != metav1.NamespaceDefault {
builderNs = env.Metadata.Namespace
}
// Filter non-matching pods
if pod.ObjectMeta.Labels[LABEL_ENV_NAME] != env.Metadata.Name ||
pod.ObjectMeta.Labels[LABEL_ENV_NAMESPACE] != builderNs ||
pod.ObjectMeta.Labels[LABEL_ENV_RESOURCEVERSION] != env.Metadata.ResourceVersion {
continue
}
@@ -138,8 +149,18 @@ func (pkgw *packageWatcher) build(buildCache *cache.Cache, pkg *crd.Package) {
break
}
uploadResp, buildLogs, err := buildPackage(pkgw.fissionClient,
pkgw.builderNamespace, pkgw.storageSvcUrl, pkg)
// Add the package getter rolebinding to builder sa
// we continue here if role binding was not setup succeesffully. this is because without this, the fetcher wont be able to fetch the source pkg into the container and
// the build will fail eventually
err := fission.SetupRoleBinding(pkgw.k8sClient, fission.PackageGetterRB, pkg.Metadata.Namespace, fission.PackageGetterCR, fission.ClusterRole, fission.FissionBuilderSA, builderNs)
if err != nil {
log.Printf("Error : %v in setting up the role binding %s for pkg : %s.%s", err, fission.PackageGetterRB, pkg.Metadata.Name, pkg.Metadata.Namespace)
continue
} else {
log.Printf("Setup rolebinding for sa : %s.%s for pkg : %s.%s", fission.FissionBuilderSA, builderNs, pkg.Metadata.Name, pkg.Metadata.Namespace)
}
uploadResp, buildLogs, err := buildPackage(pkgw.fissionClient, builderNs, pkgw.storageSvcUrl, pkg)
if err != nil {
log.Printf("Error building package %v: %v", pkg.Metadata.Name, err)
updatePackage(pkgw.fissionClient, pkg, fission.BuildStatusFailed, buildLogs, nil)
@@ -149,7 +170,7 @@ func (pkgw *packageWatcher) build(buildCache *cache.Cache, pkg *crd.Package) {
log.Printf("Start updating info of package: %v", pkg.Metadata.Name)
fnList, err := pkgw.fissionClient.
Functions(metav1.NamespaceDefault).List(metav1.ListOptions{})
Functions(metav1.NamespaceAll).List(metav1.ListOptions{})
if err != nil {
e := fmt.Sprintf("Error getting function list: %v", err)
log.Println(e)
@@ -191,13 +212,18 @@ func (pkgw *packageWatcher) build(buildCache *cache.Cache, pkg *crd.Package) {
// build timeout
updatePackage(pkgw.fissionClient, pkg,
fission.BuildStatusFailed, "Build timeout due to environment builder not ready", nil)
log.Printf("Max retries exceeded in building the source pkg : %s.%s, timeout due to environment builder not ready",
pkg.Metadata.Name, pkg.Metadata.Namespace)
return
}
func (pkgw *packageWatcher) watchPackages() {
func (pkgw *packageWatcher) watchPackages(fissionClient *crd.FissionClient,
kubernetesClient *kubernetes.Clientset, builderNamespace string) {
buildCache := cache.MakeCache(0, 0)
lw := k8sCache.NewListWatchFromClient(pkgw.fissionClient.GetCrdClient(), "packages", apiv1.NamespaceDefault, fields.Everything())
_, controller := k8sCache.NewInformer(lw, &crd.Package{}, 60*time.Second, k8sCache.ResourceEventHandlerFuncs{
lw := k8sCache.NewListWatchFromClient(pkgw.fissionClient.GetCrdClient(), "packages", apiv1.NamespaceAll, fields.Everything())
pkgStore, controller := k8sCache.NewInformer(lw, &crd.Package{}, 60*time.Second, k8sCache.ResourceEventHandlerFuncs{
AddFunc: func(obj interface{}) {
pkg := obj.(*crd.Package)
go pkgw.build(buildCache, pkg)
@@ -207,5 +233,6 @@ func (pkgw *packageWatcher) watchPackages() {
go pkgw.build(buildCache, pkg)
},
})
pkgw.pkgStore = pkgStore
controller.Run(make(chan struct{}))
}
+2 -2
View File
@@ -1,6 +1,6 @@
apiVersion: v1
name: fission-all
version: 0.6.0
version: 0.9.2
description: Fission is a fast serverless framework for Kubernetes.
keywords:
- fission
@@ -12,4 +12,4 @@ maintainers:
- name: Ta Ching Chen
email: contact@tachingchen.com
engine: gotpl
appVersion: 0.6.0
appVersion: 0.9.2
+3 -3
View File
@@ -1,13 +1,13 @@
1. Install the client CLI.
Mac:
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.6.0/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.9.2/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
Linux:
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.6.0/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.9.2/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
Windows:
For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/0.6.0/fission-cli-windows.exe
For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/0.9.2/fission-cli-windows.exe
2. You're ready to use Fission!
+80 -50
View File
@@ -22,6 +22,37 @@ metadata:
istio-injection: enabled
{{- end }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: secret-configmap-getter
rules:
- apiGroups:
- '*'
resources:
- secrets
- configmaps
verbs:
- get
- watch
- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: package-getter
rules:
- apiGroups:
- '*'
resources:
- packages
verbs:
- get
- watch
- list
---
apiVersion: v1
kind: ServiceAccount
@@ -44,21 +75,6 @@ roleRef:
name: admin
apiGroup: rbac.authorization.k8s.io
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: fission-function-admin
namespace: {{ .Values.functionNamespace }}
subjects:
- kind: ServiceAccount
name: fission-svc
namespace: {{ .Release.Namespace }}
roleRef:
kind: ClusterRole
name: admin
apiGroup: rbac.authorization.k8s.io
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
@@ -79,21 +95,6 @@ kind: ServiceAccount
metadata:
name: fission-fetcher
namespace: {{ .Values.functionNamespace }}
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: fission-fetcher-crd
subjects:
- kind: ServiceAccount
name: fission-fetcher
namespace: {{ .Values.functionNamespace }}
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
---
apiVersion: v1
kind: ServiceAccount
@@ -101,20 +102,6 @@ metadata:
name: fission-builder
namespace: {{ .Values.builderNamespace }}
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: fission-builder-crd
subjects:
- kind: ServiceAccount
name: fission-builder
namespace: {{ .Values.builderNamespace }}
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
---
apiVersion: extensions/v1beta1
kind: Deployment
@@ -139,8 +126,10 @@ spec:
env:
- name: FISSION_FUNCTION_NAMESPACE
value: "{{ .Values.functionNamespace }}"
- name: ENABLE_ISTIO
value: "{{ .Values.enableIstio }}"
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
readinessProbe:
httpGet:
path: "/healthz"
@@ -170,6 +159,10 @@ spec:
labels:
application: fission-router
svc: router
annotations:
prometheus.io/scrape: "true"
prometheus.io/path: "/metrics"
prometheus.io/port: "8080"
spec:
containers:
- name: router
@@ -177,6 +170,19 @@ spec:
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--routerPort", "8888", "--executorUrl", "http://executor.{{ .Release.Namespace }}"]
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: ROUTER_ROUND_TRIP_TIMEOUT
value: {{ .Values.routerRoundTripTimeout | default "50ms" | quote }}
- name: ROUTER_ROUNDTRIP_TIMEOUT_EXPONENT
value: {{ .Values.routerRoundTripTimeoutExponent | default 2 | quote }}
- name: ROUTER_ROUND_TRIP_KEEP_ALIVE_TIME
value: {{ .Values.routerRoundTripKeepAliveTime | default "30s" | quote }}
- name: ROUTER_ROUND_TRIP_MAX_RETRIES
value: {{ .Values.routerRoundTripMaxRetries | default 10 | quote }}
readinessProbe:
httpGet:
path: "/router-healthz"
@@ -190,6 +196,11 @@ spec:
port: 8888
initialDelaySeconds: 35
periodSeconds: 5
ports:
- containerPort: 8080
name: metrics
- containerPort: 8888
name: http
serviceAccount: fission-svc
---
@@ -221,6 +232,10 @@ spec:
metadata:
labels:
svc: executor
annotations:
prometheus.io/scrape: "true"
prometheus.io/path: "/metrics"
prometheus.io/port: "8080"
spec:
containers:
- name: executor
@@ -237,6 +252,14 @@ spec:
value: "{{ .Values.pullPolicy }}"
- name: ENABLE_ISTIO
value: "{{ .Values.enableIstio }}"
- name: FETCHER_MINCPU
value: {{ .Values.fetcherMinCpu | default "10m" | quote }}
- name: FETCHER_MINMEM
value: {{ .Values.fetcherMinMem | default "16Mi" | quote }}
- name: FETCHER_MAXCPU
value: {{ .Values.fetcherMaxCpu | default "40m" | quote }}
- name: FETCHER_MAXMEM
value: {{ .Values.fetcherMaxMem | default "128Mi" | quote }}
readinessProbe:
httpGet:
path: "/healthz"
@@ -250,6 +273,11 @@ spec:
port: 8888
initialDelaySeconds: 35
periodSeconds: 5
ports:
- containerPort: 8080
name: metrics
- containerPort: 8888
name: http
serviceAccount: fission-svc
---
@@ -300,7 +328,7 @@ spec:
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--kubewatcher"]
args: ["--kubewatcher", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
serviceAccount: fission-svc
---
@@ -449,7 +477,8 @@ spec:
- name: fission-log
hostPath:
path: /var/log/fission
updateStrategy:
type: RollingUpdate
---
apiVersion: extensions/v1beta1
kind: Deployment
@@ -469,7 +498,7 @@ spec:
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--timer"]
args: ["--timer", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
serviceAccount: fission-svc
#
@@ -547,7 +576,7 @@ spec:
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--mqt"]
args: ["--mqt", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
env:
- name: MESSAGE_QUEUE_TYPE
value: {{ .Values.messageQueue.type }}
@@ -578,6 +607,7 @@ spec:
metadata:
labels:
svc: storagesvc
application: fission-storage
spec:
containers:
- name: storagesvc
@@ -13,7 +13,7 @@ metadata:
annotations:
# This is what defines this resource as a hook. Without this line, the
# job is considered part of the release.
"helm.sh/hook": post-install,post-upgrade
"helm.sh/hook": post-install
"helm.sh/hook-delete-policy": hook-succeeded
spec:
template:
@@ -0,0 +1,35 @@
{{ if .Values.analytics }}
apiVersion: batch/v1
kind: Job
metadata:
name: {{ template "fullname" . }}-{{ .Chart.Version }}
labels:
# The "release" convention makes it easy to tie a release to all of the
# Kubernetes resources that were created as part of that release.
release: {{ .Release.Name }}
# This makes it easy to audit chart usage.
chart: {{ .Chart.Name }}-{{ .Chart.Version }}
app: {{ template "name" . }}
annotations:
# This is what defines this resource as a hook. Without this line, the
# job is considered part of the release.
"helm.sh/hook": post-upgrade
"helm.sh/hook-delete-policy": hook-succeeded
spec:
template:
metadata:
name: {{ template "fullname" . }}
labels:
release: {{ .Release.Name }}
app: {{ template "name" . }}
annotations:
{{- if .Values.enableIstio }}
"sidecar.istio.io/inject": "false"
{{- end }}
spec:
restartPolicy: Never
containers:
- name: post-upgrade-job
image: "fission/alpinecurl"
command: ["sh", "-c", "/usr/bin/curl -m 5 -H \"Content-Type: application/json\" -X POST -d '{\"type\": \"helm-post-upgrade\", \"chartName\": \"{{ .Chart.Name }}\", \"chartVersion\": \"{{ .Chart.Version }}\"}' https://g.fission.sh/analytics || true"]
{{ end }}
@@ -0,0 +1,33 @@
apiVersion: batch/v1
kind: Job
metadata:
name: {{ template "fullname" . }}-{{ .Chart.Version }}-{{ randNumeric 3 }}
labels:
# The "release" convention makes it easy to tie a release to all of the
# Kubernetes resources that were created as part of that release.
release: "{{ .Release.Name }}"
# This makes it easy to audit chart usage.
chart: {{ .Chart.Name }}-{{ .Chart.Version }}
app: {{ template "name" . }}
annotations:
# This is what defines this resource as a hook. Without this line, the
# job is considered part of the release.
"helm.sh/hook": pre-upgrade
"helm.sh/hook-delete-policy": hook-succeeded
spec:
backoffLimit: 0
template:
metadata:
name: {{ template "fullname" . }}
labels:
release: "{{ .Release.Name }}"
app: {{ template "name" . }}
spec:
restartPolicy: Never
containers:
- name: pre-upgrade-job
image: {{ .Values.preUpgradeChecksImage }}:{{ .Values.imageTag }}
imagePullPolicy: {{ .Values.pullPolicy }}
command: [ "/pre-upgrade-checks" ]
args: ["--fn-pod-namespace", "{{ .Values.functionNamespace }}", "--envbuilder-namespace", "{{ .Values.builderNamespace }}"]
serviceAccount: fission-svc
+39
View File
@@ -0,0 +1,39 @@
apiVersion: v1
kind: Service
metadata:
name: redis
labels:
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
spec:
ports:
- port: 6379
targetPort: 6379
protocol: TCP
name: redis
selector:
app: redis
---
apiVersion: apps/v1beta2
kind: StatefulSet
metadata:
name: redis
labels:
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
spec:
selector:
matchLabels:
app: redis
serviceName: redis
replicas: 1
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: redis:3.2-alpine
imagePullPolicy: Always
ports:
- containerPort: 6379
name: redis
+1
View File
@@ -65,6 +65,7 @@ metadata:
name: storagesvc
labels:
svc: storagesvc
application: fission-storage
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
spec:
type: ClusterIP
+6 -3
View File
@@ -16,13 +16,13 @@ image: fission/fission-bundle
pullPolicy: IfNotPresent
## Fission image version
imageTag: 0.6.0
imageTag: 0.9.2
## Fission fetcher repository
fetcherImage: fission/fetcher
## Fission fetcher image version
fetcherImageTag: 0.6.0
fetcherImageTag: 0.9.2
## Port at which Fission controller service should be exposed
controllerPort: 31313
@@ -48,7 +48,7 @@ enableIstio: false
logger:
influxdbAdmin: "admin"
fluentdImage: fission/fluentd
fluentdImageTag: 0.6.0
fluentdImageTag: 0.9.2
## Type of Queue you would like to use
## currently supports nats-streaming, azure-storage-queue
@@ -91,3 +91,6 @@ heapster: false
## This interval configures the frequency at which it runs inside the storagesvc pod.
## The value is in minutes.
pruneInterval: 60
## Fission pre-install/pre-upgrade checks live in this image
preUpgradeChecksImage: fission/pre-upgrade-checks
+2 -2
View File
@@ -1,6 +1,6 @@
apiVersion: v1
name: fission-core
version: 0.6.0
version: 0.9.2
description: Fission is a fast serverless framework for Kubernetes.
keywords:
- fission
@@ -12,4 +12,4 @@ maintainers:
- name: Ta Ching Chen
email: contact@tachingchen.com
engine: gotpl
appVersion: 0.6.0
appVersion: 0.9.2
+3 -3
View File
@@ -1,13 +1,13 @@
1. Install the client CLI.
Mac:
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.6.0/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.9.2/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
Linux:
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.6.0/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.9.2/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
Windows:
For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/0.6.0/fission-cli-windows.exe
For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/0.9.2/fission-cli-windows.exe
2. You're ready to use Fission!
+65 -53
View File
@@ -22,6 +22,37 @@ metadata:
istio-injection: enabled
{{- end }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: secret-configmap-getter
rules:
- apiGroups:
- '*'
resources:
- secrets
- configmaps
verbs:
- get
- watch
- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: package-getter
rules:
- apiGroups:
- '*'
resources:
- packages
verbs:
- get
- watch
- list
---
apiVersion: v1
kind: ServiceAccount
@@ -44,21 +75,6 @@ roleRef:
name: admin
apiGroup: rbac.authorization.k8s.io
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: fission-function-admin
namespace: {{ .Values.functionNamespace }}
subjects:
- kind: ServiceAccount
name: fission-svc
namespace: {{ .Release.Namespace }}
roleRef:
kind: ClusterRole
name: admin
apiGroup: rbac.authorization.k8s.io
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
@@ -80,20 +96,6 @@ metadata:
name: fission-fetcher
namespace: {{ .Values.functionNamespace }}
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: fission-fetcher-crd
subjects:
- kind: ServiceAccount
name: fission-fetcher
namespace: {{ .Values.functionNamespace }}
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
---
apiVersion: v1
kind: ServiceAccount
@@ -101,20 +103,6 @@ metadata:
name: fission-builder
namespace: {{ .Values.builderNamespace }}
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: fission-builder-crd
subjects:
- kind: ServiceAccount
name: fission-builder
namespace: {{ .Values.builderNamespace }}
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
---
apiVersion: extensions/v1beta1
kind: Deployment
@@ -139,19 +127,21 @@ spec:
env:
- name: FISSION_FUNCTION_NAMESPACE
value: "{{ .Values.functionNamespace }}"
- name: ENABLE_ISTIO
value: "{{ .Values.enableIstio }}"
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
readinessProbe:
httpGet:
path: "/healthz"
port: "8888"
port: 8888
initialDelaySeconds: 1
periodSeconds: 1
failureThreshold: 30
livenessProbe:
httpGet:
path: "/healthz"
port: "8888"
port: 8888
initialDelaySeconds: 35
periodSeconds: 5
serviceAccount: fission-svc
@@ -177,17 +167,30 @@ spec:
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--routerPort", "8888", "--executorUrl", "http://executor.{{ .Release.Namespace }}"]
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: ROUTER_ROUND_TRIP_TIMEOUT
value: {{ .Values.routerRoundTripTimeout | default "50ms" | quote }}
- name: ROUTER_ROUNDTRIP_TIMEOUT_EXPONENT
value: {{ .Values.routerRoundTripTimeoutExponent | default 2 | quote }}
- name: ROUTER_ROUND_TRIP_KEEP_ALIVE_TIME
value: {{ .Values.routerRoundTripKeepAliveTime | default "30s" | quote }}
- name: ROUTER_ROUND_TRIP_MAX_RETRIES
value: {{ .Values.routerRoundTripMaxRetries | default 10 | quote }}
readinessProbe:
httpGet:
path: "/router-healthz"
port: "8888"
port: 8888
initialDelaySeconds: 1
periodSeconds: 1
failureThreshold: 30
livenessProbe:
httpGet:
path: "/router-healthz"
port: "8888"
port: 8888
initialDelaySeconds: 35
periodSeconds: 5
serviceAccount: fission-svc
@@ -235,17 +238,25 @@ spec:
value: "{{ .Values.pullPolicy }}"
- name: ENABLE_ISTIO
value: "{{ .Values.enableIstio }}"
- name: FETCHER_MINCPU
value: {{ .Values.fetcherMinCpu | default "10m" | quote }}
- name: FETCHER_MINMEM
value: {{ .Values.fetcherMinMem | default "16Mi" | quote }}
- name: FETCHER_MAXCPU
value: {{ .Values.fetcherMaxCpu | default "40m" | quote }}
- name: FETCHER_MAXMEM
value: {{ .Values.fetcherMaxMem | default "128Mi" | quote }}
readinessProbe:
httpGet:
path: "/healthz"
port: "8888"
port: 8888
initialDelaySeconds: 1
periodSeconds: 1
failureThreshold: 30
livenessProbe:
httpGet:
path: "/healthz"
port: "8888"
port: 8888
initialDelaySeconds: 35
periodSeconds: 5
serviceAccount: fission-svc
@@ -298,7 +309,7 @@ spec:
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--kubewatcher"]
args: ["--kubewatcher", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
serviceAccount: fission-svc
---
@@ -320,7 +331,7 @@ spec:
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ .Values.pullPolicy }}
command: ["/fission-bundle"]
args: ["--timer"]
args: ["--timer", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
serviceAccount: fission-svc
---
@@ -336,6 +347,7 @@ spec:
metadata:
labels:
svc: storagesvc
application: fission-storage
spec:
containers:
- name: storagesvc
@@ -13,7 +13,7 @@ metadata:
annotations:
# This is what defines this resource as a hook. Without this line, the
# job is considered part of the release.
"helm.sh/hook": post-install,post-upgrade
"helm.sh/hook": post-install
"helm.sh/hook-delete-policy": hook-succeeded
spec:
template:
@@ -0,0 +1,35 @@
{{ if .Values.analytics }}
apiVersion: batch/v1
kind: Job
metadata:
name: {{ template "fullname" . }}-{{ .Chart.Version }}
labels:
# The "release" convention makes it easy to tie a release to all of the
# Kubernetes resources that were created as part of that release.
release: {{ .Release.Name }}
# This makes it easy to audit chart usage.
chart: {{ .Chart.Name }}-{{ .Chart.Version }}
app: {{ template "name" . }}
annotations:
# This is what defines this resource as a hook. Without this line, the
# job is considered part of the release.
"helm.sh/hook": post-upgrade
"helm.sh/hook-delete-policy": hook-succeeded
spec:
template:
metadata:
name: {{ template "fullname" . }}
labels:
release: {{ .Release.Name }}
app: {{ template "name" . }}
annotations:
{{- if .Values.enableIstio }}
"sidecar.istio.io/inject": "false"
{{- end }}
spec:
restartPolicy: Never
containers:
- name: post-upgrade-job
image: "fission/alpinecurl"
command: ["sh", "-c", "/usr/bin/curl -m 5 -H \"Content-Type: application/json\" -X POST -d '{\"type\": \"helm-post-upgrade\", \"chartName\": \"{{ .Chart.Name }}\", \"chartVersion\": \"{{ .Chart.Version }}\"}' https://g.fission.sh/analytics || true"]
{{ end }}
@@ -0,0 +1,33 @@
apiVersion: batch/v1
kind: Job
metadata:
name: {{ template "fullname" . }}-{{ .Chart.Version }}-{{ randNumeric 3 }}
labels:
# The "release" convention makes it easy to tie a release to all of the
# Kubernetes resources that were created as part of that release.
release: "{{ .Release.Name }}"
# This makes it easy to audit chart usage.
chart: {{ .Chart.Name }}-{{ .Chart.Version }}
app: {{ template "name" . }}
annotations:
# This is what defines this resource as a hook. Without this line, the
# job is considered part of the release.
"helm.sh/hook": pre-upgrade
"helm.sh/hook-delete-policy": hook-succeeded
spec:
backoffLimit: 0
template:
metadata:
name: {{ template "fullname" . }}
labels:
release: "{{ .Release.Name }}"
app: {{ template "name" . }}
spec:
restartPolicy: Never
containers:
- name: pre-upgrade-job
image: {{ .Values.preUpgradeChecksImage }}:{{ .Values.imageTag }}
imagePullPolicy: {{ .Values.pullPolicy }}
command: [ "/pre-upgrade-checks" ]
args: ["--fn-pod-namespace", "{{ .Values.functionNamespace }}", "--envbuilder-namespace", "{{ .Values.builderNamespace }}"]
serviceAccount: fission-svc
+1
View File
@@ -44,6 +44,7 @@ metadata:
name: storagesvc
labels:
svc: storagesvc
application: fission-storage
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
spec:
type: ClusterIP
+5 -2
View File
@@ -13,7 +13,7 @@ routerServiceType: LoadBalancer
image: fission/fission-bundle
## Fission image version
imageTag: 0.6.0
imageTag: 0.9.2
## Image pull policy
pullPolicy: IfNotPresent
@@ -22,7 +22,7 @@ pullPolicy: IfNotPresent
fetcherImage: fission/fetcher
## Fission fetcher image version
fetcherImageTag: 0.6.0
fetcherImageTag: 0.9.2
## Port at which Fission controller service should be exposed
controllerPort: 31313
@@ -62,3 +62,6 @@ analytics: true
## This interval configures the frequency at which it runs inside the storagesvc pod.
## The value is in minutes.
pruneInterval: 60
## Fission pre-install/pre-upgrade checks live in this image
preUpgradeChecksImage: fission/pre-upgrade-checks
+72 -1
View File
@@ -19,14 +19,24 @@ package fission
import (
"fmt"
"net"
"net/http"
"os"
"os/signal"
"runtime/debug"
"strings"
"syscall"
"github.com/gorilla/handlers"
"github.com/imdario/mergo"
apiv1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func UrlForFunction(name string) string {
func UrlForFunction(name, namespace string) string {
prefix := "/fission-function"
if namespace != metav1.NamespaceDefault {
prefix = fmt.Sprintf("/fission-function/%s", namespace)
}
return fmt.Sprintf("%v/%v", prefix, name)
}
@@ -52,3 +62,64 @@ func IsNetworkError(err error) bool {
func GetFunctionIstioServiceName(fnName, fnNamespace string) string {
return fmt.Sprintf("istio-%v-%v", fnName, fnNamespace)
}
func LoggingMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
requestURI := r.RequestURI
if !strings.Contains(requestURI, "healthz") {
// Call the next handler, which can be another middleware in the chain, or the final handler.
handlers.LoggingHandler(os.Stdout, next).ServeHTTP(w, r)
}
})
}
// MergeContainerSpecs merges container specs using a predefined order.
//
// The order of the arguments indicates which spec has precedence (lower index takes precedence over higher indexes).
// Slices and maps are merged; other fields are set only if they are a zero value.
func MergeContainerSpecs(specs ...*apiv1.Container) apiv1.Container {
result := &apiv1.Container{}
for _, spec := range specs {
if spec == nil {
continue
}
err := mergo.Merge(result, spec)
if err != nil {
panic(err)
}
}
return *result
}
// IsNetworkDialError returns true if its a network dial error
func IsNetworkDialError(err error) bool {
netErr, ok := err.(net.Error)
if !ok {
return false
}
netOpErr, ok := netErr.(*net.OpError)
if !ok {
return false
}
if netOpErr.Op == "dial" {
return true
}
return false
}
// IsReadyPod checks that all containers in a pod are ready and returns true if so
func IsReadyPod(pod *apiv1.Pod) bool {
// since its a utility function, just ensuring there is no nil pointer exception
if pod == nil {
return false
}
for _, cStatus := range pod.Status.ContainerStatuses {
if !cStatus.Ready {
return false
}
}
return true
}
+258
View File
@@ -0,0 +1,258 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package fission
import (
"fmt"
"log"
apiv1 "k8s.io/api/core/v1"
rbac "k8s.io/api/rbac/v1beta1"
k8serrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
"k8s.io/apimachinery/pkg/util/json"
"k8s.io/client-go/kubernetes"
)
// This file has util functions needed for setting up and cleaning up RBAC objects.
const (
MaxRetries = 10
)
// MakeSAObj returns a ServiceAccount object with the given SA name and namespace
func MakeSAObj(sa, ns string) *apiv1.ServiceAccount {
return &apiv1.ServiceAccount{
ObjectMeta: metav1.ObjectMeta{
Namespace: ns,
Name: sa,
},
}
}
// SetupSA checks if a service account is present in the namespace, if not creates it.
func SetupSA(k8sClient *kubernetes.Clientset, sa, ns string) (*apiv1.ServiceAccount, error) {
saObj, err := k8sClient.CoreV1().ServiceAccounts(ns).Get(sa, metav1.GetOptions{})
if err == nil {
return saObj, nil
}
if k8serrors.IsNotFound(err) {
saObj = MakeSAObj(sa, ns)
saObj, err = k8sClient.CoreV1().ServiceAccounts(ns).Create(saObj)
}
return saObj, err
}
// makeRoleBindingObj is a helper function called from other functions in this file only.
// given a rolebinging name and namespace, it makes a rolebinding object mapping the role to the SA of the namespace.
func makeRoleBindingObj(roleBinding, roleBindingNs, role, roleKind, sa, saNamespace string) *rbac.RoleBinding {
return &rbac.RoleBinding{
ObjectMeta: metav1.ObjectMeta{
Name: roleBinding,
Namespace: roleBindingNs,
},
Subjects: []rbac.Subject{
{
Kind: "ServiceAccount",
Name: sa,
Namespace: saNamespace,
},
},
RoleRef: rbac.RoleRef{
Kind: roleKind,
Name: role,
},
}
}
// isSAInRoleBinding checkis if a service account is present in the rolebinding object
func isSAInRoleBinding(rbObj *rbac.RoleBinding, sa, ns string) bool {
for _, subject := range rbObj.Subjects {
if subject.Name == sa && subject.Namespace == ns {
return true
}
}
return false
}
// PatchSpec is a standard struct needed for JSON merge.
type PatchSpec struct {
Op string `json:"op"`
Path string `json:"path"`
Value rbac.Subject `json:"value"`
}
// AddSaToRoleBindingWithRetries adds a service account to a rolebinding object. IT retries on already exists and conflict errors.
func AddSaToRoleBindingWithRetries(k8sClient *kubernetes.Clientset, roleBinding, roleBindingNs, sa, saNamespace, role, roleKind string) (err error) {
patch := PatchSpec{}
patch.Op = "add"
patch.Path = "/subjects/-"
patch.Value = rbac.Subject{
Kind: "ServiceAccount",
Name: sa,
Namespace: saNamespace,
}
patchJson, err := json.Marshal([]PatchSpec{patch})
if err != nil {
log.Printf("Error marshalling patch into json")
return err
}
for i := 0; i < MaxRetries; i++ {
_, err = k8sClient.RbacV1beta1().RoleBindings(roleBindingNs).Patch(roleBinding, types.JSONPatchType, patchJson)
if err == nil {
log.Printf("Patched rolebinding : %s.%s", roleBinding, roleBindingNs)
return err
}
if k8serrors.IsNotFound(err) {
// someone may have deleted the object between us checking if the object is present and deciding to patch
// so just create the object again
rbObj := makeRoleBindingObj(roleBinding, roleBindingNs, role, roleKind, sa, saNamespace)
rbObj, err = k8sClient.RbacV1beta1().RoleBindings(roleBindingNs).Create(rbObj)
if err == nil {
log.Printf("Created rolebinding : %s.%s", roleBinding, roleBindingNs)
return err
}
if k8serrors.IsAlreadyExists(err) {
log.Printf("rolebinding object : %s.%s already exists, retrying patch, err: %v", roleBinding, roleBindingNs, err)
continue
}
log.Printf("Error returned by rolebinding create, %v", err)
return err
}
if k8serrors.IsConflict(err) {
// TODO : Need to test this out, not able to simulate conflicts yet.
// Initially, my understanding was that patch can never error on conflict because Api server will handle the conflicts for us.
// but one CI run did show patch errored out on conflict : https://api.travis-ci.org/v3/job/373161490/log.txt, look for :
// Error returned by rolebinding patch : <some more text> there is a meaningful conflict (firstResourceVersion: "35482724", currentResourceVersion: "35482849")
// so, m guessing retrying patch should help. will watch out for any such conflicts and fix the issue if any
log.Printf("Conflict reported on patch of rolebinding : %s.%s. Retrying patch operation, err : %v", roleBinding, roleBindingNs, err)
continue
}
log.Printf("Error returned by rolebinding patch : %v", err)
return err
}
log.Printf("Exceeded maxretries : %d adding SA: %s.%s to rolebinding : %s.%s, giving up, err: %v", MaxRetries, sa, saNamespace, roleBinding, roleBindingNs, err)
return err
}
// RemoveSAFromRoleBindingWithRetries removes an SA from the rolebinding passed as parameter. If this is the only SA in
// the rolebinding, then it deletes the rolebinding object.
func RemoveSAFromRoleBindingWithRetries(k8sClient *kubernetes.Clientset, roleBinding, roleBindingNs string, saToRemove map[string]bool) (err error) {
for i := 0; i < MaxRetries; i++ {
rbObj, err := k8sClient.RbacV1beta1().RoleBindings(roleBindingNs).Get(
roleBinding, metav1.GetOptions{})
if err != nil {
// silently ignoring the error. there's no need for us to remove sa anymore.
log.Printf("rolebinding %s.%s not found, but ignoring the error since we're cleaning up", roleBinding, roleBindingNs)
return nil
}
subjects := rbObj.Subjects
newSubjects := make([]rbac.Subject, 0)
// TODO : optimize it.
for _, item := range subjects {
if _, ok := saToRemove[MakeSAMapKey(item.Name, item.Namespace)]; ok {
continue
}
newSubjects = append(newSubjects, rbac.Subject{
Kind: "ServiceAccount",
Name: item.Name,
Namespace: item.Namespace,
})
}
if len(newSubjects) == 0 {
return DeleteRoleBinding(k8sClient, roleBinding, roleBindingNs)
}
rbObj.Subjects = newSubjects
// cant use patch for deletes, the results become in-deterministic, so using update.
_, err = k8sClient.RbacV1beta1().RoleBindings(rbObj.Namespace).Update(rbObj)
switch {
case err == nil:
log.Printf("Removed sa's : %v from rolebinding : %s.%s", saToRemove, roleBinding, roleBindingNs)
return nil
case k8serrors.IsConflict(err):
log.Printf("Conflict in update of rolebinding: %s.%s, retrying", roleBinding, roleBindingNs)
continue
default:
log.Printf("Rolebinding Update Errored out : %v", err)
return err
}
}
log.Printf("Maxretries : %d exceeded for removing SA's : %v from rolebinding %s.%s, giving up, err: %v", MaxRetries, saToRemove, roleBinding, roleBindingNs, err)
return err
}
// SetupRoleBinding adds a role to a service account if the rolebinding object is already present in the namespace.
// if not, it creates a rolebinding object granting the role to the SA in the namespace.
func SetupRoleBinding(k8sClient *kubernetes.Clientset, roleBinding, roleBindingNs, role, roleKind, sa, saNamespace string) error {
// get the role binding object
rbObj, err := k8sClient.RbacV1beta1().RoleBindings(roleBindingNs).Get(
roleBinding, metav1.GetOptions{})
if err == nil {
if !isSAInRoleBinding(rbObj, sa, saNamespace) {
return AddSaToRoleBindingWithRetries(k8sClient, roleBinding, roleBindingNs, sa, saNamespace, role, roleKind)
}
log.Printf("SA : %s.%s already present in rolebinding : %s.%s, so nothing to add", sa, saNamespace, roleBinding, roleBindingNs)
return nil
}
// if role binding is missing, create it. also add this sa to the binding.
if k8serrors.IsNotFound(err) {
log.Printf("Rolebinding %s does NOT exist in ns %s. Creating it", roleBinding, roleBindingNs)
rbObj = makeRoleBindingObj(roleBinding, roleBindingNs, role, roleKind, sa, saNamespace)
rbObj, err = k8sClient.RbacV1beta1().RoleBindings(roleBindingNs).Create(rbObj)
if k8serrors.IsAlreadyExists(err) {
err = AddSaToRoleBindingWithRetries(k8sClient, roleBinding, roleBindingNs, sa, saNamespace, role, roleKind)
}
}
return err
}
// DeleteRoleBinding deletes a rolebinding object. if k8s throws an error that the rolebinding is not there, it just
// returns silently.
func DeleteRoleBinding(k8sClient *kubernetes.Clientset, roleBinding, roleBindingNs string) error {
// if deleteRoleBinding is invoked by 2 fission services at the same time for the same rolebinding,
// the first call will succeed while the 2nd will fail with isNotFound. but we dont want to error out then.
err := k8sClient.RbacV1beta1().RoleBindings(roleBindingNs).Delete(roleBinding, &metav1.DeleteOptions{})
if err == nil || k8serrors.IsNotFound(err) {
return nil
}
return err
}
func MakeSAMapKey(saName, saNamespace string) string {
return fmt.Sprintf("%s-%s", saName, saNamespace)
}
+67 -13
View File
@@ -21,13 +21,13 @@ import (
"net/http"
"os"
"runtime/debug"
"strconv"
"strings"
"github.com/gorilla/handlers"
"github.com/gorilla/mux"
log "github.com/sirupsen/logrus"
apiv1 "k8s.io/api/core/v1"
kerrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
"github.com/fission/fission"
@@ -35,6 +35,15 @@ import (
"github.com/fission/fission/fission/logdb"
)
var podNamespace string
func init() {
podNamespace = os.Getenv("POD_NAMESPACE")
if podNamespace == "" {
podNamespace = "fission"
}
}
type (
API struct {
fissionClient *crd.FissionClient
@@ -84,14 +93,6 @@ func MakeAPI() (*API, error) {
api.functionNamespace = "fission-function"
}
if len(os.Getenv("ENABLE_ISTIO")) > 0 {
istio, err := strconv.ParseBool(os.Getenv("ENABLE_ISTIO"))
if err != nil {
log.Println("Failed to parse ENABLE_ISTIO")
}
api.useIstio = istio
}
return api, err
}
@@ -119,6 +120,31 @@ func (api *API) respondWithError(w http.ResponseWriter, err error) {
http.Error(w, msg, code)
}
func (api *API) extractQueryParamFromRequest(r *http.Request, queryParam string) string {
values := r.URL.Query()
return values.Get(queryParam)
}
// check if namespace exists, if not create it.
func (api *API) createNsIfNotExists(ns string) error {
if ns == metav1.NamespaceDefault {
// we dont have to create default ns
return nil
}
_, err := api.kubernetesClient.CoreV1().Namespaces().Get(ns, metav1.GetOptions{})
if err != nil && kerrors.IsNotFound(err) {
ns := &apiv1.Namespace{
ObjectMeta: metav1.ObjectMeta{
Name: ns,
},
}
_, err = api.kubernetesClient.CoreV1().Namespaces().Create(ns)
}
return err
}
func (api *API) getLogDBConfig(dbType string) logDBConfig {
dbType = strings.ToUpper(dbType)
// retrieve db auth config from the env
@@ -138,7 +164,7 @@ func (api *API) getLogDBConfig(dbType string) logDBConfig {
func (api *API) HomeHandler(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
fmt.Fprintf(w, "{\"message\": \"Fission API\", \"version\": \"0.6.0\"}\n")
fmt.Fprintf(w, fission.ApiInfo().String())
}
func (api *API) ApiVersionMismatchHandler(w http.ResponseWriter, r *http.Request) {
@@ -150,6 +176,18 @@ func (api *API) HealthHandler(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
}
func (api *API) GetSvcName(w http.ResponseWriter, r *http.Request) {
appLabelSelector := "application=" + r.URL.Query().Get("application")
services, err := api.kubernetesClient.CoreV1().Services(podNamespace).List(metav1.ListOptions{
LabelSelector: appLabelSelector,
})
if err != nil || len(services.Items) > 1 || len(services.Items) == 0 {
api.respondWithError(w, err)
}
service := services.Items[0]
fmt.Fprintf(w, service.Name+"."+podNamespace)
}
func (api *API) Serve(port int) {
r := mux.NewRouter()
r.HandleFunc("/healthz", api.HealthHandler).Methods("GET")
@@ -199,15 +237,31 @@ func (api *API) Serve(port int) {
r.HandleFunc("/v2/triggers/messagequeue/{mqTrigger}", api.MessageQueueTriggerApiUpdate).Methods("PUT")
r.HandleFunc("/v2/triggers/messagequeue/{mqTrigger}", api.MessageQueueTriggerApiDelete).Methods("DELETE")
r.HandleFunc("/v2/deleteTpr", api.Tpr2crdApi).Methods("DELETE")
r.HandleFunc("/v2/recorders", api.RecorderApiList).Methods("GET")
r.HandleFunc("/v2/recorders", api.RecorderApiCreate).Methods("POST")
r.HandleFunc("/v2/recorders/{recorder}", api.RecorderApiGet).Methods("GET")
r.HandleFunc("/v2/recorders/{recorder}", api.RecorderApiUpdate).Methods("PUT")
r.HandleFunc("/v2/recorders/{recorder}", api.RecorderApiDelete).Methods("DELETE")
r.HandleFunc("/v2/records", api.RecordsApiListAll).Methods("GET")
r.HandleFunc("/v2/records/function/{function}", api.RecordsApiFilterByFunction).Methods("GET")
r.HandleFunc("/v2/records/trigger/{trigger}", api.RecordsApiFilterByTrigger).Methods("GET")
r.HandleFunc("/v2/records/time", api.RecordsApiFilterByTime).Methods("GET")
r.HandleFunc("/v2/replay/{reqUID}", api.ReplayByReqUID).Methods("GET")
r.HandleFunc("/v2/secrets/{secret}", api.SecretGet).Methods("GET")
r.HandleFunc("/v2/configmaps/{configmap}", api.ConfigMapGet).Methods("GET")
r.HandleFunc("/proxy/{dbType}", api.FunctionLogsApiPost).Methods("POST")
r.HandleFunc("/proxy/storage/v1/archive", api.StorageServiceProxy)
r.HandleFunc("/proxy/logs/{function}", api.FunctionPodLogs).Methods("POST")
r.HandleFunc("/proxy/workflows-apiserver/{path:.*}", api.WorkflowApiserverProxy)
r.HandleFunc("/proxy/svcname", api.GetSvcName).Queries("application", "").Methods("GET")
address := fmt.Sprintf(":%v", port)
log.WithFields(log.Fields{"port": port}).Info("Server started")
log.Fatal(http.ListenAndServe(address, handlers.LoggingHandler(os.Stdout, r)))
r.Use(fission.LoggingMiddleware)
log.Fatal(http.ListenAndServe(address, r))
}
+21 -17
View File
@@ -23,11 +23,13 @@ import (
"log"
"net/http"
"os"
"reflect"
"strings"
"testing"
"time"
"k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/pkg/api/v1"
"github.com/fission/fission"
"github.com/fission/fission/controller/client"
@@ -68,9 +70,8 @@ func assertNotFoundFailure(err error, name string) {
func assertCronSpecFails(err error) {
assert(err != nil, "using an invalid cron spec must fail")
fe, ok := err.(fission.Error)
assert(ok, "error must be a fission Error")
assert(fe.Code == fission.ErrorInvalidArgument, "error must be a invalid argument error")
ok := strings.Contains(err.Error(), "not a valid cron spec")
assert(ok, "invalid cron spec must fail")
}
func TestFunctionApi(t *testing.T) {
@@ -81,10 +82,16 @@ func TestFunctionApi(t *testing.T) {
},
Spec: fission.FunctionSpec{
Environment: fission.EnvironmentReference{
Name: "nodejs",
Name: "nodejs",
Namespace: metav1.NamespaceDefault,
},
Package: fission.FunctionPackageRef{
FunctionName: "xxx",
PackageRef: fission.PackageRef{
Namespace: metav1.NamespaceDefault,
Name: "xxx",
ResourceVersion: "12345",
},
},
},
}
@@ -115,7 +122,7 @@ func TestFunctionApi(t *testing.T) {
panicIf(err)
defer g.client.FunctionDelete(m2)
funcs, err := g.client.FunctionList()
funcs, err := g.client.FunctionList(metav1.NamespaceDefault)
panicIf(err)
assert(len(funcs) == 2, fmt.Sprintf("created two functions, but found %v", len(funcs)))
@@ -141,6 +148,7 @@ func TestHTTPTriggerApi(t *testing.T) {
Namespace: metav1.NamespaceDefault,
},
Spec: fission.HTTPTriggerSpec{
Method: http.MethodGet,
RelativeURL: "/hello",
FunctionReference: fission.FunctionReference{
Type: fission.FunctionReferenceTypeFunctionName,
@@ -180,7 +188,7 @@ func TestHTTPTriggerApi(t *testing.T) {
panicIf(err)
defer g.client.HTTPTriggerDelete(m2)
ts, err := g.client.HTTPTriggerList()
ts, err := g.client.HTTPTriggerList(metav1.NamespaceDefault)
panicIf(err)
assert(len(ts) == 2, fmt.Sprintf("created two triggers, but found %v", len(ts)))
}
@@ -214,11 +222,7 @@ func TestEnvironmentApi(t *testing.T) {
e, err := g.client.EnvironmentGet(m)
panicIf(err)
assert(testEnv.Spec.AllowedFunctionsPerContainer == e.Spec.AllowedFunctionsPerContainer, "env AllowedFunctionsPerContainer should match after reading")
assert(testEnv.Spec.Poolsize == e.Spec.Poolsize, "env Poolsize should match after reading")
assert(testEnv.Spec.Builder == e.Spec.Builder, "env Builder should match after reading")
assert(testEnv.Spec.Runtime == e.Spec.Runtime, "env Runtime should match after reading")
assert(testEnv.Spec.Version == e.Spec.Version, "env Version should match after reading")
assert(reflect.DeepEqual(testEnv.Spec, e.Spec), "env should match after reading")
testEnv.Metadata.ResourceVersion = m.ResourceVersion
testEnv.Spec.Runtime.Image = "another-img"
@@ -231,7 +235,7 @@ func TestEnvironmentApi(t *testing.T) {
panicIf(err)
defer g.client.EnvironmentDelete(m2)
ts, err := g.client.EnvironmentList()
ts, err := g.client.EnvironmentList(metav1.NamespaceDefault)
panicIf(err)
assert(len(ts) == 2, fmt.Sprintf("created two envs, but found %v", len(ts)))
}
@@ -266,16 +270,16 @@ func TestWatchApi(t *testing.T) {
w, err := g.client.WatchGet(m)
panicIf(err)
assert((testWatch.Spec.Namespace == w.Spec.Namespace &&
assert(testWatch.Spec.Namespace == w.Spec.Namespace &&
testWatch.Spec.Type == w.Spec.Type &&
testWatch.Spec.FunctionReference == w.Spec.FunctionReference), "watch should match after reading")
testWatch.Spec.FunctionReference == w.Spec.FunctionReference, "watch should match after reading")
testWatch.Metadata.Name = "yyy"
m2, err := g.client.WatchCreate(testWatch)
panicIf(err)
defer g.client.WatchDelete(m2)
ws, err := g.client.WatchList()
ws, err := g.client.WatchList(metav1.NamespaceDefault)
panicIf(err)
assert(len(ws) == 2, fmt.Sprintf("created two watches, but found %v", len(ws)))
}
@@ -319,7 +323,7 @@ func TestTimeTriggerApi(t *testing.T) {
_, err = g.client.TimeTriggerCreate(testTrigger)
assertCronSpecFails(err)
ts, err := g.client.TimeTriggerList()
ts, err := g.client.TimeTriggerList(metav1.NamespaceDefault)
panicIf(err)
assert(len(ts) == 1, fmt.Sprintf("created two time triggers, but found %v", len(ts)))
}
+24
View File
@@ -18,7 +18,9 @@ package client
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io/ioutil"
"net/http"
"strings"
@@ -88,3 +90,25 @@ func (c *Client) handleCreateResponse(resp *http.Response) ([]byte, error) {
body, err := ioutil.ReadAll(resp.Body)
return body, err
}
func (c *Client) ServerInfo() (*fission.ServerInfo, error) {
url := fmt.Sprintf(c.Url)
resp, err := http.Get(url)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
info := &fission.ServerInfo{}
err = json.Unmarshal(body, info)
if err != nil {
return nil, err
}
return info, nil
}
+99
View File
@@ -0,0 +1,99 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
apiv1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func (c *Client) SecretGet(m *metav1.ObjectMeta) (*apiv1.Secret, error) {
relativeUrl := fmt.Sprintf("secrets/%v", m.Name)
relativeUrl += fmt.Sprintf("?namespace=%v", m.Namespace)
resp, err := http.Get(c.url(relativeUrl))
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := c.handleResponse(resp)
if err != nil {
return nil, err
}
var secret apiv1.Secret
err = json.Unmarshal(body, &secret)
if err != nil {
return nil, err
}
return &secret, nil
}
func (c *Client) ConfigMapGet(m *metav1.ObjectMeta) (*apiv1.ConfigMap, error) {
relativeUrl := fmt.Sprintf("configmaps/%v", m.Name)
relativeUrl += fmt.Sprintf("?namespace=%v", m.Namespace)
resp, err := http.Get(c.url(relativeUrl))
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := c.handleResponse(resp)
if err != nil {
return nil, err
}
var configMap apiv1.ConfigMap
err = json.Unmarshal(body, &configMap)
if err != nil {
return nil, err
}
return &configMap, nil
}
func (c *Client) GetSvcURL(label string) (string, error) {
url := fmt.Sprintf("%s/proxy/svcname?"+label, c.Url)
resp, err := http.Get(url)
if err != nil {
return "", err
}
if resp == nil {
return "", fmt.Errorf("Failed to find service for given label: %v", label)
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return "", err
}
storageSvc := string(body)
return storageSvc, err
}
+14 -2
View File
@@ -25,9 +25,15 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission/crd"
fv1 "github.com/fission/fission/pkg/apis/fission.io/v1"
)
func (c *Client) EnvironmentCreate(env *crd.Environment) (*metav1.ObjectMeta, error) {
err := env.Validate()
if err != nil {
return nil, fv1.AggregateValidationErrors("Environment", err)
}
reqbody, err := json.Marshal(env)
if err != nil {
return nil, err
@@ -78,6 +84,11 @@ func (c *Client) EnvironmentGet(m *metav1.ObjectMeta) (*crd.Environment, error)
}
func (c *Client) EnvironmentUpdate(env *crd.Environment) (*metav1.ObjectMeta, error) {
err := env.Validate()
if err != nil {
return nil, fv1.AggregateValidationErrors("Environment", err)
}
reqbody, err := json.Marshal(env)
if err != nil {
return nil, err
@@ -110,8 +121,9 @@ func (c *Client) EnvironmentDelete(m *metav1.ObjectMeta) error {
return c.delete(relativeUrl)
}
func (c *Client) EnvironmentList() ([]crd.Environment, error) {
resp, err := http.Get(c.url("environments"))
func (c *Client) EnvironmentList(ns string) ([]crd.Environment, error) {
relativeUrl := fmt.Sprintf("environments?namespace=%v", ns)
resp, err := http.Get(c.url(relativeUrl))
if err != nil {
return nil, err
}
+13 -2
View File
@@ -25,9 +25,14 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission/crd"
fv1 "github.com/fission/fission/pkg/apis/fission.io/v1"
)
func (c *Client) FunctionCreate(f *crd.Function) (*metav1.ObjectMeta, error) {
err := f.Validate()
if err != nil {
return nil, fv1.AggregateValidationErrors("Function", err)
}
reqbody, err := json.Marshal(f)
if err != nil {
@@ -93,6 +98,11 @@ func (c *Client) FunctionGetRawDeployment(m *metav1.ObjectMeta) ([]byte, error)
}
func (c *Client) FunctionUpdate(f *crd.Function) (*metav1.ObjectMeta, error) {
err := f.Validate()
if err != nil {
return nil, fv1.AggregateValidationErrors("Function", err)
}
reqbody, err := json.Marshal(f)
if err != nil {
return nil, err
@@ -124,8 +134,9 @@ func (c *Client) FunctionDelete(m *metav1.ObjectMeta) error {
return c.delete(relativeUrl)
}
func (c *Client) FunctionList() ([]crd.Function, error) {
resp, err := http.Get(c.url("functions"))
func (c *Client) FunctionList(functionNamespace string) ([]crd.Function, error) {
relativeUrl := fmt.Sprintf("functions?namespace=%v", functionNamespace)
resp, err := http.Get(c.url(relativeUrl))
if err != nil {
return nil, err
}
+14 -2
View File
@@ -25,9 +25,15 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission/crd"
fv1 "github.com/fission/fission/pkg/apis/fission.io/v1"
)
func (c *Client) HTTPTriggerCreate(t *crd.HTTPTrigger) (*metav1.ObjectMeta, error) {
err := t.Validate()
if err != nil {
return nil, fv1.AggregateValidationErrors("HTTPTrigger", err)
}
reqbody, err := json.Marshal(t)
if err != nil {
return nil, err
@@ -78,6 +84,11 @@ func (c *Client) HTTPTriggerGet(m *metav1.ObjectMeta) (*crd.HTTPTrigger, error)
}
func (c *Client) HTTPTriggerUpdate(t *crd.HTTPTrigger) (*metav1.ObjectMeta, error) {
err := t.Validate()
if err != nil {
return nil, fv1.AggregateValidationErrors("HTTPTrigger", err)
}
reqbody, err := json.Marshal(t)
if err != nil {
return nil, err
@@ -109,8 +120,9 @@ func (c *Client) HTTPTriggerDelete(m *metav1.ObjectMeta) error {
return c.delete(relativeUrl)
}
func (c *Client) HTTPTriggerList() ([]crd.HTTPTrigger, error) {
resp, err := http.Get(c.url("triggers/http"))
func (c *Client) HTTPTriggerList(triggerNamespace string) ([]crd.HTTPTrigger, error) {
relativeUrl := fmt.Sprintf("triggers/http?namespace=%v", triggerNamespace)
resp, err := http.Get(c.url(relativeUrl))
if err != nil {
return nil, err
}
+9 -2
View File
@@ -26,9 +26,15 @@ import (
"github.com/fission/fission"
"github.com/fission/fission/crd"
fv1 "github.com/fission/fission/pkg/apis/fission.io/v1"
)
func (c *Client) WatchCreate(w *crd.KubernetesWatchTrigger) (*metav1.ObjectMeta, error) {
err := w.Validate()
if err != nil {
return nil, fv1.AggregateValidationErrors("KubernetesWatchTrigger", err)
}
reqbody, err := json.Marshal(w)
if err != nil {
return nil, err
@@ -89,8 +95,9 @@ func (c *Client) WatchDelete(m *metav1.ObjectMeta) error {
return c.delete(relativeUrl)
}
func (c *Client) WatchList() ([]crd.KubernetesWatchTrigger, error) {
resp, err := http.Get(c.url("watches"))
func (c *Client) WatchList(ns string) ([]crd.KubernetesWatchTrigger, error) {
relativeUrl := fmt.Sprintf("watches?namespace=%v", ns)
resp, err := http.Get(c.url(relativeUrl))
if err != nil {
return nil, err
}
+13 -2
View File
@@ -25,9 +25,15 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission/crd"
fv1 "github.com/fission/fission/pkg/apis/fission.io/v1"
)
func (c *Client) MessageQueueTriggerCreate(t *crd.MessageQueueTrigger) (*metav1.ObjectMeta, error) {
err := t.Validate()
if err != nil {
return nil, fv1.AggregateValidationErrors("MessageQueueTrigger", err)
}
reqbody, err := json.Marshal(t)
if err != nil {
return nil, err
@@ -78,6 +84,11 @@ func (c *Client) MessageQueueTriggerGet(m *metav1.ObjectMeta) (*crd.MessageQueue
}
func (c *Client) MessageQueueTriggerUpdate(mqTrigger *crd.MessageQueueTrigger) (*metav1.ObjectMeta, error) {
err := mqTrigger.Validate()
if err != nil {
return nil, fv1.AggregateValidationErrors("MessageQueueTrigger", err)
}
reqbody, err := json.Marshal(mqTrigger)
if err != nil {
return nil, err
@@ -109,11 +120,11 @@ func (c *Client) MessageQueueTriggerDelete(m *metav1.ObjectMeta) error {
return c.delete(relativeUrl)
}
func (c *Client) MessageQueueTriggerList(mqType string) ([]crd.MessageQueueTrigger, error) {
func (c *Client) MessageQueueTriggerList(mqType string, ns string) ([]crd.MessageQueueTrigger, error) {
relativeUrl := "triggers/messagequeue"
if len(mqType) > 0 {
// TODO remove this, replace with field selector
relativeUrl += fmt.Sprintf("?mqtype=%v", mqType)
relativeUrl += fmt.Sprintf("?mqtype=%v&namespace=%v", mqType, ns)
}
resp, err := http.Get(c.url(relativeUrl))
+13 -2
View File
@@ -25,9 +25,14 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission/crd"
fv1 "github.com/fission/fission/pkg/apis/fission.io/v1"
)
func (c *Client) PackageCreate(f *crd.Package) (*metav1.ObjectMeta, error) {
err := f.Validate()
if err != nil {
return nil, fv1.AggregateValidationErrors("Package", err)
}
reqbody, err := json.Marshal(f)
if err != nil {
@@ -79,6 +84,11 @@ func (c *Client) PackageGet(m *metav1.ObjectMeta) (*crd.Package, error) {
}
func (c *Client) PackageUpdate(f *crd.Package) (*metav1.ObjectMeta, error) {
err := f.Validate()
if err != nil {
return nil, fv1.AggregateValidationErrors("Package", err)
}
reqbody, err := json.Marshal(f)
if err != nil {
return nil, err
@@ -110,8 +120,9 @@ func (c *Client) PackageDelete(m *metav1.ObjectMeta) error {
return c.delete(relativeUrl)
}
func (c *Client) PackageList() ([]crd.Package, error) {
resp, err := http.Get(c.url("packages"))
func (c *Client) PackageList(pkgNamespace string) ([]crd.Package, error) {
relativeUrl := fmt.Sprintf("packages?namespace=%v", pkgNamespace)
resp, err := http.Get(c.url(relativeUrl))
if err != nil {
return nil, err
}
+239
View File
@@ -0,0 +1,239 @@
/*
Copyright 2018 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
import (
"bytes"
"encoding/json"
"fmt"
"net/http"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission/crd"
fv1 "github.com/fission/fission/pkg/apis/fission.io/v1"
"github.com/fission/fission/redis/build/gen"
)
func (c *Client) RecorderCreate(r *crd.Recorder) (*metav1.ObjectMeta, error) {
err := r.Validate()
if err != nil {
return nil, fv1.AggregateValidationErrors("Recorder", err)
}
reqbody, err := json.Marshal(r)
if err != nil {
return nil, err
}
resp, err := http.Post(c.url("recorders"), "application/json", bytes.NewReader(reqbody))
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := c.handleCreateResponse(resp)
if err != nil {
return nil, err
}
var m metav1.ObjectMeta
err = json.Unmarshal(body, &m)
if err != nil {
return nil, err
}
return &m, nil
}
func (c *Client) RecorderGet(m *metav1.ObjectMeta) (*crd.Recorder, error) {
relativeUrl := fmt.Sprintf("recorders/%v", m.Name)
relativeUrl += fmt.Sprintf("?namespace=%v", m.Namespace)
resp, err := http.Get(c.url(relativeUrl))
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := c.handleResponse(resp)
if err != nil {
return nil, err
}
var r crd.Recorder
err = json.Unmarshal(body, &r)
if err != nil {
return nil, err
}
return &r, nil
}
func (c *Client) RecorderUpdate(recorder *crd.Recorder) (*metav1.ObjectMeta, error) {
err := recorder.Validate()
if err != nil {
return nil, fv1.AggregateValidationErrors("Recorder", err)
}
reqbody, err := json.Marshal(recorder)
if err != nil {
return nil, err
}
relativeUrl := fmt.Sprintf("recorders/%v", recorder.Metadata.Name)
resp, err := c.put(relativeUrl, "application/json", reqbody)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := c.handleResponse(resp)
if err != nil {
return nil, err
}
var m metav1.ObjectMeta
err = json.Unmarshal(body, &m)
if err != nil {
return nil, err
}
return &m, nil
}
func (c *Client) RecorderDelete(m *metav1.ObjectMeta) error {
relativeUrl := fmt.Sprintf("recorders/%v", m.Name)
relativeUrl += fmt.Sprintf("?namespace=%v", m.Namespace)
return c.delete(relativeUrl)
}
func (c *Client) RecorderList(ns string) ([]crd.Recorder, error) {
relativeUrl := "recorders"
resp, err := http.Get(c.url(relativeUrl))
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := c.handleResponse(resp)
if err != nil {
return nil, err
}
recorders := make([]crd.Recorder, 0)
err = json.Unmarshal(body, &recorders)
if err != nil {
return nil, err
}
return recorders, nil
}
// TODO: Move to different file?
func (c *Client) RecordsByFunction(function string) ([]*redisCache.RecordedEntry, error) {
relativeUrl := fmt.Sprintf("records/function/%v", function)
resp, err := http.Get(c.url(relativeUrl))
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := c.handleResponse(resp)
if err != nil {
return nil, err
}
records := make([]*redisCache.RecordedEntry, 0)
err = json.Unmarshal(body, &records)
if err != nil {
return nil, err
}
return records, nil
}
func (c *Client) RecordsAll() ([]*redisCache.RecordedEntry, error) {
relativeUrl := "records"
resp, err := http.Get(c.url(relativeUrl))
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := c.handleResponse(resp)
if err != nil {
return nil, err
}
records := make([]*redisCache.RecordedEntry, 0)
err = json.Unmarshal(body, &records)
if err != nil {
return nil, err
}
return records, nil
}
func (c *Client) RecordsByTrigger(trigger string) ([]*redisCache.RecordedEntry, error) {
relativeUrl := fmt.Sprintf("records/trigger/%v", trigger)
resp, err := http.Get(c.url(relativeUrl))
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := c.handleResponse(resp)
if err != nil {
return nil, err
}
records := make([]*redisCache.RecordedEntry, 0)
err = json.Unmarshal(body, &records)
if err != nil {
return nil, err
}
return records, nil
}
func (c *Client) RecordsByTime(from string, to string) ([]*redisCache.RecordedEntry, error) {
relativeUrl := "records/time"
relativeUrl += fmt.Sprintf("?from=%v&to=%v", from, to)
resp, err := http.Get(c.url(relativeUrl))
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := c.handleResponse(resp)
if err != nil {
return nil, err
}
records := make([]*redisCache.RecordedEntry, 0)
err = json.Unmarshal(body, &records)
if err != nil {
return nil, err
}
return records, nil
}
+47
View File
@@ -0,0 +1,47 @@
/*
Copyright 2018 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
import (
"encoding/json"
"fmt"
"net/http"
)
func (c *Client) ReplayByReqUID(reqUID string) ([]string, error) {
relativeUrl := fmt.Sprintf("replay/%v", reqUID)
resp, err := http.Get(c.url(relativeUrl))
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := c.handleResponse(resp)
if err != nil {
return nil, err
}
replayed := make([]string, 0)
err = json.Unmarshal(body, &replayed)
if err != nil {
return nil, err
}
return replayed, nil
}
+14 -2
View File
@@ -25,9 +25,15 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission/crd"
fv1 "github.com/fission/fission/pkg/apis/fission.io/v1"
)
func (c *Client) TimeTriggerCreate(t *crd.TimeTrigger) (*metav1.ObjectMeta, error) {
err := t.Validate()
if err != nil {
return nil, fv1.AggregateValidationErrors("TimeTrigger", err)
}
reqbody, err := json.Marshal(t)
if err != nil {
return nil, err
@@ -78,6 +84,11 @@ func (c *Client) TimeTriggerGet(m *metav1.ObjectMeta) (*crd.TimeTrigger, error)
}
func (c *Client) TimeTriggerUpdate(t *crd.TimeTrigger) (*metav1.ObjectMeta, error) {
err := t.Validate()
if err != nil {
return nil, fv1.AggregateValidationErrors("TimeTrigger", err)
}
reqbody, err := json.Marshal(t)
if err != nil {
return nil, err
@@ -109,8 +120,9 @@ func (c *Client) TimeTriggerDelete(m *metav1.ObjectMeta) error {
return c.delete(relativeUrl)
}
func (c *Client) TimeTriggerList() ([]crd.TimeTrigger, error) {
resp, err := http.Get(c.url("triggers/time"))
func (c *Client) TimeTriggerList(ns string) ([]crd.TimeTrigger, error) {
relativeUrl := fmt.Sprintf("triggers/time?namespace=%v", ns)
resp, err := http.Get(c.url(relativeUrl))
if err != nil {
return nil, err
}
+49
View File
@@ -0,0 +1,49 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package controller
import (
"encoding/json"
"net/http"
"github.com/gorilla/mux"
log "github.com/sirupsen/logrus"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func (a *API) ConfigMapGet(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["configmap"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
configMap, err := a.kubernetesClient.CoreV1().ConfigMaps(ns).Get(name, metav1.GetOptions{})
if err != nil {
log.Printf("Error getting config map: %s from ns: %s", name, ns)
a.respondWithError(w, err)
return
}
resp, err := json.Marshal(configMap)
if err != nil {
a.respondWithError(w, err)
return
}
a.respondWithSuccess(w, resp)
}
+4 -1
View File
@@ -37,7 +37,10 @@ func Start(port int) {
log.Fatalf("Failed to create fission CRDs: %v", err)
}
fc.WaitForCRDs()
err = fc.WaitForCRDs()
if err != nil {
log.Fatalf("Error waiting for CRDs: %v", err)
}
api, err := MakeAPI()
if err != nil {
-11
View File
@@ -17,9 +17,6 @@ limitations under the License.
package controller
import (
"errors"
"regexp"
"github.com/fission/fission/crd"
)
@@ -30,11 +27,3 @@ func makeCRDBackedAPI() (*API, error) {
}
return &API{fissionClient: fissionClient, kubernetesClient: kubernetesClient}, nil
}
func validateResourceName(name string) error {
re := regexp.MustCompile(`[a-z0-9]([-a-z0-9]*[a-z0-9])?`)
if len(re.FindString(name)) != len(name) {
return errors.New("Name must consist of lower case alphanumeric characters or '-', and must start and end with an alphanumeric character (e.g. 'my-name', or '123-abc', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?'")
}
return nil
}
+12 -4
View File
@@ -30,7 +30,12 @@ import (
)
func (a *API) EnvironmentApiList(w http.ResponseWriter, r *http.Request) {
envs, err := a.fissionClient.Environments(metav1.NamespaceAll).List(metav1.ListOptions{})
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceAll
}
envs, err := a.fissionClient.Environments(ns).List(metav1.ListOptions{})
if err != nil {
a.respondWithError(w, err)
return
@@ -60,7 +65,8 @@ func (a *API) EnvironmentApiCreate(w http.ResponseWriter, r *http.Request) {
return
}
err = validateResourceName(env.Metadata.Name)
// check if namespace exists, if not create it.
err = a.createNsIfNotExists(env.Metadata.Namespace)
if err != nil {
a.respondWithError(w, err)
return
@@ -85,7 +91,8 @@ func (a *API) EnvironmentApiCreate(w http.ResponseWriter, r *http.Request) {
func (a *API) EnvironmentApiGet(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["environment"]
ns := vars["namespace"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
@@ -146,7 +153,8 @@ func (a *API) EnvironmentApiUpdate(w http.ResponseWriter, r *http.Request) {
func (a *API) EnvironmentApiDelete(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["environment"]
ns := vars["namespace"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
+12 -79
View File
@@ -28,11 +28,8 @@ import (
"github.com/gorilla/mux"
log "github.com/sirupsen/logrus"
apiv1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/labels"
"k8s.io/apimachinery/pkg/util/intstr"
"k8s.io/client-go/pkg/api"
apiv1 "k8s.io/client-go/pkg/api/v1"
restclient "k8s.io/client-go/rest"
"github.com/fission/fission"
@@ -46,7 +43,12 @@ func (a *API) getIstioServiceLabels(fnName string) map[string]string {
}
func (a *API) FunctionApiList(w http.ResponseWriter, r *http.Request) {
funcs, err := a.fissionClient.Functions(metav1.NamespaceAll).List(metav1.ListOptions{})
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceAll
}
funcs, err := a.fissionClient.Functions(ns).List(metav1.ListOptions{})
if err != nil {
a.respondWithError(w, err)
return
@@ -75,7 +77,8 @@ func (a *API) FunctionApiCreate(w http.ResponseWriter, r *http.Request) {
return
}
err = validateResourceName(f.Metadata.Name)
// check if namespace exists, if not create it.
err = a.createNsIfNotExists(f.Metadata.Namespace)
if err != nil {
a.respondWithError(w, err)
return
@@ -93,62 +96,6 @@ func (a *API) FunctionApiCreate(w http.ResponseWriter, r *http.Request) {
return
}
// Since istio only allows accessing pod through k8s service,
// for the functions with executor type "poolmgr" we need to
// create a service for sending requests to pod in pool.
// Functions with executor type "Newdeploy" is specialized at
// pod starts. In this case, just ignore such functions.
fnExecutorType := f.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType
if a.useIstio && fnExecutorType == fission.ExecutorTypePoolmgr {
// create a same name service for function
// since istio only allows the traffic to service
sel := map[string]string{
"functionName": fnew.Metadata.Name,
"functionUid": string(fnew.Metadata.UID),
}
// service for accepting user traffic
svc := apiv1.Service{
ObjectMeta: metav1.ObjectMeta{
Namespace: a.functionNamespace,
Name: fission.GetFunctionIstioServiceName(f.Metadata.Name, f.Metadata.Namespace),
Labels: a.getIstioServiceLabels(f.Metadata.Name),
},
Spec: apiv1.ServiceSpec{
Type: apiv1.ServiceTypeClusterIP,
Ports: []apiv1.ServicePort{
// Service port name should begin with a recognized prefix, or the traffic will be
// treated as TCP traffic. (https://istio.io/docs/setup/kubernetes/sidecar-injection.html)
// Originally the ports' name are similar to "http-fetch" and "http-specialize".
// But for istio 0.5.1, istio-proxy return unexpected 431 error with such naming.
// https://github.com/istio/istio/issues/928
// Workaround: remove prefix
// TODO: prepend prefix once the bug fixed
{
Name: "fetch",
Protocol: apiv1.ProtocolTCP,
Port: 8000,
TargetPort: intstr.FromInt(8000),
},
{
Name: "specialize",
Protocol: apiv1.ProtocolTCP,
Port: 8888,
TargetPort: intstr.FromInt(8888),
},
},
Selector: sel,
},
}
_, err = a.kubernetesClient.CoreV1().Services(a.functionNamespace).Create(&svc)
if err != nil {
a.respondWithError(w, err)
return
}
}
w.WriteHeader(http.StatusCreated)
a.respondWithSuccess(w, resp)
}
@@ -156,7 +103,7 @@ func (a *API) FunctionApiCreate(w http.ResponseWriter, r *http.Request) {
func (a *API) FunctionApiGet(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["function"]
ns := vars["namespace"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
@@ -215,7 +162,7 @@ func (a *API) FunctionApiUpdate(w http.ResponseWriter, r *http.Request) {
func (a *API) FunctionApiDelete(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["function"]
ns := vars["namespace"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
@@ -226,20 +173,6 @@ func (a *API) FunctionApiDelete(w http.ResponseWriter, r *http.Request) {
return
}
if a.useIstio {
// delete all istio services belong to the function
sel := a.getIstioServiceLabels(name)
svcList, err := a.kubernetesClient.CoreV1().Services(a.functionNamespace).List(metav1.ListOptions{
LabelSelector: labels.Set(sel).AsSelector().String(),
})
for _, svc := range svcList.Items {
err = a.kubernetesClient.CoreV1().Services(a.functionNamespace).Delete(svc.ObjectMeta.Name, &metav1.DeleteOptions{})
// log error and continue
log.Printf("Failed to delete service %v: %v", svc.ObjectMeta.Name, err)
continue
}
}
a.respondWithSuccess(w, []byte(""))
}
@@ -283,7 +216,7 @@ func (a *API) FunctionPodLogs(w http.ResponseWriter, r *http.Request) {
ns = "fission-function"
}
f, err := a.fissionClient.Functions(api.NamespaceDefault).Get(fnName)
f, err := a.fissionClient.Functions(ns).Get(fnName)
if err != nil {
a.respondWithError(w, err)
return
+12 -6
View File
@@ -30,7 +30,12 @@ import (
)
func (a *API) HTTPTriggerApiList(w http.ResponseWriter, r *http.Request) {
triggers, err := a.fissionClient.HTTPTriggers(metav1.NamespaceAll).List(metav1.ListOptions{})
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceAll
}
triggers, err := a.fissionClient.HTTPTriggers(ns).List(metav1.ListOptions{})
if err != nil {
a.respondWithError(w, err)
return
@@ -74,14 +79,15 @@ func (a *API) HTTPTriggerApiCreate(w http.ResponseWriter, r *http.Request) {
return
}
err = validateResourceName(t.Metadata.Name)
// Ensure we don't have a duplicate HTTP route defined (same URL and method)
err = a.checkHTTPTriggerDuplicates(&t)
if err != nil {
a.respondWithError(w, err)
return
}
// Ensure we don't have a duplicate HTTP route defined (same URL and method)
err = a.checkHTTPTriggerDuplicates(&t)
// check if namespace exists, if not create it.
err = a.createNsIfNotExists(t.Metadata.Namespace)
if err != nil {
a.respondWithError(w, err)
return
@@ -106,7 +112,7 @@ func (a *API) HTTPTriggerApiCreate(w http.ResponseWriter, r *http.Request) {
func (a *API) HTTPTriggerApiGet(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["httpTrigger"]
ns := vars["namespace"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
@@ -172,7 +178,7 @@ func (a *API) HTTPTriggerApiUpdate(w http.ResponseWriter, r *http.Request) {
func (a *API) HTTPTriggerApiDelete(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["httpTrigger"]
ns := vars["namespace"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
+10 -4
View File
@@ -30,7 +30,12 @@ import (
func (a *API) MessageQueueTriggerApiList(w http.ResponseWriter, r *http.Request) {
//mqType := r.FormValue("mqtype") // ignored for now
triggers, err := a.fissionClient.MessageQueueTriggers(metav1.NamespaceAll).List(metav1.ListOptions{})
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceAll
}
triggers, err := a.fissionClient.MessageQueueTriggers(ns).List(metav1.ListOptions{})
if err != nil {
a.respondWithError(w, err)
return
@@ -57,7 +62,8 @@ func (a *API) MessageQueueTriggerApiCreate(w http.ResponseWriter, r *http.Reques
return
}
err = validateResourceName(mqTrigger.Metadata.Name)
// check if namespace exists, if not create it.
err = a.createNsIfNotExists(mqTrigger.Metadata.Namespace)
if err != nil {
a.respondWithError(w, err)
return
@@ -81,7 +87,7 @@ func (a *API) MessageQueueTriggerApiCreate(w http.ResponseWriter, r *http.Reques
func (a *API) MessageQueueTriggerApiGet(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["mqTrigger"]
ns := vars["namespace"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
@@ -139,7 +145,7 @@ func (a *API) MessageQueueTriggerApiUpdate(w http.ResponseWriter, r *http.Reques
func (a *API) MessageQueueTriggerApiDelete(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["mqTrigger"]
ns := vars["namespace"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
+22 -13
View File
@@ -18,9 +18,11 @@ package controller
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"github.com/dustin/go-humanize"
"github.com/gorilla/mux"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -29,7 +31,11 @@ import (
)
func (a *API) PackageApiList(w http.ResponseWriter, r *http.Request) {
funcs, err := a.fissionClient.Packages(metav1.NamespaceAll).List(metav1.ListOptions{})
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceAll
}
funcs, err := a.fissionClient.Packages(ns).List(metav1.ListOptions{})
if err != nil {
a.respondWithError(w, err)
return
@@ -58,20 +64,23 @@ func (a *API) PackageApiCreate(w http.ResponseWriter, r *http.Request) {
return
}
err = validateResourceName(f.Metadata.Name)
if err != nil {
// Ensure size limits
if len(f.Spec.Source.Literal) > int(fission.ArchiveLiteralSizeLimit) {
err := fission.MakeError(fission.ErrorInvalidArgument,
fmt.Sprintf("Package literal larger than %s", humanize.Bytes(uint64(fission.ArchiveLiteralSizeLimit))))
a.respondWithError(w, err)
return
}
if len(f.Spec.Deployment.Literal) > int(fission.ArchiveLiteralSizeLimit) {
err := fission.MakeError(fission.ErrorInvalidArgument,
fmt.Sprintf("Package literal larger than %s", humanize.Bytes(uint64(fission.ArchiveLiteralSizeLimit))))
a.respondWithError(w, err)
return
}
// Ensure size limits
if len(f.Spec.Source.Literal) > 256*1024 {
err := fission.MakeError(fission.ErrorInvalidArgument, "Package literal larger than 256K")
a.respondWithError(w, err)
return
}
if len(f.Spec.Deployment.Literal) > 256*1024 {
err := fission.MakeError(fission.ErrorInvalidArgument, "Package literal larger than 256K")
// check if namespace exists, if not create it.
err = a.createNsIfNotExists(f.Metadata.Namespace)
if err != nil {
a.respondWithError(w, err)
return
}
@@ -95,7 +104,7 @@ func (a *API) PackageApiCreate(w http.ResponseWriter, r *http.Request) {
func (a *API) PackageApiGet(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["package"]
ns := vars["namespace"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
@@ -160,7 +169,7 @@ func (a *API) PackageApiUpdate(w http.ResponseWriter, r *http.Request) {
func (a *API) PackageApiDelete(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["package"]
ns := vars["namespace"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
+148
View File
@@ -0,0 +1,148 @@
/*
Copyright 2018 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package controller
import (
"encoding/json"
"io/ioutil"
"net/http"
"github.com/gorilla/mux"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission"
"github.com/fission/fission/crd"
)
func (a *API) RecorderApiList(w http.ResponseWriter, r *http.Request) {
recorders, err := a.fissionClient.Recorders(metav1.NamespaceAll).List(metav1.ListOptions{})
if err != nil {
a.respondWithError(w, err)
return
}
resp, err := json.Marshal(recorders.Items)
if err != nil {
a.respondWithError(w, err)
return
}
a.respondWithSuccess(w, resp)
}
func (a *API) RecorderApiCreate(w http.ResponseWriter, r *http.Request) {
body, err := ioutil.ReadAll(r.Body)
if err != nil {
a.respondWithError(w, err)
return
}
var recorder crd.Recorder
err = json.Unmarshal(body, &recorder)
if err != nil {
a.respondWithError(w, err)
return
}
tnew, err := a.fissionClient.Recorders(recorder.Metadata.Namespace).Create(&recorder)
if err != nil {
a.respondWithError(w, err)
return
}
resp, err := json.Marshal(tnew.Metadata)
if err != nil {
a.respondWithError(w, err)
return
}
w.WriteHeader(http.StatusCreated)
a.respondWithSuccess(w, resp)
}
func (a *API) RecorderApiGet(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["recorder"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
recorder, err := a.fissionClient.Recorders(ns).Get(name)
if err != nil {
a.respondWithError(w, err)
return
}
resp, err := json.Marshal(recorder)
if err != nil {
a.respondWithError(w, err)
return
}
a.respondWithSuccess(w, resp)
}
func (a *API) RecorderApiUpdate(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["recorder"]
body, err := ioutil.ReadAll(r.Body)
if err != nil {
a.respondWithError(w, err)
return
}
var recorder crd.Recorder
err = json.Unmarshal(body, &recorder)
if err != nil {
a.respondWithError(w, err)
return
}
if name != recorder.Metadata.Name {
err = fission.MakeError(fission.ErrorInvalidArgument, "Recorder name doesn't match URL")
a.respondWithError(w, err)
return
}
rnew, err := a.fissionClient.Recorders(recorder.Metadata.Namespace).Update(&recorder)
if err != nil {
a.respondWithError(w, err)
return
}
resp, err := json.Marshal(rnew.Metadata)
if err != nil {
a.respondWithError(w, err)
return
}
a.respondWithSuccess(w, resp)
}
func (a *API) RecorderApiDelete(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["recorder"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
err := a.fissionClient.Recorders(ns).Delete(name, &metav1.DeleteOptions{})
if err != nil {
a.respondWithError(w, err)
return
}
a.respondWithSuccess(w, []byte(""))
}
+95
View File
@@ -0,0 +1,95 @@
/*
Copyright 2018 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package controller
import (
"net/http"
"github.com/gorilla/mux"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission/redis"
)
func (a *API) RecordsApiListAll(w http.ResponseWriter, r *http.Request) {
resp, err := redis.RecordsListAll()
if err != nil {
a.respondWithError(w, err)
return
}
a.respondWithSuccess(w, resp)
}
func (a *API) RecordsApiFilterByFunction(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
query := vars["function"]
recorders, err := a.fissionClient.Recorders(metav1.NamespaceAll).List(metav1.ListOptions{})
if err != nil {
a.respondWithError(w, err)
return
}
triggers, err := a.fissionClient.HTTPTriggers(metav1.NamespaceAll).List(metav1.ListOptions{})
if err != nil {
a.respondWithError(w, err)
return
}
resp, err := redis.RecordsFilterByFunction(query, recorders, triggers)
if err != nil {
a.respondWithError(w, err)
return
}
a.respondWithSuccess(w, resp)
}
func (a *API) RecordsApiFilterByTrigger(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
query := vars["trigger"]
recorders, err := a.fissionClient.Recorders(metav1.NamespaceAll).List(metav1.ListOptions{})
if err != nil {
a.respondWithError(w, err)
return
}
triggers, err := a.fissionClient.HTTPTriggers(metav1.NamespaceAll).List(metav1.ListOptions{})
if err != nil {
a.respondWithError(w, err)
return
}
resp, err := redis.RecordsFilterByTrigger(query, recorders, triggers)
if err != nil {
a.respondWithError(w, err)
return
}
a.respondWithSuccess(w, resp)
}
func (a *API) RecordsApiFilterByTime(w http.ResponseWriter, r *http.Request) {
from := r.FormValue("from")
to := r.FormValue("to")
resp, err := redis.RecordsFilterByTime(from, to)
if err != nil {
a.respondWithError(w, err)
return
}
a.respondWithSuccess(w, resp)
}
+40
View File
@@ -0,0 +1,40 @@
/*
Copyright 2018 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package controller
import (
"fmt"
"net/http"
"github.com/gorilla/mux"
"github.com/fission/fission/redis"
)
func (a *API) ReplayByReqUID(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
queriedID := vars["reqUID"]
routerUrl := fmt.Sprintf("http://router.%v", podNamespace)
resp, err := redis.ReplayByReqUID(routerUrl, queriedID)
if err != nil {
a.respondWithError(w, err)
return
}
a.respondWithSuccess(w, resp)
}
+49
View File
@@ -0,0 +1,49 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package controller
import (
"encoding/json"
"net/http"
"github.com/gorilla/mux"
log "github.com/sirupsen/logrus"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func (a *API) SecretGet(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["secret"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
secret, err := a.kubernetesClient.CoreV1().Secrets(ns).Get(name, metav1.GetOptions{})
if err != nil {
log.Printf("Error getting secret: %s from ns: %s", name, ns)
a.respondWithError(w, err)
return
}
resp, err := json.Marshal(secret)
if err != nil {
a.respondWithError(w, err)
return
}
a.respondWithSuccess(w, resp)
}
+1 -1
View File
@@ -30,7 +30,7 @@ func (api *API) StorageServiceProxy(w http.ResponseWriter, r *http.Request) {
if err != nil {
msg := fmt.Sprintf("Error parsing url %v: %v", u, err)
log.Println(msg)
http.Error(w, msg, 500)
http.Error(w, msg, http.StatusInternalServerError)
return
}
director := func(req *http.Request) {
+15 -9
View File
@@ -30,7 +30,12 @@ import (
)
func (a *API) TimeTriggerApiList(w http.ResponseWriter, r *http.Request) {
triggers, err := a.fissionClient.TimeTriggers(metav1.NamespaceAll).List(metav1.ListOptions{})
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceAll
}
triggers, err := a.fissionClient.TimeTriggers(ns).List(metav1.ListOptions{})
if err != nil {
a.respondWithError(w, err)
return
@@ -59,12 +64,6 @@ func (a *API) TimeTriggerApiCreate(w http.ResponseWriter, r *http.Request) {
return
}
err = validateResourceName(t.Metadata.Name)
if err != nil {
a.respondWithError(w, err)
return
}
// validate
_, err = cron.Parse(t.Spec.Cron)
if err != nil {
@@ -73,6 +72,13 @@ func (a *API) TimeTriggerApiCreate(w http.ResponseWriter, r *http.Request) {
return
}
// check if namespace exists, if not create it.
err = a.createNsIfNotExists(t.Metadata.Namespace)
if err != nil {
a.respondWithError(w, err)
return
}
tnew, err := a.fissionClient.TimeTriggers(t.Metadata.Namespace).Create(&t)
if err != nil {
a.respondWithError(w, err)
@@ -92,7 +98,7 @@ func (a *API) TimeTriggerApiCreate(w http.ResponseWriter, r *http.Request) {
func (a *API) TimeTriggerApiGet(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["timeTrigger"]
ns := vars["namespace"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
@@ -159,7 +165,7 @@ func (a *API) TimeTriggerApiUpdate(w http.ResponseWriter, r *http.Request) {
func (a *API) TimeTriggerApiDelete(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["timeTrigger"]
ns := vars["namespace"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
-64
View File
@@ -1,64 +0,0 @@
/*
Copyright 2017 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package controller
import (
"net/http"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission/crd"
)
func (a *API) Tpr2crdApi(w http.ResponseWriter, r *http.Request) {
_, kubeClient, _, err := crd.MakeFissionClient()
if err != nil {
a.respondWithError(w, err)
return
}
fissionTprs := []string{
"function.fission.io",
"environment.fission.io",
"httptrigger.fission.io",
"kuberneteswatchtrigger.fission.io",
"timetrigger.fission.io",
"messagequeuetrigger.fission.io",
"package.fission.io",
}
tprList, err := kubeClient.ThirdPartyResources().List(metav1.ListOptions{})
if err != nil {
a.respondWithError(w, err)
return
}
for _, tpr := range tprList.Items {
for _, tprName := range fissionTprs {
if tpr.Name == tprName {
err := kubeClient.ThirdPartyResources().Delete(tpr.Name, &metav1.DeleteOptions{})
if err != nil {
a.respondWithError(w, err)
return
}
break
}
}
}
a.respondWithSuccess(w, nil)
}
+12 -6
View File
@@ -29,7 +29,12 @@ import (
)
func (a *API) WatchApiList(w http.ResponseWriter, r *http.Request) {
watches, err := a.fissionClient.KubernetesWatchTriggers(metav1.NamespaceAll).List(metav1.ListOptions{})
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceAll
}
watches, err := a.fissionClient.KubernetesWatchTriggers(ns).List(metav1.ListOptions{})
if err != nil {
a.respondWithError(w, err)
return
@@ -58,14 +63,15 @@ func (a *API) WatchApiCreate(w http.ResponseWriter, r *http.Request) {
return
}
err = validateResourceName(watch.Metadata.Name)
// TODO check for duplicate watches
// TODO check for duplicate watches -> we probably wont need it?
// check if namespace exists, if not create it.
err = a.createNsIfNotExists(watch.Metadata.Namespace)
if err != nil {
a.respondWithError(w, err)
return
}
// TODO check for duplicate watches
wnew, err := a.fissionClient.KubernetesWatchTriggers(watch.Metadata.Namespace).Create(&watch)
if err != nil {
a.respondWithError(w, err)
@@ -85,7 +91,7 @@ func (a *API) WatchApiCreate(w http.ResponseWriter, r *http.Request) {
func (a *API) WatchApiGet(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["watch"]
ns := vars["namespace"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
@@ -113,7 +119,7 @@ func (a *API) WatchApiUpdate(w http.ResponseWriter, r *http.Request) {
func (a *API) WatchApiDelete(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
name := vars["watch"]
ns := vars["namespace"]
ns := a.extractQueryParamFromRequest(r, "namespace")
if len(ns) == 0 {
ns = metav1.NamespaceDefault
}
+1 -1
View File
@@ -13,7 +13,7 @@ func (api *API) WorkflowApiserverProxy(w http.ResponseWriter, r *http.Request) {
ssUrl, err := url.Parse(u)
if err != nil {
msg := fmt.Sprintf("Error parsing url %v: %v", u, err)
http.Error(w, msg, 500)
http.Error(w, msg, http.StatusInternalServerError)
return
}
+12 -3
View File
@@ -153,6 +153,13 @@ func configureClient(config *rest.Config) {
&metav1.ListOptions{},
&metav1.DeleteOptions{},
)
scheme.AddKnownTypes(
groupversion,
&Recorder{},
&RecorderList{},
&metav1.ListOptions{},
&metav1.DeleteOptions{},
)
return nil
})
schemeBuilder.AddToScheme(scheme.Scheme)
@@ -208,12 +215,14 @@ func (fc *FissionClient) TimeTriggers(ns string) TimeTriggerInterface {
func (fc *FissionClient) MessageQueueTriggers(ns string) MessageQueueTriggerInterface {
return MakeMessageQueueTriggerInterface(fc.crdClient, ns)
}
func (fc *FissionClient) Recorders(ns string) RecorderInterface {
return MakeRecorderInterface(fc.crdClient, ns)
}
func (fc *FissionClient) Packages(ns string) PackageInterface {
return MakePackageInterface(fc.crdClient, ns)
}
func (fc *FissionClient) WaitForCRDs() {
waitForCRDs(fc.crdClient)
func (fc *FissionClient) WaitForCRDs() error {
return waitForCRDs(fc.crdClient)
}
func (fc *FissionClient) GetCrdClient() *rest.RESTClient {
return fc.crdClient
+36 -20
View File
@@ -34,31 +34,31 @@ const (
// ensureCRD checks if the given CRD type exists, and creates it if
// needed. (Note that this creates the CRD type; it doesn't create any
// _instances_ of that type.)
func ensureCRD(clientset *apiextensionsclient.Clientset, crd *apiextensionsv1beta1.CustomResourceDefinition) error {
func ensureCRD(clientset *apiextensionsclient.Clientset, crd *apiextensionsv1beta1.CustomResourceDefinition) (err error) {
maxRetries := 5
for i := 0; i < maxRetries; i++ {
_, err := clientset.ApiextensionsV1beta1().CustomResourceDefinitions().Get(crd.ObjectMeta.Name, metav1.GetOptions{})
if err != nil {
if errors.IsNotFound(err) {
// crd resource not found error
_, err := clientset.ApiextensionsV1beta1().CustomResourceDefinitions().Create(crd)
if err != nil {
return err
}
} else {
// The requests fail to connect to k8s api server before
// istio-prxoy is ready to serve traffic. Retry again.
log.Printf("Error connecting to kubernetes api service (%v), retrying", err)
time.Sleep(500 * time.Duration(2*i) * time.Millisecond)
continue
}
for i := 0; i < maxRetries; i++ {
_, err = clientset.ApiextensionsV1beta1().CustomResourceDefinitions().Get(crd.ObjectMeta.Name, metav1.GetOptions{})
if err == nil {
return nil
}
// resource already exists
break
if errors.IsNotFound(err) {
// crd resource not found error
_, err = clientset.ApiextensionsV1beta1().CustomResourceDefinitions().Create(crd)
if err != nil {
return err
}
} else {
// The requests fail to connect to k8s api server before
// istio-prxoy is ready to serve traffic. Retry again.
log.Printf("Error connecting to kubernetes api service (%v), retrying", err)
time.Sleep(500 * time.Duration(2*i) * time.Millisecond)
continue
}
}
return nil
return err
}
// Ensure CRDs
@@ -160,6 +160,22 @@ func EnsureFissionCRDs(clientset *apiextensionsclient.Clientset) error {
},
},
},
// Recorders
{
ObjectMeta: metav1.ObjectMeta{
Name: "recorders.fission.io",
},
Spec: apiextensionsv1beta1.CustomResourceDefinitionSpec{
Group: crdGroupName,
Version: crdVersion,
Scope: apiextensionsv1beta1.NamespaceScoped,
Names: apiextensionsv1beta1.CustomResourceDefinitionNames{
Kind: "Recorder",
Plural: "recorders",
Singular: "recorder",
},
},
},
// Packages: archives containing source or binaries for one or more functions
{
ObjectMeta: metav1.ObjectMeta{
+121
View File
@@ -0,0 +1,121 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package crd
import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/watch"
"k8s.io/client-go/kubernetes/scheme"
"k8s.io/client-go/rest"
)
type (
RecorderInterface interface {
Create(*Recorder) (*Recorder, error)
Get(name string) (*Recorder, error)
Update(*Recorder) (*Recorder, error)
Delete(name string, opts *metav1.DeleteOptions) error
List(opts metav1.ListOptions) (*RecorderList, error)
Watch(opts metav1.ListOptions) (watch.Interface, error)
}
recorderClient struct {
client *rest.RESTClient
namespace string
}
)
func MakeRecorderInterface(crdClient *rest.RESTClient, namespace string) RecorderInterface {
return &recorderClient{
client: crdClient,
namespace: namespace,
}
}
func (rc *recorderClient) Create(r *Recorder) (*Recorder, error) {
var result Recorder
err := rc.client.Post().
Resource("recorders").
Namespace("default").
Body(r).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (rc *recorderClient) Get(name string) (*Recorder, error) {
var result Recorder
err := rc.client.Get().
Resource("recorders").
Namespace(rc.namespace).
Name(name).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (rc *recorderClient) Update(r *Recorder) (*Recorder, error) {
var result Recorder
err := rc.client.Put().
Resource("recorders").
Namespace(rc.namespace).
Name(r.Metadata.Name).
Body(r).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (rc *recorderClient) Delete(name string, opts *metav1.DeleteOptions) error {
return rc.client.Delete().
Namespace(rc.namespace).
Resource("recorders").
Name(name).
Body(opts).
Do().
Error()
}
func (rc *recorderClient) List(opts metav1.ListOptions) (*RecorderList, error) {
var result RecorderList
err := rc.client.Get().
Namespace(rc.namespace).
Resource("recorders").
VersionedParams(&opts, scheme.ParameterCodec).
Do().
Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (rc *recorderClient) Watch(opts metav1.ListOptions) (watch.Interface, error) {
return rc.client.Get().
Prefix("watch").
Namespace(rc.namespace).
Resource("recorders").
VersionedParams(&opts, scheme.ParameterCodec).
Watch()
}
+17 -205
View File
@@ -17,212 +17,24 @@ limitations under the License.
package crd
import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime/schema"
"github.com/fission/fission"
fv1 "github.com/fission/fission/pkg/apis/fission.io/v1"
)
//
// To add a Fission CRD type:
// 1. Create a "spec" type, for everything in the type except metadata
// 2. Create the type with metadata + the spec
// 3. Create a list type (for example see FunctionList and Function, below)
// 4. Add methods at the bottom of this file for satisfying Object and List interfaces
// 5. Add the type to configureClient in client.go
// 6. Add the type to EnsureFissionCRDs in crd.go
// 7. Add tests to crd_test.go
// 8. Add a CRUD Interface type (analogous to FunctionInterface in function.go)
// 9. Add a getter method for your interface type to FissionClient in client.go
//
type (
// Packages. Think of these as function-level images.
Package struct {
metav1.TypeMeta `json:",inline"`
Metadata metav1.ObjectMeta `json:"metadata"`
Spec fission.PackageSpec `json:"spec"`
Status fission.PackageStatus `json:"status"`
}
PackageList struct {
metav1.TypeMeta `json:",inline"`
Metadata metav1.ListMeta `json:"metadata"`
Items []Package `json:"items"`
}
// Functions.
Function struct {
metav1.TypeMeta `json:",inline"`
Metadata metav1.ObjectMeta `json:"metadata"`
Spec fission.FunctionSpec `json:"spec"`
}
FunctionList struct {
metav1.TypeMeta `json:",inline"`
Metadata metav1.ListMeta `json:"metadata"`
Items []Function `json:"items"`
}
// Environments.
Environment struct {
metav1.TypeMeta `json:",inline"`
Metadata metav1.ObjectMeta `json:"metadata"`
Spec fission.EnvironmentSpec `json:"spec"`
}
EnvironmentList struct {
metav1.TypeMeta `json:",inline"`
Metadata metav1.ListMeta `json:"metadata"`
Items []Environment `json:"items"`
}
HTTPTrigger struct {
metav1.TypeMeta `json:",inline"`
Metadata metav1.ObjectMeta `json:"metadata"`
Spec fission.HTTPTriggerSpec `json:"spec"`
}
HTTPTriggerList struct {
metav1.TypeMeta `json:",inline"`
Metadata metav1.ListMeta `json:"metadata"`
Items []HTTPTrigger `json:"items"`
}
// Kubernetes Watches as triggers
KubernetesWatchTrigger struct {
metav1.TypeMeta `json:",inline"`
Metadata metav1.ObjectMeta `json:"metadata"`
Spec fission.KubernetesWatchTriggerSpec `json:"spec"`
}
KubernetesWatchTriggerList struct {
metav1.TypeMeta `json:",inline"`
Metadata metav1.ListMeta `json:"metadata"`
Items []KubernetesWatchTrigger `json:"items"`
}
// Time triggers
TimeTrigger struct {
metav1.TypeMeta `json:",inline"`
Metadata metav1.ObjectMeta `json:"metadata"`
Spec fission.TimeTriggerSpec `json:"spec"`
}
TimeTriggerList struct {
metav1.TypeMeta `json:",inline"`
Metadata metav1.ListMeta `json:"metadata"`
Items []TimeTrigger `json:"items"`
}
// Message Queue triggers
MessageQueueTrigger struct {
metav1.TypeMeta `json:",inline"`
Metadata metav1.ObjectMeta `json:"metadata"`
Spec fission.MessageQueueTriggerSpec `json:"spec"`
}
MessageQueueTriggerList struct {
metav1.TypeMeta `json:",inline"`
Metadata metav1.ListMeta `json:"metadata"`
Items []MessageQueueTrigger `json:"items"`
}
Package = fv1.Package
PackageList = fv1.PackageList
Function = fv1.Function
FunctionList = fv1.FunctionList
Environment = fv1.Environment
EnvironmentList = fv1.EnvironmentList
HTTPTrigger = fv1.HTTPTrigger
HTTPTriggerList = fv1.HTTPTriggerList
KubernetesWatchTrigger = fv1.KubernetesWatchTrigger
KubernetesWatchTriggerList = fv1.KubernetesWatchTriggerList
TimeTrigger = fv1.TimeTrigger
TimeTriggerList = fv1.TimeTriggerList
MessageQueueTrigger = fv1.MessageQueueTrigger
MessageQueueTriggerList = fv1.MessageQueueTriggerList
Recorder = fv1.Recorder
RecorderList = fv1.RecorderList
)
// Each CRD type needs:
// GetObjectKind (to satisfy the Object interface)
//
// In addition, each singular CRD type needs:
// GetObjectMeta (to satisfy the ObjectMetaAccessor interface)
//
// And each list CRD type needs:
// GetListMeta (to satisfy the ListMetaAccessor interface)
func (f *Function) GetObjectKind() schema.ObjectKind {
return &f.TypeMeta
}
func (e *Environment) GetObjectKind() schema.ObjectKind {
return &e.TypeMeta
}
func (ht *HTTPTrigger) GetObjectKind() schema.ObjectKind {
return &ht.TypeMeta
}
func (w *KubernetesWatchTrigger) GetObjectKind() schema.ObjectKind {
return &w.TypeMeta
}
func (w *TimeTrigger) GetObjectKind() schema.ObjectKind {
return &w.TypeMeta
}
func (w *MessageQueueTrigger) GetObjectKind() schema.ObjectKind {
return &w.TypeMeta
}
func (w *Package) GetObjectKind() schema.ObjectKind {
return &w.TypeMeta
}
func (f *Function) GetObjectMeta() metav1.Object {
return &f.Metadata
}
func (e *Environment) GetObjectMeta() metav1.Object {
return &e.Metadata
}
func (ht *HTTPTrigger) GetObjectMeta() metav1.Object {
return &ht.Metadata
}
func (w *KubernetesWatchTrigger) GetObjectMeta() metav1.Object {
return &w.Metadata
}
func (w *TimeTrigger) GetObjectMeta() metav1.Object {
return &w.Metadata
}
func (w *MessageQueueTrigger) GetObjectMeta() metav1.Object {
return &w.Metadata
}
func (w *Package) GetObjectMeta() metav1.Object {
return &w.Metadata
}
func (fl *FunctionList) GetObjectKind() schema.ObjectKind {
return &fl.TypeMeta
}
func (el *EnvironmentList) GetObjectKind() schema.ObjectKind {
return &el.TypeMeta
}
func (hl *HTTPTriggerList) GetObjectKind() schema.ObjectKind {
return &hl.TypeMeta
}
func (wl *KubernetesWatchTriggerList) GetObjectKind() schema.ObjectKind {
return &wl.TypeMeta
}
func (wl *TimeTriggerList) GetObjectKind() schema.ObjectKind {
return &wl.TypeMeta
}
func (wl *MessageQueueTriggerList) GetObjectKind() schema.ObjectKind {
return &wl.TypeMeta
}
func (wl *PackageList) GetObjectKind() schema.ObjectKind {
return &wl.TypeMeta
}
func (fl *FunctionList) GetListMeta() metav1.List {
return &fl.Metadata
}
func (el *EnvironmentList) GetListMeta() metav1.List {
return &el.Metadata
}
func (hl *HTTPTriggerList) GetListMeta() metav1.List {
return &hl.Metadata
}
func (wl *KubernetesWatchTriggerList) GetListMeta() metav1.List {
return &wl.Metadata
}
func (wl *TimeTriggerList) GetListMeta() metav1.List {
return &wl.Metadata
}
func (wl *MessageQueueTriggerList) GetListMeta() metav1.List {
return &wl.Metadata
}
func (wl *PackageList) GetListMeta() metav1.List {
return &wl.Metadata
}
+21 -8
View File
@@ -4,22 +4,30 @@ The `binary` runtime is a go server that uses a subprocess to invoke executables
Use Cases
- Execute bash scripts
- Run executables of languages that have no dedicated environment yet.
- Execute arbitrary binaries (such as common sysadmin tools)
- Get support in _any_ programming language by executing the generated executable.
## Words of Caution
The environment runs on an alpine image with some additional utilty commandline tools installed, such as 'grep'.
However, in case you want to make use of more ecsoteric commandline tools, you should add the relevant apk to the
⚠️ **Words of Caution** ⚠️
The environment runs on an alpine image with some additional utility commandline tools installed, such as 'grep'.
However, in case you want to make use of more esoteric commandline tools, you should add the relevant apk to the
Dockerfile and build a new binary environment. See 'Compiling' for instructions.
When executing functions using binaries, **ensure that the executable is built for the right architecture**.
Using the default binary environment this means that the binary should be build for Linux.
## Usage
To get started with the latest binary environment:
```bash
fission env create --name binary --image fission/binary-env --builder fission/binary-builder
```
The interface to the executable used by this environment is somewhat similar to a [CGI interface](https://en.wikipedia.org/wiki/Common_Gateway_Interface).
This means that any HTTP headers are converted to environment variables of the form "HTTP_<header-name>". For example these
are some of frequently occurring headers:
```
```bash
# Request Metadata
CONTENT_LENGTH
REQUEST_URI
@@ -28,18 +36,23 @@ REQUEST_METHOD
# HTTP Headers
HTTP_ACCEPT
HTTP_USER-AGENT
...
HTTP_CONTENT-TYPE
# ...
```
The body of HTTP piped over the STDIN to the executable.
All output that is provided to the server over the STDOUT will be transformed into the HTTP response.
See the [README](../../examples/binary/README.md) in the binary examples directory for additional usage instructions.
## Compiling
Build the Dockerfile:
To build the runtime environment:
```bash
docker build --tag=${USER}/binary-env .
```
See the [README](../../examples/binary/README.md) in the binary examples directory for usage instructions.
To build the builder environment:
```bash
(cd builder/ && docker build --tag=${USER}/binary-builder .)
```
+1 -1
View File
@@ -46,7 +46,7 @@ type (
func (bs *BinaryServer) SpecializeHandler(w http.ResponseWriter, r *http.Request) {
if specialized {
w.WriteHeader(400)
w.WriteHeader(http.StatusBadRequest)
w.Write([]byte("Not a generic container"))
return
}
+8 -2
View File
@@ -1,9 +1,15 @@
FROM microsoft/dotnet:1.1-sdk AS builder
COPY * /proj/
RUN cd /proj && ./project-build.sh
# Build env image
FROM microsoft/dotnet:1.1.0-runtime
WORKDIR /fission-workdir
COPY out .
COPY --from=builder /proj/out .
EXPOSE 8888
ENTRYPOINT ["dotnet"]
CMD ["fission-dotnet.dll"]
CMD ["fission-dotnet.dll"]
+2 -3
View File
@@ -1,6 +1,5 @@
#!/bin/sh
THIS_DIR=$(realpath $(dirname $0))
docker run -v $THIS_DIR:/proj microsoft/dotnet:1.1-sdk sh -c "cd /proj && ./project-build.sh"
# clean up
rm -rf bin obj out
+8 -2
View File
@@ -1,9 +1,15 @@
FROM microsoft/dotnet:2.0.0-sdk AS builder
COPY * /proj/
RUN cd /proj && ./project-build.sh
# Build env image
FROM microsoft/dotnet:2.0-runtime
WORKDIR /fission-workdir
COPY out .
COPY --from=builder /proj/out .
EXPOSE 8888
ENTRYPOINT ["dotnet"]
CMD ["fission-dotnet20.dll"]
CMD ["fission-dotnet20.dll"]
+2 -4
View File
@@ -1,6 +1,4 @@
#!/bin/sh
THIS_DIR=$(realpath $(dirname $0))
docker run -v $THIS_DIR:/proj microsoft/dotnet:2.0.0-sdk sh -c "cd /proj && ./project-build.sh"
# clean up
rm -rf bin obj out
+16 -1
View File
@@ -1,2 +1,17 @@
#!/bin/sh
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -gcflags=-trimpath=$GOPATH -asmflags=-trimpath=$GOPATH -o fetcher .
version=$1
if [ -z $version ]; then
version=$(git rev-parse HEAD)
fi
date=$2
if [ -z $date ]; then
date=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
fi
gitcommit=$3
if [ -z $gitcommit ]; then
gitcommit=$(git rev-parse HEAD)
fi
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -gcflags=-trimpath=$GOPATH -asmflags=-trimpath=$GOPATH -ldflags "-X github.com/fission/fission.GitCommit=$gitcommit -X github.com/fission/fission.BuildDate=$date -X github.com/fission/fission.Version=$version" -o fetcher .
+2 -1
View File
@@ -71,6 +71,7 @@ func main() {
mux := http.NewServeMux()
mux.HandleFunc("/", fetcher.FetchHandler)
mux.HandleFunc("/upload", fetcher.UploadHandler)
mux.HandleFunc("/version", fetcher.VersionHandler)
mux.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
})
@@ -97,7 +98,7 @@ func specializePod(f *fetcher.Fetcher, fetchPayload *string, loadPayload *string
_, err = f.FetchSecretsAndCfgMaps(fetchReq.Secrets, fetchReq.ConfigMaps)
if err != nil {
log.Fatalf("Error fetching secerts/configmaps: %v", err)
log.Fatalf("Error fetching secrets/configmaps: %v", err)
return
}
+57 -18
View File
@@ -36,13 +36,15 @@ type (
Filename string `json:"filename"`
Secrets []fission.SecretReference `json:"secretList"`
ConfigMaps []fission.ConfigMapReference `json:"configMapList"`
KeepArchive bool `json:"keeparchive"`
}
// UploadRequest send from builder manager describes which
// deployment package should be upload to storage service.
UploadRequest struct {
Filename string `json:"filename"`
StorageSvcUrl string `json:"storagesvcurl"`
Filename string `json:"filename"`
StorageSvcUrl string `json:"storagesvcurl"`
ArchivePackage bool `json:"archivepackage"`
}
// UploadResponse defines the download url of an archive and
@@ -163,6 +165,11 @@ func writeSecretOrConfigMap(dataMap map[string][]byte, dirPath string) error {
return nil
}
func (fetcher *Fetcher) VersionHandler(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
fmt.Fprintf(w, fission.BuildInfo().String())
}
func (fetcher *Fetcher) FetchHandler(w http.ResponseWriter, r *http.Request) {
if r.Method != "POST" {
http.Error(w, "only POST is supported on this endpoint", http.StatusMethodNotAllowed)
@@ -212,6 +219,19 @@ func (fetcher *Fetcher) FetchHandler(w http.ResponseWriter, r *http.Request) {
// Fetch takes FetchRequest and makes the fetch call
// It returns the HTTP code and error if any
func (fetcher *Fetcher) Fetch(req FetchRequest) (int, error) {
// check that the requested filename is not an empty string and error out if so
if len(req.Filename) == 0 {
e := fmt.Sprintf("Fetch request received for an empty file name, request: %v", req)
log.Printf(e)
return http.StatusBadRequest, errors.New(e)
}
// verify first if the file already exists.
if _, err := os.Stat(filepath.Join(fetcher.sharedVolumePath, req.Filename)); err == nil {
log.Printf("Requested file: %s already exists at %s. Skipping fetch", req.Filename, fetcher.sharedVolumePath)
return http.StatusOK, nil
}
tmpFile := req.Filename + ".tmp"
tmpPath := filepath.Join(fetcher.sharedVolumePath, tmpFile)
@@ -221,7 +241,7 @@ func (fetcher *Fetcher) Fetch(req FetchRequest) (int, error) {
if err != nil {
e := fmt.Sprintf("Failed to download url %v: %v", req.Url, err)
log.Printf(e)
return 400, errors.New(e)
return http.StatusBadRequest, errors.New(e)
}
} else {
// get pkg
@@ -229,13 +249,22 @@ func (fetcher *Fetcher) Fetch(req FetchRequest) (int, error) {
if err != nil {
e := fmt.Sprintf("Failed to get package: %v", err)
log.Printf(e)
return 500, errors.New(e)
return http.StatusInternalServerError, errors.New(e)
}
var archive *fission.Archive
if req.FetchType == FETCH_SOURCE {
archive = &pkg.Spec.Source
} else if req.FetchType == FETCH_DEPLOYMENT {
// sometimes, the user may invoke the function even before the source code is built into a deploy pkg.
// this results in executor sending a fetch request of type FETCH_DEPLOYMENT and since pkg.Spec.Deployment.Url will be empty,
// we hit this "Get : unsupported protocol scheme "" error.
// it may be useful to the user if we can send a more meaningful error in such a scenario.
if pkg.Status.BuildStatus != fission.BuildStatusSucceeded && pkg.Status.BuildStatus != fission.BuildStatusNone {
e := fmt.Sprintf("Build status for the function's pkg : %s.%s is : %s, can't fetch deployment", pkg.Metadata.Name, pkg.Metadata.Namespace, pkg.Status.BuildStatus)
log.Printf(e)
return http.StatusInternalServerError, errors.New(e)
}
archive = &pkg.Spec.Deployment
}
// get package data as literal or by url
@@ -245,7 +274,7 @@ func (fetcher *Fetcher) Fetch(req FetchRequest) (int, error) {
if err != nil {
e := fmt.Sprintf("Failed to write file %v: %v", tmpPath, err)
log.Printf(e)
return 500, errors.New(e)
return http.StatusInternalServerError, errors.New(e)
}
} else {
// download and verify
@@ -253,27 +282,27 @@ func (fetcher *Fetcher) Fetch(req FetchRequest) (int, error) {
if err != nil {
e := fmt.Sprintf("Failed to download url %v: %v", req.Url, err)
log.Printf(e)
return 400, errors.New(e)
return http.StatusBadRequest, errors.New(e)
}
err = verifyChecksum(tmpPath, &archive.Checksum)
if err != nil {
e := fmt.Sprintf("Failed to verify checksum: %v", err)
log.Printf(e)
return 400, errors.New(e)
return http.StatusBadRequest, errors.New(e)
}
}
}
// check file type here, if the file is a zip file unarchive it.
if archiver.Zip.Match(tmpPath) {
if archiver.Zip.Match(tmpPath) && !req.KeepArchive {
// unarchive tmp file to a tmp unarchive path
tmpUnarchivePath := filepath.Join(fetcher.sharedVolumePath, uuid.NewV4().String())
err := fetcher.unarchive(tmpPath, tmpUnarchivePath)
if err != nil {
log.Println(err.Error())
return 500, err
return http.StatusInternalServerError, err
}
tmpPath = tmpUnarchivePath
}
@@ -281,11 +310,11 @@ func (fetcher *Fetcher) Fetch(req FetchRequest) (int, error) {
err := fetcher.rename(tmpPath, filepath.Join(fetcher.sharedVolumePath, req.Filename))
if err != nil {
log.Println(err.Error())
return 500, err
return http.StatusInternalServerError, err
}
log.Printf("Successfully placed at %v", filepath.Join(fetcher.sharedVolumePath, req.Filename))
return 200, nil
return http.StatusOK, nil
}
// FetchSecretsAndCfgMaps fetches secrets and configmaps specified by user
@@ -393,12 +422,22 @@ func (fetcher *Fetcher) UploadHandler(w http.ResponseWriter, r *http.Request) {
srcFilepath := filepath.Join(fetcher.sharedVolumePath, req.Filename)
dstFilepath := filepath.Join(fetcher.sharedVolumePath, zipFilename)
err = fetcher.archive(srcFilepath, dstFilepath)
if err != nil {
e := fmt.Sprintf("Error archiving zip file: %v", err)
log.Println(e)
http.Error(w, e, http.StatusInternalServerError)
return
if req.ArchivePackage {
err = fetcher.archive(srcFilepath, dstFilepath)
if err != nil {
e := fmt.Sprintf("Error archiving zip file: %v", err)
log.Println(e)
http.Error(w, e, http.StatusInternalServerError)
return
}
} else {
err = os.Rename(srcFilepath, dstFilepath)
if err != nil {
e := fmt.Sprintf("Error renaming the archive: %v", err)
log.Println(e)
http.Error(w, e, http.StatusInternalServerError)
return
}
}
log.Println("Starting upload...")
+7 -2
View File
@@ -1,6 +1,6 @@
ARG GO_VERSION=1.9.2
FROM golang:${GO_VERSION}
FROM golang:${GO_VERSION} AS builder
ENV GOPATH /usr
ENV APP ${GOPATH}/src/github.com/fission/fission/environments/go
@@ -10,7 +10,12 @@ ADD server.go ${APP}
WORKDIR ${APP}
RUN go get
RUN go build -o /server server.go
RUN go build -a -o /server server.go
FROM ubuntu:18.04
WORKDIR /
COPY --from=builder /server /
RUN apt update && apt install -y ca-certificates && rm -rf /var/lib/apt/lists/*
ENTRYPOINT ["/server"]
EXPOSE 8888
+2 -2
View File
@@ -5,9 +5,9 @@ FROM ${BUILDER_IMAGE}
FROM golang:${GO_VERSION}
COPY --from=0 /builder /builder
ENV GOPATH /usr
WORKDIR ${GOPATH}
COPY --from=0 /builder /builder
ADD build.sh /usr/local/bin/build
+14 -12
View File
@@ -1,19 +1,21 @@
#!/bin/sh
set -eux
srcDir=${GOPATH}/src/$(basename ${SRC_PKG})
trap "rm -rf ${srcDir}" EXIT
if [ -d ${SRC_PKG} ]
then
echo "Building in directory ${SRC_PKG}"
cd ${SRC_PKG}
go build -buildmode=plugin -i -o ${DEPLOY_PKG} .
echo "Building in directory ${srcDir}"
ln -sf ${SRC_PKG} ${srcDir}
elif [ -f ${SRC_PKG} ]
then
fn=$(basename ${SRC_PKG})
d=/tmp/$fn
mkdir $d
trap "rm -rf /tmp/$fn" EXIT
echo "Building file ${SRC_PKG} in $d"
cd $d
cp ${SRC_PKG} ./function.go
go build -buildmode=plugin -i -o ${DEPLOY_PKG} .
echo "Building file ${SRC_PKG} in ${srcDir}"
mkdir -p ${srcDir}
cp ${SRC_PKG} ${srcDir}/function.go
fi
cd ${srcDir}
go build -buildmode=plugin -i -o ${DEPLOY_PKG} .
+7
View File
@@ -0,0 +1,7 @@
.springBeans
.project
.mvn
.settings
.classpath
target/
bin/
+10
View File
@@ -0,0 +1,10 @@
FROM maven:3.5-jdk-8 as BUILD
COPY src /usr/src/myapp/src
COPY pom.xml /usr/src/myapp
RUN mvn -f /usr/src/myapp/pom.xml clean package
FROM openjdk:8-jdk-alpine
VOLUME /tmp
COPY --from=BUILD /usr/src/myapp/target/env-java-0.0.1-SNAPSHOT.jar /app.jar
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar","--server.port=8888"]
EXPOSE 8888

Some files were not shown because too many files have changed in this diff Show More