Commit Graph
92 Commits
Author SHA1 Message Date
Ta-Ching ChenandGitHub a021f5003a Fix “rm: missing operand” in release script (#779) 2018-07-06 14:15:20 +08:00
Ta-Ching ChenandGitHub 6cffe7b727 Fix go env panic “plugin: not implemented” (#784) 2018-07-06 12:24:46 +08:00
Ta-Ching ChenandGitHub 58ee3541db Change flag name to KeepArchive for backward compatibility (#787) 2018-07-06 03:13:07 +08:00
Ta-Ching Chenandsmruthi2187 bb049a7639 Move build process from host to docker container during release process (#682) 2018-07-03 21:14:06 -07:00
Ta-Ching ChenandGitHub ba44ca0d87 Add release steps to render & upload fission installation YAML (#745) 2018-06-26 13:41:43 +08:00
Ta-Ching ChenandGitHub d7ed94dfe4 Fix executor not reaps idle function pods for functions with executortype newdeploy (#744) 2018-06-21 17:54:42 +08:00
Ta-Ching ChenandGitHub 7514166595 Fix executor wrongly passes loop variable reference to function (#751) 2018-06-19 09:55:41 +08:00
Ta-Ching ChenandGitHub cb45f0a455 Add time trigger cron spec examination tool (#680) 2018-06-04 19:02:54 +08:00
Ta-Ching ChenandGitHub 60e46745ce Update k8s dependencies to 1.10 (#687) 2018-06-01 15:43:19 +08:00
Ta-Ching ChenandGitHub 9ddaffd858 Check spec directory exists before reading spec files (#709)
* Check spec directory exists before reading spec files
* Fix spec example
2018-05-31 03:01:40 +08:00
Ta-Ching ChenandGitHub d0942fb7a5 Replace Werkzeug with Bjoern as underlying WSGI server (#672) 2018-05-09 14:00:40 +08:00
Ta-Ching ChenandGitHub e58dac5c5c Version -> 0.7.2 (#670) 2018-05-05 20:01:23 +08:00
Ta-Ching ChenandGitHub 58260d3f1f Add benchmark script (#666) 2018-05-05 12:42:16 +08:00
Ta-Ching ChenandGitHub 5a5927c1be Delete deployment with proper delete propagation policy (#630) 2018-05-01 23:25:50 +08:00
Ta-Ching ChenandGitHub 67b6cb5530 Fix newdeploy backend failed to delete deployment due to incorrect resource version (#657) 2018-05-01 16:48:34 +08:00
Ta-Ching ChenandGitHub 91b6e3771f Change time precision for fluentd influxdb plugin to nano second (#646) 2018-04-26 20:57:55 +08:00
Ta-Ching ChenandGitHub 7353d4e9c8 Fix test cases occasional failure (#647) 2018-04-26 14:39:16 +08:00
Ta-Ching ChenandGitHub 9a949e8542 Fix SEGFAULT issue when buildmgr failed to update package (#635) 2018-04-21 23:17:33 +08:00
Ta-Ching ChenandGitHub 87feda395e Fix executor does not reap specialized function pod when env no longer exists (#633) 2018-04-19 07:39:37 +08:00
Ta-Ching ChenandGitHub 14ae582d3f Fix SEGFAULT issue when build manager failed to update package (#626) 2018-04-17 09:36:11 +08:00
Ta-Ching ChenandGitHub f07f28eeba Remove confusing default value description & add default value to cli flag (#619) 2018-04-16 17:27:10 +08:00
Ta-Ching ChenandGitHub 1d4368c23a Add validate function to crd resource and do validate before creation/update (#580) 2018-04-08 00:56:13 +08:00
Ta-Ching ChenandGitHub 5fc04e681d Fix components crash before crds creation (#602)
* Wait for CRDs creation for 30 sec when component start
* Fix ensureCRD return nil while the error is not empty
2018-04-04 15:40:55 +08:00
Ta-Ching ChenandGitHub b6fa43f5ac Add deprecated message to subcommand pods (#592) 2018-04-04 01:26:27 +08:00
Ta-Ching ChenandGitHub 9067431e48 Use a separate controller loop to watch functions change and create a service (#544) 2018-04-04 01:17:51 +08:00
Ta-Ching ChenandGitHub 045ffe8ffc Add steps for how to set FISSION_ROUTER env variable & update docs (#593) 2018-03-31 01:10:40 +08:00
Ta-Ching ChenandSoam Vasani 27d3b80905 Add post-upgrade-job to track fission upgrade (#564) 2018-03-26 12:28:35 -07:00
Ta-Ching ChenandGitHub 465f15a377 Show fission deployment version with cli (#538)
Add version information to binary through go ldflags
2018-03-19 14:14:55 +08:00
Ta-Ching ChenandGitHub 4fd884820e Fix executor failed to clean cache & kubeobjs after function deleted (#534) 2018-03-14 02:29:12 +08:00
Ta-Ching ChenandGitHub 896303cddf Always retry for function pod specialization when istio feature is enabled. (#536) 2018-03-11 23:17:05 +08:00
Ta-Ching ChenandGitHub b4b97af805 Show warning when trying to create a route with non-existent function (#539) 2018-03-09 02:45:30 +08:00
Ta-Ching ChenandGitHub dd4dc593d4 Fix executor tries to create a new deployment when a function is updated (#524)
Newdeploy manager now checks the existence of function service cache by function UID before trying to create a new deployment. And return the cached fsvc directly if the cache exists.
2018-03-08 02:52:36 +08:00
Ta-Ching ChenandGitHub b6430a258b Fix CLI not update function's secret/configmap correctly (#512)
This PR addressed some issues introduced in PR399. Also, now env builders and function pods can mount shared secret/configmap volumes correctly.
2018-02-28 05:28:05 +08:00
Ta-Ching ChenandGitHub 78ac09fef0 Istio integration (#421)
This the very first step for fission to integrate with Istio, which is an open platform to connect, manage, and secure microservices. With Istio, users are able to monitor functions usage and trace requests latency through dashboards. For more information, please visit http://fission.io/docs/
2018-02-28 04:09:32 +08:00
Ta-Ching ChenandGitHub d72eefa9bc Fix post-install-job container failure due to curl command not found (#514) 2018-02-27 18:08:16 +08:00
Ta-Ching ChenandSoam Vasani e90fc20f0a Improve release script (#494)
* Add step for releasing environment builder image

* Strip $GOPATH from strack trace
2018-02-12 15:42:23 -08:00
Ta-Ching ChenandGitHub e9fcefd05c Update SHA256 HASH in CHANGELOG.md due to binaries update (#493)
Update SHA256 HASH in CHANGELOG.md due to binaries update due to build flags change.
(Strip GOPATH from stack trace)
2018-02-10 00:47:44 +08:00
Ta-Ching ChenandGitHub a7fe3b203b Add simple usage doc for accessing secret/configmap in function (#484) 2018-02-08 13:05:30 +08:00
Ta-Ching ChenandGitHub e4b1f57999 Add upgrade guide from 0.4.x to 0.5.0 (#490) 2018-02-08 13:05:04 +08:00
Ta-Ching ChenandGitHub e24b6f82f0 Version -> 0.5.0 (#489) 2018-02-08 03:39:02 +08:00
Ta-Ching ChenandGitHub ed1384e9ff Fix binary environment build failure due to package not found (#487)
Subpackage k8s.io/client-go/pkg/api/v1 is no longer exist in HEAD.
(https://github.com/kubernetes-incubator/external-storage/issues/324)
For now, just duplicate the FunctionLoadRequest struct like what go environment did.
2018-02-07 21:44:12 +08:00
Ta-Ching ChenandGitHub 85cc1ab284 Migrate project.json to dotnet.csproj & do build in dotnet container (#488) 2018-02-07 21:43:51 +08:00
Ta-Ching ChenandSoam Vasani cd4215f5bc Fix potential nil pointer problem when both watchResources and waitForBuild are equal to false (#485) 2018-02-06 15:14:47 -08:00
Ta-Ching ChenandGitHub aeec590ded Fix possible context leak problem (#483) 2018-02-06 21:34:01 +08:00
Ta-Ching ChenandGitHub 650499d55b Add glide flag to strip nested vendor (#480)
Adds --strip-vendor (-v) to travis.yml when installing dependencies.
For more information about the flag, please check dependency flattening (https://glide.readthedocs.io/en/stable/getting-started/)
2018-02-06 02:31:06 +08:00
Ta-Ching ChenandGitHub eccbc58190 Add readiness probe to go env (#461) 2018-01-30 21:36:13 +08:00
Ta-Ching ChenandSoam Vasani 1bdfaffa8d Show warning when user tries to create a function with a non-existent environment (#436)
Show warning when user try to create a function with non-existent environment
2018-01-26 16:35:17 -08:00
Ta-Ching ChenandGitHub c1bdb7b71a Block build requests until environment builder is ready (#437)
* Add readiness probe
* Remove builder manager http api interface since we don’t use/need it
* Check environment builder status and block build requests until builder is ready
* Replace deprecated api extension interface
* Add healthy check to python env
2018-01-26 01:13:36 +08:00
Ta-Ching Chen 5424f9a15e Fix fission bundle build failure 2018-01-24 15:38:52 +08:00
Ta-Ching Chen b034d09843 Update docs-site 2018-01-23 13:27:10 +08:00
Ta-Ching ChenandSoam Vasani ab7c749abd Add go vet check (#430)
* Add go vet check

* Replace go vet with go tool vet to avoid printing exit message
2018-01-22 09:28:16 -08:00
Ta-Ching ChenandSoam Vasani 1efa7806b2 Fix python environment failed to launch due to port needs integer not string (#451) 2018-01-20 06:06:50 -08:00
Ta-Ching ChenandSoam Vasani 4109071ff6 Improve travi-ci test scripts (#434)
Make our travis-based integration test less flaky:

* Wait for tiller ready before installation

* Remove all existing old releases before installation
2018-01-01 11:34:11 +05:30
Ta-Ching ChenandSoam Vasani a3a3030eb8 Fix glide failed to check out github.com/dsnet/compress & update gorilla pkg version (#429) 2017-12-14 05:23:49 +05:30
Ta-Ching ChenandSoam Vasani 92426379e7 [Issue 423] build logs not saved on build error (#426)
* Fix builder return empty buildlogs when encounters errors

* Append error logs to build logs

* Builder manager updates package with build logs plus error log

* Fix potential nil pointer bug

* Replace numeric status code with built-in const

* Remove no use parameter

* Fix go fmt
2017-12-06 22:20:41 -06:00
Ta-Ching ChenandSoam Vasani 7f628ecd66 Overwrite request host with internal host to prevent request rejection (#419) 2017-11-30 23:58:14 -08:00
Ta-Ching ChenandSoam Vasani 6fc718378a Fix pool manager crash problem if failed at http call (#418) 2017-11-30 13:40:17 -08:00
Ta-Ching ChenandSoam Vasani 9c51e8481f Update nats dependencies (#411) 2017-11-28 16:12:31 -08:00
Ta-Ching ChenandSoam Vasani 685a12d3a3 Add package command (#385)
Add package command support: This PR adds package command to CLI. package provides some useful subcommands to use such as packages CRUD and display package detail information. Also it is able to reuse existing package at function creation.

* Check function existence before creating package

* Check package existence

* Add support for downloading archive from given url

The functionality was supported before 2f3ff18551. Add this functionality back for more flexible usage.

* Add output flag to save archive content in specific file

* Add logs when package create/update

* Fix fnCreate requires environment argument when a pkg is specified

* Fix fnUpdate failed to update function pkg info when a package is specified

* Add package command test

* Fix wrong python test image in test script

* Remove package description fields

* Fix package command not update package but creating a new one

* Set package as pending status only when there is no deploy archive

* Rename function from fetchArchiveFromArbitraryURL to downloadToTempFile

* Use io.Copy to prevent loading all body into memory

* Revise some messages

* Allow user to update package build command

* Allow user to update package content when using function update

* Retrieve pkgName from function packageref if it’s not specified

* Fix function failed to update due to resource conflict

* Fix test case failure due to single quote

* kick ci

* Fix failed to update function packageRef

* kick ci

* kick ci
2017-11-25 06:54:43 -08:00
Ta-Ching ChenandSoam Vasani cce23cf042 Use common controller/store for httpTriggerSet and functionReferenceResolver (#390)
This fixes a bug where functionReferenceResolver returned out-of-date function metadata and caused the router to proxy requests to old function pods.

It also uses the go context package to shutdown the controller when the router is shutting down.
2017-11-09 07:07:14 -08:00
Ta-Ching ChenandSoam Vasani 0fd0452e64 Delete failed helm releases to prevent test case failure (#393) 2017-11-09 06:04:06 -08:00
Ta-Ching ChenandSoam Vasani a2e3e415bd Builder manager bugfixes (#367)
This contains a few different builder manager fixes:

* Update the zip file structure to avoid an extra subdirectory
* Annotate packages with what functions are using them
* Only trigger builds when there is no deployment archive
* Python environment loadpath bugfixes
* Other bugfixes
2017-11-06 23:01:46 -08:00
Ta-Ching ChenandSoam Vasani b8f6a4ce04 Use k8s client store to sync functions and triggers for fast synchronization (#382) 2017-11-06 12:58:19 -08:00
Ta-Ching ChenandSoam Vasani 46406cadb4 Switch from ThirdPartyResources to CustomResourceDefinitions (#381)
Switch Fission's storage over to the new CustomResourceDefinitions, from the deprecated ThirdPartyResources. This allows us to be compatible with Kubernets 1.8 and onwards.

This also adds a CLI tool for dumping state from an old fission version and restoring state into new CRDs.

The storage service is unaffected by this change.
2017-11-04 14:55:16 -07:00
Ta-Ching ChenandSoam Vasani 71fad66dbf Use Kubernetes informer and cache in router (#376)
Use Kubernetes client's informer and cache to watch functions and triggers in the router. This replaces the polling loop we were using so far.
2017-10-25 15:10:44 -07:00
Ta-Ching ChenandSoam Vasani 31f255e5de Update k8s client version to 4.0.0 (#351)
Move to client-go v4.0.0 from 1.5.
2017-09-29 07:37:36 -07:00
Ta-Ching ChenandSoam Vasani c8ff318747 Add builder manager support (#308)
This change orchestrates function builds.

Environments (in v2) define a builder image, just like they do a runtime image. The builder image contains a build script that's invoked with source and deployment paths (as env vars).

The buildermgr watches for environments with build images defined, and creates build deployments and services.

Functions can define source and deployment. Buildermgr watches for functions with source code (and build status == pending) and invokes the environment's builder when appropriate. It captures logs from the build and sets the build status (success/failure) and build lots into the PackageStatus.
2017-09-25 07:53:09 -07:00
Ta-Ching ChenandSoam Vasani fcf39a364c Use Containers to find matched storage containers (#350) (#353)
Use stow api correctly to find our function storage container on restarts.
2017-09-24 10:36:06 -07:00
Ta-Ching ChenandSoam Vasani 3ffa5f2874 Fix storage service failed to start after restarting it (#352) 2017-09-24 08:21:13 -07:00
Ta-Ching ChenandSoam Vasani 67fdb8bd0c Fix missing checksum from CLI uploads (issue #342) (#343)
Archive upload from the CLI wasn't filling in the checksum field, causing validation to fail.
2017-09-22 12:27:42 -07:00
Ta-Ching ChenandSoam Vasani ebd3c9f989 Use latest function metadata to check cached function service. (#316)
Watch functions in the router and use that to trigger functionReferenceResolver cache invalidation.

We might want to rate-limit syncTriggers in a future change, since we might be triggering it more often than needed.
2017-09-08 13:53:02 -07:00
Ta-Ching ChenandSoam Vasani 5249f72b34 Unarchive zip file after fetcher downloads the package (#301)
Fetcher now uncompresses archive files. 

This is triggered by automatically detecting the file format, although maybe we should make that explicit or at least move the detection into the client.
2017-09-01 07:50:47 -07:00
Ta-Ching ChenandSoam Vasani 38b7f45e1b Add Environment v2 Builder (#298)
Build server and client for environments.

This build server will run inside environment builders and invoke environment-specific scripts for doing builds. It provides a uniform language-independent API for builds to the fission build manager.

The change also includes an implementation of a builder for Python -- this "builder" acts on requirements.txt and runs `pip install` to collect all function deps.
2017-08-31 10:56:40 -07:00
Ta-Ching ChenandSoam Vasani c541390ad6 Add env builder & srcpkg through cli (#296)
CLI support for specifying an environment builder and source packages.
2017-08-28 10:44:31 -07:00
Ta-Ching ChenandSoam Vasani 190ef0fe22 Set fetcher image through poolmgr env (#280)
Add a helm value for the fetcher image. Pass the fetcher image path to poolmgr through an env var.
2017-08-18 15:12:10 -07:00
Ta-Ching ChenandSoam Vasani d833e1e9aa Fix fetcher failed to access TPR if RBAC is enabled (#288) 2017-08-18 15:01:21 -07:00
Ta-Ching ChenandSoam Vasani 2dfa52da47 Set message content-type based on the trigger.Spec.ContentType (#279) 2017-08-17 10:30:24 -07:00
Ta-Ching ChenandSoam Vasani 96c1de7fbe Parse metadata.Name before creating tpr resource (#284) 2017-08-14 08:55:31 -07:00
Ta-Ching ChenandSoam Vasani 35519b3527 Remove etcd from fission-all chart (#282)
Since we use Kubernetes APIs for storing our state now, etcd can be removed.
2017-08-12 16:04:30 -07:00
Ta-Ching ChenandSoam Vasani 0a2c308b40 Update/Add fission-core & fission-all helm charts (#239)
Improve fission helm chart; split it into "fission core" and "fission all". fission-core is the minimal install, and fission-all will have all the bells and whistles.

* Update/Add fission-core & fission-all helm charts
* Fix incompatible types for comparison
* Add message queue trigger
* Rename openshiftRBAC to openshift
* Update documentation & comments
* Fix logger & kubewatcher failed to access kubernetes resource due to RBAC
2017-07-31 14:10:26 -07:00
Ta-Ching ChenandSoam Vasani e1ec460da8 Fix nats trigger replies message to non-existing resptopic (#260) 2017-07-26 11:43:23 -07:00
Ta-Ching ChenandSoam Vasani b052e67e9f Fix logger prints wrong log (#263) 2017-07-20 13:44:23 -07:00
Ta-Ching ChenandSoam Vasani 0412c9a876 Fix unstoppable kubewatcher (#208)
Fix a bug due to which watches could never be stopped; simplify control flow a bit.
2017-07-10 16:01:48 -07:00
Ta-Ching ChenandSoam Vasani fb29f897fe Add message queue trigger support (#218)
Adds support for message queue triggers based on the NATS-Streaming message queue.

This lets the user map a queue topic to a function, and optionally send the function's response into another queue topic.

The message queue trigger manager is designed to be message queue independent, so we should be able add support for more queues by adding implementations for the relatively simple mqtrigger.MessageQueue interface.
2017-07-05 17:19:58 -07:00
Ta-Ching ChenandSoam Vasani bafe196bb3 Fission update must require at least one change to function (#241) 2017-06-26 22:22:09 -07:00
Ta-Ching ChenandSoam Vasani cc66b245d0 Fix http response body not closed correctly, return immediately on error (#210)
* Fix controller timer handler not return when error occurred

* Fix controller not close http response correctly

This PR aims to fix controller not close response correctly. Also, I’ve check all over the project there is no more resource leak issues exist after this patch.

* Fix controller not return immediately when error occurred

* Sort import
2017-06-17 13:10:54 -07:00
Ta-Ching ChenandSoam Vasani 913207ae57 Fix pool contains wrong environment metadata (#221)
The generic pool manager creates a generic pool with the reference of environment metadata. Since the reference is address of loop variable, so the content of metadata will be replaced with the last element of envs. So pool manager sets up logging with wrong env.
2017-06-08 11:11:58 -07:00
Ta-Ching ChenandSoam Vasani 9162f5d3ea Print log when timetrigger is removed (#209) 2017-05-30 14:21:41 -07:00
Ta-Ching ChenandSoam Vasani 9669f422af Retrieve function logs from controller (#207)
Establish a proxy server from the controller to the log database.

Redirect query commands send from client to database then proxy back the db response.

Use parameter binding instead of fmt.Sprintf to prevent SQL injection.
2017-05-30 14:19:22 -07:00
Ta-Ching ChenandSoam Vasani 479b7d05be Add function logs support (#53) (#131)
Add function log aggregation and persistence using Fluentd and InfluxDB.

Fluentd and a helper sidecar run as a daemonset.  The poolmgr sets up logging for each function pod, using the helper sidecar. Fluentd forwards logs to InfluxDB, which is run as a deployment and service. The client CLI directly queries InfluxDB for logs.

Fluentd supports many outputs besides InfluxDB, so we aren't very tied to InfluxDB.

The setup is somewhat manual, which we should be able to improve by integrating this into the helm chart.

Diagram of component interactions: https://cloud.githubusercontent.com/assets/202578/23100399/b0e3ea00-f6ba-11e6-8f2f-6588cfef2e84.png
2017-03-22 15:59:42 -07:00
Ta-Ching ChenandSoam Vasani 8182470ed2 Fixed pod has no ip (#139) (#141)
Fixes #139.  The code before this change was saving up pointers to the loop variable in a slice. So loop was useless and we always chose the last pod in the list, whether it was ready or not.  With this fix, choosePod should always return a ready pod.

Also adds a check for the pod phase, before looping over the containers in the pod, and discard pods that aren't running yet.
2017-03-05 11:01:47 -08:00
Ta-Ching ChenandSoam Vasani 8aa96f5f79 Fix failure to delete function when function's file doesn't exist (#118) 2017-02-10 23:38:13 -08:00