Commit Graph
100 Commits
Author SHA1 Message Date
Ta-Ching ChenandTa-Ching Chen 48e7a13c0f Fission 1.7.0-rc.1 2019-11-18 10:06:32 +00:00
Ta-Ching ChenandGitHub ef0a2fa340 Refactor canary config command (#1370) 2019-11-01 15:36:40 +08:00
Ta-Ching ChenandGitHub e22cdee5f5 Refactor kubewatch command (#1365) 2019-10-31 10:48:27 +08:00
Ta-Ching ChenandGitHub a151f89eec Move route creation to independent function (#1362) 2019-10-30 05:47:51 +08:00
Ta-Ching ChenandGitHub d20dc9aa64 Allow using URL as archive source when creating functions (#1360)
In the case of large files, it takes a long time for the user to download
the source from the URL and upload it to StorgeSvc through CLI.

This PR allows a user to use URL as the function source when creating a function
and provides a new flag "--keeparchiveurl" to let the user to decided
whether the CLI should download the file first or store the file URL in the
archive directly. If "--keeparchiveurl" is true, then no checksum will be
generated, it's the user's responsibility to ensure the file won't be changed.
2019-10-28 22:37:13 +08:00
Ta-Ching ChenandGitHub 52b5cb0902 Remove UID from CLI output (#1361) 2019-10-28 19:34:02 +08:00
Ta-Ching ChenandGitHub bc5c181fc2 Update fission architecture doc (#1356) 2019-10-22 19:46:57 +08:00
Ta-Ching ChenandGitHub fed30e1b24 Refactor plugin & version subcommands (#1359) 2019-10-22 17:57:56 +08:00
Ta-Ching ChenandGitHub e42f81a7a1 Refactor package CLI command (#1345) 2019-10-13 01:30:20 +08:00
Ta-Ching ChenandGitHub f30e7df43d Fix release script wrongly tags image name & sed problem on Linux (#1353) 2019-10-12 21:38:13 +08:00
Ta-Ching ChenandGitHub c930c69629 Update CHANGELOG for 1.6.0 (#1352) 2019-10-11 04:07:38 +08:00
Ta-Ching Chen 08b6d43ac1 Fission 1.6.0 2019-10-11 01:11:58 +08:00
Ta-Ching ChenandGitHub 53b075a315 Move statefulset to apps/v1 (#1350) 2019-10-10 23:42:40 +08:00
Ta-Ching ChenandGitHub b8311d43ce Fix newdeploy failed to find serviceEntry in cache (#1349)
When a function with executor type newdeploy got created, Newdeploy
is expected to create deployment/HPA/service for it and insert serviceEntry
to the cache for later use. Once clients call the function, newdeploy returns
the serviceEntry to the router.

However, the log shows that the newdeploy was unable to find the entry and
prints "Resource not found - key 'xxx' not found". The root cause is that the
informer controller instead of processing items in parallel, it dispatches XXFunc
to process items one by one. So if there is any problem during the creation of the
kubernetes resource, it takes a longer time to process the next item and hence
the serviceEntry was not inserted before clients call the function.

This PR lets the newdeploy to process items in extra goroutines instead of blocking
the process loop. It's a workaround to solve the problem above, we should consider
using workqueue to solve it in the future.
2019-10-10 23:32:36 +08:00
Ta-Ching ChenandGitHub 1075c5da13 Support encoded path in router (#1347)
Encoded path means the URL path contains encoded string like "/foo%2Fbar".
Gorilla/Mux by default doesn't enable encoded path support, you need to enable
it when initializing the router.

This PR adds a new environment variable USE_ENCODED_PATH to the router
deployment to enable encoded path support.
2019-10-10 02:23:41 +08:00
Ta-Ching ChenandGitHub 59c8569190 Support custom private go vendor (#1346) 2019-10-10 01:44:44 +08:00
Ta-Ching ChenandGitHub d358a29b17 Fix the namespace mismatch problem when deploying with a single YAML file (#1344)
If a user deploys fission in the namespace which is different from the one in the single YAML file generated by helm, fission components won't be able to talk to each other due to the wrong namespace appends after the service address.

This PR adds --namespace when generating the YAML file to prevent the mismatch problem.
2019-10-09 01:14:56 +08:00
Ta-Ching ChenandGitHub e253d2a376 Move charts executor service to the right place (#1343) 2019-10-07 18:44:58 +08:00
Ta-Ching ChenandGitHub bfd005a295 Allow to deploy router as DaemonSet (#1342)
ReplicaSet generated by Deployment distributes pods to nodes based
on nodes resource usage, which means in some cases the newly created
pods may be assigned to only a few nodes. The requests will go to the
same node if there multiple pods on it and may increase the node
resource usage and overall request latency.

To solve the problem mentioned above, this PR allows users to deploy
the router as DaemonSet so that we can distribute requests across all
nodes for better workload distribution and lower latency.
2019-10-07 18:09:36 +08:00
Ta-Ching ChenandGitHub b18462b10b Fix executor doesn't apply user-configured container spec correctly (#1339)
The merge function executor used wasn't merge container correctly, and it
didn't merge all fields in spec except volumeMount & Env which confused people.

To apply the user-configured container correctly, this PR changes the way of merge
and follows rules:

1. Slices are merged and return an error if the elements in the slice have name conflicts.
2. Maps are merged, the value of map of dst container are overridden if the key is the same.
3. The rest of the fields of dst container are overridden directly.
2019-10-04 05:16:04 +08:00
Ta-Ching ChenandGitHub 30dba5dae9 Bump jvm builder JDK version (#1340) 2019-10-03 14:24:22 +08:00
Ta-Ching ChenandGitHub c61442ee76 Remove too verbose and unhelpful debug log (#1336) 2019-10-02 01:35:23 +08:00
Ta-Ching ChenandGitHub 7cdc146515 Move Deployment API group from extensions/v1beta1 to apps/v1 (#1331)
In 1.9, kubernetes prompted Deployment to apps/v1 and finally deprecated
extensions/v1beta1 support in 1.16. Users will see the following error message
in log when fission components try to submit Deployment to the k8s API server
and specify extensions/v1beta1 as API group.

error: unable to recognize "deployment": no matches for kind "Deployment" in version "extensions/v1beta1"

This PR updates call to k8s API server and apiKind in chart files to avoid fission failure in 1.16.
2019-09-29 22:16:33 +08:00
Ta-Ching ChenandGitHub e734fa3554 Fix router health route not found if no HTTP triggers created (#1333)
Router readiness probe failed due to kubelet cannot find router healthz endpoint.
This is caused that the healthz endpoint is added only when there is any update request
send to the updateRouterRequestChannel, and makes router failed.

This PR sends an update request right before the router is started to avoid the problem.
2019-09-28 17:43:04 +08:00
Ta-Ching ChenandGitHub 71b587b58e Use ErrorHandler to handle proxy error (#1310)
The router prints error no matter what error type it is.
It's useful for troubleshooting, however, it also prints
the context canceled error, which means that users abort
request before reply and its really normal nowadays. Also,
the router returns 502 if error is not nil and may confused
client if it's a timeout error.

To solve these problems, this PR adds an error handler to
reverse proxy to examine the return error and change the
status code when needed.
2019-09-28 14:22:20 +08:00
Ta-Ching ChenandGitHub 81ae6f38fc Fix spec doesn't update status of failed package when applying spec files (#1332)
Previously, the spec doesn't update the package status if nothing
in the spec file changed. Due to this, the failed package will
always stay in the failed state.

This PR adds a check to see whether a package is in the failed state.
If yes, then changes the state to pending for builder manager to rebuild it.
2019-09-28 04:35:57 +08:00
Ta-Ching ChenandGitHub 7fbb4098a4 Fix empty host value when list http triggers (#1328) 2019-09-28 00:28:14 +08:00
Ta-Ching ChenandGitHub f8af8b91f5 List/Delete HTTP triggers by function (#1327)
Previously, a user has to delete HTTP triggers point to
the same function one by one. This PR adds new flag --function
to list & delete commands. So that user can delete or list all
triggers with the same function. However, this feature is not
working for triggers with canary deployment setup.
2019-09-27 17:21:11 +08:00
Ta-Ching ChenandGitHub b5341edec0 Add Ingress TLS support (#1326)
This PR aims to add the Ingress TLS support by specifying the
TLS secret when creating/updating the HTTP trigger.

Command example:

fission route create --name foo \
    --url /foo/{bar} --function foofn --createingress \
    --ingressannotation "nginx.ingress.kubernetes.io/ssl-redirect=false" \
    --ingressannotation "nginx.ingress.kubernetes.io/use-regex=true" \
    --ingressrule "*=/foo/*"
    --ingresstls "foobartls"
2019-09-27 00:49:23 +08:00
Ta-Ching ChenandGitHub 49d60b19f3 Add Ingress host, path and annotations support (#1325)
For each ingress controller, the format of ingress host,
path and annotations are different. To support different
kinds of controller, this PR adds new ingress config field
to http trigger spec. A user can set annotations, host and
path based on the type of underlying ingress controller
with CLI.

Command example:

fission route create --name foo \
    --url /foo/{bar} --function foofn --createingress \
    --ingressannotation "nginx.ingress.kubernetes.io/ssl-redirect=false" \
    --ingressannotation "nginx.ingress.kubernetes.io/use-regex=true" \
    --ingressrule "*=/foo/*"
2019-09-26 16:54:16 +08:00
Ta-Ching ChenandGitHub 82a7acf408 Set maxSurge to 20% for safe rolling upgrade (#1321) 2019-09-23 16:11:56 +08:00
Ta-Ching ChenandGitHub 7bd57c208a Disable revision history for newdeploy (#1319) 2019-09-23 14:25:49 +08:00
Ta-Ching ChenandGitHub 27ec605955 Pre-built docker go module dependency cache image (#1318) 2019-09-18 05:07:53 +08:00
Ta-Ching ChenandGitHub c94e02b302 Fix newdeploy doesn't handle error properly (#1316) 2019-09-16 15:43:36 +08:00
Ta-Ching ChenandGitHub 5d2d65f05c Fix failed to update specialization timeout (#1315) 2019-09-16 13:21:25 +08:00
Ta-Ching ChenandGitHub 6d3d277cb9 Fix potential nil pointer problem when using multierror pkg (#1311) 2019-09-16 11:03:23 +08:00
Ta-Ching ChenandGitHub 4d654c6fe7 Add codecov config file (#1314) 2019-09-16 09:20:49 +08:00
Ta-Ching ChenandGitHub ecb97db913 Integrate with Codecov (#1308) 2019-09-11 14:52:45 +08:00
Ta-Ching ChenandGitHub 082c244b0b Fix executor unable to list secrets/configmaps (#1307) 2019-09-11 10:47:20 +08:00
Ta-Ching ChenandGitHub 78026ac573 Add specialization timeout default value to CLI (#1305) 2019-09-10 17:24:35 +08:00
Ta-Ching ChenandGitHub 4cfd459317 Update CHANGELOG for 1.5.0 (#1304) 2019-09-09 17:06:44 +08:00
Ta-Ching ChenandGitHub e481c234ee Fission 1.5.0 (#1303) 2019-09-09 15:38:27 +08:00
Ta-Ching ChenandGitHub 0d872020e3 Refactor support dump CLI (#1301) 2019-09-04 14:42:47 +08:00
Ta-Ching ChenandGitHub 58811ab522 Fix unit tests failure (#1297) 2019-08-30 23:39:45 +08:00
Ta-Ching ChenandGitHub ef8289ff73 Support nodejs function with 0 argument (#1295) 2019-08-30 15:31:04 +08:00
Ta-Ching ChenandGitHub 86b506c9b6 Add troubleshooting guide link (#1292) 2019-08-28 14:41:13 +08:00
Ta-Ching ChenandGitHub e762471d78 Bump Go version to 1.12 (#1290) 2019-08-28 13:47:40 +08:00
Ta-Ching ChenandGitHub c49377569e Refactor environment CLI command (#1265) 2019-08-27 15:43:59 +08:00
Ta-Ching ChenandGitHub 560aa34ce2 Fix typo extraCoreComponmentPodConfig -> extraCoreComponentPodConfig (#1287) 2019-08-27 14:12:51 +08:00
Ta-Ching ChenandGitHub ee18a88664 Update go dependencies (#1240)
NOTES for update k8s dependencies

In go.mod

require (
    k8s.io/api kubernetes-x.x.x
    k8s.io/apiextensions-apiserver kubernetes-x.x.x
    k8s.io/apimachinery kubernetes-x.x.x
    // check here https://github.com/kubernetes/client-go#compatibility-matrix
    k8s.io/client-go v12.0.0+incompatible
)
2019-08-23 16:17:12 +08:00
Ta-Ching ChenandGitHub 97b9350651 Improve CI integration test script (#1285) 2019-08-22 16:01:53 +08:00
Ta-Ching ChenandGitHub b9997b2ece Add swagger (OpenAPI 2.0) support (#1245)
* Add swagger (OpenAPI 2.0) support
* Gererate swagger_doc before building bundle image
2019-08-21 15:49:41 +08:00
Ta-Ching ChenandGitHub 8cafe1ebf3 Set full URL path to request header (#1266) 2019-08-20 16:44:21 +08:00
Ta-Ching ChenandGitHub 4dccfcc156 Fix go 1.9.2 & 1.11.4 shows "go mod not found" (#1276) 2019-08-20 10:12:48 +08:00
Ta-Ching ChenandGitHub a6e75b4da6 Use fuzzy version for openJDK dependency (#1269) 2019-08-13 16:17:10 +08:00
Ta-Ching ChenandGitHub 7bbcaf2174 Allow to set deployment config uid during initialization (#1249) 2019-08-01 22:53:04 +08:00
Ta-Ching ChenandVishal ac21c037d6 Fix release script tagging wrong image name (#1257) 2019-07-30 14:51:25 +05:30
Ta-Ching ChenandGitHub 3b47daa8fc Update tensorflow serving image name in env README (#1256) 2019-07-30 14:30:16 +08:00
Ta-Ching ChenandGitHub 74ad096158 Fix wrongly replace spec api version (#1254) 2019-07-29 13:49:05 +08:00
Ta-Ching ChenandGitHub 655a76447e Fix CI unable to start test due to the same travis build ID (#1246) 2019-07-25 15:54:12 +08:00
Ta-Ching ChenandGitHub efe1209550 Update flask version to resolve CVE alert (#1243) 2019-07-23 11:08:23 +08:00
Ta-Ching ChenandGitHub 1beaa9ec13 Change log-level for better performance and less annoying logs (#1231)
This PR removes not so useful logs and changes most of Info level
log to Debug/Error level in hot path while preserving some of them
that is helpful for troubleshooting.
2019-07-19 12:38:36 +08:00
Ta-Ching ChenandGitHub 6e00733f68 Fix roundtripper doesn't increase request timeout setting after each retry (#1216) 2019-07-19 00:46:57 +08:00
Ta-Ching ChenandGitHub 971a314d63 Fix fetcher client doesn't handle error properly (#1238) 2019-07-15 21:01:32 +08:00
Ta-Ching ChenandGitHub 40e1689e49 Fix build failed due to script unable to find configmap (#1237) 2019-07-15 17:54:37 +08:00
Ta-Ching ChenandGitHub fe8fa94c77 Enable concurrent CI builds (#1236)
Currently, only one CI build can be started at the same time due to we only have only one testing cluster. This PR aims to enable multiple concurrent builds can be triggered to reduce the waiting time for CI builds. The major changes listed below:

1. Each build generate unique images for testing and debugging
2. Check whether the testing cluster is being used by another build.
3. Cache docker image & go mod after build finished
4. Improve dockerfiles for reusing docker build cache
2019-07-15 16:33:26 +08:00
Ta-Ching ChenandGitHub a9dd4bf81a Fix fetcher of newdeploy pod error out when starting up (#1210)
* Connect to k8s API server to ensure network connectivity
* Fix readiness probe path
2019-07-14 21:09:35 +08:00
Ta-Ching ChenandGitHub 56beac6508 Make router keep-alive setting configurable (#1225) 2019-07-12 12:12:04 +08:00
Ta-Ching ChenandGitHub b0b29da388 Replace localhost with 127.0.0.1 to prevent DNS resolving problem (#1227) 2019-07-12 09:16:10 +08:00
Ta-Ching ChenandGitHub c1cb9603a5 Fix release script wrongly tags builder as env image (#1226) 2019-07-10 16:03:14 +08:00
Ta-Ching ChenandGitHub 58722f849a Enable go module support for go environment (#1152) 2019-07-10 14:00:29 +08:00
Ta-Ching ChenandGitHub 9e7dd5aa27 Fix nil pointer when CLI unable to get server version (#1223) 2019-07-07 09:55:00 +08:00
Ta-Ching ChenandGitHub 9d13081803 Allow to set log level through environment variable (#1217) 2019-07-07 04:39:42 +08:00
Ta-Ching ChenandGitHub 78a774cfd9 Reuse go docker build cache (#1218) 2019-07-06 09:30:27 +08:00
Ta-Ching ChenandGitHub 9f51f9064d Fix poolmanager specializes a function pod repeatedly if istio is enabled (#1208) 2019-07-03 11:26:35 +08:00
Ta-Ching ChenandGitHub b229d6e1b0 Add experimental environment: tensorflow-serving (#1212) 2019-07-02 15:43:55 +08:00
Ta-Ching ChenandGitHub 2ba66afb9f Fix Istio-proxy cannot collect HTTP-level information (#1209) 2019-07-02 00:08:29 +08:00
Ta-Ching ChenandGitHub 981bb21756 Reuse maven build cache in JVM environment build process (#1207) 2019-06-20 17:03:17 +08:00
Ta-Ching ChenandGitHub a297a67385 Fix executor panic problem when specialization timeout (#1211) 2019-06-20 13:56:02 +08:00
Ta-Ching ChenandGitHub e189ac4fc2 Fallback to get user home directory from env (#1203) 2019-06-05 12:56:30 +08:00
Ta-Ching Chen 4bc5d49792 Add ruby/dotnet20 environment builder to release script 2019-06-03 20:57:55 +08:00
Ta-Ching Chen 3860039f9a Update v1.3.0 changelog 2019-06-03 20:15:49 +08:00
Ta-Ching Chen cb8605cf60 Fission 1.3.0 2019-06-03 18:26:34 +08:00
Ta-Ching ChenandGitHub 4c4bb1bc2e Check fission CLI & server git commit SHA before test (#1200)
Check fission CLI & server git commit SHA before test 
to make sure we test against with the same commit.
2019-06-03 14:27:58 +08:00
Ta-Ching ChenandGitHub a3fd606197 Update bug issue templates (#1198) 2019-06-03 08:24:07 +08:00
Ta-Ching ChenandGitHub 93d4f2c77a Add static code analysis to CI test (#1197) 2019-06-02 14:29:38 +08:00
Ta-Ching ChenandGitHub 1562cba420 Add readiness/liveness probes to nat-streaming (#1199) 2019-06-02 13:41:24 +08:00
Ta-Ching ChenandGitHub 1d266537b0 Show warning message if spec alters poolsize while env version < 3 (#1193) 2019-05-31 17:49:56 +08:00
Ta-Ching ChenandGitHub a0e9a39511 Move packages to proejct/pkg to follow go project folder structure convention (#1190) 2019-05-31 16:28:55 +08:00
Ta-Ching ChenandGitHub 56fa8fe766 Remove prometheus server connectivity test during controller initialization (#1179)
The prometheus server may not come up in a short time or when a cluster doesn’t have a storage class config. Because of this, the connectivity test will increase controller initialization time and so the controller will fail at readiness probe detection. Since we don’t want to delay the time that controller comes up also canary config manager prints error log when it failed at connecting to prometheus server, remove the connectivity test here.
2019-05-10 14:25:24 +08:00
Ta-Ching ChenandGitHub 8015ab9a1e Change log level in executor for better log reading/troubleshooting (#1163) 2019-04-15 23:02:55 +08:00
Ta-Ching ChenandGitHub 94046eeffd Fix TravisCI go environment version to avoid go bugs (#1154)
From go release history (https://golang.org/doc/devel/release.html#go1.11)
and travis ci build history, looks like 1.11.7 and 1.11.8 contains unknown
bugs that breaks build. To avoid this, fix go version to 1.11.6 for now.
2019-04-09 16:38:40 +08:00
Ta-Ching ChenandGitHub 1662bafb50 Migrate from glide to official dependencies management tool: Go Module (#1136) 2019-04-04 22:33:26 +08:00
Ta-Ching ChenandGitHub 913888a29d Fix misleading log when setup portforward (#1134) 2019-04-01 17:03:19 +08:00
Ta-Ching ChenandGitHub cdcc6fb3ab Add connection lost handler for NATS-streaming (#1125) 2019-03-19 21:13:49 +08:00
Ta-Ching ChenandGitHub 3a8b9ccd80 Bump base image version of Go environment to 1.11.4 (#1026) 2019-03-19 18:29:19 +08:00
Ta-Ching ChenandVishal 49e0c3587b Fix canary config manager creation error in controller (#1105)
Fixing failure of the controller startup when Prometheus is provided as env variable and it fails to parse correctly
2019-03-06 15:50:09 +05:30
Ta-Ching ChenandGitHub 8b0a201f69 Fix executor tries to create same name deployment (#1082)
The root cause of the issue was introduced by PR https://github.com/fission/fission/pull/1009/files .
To be short, even the CRD of environment was delete, it still takes time for executor (poolmgr) to destroy env pool. In our cases, the previous test creates an env and delete it when test finished, then the next one creates the same name env, but failed to create pool due to the deploy name conflict. So the executor selects the pod from the first created env pool. Then, executor starts to delete the env pool, and makes the pod state became Termination state. To fix this problem, a unique name of deployment will be returned after this PR to prevent the name conflict.
2019-01-30 22:14:14 +08:00
Ta-Ching ChenandGitHub f24b016346 Fix goroutines change DialContext of http.DefaultTransport in router (#1063) 2019-01-26 04:48:10 +08:00
Ta-Ching ChenandGitHub 5ff875b1ac Fix fluentd plugin version to prevent version incompatible problem (#1076) 2019-01-23 20:18:37 +08:00