Add benchmark script (#666 )

Meaningful error message when fetch request is received for a package when build is not successful. (#661 )
Fixed the issue with update wiping original values & added resources in listing, fixes #654 (#663 )
2018-05-05 12:42:16 +08:00 · 2018-05-03 22:51:48 +08:00 · 2018-05-02 22:29:08 +08:00 · 2018-05-01 23:25:50 +08:00 · 2018-05-01 16:48:34 +08:00 · 2018-04-30 14:14:46 +08:00
126 changed files with 3853 additions and 980 deletions
@@ -39,9 +39,11 @@ script:
  - hack/verify-govet.sh
  - hack/runtests.sh
  - test/build_and_test.sh
+  - test/upgrade/fission_upgrade_test.sh

 notifications:
  slack:
-    secure: YZ34vsfw1TtftJypg1MyP4+ihONI4gaxeS3FghBQli6+EezjzcxOXyj5VD+x0ucXfDeTaDrFfmVN0SAGObOoZUE+ea9KAoTo50tRLaD9kwOTACiekZalC4uuBguH0D1/A6vlbU8dchsr9mvIhbisG6mTncdPtqGYYHtyBQme6ngmmHbVAQFZcIBHmNuDb/HWhSr8KMEuyB6+mBXLYELHnXnf26cOhdGNaagqCOJTiemX85RGwIuOPxyBhKDFMLyHohDT7FJMH/qijveE6YgOYTQC5nYc2Np1KvC7hQkIu4nuyczyYlrNQl/TWv+SVI8PjIs0PYuCuD3gUqoVEi8d94HbrOPzFEpbwDS9P4qL39DGmco1Q56Vqxe6sRI0vDWPb5gCP1lSgs3PMECVn7Wor/pTvcL+C+U2jLwWJUl0vbyWCL7ngl/3iTssV7qBpUrI7Oclwp8LrQo9fPj0DL4gE9rNanpEWjQ6yPGaysIL1zLHtRghhm52A22NJGp71jkS2KEpLi6ZWFYjMeuXw5eOQFhqFlzyRJOmLYa3B607TLWuyo2L2CxAfMmq0FGfemvrkLZIWtlQKK4y9ImpsURwaGT2XCtThFtHl77wEss913nC+T2dX3O5Bl0UmxFd5S3mVM109I8c4lDosxnAjRfS9MheFlrG0gjSJSBCw57x7f0=
+    rooms:
+      secure: YZ34vsfw1TtftJypg1MyP4+ihONI4gaxeS3FghBQli6+EezjzcxOXyj5VD+x0ucXfDeTaDrFfmVN0SAGObOoZUE+ea9KAoTo50tRLaD9kwOTACiekZalC4uuBguH0D1/A6vlbU8dchsr9mvIhbisG6mTncdPtqGYYHtyBQme6ngmmHbVAQFZcIBHmNuDb/HWhSr8KMEuyB6+mBXLYELHnXnf26cOhdGNaagqCOJTiemX85RGwIuOPxyBhKDFMLyHohDT7FJMH/qijveE6YgOYTQC5nYc2Np1KvC7hQkIu4nuyczyYlrNQl/TWv+SVI8PjIs0PYuCuD3gUqoVEi8d94HbrOPzFEpbwDS9P4qL39DGmco1Q56Vqxe6sRI0vDWPb5gCP1lSgs3PMECVn7Wor/pTvcL+C+U2jLwWJUl0vbyWCL7ngl/3iTssV7qBpUrI7Oclwp8LrQo9fPj0DL4gE9rNanpEWjQ6yPGaysIL1zLHtRghhm52A22NJGp71jkS2KEpLi6ZWFYjMeuXw5eOQFhqFlzyRJOmLYa3B607TLWuyo2L2CxAfMmq0FGfemvrkLZIWtlQKK4y9ImpsURwaGT2XCtThFtHl77wEss913nC+T2dX3O5Bl0UmxFd5S3mVM109I8c4lDosxnAjRfS9MheFlrG0gjSJSBCw57x7f0=
    on_success: change
    on_failure: always
@@ -1,17 +1,107 @@
-# 0.5.0
-[Documentation](http://fission.io/docs/0.5.0/)
-## Downloads for 0.5.0
+# 0.7.1
+[Documentation](http://fission.io/docs/0.7.1/)
+## Downloads for 0.7.1
+

 filename | sha256 hash
 -------- | -----------
-[fission-core-0.5.0.tgz](https://github.com/fission/fission/releases/download/0.5.0/fission-core-0.5.0.tgz) | `6ee668de0870a52d16a051bb5d60c12e523a87c54269b321bdab6dd97f883ee4`
-[fission-all-0.5.0.tgz](https://github.com/fission/fission/releases/download/0.5.0/fission-all-0.5.0.tgz) | `2f72fe4ba836d84e60538ecf401a508384bb194c979688be69f1370af0c071bb`
-[fission-cli-osx](https://github.com/fission/fission/releases/download/0.5.0/fission-cli-osx) | `1b7de8a5bbf7710ba66997003e7a63f9887ef89d7f7e027c31a78f2f51a11536`
-[fission-cli-linux](https://github.com/fission/fission/releases/download/0.5.0/fission-cli-linux) | `7bbff6d166daf16f6e08b82753cbd6bd0d407e12372a0c6612b6f1a11ed39e09`
-[fission-cli-windows.exe](https://github.com/fission/fission/releases/download/0.5.0/fission-cli-windows.exe) | `5db18472bd071daf39ad681c6ad3b4fd4e89ccb243f81e7ea67785e0c7b2f209`
+[fission-core-0.7.1.tgz](https://github.com/fission/fission/releases/download/0.7.1/fission-core-0.7.1.tgz) | `5d5606c79b5d83c2e88c371b6bea4a49a599f1a1e26674db3388a63278a47c2c`
+[fission-all-0.7.1.tgz](https://github.com/fission/fission/releases/download/0.7.1/fission-all-0.7.1.tgz) | `0ea6887f39c0f59463b49e632294152c926b62b5afc65202c80045ff71735b05`
+[fission-cli-osx](https://github.com/fission/fission/releases/download/0.7.1/fission-cli-osx) | `765b35bd1e4565b9819bed157ca0206a0e5217394d0b253a0cdd9a318ef4d9a5`
+[fission-cli-linux](https://github.com/fission/fission/releases/download/0.7.1/fission-cli-linux) | `5eacdff0a0f3d6d3a1c2aa8d6164bc6524b1f001298cb2e82b60cf1c762a5a9e`
+[fission-cli-windows.exe](https://github.com/fission/fission/releases/download/0.7.1/fission-cli-windows.exe) | `c237b1ee1a8f2b26038555388a438e4fff5725c90d4128baaa32a77ccf990d2e`

 # Change Log

+## [0.7.1](https://github.com/fission/fission/tree/0.7.1) (2018-04-10)
+[Full Changelog](https://github.com/fission/fission/compare/0.7.0...0.7.1)
+
+**Merged pull requests:**
+
+- Prevent releasing idle connections because transport is shared. [\#609](https://github.com/fission/fission/pull/609) ([smruthi2187](https://github.com/smruthi2187))
+- Fix components crash before crds creation [\#602](https://github.com/fission/fission/pull/602) ([life1347](https://github.com/life1347))
+- updates to changelog. [\#598](https://github.com/fission/fission/pull/598) ([smruthi2187](https://github.com/smruthi2187))
+- changes needed for release 0.7.0 [\#597](https://github.com/fission/fission/pull/597) ([smruthi2187](https://github.com/smruthi2187))
+- `fission X create --spec` flags for env and trigger create commands [\#607](https://github.com/fission/fission/pull/607) ([soamvasani](https://github.com/soamvasani))
+- Updating releasing guideliness with a few more details. [\#599](https://github.com/fission/fission/pull/599) ([smruthi2187](https://github.com/smruthi2187))
+- Add deprecated message to subcommand pods [\#592](https://github.com/fission/fission/pull/592) ([life1347](https://github.com/life1347))
+- Add validate function to crd resource and do validate before creation/update [\#580](https://github.com/fission/fission/pull/580) ([life1347](https://github.com/life1347))
+- Invalidate stale router cache entry with podIP's for deleted pods. [\#546](https://github.com/fission/fission/pull/546) ([smruthi2187](https://github.com/smruthi2187))
+- Use a separate controller loop to watch functions change and create a service [\#544](https://github.com/fission/fission/pull/544) ([life1347](https://github.com/life1347))
+- E2E test for NATS-streaming trigger [\#338](https://github.com/fission/fission/pull/338) ([soamvasani](https://github.com/soamvasani))
+
+## [0.7.0](https://github.com/fission/fission/tree/0.7.0) (2018-04-02)
+[Full Changelog](https://github.com/fission/fission/compare/0.6.1...0.7.0)
+
+**Merged pull requests:**
+
+- bug fix: spec dir flag [\#595](https://github.com/fission/fission/pull/595) ([xiekeyang](https://github.com/xiekeyang))
+- Add steps to set FISSION\_ROUTER env variable & update docs [\#593](https://github.com/fission/fission/pull/593) ([life1347](https://github.com/life1347))
+- Adding routerUrl parameter for kubewatch, timer, message queue trigge… [\#591](https://github.com/fission/fission/pull/591) ([smruthi2187](https://github.com/smruthi2187))
+- Uses proper way to get server URL [\#587](https://github.com/fission/fission/pull/587) ([vishal-biyani](https://github.com/vishal-biyani))
+- Check if the requested file already exists in fetcher and skip fetch [\#584](https://github.com/fission/fission/pull/584) ([smruthi2187](https://github.com/smruthi2187))
+- Add golang example to installation guide [\#578](https://github.com/fission/fission/pull/578) ([clee](https://github.com/clee))
+- Fixes the issue \#559 with env versions [\#569](https://github.com/fission/fission/pull/569) ([vishal-biyani](https://github.com/vishal-biyani))
+- Add post-upgrade-job to track fission upgrade [\#564](https://github.com/fission/fission/pull/564) ([life1347](https://github.com/life1347))
+- Prepending a slash to user input url if missing. [\#547](https://github.com/fission/fission/pull/547) ([smruthi2187](https://github.com/smruthi2187))
+- Add verbosity flag and verbose logs for portforwarder [\#575](https://github.com/fission/fission/pull/575) ([soamvasani](https://github.com/soamvasani))
+- Spec validator, better errors, apply waits for previous build  [\#560](https://github.com/fission/fission/pull/560) ([soamvasani](https://github.com/soamvasani))
+- Tests for function update [\#550](https://github.com/fission/fission/pull/550) ([vishal-biyani](https://github.com/vishal-biyani))
+- Show fission deployment version with cli [\#538](https://github.com/fission/fission/pull/538) ([life1347](https://github.com/life1347))
+
+## [0.6.1](https://github.com/fission/fission/tree/0.6.1) (2018-03-22)
+[Full Changelog](https://github.com/fission/fission/compare/0.6.0...0.6.1)
+
+**Merged pull requests:**
+
+- This change fixes an error in a yaml file in the fission-core chart. [\#563](https://github.com/fission/fission/pull/563) ([yiqunding](https://github.com/yiqunding))
+- \[ci skip\] update release number [\#561](https://github.com/fission/fission/pull/561) ([appleboy](https://github.com/appleboy))
+- Fixes \#537 - warning should not be given when updating to newdeploy [\#545](https://github.com/fission/fission/pull/545) ([vishal-biyani](https://github.com/vishal-biyani))
+- Docs update [\#542](https://github.com/fission/fission/pull/542) ([soamvasani](https://github.com/soamvasani))
+- Release script updates [\#541](https://github.com/fission/fission/pull/541) ([soamvasani](https://github.com/soamvasani))
+- Show warning when trying to create a route with non-existent function \(\#238\) [\#539](https://github.com/fission/fission/pull/539) ([life1347](https://github.com/life1347))
+- Fix executor failed to clean cache & kubeobjs after function deleted \(\#533\) [\#534](https://github.com/fission/fission/pull/534) ([life1347](https://github.com/life1347))
+- Delete healthz log [\#525](https://github.com/fission/fission/pull/525) ([smruthi2187](https://github.com/smruthi2187))
+- Always retry when istio is enabled. [\#536](https://github.com/fission/fission/pull/536) ([life1347](https://github.com/life1347))
+- Fix executor tries to create a new deployment when a function is updated [\#524](https://github.com/fission/fission/pull/524) ([life1347](https://github.com/life1347))
+- Add container spec config options  to \(build\) environments [\#413](https://github.com/fission/fission/pull/413) ([erwinvaneyk](https://github.com/erwinvaneyk))
+
+## [0.6.0](https://github.com/fission/fission/tree/0.6.0) (2018-03-01)
+[Full Changelog](https://github.com/fission/fission/compare/latest...0.6.0)
+
+**Merged pull requests:**
+
+- Detect fission namespace in cli [\#519](https://github.com/fission/fission/pull/519) ([soamvasani](https://github.com/soamvasani))
+- Default values for FISSION\_\* env vars [\#518](https://github.com/fission/fission/pull/518) ([soamvasani](https://github.com/soamvasani))
+- Add chart version to job name [\#516](https://github.com/fission/fission/pull/516) ([soamvasani](https://github.com/soamvasani))
+- Fix CLI not update function's secret/configmap correctly [\#512](https://github.com/fission/fission/pull/512) ([life1347](https://github.com/life1347))
+- Adds latest tags and pushes to dockerhub for fetcher and fission-bundle [\#509](https://github.com/fission/fission/pull/509) ([vishal-biyani](https://github.com/vishal-biyani))
+- Fixes the backward compatibility with older environment versions [\#508](https://github.com/fission/fission/pull/508) ([vishal-biyani](https://github.com/vishal-biyani))
+- Update Fn: Executor New Deployment [\#504](https://github.com/fission/fission/pull/504) ([vishal-biyani](https://github.com/vishal-biyani))
+- Adds default resources for fetcher pod [\#500](https://github.com/fission/fission/pull/500) ([vishal-biyani](https://github.com/vishal-biyani))
+- Documentation Revamp [\#496](https://github.com/fission/fission/pull/496) ([vishal-biyani](https://github.com/vishal-biyani))
+- Delete and list orphan pkgs [\#468](https://github.com/fission/fission/pull/468) ([smruthi2187](https://github.com/smruthi2187))
+- Service type ClusterIP - Controller port forward through CLI [\#431](https://github.com/fission/fission/pull/431) ([prithviramesh](https://github.com/prithviramesh))
+- Istio integration [\#421](https://github.com/fission/fission/pull/421) ([life1347](https://github.com/life1347))
+- Implement support for Azure storage message queue triggers [\#371](https://github.com/fission/fission/pull/371) ([peterhuene](https://github.com/peterhuene))
+
+## [latest](https://github.com/fission/fission/tree/latest) (2018-03-01)
+[Full Changelog](https://github.com/fission/fission/compare/0.5.0...latest)
+
+**Merged pull requests:**
+
+- Release checklist [\#522](https://github.com/fission/fission/pull/522) ([soamvasani](https://github.com/soamvasani))
+- Fix post-install-job container failure due to command not found [\#514](https://github.com/fission/fission/pull/514) ([life1347](https://github.com/life1347))
+- Replace the release with the latest tag. [\#513](https://github.com/fission/fission/pull/513) ([smruthi2187](https://github.com/smruthi2187))
+- Go: Set image to right version, update example readme [\#497](https://github.com/fission/fission/pull/497) ([soamvasani](https://github.com/soamvasani))
+- Remove a noisy log from router [\#495](https://github.com/fission/fission/pull/495) ([soamvasani](https://github.com/soamvasani))
+- Improve release script [\#494](https://github.com/fission/fission/pull/494) ([life1347](https://github.com/life1347))
+- Update SHA256 HASH in CHANGELOG.md due to binaries update [\#493](https://github.com/fission/fission/pull/493) ([life1347](https://github.com/life1347))
+- Go builder for single file functions [\#492](https://github.com/fission/fission/pull/492) ([soamvasani](https://github.com/soamvasani))
+- CI modifications [\#491](https://github.com/fission/fission/pull/491) ([smruthi2187](https://github.com/smruthi2187))
+- Add upgrade guide from 0.4.x to 0.5.0 [\#490](https://github.com/fission/fission/pull/490) ([life1347](https://github.com/life1347))
+- Version -\> 0.5.0 [\#489](https://github.com/fission/fission/pull/489) ([life1347](https://github.com/life1347))
+
 ## [0.5.0](https://github.com/fission/fission/tree/0.5.0) (2018-02-07)
 [Full Changelog](https://github.com/fission/fission/compare/0.4.1...0.5.0)

@@ -31,6 +121,7 @@ filename | sha256 hash
 - NewDeploy Doc [\#432](https://github.com/fission/fission/pull/432) ([vishal-biyani](https://github.com/vishal-biyani))
 - Add go vet check [\#430](https://github.com/fission/fission/pull/430) ([life1347](https://github.com/life1347))
 - Fix potential nil pointer problem [\#485](https://github.com/fission/fission/pull/485) ([life1347](https://github.com/life1347))
+- Add simple usage doc for accessing secret/configmap in function [\#484](https://github.com/fission/fission/pull/484) ([life1347](https://github.com/life1347))
 - Helm hook bugfixes: run on upgrade, delete on completion [\#473](https://github.com/fission/fission/pull/473) ([soamvasani](https://github.com/soamvasani))
 - Archive pruner [\#471](https://github.com/fission/fission/pull/471) ([smruthi2187](https://github.com/smruthi2187))
 - Build and push fluentd image on release; update chart to use that image [\#462](https://github.com/fission/fission/pull/462) ([soamvasani](https://github.com/soamvasani))
@@ -236,7 +327,7 @@ filename | sha256 hash
 - Upgrade node environment to Node.js 7.6.0+ [\#151](https://github.com/fission/fission/pull/151) ([RobertHerhold](https://github.com/RobertHerhold))
 - use fmt.Errorf instead of error.New\(\) [\#149](https://github.com/fission/fission/pull/149) ([maxwell92](https://github.com/maxwell92))
 - Return 201 for created resources [\#148](https://github.com/fission/fission/pull/148) ([RobertHerhold](https://github.com/RobertHerhold))
- Set correct Content-Type in the http response [\#147](https://github.com/fission/fission/pull/147) ([LingxianKong](https://github.com/LingxianKong))
+- Set correct Content-Type in the http response [\#147](https://github.com/fission/fission/pull/147) ([lingxiankong](https://github.com/lingxiankong))
 - Make it more clear where to clone this repo [\#145](https://github.com/fission/fission/pull/145) ([RobertHerhold](https://github.com/RobertHerhold))
 - Fix function delete with uid [\#142](https://github.com/fission/fission/pull/142) ([yqf3139](https://github.com/yqf3139))
 - Fixed pod has no ip \(\#139\) [\#141](https://github.com/fission/fission/pull/141) ([life1347](https://github.com/life1347))
@@ -1,4 +1,4 @@
-baseURL = "http://fission.io/docs/0.5.0"
+baseURL = "http://fission.io/docs/0.6.1"
 languageCode = "en-US"
 defaultContentLanguage = "en"

@@ -92,8 +92,8 @@ Usage
  $ fission route create --method GET --url /hello --function hello

  # Run the function.  This takes about 100msec the first time.
-  $ curl http://$FISSION_ROUTER/hello
+  $ fission function test --name hello
  Hello, world!
 ```

-See the [examples](examples) directory for more.
+See the [examples](examples) directory for more.
@@ -134,6 +134,19 @@ $ fission function test --name hello
 Hello, world!
 ```

+For a compiled language like Go:
+
+```sh
+$ fission env create --name go --image fission/go-env:0.6.0 --builder fission/go-builder:0.6.0
+
+$ curl -LO https://raw.githubusercontent.com/fission/fission/master/examples/go/hello.go
+
+$ fission function create --name gohello --env go --src hello.go --entrypoint Handler
+
+$ fission function test --name gohello
+Hello, world!
+```
+
 ### What's next?

 If something went wrong, we'd love to help -- please [drop by the
@@ -9,7 +9,7 @@ weight: 42
 You can create an environment on your cluster from an image for that language. Optionally, you can specify CPU and memory resource limits. You can also specify the number of initially pre-warmed pods, which is called the poolsize.

 ```
-fission env create --name node --image fission/node-env:0.4.0 --mincpu 40 --maxcpu 80 --minmemory 64 --maxmemory 128 --poolsize 4
+$ fission env create --name node --image fission/node-env:0.4.0 --mincpu 40 --maxcpu 80 --minmemory 64 --maxmemory 128 --poolsize 4
 ```

 In case of pool based executor, the resources specified for environment are used for function pod as well. In case of new deployment executor, you can override the resources when you create a function.
@@ -19,7 +19,7 @@ In case of pool based executor, the resources specified for environment are used
 When you create an environment, you can specify a builder image and builder command which will be used for building from source code. You can override the build command when creating a function. For more details on builder and packages you should check out examples in [Functions](../functions) and [packages](../package)

 ```
-fission env create --name python --image fission/python-env:latest --builder fission/python-builder:latest
+$ fission env create --name python --image fission/python-env:latest --builder fission/python-builder:latest
 ```

 ### Viewing environment information
@@ -30,7 +30,7 @@ You can list the environments or view information of an individual environment:
 $ fission env list
 NAME UID                                  IMAGE                  POOLSIZE MINCPU MAXCPU MINMEMORY MAXMEMORY
 node ac84d62e-001f-11e8-85c9-42010aa00010 fission/node-env:0.4.0 4        40m    80m    64Mi      128Mi
-$ 
+
 $ fission env get --name node
 NAME UID                                  IMAGE
 node ac84d62e-001f-11e8-85c9-42010aa00010 fission/node-env:0.4.0
@@ -62,7 +62,7 @@ Status code distribution:
 While the load is being generated, we will watch the HorizontalPodAutoscaler and how it scales over period of time. As you can notice, the number of pods is scaled from 1 to 3 after the load rises from 8 - 103%. After the load generator stops, it takes a few iterations to scale down from 3 to 1 pod. 

 ```
-$ k -n fission-function get hpa -w
+$ kubectl -n fission-function get hpa -w
 NAME             REFERENCE                   TARGETS    MINPODS   MAXPODS   REPLICAS   AGE
 hello-qoxmothj   Deployment/hello-qoxmothj   5% / 50%   1         6         1          3m
 hello-qoxmothj   Deployment/hello-qoxmothj   8% / 50%   1         6         1         3m
@@ -83,4 +83,4 @@ hello-qoxmothj   Deployment/hello-qoxmothj   7% / 50%   1         6         1
 hello-qoxmothj   Deployment/hello-qoxmothj   6% / 50%   1         6         1         11m
 hello-qoxmothj   Deployment/hello-qoxmothj   6% / 50%   1         6         1         12m
 hello-qoxmothj   Deployment/hello-qoxmothj   6% / 50%   1         6         1         12m
-```
+```
@@ -10,7 +10,7 @@ Before creating a function the environment should be created, we will assume tha

 Let's create a simple code snippet in nodejs which will output Hello world:

-```
+``` js
 module.exports = async function(context) {
    return {
        status: 200,
@@ -19,36 +19,65 @@ module.exports = async function(context) {
 }
 ```

+Let's create a function based on pool based executor.
+
+``` bash
+$ fission fn create --name hello --code hello.js --env node --executortype poolmgr
+```
+
+### Access function
+
+Before accessing function, we need to set the `FISSION_ROUTER` environment variable first. It is needed for the examples below to work
+
+#### Minikube
+
+If you're using minikube, use these commands:
+
+``` bash
+$ export FISSION_ROUTER=$(minikube ip):$(kubectl -n fission get svc router -o jsonpath='{...nodePort}')
+```
+
+#### Cloud setups
+
+Save the external IP addresses of router services in FISSION_ROUTER.
+
+Wait for services to get IP addresses (check this with `kubectl --namespace fission get svc`).  Then:
+
+``` bash
+# AWS
+$ export FISSION_ROUTER=$(kubectl --namespace fission get svc router -o=jsonpath='{..hostname}')
+
+# GCP
+$ export FISSION_ROUTER=$(kubectl --namespace fission get svc router -o=jsonpath='{..ip}')
+```
+
+#### Create a HTTP trigger
+
 Let's create a route for the function which can be used for making HTTP requests:

-```
+``` bash
 $ fission route create --function hello --url /hello
 trigger '5327e9a7-6d87-4533-a4fb-c67f55b1e492' created
 ```

-Let's create a function based on pool based executor.
-
-```
-fission fn create --name hello --code hello.js --env node --executortype poolmgr
-```
-
 When you hit this function's URL , you get a response:

-```
+``` bash
 $ curl http://$FISSION_ROUTER/hello
 Hello, world!
 ```
+
 Similarly you can create a new deployment executor type function and provide minmum and maximum scale for the function. 

-```
-fission fn create --name hello --code hello.js --env node --minscale 1 --maxscale 5  --executortype newdeploy
+``` bash
+$ fission fn create --name hello --code hello.js --env node --minscale 1 --maxscale 5  --executortype newdeploy
 ```

 ### View & update function source code

 You can look at the source code associated with given function:

-```
+``` bash
 $ fission fn get --name hello
 module.exports = async function(context) {
    return {
@@ -60,7 +89,7 @@ module.exports = async function(context) {

 Let's say you want to update the function to output "Hello Fission" instead of "Hello world", you can update the source file and update the source code for function:

-```
+``` bash
 $ fission fn update --name hello --code ../hello.js 
 package 'hello-js-ku9s' updated
 function 'hello' updated
@@ -68,7 +97,7 @@ function 'hello' updated

 Let's verify that the function now respond with a different output than earlier:

-```
+``` bash
 $ curl http://$FISSION_ROUTER/hello
 Hello, Fission!
 ```
@@ -76,13 +105,13 @@ Hello, Fission!
 ### Test and debug function

 You can directly test a function using test command. If the function call succeeds, it will output the function's response. 
-```
+``` bash
 $ fission fn test --name hello
 Hello, Fission!
 ```

 But if there is an error in function execution then the logs of function execution are displayed:
-```
+``` bash
 $ fission fn test --name hello
 Error calling function hello: 500 Internal server error (fission)

@@ -97,7 +126,7 @@ user code load error: SyntaxError: Unexpected token function
 ```

 You can also look at function execution logs explicitly:
-```
+``` bash
 $ fission fn logs --name hello
 [2018-02-16 08:41:43 +0000 UTC] 2018/02/16 08:41:43 fetcher received fetch request and started downloading: {1 {hello-js-rqew  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] nil [] }   user [] []}
 [2018-02-16 08:41:43 +0000 UTC] 2018/02/16 08:41:43 Successfully placed at /userfunc/user
@@ -119,15 +148,17 @@ You can attach the source/deployment packages to a function or explicitly create

 Let's take a simple python function which has dependency on a python pyyaml module. We can specify the dependencies in requirements.txt and a simple command to build from source. The tree structure of directory looks like:

-```
+``` bash
 sourcepkg/
 ├── __init__.py
 ├── build.sh
 ├── requirements.txt
 └── user.py
 ```
+
 And the file contents:
-```
+
+``` bash
 $ cat user.py 
 import sys
 import yaml
@@ -152,13 +183,14 @@ pip3 install -r ${SRC_PKG}/requirements.txt -t ${SRC_PKG} && cp -r ${SRC_PKG} ${

 You first need to create an environment with environment image and python-builder image specified:

+``` bash
+$ fission env create --name python --image fission/python-env:latest --builder fission/python-builder:latest --mincpu 40 --maxcpu 80 --minmemory 64 --maxmemory 128 --poolsize 2
 ```
-$fission env create --name python --image fission/python-env:latest --builder fission/python-builder:latest --mincpu 40 --maxcpu 80 --minmemory 64 --maxmemory 128 --poolsize 2
-```
+
 Now let's zip the directory containing the source files and create a function with source package:

-```
-$zip -jr demo-src-pkg.zip sourcepkg/
+``` bash
+$ zip -jr demo-src-pkg.zip sourcepkg/
  adding: __init__.py (stored 0%)
  adding: build.sh (deflated 24%)
  adding: requirements.txt (stored 0%)
@@ -169,10 +201,11 @@ function 'hellopy' created

 $ fission route create --function hellopy --url /hellopy
 ```
+
 Once we create the function, the build process is started. You can check logs of the builder in fission-builder namespace:

-```
-$ k -n fission-builder logs -f py3-4214348-59555d9bd8-ks7m4 builder
+``` bash
+$ kubectl -n fission-builder logs -f py3-4214348-59555d9bd8-ks7m4 builder
 2018/02/16 11:44:21 Builder received request: {demo-src-pkg-zip-ninf-djtswo ./build.sh}
 2018/02/16 11:44:21 Starting build...

@@ -189,8 +222,8 @@ Successfully installed pyyaml-3.12
 ```

 Once the build has succeeded, you can hit the function URL to test the function:
-```
-$curl http://$FISSION_ROUTER/hellopy
+``` bash
+$ curl http://$FISSION_ROUTER/hellopy
 a: 1
 b: {c: 3, d: 4}
 ```
@@ -201,7 +234,7 @@ In some cases you have a pre-built deployment package which you need to deploy t

 We will use a simple python file in a directory and turn it into a deployment package:

-```
+``` bash
 $ cat testDir/hello.py
 def main():
    return "Hello, world!"
@@ -209,9 +242,10 @@ def main():
 $zip -jr demo-deploy-pkg.zip testDir/

 ```
+
 Let's use the deployment package to create a function and route and then test it.

-```
+``` bash
 $ fission fn create --name hellopy --env python --deploy demo-deploy-pkg.zip --entrypoint "hello.main"
 function 'hellopy' created

@@ -225,7 +259,7 @@ Hello, world!

 You can retrieve metadata information of a single function or list all functions to look at basic information of functions:

-```
+``` bash
 $ fission fn getmeta --name hello
 NAME  UID                                  ENV
 hello 34234b50-12f5-11e8-85c9-42010aa00010 node
@@ -234,5 +268,4 @@ $ fission fn list
 NAME   UID                                  ENV  EXECUTORTYPE MINSCALE MAXSCALE TARGETCPU
 hello  34234b50-12f5-11e8-85c9-42010aa00010 node poolmgr      0        1        80
 hello2 e37a46e3-12f4-11e8-85c9-42010aa00010 node newdeploy    1        5        80
-
-```
+```
@@ -123,7 +123,7 @@ Build Logs:
 Finally you can create a function with the package and test the function:

 ```
-$fission fn create --name deploypy --pkg demo-deploy-pkg-zip-whzl --entrypoint "hello.main"
+$ fission fn create --name deploypy --pkg demo-deploy-pkg-zip-whzl --entrypoint "hello.main"

 $curl http://$FISSION_ROUTER/deploypy
 Hello, world!
@@ -0,0 +1,209 @@
+# Testing Proposal
+
+This proposal was initially started as a upgrade testing proposal but soon problems that were posed resulted in a bigger proposal.
+### Kinds of testing
+
+Most of current integration tests are CLI driven. Fission CLI is used to test execute various test cases. In future we would have to also focus on API level testing as a UI is built for Fission. 
+## Needs & patterns
+
+This section only explains the problems/best practices without going into tooling and language used for implementation.
+### Separating the test & data
+
+Seperating the tests from test data has two aspects - one is separation of concerns and second is scaling the tests without touching the test logic. The test data is a simple data structure which holds all information and test can take data and execute the logic.
+
+As an example today we test "Hello world" for nodejs environment with a simple hello.js like this:
+
+```
+fission env create --name nodejs --image fission/node-env
+fission fn create --name $fn --env nodejs --code $ROOT/examples/nodejs/hello.js
+fission route create --function $fn --url /$fn --method GET
+response=$(curl http://$FISSION_ROUTER/$fn)
+```
+The variables here are environment image, function code & route URL.
+
+If tomorrow if we had to scale this test for all environments, we will have to repeat ourselves. (Violate DRY principle). Instead of that if we encapsulate the test setup & test in a simple function:
+
+```
+test_hello_env(envImage, codePath, routeURL){
+
+}
+```
+
+And feed it with a dictionary which has all possible combination of tests:
+
+```
+{
+    node: {"fission/node-env", "test/hello.js", "/hellonode"},
+    python: {"fission/python-env", "test/hello.py", "/hellopy"},
+    golang: {"fission/go-env", "test/hello.go", "/hellogo"},
+    binray: {"fission/binary-env", "test/hello.sh", "/hellobinary"},
+}
+
+```
+This would achieve a few things:
+
+- Separate the test execution logic from the data it needs clearly.
+- For adding new kind of environments, you just need to add one more entry into data structure.
+
+Testing all environments may not be most apt example for this, but there can be potential use cases like this.
+### Separating the test & setup/teardown
+
+When we run a test there are typically three distinct phases:
+
+- Setup (Create env, fn, route)
+- Test (Curl the function)
+- Cleanup (Delete fn, route & env)
+
+It should be possible to separate the before test and after test parts from actual tests at two levels:
+- Each test
+- A whole test suite
+
+The ability to have clean and separate before and after blocks, apart from separation of concerns, enables:
+
+- Running a suite of tests for same setup (See tagging for suite of tests)
+### Tagging tests & running a selection
+
+Over a period of time as tests grow, there will be unit, smkoe, integration, performance, soak tests and so on. Ability to run a perticular test suite only or a combination of them makes it easy to run for specific purpose. 
+### Measuring test times
+
+[Good to have, not a must] Measuring time for tests and reporting somewhere helps over time to monitor trends. Although this job is better done by performance/benchmark tests so it is not a strict requirement
+
+### Cleaner Logging
+It would be good to have cleaner/relevant logging as part of build & test. For example something that Ginkgo framework does is it shows error logs only for failed tests.
+### Tests in Parallel 
+
+It would be good to be able to run tests in parallel.
+
+## Evalutaing the tools/alternatives
+### BATS
+
+Bash Automated Testing System is like a enhanced version of bash with support for @test tags and before and after steps & ability to skip tests etc. While it enhances the bash to certain extent, the overall improvement is only marginal.
+
+```
+#!/usr/bin/env bats
+
+@test "addition using bc" {
+  result="$(echo 2+2 | bc)"
+  [ "$result" -eq 4 ]
+}
+
+$ bats addition.bats
+ ✓ addition using bc
+ ✓ addition using dc
+
+2 tests, 0 failures
+
+```
+#### Links
+- Bats repo: https://github.com/sstephenson/bats
+- Runc uses Bats https://github.com/opencontainers/runc/tree/master/tests/integration
+### Go Test
+
+The testing package of Go also is quite feature rich for most of the use cases we need. Go 1.7 onwards there is support for setup & teardown parts and parallelism etc. 
+#### Go - Testing package
+
+- Support for setup and teardown based on https://golang.org/pkg/testing/#hdr-Main
+- Go testing already supports and has examples of table driven tests (Separating test & data), measuring test times and parallel tests
+#### Shell execution: Go's Exec Library
+GO provides a built in Exec library for working with CLI commands. The package seems good enough for us to work, though a few working examples will help decide better
+
+https://golang.org/pkg/os/exec 
+
+### Using CLI package
+
+Currently we build a CLI and then execute the tests. The tests basically call one of functions from CLI package. If we decide to use a go lang based framework, then we can import the CLI package and then call those functions  by providing them context. This is as good as calling the Fission from CLI, with added benefit of programmibility of Go langugage.
+
+```
+func TestSomething(t *testing.T) {
+    // Build the Cli context with flags etc.
+    ctx := cli.Context{}
+
+    // Pass the ctx to create function
+    fnCreate(ctx)
+}
+```
+
+Some of benefits of using above pattern are:
+- We can build a small framework around above core where we can pass various flag combinations etc. and exercise all flags in great detail
+- We can use rest of Go testing library and other libraries to build matchers, looping, parallelism etc.
+- It allows us to exercise the logic in CLI as well as validate the API at the same time.
+
+### Ginkgo & Gomega
+
+Ginkgo is a BDD framework which works with Gomega matcher library. I will state relevant portions of these two frameworks which can be utilized:
+
+From Ginkgo:
+
+- Global `BeforeSuite` and after `AfterSuite` can be used to have global setup and tear down phases
+- For tests `BeforeEach` and `AfterEach` and more such varients to do before and after test tasks.
+
+From Gomega:
+
+Gomega is a matcher library but the `gexec` library makes it really easy to interact with OS execution environment. Some working examples:
+
+- Build and cleanup the Fission CLI before & after the tests
+```
+var fissionCli string
+
+BeforeSuite(func() {
+    var err error
+    fissionCli, err = gexec.Build("github.com/fission/fission")
+    Ω(err).ShouldNot(HaveOccurred())
+})
+
+AfterSuite(func() {
+    gexec.CleanupBuildArtifacts()
+})
+```
+
+- Following will run Fission commands with Fission CLI and print error if there is one (verbosity is configurable)
+
+```
+command := exec.Command(fissionCli, "fission env create --name nodejs --image fission/node-env")
+session, err := gexec.Start(command, GinkgoWriter, GinkgoWriter)
+Ω(err).ShouldNot(HaveOccurred())
+```
+
+- Use Fission CLI's output to validate test results
+
+```
+Eventually(session.Out).Should(gbytes.Say("hello [A-Za-z], world"))
+```
+#### Gomega Matchers
+
+- Gomega provides quite a few built in matchers - so you don't have to code those small usual checks, for example:
+```
+Ω(ACTUAL).Should(BeTrue()) // The output should be true
+
+Ω(ACTUAL).Should(BeAnExistingFile()) //  The file should already exist
+
+```
+There are many more matchers which cane be found here: http://onsi.github.io/gomega/#provided-matchers 
+
+- We can build custom mathers in Go language for reusable logic.
+
+#### Links
+Ginkgo: http://onsi.github.io/ginkgo/
+Gomega: http://onsi.github.io/gomega/
+## Thoughts & Next actions
+
+Based on the discussion with team, here are current thoughts and next action items:
+
+### Thoughts
+- As far as possible we should stick to Go's built in testing package
+- Ginkgo's cleaner logging feature (Onlu log if there are errors) - is very useful. We can decide to incorporate this in future.
+- Gomega's (gexec)[http://onsi.github.io/gomega/#gexec-testing-external-processes] is really neat and some matchers can be used if necessary
+
+### Action items
+- How will upgrade test for Fission fit in the framework?
+- How will migration of tests happen over time:
+  - Aim is to keep existing tests around so that enough validation is in place
+  - May be migrate one test at a time
+  - How much of current setup etc. will move into framework? For example it is clear that helm commands should be part of test framework as part of setup/teardown. But other sections may or may not be. A RCA needs to be done to analyze and come up with clear demarkation.
+
+## References
+
+- EngineYard uses BATS: https://www.engineyard.com/blog/bats-test-command-line-tools
+- AWS CLI Tests, written in Python (CLI itself is also in Python): https://github.com/aws/aws-cli/tree/develop/tests
+- Kubernetes uses Ginkgo and Gomega extensively: https://github.com/kubernetes/kubernetes/search?l=Go&q=onsi&type=
+- Hashicorp's Mitchell's talk on Advanced testing with go talks about some good patterns to use: https://www.youtube.com/watch?v=8hQG7QlcLBk 
@@ -88,7 +88,7 @@ Usage
  $ fission route create --method GET --url /hello --function hello

  # Run the function.  This takes about 100msec the first time.
-  $ curl http://$FISSION_ROUTER/hello
+  $ fission function test --name hello
  Hello, world!
 ```

@@ -97,12 +97,12 @@ See the [examples](examples) directory for more.
 Running Fission on your Cluster
 ===============================

-See the [installation guide](http://fission.io/docs/latest/installation/).
+See the [installation guide](http://docs.fission.io/latest/installation).

 Compiling Fission
 =================

-See the [compilation guide](http://fission.io/docs/latest/compilation/).
+See the [compilation guide](http://docs.fission.io/latest/contributing/compiling/).

 Status
 ======
@@ -31,6 +31,11 @@ func Start(storageSvcUrl string, envBuilderNamespace string) error {
 		return err
 	}

+	err = fissionClient.WaitForCRDs()
+	if err != nil {
+		log.Fatalf("Error waiting for CRDs: %v", err)
+	}
+
 	envWatcher := makeEnvironmentWatcher(fissionClient, kubernetesClient, envBuilderNamespace)
 	go envWatcher.watchEnvironments()

@@ -45,6 +45,11 @@ const (
 	LABEL_ENV_RESOURCEVERSION = "envResourceVersion"
 )

+var (
+	deletePropagation = metav1.DeletePropagationBackground
+	delOpt            = metav1.DeleteOptions{PropagationPolicy: &deletePropagation}
+)
+
 type (
 	builderInfo struct {
 		envMetadata *metav1.ObjectMeta
@@ -341,17 +346,9 @@ func (envw *environmentWatcher) deleteBuilderService(sel map[string]string) erro
 	}
 	for _, svc := range svcList {
 		log.Printf("Removing builder service: %v", svc.ObjectMeta.Name)
-
-		// cascading deletion
-		// https://kubernetes.io/docs/concepts/workloads/controllers/garbage-collection/
-		falseVal := false
-		delOpt := &metav1.DeleteOptions{
-			OrphanDependents: &falseVal,
-		}
-
 		err = envw.kubernetesClient.
 			Services(envw.builderNamespace).
-			Delete(svc.ObjectMeta.Name, delOpt)
+			Delete(svc.ObjectMeta.Name, nil)
 		if err != nil {
 			return fmt.Errorf("Error deleting builder service: %v", err)
 		}
@@ -366,15 +363,9 @@ func (envw *environmentWatcher) deleteBuilderDeployment(sel map[string]string) e
 	}
 	for _, deploy := range deployList {
 		log.Printf("Removing builder deployment: %v", deploy.ObjectMeta.Name)
-
-		falseVal := false
-		delOpt := &metav1.DeleteOptions{
-			OrphanDependents: &falseVal,
-		}
-
 		err = envw.kubernetesClient.ExtensionsV1beta1().
 			Deployments(envw.builderNamespace).
-			Delete(deploy.ObjectMeta.Name, delOpt)
+			Delete(deploy.ObjectMeta.Name, &delOpt)
 		if err != nil {
 			return fmt.Errorf("Error deleteing builder deployment: %v", err)
 		}
@@ -67,28 +67,28 @@ func makePackageWatcher(fissionClient *crd.FissionClient, getter k8sCache.Getter
 // 5. Update package resource in package ref of functions that share the same package
 // 6. Update package status to succeed state
 // *. Update package status to failed state,if any one of steps above failed/time out
-func (pkgw *packageWatcher) build(buildCache *cache.Cache, pkg *crd.Package) {
+func (pkgw *packageWatcher) build(buildCache *cache.Cache, srcpkg *crd.Package) {

 	// Ignore non-pending state packages.
-	if pkg.Status.BuildStatus != fission.BuildStatusPending {
+	if srcpkg.Status.BuildStatus != fission.BuildStatusPending {
 		return
 	}

 	// Ignore duplicate build requests
-	key := fmt.Sprintf("%v-%v", pkg.Metadata.Name, pkg.Metadata.ResourceVersion)
-	err, _ := buildCache.Set(key, pkg)
+	key := fmt.Sprintf("%v-%v", srcpkg.Metadata.Name, srcpkg.Metadata.ResourceVersion)
+	err, _ := buildCache.Set(key, srcpkg)
 	if err != nil {
 		return
 	}
 	defer buildCache.Delete(key)

-	log.Printf("Start build for package %v with resource version %v", pkg.Metadata.Name, pkg.Metadata.ResourceVersion)
+	log.Printf("Start build for package %v with resource version %v", srcpkg.Metadata.Name, srcpkg.Metadata.ResourceVersion)

-	pkg, err = updatePackage(pkgw.fissionClient, pkg, fission.BuildStatusRunning, "", nil)
+	pkg, err := updatePackage(pkgw.fissionClient, srcpkg, fission.BuildStatusRunning, "", nil)
 	if err != nil {
 		e := fmt.Sprintf("Error setting package pending state: %v", err)
 		log.Println(e)
-		updatePackage(pkgw.fissionClient, pkg, fission.BuildStatusFailed, e, nil)
+		updatePackage(pkgw.fissionClient, srcpkg, fission.BuildStatusFailed, e, nil)
 		return
 	}

@@ -1,6 +1,6 @@
 apiVersion: v1
 name: fission-all
-version: 0.6.1
+version: 0.7.1
 description: Fission is a fast serverless framework for Kubernetes.
 keywords:
 - fission
@@ -12,4 +12,4 @@ maintainers:
  - name: Ta Ching Chen
    email: contact@tachingchen.com
 engine: gotpl
-appVersion: 0.6.1
+appVersion: 0.7.1
@@ -1,13 +1,13 @@
 1. Install the client CLI.

 Mac:
-  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.6.1/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
+  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.7.1/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/

 Linux:
-  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.6.1/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
+  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.7.1/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/

 Windows:
-  For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/0.6.1/fission-cli-windows.exe
+  For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/0.7.1/fission-cli-windows.exe

 2. You're ready to use Fission!

@@ -139,8 +139,6 @@ spec:
        env:
          - name: FISSION_FUNCTION_NAMESPACE
            value: "{{ .Values.functionNamespace }}"
-          - name: ENABLE_ISTIO
-            value: "{{ .Values.enableIstio }}"
        readinessProbe:
          httpGet:
            path: "/healthz"
@@ -300,7 +298,7 @@ spec:
        image: "{{ .Values.image }}:{{ .Values.imageTag }}"
        imagePullPolicy: {{ .Values.pullPolicy }}
        command: ["/fission-bundle"]
-        args: ["--kubewatcher"]
+        args: ["--kubewatcher", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
      serviceAccount: fission-svc

 ---
@@ -469,7 +467,7 @@ spec:
        image: "{{ .Values.image }}:{{ .Values.imageTag }}"
        imagePullPolicy: {{ .Values.pullPolicy }}
        command: ["/fission-bundle"]
-        args: ["--timer"]
+        args: ["--timer", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
      serviceAccount: fission-svc

 #
@@ -547,7 +545,7 @@ spec:
        image: "{{ .Values.image }}:{{ .Values.imageTag }}"
        imagePullPolicy: {{ .Values.pullPolicy }}
        command: ["/fission-bundle"]
-        args: ["--mqt"]
+        args: ["--mqt", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
        env:
        - name: MESSAGE_QUEUE_TYPE
          value: {{ .Values.messageQueue.type }}
@@ -13,7 +13,7 @@ metadata:
  annotations:
    # This is what defines this resource as a hook. Without this line, the
    # job is considered part of the release.
-    "helm.sh/hook": post-install,post-upgrade
+    "helm.sh/hook": post-install
    "helm.sh/hook-delete-policy": hook-succeeded
 spec:
  template:
@@ -0,0 +1,35 @@
+{{ if .Values.analytics }}
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: {{ template "fullname" . }}-{{ .Chart.Version }}
+  labels:
+    # The "release" convention makes it easy to tie a release to all of the
+    # Kubernetes resources that were created as part of that release.
+    release: {{ .Release.Name }}
+    # This makes it easy to audit chart usage.
+    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    app: {{ template "name" . }}
+  annotations:
+    # This is what defines this resource as a hook. Without this line, the
+    # job is considered part of the release.
+    "helm.sh/hook": post-upgrade
+    "helm.sh/hook-delete-policy": hook-succeeded
+spec:
+  template:
+    metadata:
+      name: {{ template "fullname" . }}
+      labels:
+        release: {{ .Release.Name }}
+        app: {{ template "name" . }}
+      annotations:
+        {{- if .Values.enableIstio }}
+        "sidecar.istio.io/inject": "false"
+        {{- end }}
+    spec:
+      restartPolicy: Never
+      containers:
+        - name: post-upgrade-job
+          image: "fission/alpinecurl"
+          command: ["sh", "-c", "/usr/bin/curl -m 5 -H \"Content-Type: application/json\" -X POST -d '{\"type\": \"helm-post-upgrade\", \"chartName\": \"{{ .Chart.Name }}\", \"chartVersion\": \"{{ .Chart.Version }}\"}' https://g.fission.sh/analytics || true"]
+{{ end }}
@@ -16,13 +16,13 @@ image: fission/fission-bundle
 pullPolicy: IfNotPresent

 ## Fission image version
-imageTag: 0.6.1
+imageTag: 0.7.1

 ## Fission fetcher repository
 fetcherImage: fission/fetcher

 ## Fission fetcher image version
-fetcherImageTag: 0.6.1
+fetcherImageTag: 0.7.1

 ## Port at which Fission controller service should be exposed
 controllerPort: 31313
@@ -48,7 +48,7 @@ enableIstio: false
 logger:
  influxdbAdmin: "admin"
  fluentdImage: fission/fluentd
-  fluentdImageTag: 0.6.1
+  fluentdImageTag: 0.7.1

 ## Type of Queue you would like to use
 ## currently supports nats-streaming, azure-storage-queue
@@ -1,6 +1,6 @@
 apiVersion: v1
 name: fission-core
-version: 0.6.1
+version: 0.7.1
 description: Fission is a fast serverless framework for Kubernetes.
 keywords:
 - fission
@@ -12,4 +12,4 @@ maintainers:
  - name: Ta Ching Chen
    email: contact@tachingchen.com
 engine: gotpl
-appVersion: 0.6.1
+appVersion: 0.7.1
@@ -1,13 +1,13 @@
 1. Install the client CLI.

 Mac:
-  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.6.1/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
+  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.7.1/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/

 Linux:
-  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.6.1/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
+  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.7.1/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/

 Windows:
-  For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/0.6.1/fission-cli-windows.exe
+  For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/0.7.1/fission-cli-windows.exe

 2. You're ready to use Fission!

@@ -139,8 +139,6 @@ spec:
        env:
          - name: FISSION_FUNCTION_NAMESPACE
            value: "{{ .Values.functionNamespace }}"
-          - name: ENABLE_ISTIO
-            value: "{{ .Values.enableIstio }}"
        readinessProbe:
          httpGet:
            path: "/healthz"
@@ -298,7 +296,7 @@ spec:
        image: "{{ .Values.image }}:{{ .Values.imageTag }}"
        imagePullPolicy: {{ .Values.pullPolicy }}
        command: ["/fission-bundle"]
-        args: ["--kubewatcher"]
+        args: ["--kubewatcher", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
      serviceAccount: fission-svc

 ---
@@ -320,7 +318,7 @@ spec:
        image: "{{ .Values.image }}:{{ .Values.imageTag }}"
        imagePullPolicy: {{ .Values.pullPolicy }}
        command: ["/fission-bundle"]
-        args: ["--timer"]
+        args: ["--timer", "--routerUrl", "http://router.{{ .Release.Namespace }}"]
      serviceAccount: fission-svc

 ---
@@ -13,7 +13,7 @@ metadata:
  annotations:
    # This is what defines this resource as a hook. Without this line, the
    # job is considered part of the release.
-    "helm.sh/hook": post-install,post-upgrade
+    "helm.sh/hook": post-install
    "helm.sh/hook-delete-policy": hook-succeeded
 spec:
  template:
@@ -0,0 +1,35 @@
+{{ if .Values.analytics }}
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: {{ template "fullname" . }}-{{ .Chart.Version }}
+  labels:
+    # The "release" convention makes it easy to tie a release to all of the
+    # Kubernetes resources that were created as part of that release.
+    release: {{ .Release.Name }}
+    # This makes it easy to audit chart usage.
+    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    app: {{ template "name" . }}
+  annotations:
+    # This is what defines this resource as a hook. Without this line, the
+    # job is considered part of the release.
+    "helm.sh/hook": post-upgrade
+    "helm.sh/hook-delete-policy": hook-succeeded
+spec:
+  template:
+    metadata:
+      name: {{ template "fullname" . }}
+      labels:
+        release: {{ .Release.Name }}
+        app: {{ template "name" . }}
+      annotations:
+        {{- if .Values.enableIstio }}
+        "sidecar.istio.io/inject": "false"
+        {{- end }}
+    spec:
+      restartPolicy: Never
+      containers:
+        - name: post-upgrade-job
+          image: "fission/alpinecurl"
+          command: ["sh", "-c", "/usr/bin/curl -m 5 -H \"Content-Type: application/json\" -X POST -d '{\"type\": \"helm-post-upgrade\", \"chartName\": \"{{ .Chart.Name }}\", \"chartVersion\": \"{{ .Chart.Version }}\"}' https://g.fission.sh/analytics || true"]
+{{ end }}
@@ -13,7 +13,7 @@ routerServiceType: LoadBalancer
 image: fission/fission-bundle

 ## Fission image version
-imageTag: 0.6.1
+imageTag: 0.7.1

 ## Image pull policy
 pullPolicy: IfNotPresent
@@ -22,7 +22,7 @@ pullPolicy: IfNotPresent
 fetcherImage: fission/fetcher

 ## Fission fetcher image version
-fetcherImageTag: 0.6.1
+fetcherImageTag: 0.7.1

 ## Port at which Fission controller service should be exposed
 controllerPort: 31313
@@ -87,3 +87,35 @@ func MergeContainerSpecs(specs ...*apiv1.Container) apiv1.Container {
 	}
 	return *result
 }
+
+// IsNetworkDialError returns true if its a network dial error
+func IsNetworkDialError(err error) bool {
+	netErr, ok := err.(net.Error)
+	if !ok {
+		return false
+	}
+	netOpErr, ok := netErr.(*net.OpError)
+	if !ok {
+		return false
+	}
+	if netOpErr.Op == "dial" {
+		return true
+	}
+	return false
+}
+
+// IsReadyPod checks that all containers in a pod are ready and returns true if so
+func IsReadyPod(pod *apiv1.Pod) bool {
+	// since its a utility function, just ensuring there is no nil pointer exception
+	if pod == nil {
+		return false
+	}
+
+	for _, cStatus := range pod.Status.ContainerStatuses {
+		if !cStatus.Ready {
+			return false
+		}
+	}
+
+	return true
+}
@@ -21,7 +21,6 @@ import (
 	"net/http"
 	"os"
 	"runtime/debug"
-	"strconv"
 	"strings"

 	"github.com/gorilla/mux"
@@ -83,14 +82,6 @@ func MakeAPI() (*API, error) {
 		api.functionNamespace = "fission-function"
 	}

-	if len(os.Getenv("ENABLE_ISTIO")) > 0 {
-		istio, err := strconv.ParseBool(os.Getenv("ENABLE_ISTIO"))
-		if err != nil {
-			log.Println("Failed to parse ENABLE_ISTIO")
-		}
-		api.useIstio = istio
-	}
-
 	return api, err
 }

@@ -24,6 +24,7 @@ import (
 	"net/http"
 	"os"
 	"reflect"
+	"strings"
 	"testing"
 	"time"

@@ -69,9 +70,8 @@ func assertNotFoundFailure(err error, name string) {

 func assertCronSpecFails(err error) {
 	assert(err != nil, "using an invalid cron spec must fail")
-	fe, ok := err.(fission.Error)
-	assert(ok, "error must be a fission Error")
-	assert(fe.Code == fission.ErrorInvalidArgument, "error must be a invalid argument error")
+	ok := strings.Contains(err.Error(), "not a valid cron spec")
+	assert(ok, "invalid cron spec must fail")
 }

 func TestFunctionApi(t *testing.T) {
@@ -82,10 +82,16 @@ func TestFunctionApi(t *testing.T) {
 		},
 		Spec: fission.FunctionSpec{
 			Environment: fission.EnvironmentReference{
-				Name: "nodejs",
+				Name:      "nodejs",
+				Namespace: metav1.NamespaceDefault,
 			},
 			Package: fission.FunctionPackageRef{
 				FunctionName: "xxx",
+				PackageRef: fission.PackageRef{
+					Namespace:       metav1.NamespaceDefault,
+					Name:            "xxx",
+					ResourceVersion: "12345",
+				},
 			},
 		},
 	}
@@ -142,6 +148,7 @@ func TestHTTPTriggerApi(t *testing.T) {
 			Namespace: metav1.NamespaceDefault,
 		},
 		Spec: fission.HTTPTriggerSpec{
+			Method:      http.MethodGet,
 			RelativeURL: "/hello",
 			FunctionReference: fission.FunctionReference{
 				Type: fission.FunctionReferenceTypeFunctionName,
@@ -24,10 +24,16 @@ import (

 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

+	"github.com/fission/fission"
 	"github.com/fission/fission/crd"
 )

 func (c *Client) EnvironmentCreate(env *crd.Environment) (*metav1.ObjectMeta, error) {
+	err := env.Validate()
+	if err != nil {
+		return nil, fission.AggregateValidationErrors("Environment", err)
+	}
+
 	reqbody, err := json.Marshal(env)
 	if err != nil {
 		return nil, err
@@ -78,6 +84,11 @@ func (c *Client) EnvironmentGet(m *metav1.ObjectMeta) (*crd.Environment, error)
 }

 func (c *Client) EnvironmentUpdate(env *crd.Environment) (*metav1.ObjectMeta, error) {
+	err := env.Validate()
+	if err != nil {
+		return nil, fission.AggregateValidationErrors("Environment", err)
+	}
+
 	reqbody, err := json.Marshal(env)
 	if err != nil {
 		return nil, err
@@ -24,10 +24,15 @@ import (

 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

+	"github.com/fission/fission"
 	"github.com/fission/fission/crd"
 )

 func (c *Client) FunctionCreate(f *crd.Function) (*metav1.ObjectMeta, error) {
+	err := f.Validate()
+	if err != nil {
+		return nil, fission.AggregateValidationErrors("Function", err)
+	}

 	reqbody, err := json.Marshal(f)
 	if err != nil {
@@ -93,6 +98,11 @@ func (c *Client) FunctionGetRawDeployment(m *metav1.ObjectMeta) ([]byte, error)
 }

 func (c *Client) FunctionUpdate(f *crd.Function) (*metav1.ObjectMeta, error) {
+	err := f.Validate()
+	if err != nil {
+		return nil, fission.AggregateValidationErrors("Function", err)
+	}
+
 	reqbody, err := json.Marshal(f)
 	if err != nil {
 		return nil, err
@@ -24,10 +24,16 @@ import (

 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

+	"github.com/fission/fission"
 	"github.com/fission/fission/crd"
 )

 func (c *Client) HTTPTriggerCreate(t *crd.HTTPTrigger) (*metav1.ObjectMeta, error) {
+	err := t.Validate()
+	if err != nil {
+		return nil, fission.AggregateValidationErrors("HTTPTrigger", err)
+	}
+
 	reqbody, err := json.Marshal(t)
 	if err != nil {
 		return nil, err
@@ -78,6 +84,11 @@ func (c *Client) HTTPTriggerGet(m *metav1.ObjectMeta) (*crd.HTTPTrigger, error)
 }

 func (c *Client) HTTPTriggerUpdate(t *crd.HTTPTrigger) (*metav1.ObjectMeta, error) {
+	err := t.Validate()
+	if err != nil {
+		return nil, fission.AggregateValidationErrors("HTTPTrigger", err)
+	}
+
 	reqbody, err := json.Marshal(t)
 	if err != nil {
 		return nil, err
@@ -29,6 +29,11 @@ import (
 )

 func (c *Client) WatchCreate(w *crd.KubernetesWatchTrigger) (*metav1.ObjectMeta, error) {
+	err := w.Validate()
+	if err != nil {
+		return nil, fission.AggregateValidationErrors("KubernetesWatchTrigger", err)
+	}
+
 	reqbody, err := json.Marshal(w)
 	if err != nil {
 		return nil, err
@@ -24,10 +24,16 @@ import (

 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

+	"github.com/fission/fission"
 	"github.com/fission/fission/crd"
 )

 func (c *Client) MessageQueueTriggerCreate(t *crd.MessageQueueTrigger) (*metav1.ObjectMeta, error) {
+	err := t.Validate()
+	if err != nil {
+		return nil, fission.AggregateValidationErrors("MessageQueueTrigger", err)
+	}
+
 	reqbody, err := json.Marshal(t)
 	if err != nil {
 		return nil, err
@@ -78,6 +84,11 @@ func (c *Client) MessageQueueTriggerGet(m *metav1.ObjectMeta) (*crd.MessageQueue
 }

 func (c *Client) MessageQueueTriggerUpdate(mqTrigger *crd.MessageQueueTrigger) (*metav1.ObjectMeta, error) {
+	err := mqTrigger.Validate()
+	if err != nil {
+		return nil, fission.AggregateValidationErrors("MessageQueueTrigger", err)
+	}
+
 	reqbody, err := json.Marshal(mqTrigger)
 	if err != nil {
 		return nil, err
@@ -24,10 +24,15 @@ import (

 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

+	"github.com/fission/fission"
 	"github.com/fission/fission/crd"
 )

 func (c *Client) PackageCreate(f *crd.Package) (*metav1.ObjectMeta, error) {
+	err := f.Validate()
+	if err != nil {
+		return nil, fission.AggregateValidationErrors("Package", err)
+	}

 	reqbody, err := json.Marshal(f)
 	if err != nil {
@@ -79,6 +84,11 @@ func (c *Client) PackageGet(m *metav1.ObjectMeta) (*crd.Package, error) {
 }

 func (c *Client) PackageUpdate(f *crd.Package) (*metav1.ObjectMeta, error) {
+	err := f.Validate()
+	if err != nil {
+		return nil, fission.AggregateValidationErrors("Package", err)
+	}
+
 	reqbody, err := json.Marshal(f)
 	if err != nil {
 		return nil, err
@@ -24,10 +24,16 @@ import (

 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

+	"github.com/fission/fission"
 	"github.com/fission/fission/crd"
 )

 func (c *Client) TimeTriggerCreate(t *crd.TimeTrigger) (*metav1.ObjectMeta, error) {
+	err := t.Validate()
+	if err != nil {
+		return nil, fission.AggregateValidationErrors("TimeTrigger", err)
+	}
+
 	reqbody, err := json.Marshal(t)
 	if err != nil {
 		return nil, err
@@ -78,6 +84,11 @@ func (c *Client) TimeTriggerGet(m *metav1.ObjectMeta) (*crd.TimeTrigger, error)
 }

 func (c *Client) TimeTriggerUpdate(t *crd.TimeTrigger) (*metav1.ObjectMeta, error) {
+	err := t.Validate()
+	if err != nil {
+		return nil, fission.AggregateValidationErrors("TimeTrigger", err)
+	}
+
 	reqbody, err := json.Marshal(t)
 	if err != nil {
 		return nil, err
@@ -37,7 +37,10 @@ func Start(port int) {
 		log.Fatalf("Failed to create fission CRDs: %v", err)
 	}

-	fc.WaitForCRDs()
+	err = fc.WaitForCRDs()
+	if err != nil {
+		log.Fatalf("Error waiting for CRDs: %v", err)
+	}

 	api, err := MakeAPI()
 	if err != nil {
@@ -17,9 +17,6 @@ limitations under the License.
 package controller

 import (
-	"errors"
-	"regexp"
-
 	"github.com/fission/fission/crd"
 )

@@ -30,11 +27,3 @@ func makeCRDBackedAPI() (*API, error) {
 	}
 	return &API{fissionClient: fissionClient, kubernetesClient: kubernetesClient}, nil
 }
-
-func validateResourceName(name string) error {
-	re := regexp.MustCompile(`[a-z0-9]([-a-z0-9]*[a-z0-9])?`)
-	if len(re.FindString(name)) != len(name) {
-		return errors.New("Name must consist of lower case alphanumeric characters or '-', and must start and end with an alphanumeric character (e.g. 'my-name',  or '123-abc', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?'")
-	}
-	return nil
-}
@@ -60,12 +60,6 @@ func (a *API) EnvironmentApiCreate(w http.ResponseWriter, r *http.Request) {
 		return
 	}

-	err = validateResourceName(env.Metadata.Name)
-	if err != nil {
-		a.respondWithError(w, err)
-		return
-	}
-
 	enew, err := a.fissionClient.Environments(env.Metadata.Namespace).Create(&env)
 	if err != nil {
 		a.respondWithError(w, err)
@@ -29,8 +29,6 @@ import (
 	"github.com/gorilla/mux"
 	log "github.com/sirupsen/logrus"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
-	"k8s.io/apimachinery/pkg/labels"
-	"k8s.io/apimachinery/pkg/util/intstr"
 	"k8s.io/client-go/pkg/api"
 	apiv1 "k8s.io/client-go/pkg/api/v1"
 	restclient "k8s.io/client-go/rest"
@@ -75,12 +73,6 @@ func (a *API) FunctionApiCreate(w http.ResponseWriter, r *http.Request) {
 		return
 	}

-	err = validateResourceName(f.Metadata.Name)
-	if err != nil {
-		a.respondWithError(w, err)
-		return
-	}
-
 	fnew, err := a.fissionClient.Functions(f.Metadata.Namespace).Create(&f)
 	if err != nil {
 		a.respondWithError(w, err)
@@ -93,62 +85,6 @@ func (a *API) FunctionApiCreate(w http.ResponseWriter, r *http.Request) {
 		return
 	}

-	// Since istio only allows accessing pod through k8s service,
-	// for the functions with executor type "poolmgr" we need to
-	// create a service for sending requests to pod in pool.
-	// Functions with executor type "Newdeploy" is specialized at
-	// pod starts. In this case, just ignore such functions.
-	fnExecutorType := f.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType
-
-	if a.useIstio && fnExecutorType == fission.ExecutorTypePoolmgr {
-		// create a same name service for function
-		// since istio only allows the traffic to service
-
-		sel := map[string]string{
-			"functionName": fnew.Metadata.Name,
-			"functionUid":  string(fnew.Metadata.UID),
-		}
-
-		// service for accepting user traffic
-		svc := apiv1.Service{
-			ObjectMeta: metav1.ObjectMeta{
-				Namespace: a.functionNamespace,
-				Name:      fission.GetFunctionIstioServiceName(f.Metadata.Name, f.Metadata.Namespace),
-				Labels:    a.getIstioServiceLabels(f.Metadata.Name),
-			},
-			Spec: apiv1.ServiceSpec{
-				Type: apiv1.ServiceTypeClusterIP,
-				Ports: []apiv1.ServicePort{
-					// Service port name should begin with a recognized prefix, or the traffic will be
-					// treated as TCP traffic. (https://istio.io/docs/setup/kubernetes/sidecar-injection.html)
-					// Originally the ports' name are similar to "http-fetch" and "http-specialize".
-					// But for istio 0.5.1, istio-proxy return unexpected 431 error with such naming.
-					// https://github.com/istio/istio/issues/928
-					// Workaround: remove prefix
-					// TODO: prepend prefix once the bug fixed
-					{
-						Name:       "fetch",
-						Protocol:   apiv1.ProtocolTCP,
-						Port:       8000,
-						TargetPort: intstr.FromInt(8000),
-					},
-					{
-						Name:       "specialize",
-						Protocol:   apiv1.ProtocolTCP,
-						Port:       8888,
-						TargetPort: intstr.FromInt(8888),
-					},
-				},
-				Selector: sel,
-			},
-		}
-		_, err = a.kubernetesClient.CoreV1().Services(a.functionNamespace).Create(&svc)
-		if err != nil {
-			a.respondWithError(w, err)
-			return
-		}
-	}
-
 	w.WriteHeader(http.StatusCreated)
 	a.respondWithSuccess(w, resp)
 }
@@ -226,20 +162,6 @@ func (a *API) FunctionApiDelete(w http.ResponseWriter, r *http.Request) {
 		return
 	}

-	if a.useIstio {
-		// delete all istio services belong to the function
-		sel := a.getIstioServiceLabels(name)
-		svcList, err := a.kubernetesClient.CoreV1().Services(a.functionNamespace).List(metav1.ListOptions{
-			LabelSelector: labels.Set(sel).AsSelector().String(),
-		})
-		for _, svc := range svcList.Items {
-			err = a.kubernetesClient.CoreV1().Services(a.functionNamespace).Delete(svc.ObjectMeta.Name, &metav1.DeleteOptions{})
-			// log error and continue
-			log.Printf("Failed to delete service %v: %v", svc.ObjectMeta.Name, err)
-			continue
-		}
-	}
-
 	a.respondWithSuccess(w, []byte(""))
 }

@@ -74,12 +74,6 @@ func (a *API) HTTPTriggerApiCreate(w http.ResponseWriter, r *http.Request) {
 		return
 	}

-	err = validateResourceName(t.Metadata.Name)
-	if err != nil {
-		a.respondWithError(w, err)
-		return
-	}
-
 	// Ensure we don't have a duplicate HTTP route defined (same URL and method)
 	err = a.checkHTTPTriggerDuplicates(&t)
 	if err != nil {
@@ -57,12 +57,6 @@ func (a *API) MessageQueueTriggerApiCreate(w http.ResponseWriter, r *http.Reques
 		return
 	}

-	err = validateResourceName(mqTrigger.Metadata.Name)
-	if err != nil {
-		a.respondWithError(w, err)
-		return
-	}
-
 	tnew, err := a.fissionClient.MessageQueueTriggers(mqTrigger.Metadata.Namespace).Create(&mqTrigger)
 	if err != nil {
 		a.respondWithError(w, err)
@@ -58,12 +58,6 @@ func (a *API) PackageApiCreate(w http.ResponseWriter, r *http.Request) {
 		return
 	}

-	err = validateResourceName(f.Metadata.Name)
-	if err != nil {
-		a.respondWithError(w, err)
-		return
-	}
-
 	// Ensure size limits
 	if len(f.Spec.Source.Literal) > 256*1024 {
 		err := fission.MakeError(fission.ErrorInvalidArgument, "Package literal larger than 256K")
@@ -59,12 +59,6 @@ func (a *API) TimeTriggerApiCreate(w http.ResponseWriter, r *http.Request) {
 		return
 	}

-	err = validateResourceName(t.Metadata.Name)
-	if err != nil {
-		a.respondWithError(w, err)
-		return
-	}
-
 	// validate
 	_, err = cron.Parse(t.Spec.Cron)
 	if err != nil {
@@ -58,12 +58,6 @@ func (a *API) WatchApiCreate(w http.ResponseWriter, r *http.Request) {
 		return
 	}

-	err = validateResourceName(watch.Metadata.Name)
-	if err != nil {
-		a.respondWithError(w, err)
-		return
-	}
-
 	// TODO check for duplicate watches

 	wnew, err := a.fissionClient.KubernetesWatchTriggers(watch.Metadata.Namespace).Create(&watch)
@@ -212,8 +212,8 @@ func (fc *FissionClient) Packages(ns string) PackageInterface {
 	return MakePackageInterface(fc.crdClient, ns)
 }

-func (fc *FissionClient) WaitForCRDs() {
-	waitForCRDs(fc.crdClient)
+func (fc *FissionClient) WaitForCRDs() error {
+	return waitForCRDs(fc.crdClient)
 }
 func (fc *FissionClient) GetCrdClient() *rest.RESTClient {
 	return fc.crdClient
@@ -34,31 +34,31 @@ const (
 // ensureCRD checks if the given CRD type exists, and creates it if
 // needed. (Note that this creates the CRD type; it doesn't create any
 // _instances_ of that type.)
-func ensureCRD(clientset *apiextensionsclient.Clientset, crd *apiextensionsv1beta1.CustomResourceDefinition) error {
+func ensureCRD(clientset *apiextensionsclient.Clientset, crd *apiextensionsv1beta1.CustomResourceDefinition) (err error) {
 	maxRetries := 5
-	for i := 0; i < maxRetries; i++ {
-		_, err := clientset.ApiextensionsV1beta1().CustomResourceDefinitions().Get(crd.ObjectMeta.Name, metav1.GetOptions{})

-		if err != nil {
-			if errors.IsNotFound(err) {
-				// crd resource not found error
-				_, err := clientset.ApiextensionsV1beta1().CustomResourceDefinitions().Create(crd)
-				if err != nil {
-					return err
-				}
-			} else {
-				// The requests fail to connect to k8s api server before
-				// istio-prxoy is ready to serve traffic. Retry again.
-				log.Printf("Error connecting to kubernetes api service (%v), retrying", err)
-				time.Sleep(500 * time.Duration(2*i) * time.Millisecond)
-				continue
-			}
+	for i := 0; i < maxRetries; i++ {
+		_, err = clientset.ApiextensionsV1beta1().CustomResourceDefinitions().Get(crd.ObjectMeta.Name, metav1.GetOptions{})
+		if err == nil {
+			return nil
 		}

-		// resource already exists
-		break
+		if errors.IsNotFound(err) {
+			// crd resource not found error
+			_, err = clientset.ApiextensionsV1beta1().CustomResourceDefinitions().Create(crd)
+			if err != nil {
+				return err
+			}
+		} else {
+			// The requests fail to connect to k8s api server before
+			// istio-prxoy is ready to serve traffic. Retry again.
+			log.Printf("Error connecting to kubernetes api service (%v), retrying", err)
+			time.Sleep(500 * time.Duration(2*i) * time.Millisecond)
+			continue
+		}
 	}
-	return nil
+
+	return err
 }

 // Ensure CRDs
@@ -17,6 +17,7 @@ limitations under the License.
 package crd

 import (
+	"github.com/hashicorp/go-multierror"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/runtime/schema"

@@ -151,14 +152,14 @@ func (ht *HTTPTrigger) GetObjectKind() schema.ObjectKind {
 func (w *KubernetesWatchTrigger) GetObjectKind() schema.ObjectKind {
 	return &w.TypeMeta
 }
-func (w *TimeTrigger) GetObjectKind() schema.ObjectKind {
-	return &w.TypeMeta
+func (t *TimeTrigger) GetObjectKind() schema.ObjectKind {
+	return &t.TypeMeta
 }
-func (w *MessageQueueTrigger) GetObjectKind() schema.ObjectKind {
-	return &w.TypeMeta
+func (m *MessageQueueTrigger) GetObjectKind() schema.ObjectKind {
+	return &m.TypeMeta
 }
-func (w *Package) GetObjectKind() schema.ObjectKind {
-	return &w.TypeMeta
+func (p *Package) GetObjectKind() schema.ObjectKind {
+	return &p.TypeMeta
 }

 func (f *Function) GetObjectMeta() metav1.Object {
@@ -173,14 +174,14 @@ func (ht *HTTPTrigger) GetObjectMeta() metav1.Object {
 func (w *KubernetesWatchTrigger) GetObjectMeta() metav1.Object {
 	return &w.Metadata
 }
-func (w *TimeTrigger) GetObjectMeta() metav1.Object {
-	return &w.Metadata
+func (t *TimeTrigger) GetObjectMeta() metav1.Object {
+	return &t.Metadata
 }
-func (w *MessageQueueTrigger) GetObjectMeta() metav1.Object {
-	return &w.Metadata
+func (m *MessageQueueTrigger) GetObjectMeta() metav1.Object {
+	return &m.Metadata
 }
-func (w *Package) GetObjectMeta() metav1.Object {
-	return &w.Metadata
+func (p *Package) GetObjectMeta() metav1.Object {
+	return &p.Metadata
 }

 func (fl *FunctionList) GetObjectKind() schema.ObjectKind {
@@ -198,11 +199,11 @@ func (wl *KubernetesWatchTriggerList) GetObjectKind() schema.ObjectKind {
 func (wl *TimeTriggerList) GetObjectKind() schema.ObjectKind {
 	return &wl.TypeMeta
 }
-func (wl *MessageQueueTriggerList) GetObjectKind() schema.ObjectKind {
-	return &wl.TypeMeta
+func (ml *MessageQueueTriggerList) GetObjectKind() schema.ObjectKind {
+	return &ml.TypeMeta
 }
-func (wl *PackageList) GetObjectKind() schema.ObjectKind {
-	return &wl.TypeMeta
+func (pl *PackageList) GetObjectKind() schema.ObjectKind {
+	return &pl.TypeMeta
 }

 func (fl *FunctionList) GetListMeta() metav1.List {
@@ -220,9 +221,141 @@ func (wl *KubernetesWatchTriggerList) GetListMeta() metav1.List {
 func (wl *TimeTriggerList) GetListMeta() metav1.List {
 	return &wl.Metadata
 }
-func (wl *MessageQueueTriggerList) GetListMeta() metav1.List {
-	return &wl.Metadata
+func (ml *MessageQueueTriggerList) GetListMeta() metav1.List {
+	return &ml.Metadata
 }
-func (wl *PackageList) GetListMeta() metav1.List {
-	return &wl.Metadata
+func (pl *PackageList) GetListMeta() metav1.List {
+	return &pl.Metadata
+}
+
+func validateMetadata(field string, m metav1.ObjectMeta) error {
+	return fission.ValidateKubeReference(field, m.Name, m.Namespace)
+}
+
+func (p *Package) Validate() error {
+	var result *multierror.Error
+
+	result = multierror.Append(result,
+		validateMetadata("Package", p.Metadata),
+		p.Spec.Validate(),
+		p.Status.Validate())
+
+	return result.ErrorOrNil()
+}
+
+func (pl *PackageList) Validate() error {
+	var result *multierror.Error
+	// not validate ListMeta
+	for _, p := range pl.Items {
+		result = multierror.Append(result, p.Validate())
+	}
+	return result.ErrorOrNil()
+}
+
+func (f *Function) Validate() error {
+	var result *multierror.Error
+
+	result = multierror.Append(result,
+		validateMetadata("Function", f.Metadata),
+		f.Spec.Validate())
+
+	return result.ErrorOrNil()
+}
+
+func (fl *FunctionList) Validate() error {
+	var result *multierror.Error
+	for _, f := range fl.Items {
+		result = multierror.Append(result, f.Validate())
+	}
+	return result.ErrorOrNil()
+}
+
+func (e *Environment) Validate() error {
+	var result *multierror.Error
+
+	result = multierror.Append(result,
+		validateMetadata("Environment", e.Metadata),
+		e.Spec.Validate())
+
+	return result.ErrorOrNil()
+}
+
+func (el *EnvironmentList) Validate() error {
+	var result *multierror.Error
+	for _, e := range el.Items {
+		result = multierror.Append(result, e.Validate())
+	}
+	return result.ErrorOrNil()
+}
+
+func (h *HTTPTrigger) Validate() error {
+	var result *multierror.Error
+
+	result = multierror.Append(result,
+		validateMetadata("HTTPTrigger", h.Metadata),
+		h.Spec.Validate())
+
+	return result.ErrorOrNil()
+}
+
+func (hl *HTTPTriggerList) Validate() error {
+	var result *multierror.Error
+	for _, h := range hl.Items {
+		result = multierror.Append(result, h.Validate())
+	}
+	return result.ErrorOrNil()
+}
+
+func (k *KubernetesWatchTrigger) Validate() error {
+	var result *multierror.Error
+
+	result = multierror.Append(result,
+		validateMetadata("KubernetesWatchTrigger", k.Metadata),
+		k.Spec.Validate())
+
+	return result.ErrorOrNil()
+}
+
+func (kl *KubernetesWatchTriggerList) Validate() error {
+	var result *multierror.Error
+	for _, k := range kl.Items {
+		result = multierror.Append(result, k.Validate())
+	}
+	return result
+}
+
+func (t *TimeTrigger) Validate() error {
+	var result *multierror.Error
+
+	result = multierror.Append(result,
+		validateMetadata("TimeTrigger", t.Metadata),
+		t.Spec.Validate())
+
+	return result.ErrorOrNil()
+}
+
+func (tl *TimeTriggerList) Validate() error {
+	var result *multierror.Error
+	for _, t := range tl.Items {
+		result = multierror.Append(result, t.Validate())
+	}
+	return result.ErrorOrNil()
+}
+
+func (m *MessageQueueTrigger) Validate() error {
+	var result *multierror.Error
+
+	result = multierror.Append(result,
+		validateMetadata("MessageQueueTrigger", m.Metadata),
+		m.Spec.Validate())
+
+	return result.ErrorOrNil()
+}
+
+func (ml *MessageQueueTriggerList) Validate() error {
+	var result *multierror.Error
+	for _, m := range ml.Items {
+		result = multierror.Append(result, m.Validate())
+	}
+	return result.ErrorOrNil()
 }
@@ -217,6 +217,19 @@ func (fetcher *Fetcher) FetchHandler(w http.ResponseWriter, r *http.Request) {
 // Fetch takes FetchRequest and makes the fetch call
 // It returns the HTTP code and error if any
 func (fetcher *Fetcher) Fetch(req FetchRequest) (int, error) {
+	// check that the requested filename is not an empty string and error out if so
+	if len(req.Filename) == 0 {
+		e := fmt.Sprintf("Fetch request received for an empty file name, request: %v", req)
+		log.Printf(e)
+		return 400, errors.New(e)
+	}
+
+	// verify first if the file already exists.
+	if _, err := os.Stat(filepath.Join(fetcher.sharedVolumePath, req.Filename)); err == nil {
+		log.Printf("Requested file: %s already exists at %s. Skipping fetch", req.Filename, fetcher.sharedVolumePath)
+		return 200, nil
+	}
+
 	tmpFile := req.Filename + ".tmp"
 	tmpPath := filepath.Join(fetcher.sharedVolumePath, tmpFile)

@@ -241,6 +254,15 @@ func (fetcher *Fetcher) Fetch(req FetchRequest) (int, error) {
 		if req.FetchType == FETCH_SOURCE {
 			archive = &pkg.Spec.Source
 		} else if req.FetchType == FETCH_DEPLOYMENT {
+			// sometimes, the user may invoke the function even before the source code is built into a deploy pkg.
+			// this results in executor sending a fetch request of type FETCH_DEPLOYMENT and since pkg.Spec.Deployment.Url will be empty,
+			// we hit this "Get : unsupported protocol scheme "" error.
+			// it may be useful to the user if we can send a more meaningful error in such a scenario.
+			if pkg.Status.BuildStatus != fission.BuildStatusSucceeded && pkg.Status.BuildStatus != fission.BuildStatusNone {
+				e := fmt.Sprintf("Build status for the function's pkg : %s.%s is : %s, can't fetch deployment", pkg.Metadata.Name, pkg.Metadata.Namespace, pkg.Status.BuildStatus)
+				log.Printf(e)
+				return 500, errors.New(e)
+			}
 			archive = &pkg.Spec.Deployment
 		}
 		// get package data as literal or by url
@@ -57,13 +57,27 @@ func (executor *Executor) getServiceForFunctionApi(w http.ResponseWriter, r *htt
 	w.Write([]byte(serviceName))
 }

+// getServiceForFunction first checks if this function's service is cached, if yes, it validates the address.
+// if it's a valid address, just returns it.
+// else, invalidates its cache entry and makes a new request to create a service for this function and finally responds
+// with new address or error.
+//
+// checking for the validity of the address causes a little more over-head than desired. but, it ensures that
+// stale addresses are not returned to the router.
+// To make it optimal, plan is to add an eager cache invalidator function that watches for pod deletion events and
+// invalidates the cache entry if the pod address was cached.
 func (executor *Executor) getServiceForFunction(m *metav1.ObjectMeta) (string, error) {
 	// Check function -> svc cache
 	log.Printf("[%v] Checking for cached function service", m.Name)
 	fsvc, err := executor.fsCache.GetByFunction(m)
 	if err == nil {
-		// Cached, return svc address
-		return fsvc.Address, nil
+		if executor.isValidAddress(fsvc) {
+			// Cached, return svc address
+			return fsvc.Address, nil
+		} else {
+			log.Printf("[%v] Deleting cache entry for invalid address : %s", m.Name, fsvc.Address)
+			executor.fsCache.DeleteEntry(fsvc)
+		}
 	}

 	respChan := make(chan *createFuncServiceResponse)
@@ -111,6 +125,7 @@ func (executor *Executor) Serve(port int) {
 	ctx, cancel := context.WithCancel(context.Background())
 	defer cancel()
 	executor.ndm.Run(ctx)
+	executor.gpm.Run(ctx)
 	r.Use(fission.LoggingMiddleware)
 	log.Fatal(http.ListenAndServe(address, r))
 }
@@ -22,9 +22,8 @@ import (
 	"strings"
 	"time"

-	"k8s.io/apimachinery/pkg/api/errors"
 	meta_v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
-	"k8s.io/apimachinery/pkg/labels"
+	"k8s.io/apimachinery/pkg/types"
 	"k8s.io/client-go/kubernetes"
 	"k8s.io/client-go/pkg/api"

@@ -33,6 +32,11 @@ import (
 	"github.com/fission/fission/executor/fscache"
 )

+var (
+	deletePropagation = meta_v1.DeletePropagationBackground
+	delOpt            = meta_v1.DeleteOptions{PropagationPolicy: &deletePropagation}
+)
+
 // cleanupObjects cleans up resources created by old executortype instances
 func cleanupObjects(kubernetesClient *kubernetes.Clientset,
 	namespace string,
@@ -65,14 +69,6 @@ func cleanup(client *kubernetes.Clientset, namespace string, instanceId string)
 	if err != nil {
 		return err
 	}
-	// See K8s #33845 and related bugs: deleting a deployment
-	// through the API doesn't cause the associated ReplicaSet to
-	// be deleted.  (Fixed recently, but we may be running a
-	// version before the fix.)
-	err = cleanupReplicaSets(client, namespace, instanceId)
-	if err != nil {
-		return err
-	}

 	// Pods might still be running user functions, so we give them
 	// a few minutes before terminating them.  This time is the
@@ -104,52 +100,46 @@ func idleObjectReaper(kubeClient *kubernetes.Clientset,
 			log.Fatalf("Failed to get environment list: %v", err)
 		}

+		envList := make(map[types.UID]struct{})
 		for i := range envs.Items {
 			env := envs.Items[i]
-			if env.Spec.AllowedFunctionsPerContainer == fission.AllowedFunctionsPerContainerInfinite {
-				continue
+			envList[env.Metadata.UID] = struct{}{}
+		}
+
+		funcSvcs, err := fsCache.ListOld(idlePodReapTime)
+		if err != nil {
+			log.Printf("Error reaping idle pods: %v", err)
+			continue
+		}
+
+		for _, fsvc := range funcSvcs {
+			if _, ok := envList[fsvc.Environment.Metadata.UID]; !ok {
+				log.Printf("Environment %v for function %v no longer exists",
+					fsvc.Environment.Metadata.Name, fsvc.Name)
 			}
-			funcSvcs, err := fsCache.ListOld(&env.Metadata, idlePodReapTime)
-			if err != nil {
-				log.Printf("Error reaping idle pods: %v", err)
+
+			if fsvc.Environment.Spec.AllowedFunctionsPerContainer == fission.AllowedFunctionsPerContainerInfinite {
 				continue
 			}

-			for _, fsvc := range funcSvcs {
-
-				fn, err := fissionClient.Functions(fsvc.Function.Namespace).Get(fsvc.Function.Name)
-				if err == nil {
-					// Ignore functions of NewDeploy ExecutorType with MinScale > 0
-					if fn.Spec.InvokeStrategy.ExecutionStrategy.MinScale > 0 &&
-						fn.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType == fission.ExecutorTypeNewdeploy {
-						continue
+			// Newdeploy manager handles the function delete event and clean cache/kubeobjs itself,
+			// so we ignore the function service cache with newdepoy executor type here.
+			if fsvc.Executor != fscache.NEWDEPLOY {
+				deleted, err := fsCache.DeleteOld(fsvc, idlePodReapTime)
+				if err != nil {
+					log.Printf("Error deleting Kubernetes objects for fsvc '%v': %v", fsvc, err)
+					log.Printf("Object Name| Object Kind | Object Namespace")
+					for _, kubeobj := range fsvc.KubernetesObjects {
+						log.Printf("%v | %v | %v", kubeobj.Name, kubeobj.Kind, kubeobj.Namespace)
 					}
 				}

-				// Return errors not equal to "is not found" error
-				if err != nil && !errors.IsNotFound(err) {
-					log.Printf("Error getting function: %v", fsvc.Function.Name)
+				if !deleted {
 					continue
 				}

-				// Newdeploy manager handles the function delete event and clean cache/kubeobjs itself,
-				// so we ignore the function service cache with newdepoy executor type here.
-				if fsvc.Executor != fscache.NEWDEPLOY {
-					deleted, err := fsCache.DeleteOld(fsvc, idlePodReapTime)
-					if err != nil {
-						log.Printf("Error deleting Kubernetes objects for fsvc '%v': %v", fsvc, err)
-						log.Printf("Object Name| Object Kind | Object Namespace")
-						for _, kubeobj := range fsvc.KubernetesObjects {
-							log.Printf("%v | %v | %v", kubeobj.Name, kubeobj.Kind, kubeobj.Namespace)
-						}
-					}
-
-					if !deleted {
-						continue
-					}
-					for _, kubeobj := range fsvc.KubernetesObjects {
-						deleteKubeobject(kubeClient, &kubeobj)
-					}
+				for _, kubeobj := range fsvc.KubernetesObjects {
+					deleteKubeobject(kubeClient, &kubeobj)
 				}
 			}
 		}
@@ -167,10 +157,8 @@ func deleteKubeobject(kubeClient *kubernetes.Clientset, kubeobj *api.ObjectRefer
 		logErr(fmt.Sprintf("cleaning up service %v ", kubeobj.Name), err)

 	case "deployment":
-		depl, err := kubeClient.ExtensionsV1beta1().Deployments(kubeobj.Namespace).Get(kubeobj.Name, meta_v1.GetOptions{})
-		err = kubeClient.ExtensionsV1beta1().Deployments(kubeobj.Namespace).Delete(kubeobj.Name, nil)
+		err := kubeClient.ExtensionsV1beta1().Deployments(kubeobj.Namespace).Delete(kubeobj.Name, &delOpt)
 		logErr(fmt.Sprintf("cleaning up deployment %v ", kubeobj.Name), err)
-		cleanupDeploymentObjects(kubeClient, kubeobj.Namespace, depl.Labels)

 	case "horizontalpodautoscaler":
 		err := kubeClient.AutoscalingV1().HorizontalPodAutoscalers(kubeobj.Namespace).Delete(kubeobj.Name, nil)
@@ -182,22 +170,6 @@ func deleteKubeobject(kubeClient *kubernetes.Clientset, kubeobj *api.ObjectRefer
 	}
 }

-func cleanupDeploymentObjects(kubeClient *kubernetes.Clientset, namespace string, sel map[string]string) {
-	rsList, err := kubeClient.ExtensionsV1beta1().ReplicaSets(namespace).List(meta_v1.ListOptions{LabelSelector: labels.Set(sel).AsSelector().String()})
-	logErr("Getting replicaset for deployment ", err)
-	for _, rs := range rsList.Items {
-		err = kubeClient.ExtensionsV1beta1().ReplicaSets(namespace).Delete(rs.Name, nil)
-		logErr(fmt.Sprintf("Cleaning replicaset %v for deployment", rs.Name), err)
-	}
-
-	podList, err := kubeClient.CoreV1().Pods(namespace).List(meta_v1.ListOptions{LabelSelector: labels.Set(sel).AsSelector().String()})
-	logErr("Getting pods for deployment ", err)
-	for _, pod := range podList.Items {
-		err = kubeClient.CoreV1().Pods(namespace).Delete(pod.Name, nil)
-		logErr(fmt.Sprintf("Cleaning pod %v for deployment", pod.Name), err)
-	}
-}
-
 func cleanupDeployments(client *kubernetes.Clientset, namespace string, instanceId string) error {
 	deploymentList, err := client.ExtensionsV1beta1().Deployments(namespace).List(meta_v1.ListOptions{})
 	if err != nil {
@@ -207,7 +179,7 @@ func cleanupDeployments(client *kubernetes.Clientset, namespace string, instance
 		id, ok := dep.ObjectMeta.Labels[fission.EXECUTOR_INSTANCEID_LABEL]
 		if ok && id != instanceId {
 			log.Printf("Cleaning up deployment %v", dep.ObjectMeta.Name)
-			err := client.ExtensionsV1beta1().Deployments(namespace).Delete(dep.ObjectMeta.Name, nil)
+			err := client.ExtensionsV1beta1().Deployments(namespace).Delete(dep.ObjectMeta.Name, &delOpt)
 			logErr("cleaning up deployment", err)
 			// ignore err
 		}
@@ -215,7 +187,7 @@ func cleanupDeployments(client *kubernetes.Clientset, namespace string, instance
 		pid, pok := dep.ObjectMeta.Labels[fission.POOLMGR_INSTANCEID_LABEL]
 		if pok && pid != instanceId {
 			log.Printf("Cleaning up deployment %v", dep.ObjectMeta.Name)
-			err := client.ExtensionsV1beta1().Deployments(namespace).Delete(dep.ObjectMeta.Name, nil)
+			err := client.ExtensionsV1beta1().Deployments(namespace).Delete(dep.ObjectMeta.Name, &delOpt)
 			logErr("cleaning up deployment", err)
 			// ignore err
 		}
@@ -223,35 +195,13 @@ func cleanupDeployments(client *kubernetes.Clientset, namespace string, instance
 	return nil
 }

-func cleanupReplicaSets(client *kubernetes.Clientset, namespace string, instanceId string) error {
-	rsList, err := client.ExtensionsV1beta1().ReplicaSets(namespace).List(meta_v1.ListOptions{})
-	if err != nil {
-		return err
-	}
-	for _, rs := range rsList.Items {
-		id, ok := rs.ObjectMeta.Labels[fission.EXECUTOR_INSTANCEID_LABEL]
-		if ok && id != instanceId {
-			log.Printf("Cleaning up replicaset %v", rs.ObjectMeta.Name)
-			err := client.ExtensionsV1beta1().ReplicaSets(namespace).Delete(rs.ObjectMeta.Name, nil)
-			logErr("cleaning up replicaset", err)
-		}
-		// Backward compatibility with older label name
-		pid, pok := rs.ObjectMeta.Labels[fission.POOLMGR_INSTANCEID_LABEL]
-		if pok && pid != instanceId {
-			log.Printf("Cleaning up replicaset %v", rs.ObjectMeta.Name)
-			err := client.ExtensionsV1beta1().ReplicaSets(namespace).Delete(rs.ObjectMeta.Name, nil)
-			logErr("cleaning up replicaset", err)
-		}
-	}
-	return nil
-}
-
 func cleanupPods(client *kubernetes.Clientset, namespace string, instanceId string) error {
 	podList, err := client.CoreV1().Pods(namespace).List(meta_v1.ListOptions{})
 	if err != nil {
 		return err
 	}
 	for _, pod := range podList.Items {
+		log.Printf("Clean pod: %v", pod.ObjectMeta.Name)
 		id, ok := pod.ObjectMeta.Labels[fission.EXECUTOR_INSTANCEID_LABEL]
 		if ok && id != instanceId {
 			log.Printf("Cleaning up pod %v", pod.ObjectMeta.Name)
@@ -119,6 +119,15 @@ func (executor *Executor) serveCreateFuncServices() {
 	}
 }

+func (executor *Executor) getFunctionExecutorType(meta *metav1.ObjectMeta) (fission.ExecutorType, error) {
+	fn, err := executor.fissionClient.Functions(meta.Namespace).Get(meta.Name)
+	if err != nil {
+		return "", err
+	}
+	return fn.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType, nil
+
+}
+
 func (executor *Executor) createServiceForFunction(meta *metav1.ObjectMeta) (*fscache.FuncSvc, error) {
 	log.Printf("[%v] No cached function service found, creating one", meta.Name)

@@ -129,14 +138,12 @@ func (executor *Executor) createServiceForFunction(meta *metav1.ObjectMeta) (*fs
 		return nil, err
 	}

-	fn, err := executor.fissionClient.
-		Functions(meta.Namespace).
-		Get(meta.Name)
+	executorType, err := executor.getFunctionExecutorType(meta)
 	if err != nil {
 		return nil, err
 	}

-	switch fn.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType {
+	switch executorType {
 	case fission.ExecutorTypeNewdeploy:
 		fs, err := executor.ndm.GetFuncSvc(meta)
 		return fs, err
@@ -182,6 +189,15 @@ func (executor *Executor) getFunctionEnv(m *metav1.ObjectMeta) (*crd.Environment
 	return env, nil
 }

+// isValidAddress invokes isValidService or isValidPod depending on the type of executor
+func (executor *Executor) isValidAddress(fsvc *fscache.FuncSvc) bool {
+	if fsvc.Executor == fscache.NEWDEPLOY {
+		return executor.ndm.IsValidService(fsvc.Address)
+	} else {
+		return executor.gpm.IsValidPod(fsvc.KubernetesObjects, fsvc.Address)
+	}
+}
+
 func dumpStackTrace() {
 	debug.PrintStack()
 }
@@ -193,6 +209,12 @@ func StartExecutor(fissionNamespace string, functionNamespace string, port int)
 	fission.SetupStackTraceHandler()

 	fissionClient, kubernetesClient, _, err := crd.MakeFissionClient()
+
+	err = fissionClient.WaitForCRDs()
+	if err != nil {
+		log.Fatalf("Error waiting for CRDs: %v", err)
+	}
+
 	restClient := fissionClient.GetCrdClient()
 	if err != nil {
 		log.Printf("Failed to get kubernetes client: %v", err)
@@ -204,8 +226,9 @@ func StartExecutor(fissionNamespace string, functionNamespace string, port int)
 	poolID := strings.ToLower(uniuri.NewLen(8))
 	cleanupObjects(kubernetesClient, functionNamespace, poolID)
 	go idleObjectReaper(kubernetesClient, fissionClient, fsCache, time.Minute*2)
+
 	gpm := poolmgr.MakeGenericPoolManager(
-		fissionClient, kubernetesClient, fissionNamespace,
+		fissionClient, kubernetesClient,
 		functionNamespace, fsCache, poolID)

 	ndm := newdeploy.MakeNewDeploy(
@@ -138,7 +138,11 @@ func TestExecutor(t *testing.T) {
 	if err != nil {
 		log.Panicf("failed to ensure crds: %v", err)
 	}
-	fissionClient.WaitForCRDs()
+
+	err = fissionClient.WaitForCRDs()
+	if err != nil {
+		log.Panicf("failed to wait crds: %v", err)
+	}

 	// create an env on the cluster
 	env, err := fissionClient.Environments(fissionNs).Create(&crd.Environment{
@@ -68,7 +68,6 @@ type (
 		address           string
 		kubernetesObjects []api.ObjectReference
 		age               time.Duration
-		env               *metav1.ObjectMeta // used for ListOld
 		responseChannel   chan *fscResponse
 	}
 	fscResponse struct {
@@ -117,8 +116,7 @@ func (fsc *FunctionServiceCache) service() {
 			funcObjects := make([]*FuncSvc, 0)
 			for _, funcSvc := range fscs {
 				fsvc := funcSvc.(*FuncSvc)
-				if fsvc.Environment.Metadata.UID == req.env.UID &&
-					time.Since(fsvc.Atime) > req.age {
+				if time.Since(fsvc.Atime) > req.age {
 					funcObjects = append(funcObjects, fsvc)
 				}
 			}
@@ -247,24 +245,27 @@ func (fsc *FunctionServiceCache) _touchByAddress(address string) error {
 	return nil
 }

+func (fsc *FunctionServiceCache) DeleteEntry(fsvc *FuncSvc) {
+	fsc.byFunction.Delete(crd.CacheKey(fsvc.Function))
+	fsc.byAddress.Delete(fsvc.Address)
+	fsc.byFunctionUID.Delete(fsvc.Function.UID)
+}
+
 func (fsc *FunctionServiceCache) DeleteOld(fsvc *FuncSvc, minAge time.Duration) (bool, error) {
 	if time.Since(fsvc.Atime) < minAge {
 		return false, nil
 	}

-	fsc.byFunction.Delete(crd.CacheKey(fsvc.Function))
-	fsc.byAddress.Delete(fsvc.Address)
-	fsc.byFunctionUID.Delete(fsvc.Function.UID)
+	fsc.DeleteEntry(fsvc)

 	return true, nil
 }

-func (fsc *FunctionServiceCache) ListOld(env *metav1.ObjectMeta, age time.Duration) ([]*FuncSvc, error) {
+func (fsc *FunctionServiceCache) ListOld(age time.Duration) ([]*FuncSvc, error) {
 	responseChannel := make(chan *fscResponse)
 	fsc.requestChannel <- &fscRequest{
 		requestType:     LISTOLD,
 		age:             age,
-		env:             env,
 		responseChannel: responseChannel,
 	}
 	resp := <-responseChannel
@@ -116,7 +116,10 @@ func (deploy *NewDeploy) getDeploymentSpec(fn *crd.Function, env *crd.Environmen
 		replicas = 1
 	}
 	targetFilename := "user"
-	var gracePeriodSeconds int64 = 6 * 60
+	gracePeriodSeconds := int64(6 * 60)
+	if env.Spec.TerminationGracePeriod > 0 {
+		gracePeriodSeconds = env.Spec.TerminationGracePeriod
+	}

 	fetchReq := &fetcher.FetchRequest{
 		FetchType: fetcher.FETCH_DEPLOYMENT,
@@ -22,6 +22,7 @@ import (
 	"log"
 	"os"
 	"strconv"
+	"strings"
 	"time"

 	"github.com/pkg/errors"
@@ -479,9 +480,14 @@ func (deploy *NewDeploy) fnDelete(fn *crd.Function) (*fscache.FuncSvc, error) {

 	var delError error

-	fsvc, err := deploy.fsCache.GetByFunction(&fn.Metadata)
+	// GetByFunction uses resource version as part of cache key, however,
+	// the resource version in function metadata will be changed when a function
+	// is deleted and cause newdeploy backend fails to delete the entry.
+	// Use GetByFunctionUID instead of GetByFunction here to find correct
+	// fsvc entry.
+	fsvc, err := deploy.fsCache.GetByFunctionUID(fn.Metadata.UID)
 	if err != nil {
-		log.Printf("fsvc not fonud in cache: %v", fn.Metadata)
+		log.Printf("fsvc not found in cache: %v", fn.Metadata)
 		delError = err
 		return nil, err
 	}
@@ -553,3 +559,18 @@ func (deploy *NewDeploy) updateKubeObjRefRV(fsvc *fscache.FuncSvc, objKind strin
 func updateStatus(fn *crd.Function, err error, message string) {
 	log.Printf(message, err)
 }
+
+// IsValidService does a get on the service address to ensure it's a valid service. returns true if it is, else false.
+func (deploy *NewDeploy) IsValidService(svc string) bool {
+	service := strings.Split(svc, ".")
+	if len(service) == 0 {
+		return false
+	}
+	svcObj, err := deploy.kubernetesClient.CoreV1().Services(service[1]).Get(service[0], metav1.GetOptions{})
+	if err == nil {
+		log.Printf("Valid service address : %s", svcObj.Spec.ClusterIP)
+		return true
+	}
+
+	return false
+}
@@ -0,0 +1,125 @@
+/*
+Copyright 2018 The Fission Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package poolmgr
+
+import (
+	"time"
+
+	log "github.com/sirupsen/logrus"
+	kerrors "k8s.io/apimachinery/pkg/api/errors"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/fields"
+	"k8s.io/apimachinery/pkg/util/intstr"
+	"k8s.io/client-go/kubernetes"
+	apiv1 "k8s.io/client-go/pkg/api/v1"
+	"k8s.io/client-go/rest"
+	k8sCache "k8s.io/client-go/tools/cache"
+
+	"github.com/fission/fission"
+	"github.com/fission/fission/crd"
+)
+
+func getIstioServiceLabels(fnName string) map[string]string {
+	return map[string]string{
+		"functionName": fnName,
+	}
+}
+
+func makeFuncIstioServiceRegister(crdClient *rest.RESTClient,
+	kubernetesClient *kubernetes.Clientset, fnNamespace string) k8sCache.Controller {
+
+	resyncPeriod := 30 * time.Second
+	lw := k8sCache.NewListWatchFromClient(crdClient, "functions", metav1.NamespaceDefault, fields.Everything())
+	_, controller := k8sCache.NewInformer(lw, &crd.Function{}, resyncPeriod,
+		k8sCache.ResourceEventHandlerFuncs{
+			AddFunc: func(obj interface{}) {
+
+				fn := obj.(*crd.Function)
+
+				// Since istio only allows accessing pod through k8s service,
+				// for the functions with executor type "poolmgr" we need to
+				// create a service for sending requests to pod in pool.
+				// Functions with executor type "Newdeploy" is specialized at
+				// pod starts. In this case, just ignore such functions.
+				fnExecutorType := fn.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType
+				if fnExecutorType != fission.ExecutorTypePoolmgr {
+					return
+				}
+
+				// create a same name service for function
+				// since istio only allows the traffic to service
+				sel := map[string]string{
+					"functionName": fn.Metadata.Name,
+					"functionUid":  string(fn.Metadata.UID),
+				}
+
+				svcName := fission.GetFunctionIstioServiceName(fn.Metadata.Name, fn.Metadata.Namespace)
+
+				// service for accepting user traffic
+				svc := apiv1.Service{
+					ObjectMeta: metav1.ObjectMeta{
+						Namespace: fnNamespace,
+						Name:      svcName,
+						Labels:    getIstioServiceLabels(fn.Metadata.Name),
+					},
+					Spec: apiv1.ServiceSpec{
+						Type: apiv1.ServiceTypeClusterIP,
+						Ports: []apiv1.ServicePort{
+							// Service port name should begin with a recognized prefix, or the traffic will be
+							// treated as TCP traffic. (https://istio.io/docs/setup/kubernetes/sidecar-injection.html)
+							// Originally the ports' name are similar to "http-fetch" and "http-specialize".
+							// But for istio 0.5.1, istio-proxy return unexpected 431 error with such naming.
+							// https://github.com/istio/istio/issues/928
+							// Workaround: remove prefix
+							// TODO: prepend prefix once the bug fixed
+							{
+								Name:       "fetch",
+								Protocol:   apiv1.ProtocolTCP,
+								Port:       8000,
+								TargetPort: intstr.FromInt(8000),
+							},
+							{
+								Name:       "specialize",
+								Protocol:   apiv1.ProtocolTCP,
+								Port:       8888,
+								TargetPort: intstr.FromInt(8888),
+							},
+						},
+						Selector: sel,
+					},
+				}
+
+				// create function istio service if it does not exist
+				_, err := kubernetesClient.CoreV1().Services(fnNamespace).Create(&svc)
+				if err != nil && !kerrors.IsAlreadyExists(err) {
+					log.Printf("Error creating function istio service: %v", err)
+				}
+			},
+			DeleteFunc: func(obj interface{}) {
+				fn := obj.(*crd.Function)
+				svcName := fission.GetFunctionIstioServiceName(fn.Metadata.Name, fn.Metadata.Namespace)
+				// delete function istio service
+				err := kubernetesClient.CoreV1().Services(fnNamespace).Delete(svcName, nil)
+				if err != nil && !kerrors.IsNotFound(err) {
+					log.Printf("Error deleting function istio service: %v", err)
+				}
+			},
+			UpdateFunc: func(oldObj, newObj interface{}) {},
+		})
+
+	return controller
+}
@@ -28,7 +28,6 @@ import (
 	"net/url"
 	"os"
 	"path/filepath"
-	"strconv"
 	"strings"
 	"time"

@@ -105,7 +104,8 @@ func MakeGenericPool(
 	initialReplicas int32,
 	namespace string,
 	fsCache *fscache.FunctionServiceCache,
-	instanceId string) (*GenericPool, error) {
+	instanceId string,
+	enableIstio bool) (*GenericPool, error) {

 	log.Printf("Creating pool for environment %v", env.Metadata)

@@ -122,16 +122,6 @@ func MakeGenericPool(
 		runtimeImagePullPolicy = "IfNotPresent"
 	}

-	enableIstio := false
-
-	if len(os.Getenv("ENABLE_ISTIO")) > 0 {
-		istio, err := strconv.ParseBool(os.Getenv("ENABLE_ISTIO"))
-		if err != nil {
-			log.Println("Failed to parse ENABLE_ISTIO")
-		}
-		enableIstio = istio
-	}
-
 	// TODO: in general we need to provide the user a way to configure pools.  Initial
 	// replicas, autoscaling params, various timeouts, etc.
 	gp := &GenericPool{
@@ -479,7 +469,10 @@ func (gp *GenericPool) createPool() error {

 	// Use long terminationGracePeriodSeconds for connection draining in case that
 	// pod still runs user functions.
-	var gracePeriodSeconds int64 = 6 * 60
+	gracePeriodSeconds := int64(6 * 60)
+	if gp.env.Spec.TerminationGracePeriod > 0 {
+		gracePeriodSeconds = gp.env.Spec.TerminationGracePeriod
+	}

 	podAnnotation := make(map[string]string)

@@ -750,7 +743,7 @@ func (gp *GenericPool) GetFuncSvc(m *metav1.ObjectMeta) (*fscache.FuncSvc, error
 	log.Printf("Specialized pod: %v", pod.ObjectMeta.Name)

 	var svcHost string
-	if gp.useSvc {
+	if gp.useSvc && !gp.useIstio {
 		svcName := fmt.Sprintf("svc-%v", m.Name)
 		if len(m.UID) > 0 {
 			svcName = fmt.Sprintf("%s-%v", svcName, m.UID)
@@ -809,40 +802,15 @@ func (gp *GenericPool) GetFuncSvc(m *metav1.ObjectMeta) (*fscache.FuncSvc, error

 // destroys the pool -- the deployment, replicaset and pods
 func (gp *GenericPool) destroy() error {
-	// Destroy deployment
-	err := gp.kubernetesClient.ExtensionsV1beta1().Deployments(gp.namespace).Delete(gp.deployment.ObjectMeta.Name, nil)
+	deletePropagation := metav1.DeletePropagationBackground
+	delOpt := metav1.DeleteOptions{
+		PropagationPolicy: &deletePropagation,
+	}
+	err := gp.kubernetesClient.ExtensionsV1beta1().
+		Deployments(gp.namespace).Delete(gp.deployment.ObjectMeta.Name, &delOpt)
 	if err != nil {
 		log.Printf("Error destroying deployment: %v", err)
 		return err
 	}
-
-	// Destroy ReplicaSet.  Pre-1.6 K8s versions don't do this
-	// automatically but post-1.6 K8s will, and may beat us to it,
-	// so don't error out if we fail.
-	rsList, err := gp.kubernetesClient.ExtensionsV1beta1().ReplicaSets(gp.namespace).List(metav1.ListOptions{
-		LabelSelector: labels.Set(gp.labelsForPool).AsSelector().String(),
-	})
-	if len(rsList.Items) >= 0 {
-		for _, rs := range rsList.Items {
-			err = gp.kubernetesClient.ExtensionsV1beta1().ReplicaSets(gp.namespace).Delete(rs.ObjectMeta.Name, nil)
-			if err != nil {
-				log.Printf("Error deleting replicaset, ignoring: %v", err)
-			}
-		}
-	}
-
-	// Destroy Pods.  See note above.
-	podList, err := gp.kubernetesClient.CoreV1().Pods(gp.namespace).List(metav1.ListOptions{
-		LabelSelector: labels.Set(gp.labelsForPool).AsSelector().String(),
-	})
-	if len(podList.Items) >= 0 {
-		for _, pod := range podList.Items {
-			err = gp.kubernetesClient.CoreV1().Pods(gp.namespace).Delete(pod.ObjectMeta.Name, nil)
-			if err != nil {
-				log.Printf("Error deleting pod, ignoring: %v", err)
-			}
-		}
-	}
-
 	return nil
 }
@@ -17,11 +17,17 @@ limitations under the License.
 package poolmgr

 import (
+	"context"
 	"log"
+	"os"
+	"strconv"
+	"strings"
 	"time"

 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/client-go/kubernetes"
+	"k8s.io/client-go/pkg/api"
+	k8sCache "k8s.io/client-go/tools/cache"

 	"github.com/fission/fission"
 	"github.com/fission/fission/crd"
@@ -45,6 +51,9 @@ type (
 		fsCache        *fscache.FunctionServiceCache
 		instanceId     string
 		requestChannel chan *request
+
+		enableIstio          bool
+		istioServiceRegister k8sCache.Controller
 	}
 	request struct {
 		requestType
@@ -61,7 +70,6 @@ type (
 func MakeGenericPoolManager(
 	fissionClient *crd.FissionClient,
 	kubernetesClient *kubernetes.Clientset,
-	fissionNamespace string,
 	functionNamespace string,
 	fsCache *fscache.FunctionServiceCache,
 	instanceId string) *GenericPoolManager {
@@ -78,9 +86,28 @@ func MakeGenericPoolManager(
 	go gpm.service()
 	go gpm.eagerPoolCreator()

+	if len(os.Getenv("ENABLE_ISTIO")) > 0 {
+		istio, err := strconv.ParseBool(os.Getenv("ENABLE_ISTIO"))
+		if err != nil {
+			log.Println("Failed to parse ENABLE_ISTIO")
+		}
+		gpm.enableIstio = istio
+
+		if gpm.enableIstio {
+			gpm.istioServiceRegister = makeFuncIstioServiceRegister(
+				gpm.fissionClient.GetCrdClient(), gpm.kubernetesClient, functionNamespace)
+		}
+	}
+
 	return gpm
 }

+func (gpm *GenericPoolManager) Run(ctx context.Context) {
+	if gpm.enableIstio && gpm.istioServiceRegister != nil {
+		go gpm.istioServiceRegister.Run(ctx.Done())
+	}
+}
+
 func (gpm *GenericPoolManager) service() {
 	for {
 		req := <-gpm.requestChannel
@@ -97,7 +124,7 @@ func (gpm *GenericPoolManager) service() {

 				pool, err = MakeGenericPool(
 					gpm.fissionClient, gpm.kubernetesClient, req.env, poolsize,
-					gpm.namespace, gpm.fsCache, gpm.instanceId)
+					gpm.namespace, gpm.fsCache, gpm.instanceId, gpm.enableIstio)
 				if err != nil {
 					req.responseChannel <- &response{error: err}
 					continue
@@ -192,3 +219,18 @@ func (gpm *GenericPoolManager) getEnvPoolsize(env *crd.Environment) int32 {
 	}
 	return poolsize
 }
+
+// IsValidPod checks if pod is not deleted and that it has the address passed as the argument. Also checks that all the
+// containers in it are reporting a ready status for the healthCheck.
+func (gpm *GenericPoolManager) IsValidPod(kubeObjects []api.ObjectReference, podAddress string) bool {
+	for _, obj := range kubeObjects {
+		if obj.Kind == "pod" {
+			pod, err := gpm.kubernetesClient.CoreV1().Pods(obj.Namespace).Get(obj.Name, metav1.GetOptions{})
+			if err == nil && strings.Contains(podAddress, pod.Status.PodIP) && fission.IsReadyPod(pod) {
+				log.Printf("Valid pod address : %s", podAddress)
+				return true
+			}
+		}
+	}
+	return false
+}
@@ -40,6 +40,11 @@ import (
 	storageSvcClient "github.com/fission/fission/storagesvc/client"
 )

+var (
+	// global verbosity of our CLI
+	verbosity int
+)
+
 func fatal(msg string) {
 	os.Stderr.WriteString(msg + "\n")
 	os.Exit(1)
@@ -49,10 +54,16 @@ func warn(msg string) {
 	os.Stderr.WriteString(msg + "\n")
 }

-func getClient(serverUrl string) *client.Client {
+func verbose(msglevel int, format string, args ...interface{}) {
+	if verbosity >= msglevel {
+		fmt.Printf(format+"\n", args...)
+	}
+}

+func getClient(serverUrl string) *client.Client {
 	if len(serverUrl) == 0 {
-		fatal("Need --server or FISSION_URL set to your fission server.")
+		// starts local portforwarder etc.
+		serverUrl = getServerUrl()
 	}

 	isHTTPS := strings.Index(serverUrl, "https://") == 0
@@ -55,22 +55,28 @@ func envCreate(c *cli.Context) error {
 	envBuilderImg := c.String("builder")
 	envBuildCmd := c.String("buildcmd")
 	envExternalNetwork := c.Bool("externalnetwork")
+	envGracePeriod := c.Int64("period")
+	if envGracePeriod <= 0 {
+		envGracePeriod = 360
+	}

 	if len(envBuilderImg) > 0 {
-		envVersion = 2
+		if !c.IsSet("version") {
+			envVersion = 2
+		}
 		if len(envBuildCmd) == 0 {
 			envBuildCmd = "build"
 		}
 	}

-	resourceReq := getResourceReq(c)
-
 	// Environment API interface version is not specified and
 	// builder image is empty, set default interface version
 	if envVersion == 0 {
 		envVersion = 1
 	}

+	resourceReq := getResourceReq(c, v1.ResourceRequirements{})
+
 	env := &crd.Environment{
 		Metadata: metav1.ObjectMeta{
 			Name:      envName,
@@ -88,9 +94,18 @@ func envCreate(c *cli.Context) error {
 			Poolsize:                     poolsize,
 			Resources:                    resourceReq,
 			AllowAccessToExternalNetwork: envExternalNetwork,
+			TerminationGracePeriod:       envGracePeriod,
 		},
 	}

+	// if we're writing a spec, don't call the API
+	if c.Bool("spec") {
+		specFile := fmt.Sprintf("env-%v.yaml", envName)
+		err := specSave(*env, specFile)
+		checkErr(err, "create environment spec")
+		return nil
+	}
+
 	_, err := client.EnvironmentCreate(env)
 	checkErr(err, "create environment")

@@ -163,6 +178,10 @@ func envUpdate(c *cli.Context) error {
 		env.Spec.Poolsize = c.Int("poolsize")
 	}

+	if c.IsSet("period") {
+		env.Spec.TerminationGracePeriod = c.Int64("period")
+	}
+
 	env.Spec.AllowAccessToExternalNetwork = envExternalNetwork

 	_, err = client.EnvironmentUpdate(env)
@@ -198,66 +217,75 @@ func envList(c *cli.Context) error {
 	checkErr(err, "list environments")

 	w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
-	fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\n", "NAME", "UID", "IMAGE", "POOLSIZE", "MINCPU", "MAXCPU", "MINMEMORY", "MAXMEMORY")
+	fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\n", "NAME", "UID", "IMAGE", "POOLSIZE", "MINCPU", "MAXCPU", "MINMEMORY", "MAXMEMORY", "EXTNET", "GRACETIME")
 	for _, env := range envs {
-		fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\n",
+		fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\n",
 			env.Metadata.Name, env.Metadata.UID, env.Spec.Runtime.Image, env.Spec.Poolsize,
 			env.Spec.Resources.Requests.Cpu(), env.Spec.Resources.Limits.Cpu(),
-			env.Spec.Resources.Requests.Memory(), env.Spec.Resources.Limits.Memory())
+			env.Spec.Resources.Requests.Memory(), env.Spec.Resources.Limits.Memory(),
+			env.Spec.AllowAccessToExternalNetwork, env.Spec.TerminationGracePeriod)
 	}
 	w.Flush()

 	return nil
 }

-func getResourceReq(c *cli.Context) v1.ResourceRequirements {
-	if c.IsSet("mincpu") || c.IsSet("maxcpu") || c.IsSet("minmemory") || c.IsSet("maxmemory") {
-		mincpu := c.Int("mincpu")
-		maxcpu := c.Int("maxcpu")
-		minmem := c.Int("minmemory")
-		maxmem := c.Int("maxmemory")
+func getResourceReq(c *cli.Context, resources v1.ResourceRequirements) v1.ResourceRequirements {

-		requestResources := make(map[v1.ResourceName]resource.Quantity)
+	var requestResources map[v1.ResourceName]resource.Quantity

-		if mincpu != 0 {
-			cpuRequest, err := resource.ParseQuantity(strconv.Itoa(mincpu) + "m")
-			if err != nil {
-				fatal("Failed to parse mincpu")
-			}
-			requestResources[v1.ResourceCPU] = cpuRequest
-		}
-
-		if minmem != 0 {
-			memRequest, err := resource.ParseQuantity(strconv.Itoa(minmem) + "Mi")
-			if err != nil {
-				fatal("Failed to parse minmemory")
-			}
-			requestResources[v1.ResourceMemory] = memRequest
-		}
-
-		limitResources := make(map[v1.ResourceName]resource.Quantity)
-
-		if maxcpu != 0 {
-			cpuLimit, err := resource.ParseQuantity(strconv.Itoa(maxcpu) + "m")
-			if err != nil {
-				fatal("Failed to parse maxcpu")
-			}
-			limitResources[v1.ResourceCPU] = cpuLimit
-		}
-
-		if maxmem != 0 {
-			memLimit, err := resource.ParseQuantity(strconv.Itoa(maxmem) + "Mi")
-			if err != nil {
-				fatal("Failed to parse maxmemory")
-			}
-			limitResources[v1.ResourceMemory] = memLimit
-		}
-
-		resources := v1.ResourceRequirements{
-			Requests: requestResources,
-			Limits:   limitResources,
-		}
-		return resources
+	if len(resources.Requests) == 0 {
+		requestResources = make(map[v1.ResourceName]resource.Quantity)
+	} else {
+		requestResources = resources.Requests
 	}
-	return v1.ResourceRequirements{}
+
+	if c.IsSet("mincpu") {
+		mincpu := c.Int("mincpu")
+		cpuRequest, err := resource.ParseQuantity(strconv.Itoa(mincpu) + "m")
+		if err != nil {
+			fatal("Failed to parse mincpu")
+		}
+		requestResources[v1.ResourceCPU] = cpuRequest
+	}
+
+	if c.IsSet("minmemory") {
+		minmem := c.Int("minmemory")
+		memRequest, err := resource.ParseQuantity(strconv.Itoa(minmem) + "Mi")
+		if err != nil {
+			fatal("Failed to parse minmemory")
+		}
+		requestResources[v1.ResourceMemory] = memRequest
+	}
+
+	var limitResources map[v1.ResourceName]resource.Quantity
+	if len(resources.Limits) == 0 {
+		limitResources = make(map[v1.ResourceName]resource.Quantity)
+	} else {
+		limitResources = resources.Limits
+	}
+
+	if c.IsSet("maxcpu") {
+		maxcpu := c.Int("maxcpu")
+		cpuLimit, err := resource.ParseQuantity(strconv.Itoa(maxcpu) + "m")
+		if err != nil {
+			fatal("Failed to parse maxcpu")
+		}
+		limitResources[v1.ResourceCPU] = cpuLimit
+	}
+
+	if c.IsSet("maxmemory") {
+		maxmem := c.Int("maxmemory")
+		memLimit, err := resource.ParseQuantity(strconv.Itoa(maxmem) + "Mi")
+		if err != nil {
+			fatal("Failed to parse maxmemory")
+		}
+		limitResources[v1.ResourceMemory] = memLimit
+	}
+
+	resources = v1.ResourceRequirements{
+		Requests: requestResources,
+		Limits:   limitResources,
+	}
+	return resources
 }
@@ -31,6 +31,7 @@ import (
 	"github.com/satori/go.uuid"
 	"github.com/urfave/cli"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	apiv1 "k8s.io/client-go/pkg/api/v1"

 	"github.com/fission/fission"
 	"github.com/fission/fission/crd"
@@ -43,7 +44,7 @@ func printPodLogs(c *cli.Context) error {
 		fatal("Need --name argument.")
 	}

-	queryURL, err := url.Parse(c.GlobalString("server"))
+	queryURL, err := url.Parse(getServerUrl())
 	checkErr(err, "parse the base URL")
 	queryURL.Path = fmt.Sprintf("/proxy/logs/%s", fnName)

@@ -117,10 +118,6 @@ func getTargetCPU(c *cli.Context) int {
 func fnCreate(c *cli.Context) error {
 	client := getClient(c.GlobalString("server"))

-	if len(c.String("package")) > 0 {
-		fatal("--package is deprecated, please use --deploy instead.")
-	}
-
 	fnName := c.String("name")
 	if len(fnName) == 0 {
 		fatal("Need --name argument.")
@@ -200,7 +197,7 @@ func fnCreate(c *cli.Context) error {
 	}

 	invokeStrategy := getInvokeStrategy(c.Int("minscale"), c.Int("maxscale"), c.String("executortype"), getTargetCPU(c))
-	resourceReq := getResourceReq(c)
+	resourceReq := getResourceReq(c, apiv1.ResourceRequirements{})
 	if (c.IsSet("mincpu") || c.IsSet("maxcpu") || c.IsSet("minmemory") || c.IsSet("maxmemory")) &&
 		invokeStrategy.ExecutionStrategy.ExecutorType == fission.ExecutorTypePoolmgr {
 		warn("CPU/Memory specified for function with pool manager executor will be ignored in favor of resources specified at environment")
@@ -274,6 +271,10 @@ func fnCreate(c *cli.Context) error {
 	if len(triggerUrl) == 0 {
 		return nil
 	}
+	if !strings.HasPrefix(triggerUrl, "/") {
+		triggerUrl = fmt.Sprintf("/%s", triggerUrl)
+	}
+
 	method := c.String("method")
 	if len(method) == 0 {
 		method = "GET"
@@ -387,10 +388,8 @@ func fnUpdate(c *cli.Context) error {
 	secretNameSpace := c.String("secretNamespace")
 	cfgMapNameSpace := c.String("configmapNamespace")

-	if len(envName) == 0 && len(deployArchiveName) == 0 && len(srcArchiveName) == 0 && len(pkgName) == 0 &&
-		len(entrypoint) == 0 && len(buildcmd) == 0 && len(secretName) == 0 && len(secretNameSpace) == 0 &&
-		len(cfgMapName) == 0 && len(cfgMapNameSpace) == 0 {
-		fatal("Need --env or --deploy or --src or --pkg or --entrypoint or --buildcmd or --secret or --secretNamespace or --configmap or --configmapNamespace argument.")
+	if len(srcArchiveName) > 0 && len(deployArchiveName) > 0 {
+		fatal("Need either of --src or --deploy and not both arguments.")
 	}

 	if len(secretName) > 0 {
@@ -471,9 +470,11 @@ func fnUpdate(c *cli.Context) error {
 		ResourceVersion: pkgMetadata.ResourceVersion,
 	}

-	function.Spec.Resources = getResourceReq(c)
+	function.Spec.Resources = getResourceReq(c, function.Spec.Resources)

-	function.Spec.InvokeStrategy.ExecutionStrategy.TargetCPUPercent = getTargetCPU(c)
+	if c.IsSet("targetcpu") {
+		function.Spec.InvokeStrategy.ExecutionStrategy.TargetCPUPercent = getTargetCPU(c)
+	}

 	if c.IsSet("minscale") {
 		minscale := c.Int("minscale")
@@ -498,7 +499,7 @@ func fnUpdate(c *cli.Context) error {
 		function.Spec.InvokeStrategy.ExecutionStrategy.MaxScale = maxscale
 	}

-	if c.String("executortype") != "" {
+	if c.IsSet("executortype") {
 		var fnExecutor fission.ExecutorType
 		switch c.String("executortype") {
 		case "":
@@ -552,13 +553,19 @@ func fnList(c *cli.Context) error {

 	w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)

-	fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\n", "NAME", "UID", "ENV", "EXECUTORTYPE", "MINSCALE", "MAXSCALE", "TARGETCPU")
+	fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\n", "NAME", "UID", "ENV", "EXECUTORTYPE", "MINSCALE", "MAXSCALE", "MINCPU", "MAXCPU", "MINMEMORY", "MAXMEMORY", "TARGETCPU")
 	for _, f := range fns {
-		fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\n",
+		mincpu := f.Spec.Resources.Requests.Cpu
+		mincpu().Value()
+		fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\n",
 			f.Metadata.Name, f.Metadata.UID, f.Spec.Environment.Name,
 			f.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType,
 			f.Spec.InvokeStrategy.ExecutionStrategy.MinScale,
 			f.Spec.InvokeStrategy.ExecutionStrategy.MaxScale,
+			f.Spec.Resources.Requests.Cpu().String(),
+			f.Spec.Resources.Limits.Cpu().String(),
+			f.Spec.Resources.Requests.Memory().String(),
+			f.Spec.Resources.Limits.Memory().String(),
 			f.Spec.InvokeStrategy.ExecutionStrategy.TargetCPUPercent)
 	}
 	w.Flush()
@@ -595,7 +602,7 @@ func fnLogs(c *cli.Context) error {
 	checkErr(err, "get function")

 	// request the controller to establish a proxy server to the database.
-	logDB, err := logdb.GetLogDB(dbType, c.GlobalString("server"))
+	logDB, err := logdb.GetLogDB(dbType, getServerUrl())
 	if err != nil {
 		fatal("failed to connect log database")
 	}
@@ -647,48 +654,6 @@ func fnLogs(c *cli.Context) error {
 	}
 }

-func fnPods(c *cli.Context) error {
-	client := getClient(c.GlobalString("server"))
-
-	fnName := c.String("name")
-	if len(fnName) == 0 {
-		fatal("Need name of function, use --name")
-	}
-
-	dbType := c.String("dbtype")
-	if len(dbType) == 0 {
-		dbType = logdb.INFLUXDB
-	}
-
-	m := &metav1.ObjectMeta{Name: fnName}
-
-	f, err := client.FunctionGet(m)
-	checkErr(err, "get function")
-
-	// client first sends db query to the controller, then the controller
-	// will establish a proxy server that bridges the client and the database.
-	logDB, err := logdb.GetLogDB(dbType, c.GlobalString("server"))
-	if err != nil {
-		fatal("failed to connect log database")
-	}
-
-	logFilter := logdb.LogFilter{
-		Function: f.Metadata.Name,
-		FuncUid:  string(f.Metadata.UID),
-	}
-	pods, err := logDB.GetPods(logFilter)
-	if err != nil {
-		fatal("failed to get pods of function")
-		return err
-	}
-	fmt.Printf("NAME\t\n")
-	for _, pod := range pods {
-		fmt.Println(pod)
-	}
-
-	return err
-}
-
 func fnTest(c *cli.Context) error {
 	fnName := c.String("name")
 	if len(fnName) == 0 {
@@ -63,6 +63,7 @@ func checkFunctionExistence(fissionClient *client.Client, fnName string) {
 		Name:      fnName,
 		Namespace: metav1.NamespaceDefault,
 	}
+
 	_, err := fissionClient.FunctionGet(meta)
 	if err != nil {
 		fmt.Printf("function '%v' does not exist, use 'fission function create --name %v ...' to create the function\n", fnName, fnName)
@@ -80,6 +81,10 @@ func htCreate(c *cli.Context) error {
 	if len(triggerUrl) == 0 {
 		fatal("Need a trigger URL, use --url")
 	}
+	if !strings.HasPrefix(triggerUrl, "/") {
+		triggerUrl = fmt.Sprintf("/%s", triggerUrl)
+	}
+
 	method := c.String("method")
 	if len(method) == 0 {
 		method = "GET"
@@ -105,6 +110,14 @@ func htCreate(c *cli.Context) error {
 		},
 	}

+	// if we're writing a spec, don't call the API
+	if c.Bool("spec") {
+		specFile := fmt.Sprintf("route-%v.yaml", triggerName)
+		err := specSave(*ht, specFile)
+		checkErr(err, "create HTTP trigger spec")
+		return nil
+	}
+
 	_, err := client.HTTPTriggerCreate(ht)
 	checkErr(err, "create HTTP trigger")

@@ -45,28 +45,6 @@ type InfluxDB struct {
 	endpoint string
 }

-func (influx InfluxDB) GetPods(filter LogFilter) ([]string, error) {
-	parameters := make(map[string]interface{})
-	parameters["funcuid"] = filter.FuncUid
-
-	queryCmd := "select * from \"log\" where \"funcuid\" = $funcuid group by \"pod\""
-	query := influxdbClient.NewQueryWithParameters(queryCmd, INFLUXDB_DATABASE, "", parameters)
-
-	response, err := influx.query(query)
-	if err != nil /*|| response.Err != ""*/ {
-		return []string{}, err
-	}
-	pods := []string{}
-	for _, r := range response.Results {
-		for _, series := range r.Series {
-			for _, pod := range series.Tags {
-				pods = append(pods, pod)
-			}
-		}
-	}
-	return pods, nil
-}
-
 func makeIndexMap(cols []string) map[string]int {
 	indexMap := make(map[string]int, len(cols))
 	for i := range cols {
@@ -98,7 +76,7 @@ func (influx InfluxDB) GetLogs(filter LogFilter) ([]LogEntry, error) {
 	logEntries := []LogEntry{}
 	response, err := influx.query(query)
 	if err != nil {
-		return logEntries, nil
+		return logEntries, err
 	}
 	for _, r := range response.Results {
 		for _, series := range r.Series {
@@ -27,7 +27,6 @@ const (
 )

 type LogDatabase interface {
-	GetPods(LogFilter) ([]string, error)
 	GetLogs(LogFilter) ([]LogEntry, error)
 }

@@ -60,29 +60,38 @@ func getFissionAPIVersion(apiUrl string) (string, error) {
 	return strings.TrimRight(string(body), "\n"), nil
 }

-func main() {
-	app := cli.NewApp()
-	app.Name = "fission"
-	app.Usage = "Serverless functions for Kubernetes"
-	app.Version = version.Version
-
-	// fetch the FISSION_URL env variable. If not set, port-forward to controller.
-	var value string
+func getServerUrl() string {
+	var serverUrl string
+	// Use FISSION_URL env variable if set; otherwise, port-forward to controller.
 	fissionUrl := os.Getenv("FISSION_URL")
 	if len(fissionUrl) == 0 {
 		fissionNamespace := getFissionNamespace()
 		kubeConfig := getKubeConfigPath()
 		localPort := setupPortForward(
 			kubeConfig, fissionNamespace, "application=fission-api")
-		value = "http://127.0.0.1:" + localPort
+		serverUrl = "http://127.0.0.1:" + localPort
 	} else {
-		value = fissionUrl
+		serverUrl = fissionUrl
 	}
+	return serverUrl
+}
+
+func cliHook(c *cli.Context) error {
+	verbosity = c.Int("verbosity")
+	verbose(2, "Verbosity = 2")
+	return nil
+}
+
+func main() {
+	app := cli.NewApp()
+	app.Name = "fission"
+	app.Usage = "Serverless functions for Kubernetes"
+	app.Version = version.Version

 	cli.VersionPrinter = func(c *cli.Context) {
 		clientVer := version.VersionInfo().String()
 		fmt.Printf("Client Version: %v\n", clientVer)
-		serverVer, err := getFissionAPIVersion(value)
+		serverVer, err := getFissionAPIVersion(getServerUrl())
 		if err != nil {
 			fmt.Printf("Error getting Fission API version: %v", err)
 		} else {
@@ -91,11 +100,15 @@ func main() {
 	}

 	app.Flags = []cli.Flag{
-		cli.StringFlag{Name: "server", Value: value, Usage: "Fission server URL"},
+		cli.StringFlag{Name: "server", Value: "", Usage: "Fission server URL"},
+		cli.IntFlag{Name: "verbosity", Value: 1, Usage: "CLI verbosity (0 is quiet, 1 is the default, 2 is verbose.)"},
 	}

+	// all resource create commands accept --spec
+	specSaveFlag := cli.BoolFlag{Name: "spec", Usage: "Save to the spec directory instead of creating on cluster"}
+
 	// trigger method and url flags (used in function and route CLIs)
-	htMethodFlag := cli.StringFlag{Name: "method", Usage: "HTTP Method: GET|POST|PUT|DELETE|HEAD; defaults to GET"}
+	htMethodFlag := cli.StringFlag{Name: "method", Value: "GET", Usage: "HTTP Method: GET|POST|PUT|DELETE|HEAD"}
 	htUrlFlag := cli.StringFlag{Name: "url", Usage: "URL pattern (See gorilla/mux supported patterns)"}

 	// Resource & scale related flags (Used in env and function)
@@ -105,13 +118,12 @@ func main() {
 	maxMem := cli.StringFlag{Name: "maxmemory", Usage: "Maximum memory to be assigned to pod (In megabyte)"}
 	minScale := cli.StringFlag{Name: "minscale", Usage: "Minimum number of pods (Uses resource inputs to configure HPA)"}
 	maxScale := cli.StringFlag{Name: "maxscale", Usage: "Maximum number of pods (Uses resource inputs to configure HPA)"}
-	targetcpu := cli.StringFlag{Name: "targetcpu", Usage: "Target average CPU across pods for scaling (In percentage, defaults to 80)"}
+	targetcpu := cli.IntFlag{Name: "targetcpu", Value: 80, Usage: "Target average CPU usage percentage across pods for scaling"}

 	// functions
 	fnNameFlag := cli.StringFlag{Name: "name", Usage: "function name"}
 	fnEnvNameFlag := cli.StringFlag{Name: "env", Usage: "environment name for function"}
 	fnCodeFlag := cli.StringFlag{Name: "code", Usage: "local path or URL for source code"}
-	fnPackageFlag := cli.StringFlag{Name: "package", Usage: "(Deprecated) local path or URL for binary package"}
 	fnDeployArchiveFlag := cli.StringFlag{Name: "deployarchive, deploy", Usage: "local path or URL for deployment archive"}
 	fnSrcArchiveFlag := cli.StringFlag{Name: "sourcearchive, src", Usage: "local path or URL for source archive"}
 	fnPkgNameFlag := cli.StringFlag{Name: "pkgname, pkg", Usage: "Name of the existing package (--deploy and --src and --env will be ignored)"}
@@ -129,26 +141,24 @@ func main() {
 	fnCfgMapnsFlag := cli.StringFlag{Name: "configmapNamespace", Usage: "namespace of configmap"}
 	fnLogCountFlag := cli.StringFlag{Name: "recordcount", Usage: "the n most recent log records"}
 	fnForceFlag := cli.BoolFlag{Name: "force", Usage: "Force update a package even if it is used by one or more functions"}
-	fnExecutorTypeFlag := cli.StringFlag{Name: "executortype", Usage: "Executor type for execution; one of 'poolmgr', 'newdeploy' defaults to 'poolmgr'"}
-	fnSpecSaveFlag := cli.BoolFlag{Name: "spec", Usage: "Save function to the spec directory instead of creating it"}
+	fnExecutorTypeFlag := cli.StringFlag{Name: "executortype", Value: "poolmgr", Usage: "Executor type for execution; one of 'poolmgr', 'newdeploy'"}

 	fnSubcommands := []cli.Command{
-		{Name: "create", Usage: "Create new function (and optionally, an HTTP route to it)", Flags: []cli.Flag{fnNameFlag, fnEnvNameFlag, fnSpecSaveFlag, fnCodeFlag, fnPackageFlag, fnSrcArchiveFlag, fnDeployArchiveFlag, fnEntryPointFlag, fnBuildCmdFlag, fnPkgNameFlag, htUrlFlag, htMethodFlag, minCpu, maxCpu, minMem, maxMem, minScale, maxScale, fnExecutorTypeFlag, targetcpu, fnCfgMapFlag, fnSecretFlag, fnSecretnsFlag, fnCfgMapnsFlag}, Action: fnCreate},
+		{Name: "create", Usage: "Create new function (and optionally, an HTTP route to it)", Flags: []cli.Flag{fnNameFlag, fnEnvNameFlag, specSaveFlag, fnCodeFlag, fnSrcArchiveFlag, fnDeployArchiveFlag, fnEntryPointFlag, fnBuildCmdFlag, fnPkgNameFlag, htUrlFlag, htMethodFlag, minCpu, maxCpu, minMem, maxMem, minScale, maxScale, fnExecutorTypeFlag, targetcpu, fnCfgMapFlag, fnSecretFlag, fnSecretnsFlag, fnCfgMapnsFlag}, Action: fnCreate},
 		{Name: "get", Usage: "Get function source code", Flags: []cli.Flag{fnNameFlag}, Action: fnGet},
 		{Name: "getmeta", Usage: "Get function metadata", Flags: []cli.Flag{fnNameFlag}, Action: fnGetMeta},
-		{Name: "update", Usage: "Update function source code", Flags: []cli.Flag{fnNameFlag, fnEnvNameFlag, fnCodeFlag, fnPackageFlag, fnSrcArchiveFlag, fnDeployArchiveFlag, fnEntryPointFlag, fnPkgNameFlag, fnBuildCmdFlag, fnForceFlag, minCpu, maxCpu, minMem, maxMem, minScale, maxScale, fnExecutorTypeFlag, targetcpu}, Action: fnUpdate},
+		{Name: "update", Usage: "Update function source code", Flags: []cli.Flag{fnNameFlag, fnEnvNameFlag, fnCodeFlag, fnSrcArchiveFlag, fnDeployArchiveFlag, fnEntryPointFlag, fnPkgNameFlag, fnBuildCmdFlag, fnForceFlag, minCpu, maxCpu, minMem, maxMem, minScale, maxScale, fnExecutorTypeFlag, targetcpu}, Action: fnUpdate},
 		{Name: "delete", Usage: "Delete function", Flags: []cli.Flag{fnNameFlag}, Action: fnDelete},
 		{Name: "list", Usage: "List all functions", Flags: []cli.Flag{}, Action: fnList},
 		{Name: "logs", Usage: "Display function logs", Flags: []cli.Flag{fnNameFlag, fnPodFlag, fnFollowFlag, fnDetailFlag, fnLogDBTypeFlag, fnLogCountFlag}, Action: fnLogs},
-		{Name: "pods", Usage: "Display function pods", Flags: []cli.Flag{fnNameFlag, fnLogDBTypeFlag}, Action: fnPods},
-		{Name: "test", Usage: "Test a function", Flags: []cli.Flag{fnNameFlag, fnEnvNameFlag, fnCodeFlag, fnPackageFlag, fnSrcArchiveFlag, htMethodFlag, fnBodyFlag, fnHeaderFlag}, Action: fnTest},
+		{Name: "test", Usage: "Test a function", Flags: []cli.Flag{fnNameFlag, fnEnvNameFlag, fnCodeFlag, fnSrcArchiveFlag, htMethodFlag, fnBodyFlag, fnHeaderFlag}, Action: fnTest},
 	}

 	// httptriggers
 	htNameFlag := cli.StringFlag{Name: "name", Usage: "HTTP Trigger name"}
 	htFnNameFlag := cli.StringFlag{Name: "function", Usage: "Function name"}
 	htSubcommands := []cli.Command{
-		{Name: "create", Aliases: []string{"add"}, Usage: "Create HTTP trigger", Flags: []cli.Flag{htMethodFlag, htUrlFlag, htFnNameFlag}, Action: htCreate},
+		{Name: "create", Aliases: []string{"add"}, Usage: "Create HTTP trigger", Flags: []cli.Flag{htMethodFlag, htUrlFlag, htFnNameFlag, specSaveFlag}, Action: htCreate},
 		{Name: "get", Usage: "Get HTTP trigger", Flags: []cli.Flag{htMethodFlag, htUrlFlag}, Action: htGet},
 		{Name: "update", Usage: "Update HTTP trigger", Flags: []cli.Flag{htNameFlag, htFnNameFlag}, Action: htUpdate},
 		{Name: "delete", Usage: "Delete HTTP trigger", Flags: []cli.Flag{htNameFlag}, Action: htDelete},
@@ -160,7 +170,7 @@ func main() {
 	ttCronFlag := cli.StringFlag{Name: "cron", Usage: "Time Trigger cron spec ('0 30 * * *', '@every 5m', '@hourly')"}
 	ttFnNameFlag := cli.StringFlag{Name: "function", Usage: "Function name"}
 	ttSubcommands := []cli.Command{
-		{Name: "create", Aliases: []string{"add"}, Usage: "Create Time trigger", Flags: []cli.Flag{ttNameFlag, ttFnNameFlag, ttCronFlag}, Action: ttCreate},
+		{Name: "create", Aliases: []string{"add"}, Usage: "Create Time trigger", Flags: []cli.Flag{ttNameFlag, ttFnNameFlag, ttCronFlag, specSaveFlag}, Action: ttCreate},
 		{Name: "get", Usage: "Get Time trigger", Flags: []cli.Flag{}, Action: ttGet},
 		{Name: "update", Usage: "Update Time trigger", Flags: []cli.Flag{ttNameFlag, ttCronFlag, ttFnNameFlag}, Action: ttUpdate},
 		{Name: "delete", Usage: "Delete Time trigger", Flags: []cli.Flag{ttNameFlag}, Action: ttDelete},
@@ -170,12 +180,12 @@ func main() {
 	// Message queue trigger
 	mqtNameFlag := cli.StringFlag{Name: "name", Usage: "Message queue Trigger name"}
 	mqtFnNameFlag := cli.StringFlag{Name: "function", Usage: "Function name"}
-	mqtMQTypeFlag := cli.StringFlag{Name: "mqtype", Usage: "Message queue type, e.g. nats-streaming, azure-storage-queue  (optional; uses \"nats-streaming\" if unspecified)"}
+	mqtMQTypeFlag := cli.StringFlag{Name: "mqtype", Value: "nats-streaming", Usage: "Message queue type, e.g. nats-streaming, azure-storage-queue (optional)"}
 	mqtTopicFlag := cli.StringFlag{Name: "topic", Usage: "Message queue Topic the trigger listens on"}
 	mqtRespTopicFlag := cli.StringFlag{Name: "resptopic", Usage: "Topic that the function response is sent on (optional; response discarded if unspecified)"}
-	mqtMsgContentType := cli.StringFlag{Name: "contenttype, c", Usage: "Content type of messages that publish to the topic (optional; uses \"application/json\" if unspecified)"}
+	mqtMsgContentType := cli.StringFlag{Name: "contenttype, c", Value: "application/json", Usage: "Content type of messages that publish to the topic (optional)"}
 	mqtSubcommands := []cli.Command{
-		{Name: "create", Aliases: []string{"add"}, Usage: "Create Message queue trigger", Flags: []cli.Flag{mqtNameFlag, mqtFnNameFlag, mqtMQTypeFlag, mqtTopicFlag, mqtRespTopicFlag, mqtMsgContentType}, Action: mqtCreate},
+		{Name: "create", Aliases: []string{"add"}, Usage: "Create Message queue trigger", Flags: []cli.Flag{mqtNameFlag, mqtFnNameFlag, mqtMQTypeFlag, mqtTopicFlag, mqtRespTopicFlag, mqtMsgContentType, specSaveFlag}, Action: mqtCreate},
 		{Name: "get", Usage: "Get message queue trigger", Flags: []cli.Flag{}, Action: mqtGet},
 		{Name: "update", Usage: "Update message queue trigger", Flags: []cli.Flag{mqtNameFlag, mqtTopicFlag, mqtRespTopicFlag, mqtFnNameFlag, mqtMsgContentType}, Action: mqtUpdate},
 		{Name: "delete", Usage: "Delete message queue trigger", Flags: []cli.Flag{mqtNameFlag}, Action: mqtDelete},
@@ -184,17 +194,17 @@ func main() {

 	// environments
 	envNameFlag := cli.StringFlag{Name: "name", Usage: "Environment name"}
-	envPoolsizeFlag := cli.IntFlag{Name: "poolsize", Usage: "Size of the pool, if not specified defaults to 3"}
+	envPoolsizeFlag := cli.IntFlag{Name: "poolsize", Value: 3, Usage: "Size of the pool"}
 	envImageFlag := cli.StringFlag{Name: "image", Usage: "Environment image URL"}
 	envBuilderImageFlag := cli.StringFlag{Name: "builder", Usage: "Environment builder image URL (optional)"}
 	envBuildCmdFlag := cli.StringFlag{Name: "buildcmd", Usage: "Build command for environment builder to build source package (optional)"}
 	envExternalNetworkFlag := cli.BoolFlag{Name: "externalnetwork", Usage: "Allow environment access external network when istio feature enabled (optional, defaults to false)"}
-
-	envVersionFlag := cli.IntFlag{Name: "version", Usage: "Environment API version: defaults to 1 (means v1 interface)"}
+	envTerminationGracePeriodFlag := cli.Int64Flag{Name: "graceperiod, period", Value: 360, Usage: "The grace time (in seconds) for pod to perform connection draining before termination (optional)"}
+	envVersionFlag := cli.IntFlag{Name: "version", Value: 1, Usage: "Environment API version (1 means v1 interface)"}
 	envSubcommands := []cli.Command{
-		{Name: "create", Aliases: []string{"add"}, Usage: "Add an environment", Flags: []cli.Flag{envNameFlag, envPoolsizeFlag, envImageFlag, envBuilderImageFlag, envBuildCmdFlag, minCpu, maxCpu, minMem, maxMem, envVersionFlag, envExternalNetworkFlag}, Action: envCreate},
+		{Name: "create", Aliases: []string{"add"}, Usage: "Add an environment", Flags: []cli.Flag{envNameFlag, envPoolsizeFlag, envImageFlag, envBuilderImageFlag, envBuildCmdFlag, minCpu, maxCpu, minMem, maxMem, envVersionFlag, envExternalNetworkFlag, envTerminationGracePeriodFlag, specSaveFlag}, Action: envCreate},
 		{Name: "get", Usage: "Get environment details", Flags: []cli.Flag{envNameFlag}, Action: envGet},
-		{Name: "update", Usage: "Update environment", Flags: []cli.Flag{envNameFlag, envPoolsizeFlag, envImageFlag, envBuilderImageFlag, envBuildCmdFlag, minCpu, maxCpu, minMem, maxMem, envExternalNetworkFlag}, Action: envUpdate},
+		{Name: "update", Usage: "Update environment", Flags: []cli.Flag{envNameFlag, envPoolsizeFlag, envImageFlag, envBuilderImageFlag, envBuildCmdFlag, minCpu, maxCpu, minMem, maxMem, envExternalNetworkFlag, envTerminationGracePeriodFlag}, Action: envUpdate},
 		{Name: "delete", Usage: "Delete environment", Flags: []cli.Flag{envNameFlag}, Action: envDelete},
 		{Name: "list", Usage: "List all environments", Flags: []cli.Flag{}, Action: envList},
 	}
@@ -206,7 +216,7 @@ func main() {
 	wObjTypeFlag := cli.StringFlag{Name: "type", Usage: "Type of resource to watch (Pod, Service, etc.)"}
 	wLabelsFlag := cli.StringFlag{Name: "labels", Usage: "Label selector of the form a=b,c=d"}
 	wSubCommands := []cli.Command{
-		{Name: "create", Aliases: []string{"add"}, Usage: "Create a watch", Flags: []cli.Flag{wFnNameFlag, wNamespaceFlag, wObjTypeFlag, wLabelsFlag}, Action: wCreate},
+		{Name: "create", Aliases: []string{"add"}, Usage: "Create a watch", Flags: []cli.Flag{wFnNameFlag, wNamespaceFlag, wObjTypeFlag, wLabelsFlag, specSaveFlag}, Action: wCreate},
 		{Name: "get", Usage: "Get details about a watch", Flags: []cli.Flag{wNameFlag}, Action: wGet},
 		// TODO add update flag when supported
 		{Name: "delete", Usage: "Delete watch", Flags: []cli.Flag{wNameFlag}, Action: wDelete},
@@ -273,5 +283,6 @@ func main() {
 		{Name: "tpr2crd", Aliases: []string{}, Usage: "Migrate tool for TPR to CRD", Subcommands: migrateSubCommands},
 	}

+	app.Before = cliHook
 	app.Run(os.Args)
 }
@@ -27,7 +27,6 @@ import (
 	"github.com/fission/fission"
 	"github.com/fission/fission/controller/client"
 	"github.com/fission/fission/crd"
-	"github.com/fission/fission/mqtrigger/messageQueue"
 )

 type (
@@ -55,7 +54,7 @@ func migrateDumpTPRResource(client *client.Client, filename string) {
 	checkErr(err, "dump watches")
 	timeTriggers, err := client.TimeTriggerList()
 	checkErr(err, "dump time triggers")
-	mqTriggers, err := client.MessageQueueTriggerList(messageQueue.NATS)
+	mqTriggers, err := client.MessageQueueTriggerList(fission.MessageQueueTypeNats)
 	checkErr(err, "dump message queue triggers")

 	tprResource := TPRResource{
@@ -27,7 +27,6 @@ import (

 	"github.com/fission/fission"
 	"github.com/fission/fission/crd"
-	"github.com/fission/fission/mqtrigger/messageQueue"
 )

 func mqtCreate(c *cli.Context) error {
@@ -42,14 +41,14 @@ func mqtCreate(c *cli.Context) error {
 		fatal("Need a function name to create a trigger, use --function")
 	}

-	mqType := c.String("mqtype")
-	switch mqType {
+	var mqType fission.MessageQueueType
+	switch c.String("mqtype") {
 	case "":
-		mqType = messageQueue.NATS
-	case messageQueue.NATS:
-		mqType = messageQueue.NATS
-	case messageQueue.ASQ:
-		mqType = messageQueue.ASQ
+		mqType = fission.MessageQueueTypeNats
+	case fission.MessageQueueTypeNats:
+		mqType = fission.MessageQueueTypeNats
+	case fission.MessageQueueTypeASQ:
+		mqType = fission.MessageQueueTypeASQ
 	default:
 		fatal("Unknown message queue type, currently only \"nats-streaming, azure-storage-queue \" is supported")
 	}
@@ -74,7 +73,7 @@ func mqtCreate(c *cli.Context) error {

 	checkMQTopicAvailability(mqType, topic, respTopic)

-	mqt := crd.MessageQueueTrigger{
+	mqt := &crd.MessageQueueTrigger{
 		Metadata: metav1.ObjectMeta{
 			Name:      mqtName,
 			Namespace: metav1.NamespaceDefault,
@@ -91,7 +90,15 @@ func mqtCreate(c *cli.Context) error {
 		},
 	}

-	_, err := client.MessageQueueTriggerCreate(&mqt)
+	// if we're writing a spec, don't call the API
+	if c.Bool("spec") {
+		specFile := fmt.Sprintf("mqtrigger-%v.yaml", mqtName)
+		err := specSave(*mqt, specFile)
+		checkErr(err, "create message queue trigger spec")
+		return nil
+	}
+
+	_, err := client.MessageQueueTriggerCreate(mqt)
 	checkErr(err, "create message queue trigger")

 	fmt.Printf("trigger '%s' created\n", mqtName)
@@ -186,9 +193,9 @@ func mqtList(c *cli.Context) error {
 	return nil
 }

-func checkMQTopicAvailability(mqType string, topics ...string) {
+func checkMQTopicAvailability(mqType fission.MessageQueueType, topics ...string) {
 	for _, t := range topics {
-		if len(t) > 0 && !messageQueue.IsTopicValid(mqType, t) {
+		if len(t) > 0 && !fission.IsTopicValid(mqType, t) {
 			fatal(fmt.Sprintf("Invalid topic for %s: %s", mqType, t))
 		}
 	}
@@ -98,6 +98,10 @@ func pkgUpdate(c *cli.Context) error {
 	deployArchiveName := c.String("deploy")
 	buildcmd := c.String("buildcmd")

+	if len(srcArchiveName) > 0 && len(deployArchiveName) > 0 {
+		fatal("Need either of --src or --deploy and not both arguments.")
+	}
+
 	if len(srcArchiveName) == 0 && len(deployArchiveName) == 0 &&
 		len(envName) == 0 && len(buildcmd) == 0 {
 		fatal("Need --env or --src or --deploy or --buildcmd argument.")
@@ -158,9 +162,8 @@ func updatePackage(client *client.Client, pkg *crd.Package, envName,
 		pkg.Spec.Deployment = *deployArchiveMetadata
 	}

-	// Set package as pending status only when there is no
-	// deploy archive.
-	if needToBuild && len(pkg.Spec.Deployment.Type) == 0 {
+	// Set package as pending status when needToBuild is true
+	if needToBuild {
 		// change into pending state to trigger package build
 		pkg.Status = fission.PackageStatus{
 			BuildStatus: fission.BuildStatusPending,
@@ -52,6 +52,8 @@ func runPortForward(kubeConfig string, labelSelector string, localPort string, f
 		fatal(fmt.Sprintf("Failed to connect to Kubernetes: %s", err))
 	}

+	verbose(2, "Connected to Kubernetes API")
+
 	// if fission namespace is unset, try to find a fission pod in any namespace
 	if len(fissionNamespace) == 0 {
 		fissionNamespace = meta_v1.NamespaceAll
@@ -93,6 +95,7 @@ func runPortForward(kubeConfig string, labelSelector string, localPort string, f
 	for _, servicePort := range service.Spec.Ports {
 		targetPort = servicePort.TargetPort.String()
 	}
+	verbose(2, "Connecting to port %v on pod %v/%v", targetPort, podNameSpace, podNameSpace)

 	stopChannel := make(chan struct{}, 1)
 	readyChannel := make(chan struct{})
@@ -113,12 +116,17 @@ func runPortForward(kubeConfig string, labelSelector string, localPort string, f
 		fatal(msg)
 	}

-	fw, err := portforward.New(dialer, ports, stopChannel, readyChannel, nil, os.Stderr)
+	outStream := os.Stdout
+	if verbosity < 2 {
+		outStream = nil
+	}
+	fw, err := portforward.New(dialer, ports, stopChannel, readyChannel, outStream, os.Stderr)
 	if err != nil {
 		msg := fmt.Sprintf("portforward.new errored out :%v", err.Error())
 		fatal(msg)
 	}

+	verbose(2, "Starting port forwarder")
 	return fw.ForwardPorts()
 }

@@ -128,11 +136,15 @@ func runPortForward(kubeConfig string, labelSelector string, localPort string, f
 // its targetPort. Once the port forward is started, wait for it to
 // start accepting connections before returning.
 func setupPortForward(kubeConfig, namespace, labelSelector string) string {
+	verbose(2, "Setting up port forward to %s in namespace %s using the kubeconfig at %s",
+		labelSelector, namespace, kubeConfig)
+
 	localPort, err := findFreePort()
 	if err != nil {
 		fatal(fmt.Sprintf("Error finding unused port :%v", err.Error()))
 	}

+	verbose(2, "Waiting for local port %v", localPort)
 	for {
 		conn, _ := net.DialTimeout("tcp",
 			net.JoinHostPort("", localPort), time.Millisecond)
@@ -144,6 +156,7 @@ func setupPortForward(kubeConfig, namespace, labelSelector string) string {
 		time.Sleep(time.Millisecond * 50)
 	}

+	verbose(2, "Starting port forward from local port %v", localPort)
 	go func() {
 		err := runPortForward(kubeConfig, labelSelector, localPort, namespace)
 		if err != nil {
@@ -151,6 +164,7 @@ func setupPortForward(kubeConfig, namespace, labelSelector string) string {
 		}
 	}()

+	verbose(2, "Waiting for port forward %v to start...", localPort)
 	for {
 		conn, _ := net.DialTimeout("tcp",
 			net.JoinHostPort("", localPort), time.Millisecond)
@@ -161,5 +175,7 @@ func setupPortForward(kubeConfig, namespace, labelSelector string) string {
 		time.Sleep(time.Millisecond * 50)
 	}

+	verbose(2, "Port forward from local port %v started", localPort)
+
 	return localPort
 }
@@ -29,6 +29,7 @@ import (

 	"github.com/fsnotify/fsnotify"
 	"github.com/ghodss/yaml"
+	multierror "github.com/hashicorp/go-multierror"
 	"github.com/mholt/archiver"
 	"github.com/pkg/errors"
 	"github.com/satori/go.uuid"
@@ -100,7 +101,7 @@ type (
 		messageQueueTriggers    []crd.MessageQueueTrigger
 		archiveUploadSpecs      []ArchiveUploadSpec

-		sourceMap SourceMap
+		sourceMap sourceMap
 	}

 	resourceApplyStatus struct {
@@ -109,13 +110,18 @@ type (
 		deleted []*metav1.ObjectMeta
 	}

-	SourceMap struct {
-		// xxx
+	location struct {
+		path string
+		line int
+	}
+	sourceMap struct {
+		// kind -> namespace -> name -> location
+		locations map[string](map[string](map[string]location))
 	}
 )

 func getSpecDir(c *cli.Context) string {
-	specDir := c.String("specs")
+	specDir := c.String("specdir")
 	if len(specDir) == 0 {
 		specDir = "specs"
 	}
@@ -190,28 +196,205 @@ func specInit(c *cli.Context) error {
 	return nil
 }

+// validateFunctionReference checks a function reference
+func (fr *FissionResources) validateFunctionReference(functions map[string]bool, kind string, meta *metav1.ObjectMeta, funcRef fission.FunctionReference) error {
+	if funcRef.Type == fission.FunctionReferenceTypeFunctionName {
+		// triggers only reference functions in their own namespace
+		namespace := meta.Namespace
+		name := funcRef.Name
+		m := &metav1.ObjectMeta{
+			Namespace: namespace,
+			Name:      name,
+		}
+		if _, ok := functions[mapKey(m)]; !ok {
+			return fmt.Errorf("%v: %v '%v' references unknown function '%v'",
+				fr.sourceMap.locations[kind][meta.Namespace][meta.Name],
+				kind,
+				meta.Name,
+				name)
+		} else {
+			functions[mapKey(m)] = true
+		}
+	}
+	return nil
+}
+
 // specValidate parses a set of specs and checks for references to
 // resources that don't exist.
 func specValidate(c *cli.Context) error {
-	//specDir := getSpecDir(c)
+	// this will error on parse errors and on duplicates
+	specDir := getSpecDir(c)
+	fr, err := readSpecs(specDir)
+	checkErr(err, "read specs")

-	// parse all specs
-	// verify references:
-	//   functions from triggers
-	//   packages from functions
+	// this does the rest of the checks, like dangling refs
+	err = fr.validate()
+	if err != nil {
+		fmt.Printf("Error validating specs: %v", err)
+	}

-	// find unreferenced uploads
+	return nil
+}
+
+func (fr *FissionResources) validate() error {
+	var result *multierror.Error
+
+	// check references: both dangling refs + garbage
+	//   packages -> archives
+	//   functions -> packages
+	//   functions -> environments [TODO]
+	//   triggers -> functions
+
+	// index archives
+	archives := make(map[string]bool)
+	for _, a := range fr.archiveUploadSpecs {
+		archives[a.Name] = false
+	}
+
+	// index packages, check outgoing refs, mark archives that are referenced
+	packages := make(map[string]bool)
+	for _, p := range fr.packages {
+		packages[mapKey(&p.Metadata)] = false
+
+		// check archive refs from package
+		aname := strings.TrimPrefix(p.Spec.Source.URL, ARCHIVE_URL_PREFIX)
+		if len(aname) > 0 {
+			if _, ok := archives[aname]; !ok {
+				result = multierror.Append(result, fmt.Errorf(
+					"%v: package '%v' references unknown source archive %v%v",
+					fr.sourceMap.locations["Package"][p.Metadata.Namespace][p.Metadata.Name],
+					p.Metadata.Name,
+					ARCHIVE_URL_PREFIX,
+					aname))
+			} else {
+				archives[aname] = true
+			}
+		}
+
+		aname = strings.TrimPrefix(p.Spec.Deployment.URL, ARCHIVE_URL_PREFIX)
+		if len(aname) > 0 {
+			if _, ok := archives[aname]; !ok {
+				result = multierror.Append(result, fmt.Errorf(
+					"%v: package '%v' references unknown deployment archive %v%v",
+					fr.sourceMap.locations["Package"][p.Metadata.Namespace][p.Metadata.Name],
+					p.Metadata.Name,
+					ARCHIVE_URL_PREFIX,
+					aname))
+			} else {
+				archives[aname] = true
+			}
+		}
+	}
+
+	// error on unreferenced archives
+	for name, referenced := range archives {
+		if !referenced {
+			result = multierror.Append(result, fmt.Errorf(
+				"%v: archive '%v' is not used in any package",
+				fr.sourceMap.locations["ArchiveUploadSpec"][""][name],
+				name))
+		}
+	}
+
+	// index functions, check function package refs, mark referenced packages
+	functions := make(map[string]bool)
+	for _, f := range fr.functions {
+		functions[mapKey(&f.Metadata)] = false
+
+		// check package ref from function
+		pkgMeta := &metav1.ObjectMeta{
+			Name:      f.Spec.Package.PackageRef.Name,
+			Namespace: f.Spec.Package.PackageRef.Namespace,
+		}
+		if _, ok := packages[mapKey(pkgMeta)]; !ok {
+			result = multierror.Append(result, fmt.Errorf(
+				"%v: function '%v' references unknown package %v/%v",
+				fr.sourceMap.locations["Function"][f.Metadata.Namespace][f.Metadata.Name],
+				f.Metadata.Name,
+				pkgMeta.Namespace,
+				pkgMeta.Name))
+		} else {
+			packages[mapKey(pkgMeta)] = true
+		}
+	}
+
+	// error on unreferenced packages
+	for key, referenced := range packages {
+		ks := strings.Split(key, ":")
+		namespace, name := ks[0], ks[1]
+		if !referenced {
+			result = multierror.Append(result, fmt.Errorf(
+				"%v: package '%v' is not used in any function",
+				fr.sourceMap.locations["Package"][namespace][name],
+				name))
+		}
+	}
+
+	// check function refs from triggers
+	for _, t := range fr.httpTriggers {
+		err := fr.validateFunctionReference(functions, t.Kind, &t.Metadata, t.Spec.FunctionReference)
+		if err != nil {
+			result = multierror.Append(result, err)
+		}
+	}
+	for _, t := range fr.kubernetesWatchTriggers {
+		err := fr.validateFunctionReference(functions, t.Kind, &t.Metadata, t.Spec.FunctionReference)
+		if err != nil {
+			result = multierror.Append(result, err)
+		}
+	}
+	for _, t := range fr.timeTriggers {
+		err := fr.validateFunctionReference(functions, t.Kind, &t.Metadata, t.Spec.FunctionReference)
+		if err != nil {
+			result = multierror.Append(result, err)
+		}
+	}
+	for _, t := range fr.messageQueueTriggers {
+		err := fr.validateFunctionReference(functions, t.Kind, &t.Metadata, t.Spec.FunctionReference)
+		if err != nil {
+			result = multierror.Append(result, err)
+		}
+	}
+
+	// we do not error on unreferenced functions (you can call a function through workflows,
+	// `fission function test`, etc.)
+
+	// (ErrorOrNil returns nil if there were no errors appended.)
+	return result.ErrorOrNil()
+}
+
+func (loc location) String() string {
+	return fmt.Sprintf("%v:%v", loc.path, loc.line)
+}
+
+// Keep track of source location of resources, and track duplicates
+func (fr *FissionResources) trackSourceMap(kind string, newobj *metav1.ObjectMeta, loc *location) error {
+	if _, exists := fr.sourceMap.locations[kind]; !exists {
+		fr.sourceMap.locations[kind] = make(map[string](map[string]location))
+	}
+	if _, exists := fr.sourceMap.locations[kind][newobj.Namespace]; !exists {
+		fr.sourceMap.locations[kind][newobj.Namespace] = make(map[string]location)
+	}
+
+	// check for duplicate resources
+	oldloc, exists := fr.sourceMap.locations[kind][newobj.Namespace][newobj.Name]
+	if exists {
+		return fmt.Errorf("%v: Duplicate %v '%v', first defined in %v", loc, kind, newobj.Name, oldloc)
+	}
+
+	// track new resource
+	fr.sourceMap.locations[kind][newobj.Namespace][newobj.Name] = *loc

 	return nil
 }

 // parseYaml takes one yaml document, figures out its type, parses it, and puts it in
 // the right list in the given fission resources set.
-func parseYaml(path string, b []byte, fr *FissionResources) error {
+func (fr *FissionResources) parseYaml(b []byte, loc *location) error {
+	var m *metav1.ObjectMeta

 	// Figure out the object type by unmarshaling into the TypeMeta struct; then
-	// unmarshal again into the "real" struct once we know the type.  There's almost
-	// certainly a better way to do this...
+	// unmarshal again into the "real" struct once we know the type.
 	var tm TypeMeta
 	err := yaml.Unmarshal(b, &tm)
 	switch tm.Kind {
@@ -219,57 +402,63 @@ func parseYaml(path string, b []byte, fr *FissionResources) error {
 		var v crd.Package
 		err = yaml.Unmarshal(b, &v)
 		if err != nil {
-			warn(fmt.Sprintf("Failed to parse %v in %v: %v", tm.Kind, path, err))
-			return err
+			return errors.Wrap(err, fmt.Sprintf("Failed to parse %v in %v", tm.Kind, loc))
 		}
+		m = &v.Metadata
 		fr.packages = append(fr.packages, v)
 	case "Function":
 		var v crd.Function
 		err = yaml.Unmarshal(b, &v)
 		if err != nil {
-			warn(fmt.Sprintf("Failed to parse %v in %v: %v", tm.Kind, path, err))
-			return err
+			return errors.Wrap(err, fmt.Sprintf("Failed to parse %v in %v", tm.Kind, loc))
 		}
+		m = &v.Metadata
 		fr.functions = append(fr.functions, v)
 	case "Environment":
 		var v crd.Environment
 		err = yaml.Unmarshal(b, &v)
 		if err != nil {
-			warn(fmt.Sprintf("Failed to parse %v in %v: %v", tm.Kind, path, err))
-			return err
+			return errors.Wrap(err, fmt.Sprintf("Failed to parse %v in %v", tm.Kind, loc))
 		}
+		m = &v.Metadata
 		fr.environments = append(fr.environments, v)
 	case "HTTPTrigger":
 		var v crd.HTTPTrigger
 		err = yaml.Unmarshal(b, &v)
 		if err != nil {
-			warn(fmt.Sprintf("Failed to parse %v in %v: %v", tm.Kind, path, err))
-			return err
+			return errors.Wrap(err, fmt.Sprintf("Failed to parse %v in %v", tm.Kind, loc))
 		}
+
+		// TODO move to validator
+		if !strings.HasPrefix(v.Spec.RelativeURL, "/") {
+			v.Spec.RelativeURL = fmt.Sprintf("/%s", v.Spec.RelativeURL)
+		}
+
+		m = &v.Metadata
 		fr.httpTriggers = append(fr.httpTriggers, v)
 	case "KubernetesWatchTrigger":
 		var v crd.KubernetesWatchTrigger
 		err = yaml.Unmarshal(b, &v)
 		if err != nil {
-			warn(fmt.Sprintf("Failed to parse %v in %v: %v", tm.Kind, path, err))
-			return err
+			return errors.Wrap(err, fmt.Sprintf("Failed to parse %v in %v", tm.Kind, loc))
 		}
+		m = &v.Metadata
 		fr.kubernetesWatchTriggers = append(fr.kubernetesWatchTriggers, v)
 	case "TimeTrigger":
 		var v crd.TimeTrigger
 		err = yaml.Unmarshal(b, &v)
 		if err != nil {
-			warn(fmt.Sprintf("Failed to parse %v in %v: %v", tm.Kind, path, err))
-			return err
+			return errors.Wrap(err, fmt.Sprintf("Failed to parse %v in %v", tm.Kind, loc))
 		}
+		m = &v.Metadata
 		fr.timeTriggers = append(fr.timeTriggers, v)
 	case "MessageQueueTrigger":
 		var v crd.MessageQueueTrigger
 		err = yaml.Unmarshal(b, &v)
 		if err != nil {
-			warn(fmt.Sprintf("Failed to parse %v in %v: %v", tm.Kind, path, err))
-			return err
+			return errors.Wrap(err, fmt.Sprintf("Failed to parse %v in %v", tm.Kind, loc))
 		}
+		m = &v.Metadata
 		fr.messageQueueTriggers = append(fr.messageQueueTriggers, v)

 	// The following are not CRDs
@@ -278,22 +467,32 @@ func parseYaml(path string, b []byte, fr *FissionResources) error {
 		var v DeploymentConfig
 		err = yaml.Unmarshal(b, &v)
 		if err != nil {
-			warn(fmt.Sprintf("Failed to parse %v in %v: %v", tm.Kind, path, err))
-			return err
+			return errors.Wrap(err, fmt.Sprintf("Failed to parse %v in %v", tm.Kind, loc))
 		}
 		fr.deploymentConfig = v
 	case "ArchiveUploadSpec":
 		var v ArchiveUploadSpec
 		err = yaml.Unmarshal(b, &v)
 		if err != nil {
-			warn(fmt.Sprintf("Failed to parse %v in %v: %v", tm.Kind, path, err))
-			return err
+			return errors.Wrap(err, fmt.Sprintf("Failed to parse %v in %v", tm.Kind, loc))
+		}
+		m = &metav1.ObjectMeta{
+			Name:      v.Name,
+			Namespace: "",
 		}
 		fr.archiveUploadSpecs = append(fr.archiveUploadSpecs, v)
 	default:
 		// no need to error out just because there's some extra files around;
 		// also good for compatibility.
-		warn(fmt.Sprintf("Ignoring unknown type %v in %v", tm.Kind, path))
+		warn(fmt.Sprintf("Ignoring unknown type %v in %v", tm.Kind, loc))
+	}
+
+	// add to source map, check for duplicates
+	if m != nil {
+		err = fr.trackSourceMap(tm.Kind, m, loc)
+		if err != nil {
+			return err
+		}
 	}

 	return nil
@@ -310,8 +509,14 @@ func readSpecs(specDir string) (*FissionResources, error) {
 		kubernetesWatchTriggers: make([]crd.KubernetesWatchTrigger, 0),
 		timeTriggers:            make([]crd.TimeTrigger, 0),
 		messageQueueTriggers:    make([]crd.MessageQueueTrigger, 0),
+
+		sourceMap: sourceMap{
+			locations: make(map[string](map[string](map[string]location))),
+		},
 	}

+	var result *multierror.Error
+
 	// Users can organize the specdir into subdirs if they want to.
 	err := filepath.Walk(specDir, func(path string, info os.FileInfo, err error) error {
 		// For now just read YAML files. We'll add jsonnet at some point. Skip
@@ -322,26 +527,38 @@ func readSpecs(specDir string) (*FissionResources, error) {
 		// read
 		b, err := ioutil.ReadFile(path)
 		if err != nil {
-			return err
+			result = multierror.Append(result, err)
+			return nil
 		}
 		// handle the case where there are multiple YAML docs per file. go-yaml
 		// doesn't support this directly, yet.
 		docs := bytes.Split(b, []byte("\n---"))
+		lines := 1
 		for _, doc := range docs {
 			d := []byte(strings.TrimSpace(string(doc)))
 			if len(d) != 0 {
 				// parse this document and add whatever is in it to fr
-				err = parseYaml(path, d, &fr)
+				err = fr.parseYaml(d, &location{
+					path: path,
+					line: lines,
+				})
 				if err != nil {
-					return err
+					// collect all errors so user can fix them all
+					result = multierror.Append(result, err)
 				}
 			}
+			// the separator occupies one line, hence the +1
+			lines += strings.Count(string(doc), "\n") + 1
 		}
 		return nil
 	})
+
 	if err != nil {
 		return nil, err
 	}
+	if err := result.ErrorOrNil(); err != nil {
+		return nil, err
+	}

 	return &fr, nil
 }
@@ -419,6 +636,10 @@ func specApply(c *cli.Context) error {
 		fr, err := readSpecs(specDir)
 		checkErr(err, "read specs")

+		// validate
+		err = fr.validate()
+		checkErr(err, "validate specs")
+
 		// make changes to the cluster based on the specs
 		pkgMetas, as, err := applyResources(fclient, specDir, fr, deleteResources)
 		checkErr(err, "apply specs")
@@ -787,6 +1008,27 @@ func hasDeploymentConfig(m *metav1.ObjectMeta, fr *FissionResources) bool {
 	return false
 }

+func waitForPackageBuild(fclient *client.Client, pkg *crd.Package) (*crd.Package, error) {
+	start := time.Now()
+	for {
+		if pkg.Status.BuildStatus != fission.BuildStatusRunning {
+			return pkg, nil
+		}
+		if time.Since(start) > 5*time.Minute {
+			return nil, fmt.Errorf("Package %v has been building for a while.  Giving up on waiting for it.", pkg.Metadata.Name)
+		}
+
+		// TODO watch instead
+		time.Sleep(time.Second)
+
+		var err error
+		pkg, err = fclient.PackageGet(&pkg.Metadata)
+		if err != nil {
+			return nil, err
+		}
+	}
+}
+
 func applyPackages(fclient *client.Client, fr *FissionResources, delete bool) (map[string]metav1.ObjectMeta, *resourceApplyStatus, error) {
 	// get list
 	allObjs, err := fclient.PackageList()
@@ -843,9 +1085,20 @@ func applyPackages(fclient *client.Client, fr *FissionResources, delete bool) (m
 			} else {
 				// update
 				o.Metadata.ResourceVersion = existingObj.Metadata.ResourceVersion
-				newmeta, err := fclient.PackageUpdate(&o)
+
+				// We may be racing against the package builder to update the
+				// package (a previous version might have been getting built).  So,
+				// wait for the package to have a non-running build status.
+				pkg, err := waitForPackageBuild(fclient, &o)
+				if err != nil {
+					// log and ignore
+					fmt.Printf("Error waiting for package '%v' build, ignoring\n", o.Metadata.Name)
+				}
+
+				newmeta, err := fclient.PackageUpdate(pkg)
 				if err != nil {
 					return nil, nil, err
+					// TODO check for resourceVersion conflict errors and retry
 				}
 				ras.updated = append(ras.updated, newmeta)
 				// keep track of metadata in case we need to create a reference to it
@@ -1379,7 +1632,7 @@ func applyMessageQueueTriggers(fclient *client.Client, fr *FissionResources, del
 	return metadataMap, &ras, nil
 }

-// called from `fission function create --spec`
+// called from `fission * create --spec`
 func specSave(resource interface{}, specFile string) error {
 	specDir := "specs"

@@ -1403,6 +1656,26 @@ func specSave(resource interface{}, specFile string) error {
 		typedres.TypeMeta.APIVersion = SPEC_API_VERSION
 		typedres.TypeMeta.Kind = "Function"
 		data, err = yaml.Marshal(typedres)
+	case crd.Environment:
+		typedres.TypeMeta.APIVersion = SPEC_API_VERSION
+		typedres.TypeMeta.Kind = "Environment"
+		data, err = yaml.Marshal(typedres)
+	case crd.HTTPTrigger:
+		typedres.TypeMeta.APIVersion = SPEC_API_VERSION
+		typedres.TypeMeta.Kind = "HTTPTrigger"
+		data, err = yaml.Marshal(typedres)
+	case crd.KubernetesWatchTrigger:
+		typedres.TypeMeta.APIVersion = SPEC_API_VERSION
+		typedres.TypeMeta.Kind = "KubernetesWatchTrigger"
+		data, err = yaml.Marshal(typedres)
+	case crd.MessageQueueTrigger:
+		typedres.TypeMeta.APIVersion = SPEC_API_VERSION
+		typedres.TypeMeta.Kind = "MessageQueueTrigger"
+		data, err = yaml.Marshal(typedres)
+	case crd.TimeTrigger:
+		typedres.TypeMeta.APIVersion = SPEC_API_VERSION
+		typedres.TypeMeta.Kind = "TimeTrigger"
+		data, err = yaml.Marshal(typedres)
 	default:
 		return fmt.Errorf("can't save resource %#v", resource)
 	}
@@ -59,6 +59,14 @@ func ttCreate(c *cli.Context) error {
 		},
 	}

+	// if we're writing a spec, don't call the API
+	if c.Bool("spec") {
+		specFile := fmt.Sprintf("timetrigger-%v.yaml", name)
+		err := specSave(*tt, specFile)
+		checkErr(err, "create time trigger spec")
+		return nil
+	}
+
 	_, err := client.TimeTriggerCreate(tt)
 	checkErr(err, "create Time trigger")

@@ -360,7 +360,7 @@ func upgradeRestoreState(c *cli.Context) error {
 			Metadata: *crdMetadataFromV1Metadata(&t.Metadata, v1state.NameChanges),
 			Spec: fission.MessageQueueTriggerSpec{
 				FunctionReference: *functionRefFromV1Metadata(&t.Function, v1state.NameChanges),
-				MessageQueueType:  t.MessageQueueType,
+				MessageQueueType:  fission.MessageQueueTypeNats, // only NATS is supported at that time (v1 types)
 				Topic:             t.Topic,
 				ResponseTopic:     t.ResponseTopic,
 			},
@@ -77,6 +77,14 @@ func wCreate(c *cli.Context) error {
 		},
 	}

+	// if we're writing a spec, don't call the API
+	if c.Bool("spec") {
+		specFile := fmt.Sprintf("kubewatch-%v.yaml", watchName)
+		err := specSave(*w, specFile)
+		checkErr(err, "create kubernetes watch spec")
+		return nil
+	}
+
 	_, err := client.WatchCreate(w)
 	checkErr(err, "create watch")

@@ -1,5 +1,5 @@
-hash: 3b15246c5a7ca26271ef45e705c92351c786cfbf8ca1c093026b458e527ceefa
-updated: 2018-02-10T19:04:05.240303+01:00
+hash: d77d204547318863d07beea96b4e29a90f0d6c106a957dd0ba40eb0345a77e42
+updated: 2018-03-23T17:49:04.748453691-07:00
 imports:
 - name: cloud.google.com/go
  version: 3b1ae45394a234c385be014e9a488f2bb6eef821
@@ -18,7 +18,7 @@ imports:
  - autorest/azure
  - autorest/date
 - name: github.com/coreos/etcd
-  version: 9c6d93056575da4da94382f473b0ecfcd9c1443b
+  version: 6a265731e10a5137b991c1aa3a83ecefdd149d50
  subpackages:
  - client
 - name: github.com/davecgh/go-spew
@@ -55,7 +55,7 @@ imports:
 - name: github.com/emicklei/go-restful-swagger12
  version: dcef7f55730566d41eae5db10e7d6981829720f6
 - name: github.com/fsnotify/fsnotify
-  version: c2828203cd70a50dcccfb2761f8b1f8ceef9a8e9
+  version: 4da3e2cfbabc9f751898f250b49f2439785783a1
 - name: github.com/ghodss/yaml
  version: 73d445a93680fa1a78ae23a5839bad48f32ba1ee
 - name: github.com/go-openapi/analysis
@@ -97,6 +97,10 @@ imports:
  version: abb68c488872b06c5453865fa59f4818b4ea13a4
  subpackages:
  - local
+- name: github.com/hashicorp/errwrap
+  version: 7554cd9344cec97297fa6649b055a8c98c2a1e55
+- name: github.com/hashicorp/go-multierror
+  version: b7773ae218740a7be65057fc60b366a49b538a44
 - name: github.com/hashicorp/golang-lru
  version: a0d98a5f288019575c6d1f4bb1573fef2d1fcdc4
  subpackages:
@@ -133,7 +137,7 @@ imports:
  subpackages:
  - pb
 - name: github.com/nats-io/nats-streaming-server
-  version: 33414c6f2179201f7fda8743ba89d07184e3fa77
+  version: 7889f37a10062ff8eb9ac11855ed51cddeaf4469
  subpackages:
  - spb
  - util
@@ -148,7 +152,7 @@ imports:
  subpackages:
  - xxHash32
 - name: github.com/pkg/errors
-  version: 30136e27e2ac8d167177e8a583aa4c3fea5be833
+  version: f15c970de5b76fac0b59abb32d62c17cc7bed265
 - name: github.com/PuerkitoBio/purell
  version: 8a290539e2e8629dbc4e6bad948158f790ec31f4
 - name: github.com/PuerkitoBio/urlesc
@@ -237,7 +241,7 @@ imports:
 - name: k8s.io/api
  version: 4b8fc5be9b77d91bbb6525d18591c43699a2b4e5
 - name: k8s.io/apiextensions-apiserver
-  version: fcd622fe88a4a6efcb5aea9e94ee87324ac1b036
+  version: 19d3c0f1ccfb3e4180400ff7c22fe3c879771807
  subpackages:
  - pkg/apis/apiextensions
  - pkg/apis/apiextensions/v1beta1
@@ -245,7 +249,7 @@ imports:
  - pkg/client/clientset/clientset/scheme
  - pkg/client/clientset/clientset/typed/apiextensions/v1beta1
 - name: k8s.io/apimachinery
-  version: 8ab5f3d8a330c2e9baaf84e39042db8d49034ae2
+  version: 208a6980b14bbb263f29482482eabdbcfff9f7bb
  subpackages:
  - pkg/api/equality
  - pkg/api/errors
@@ -278,22 +282,22 @@ imports:
  - pkg/util/diff
  - pkg/util/errors
  - pkg/util/framer
+  - pkg/util/httpstream
+  - pkg/util/httpstream/spdy
  - pkg/util/intstr
  - pkg/util/json
  - pkg/util/net
  - pkg/util/rand
+  - pkg/util/remotecommand
  - pkg/util/runtime
  - pkg/util/sets
  - pkg/util/validation
  - pkg/util/validation/field
  - pkg/util/wait
  - pkg/util/yaml
-  - pkg/util/httpstream
-  - pkg/util/httpstream/spdy
-  - pkg/util/remotecommand
-  - third_party/forked/golang/netutil
  - pkg/version
  - pkg/watch
+  - third_party/forked/golang/netutil
  - third_party/forked/golang/reflect
 - name: k8s.io/client-go
  version: d92e8497f71b7b4e0494e5bd204b48d34bd6f254
@@ -374,11 +378,12 @@ imports:
  - tools/clientcmd/api
  - tools/clientcmd/api/latest
  - tools/clientcmd/api/v1
+  - tools/metrics
  - tools/portforward
  - tools/remotecommand
-  - tools/metrics
  - transport
  - util/cert
+  - util/exec
  - util/flowcontrol
  - util/homedir
  - util/integer
@@ -389,4 +394,4 @@ testImports:
  subpackages:
  - difflib
 - name: github.com/stretchr/objx
-  version: 8a3f7159479fbc75b30357fbc48f380b7320f08e
+  version: 8a3f7159479fbc75b30357fbc48f380b7320f08e
@@ -60,3 +60,5 @@ import:
  version: ~1.1.0
 - package: github.com/imdario/mergo
  version: ~0.3.2
+- package: github.com/hashicorp/go-multierror
+- package: github.com/hashicorp/errwrap
@@ -19,6 +19,18 @@ Prerequisites
   
 4. Install changelog generator tool "github_changelog_generator": https://github.com/skywinder/github-changelog-generator

+5. Install realpath if you don't already have it : https://github.com/harto/realpath-osx
+
+6. Install gothub if you don't already have it : https://github.com/itchio/gothub 
+   You just have to go get github.com/itchio/gothub, that'll install gothub in $GOPATH/bin. Ensure this is already
+   in $PATH env variable.
+
+
+Check that the build is green!
+------------------------------
+
+Fix if not, etc.
+

 Update versions
 ---------------
@@ -33,6 +45,18 @@ Commit your changes, push them, open a PR, merge it.
 Update doc versions: Docs live at github.com/fission/docs.fission.io


+Update the docs versions
+------------------------
+
+Archive existing version into dist/
+
+Change config.toml baseURL.
+
+Change versions in links.
+
+git add, git commit.
+
+
 Make the release
 ----------------

@@ -47,48 +71,48 @@ Make the release

   ./hack/release.sh <VERSION>

-   This script checks to make sure you're releasing from the master
+   This script checks to make sure you're releasing from the vX.Y.Z
   branch, and that your repo is clean (no modified/staged files).

   This will take a while.

 3. Push docs update to github

-4. Copy over charts into charts repo, update helm chart index, push to github
+4. Copy over charts from build dir into charts repo [see below], update helm chart index, push to github

 5. Manually test an installation, just for a sanity check

-6. Switch the github release from pre-release to release.
+6. Git commit and push Changelog.md into master branch.

+7. Switch the github release from pre-release to release.
+
+8. Add links from release notes:
+   * Install guide in docs
+   * Changelog.md

 Update chart repo
 -----------------

-Copy the helm charts over to the fission-charts repo.
+Copy the helm charts over to the fission-charts repo

 Run `index.sh` in that repo.  (Don't edit the index.yaml in this repo
 manually.  index.sh generates it, using the helm repo CLI.)

 git add, git commit, and git push your changes.

-The chart repo is a github page, so once you push the master branch,
+The chart repo is a github page : https://github.com/fission/fission-charts, so once you push the master branch,
 you're all set.

-
-Update the docs website
-----------------------
-
-Rebuild the docs website using hugo.
-
-Create a new folder named by version under fission.io/docs.  Copy the
-generated docs public folder into this folder.
-
-Update the 'latest' symlink to point to the new version of the docs.
-
-git add, git commit, git push.
+Ask to be added to this repo if permissions are missing.


 Announce the release
 --------------------

 Announce the release on #contributors.
+
+
+Update Fission.io
+-----------------
+
+Tell #marketing channel to update link
@@ -18,8 +18,8 @@ else
    fi
 fi

-go test -v -i $(go list ./... | grep -v '/vendor/' | grep -v 'examples/go')
+go test -v -i $(go list ./... | grep -v '/vendor/' | grep -v 'examples/go' | grep -v 'benchmark')

 # The executor unit test only works with NodePort-type services for
 # now. So disable it for our travis ci tests.
-go test -v $(go list ./... | grep -v '/vendor/' | grep -v 'examples/go' | grep -v executor)
+go test -v $(go list ./... | grep -v '/vendor/' | grep -v 'examples/go' | grep -v executor | grep -v 'benchmark')
@@ -17,6 +17,8 @@ limitations under the License.
 package kubewatcher

 import (
+	"log"
+
 	"github.com/fission/fission/crd"
 	"github.com/fission/fission/publisher"
 )
@@ -26,6 +28,12 @@ func Start(routerUrl string) error {
 	if err != nil {
 		return err
 	}
+
+	err = fissionClient.WaitForCRDs()
+	if err != nil {
+		log.Fatalf("Error waiting for CRDs: %v", err)
+	}
+
 	poster := publisher.MakeWebhookPublisher(routerUrl)
 	kubeWatch := MakeKubeWatcher(kubeClient, poster)
 	MakeWatchSync(fissionClient, kubeWatch)
@@ -40,7 +40,7 @@
    user  "#{ENV['INFLUXDB_USERNAME']}"
    password  "#{ENV['INFLUXDB_PASSWD']}"
    use_ssl false
-    time_precision s
+    time_precision ns
    tag_keys ["funcuid"]
    sequence_tag _seq
    buffer_type file
@@ -30,6 +30,11 @@ func Start(routerUrl string) error {
 		log.Fatalf("Failed to get fission client: %v", err)
 	}

+	err = fissionClient.WaitForCRDs()
+	if err != nil {
+		log.Fatalf("Error waiting for CRDs: %v", err)
+	}
+
 	// Message queue type: nats is the only supported one for now
 	mqType := os.Getenv("MESSAGE_QUEUE_TYPE")
 	mqUrl := os.Getenv("MESSAGE_QUEUE_URL")
@@ -104,7 +104,7 @@ func (m *azureHTTPClientMock) Do(req *http.Request) (*http.Response, error) {

 func TestNewStorageConnectionMissingAccountName(t *testing.T) {
 	connection, err := newAzureStorageConnection(DummyRouterURL, MessageQueueConfig{
-		MQType: ASQ,
+		MQType: fission.MessageQueueTypeASQ,
 		Url:    "",
 	})
 	require.Nil(t, connection)
@@ -114,7 +114,7 @@ func TestNewStorageConnectionMissingAccountName(t *testing.T) {
 func TestNewStorageConnectionMissingAccessKey(t *testing.T) {
 	_ = os.Setenv("AZURE_STORAGE_ACCOUNT_NAME", "accountname")
 	connection, err := newAzureStorageConnection(DummyRouterURL, MessageQueueConfig{
-		MQType: ASQ,
+		MQType: fission.MessageQueueTypeASQ,
 		Url:    "",
 	})
 	_ = os.Unsetenv("AZURE_STORAGE_ACCOUNT_NAME")
@@ -291,7 +291,7 @@ func TestAzureStorageQueuePoisonMessage(t *testing.T) {
 				Type: fission.FunctionReferenceTypeFunctionName,
 				Name: FunctionName,
 			},
-			MessageQueueType: ASQ,
+			MessageQueueType: fission.MessageQueueTypeASQ,
 			Topic:            QueueName,
 			ContentType:      ContentType,
 		},
@@ -434,7 +434,7 @@ func runAzureStorageQueueTest(t *testing.T, count int, output bool) {
 				Type: fission.FunctionReferenceTypeFunctionName,
 				Name: FunctionName,
 			},
-			MessageQueueType: ASQ,
+			MessageQueueType: fission.MessageQueueTypeASQ,
 			Topic:            QueueName,
 			ResponseTopic:    responseTopic,
 			ContentType:      ContentType,
@@ -18,7 +18,6 @@ package messageQueue

 import (
 	"errors"
-	"regexp"
 	"time"

 	log "github.com/sirupsen/logrus"
@@ -28,21 +27,12 @@ import (
 	"github.com/fission/fission/crd"
 )

-const (
-	NATS string = "nats-streaming"
-	ASQ  string = "azure-storage-queue"
-)
-
 const (
 	ADD_TRIGGER requestType = iota
 	DELETE_TRIGGER
 	GET_ALL_TRIGGERS
 )

-var (
-	validAzureQueueName = regexp.MustCompile("^[a-z0-9][a-z0-9\\-]*[a-z0-9]$")
-)
-
 type (
 	messageQueueSubscription interface{}

@@ -92,9 +82,9 @@ func MakeMessageQueueTriggerManager(fissionClient *crd.FissionClient, routerUrl
 		fissionClient: fissionClient,
 	}
 	switch mqConfig.MQType {
-	case NATS:
+	case fission.MessageQueueTypeNats:
 		messageQueue, err = makeNatsMessageQueue(routerUrl, mqConfig)
-	case ASQ:
+	case fission.MessageQueueTypeASQ:
 		messageQueue, err = newAzureStorageConnection(routerUrl, mqConfig)
 	default:
 		err = errors.New("No matched message queue type found")
@@ -231,13 +221,3 @@ func (mqt *MessageQueueTriggerManager) syncTriggers() {
 		time.Sleep(3 * time.Second)
 	}
 }
-
-func IsTopicValid(mqType string, topic string) bool {
-	switch mqType {
-	case NATS:
-		return isTopicValidForNats(topic)
-	case ASQ:
-		return len(topic) >= 3 && len(topic) <= 63 && validAzureQueueName.MatchString(topic)
-	}
-	return false
-}
@@ -28,6 +28,7 @@ import (
 	"github.com/gorilla/mux"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

+	"github.com/fission/fission"
 	executorClient "github.com/fission/fission/executor/client"
 )

@@ -37,45 +38,138 @@ type functionHandler struct {
 	function *metav1.ObjectMeta
 }

-func (fh *functionHandler) getServiceForFunction() (*url.URL, error) {
-	// call executor, get a url for a function
-	svcName, err := fh.executor.GetServiceForFunction(fh.function)
-	if err != nil {
-		return nil, err
-	}
-	svcUrl, err := url.Parse(fmt.Sprintf("http://%v", svcName))
-	if err != nil {
-		return nil, err
-	}
-	return svcUrl, nil
-}
-
 // A layer on top of http.DefaultTransport, with retries.
 type RetryingRoundTripper struct {
-	maxRetries    int
-	initalTimeout time.Duration
+	maxRetries     int
+	initialTimeout time.Duration
+	funcHandler    *functionHandler
 }

-func (rrt RetryingRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
-	timeout := rrt.initalTimeout
+// RoundTrip is a custom transport with retries for http requests that forwards the request to the right serviceUrl, obtained
+// from router's cache or from executor if router entry is stale.
+//
+// It first checks if the service address for this function came from router's cache.
+// If it didn't, it makes a request to executor to get a new service for function. If that succeeds, it adds the address
+// to it's cache and makes a request to that address with transport.RoundTrip call.
+// Initial requests to new k8s services sometimes seem to fail, but retries work. So, it retries with an exponential
+// back-off for maxRetries times.
+//
+// Else if it came from the cache, it makes a transport.RoundTrip with that cached address. If the response received is
+// a network dial error (which means that the pod doesn't exist anymore), it removes the cache entry and makes a request
+// to executor to get a new service for function. It then retries transport.RoundTrip with the new address.
+//
+// At any point in time, if the response received from transport.RoundTrip is other than dial network error, it is
+// relayed as-is to the user, without any retries.
+//
+// While this RoundTripper handles the case where a previously cached address of the function pod isn't valid anymore
+// (probably because the pod got deleted somehow), by making a request to executor to get a new service for this function,
+// it doesn't handle a case where a newly specialized pod gets deleted just after the GetServiceForFunction succeeds.
+// In such a case, the RoundTripper will retry requests against the new address and give up after maxRetries.
+// However, the subsequent http call for this function will ensure the cache is invalidated.
+//
+// If GetServiceForFunction returns an error or if RoundTripper exits with an error, it get's translated into 502
+// inside ServeHttp function of the reverseProxy.
+// Earlier, GetServiceForFunction was called inside handler function and fission explicitly set http status code to 500
+// if it returned an error.
+func (roundTripper RetryingRoundTripper) RoundTrip(req *http.Request) (resp *http.Response, err error) {
+	var needExecutor, serviceUrlFromExecutor bool
+	var serviceUrl *url.URL
+
+	// set the timeout for transport context
+	timeout := roundTripper.initialTimeout
 	transport := http.DefaultTransport.(*http.Transport)

-	// Do max-1 retries; the last one uses default transport timeouts
-	for i := rrt.maxRetries - 1; i > 0; i-- {
-		// update timeout in transport
+	// cache lookup to get serviceUrl
+	serviceUrl, err = roundTripper.funcHandler.fmap.lookup(roundTripper.funcHandler.function)
+	if err != nil || serviceUrl == nil {
+		// cache miss or nil entry in cache
+		needExecutor = true
+	}
+
+	for i := 0; i < roundTripper.maxRetries-1; i++ {
+		if needExecutor {
+			log.Printf("Calling getServiceForFunction for function: %s", roundTripper.funcHandler.function.Name)
+
+			// send a request to executor to specialize a new pod
+			service, err := roundTripper.funcHandler.executor.GetServiceForFunction(
+				roundTripper.funcHandler.function)
+			if err != nil {
+				// We might want a specific error code or header for fission failures as opposed to
+				// user function bugs.
+				return nil, err
+			}
+
+			// parse the address into url
+			serviceUrl, err = url.Parse(fmt.Sprintf("http://%v", service))
+			if err != nil {
+				return nil, err
+			}
+
+			// add the address in router's cache
+			roundTripper.funcHandler.fmap.assign(roundTripper.funcHandler.function, serviceUrl)
+
+			// flag denotes that service was not obtained from cache, instead, created just now by executor
+			serviceUrlFromExecutor = true
+		}
+
+		// modify the request to reflect the service url
+		// this service url may have come from the cache lookup or from executor response
+		req.URL.Scheme = serviceUrl.Scheme
+		req.URL.Host = serviceUrl.Host
+
+		// To keep the function run container simple, it
+		// doesn't do any routing.  In the future if we have
+		// multiple functions per container, we could use the
+		// function metadata here.
+		// leave the query string intact (req.URL.RawQuery)
+		req.URL.Path = "/"
+
+		// Overwrite request host with internal host,
+		// or request will be blocked in some situations
+		// (e.g. istio-proxy)
+		req.Host = serviceUrl.Host
+
+		// over-riding default settings.
 		transport.DialContext = (&net.Dialer{
 			Timeout:   timeout,
 			KeepAlive: 30 * time.Second,
 		}).DialContext

-		resp, err := transport.RoundTrip(req)
+		// forward the request to the function service
+		resp, err = transport.RoundTrip(req)
 		if err == nil {
+			// if transport.RoundTrip succeeds and it was a cached entry, then tapService
+			if !serviceUrlFromExecutor {
+				go roundTripper.funcHandler.tapService(serviceUrl)
+			}
+			// return response back to user
 			return resp, nil
 		}

-		timeout *= time.Duration(2)
-		log.Printf("Retrying request to %v in %v", req.URL.Host, timeout)
-		time.Sleep(timeout)
+		// if transport.RoundTrip returns a non-network dial error, then relay it back to user
+		if !fission.IsNetworkDialError(err) {
+			return resp, err
+		}
+
+		// means its a newly created service and it returned a network dial error.
+		// just retry after backing off for timeout period.
+		if serviceUrlFromExecutor {
+			log.Printf("request to %s errored out. backing off for %v before retrying",
+				req.URL.Host, timeout)
+			timeout *= time.Duration(2)
+			time.Sleep(timeout)
+			needExecutor = false
+			continue
+		} else {
+			// if transport.RoundTrip returns a network dial error and serviceUrl was from cache,
+			// it means, the entry in router cache is stale, so invalidate it.
+			// also set needExecutor to true so a new service can be requested for function.
+			log.Printf("request to %s errored out. removing function : %s from router's cache "+
+				"and requesting a new service for function",
+				req.URL.Host, roundTripper.funcHandler.function.Name)
+			roundTripper.funcHandler.fmap.remove(roundTripper.funcHandler.function)
+			needExecutor = true
+		}
 	}

 	// finally, one more retry with the default timeout
@@ -90,8 +184,6 @@ func (fh *functionHandler) tapService(serviceUrl *url.URL) {
 }

 func (fh *functionHandler) handler(responseWriter http.ResponseWriter, request *http.Request) {
-	reqStartTime := time.Now()
-
 	// retrieve url params and add them to request header
 	vars := mux.Vars(request)
 	for k, v := range vars {
@@ -101,71 +193,23 @@ func (fh *functionHandler) handler(responseWriter http.ResponseWriter, request *
 	// System Params
 	MetadataToHeaders(HEADERS_FISSION_FUNCTION_PREFIX, fh.function, request)

-	// cache lookup
-	serviceUrl, err := fh.fmap.lookup(fh.function)
-	if err != nil {
-		// Cache miss: request the Pool Manager to make a new service.
-		log.Printf("Not cached, getting new service for %v", fh.function)
-
-		var poolErr error
-		serviceUrl, poolErr = fh.getServiceForFunction()
-		if poolErr != nil {
-			log.Printf("Failed to get service for function %v: %v", fh.function.Name, poolErr)
-			// We might want a specific error code or header for fission
-			// failures as opposed to user function bugs.
-			http.Error(responseWriter, "Internal server error (fission)", 500)
-			return
-		}
-
-		// add it to the map
-		fh.fmap.assign(fh.function, serviceUrl)
-	} else {
-		// if we're using our cache, asynchronously tell
-		// executor we're using this service
-		go fh.tapService(serviceUrl)
-	}
-
-	// Proxy off our request to the serviceUrl, and send the response back.
 	// TODO: As an optimization we may want to cache proxies too -- this might get us
 	// connection reuse and possibly better performance
 	director := func(req *http.Request) {
-		log.Printf("Proxying request for %v to %v", req.URL, serviceUrl.Host)
-
-		// send this request to serviceurl
-		req.URL.Scheme = serviceUrl.Scheme
-		req.URL.Host = serviceUrl.Host
-
-		// To keep the function run container simple, it
-		// doesn't do any routing.  In the future if we have
-		// multiple functions per container, we could use the
-		// function metadata here.
-		req.URL.Path = "/"
-
-		// Overwrite request host with internal host,
-		// or request will be blocked in some situations
-		// (e.g. istio-proxy)
-		req.Host = serviceUrl.Host
-
-		// leave the query string intact (req.URL.RawQuery)
-
 		if _, ok := req.Header["User-Agent"]; !ok {
 			// explicitly disable User-Agent so it's not set to default value
 			req.Header.Set("User-Agent", "")
 		}
 	}

-	// Initial requests to new k8s services sometimes seem to
-	// fail, but retries work.  So use a transport that does retries.
 	proxy := &httputil.ReverseProxy{
 		Director: director,
-		Transport: RetryingRoundTripper{
-			maxRetries:    10,
-			initalTimeout: 50 * time.Millisecond,
+		Transport: &RetryingRoundTripper{
+			initialTimeout: 50 * time.Millisecond,
+			maxRetries:     10,
+			funcHandler:    fh,
 		},
 	}
-	delay := time.Since(reqStartTime)
-	if delay > 100*time.Millisecond {
-		log.Printf("Request delay for %v: %v", serviceUrl, delay)
-	}
+
 	proxy.ServeHTTP(responseWriter, request)
 }
@@ -54,7 +54,9 @@ func TestFunctionProxying(t *testing.T) {
 	fmap := makeFunctionServiceMap(0)
 	fmap.assign(fn, backendURL)

-	fh := &functionHandler{fmap: fmap, function: fn}
+	fh := &functionHandler{fmap: fmap,
+		function: fn,
+	}
 	functionHandlerServer := httptest.NewServer(http.HandlerFunc(fh.handler))
 	fhURL := functionHandlerServer.URL

@@ -75,3 +75,8 @@ func (fmap *functionServiceMap) assign(f *metav1.ObjectMeta, serviceUrl *url.URL
 		// ignore error
 	}
 }
+
+func (fmap *functionServiceMap) remove(f *metav1.ObjectMeta) error {
+	mk := keyFromMetadata(f)
+	return fmap.cache.Delete(*mk)
+}
@@ -82,6 +82,11 @@ func Start(port int, executorUrl string) {
 		log.Fatalf("Error connecting to kubernetes API: %v", err)
 	}

+	err = fissionClient.WaitForCRDs()
+	if err != nil {
+		log.Fatalf("Error waiting for CRDs: %v", err)
+	}
+
 	restClient := fissionClient.GetCrdClient()

 	executor := executorClient.MakeClient(executorUrl)
@@ -0,0 +1,13 @@
+FROM golang:1.10.1 AS go-builder
+WORKDIR /go
+RUN go get github.com/wcharczuk/go-chart
+COPY picasso.go /go
+RUN CGO_ENABLE=0 GOOS=linux GOARCH=amd64 go build -o picasso .
+
+FROM loadimpact/k6
+WORKDIR /fission-bench
+COPY --from=go-builder /go/picasso /usr/local/bin/picasso
+RUN apk --update add --no-cache bash curl
+RUN curl -Lo fission https://github.com/fission/fission/releases/download/$(curl --silent "https://api.github.com/repos/fission/fission/releases/latest" | grep "tag_name" |sed -E 's/.*"([^"]+)".*/\1/')/fission-cli-linux && chmod +x fission && mv fission /usr/local/bin/
+
+ENTRYPOINT ["sh"]
@@ -0,0 +1,2 @@
+def main():
+    return "Hello, world!\n"
@@ -0,0 +1,249 @@
+package main
+
+import (
+	"bufio"
+	"bytes"
+	"encoding/json"
+	"errors"
+	"flag"
+	"fmt"
+	"io"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"strings"
+	"time"
+
+	"github.com/wcharczuk/go-chart"
+)
+
+var (
+	title        *string
+	file         *string
+	outputFile   *string
+	outputFormat *string
+)
+
+const (
+	PNG = "png"
+	SVG = "svg"
+)
+
+type (
+	Tags struct {
+		Group  string `json:"group"`
+		Iter   string `json:"iter"`
+		Method string `json:"method"`
+		Name   string `json:"name"`
+		Proto  string `json:"proto"`
+		Status string `json:"status"`
+		URL    string `json:"url"`
+		Vu     string `json:"vu"`
+	}
+
+	Data struct {
+		Time  time.Time `json:"time"`
+		Value float64   `json:"value"`
+		Tags  Tags      `json:"tags"`
+	}
+
+	MetricPoint struct {
+		Type   string `json:"type"`
+		Data   Data   `json:"data"`
+		Metric string `json:"metric"`
+	}
+)
+
+func init() {
+	title = flag.String("title", "Fission Benchmark", "Chart title")
+	file = flag.String("file", "", "Metric json file")
+	outputFile = flag.String("o", "chart.png", "Output file name")
+	outputFormat = flag.String("format", PNG, "Format of output file (png or svg)")
+	flag.Parse()
+}
+
+func generateContinuousSeries(file string) chart.Series {
+	f, err := os.OpenFile(file, os.O_RDONLY, 0644)
+	if err != nil {
+		fmt.Printf("Failed to open file metric point: %v", err)
+		return nil
+	}
+	defer f.Close()
+
+	reader := bufio.NewReader(f)
+
+	var points []*MetricPoint
+	var xVals []float64
+	var yVals []float64
+
+	var initTime *time.Time
+
+	for {
+		l, _, err := reader.ReadLine()
+		if err == io.EOF {
+			break
+		}
+
+		point := &MetricPoint{}
+		err = json.Unmarshal(l, point)
+		if err != nil {
+			fmt.Printf("Failed to parse metric point: %v -> %v", err, string(l))
+			return nil
+		}
+
+		if point.Type != "Point" || point.Metric != "http_req_duration" {
+			continue
+		}
+
+		points = append(points, point)
+
+		if initTime == nil {
+			initTime = &point.Data.Time
+		}
+
+		timeSinceStart := point.Data.Time.Sub(*initTime).Seconds()
+		xVals = append(xVals, timeSinceStart)
+		yVals = append(yVals, point.Data.Value)
+	}
+
+	return chart.ContinuousSeries{
+		Style: chart.Style{
+			Show:        true,
+			StrokeColor: chart.GetDefaultColor(0).WithAlpha(64),
+			FillColor:   chart.GetDefaultColor(0).WithAlpha(64),
+		},
+		XValues: xVals,
+		YValues: yVals,
+	}
+}
+
+func generateChart(title string, file string, format chart.RendererProvider, series []chart.Series) error {
+	if series == nil {
+		return errors.New("Series cannot be nil")
+	}
+
+	cs := chart.ConcatSeries(series)
+
+	graph := chart.Chart{
+		Title:      title,
+		TitleStyle: chart.StyleShow(),
+		Background: chart.Style{
+			Padding: chart.Box{
+				Top:    50,
+				Left:   25,
+				Right:  25,
+				Bottom: 10,
+			},
+		},
+		XAxis: chart.XAxis{
+			Name:      "Time (s)",
+			NameStyle: chart.StyleShow(),
+			Style:     chart.StyleShow(),
+			Range: &chart.ContinuousRange{
+				Min: 0,
+			},
+			ValueFormatter: func(v interface{}) string {
+				return fmt.Sprintf("%.2f s", v.(float64))
+			},
+		},
+		YAxis: chart.YAxis{
+			Name:      "Response Time (ms)",
+			NameStyle: chart.StyleShow(),
+			Style:     chart.StyleShow(),
+			Range: &chart.ContinuousRange{
+				Min: 0,
+			},
+			ValueFormatter: func(v interface{}) string {
+				return fmt.Sprintf("%d ms", int(v.(float64)))
+			},
+		},
+		Series: cs,
+	}
+
+	buffer := bytes.NewBuffer([]byte{})
+	err := graph.Render(format, buffer)
+	if err != nil {
+		return err
+	}
+
+	err = ioutil.WriteFile(file, buffer.Bytes(), 0644)
+	if err != nil {
+		return err
+	}
+
+	return nil
+}
+
+func listJsonFiles(path string) ([]string, error) {
+	fi, err := os.Stat(*file)
+	if err != nil {
+		return nil, err
+	}
+
+	if fi.Mode().IsRegular() {
+		return []string{path}, nil
+	}
+
+	var files []string
+
+	err = filepath.Walk(path, func(path string, info os.FileInfo, err error) error {
+		if err != nil {
+			return err
+		}
+
+		if strings.HasSuffix(path, ".json") && !info.IsDir() {
+			files = append(files, path)
+			return nil
+		}
+
+		return nil
+	})
+
+	return files, nil
+}
+
+func main() {
+
+	if file == nil || len(*file) == 0 {
+		fmt.Println("Please provide metric json file name")
+		return
+	}
+
+	var format chart.RendererProvider
+
+	if outputFormat == nil {
+		format = chart.PNG
+	} else {
+		switch strings.ToLower(*outputFormat) {
+		case PNG:
+			format = chart.PNG
+		case SVG:
+			format = chart.SVG
+		default:
+			fmt.Println("Unknown format, use png as output format")
+			*outputFormat = PNG
+		}
+	}
+
+	if len(*outputFile) == 0 {
+		fmt.Println("Please output chart png file name")
+		return
+	}
+
+	var series []chart.Series
+
+	files, err := listJsonFiles(*file)
+	if err != nil {
+		fmt.Printf("Failed to get file information: %v", err)
+		return
+	}
+
+	for _, f := range files {
+		series = append(series, generateContinuousSeries(f))
+	}
+
+	err = generateChart(*title, *outputFile, format, series)
+	if err != nil {
+		fmt.Printf("Failed to generate chart: %v\n", err)
+	}
+}
@@ -0,0 +1,88 @@
+#!/bin/bash
+
+set -euo pipefail
+
+ROOT=$(dirname $0)/../../../..
+
+for executorType in poolmgr newdeploy
+do
+    dirName="burst-load-executor-${executorType}"
+
+    # remove old data
+    rm -rf ${dirName}
+    mkdir ${dirName}
+    pushd ${dirName}
+
+    # run multiple iterations to reduce impact of imbalance of pod distribution.
+    for iteration in {1..10}
+    do
+
+        # Create a hello world function in nodejs, test it with an http trigger
+        echo "Pre-test cleanup"
+        fission env delete --name python || true
+
+        echo "Creating python env"
+        # Use short grace period time to speed up resource recycle time
+        # Use high min/max CPU so that K8S will distribute pod in different nodes
+        fission env create --name python --version 2 --image fission/python-env --period 5 --mincpu 300 --maxcpu 300 --minmemory 256 --maxmemory 256
+        trap "fission env delete --name python" EXIT
+
+        sleep 30
+
+        fn=python-hello-$(date +%s)
+
+        echo "Creating package"
+        rm -rf pkg.zip pkg/ || true
+        mkdir pkg
+        cp ../../../assets/hello.py pkg/hello.py
+
+        zip -jr pkg.zip pkg/
+        pkgName=$(fission pkg create --env python --deploy pkg.zip | cut -d' ' -f 2 | cut -d"'" -f 2)
+
+        echo "Creating function"
+        fission fn create --name $fn --env python --pkg ${pkgName} --entrypoint "hello.main" --executortype ${executorType} --minscale 3 --maxscale 3
+
+        echo "Creating route"
+        fission route create --function $fn --url /$fn --method GET
+
+        echo "Waiting for router to catch up"
+        sleep 5
+
+        fnEndpoint="http://$FISSION_ROUTER/$fn"
+        js="sample.js"
+        rawFile="raw-${iteration}.json"
+        rawUsageReport="raw-usage.txt"
+
+        # Stage 1: 15s, 10 vus
+        # Stage 2: 45s, 500 vus
+        # Please check sample.js for more detail.
+        k6 run \
+            -e FN_ENDPOINT="${fnEndpoint}" \
+            --no-connection-reuse \
+            --out json="${rawFile}" \
+            --summary-trend-stats="avg,min,med,max,p(5),p(10),p(15),p(20),p(25),p(30),p(35),p(40),p(45),p(50),p(55),p(60),p(65),p(70),p(75),p(80),p(85),p(90),p(95),p(100)" \
+            ../${js} >> ${rawUsageReport}
+
+        echo "Clean up"
+        fission fn delete --name ${fn}
+        fission env delete --name python
+        fission route list| grep ${fn}| awk '{print $1}'| xargs fission route delete --name
+        fission pkg delete --name ${pkgName}
+        rm -rf pkg.zip pkg
+
+        kubectl -n fission-function get deploy -o name|xargs -I@ bash -c "kubectl -n fission-function delete @" || true
+        kubectl -n fission-function get pod -o name|xargs -I@ bash -c "kubectl -n fission-function delete @" || true
+
+        echo "All done."
+    done
+
+    usageReport="usage.txt"
+    cat ${rawUsageReport}| grep "http_req_duration"| cut -f2 -d':' > ${usageReport}
+
+    popd
+
+    # generate report after iterations are over
+    outImage="${dirName}.png"
+    picasso -file ${dirName} -format png -o ${outImage}
+
+done
@@ -0,0 +1,17 @@
+import http from "k6/http";
+import { check } from "k6";
+
+export let options = {
+    stages: [
+        { duration: "15s", target: 10 },
+        { duration: "45s", target: 500  },
+    ]
+};
+
+export default function() {
+    let params = { timeout: 30 }
+    let res = http.get(`${__ENV.FN_ENDPOINT}`)
+    check(res, {
+        "status is 200": (r) => r.status === 200
+    });
+};
@@ -0,0 +1,93 @@
+#!/bin/bash
+
+set -euo pipefail
+
+ROOT=$(dirname $0)/../../../..
+
+for executorType in poolmgr newdeploy
+do
+    for concurrency in 100 250 500 750 1000
+    do
+
+        testDuration="60"
+        dirName="concurrency-${concurrency}-executor-${executorType}"
+
+        # remove old data
+        rm -rf ${dirName}
+        mkdir ${dirName}
+        pushd ${dirName}
+
+        # run multiple iterations to reduce impact of imbalance of pod distribution.
+        for iteration in {1..10}
+        do
+
+            # Create a hello world function in nodejs, test it with an http trigger
+            echo "Pre-test cleanup"
+            fission env delete --name python || true
+
+            echo "Creating python env"
+            # Use short grace period time to speed up resource recycle time
+            # Use high min/max CPU so that K8S will distribute pod in different nodes
+            fission env create --name python --version 2 --image fission/python-env --period 5 --mincpu 300 --maxcpu 300 --minmemory 256 --maxmemory 256
+            trap "fission env delete --name python" EXIT
+
+            sleep 30
+
+            fn=python-hello-$(date +%s)
+
+            echo "Creating package"
+            rm -rf pkg.zip pkg/ || true
+            mkdir pkg
+            cp ../../../assets/hello.py pkg/hello.py
+
+            zip -jr pkg.zip pkg/
+            pkgName=$(fission pkg create --env python --deploy pkg.zip | cut -d' ' -f 2 | cut -d"'" -f 2)
+
+            echo "Creating function"
+            fission fn create --name $fn --env python --pkg ${pkgName} --entrypoint "hello.main" --executortype ${executorType} --minscale 3 --maxscale 3
+
+            echo "Creating route"
+            fission route create --function $fn --url /$fn --method GET
+
+            echo "Waiting for router to catch up"
+            sleep 5
+
+            fnEndpoint="http://$FISSION_ROUTER/$fn"
+            js="sample.js"
+            rawFile="raw-${iteration}.json"
+            rawUsageReport="raw-usage.txt"
+
+            k6 run \
+                -e FN_ENDPOINT="${fnEndpoint}" \
+                --duration "${testDuration}s" \
+                --rps ${concurrency} \
+                --vus ${concurrency} \
+                --no-connection-reuse \
+                --out json="${rawFile}" \
+                --summary-trend-stats="avg,min,med,max,p(5),p(10),p(15),p(20),p(25),p(30),p(35),p(40),p(45),p(50),p(55),p(60),p(65),p(70),p(75),p(80),p(85),p(90),p(95),p(100)" \
+                ../${js} >> ${rawUsageReport}
+
+            echo "Clean up"
+            fission fn delete --name ${fn}
+            fission env delete --name python
+            fission route list| grep ${fn}| awk '{print $1}'| xargs fission route delete --name
+            fission pkg delete --name ${pkgName}
+            rm -rf pkg.zip pkg
+
+            kubectl -n fission-function get deploy -o name|xargs -I@ bash -c "kubectl -n fission-function delete @" || true
+            kubectl -n fission-function get pod -o name|xargs -I@ bash -c "kubectl -n fission-function delete @" || true
+
+            echo "All done."
+        done
+
+        usageReport="usage.txt"
+        cat ${rawUsageReport}| grep "http_req_duration"| cut -f2 -d':' > ${usageReport}
+
+        popd
+
+        # generate report after iterations are over
+        outImage="${dirName}.png"
+        picasso -file ${dirName} -format png -o ${outImage}
+
+    done
+done
@@ -0,0 +1,10 @@
+import http from "k6/http";
+import { check } from "k6";
+
+export default function() {
+    let params = { timeout: 30 }
+    let res = http.get(`${__ENV.FN_ENDPOINT}`)
+    check(res, {
+        "status is 200": (r) => r.status === 200
+    });
+};
@@ -0,0 +1,117 @@
+#!/bin/bash
+
+set -euo pipefail
+
+ROOT=$(dirname $0)/../../../..
+
+for executorType in poolmgr newdeploy
+do
+    for packagesize in 0 1 5 10 15 20
+    do
+
+        testDuration="5"
+        dirName="package-size-${packagesize}-executor-${executorType}"
+
+        # remove old data
+        rm -rf ${dirName}
+        mkdir ${dirName}
+        pushd ${dirName}
+
+        # run multiple iterations to reduce impact of imbalance of pod distribution.
+        for iteration in {1..10}
+        do
+
+            # Create a hello world function in nodejs, test it with an http trigger
+            echo "Pre-test cleanup"
+            fission env delete --name python || true
+
+            echo "Creating python env"
+            # Use short grace period time to speed up resource recycle time
+            # Use high min/max CPU so that K8S will distribute pod in different nodes
+
+            version=2
+
+            if [[ "${packagesize}" == "0" ]]
+            then
+                version=1
+            fi
+
+            fission env create --name python --version ${version} --image fission/python-env --period 5 --mincpu 300 --maxcpu 300 --minmemory 256 --maxmemory 256
+
+            trap "fission env delete --name python" EXIT
+
+            sleep 30
+
+            fn=python-hello-$(date +%s)
+
+            pkgName=""
+
+            if [[ "${packagesize}" == "0" ]]
+            then
+                echo "Creating function"
+                fission fn create --name $fn --env python --code ../../../assets/hello.py --executortype ${executorType} --minscale 3 --maxscale 3
+            else
+                echo "Creating package"
+                rm -rf pkg.zip pkg/ || true
+                mkdir pkg
+                cp ../../../assets/hello.py pkg/hello.py
+
+                # Create empty file with give size to simulate different size of package
+                truncate -s ${packagesize}MiB pkg/foo
+
+                zip -jr pkg.zip pkg/
+                pkgName=$(fission pkg create --env python --deploy pkg.zip | cut -d' ' -f 2 | cut -d"'" -f 2)
+
+                echo "Creating function"
+                fission fn create --name $fn --env python --pkg ${pkgName} --entrypoint "hello.main" --executortype ${executorType} --minscale 3 --maxscale 3
+            fi
+
+            echo "Creating route"
+            fission route create --function $fn --url /$fn --method GET
+
+            echo "Waiting for router to catch up"
+            sleep 5
+
+            fnEndpoint="http://$FISSION_ROUTER/$fn"
+            js="sample.js"
+            rawFile="raw-${iteration}.json"
+            rawUsageReport="raw-usage.txt"
+
+            k6 run \
+                -e FN_ENDPOINT="${fnEndpoint}" \
+                --duration "${testDuration}s" \
+                --rps 1 \
+                --vus 1 \
+                --no-connection-reuse \
+                --out json="${rawFile}" \
+                --summary-trend-stats="avg,min,med,max,p(5),p(10),p(15),p(20),p(25),p(30),p(35),p(40),p(45),p(50),p(55),p(60),p(65),p(70),p(75),p(80),p(85),p(90),p(95),p(100)" \
+                ../${js} >> ${rawUsageReport}
+
+            echo "Clean up"
+            fission fn delete --name ${fn}
+            fission env delete --name python
+            fission route list| grep ${fn}| awk '{print $1}'| xargs fission route delete --name
+
+            if [[ ! -z "${pkgName}" ]]
+            then
+                fission pkg delete --name ${pkgName} || true
+                rm -rf pkg.zip pkg
+            fi
+
+            kubectl -n fission-function get deploy -o name|xargs -I@ bash -c "kubectl -n fission-function delete @" || true
+            kubectl -n fission-function get pod -o name|xargs -I@ bash -c "kubectl -n fission-function delete @" || true
+
+            echo "All done."
+        done
+
+        usageReport="usage.txt"
+        cat ${rawUsageReport}| grep "http_req_duration"| cut -f2 -d':' > ${usageReport}
+
+        popd
+
+        # generate report after iterations are over
+        outImage="${dirName}.png"
+        picasso -file ${dirName} -format png -o ${outImage}
+
+    done
+done
@@ -0,0 +1,10 @@
+import http from "k6/http";
+import { check } from "k6";
+
+export default function() {
+    let params = { timeout: 30 }
+    let res = http.get(`${__ENV.FN_ENDPOINT}`)
+    check(res, {
+        "status is 200": (r) => r.status === 200
+    });
+};
@@ -2,6 +2,13 @@

 set -euo pipefail

+# Unbound variables cause failure, so this readable if block instead of Parameter Expansion
+if [[ ${TRAVIS_EVENT_TYPE+NOVALUE} == "cronNOVALUE" ]]
+then
+    echo "Skipping build & test, this is cron job for fission upgrade tests"
+    exit 0
+fi
+
 if [ ! -f ${HOME}/.kube/config ]
 then
    echo "Skipping end to end tests, no cluster credentials"
@@ -17,7 +24,8 @@ FLUENTD_IMAGE=gcr.io/fission-ci/fluentd
 BUILDER_IMAGE=$REPO/builder
 TAG=test
 PRUNE_INTERVAL=1 # this variable controls the interval to run archivePruner. The unit is in minutes.
-ROUTER_SERVICE_TYPE=ClusterIP
+ROUTER_SERVICE_TYPE=LoadBalancer
+SERVICE_TYPE=LoadBalancer

 dump_system_info

@@ -37,4 +45,4 @@ build_and_push_fluentd $FLUENTD_IMAGE:$TAG

 build_fission_cli

-install_and_test $IMAGE $TAG $FETCHER_IMAGE $TAG $FLUENTD_IMAGE $TAG $PRUNE_INTERVAL $ROUTER_SERVICE_TYPE
+install_and_test $IMAGE $TAG $FETCHER_IMAGE $TAG $FLUENTD_IMAGE $TAG $PRUNE_INTERVAL $ROUTER_SERVICE_TYPE $SERVICE_TYPE
@@ -1,45 +0,0 @@
-#!/bin/bash
-
-set -e
-
-clusterID="fissionMQTrigger"
-topic="foo.bar"
-resptopic="foo.foo"
-expectedRespOutput="[foo.foo]: 'Hello, World!'"
-FISSIONDIR=$GOPATH"/src/github.com/fission/fission"
-
-if [[ -z $NATS_STREAMING_URL ]]; then
-    echo "'NATS_STREAMING_URL' must not be empty. For example: export NATS_STREAMING_URL=nats://192.168.0.1:4222"
-    exit 1
-fi
-
-if [[ -z $FISSION_URL ]]; then
-    echo "'FISSION_URL' must not be empty. For example: export FISSION_URL=http://10.10.10.10"
-    exit 1
-fi
-
-cd $FISSIONDIR"/fission/"
-go build
-mv fission $FISSIONDIR"/test/mqtrigger"
-cd $FISSIONDIR"/test/mqtrigger"
-
-./fission env create --name nodejs --image fission/node-env
-./fission fn create --name hello1 --env nodejs --code main.js --method GET
-./fission route create --method GET --url /h1 --function hello1
-./fission mqtrigger create --name h1 --function hello1 --mqtype "nats-streaming" --topic "foo.bar" --resptopic "foo.foo"
-
-# wait until nats trigger is created
-sleep 5
-
-go run ./stan-pub.go -s $NATS_STREAMING_URL -c $clusterID -id clientPub $topic "" || exit 1
-
-response=$(go run ./stan-sub.go --last -s $NATS_STREAMING_URL -c $clusterID -id clientSub $resptopic 2>&1)
-
-if [[ "$response" != "$expectedRespOutput" ]]; then
-    echo "$response is not equal to $expectedRespOutput"
-    exit 1
-fi
-
-echo "Subscriber received expected response: $response"
-
-exit 0
@@ -148,6 +148,18 @@ clean_tpr_crd_resources() {
    kubectl --namespace default get crd| grep -v NAME| grep "fission.io"| awk '{print $1}'|xargs -I@ bash -c "kubectl --namespace default delete crd @"  || true
 }

+set_environment() {
+    id=$1
+    ns=f-$id
+
+    export FISSION_URL=http://$(kubectl -n $ns get svc controller -o jsonpath='{...ip}')
+    export FISSION_ROUTER=$(kubectl -n $ns get svc router -o jsonpath='{...ip}')
+    export FISSION_NATS_STREAMING_URL="http://defaultFissionAuthToken@$(kubectl -n $ns get svc nats-streaming -o jsonpath='{...ip}:{.spec.ports[0].port}')"
+
+    # set path to include cli
+    export PATH=$ROOT/fission:$PATH
+}
+
 generate_test_id() {
    echo $(date|md5sum|cut -c1-6)
 }
@@ -161,14 +173,15 @@ helm_install_fission() {
    controllerNodeport=$6
    routerNodeport=$7
    fluentdImage=$8
-    fluentdImageTag=$9
+    fluentdImageTag=${9}
    pruneInterval="${10}"
    routerServiceType=${11}
+    serviceType=${12}

    ns=f-$id
    fns=f-func-$id

-    helmVars=image=$image,imageTag=$imageTag,fetcherImage=$fetcherImage,fetcherImageTag=$fetcherImageTag,functionNamespace=$fns,controllerPort=$controllerNodeport,routerPort=$routerNodeport,pullPolicy=Always,analytics=false,logger.fluentdImage=$fluentdImage,logger.fluentdImageTag=$fluentdImageTag,pruneInterval=$pruneInterval,routerServiceType=$routerServiceType
+    helmVars=image=$image,imageTag=$imageTag,fetcherImage=$fetcherImage,fetcherImageTag=$fetcherImageTag,functionNamespace=$fns,controllerPort=$controllerNodeport,routerPort=$routerNodeport,pullPolicy=Always,analytics=false,logger.fluentdImage=$fluentdImage,logger.fluentdImageTag=$fluentdImageTag,pruneInterval=$pruneInterval,routerServiceType=$routerServiceType,serviceType=$serviceType

    timeout 30 bash -c "helm_setup"

@@ -215,6 +228,32 @@ dump_tiller_logs() {
 }
 export -f dump_tiller_logs

+wait_for_service() {
+    id=$1
+    svc=$2
+
+    ns=f-$id
+    while true
+    do
+	ip=$(kubectl -n $ns get svc $svc -o jsonpath='{...ip}')
+	if [ ! -z $ip ]
+	then
+	    break
+	fi
+	echo Waiting for service $svc...
+	sleep 1
+    done
+}
+
+wait_for_services() {
+    id=$1
+
+    wait_for_service $id controller
+    wait_for_service $id router
+
+    echo Waiting for service is routable...
+    sleep 10
+}

 helm_uninstall_fission() {(set +e
    id=$1
@@ -234,17 +273,31 @@ export -f helm_uninstall_fission
 port_forward_services() {
    id=$1
    ns=f-$id
-    port=8888
+    svc=$2
+    port=$3

-    kubectl get pods -l svc="router" -o name --namespace $ns | \
+    kubectl get pods -l svc="$svc" -o name --namespace $ns | \
        sed 's/^.*\///' | \
        xargs -I{} kubectl port-forward {} $port:$port -n $ns &
-
-    export FISSION_ROUTER="127.0.0.1:"
-    FISSION_ROUTER+="$port"
-    export PATH=$ROOT/fission:$PATH
 }

+wait_for_service() { 
+    id=$1 
+    svc=$2 
+  
+    ns=f-$id 
+    while true 
+        do 
+        ip=$(kubectl -n $ns get svc $svc -o jsonpath='{...ip}') 
+        if [ ! -z $ip ] 
+        then 
+            break 
+        fi 
+        echo Waiting for service $svc... 
+        sleep 1 
+    done 
+ } 
+
 dump_builder_pod_logs() {
    bns=$1
    builderPods=$(kubectl -n $bns get pod -o name)
@@ -367,6 +420,8 @@ dump_logs() {
    dump_fission_logs $ns $fns buildermgr
    dump_fission_logs $ns $fns executor
    dump_fission_logs $ns $fns storagesvc
+    dump_fission_logs $ns $fns mqtrigger
+    dump_fission_logs $ns $fns nats-streaming
    dump_function_pod_logs $ns $fns
    dump_builder_pod_logs $bns
    dump_fission_crds
@@ -428,15 +483,16 @@ install_and_test() {
    fluentdImageTag=$6
    pruneInterval=$7
    routerServiceType=$8
+    serviceType=$9

    controllerPort=31234
    routerPort=31235

    clean_tpr_crd_resources
-
+    
    id=$(generate_test_id)
    trap "helm_uninstall_fission $id" EXIT
-    helm_install_fission $id $image $imageTag $fetcherImage $fetcherImageTag $controllerPort $routerPort $fluentdImage $fluentdImageTag $pruneInterval $routerServiceType
+    helm_install_fission $id $image $imageTag $fetcherImage $fetcherImageTag $controllerPort $routerPort $fluentdImage $fluentdImageTag $pruneInterval $routerServiceType $serviceType
    helm status $id | grep STATUS | grep -i deployed
    if [ $? -ne 0 ]; then
        describe_all_pods $id
@@ -445,7 +501,8 @@ install_and_test() {
 	    exit 1
    fi

-    port_forward_services $id $routerPort
+    wait_for_services $id
+    set_environment $id
    run_all_tests $id

    dump_logs $id
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Ta-Ching ChenandGitHub	6a3e326f36	Add benchmark script (#666 )	2018-05-05 12:42:16 +08:00
smruthi2187andTa-Ching Chen	f0dc2554a1	Meaningful error message when fetch request is received for a package when build is not successful. (#661 )	2018-05-03 22:51:48 +08:00
VishalandTa-Ching Chen	2ece0db13f	Fixed the issue with update wiping original values & added resources in listing, fixes #654 (#663 )	2018-05-02 22:29:08 +08:00
Ta-Ching ChenandGitHub	0e3a270d27	Delete deployment with proper delete propagation policy (#630 )	2018-05-01 23:25:50 +08:00
Ta-Ching ChenandGitHub	6a7381c575	Fix newdeploy backend failed to delete deployment due to incorrect resource version (#657 )	2018-05-01 16:48:34 +08:00
VishalandTa-Ching Chen	0aff2f214d	Fixes the issue with fn test and adds relevant test cases, fixes #650 (#651 ) * Fixes the issue with fn test and adds relevant test cases, fixes #650 * Checking log size for invalid function * Testing the invalid function logs a few time beforr quitting * Disabling the test for now, linked to issue #653	2018-04-30 14:14:46 +08:00
VishalandTa-Ching Chen	dec4a39c15	Function update should be possible without change to code (#652 )	2018-04-30 11:35:19 +08:00
Ta-Ching ChenandGitHub	7577339912	Change time precision for fluentd influxdb plugin to nano second (#646 )	2018-04-26 20:57:55 +08:00
Ta-Ching ChenandGitHub	6375819195	Fix test cases occasional failure (#647 )	2018-04-26 14:39:16 +08:00
VishalandGitHub	0b34abcada	Update .travis.yml Seems a small item is missing, causing notifications not to work. Similar block in fission-workflows does work.	2018-04-25 23:15:32 +05:30
smruthi2187andTa-Ching Chen	772fdfa15c	Setting package buildStatus to pending when functions source is updated. (#637 )	2018-04-24 15:24:46 +08:00
VishalandGitHub	7198b24028	Fission upgrade tests (#605 ) Tests for upgrading Fission from the last version to current branch and testing. Currently test is very simple and more tests can be added in future	2018-04-24 12:51:17 +05:30
VishalandGitHub	eb696432d8	Testing proposal: Requirements and frameworks exploration (#581 ) Proposal for revamping the testing framework and tests in general based on needs of Fission project	2018-04-24 00:45:09 +05:30
Ta-Ching ChenandGitHub	2d359f879a	Fix SEGFAULT issue when buildmgr failed to update package (#635 )	2018-04-21 23:17:33 +08:00
smruthi2187andGitHub	94e2f11ea6	Remove port forward in tests for router, controller and nats pods (#611 )	2018-04-20 19:00:22 -07:00
VishalandTa-Ching Chen	ff01cf2452	Removed the fn pods functionality (#594 )	2018-04-19 14:32:16 +08:00
Ta-Ching ChenandGitHub	400e19a48f	Fix executor does not reap specialized function pod when env no longer exists (#633 )	2018-04-19 07:39:37 +08:00
Ta-Ching ChenandGitHub	0194f53e12	Fix SEGFAULT issue when build manager failed to update package (#626 )	2018-04-17 09:36:11 +08:00
John GallagherandSoam Vasani	fef1469c61	Update readme to point to the proper link (#628 )	2018-04-16 11:30:27 -07:00
Ta-Ching ChenandGitHub	cf53e17be5	Remove confusing default value description & add default value to cli flag (#619 )	2018-04-16 17:27:10 +08:00
smruthi2187andGitHub	449f49515a	Changes needed for release 0.7.1 (#622 ) bump up the version to 0.7.1 in charts and adding changelog.	2018-04-10 18:03:08 -07:00
Ta-Ching ChenandGitHub	d01b309fc7	Add validate function to crd resource and do validate before creation/update (#580 )	2018-04-08 00:56:13 +08:00
smruthi2187andGitHub	12299df811	Prevent releasing idle connections because transport is shared. (#609 ) The transport in RoundTripper is the default transport and there's just one object shared between different http requests and releasing the idle Connections ended up releasing the idle connections that were established with the executor service address too. So every time a http request was made to get a service for function, the transport layer ended up having to dial a new connection (that was timing out due to various reasons). By removing the CloseIdleConnections, we minimize the need for transport to dial a new connection to executor for every request, thereby minimizing the occurrence of dial timeouts.	2018-04-06 13:23:25 -07:00
Soam VasaniandGitHub	95a50dbbd6	--spec flags for env and trigger creates (#607 )	2018-04-06 09:00:01 -07:00
smruthi2187andGitHub	8014c83b02	Invalidate stale router cache entry with podIP's for deleted pods. (#546 ) The router's cache entry for a function might become stale if the pod that had the function specialized gets deleted somehow. In such a case, we'd retry getting a new service for the function from executor and retry forwarding the user request to the newly created service.	2018-04-05 13:47:47 -07:00
smruthi2187andGitHub	db2f620121	Updating releasing guideliness with a few more details. (#599 )	2018-04-05 13:44:17 -07:00
Soam VasaniandGitHub	f9ca626940	E2E test for NATS-streaming trigger (#338 ) E2E test for NATS-streaming trigger: move ad-hoc shell script into CI tests.	2018-04-05 10:07:00 -07:00
Ta-Ching ChenandGitHub	1dbf8f9243	Fix components crash before crds creation (#602 ) * Wait for CRDs creation for 30 sec when component start * Fix ensureCRD return nil while the error is not empty	2018-04-04 15:40:55 +08:00
Ta-Ching ChenandGitHub	9406720e8a	Add deprecated message to subcommand pods (#592 )	2018-04-04 01:26:27 +08:00
Ta-Ching ChenandGitHub	2a06705320	Use a separate controller loop to watch functions change and create a service (#544 )	2018-04-04 01:17:51 +08:00
smruthi2187andGitHub	d39944e04d	updates to changelog. (#598 )	2018-04-02 18:08:15 -07:00
smruthi2187andGitHub	6f6b52438a	changes needed for release 0.7.0 (#597 )	2018-04-02 15:32:22 -07:00
Soam VasaniandGitHub	5baf67f143	Spec validator, better errors, apply waits for previous build (#560 ) Adds a validator, invoked through `fission spec validate` and before `fission spec apply`. The validator checks for duplicate names, dangling references, and unused resources. Also attempts to fix a race between CLI and builder. Avoids replacing a package while it's building, instead waiting for the build status to change from pending before updating it.	2018-04-02 14:33:31 -07:00
xiekeyangandSoam Vasani	6d61f6e5eb	bug fix: spec dir flag (#595 ) The cli spec dir flag name should be `specdir`, not `specs`, that should match to: ```go specDirFlag := cli.StringFlag{Name: "specdir", Usage: "Directory to store specs, defaults to ./specs"} ``` in fission/main.go Currently it cause the specific specs dir not to work. ``` $ fission spec apply --specdir <my spec dir> Everything up to date. ```	2018-04-02 05:51:46 -07:00
Ta-Ching ChenandGitHub	6ab902e7a3	Add steps for how to set FISSION_ROUTER env variable & update docs (#593 )	2018-03-31 01:10:40 +08:00
smruthi2187andTa-Ching Chen	ed7d088bbc	Check if the requested file already exists in fetcher and skip fetch (#584 )	2018-03-30 16:33:06 +08:00
smruthi2187andGitHub	c0058153a0	Adding routerUrl parameter for kubewatch, timer, mqueue trigger deployments (#591 ) * This is to enable these triggers to be able to talk to fission router if deployed in a namespace other than fission.	2018-03-29 17:38:50 -07:00
VishalandGitHub	a8482a8b89	Function logs: Uses proper way to get server URL After changes related to port forwarding, function logs was getting empty URL, fixed to use the appropriate method instead.	2018-03-29 15:26:47 +05:30
VishalandGitHub	a23f8acffc	Tests for function update (#550 ) Tests for updates to a function of new deployment executor type. Tests check for changes in the environment, scale, secrets etc. Also checks conversion of function executor type from new deployment to pool manager and vice versa	2018-03-28 20:32:04 +05:30
smruthi2187andSoam Vasani	77be3379b2	Prepending a slash to user input url if missing. (#547 ) * Prepending a slash to user input url if missing. * Adding the url slash check for spec.	2018-03-26 12:31:02 -07:00
Ta-Ching ChenandSoam Vasani	03046170dc	Add post-upgrade-job to track fission upgrade (#564 )	2018-03-26 12:28:35 -07:00
VishalandSoam Vasani	c0acaa74c3	Fixes the issue #559 with env versions (#569 ) Do not set automatically set env version if user has already specified it.	2018-03-26 12:25:16 -07:00
Soam VasaniandGitHub	8708fdec93	Add verbosity flag and verbose logs for portforwarder (#575 ) Also, parse flags before connecting to server.	2018-03-26 12:23:45 -07:00
Chris LeeandSoam Vasani	6b188c053a	Add golang example to installation guide (#578 ) Use --src instead of --code for Go example.	2018-03-26 10:04:25 -07:00
Soam Vasani	0fd6114218	Update release checklist	2018-03-22 14:05:51 -07:00
Soam Vasani	9ae06f56c8	Update generated changelog	2018-03-22 14:03:02 -07:00