Chart version update

Add AWS instructions to setup commands, since AWS only provides service hostname, not IP.

Documentation/docs-site/config.toml

@@ -1,3 +1,4 @@
 baseURL = "http://fission.io/docs/0.3.0"
 languageCode = "en-us"
 title = "Fission: Serverless Functions for Kubernetes"
+theme = "docdock"

Documentation/docs-site/content/install.md

@@ -6,7 +6,7 @@ draft: false
 
 Welcome! This guide will get you up and running with Fission on a
 Kubernetes cluster.
-           
+
 ### Cluster preliminaries
 
 If you don't have a Kubernetes cluster, [here's a quick guide to set
@@ -48,18 +48,18 @@ First, you'll need the helm CLI:
 
 On __OS X__:
 ```
-$ curl -LO https://storage.googleapis.com/kubernetes-helm/helm-v2.6.1-darwin-amd64.tar.gz
+$ curl -LO https://storage.googleapis.com/kubernetes-helm/helm-v2.7.0-darwin-amd64.tar.gz
 
-$ tar xzf helm-v2.6.1-darwin-amd64.tar.gz
+$ tar xzf helm-v2.7.0-darwin-amd64.tar.gz
 
 $ mv darwin-amd64/helm /usr/local/bin
 ```
 
 On __Linux__:
 ```
-$ curl -LO https://storage.googleapis.com/kubernetes-helm/helm-v2.6.1-linux-amd64.tar.gz
+$ curl -LO https://storage.googleapis.com/kubernetes-helm/helm-v2.7.0-linux-amd64.tar.gz
 
-$ tar xzf helm-v2.6.1-linux-amd64.tar.gz
+$ tar xzf helm-v2.7.0-linux-amd64.tar.gz
 
 $ mv linux-amd64/helm /usr/local/bin
 ```
@@ -67,24 +67,15 @@ $ mv linux-amd64/helm /usr/local/bin
 Next, install the Helm server on your Kubernetes cluster:
 
 ```
-$ kubectl -n kube-system create sa tiller
+$ helm init
-
-$ kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
-
-$ helm init --service-account tiller
 ```
 
-(The first two commands are there to make sure that helm is allowed to
-install stuff on Kubernetes, in the common case that your cluster has
-role-based access control.)
-
-
 ### Install Fission
 
 #### Minikube
 
 ```
-$ helm install --namespace fission --set serviceType=NodePort https://github.com/fission/fission/releases/download/0.3.0/fission-all-0.3.0.tgz
+$ helm install --namespace fission --set serviceType=NodePort https://github.com/fission/fission/releases/download/0.4.0/fission-all-0.4.0.tgz
 ```
 
 The serviceType variable allows configuring the type of Kubernetes
@@ -94,7 +85,7 @@ want to expose anything outside the cluster.
 #### Cloud hosted clusters (GKE, AWS, Azure etc.)
 
 ```
-$ helm install --namespace fission https://github.com/fission/fission/releases/download/0.3.0/fission-all-0.3.0.tgz
+$ helm install --namespace fission https://github.com/fission/fission/releases/download/0.4.0/fission-all-0.4.0.tgz
 ```
 
 #### Minimal version
@@ -104,7 +95,7 @@ the NATS message queue, influxDB for logs, etc. If you want a more
 minimal setup, you can install the fission-core chart instead:
 
 ```
-$ helm install --namespace fission https://github.com/fission/fission/releases/download/0.3.0/fission-core-0.3.0.tgz
+$ helm install --namespace fission https://github.com/fission/fission/releases/download/0.4.0/fission-core-0.4.0.tgz
 ```
 
 ### Install the Fission CLI
@@ -114,19 +105,19 @@ $ helm install --namespace fission https://github.com/fission/fission/releases/d
 Get the CLI binary for Mac:
 
 ```
-$ curl -Lo fission https://github.com/fission/fission/releases/download/0.3.0/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
+$ curl -Lo fission https://github.com/fission/fission/releases/download/0.4.0/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
 ```
 
 #### Linux
 
 ```
-$ curl -Lo fission https://github.com/fission/fission/releases/download/0.3.0/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
+$ curl -Lo fission https://github.com/fission/fission/releases/download/0.4.0/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
 ```
 
 #### Windows
 
 For Windows, you can use the linux binary on WSL. Or you can download
-this windows executable: [fission.exe](https://github.com/fission/fission/releases/download/0.3.0/fission-cli-windows.exe)
+this windows executable: [fission.exe](https://github.com/fission/fission/releases/download/0.4.0/fission-cli-windows.exe)
 
 ### Set environment vars
 
@@ -149,6 +140,13 @@ FISSION_URL and FISSION_ROUTER, respectively.  Wait for services to
 get IP addresses (check this with ```kubectl --namespace fission get
 svc```).  Then:
 
+##### AWS
+```
+  $ export FISSION_URL=http://$(kubectl --namespace fission get svc controller -o=jsonpath='{..hostname}')
+  $ export FISSION_ROUTER=$(kubectl --namespace fission get svc router -o=jsonpath='{..hostname}')
+```
+
+##### GCP
 ```
   $ export FISSION_URL=http://$(kubectl --namespace fission get svc controller -o=jsonpath='{..ip}')
   $ export FISSION_ROUTER=$(kubectl --namespace fission get svc router -o=jsonpath='{..ip}')
@@ -159,7 +157,7 @@ svc```).  Then:
 Finally, you're ready to use Fission!
 
 ```
-$ fission env create --name nodejs --image fission/node-env:0.3.0
+$ fission env create --name nodejs --image fission/node-env:0.4.0
 
 $ curl -LO https://raw.githubusercontent.com/fission/fission/master/examples/nodejs/hello.js
 

Documentation/docs-site/content/upgrade-from-v0.3.md

@@ -19,9 +19,9 @@ the [upgrade guide from 0.1 to 0.2]() and then upgrade to 0.4.0.
 
 ## How to Upgrade
 
-1. Get the 0.4.0-rc CLI
+1. Get the 0.4.0 CLI
 2. Get the Fission state from v0.3 install
-3. Upgrade to Fission 0.4.0-rc
+3. Upgrade to Fission 0.4.0
 4. Upgrade Kubernetes cluster version to 1.7.x or higher
 5. Remove all TPR definition (for Kubernetes 1.7.x)
 6. Restore Fission state into CRDs
@@ -31,19 +31,19 @@ the [upgrade guide from 0.1 to 0.2]() and then upgrade to 0.4.0.
 #### OS X
 
 ```
-$ curl -Lo fission https://github.com/fission/fission/releases/download/0.4.0-rc/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
+$ curl -Lo fission https://github.com/fission/fission/releases/download/0.4.0/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
 ```
 
 #### Linux
 
 ```
-$ curl -Lo fission https://github.com/fission/fission/releases/download/0.4.0-rc/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
+$ curl -Lo fission https://github.com/fission/fission/releases/download/0.4.0/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
 ```
 
 #### Windows
 
 For Windows, you can use the linux binary on WSL. Or you can download
-this windows executable: [fission.exe](https://github.com/fission/fission/releases/download/0.4.0-rc/fission-cli-windows.exe)
+this windows executable: [fission.exe](https://github.com/fission/fission/releases/download/0.4.0/fission-cli-windows.exe)
 
 ### Get Fission state from v0.3 install
 
@@ -57,7 +57,7 @@ variable `$FISSION_URL` set to point at a v0.3 Fission server.
 This will create a JSON file with all your fission state in the
 current directory.
 
-### Upgrade to Fission 0.4.0-rc
+### Upgrade to Fission 0.4.0
 
 Upgrade fission with a command similar to this:
 
@@ -80,7 +80,7 @@ Though Kubernetes will migrate TPRs to CRDs automatically when TPR definition is
 fission tpr2crd delete
 ```
 
-### Restore your Fission state into Fission 0.4.0-rc
+### Restore your Fission state into Fission 0.4.0
 
 ```
 fission tpr2crd restore --file state.json

charts/fission-all/Chart.yaml

@@ -1,6 +1,6 @@
 apiVersion: v1
 name: fission-all
-version: 0.4.0rc
+version: 0.4.0
 description: Fission is a fast serverless framework for Kubernetes.
 keywords:
 - fission
@@ -12,4 +12,4 @@ maintainers:
   - name: Ta Ching Chen
     email: contact@tachingchen.com
 engine: gotpl
-appVersion: 0.4.0rc
+appVersion: 0.4.0

charts/fission-all/templates/NOTES.txt

@@ -1,13 +1,13 @@
 1. Install the client CLI.
 
 Mac:
-  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.4.0rc/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
+  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.4.0/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
 
 Linux:
-  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.4.0rc/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
+  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.4.0/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
 
 Windows:
-  For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/0.4.0rc/fission-cli-windows.exe
+  For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/0.4.0/fission-cli-windows.exe
 
 2. Set the FISSION_URL and FISSION_ROUTER environment variables.
 

charts/fission-all/templates/deployment.yaml

@@ -348,7 +348,7 @@ metadata:
     svc: influxdb
     chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
 spec:
-  type: ClusterIP
+  type: ClusterIP 
   ports:
   - port: 8086
     targetPort: 8086
@@ -400,25 +400,6 @@ spec:
         svc: logger
     spec:
       containers:
-        - name: logger
-          image: "{{ .Values.image }}:{{ .Values.imageTag }}"
-          imagePullPolicy: {{ .Values.pullPolicy }}
-          command: ["/fission-bundle"]
-          args: ["--logger"]
-          volumeMounts:
-            - name: container-log
-              mountPath: /var/log/containers
-              readOnly: true
-            - name: docker-log
-              mountPath: /var/lib/docker/containers
-              readOnly: true
-            - name: fission-log
-              mountPath: /var/log/fission
-              readOnly: false
-          ports:
-            - containerPort: 1234
-              hostPort: 1234
-              protocol: TCP
         - name: fluentd
           image: {{ .Values.logger.fluentdImage }}
           imagePullPolicy: {{ .Values.pullPolicy }}
@@ -439,9 +420,11 @@ spec:
                 secretKeyRef:
                   name: influxdb
                   key: password
+            - name: FLUENTD_PATH
+              value: /var/log/containers/*{{.Values.functionNamespace}}*.log
           volumeMounts:
             - name: container-log
-              mountPath: /var/log/containers
+              mountPath: /var/log/
               readOnly: true
             - name: docker-log
               mountPath: /var/lib/docker/containers
@@ -453,7 +436,7 @@ spec:
       volumes:
         - name: container-log
           hostPath:
-              path: /var/log/containers
+              path: /var/log/
         - name: docker-log
           hostPath:
               path: /var/lib/docker/containers

charts/fission-all/templates/svc.yaml

@@ -10,7 +10,9 @@ spec:
   ports:
   - port: 80
     targetPort: 8888
+{{ if eq .Values.serviceType "NodePort" }}
     nodePort: {{ .Values.routerPort }}
+{{ end }}
   selector:
     svc: router
 
@@ -27,7 +29,9 @@ spec:
   ports:
   - port: 80
     targetPort: 8888
+{{ if eq .Values.serviceType "NodePort" }}
     nodePort: {{ .Values.controllerPort }}
+{{ end }}
   selector:
     svc: controller
 
@@ -44,7 +48,9 @@ spec:
   ports:
   - port: 4222
     targetPort: 4222
+{{ if eq .Values.serviceType "NodePort" }}
     nodePort: {{ .Values.natsStreamingPort }}
+{{ end }}
   selector:
     svc: nats-streaming
 

charts/fission-all/values.yaml

@@ -10,16 +10,16 @@ serviceType: LoadBalancer
 image: fission/fission-bundle
 
 ## Image pull policy
-pullPolicy: IfNotPresent
+pullPolicy: IfNotPresent 
 
 ## Fission image version
-imageTag: 0.3.0
+imageTag: 0.4.0
 
 ## Fission fetcher repository
 fetcherImage: fission/fetcher
 
 ## Fission fetcher image version
-fetcherImageTag: 0.3.0
+fetcherImageTag: 0.4.0
 
 ## Port at which Fission controller service should be exposed
 controllerPort: 31313

charts/fission-core/Chart.yaml

@@ -1,6 +1,6 @@
 apiVersion: v1
 name: fission-core
-version: 0.4.0rc
+version: 0.4.0
 description: Fission is a fast serverless framework for Kubernetes.
 keywords:
 - fission
@@ -12,4 +12,4 @@ maintainers:
   - name: Ta Ching Chen
     email: contact@tachingchen.com
 engine: gotpl
-appVersion: 0.4.0rc
+appVersion: 0.4.0

charts/fission-core/templates/NOTES.txt

@@ -1,13 +1,13 @@
 1. Install the client CLI.
 
 Mac:
-  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.4.0rc/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
+  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.4.0/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
 
 Linux:
-  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.4.0rc/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
+  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.4.0/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
 
 Windows:
-  For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/0.4.0rc/fission-cli-windows.exe
+  For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/0.4.0/fission-cli-windows.exe
 
 2. Set the FISSION_URL and FISSION_ROUTER environment variables.
 

charts/fission-core/templates/svc.yaml

@@ -10,7 +10,9 @@ spec:
   ports:
   - port: 80
     targetPort: 8888
+{{ if eq .Values.serviceType "NodePort" }}
     nodePort: {{ .Values.routerPort }}
+{{ end }}
   selector:
     svc: router
 
@@ -27,7 +29,9 @@ spec:
   ports:
   - port: 80
     targetPort: 8888
+{{ if eq .Values.serviceType "NodePort" }}
     nodePort: {{ .Values.controllerPort }}
+{{ end }}
   selector:
     svc: controller
 

charts/fission-core/values.yaml

@@ -10,7 +10,7 @@ serviceType: LoadBalancer
 image: fission/fission-bundle
 
 ## Fission image version
-imageTag: 0.3.0
+imageTag: 0.4.0
 
 ## Image pull policy
 pullPolicy: IfNotPresent
@@ -19,7 +19,7 @@ pullPolicy: IfNotPresent
 fetcherImage: fission/fetcher
 
 ## Fission fetcher image version
-fetcherImageTag: 0.3.0
+fetcherImageTag: 0.4.0
 
 ## Port at which Fission controller service should be exposed
 controllerPort: 31313

controller/api.go

@@ -120,7 +120,7 @@ func (api *API) getLogDBConfig(dbType string) logDBConfig {
 
 func (api *API) HomeHandler(w http.ResponseWriter, r *http.Request) {
 	w.Header().Set("Content-Type", "application/json; charset=utf-8")
-	fmt.Fprintf(w, "{\"message\": \"Fission API\", \"version\": \"0.4.0rc\"}\n")
+	fmt.Fprintf(w, "{\"message\": \"Fission API\", \"version\": \"0.4.0\"}\n")
 }
 
 func (api *API) ApiVersionMismatchHandler(w http.ResponseWriter, r *http.Request) {

environments/python/Dockerfile-2.7

@@ -0,0 +1,14 @@
+FROM alpine:3.5
+
+RUN apk update
+RUN apk add --no-cache python python-dev build-base py-pip
+RUN pip install --upgrade pip
+RUN rm -r /root/.cache
+
+COPY . /app
+WORKDIR /app
+RUN pip install -r requirements.txt
+
+ENTRYPOINT ["python"]
+CMD ["server.py"]
+

examples/nodejs/README.md

@@ -82,7 +82,7 @@ This is a basic example of how you can easily use asynchronous requests in your
 $ fission function create --name stock --env nodejs --code stock.js
 
 # Map GET /stock to your new function
-$ fission route create --method GET --url /stock --function stock
+$ fission route create --method POST --url /stock --function stock
 
 # Run the function.
 $ curl -H "Content-Type: application/json" -X POST -d '{"symbol":"AAPL"}' http://$FISSION_ROUTER/stock
@@ -103,3 +103,22 @@ $ fission fn create --name kubeEventsSlack --env nodejs --code kubeEventsSlack.j
 # Watch all services in the default namespace:
 $ fission watch create --function kubeEventsSlack --type service --ns default
 ```
+
+## weather.js
+
+In this example, the Yahoo Weather API is used to current weather at a given location.
+
+### Usage
+
+```bash
+# Upload your function code to fission
+$ fission function create --name weather --env nodejs --code weather.js
+
+# Map GET /stock to your new function
+$ fission route create --method POST --url /weather --function weather
+
+# Run the function.
+$ curl -H "Content-Type: application/json" -X POST -d '{"location":"Sieteiglesias, Spain"}' http://$FISSION_ROUTER/weather
+
+{"text":"It is 2 celsius degrees in Sieteiglesias, Spain and Mostly Clear"}
+```

examples/nodejs/weather.js

@@ -0,0 +1,40 @@
+'use strict';
+
+const rp = require('request-promise-native');
+
+module.exports = async function (context) {
+    const stringBody = JSON.stringify(context.request.body);
+    const body = JSON.parse(stringBody);
+    const location = body.location;
+
+    if (!location) {
+        return {
+            status: 400,
+            body: {
+                text: 'You must provide a location.'
+            }
+        };
+    }
+
+    try {
+        const response = await rp(`https://query.yahooapis.com/v1/public/yql?q=select item.condition from weather.forecast where woeid in (select woeid from geo.places(1) where text="${location}") and u="c"&format=json`);
+        const condition = JSON.parse(response).query.results.channel.item.condition;
+        const text = condition.text;
+        const temperature = condition.temp;
+        return {
+            status: 200,
+            body: {
+                text: `It is ${temperature} celsius degrees in ${location} and ${text}`
+            },
+            headers: {
+                'Content-Type': 'application/json'
+            }
+        };
+    } catch (e) {
+        console.error(e);
+        return {
+            status: 500,
+            body: e
+        };
+    }
+}

examples/python/multifile/README.md

@@ -0,0 +1,62 @@
+This is an example of creating a deployment package with multiple
+files including some static data in text file.
+
+### Create an environment
+
+```
+fission env create --name python --image fission/python-env:0.4.0rc --version 2
+```
+
+### Create a zip file with all your files
+
+```
+zip -jr multifile.zip *.py *.txt
+```
+
+### Create a function
+
+Since there are multiple files, you have to specify an _entrypoint_ to
+for the function.  Its format is `<file name>.<function name>`. In our
+example, that's `main.main`, to run function `main` in `main.py`.
+
+```
+fission function create --name multifile --env python --code multifile.zip --entrypoint main.main
+```
+
+### Test it
+
+```
+fission function test --name multifile
+```
+
+You should see the "Hello, world" message.
+
+
+## Updating the function
+
+### Edit a file
+
+```
+echo "I said hellooooo!" > message.txt
+```
+
+### Update the deployment package
+
+```
+zip -jr multifile.zip *.py *.txt
+```
+
+### Update the function
+
+```
+fission function update --name multifile --code multifile.zip
+```
+
+### Test it
+
+```
+fission function test --name multifile
+```
+
+You should now see your new, edited message.
+

examples/python/multifile/main.py

@@ -0,0 +1,11 @@
+from flask import current_app
+import sys
+import readfile
+import os
+
+def main():
+    current_app.logger.info("Hi")
+
+    current_dir = os.path.dirname(__file__)
+
+    return readfile.readFile(os.path.join(current_dir, "message.txt"))

examples/python/multifile/message.txt

@@ -0,0 +1 @@
+Hello, world!

examples/python/multifile/readfile.py

@@ -0,0 +1,3 @@
+def readFile(name):
+    with open(name) as f:
+        return f.read()

examples/python/statuscode.py

@@ -0,0 +1,4 @@
+def main():
+  # You can return any http status code you like, simply place a comma after
+  # your return statement, and typing in the status code.
+  return "Not Found\n", 404

fission-bundle/main.go

@@ -9,7 +9,6 @@ import (
 	"github.com/fission/fission/buildermgr"
 	"github.com/fission/fission/controller"
 	"github.com/fission/fission/kubewatcher"
-	"github.com/fission/fission/logger"
 	"github.com/fission/fission/mqtrigger"
 	"github.com/fission/fission/poolmgr"
 	"github.com/fission/fission/router"
@@ -41,11 +40,6 @@ func runKubeWatcher(routerUrl string) {
 	}
 }
 
-func runLogger() {
-	logger.Start()
-	log.Fatalf("Error: Logger exited.")
-}
-
 func runTimer(routerUrl string) {
 	err := timer.Start(routerUrl)
 	if err != nil {
@@ -122,7 +116,6 @@ Usage:
   fission-bundle --kubewatcher [--routerUrl=<url>]
   fission-bundle --storageServicePort=<port> --filePath=<filePath>
   fission-bundle --builderMgrPort=<port> [--storageSvcUrl=<url>] [--envbuilder-namespace=<namespace>]
-  fission-bundle --logger
   fission-bundle --timer [--routerUrl=<url>]
   fission-bundle --mqt   [--routerUrl=<url>]
 Options:
@@ -138,7 +131,6 @@ Options:
   --filePath=<filePath>           Directory to store functions in.
   --namespace=<namespace>         Kubernetes namespace in which to run function containers. Defaults to 'fission-function'.
   --kubewatcher                   Start Kubernetes events watcher.
-  --logger                        Start logger.
   --timer                         Start Timer.
   --mqt                           Start message queue trigger.
 `
@@ -174,10 +166,6 @@ Options:
 		runKubeWatcher(routerUrl)
 	}
 
-	if arguments["--logger"] == true {
-		runLogger()
-	}
-
 	if arguments["--timer"] == true {
 		runTimer(routerUrl)
 	}

fission/logdb/influxdb.go

@@ -96,14 +96,15 @@ func (influx InfluxDB) GetLogs(filter LogFilter) ([]LogEntry, error) {
 					log.Fatal(err)
 				}
 				logEntries = append(logEntries, LogEntry{
+					//The attributes of the LogEntry are selected as relative to their position in InfluxDB's line protocol response
 					Timestamp: t,
-					Container: row[2].(string),
+					Container: row[2].(string),                            //docker_container_id
-					FuncName:  row[3].(string),
+					FuncName:  row[8].(string),                            //kubernetes_labels_functionName
-					FuncUid:   row[4].(string),
+					FuncUid:   row[3].(string),                            //funcuid
-					Message:   strings.TrimSuffix(row[5].(string), "\n"),
+					Message:   strings.TrimSuffix(row[17].(string), "\n"), //log field
-					Namespace: row[6].(string),
+					Namespace: row[14].(string),                           //kubernetes_namespace_name
-					Pod:       row[7].(string),
+					Pod:       row[15].(string),                           //kubernetes_pod_name
-					Stream:    row[8].(string),
+					Stream:    row[18].(string),                           //stream
 				})
 			}
 		}

fission/main.go

@@ -26,7 +26,7 @@ func main() {
 	app := cli.NewApp()
 	app.Name = "fission"
 	app.Usage = "Serverless functions for Kubernetes"
-	app.Version = "0.4.0rc"
+	app.Version = "0.4.0"
 
 	app.Flags = []cli.Flag{
 		cli.StringFlag{Name: "server", Usage: "Fission server URL", EnvVar: "FISSION_URL"},

hack/release.sh

@@ -128,27 +128,28 @@ build_and_push_all_envs() {
     version=$1
 
     # call with version, env dir, image name base, image name variant
-    build_and_push_env_image "$version" "nodejs" "node-env" ""
+    build_and_push_env_image "$version" "nodejs"   "node-env"     ""
-    build_and_push_env_image "$version" "nodejs" "node-env" "debian"
+    build_and_push_env_image "$version" "nodejs"   "node-env"     "debian"
-    build_and_push_env_image "$version" "binary" "binary-env" ""
+    build_and_push_env_image "$version" "binary"   "binary-env"   ""
-    build_and_push_env_image "$version" "dotnet" "dotnet-env" ""
+    build_and_push_env_image "$version" "dotnet"   "dotnet-env"   ""
     build_and_push_env_image "$version" "dotnet20" "dotnet20-env" ""    
-    build_and_push_env_image "$version" "go" "go-env" ""
+    build_and_push_env_image "$version" "go"       "go-env"       ""
-    build_and_push_env_image "$version" "perl" "perl-env" ""
+    build_and_push_env_image "$version" "perl"     "perl-env"     ""
-    build_and_push_env_image "$version" "php7" "php-env" ""
+    build_and_push_env_image "$version" "php7"     "php-env"      ""
-    build_and_push_env_image "$version" "python3" "python-env" ""
+    build_and_push_env_image "$version" "python"   "python-env"   ""
-    build_and_push_env_image "$version" "ruby" "ruby-env" ""
+    build_and_push_env_image "$version" "python"   "python-env"   "2.7"
+    build_and_push_env_image "$version" "ruby"     "ruby-env"     ""
 }
 
 build_charts() {
     version=$1
     mkdir -p $BUILDDIR/charts
     pushd $DIR/charts
-    for c in all core
+    find . -iname *.~?~ | xargs rm
+    for c in fission-all fission-core
     do
-	tgz=fission-$c-$version.tgz
+	helm package $c/
-	tar czvf $tgz fission-$c/
+	mv *.tgz $BUILDDIR/charts/
-	mv $tgz $BUILDDIR/charts/
     done
     popd
 }

logger/fluentd/Dockerfile

@@ -35,7 +35,7 @@ ENV DEBIAN_FRONTEND noninteractive
 
 # Install build tools
 RUN apt-get -qq update && \
-    apt-get install -y -qq curl ca-certificates gcc make bash sudo && \
+    apt-get install -y -qq curl ca-certificates gcc g++ make bash sudo && \
     apt-get install -y -qq --reinstall lsb-base lsb-release && \
     # Install logging agent and required gems
     /usr/bin/curl -sSL https://toolbelt.treasuredata.com/sh/install-ubuntu-xenial-td-agent2.sh | sh && \
@@ -45,6 +45,8 @@ RUN apt-get -qq update && \
     td-agent-gem install --no-document fluent-plugin-google-cloud -v 0.5.2 && \
     td-agent-gem install --no-document fluent-plugin-detect-exceptions -v 0.0.4 && \
     td-agent-gem install --no-document fluent-plugin-influxdb && \
+    td-agent-gem install --no-document fluent-plugin-kubernetes_metadata_filter && \
+    td-agent-gem install --no-document fluent-plugin-flatten-hash && \
     # Remove build tools
     apt-get remove -y -qq gcc make && \
     apt-get autoremove -y -qq && \

logger/fluentd/build.sh

@@ -1,2 +1,2 @@
 #!/bin/sh
-docker build -t fission-daemonset-fluentd:latest .
+docker build -t fission-daemonset-fluentd:latest . 

logger/fluentd/fluent.conf

@@ -4,26 +4,32 @@
 
 <source>
   type tail
-  format json
+  format json 
   time_key time
-  path /var/log/fission/*.log
+  path "#{ENV['FLUENTD_PATH']}"
   time_format %Y-%m-%dT%H:%M:%S.%NZ
   tag fission.*
   read_from_head true
   refresh_interval 5
 </source>
 
+<filter fission.**>
+   type kubernetes_metadata
+</filter>
+
+
+<filter fission.**>
+   type flatten_hash
+   separator _
+</filter>
+
 <match fission.**>
-  type record_reformer
+   type record_reformer
-  enable_ruby false
+   enable_ruby false
-  tag log
+   tag log
-  <record>
+   <record>
-    namespace ${tag_parts[4]}
+     funcuid ${kubernetes_labels_functionUid}
-    pod       ${tag_parts[5]}
+   </record>
-    container ${tag_parts[6]}
-    funcname  ${tag_parts[7]}
-    funcuid   ${tag_parts[8]}
-  </record>
 </match>
 
 <match **>
@@ -35,7 +41,7 @@
     password  "#{ENV['INFLUXDB_PASSWD']}"
     use_ssl false
     time_precision s
-    tag_keys ["funcuid", "pod"]
+    tag_keys ["funcuid"]
     sequence_tag _seq
     buffer_type file
     buffer_path /var/log/fission/fluentd.buffer
@@ -45,4 +51,4 @@
     retry_limit 10
     retry_wait 1.0
     num_threads 2
-</match>
+</match>

logger/logger.go

@@ -1,216 +0,0 @@
-/*
-Copyright 2016 The Fission Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package logger
-
-import (
-	"encoding/json"
-	"fmt"
-	"io/ioutil"
-	"net/http"
-	"os"
-	"path/filepath"
-	"strings"
-
-	"github.com/fission/fission"
-	"github.com/gorilla/handlers"
-	"github.com/gorilla/mux"
-	log "github.com/sirupsen/logrus"
-	"k8s.io/apimachinery/pkg/apis/meta/v1"
-	"k8s.io/client-go/kubernetes"
-	"k8s.io/client-go/rest"
-)
-
-func makelogRequestTracker() logRequestTracker {
-	return logRequestTracker{
-		logMap: make(map[string]LogRequest),
-	}
-}
-
-func (l logRequestTracker) Add(logReq LogRequest) {
-	l.Lock()
-	l.logMap[logReq.Pod] = logReq
-	l.Unlock()
-}
-
-func (l logRequestTracker) Get(pod string) LogRequest {
-	l.RLock()
-	logReq, ok := l.logMap[pod]
-	l.RUnlock()
-	if ok {
-		return logReq
-	}
-	return LogRequest{}
-}
-
-func (l logRequestTracker) Remove(logReq LogRequest) {
-	l.Lock()
-	delete(l.logMap, logReq.Pod)
-	l.Unlock()
-}
-
-// Get a kubernetes client using the pod's service account.  This only
-// works when we're running inside a kubernetes cluster.
-func getKubernetesClient() (*kubernetes.Clientset, error) {
-	// creates the in-cluster config
-	config, err := rest.InClusterConfig()
-	if err != nil {
-		log.Printf("Error getting kubernetes client config: %v", err)
-		return nil, err
-	}
-
-	// creates the clientset
-	clientset, err := kubernetes.NewForConfig(config)
-	if err != nil {
-		log.Printf("Error getting kubernetes client: %v", err)
-		return nil, err
-	}
-
-	return clientset, nil
-}
-
-// make sure that the targetPath is a legitimate path for security purpose
-func validateFilePath(targetPath string, expectedPathPrefix string) bool {
-	targetPath = filepath.Clean(targetPath)
-	return strings.HasPrefix(targetPath, expectedPathPrefix)
-}
-
-// The ContainerID is consist of container engine type (docker://) and uuid of container.
-// (e.g., docker://f4ca66baaa715030e20273aaf5232635a144165f1cd8e34ca5175064c245b679)
-// This function tries to extract container uuid from ContainerID.
-func parseContainerString(containerID string) (string, error) {
-	// Trim the quotes and split the type and ID.
-	parts := strings.Split(strings.Trim(containerID, "\""), "://")
-	if len(parts) != 2 {
-		return "", fmt.Errorf("invalid container ID: %q", containerID)
-	}
-	_, ID := parts[0], parts[1]
-	return ID, nil
-}
-
-func getcontainerID(kubeClient *kubernetes.Clientset, namespace, pod, container string) (string, error) {
-	podInfo, err := kubeClient.CoreV1().Pods(namespace).Get(pod, v1.GetOptions{})
-	if err != nil {
-		log.Printf("Failed to get pod info: %v", err)
-		return "", err
-	}
-	var containerID string
-	for _, c := range podInfo.Status.ContainerStatuses {
-		if c.Name == container {
-			containerID, err = parseContainerString(c.ContainerID)
-			if err != nil {
-				log.Printf("Failed to get container id: %v", err)
-				return "", err
-			}
-			return containerID, nil
-		}
-	}
-	return "", fission.MakeError(404, "no matching container is found")
-}
-
-func getContainerLogPath(logReq LogRequest) (string, bool) {
-	logPath := fmt.Sprintf("/var/lib/docker/containers/%s/%s-json.log", logReq.ContainerID, logReq.ContainerID)
-	if !validateFilePath(logPath, "/var/lib/docker/containers") {
-		return "", false
-	}
-	return logPath, true
-}
-
-func getFissionLogSymlinkPath(logReq LogRequest) (string, bool) {
-	// pass function related information through a symlink name
-	logSymLink := fmt.Sprintf("/var/log/fission/%s.%s.%s.%s.%s.log", logReq.Namespace, logReq.Pod, logReq.ContainerID, logReq.FuncName, logReq.FuncUid)
-	if !validateFilePath(logSymLink, "/var/log/fission") {
-		return "", false
-	}
-	return logSymLink, true
-}
-
-func createLogSymlink(w http.ResponseWriter, r *http.Request) {
-	body, err := ioutil.ReadAll(r.Body)
-	if err != nil {
-		http.Error(w, "Failed to read request", 500)
-		return
-	}
-	logReq := LogRequest{}
-	if err = json.Unmarshal(body, &logReq); err != nil {
-		w.Write([]byte(fmt.Sprintf("%v", err)))
-		return
-	}
-
-	kubernetesClient, err := getKubernetesClient()
-	if err != nil {
-		log.Warningf("Failed to get kubernetes client: %v", err)
-		w.WriteHeader(http.StatusInternalServerError)
-		return
-	}
-
-	containerID, err := getcontainerID(kubernetesClient, logReq.Namespace, logReq.Pod, logReq.Container)
-	if err != nil || containerID == "" {
-		log.Warningf("Failed to get container id: %v", err)
-		w.WriteHeader(http.StatusInternalServerError)
-		return
-	}
-
-	logReq.ContainerID = containerID
-	containerLogFilePath, isValidLogPath := getContainerLogPath(logReq)
-	fissionLogSymlinkPath, isValidSymlinkPath := getFissionLogSymlinkPath(logReq)
-	if !isValidLogPath || !isValidSymlinkPath {
-		w.WriteHeader(http.StatusBadRequest)
-		return
-	}
-
-	err = os.Symlink(containerLogFilePath, fissionLogSymlinkPath)
-	if err != nil {
-		w.WriteHeader(http.StatusInternalServerError)
-		return
-	}
-	logInfo.Add(logReq)
-	w.WriteHeader(http.StatusOK)
-}
-
-func removeLogSymlink(w http.ResponseWriter, r *http.Request) {
-	vars := mux.Vars(r)
-	pod := vars["pod"]
-	logReq := logInfo.Get(pod)
-	if logReq.Pod == "" {
-		w.WriteHeader(http.StatusInternalServerError)
-		return
-	}
-
-	fissionLogSymlinkPath, isValidSymlinkPath := getFissionLogSymlinkPath(logReq)
-	if !isValidSymlinkPath {
-		w.WriteHeader(http.StatusBadRequest)
-		return
-	}
-	err := os.Remove(fissionLogSymlinkPath)
-	if err != nil {
-		w.WriteHeader(http.StatusInternalServerError)
-		return
-	}
-	w.WriteHeader(http.StatusOK)
-}
-
-var logInfo logRequestTracker
-
-func Start() {
-	logInfo = makelogRequestTracker()
-	r := mux.NewRouter()
-	r.HandleFunc("/v1/log", createLogSymlink).Methods("POST")
-	r.HandleFunc("/v1/log/{pod}", removeLogSymlink).Methods("DELETE")
-	address := fmt.Sprintf(":%v", 1234)
-	log.Printf("starting logger at port %s", address)
-	log.Fatal(http.ListenAndServe(address, handlers.LoggingHandler(os.Stdout, r)))
-}

poolmgr/gp.go

@@ -43,7 +43,6 @@ import (
 	"github.com/fission/fission/crd"
 	"github.com/fission/fission/environments/fetcher"
 	fetcherClient "github.com/fission/fission/environments/fetcher/client"
-	"github.com/fission/fission/logger"
 )
 
 const POOLMGR_INSTANCEID_LABEL string = "poolmgrInstanceId"
@@ -348,9 +347,6 @@ func (gp *GenericPool) specializePod(pod *apiv1.Pod, metadata *metav1.ObjectMeta
 		return err
 	}
 
-	// Tell logging helper about this function invocation
-	gp.setupLogging(pod, metadata)
-
 	// get function run container to specialize
 	log.Printf("[%v] specializing pod", metadata.Name)
 
@@ -682,32 +678,3 @@ func (gp *GenericPool) destroy() error {
 
 	return nil
 }
-
-// Calls the logging daemonset pod on the node where the given pod is
-// running.
-func (gp *GenericPool) setupLogging(pod *apiv1.Pod, metadata *metav1.ObjectMeta) {
-	logReq := logger.LogRequest{
-		Namespace: pod.Namespace,
-		Pod:       pod.Name,
-		Container: gp.env.Metadata.Name,
-		FuncName:  metadata.Name,
-		FuncUid:   string(metadata.UID),
-	}
-	reqbody, err := json.Marshal(logReq)
-	if err != nil {
-		log.Printf("Error creating log request")
-		return
-	}
-	go func() {
-		loggerUrl := fmt.Sprintf("http://%s:1234/v1/log", pod.Status.HostIP)
-		resp, err := http.Post(loggerUrl, "application/json", bytes.NewReader(reqbody))
-		if err != nil {
-			log.Printf("Error connecting to %s log daemonset pod: %v", pod.Spec.NodeName, err)
-		} else {
-			if resp.StatusCode != 200 {
-				log.Printf("Error from %s log daemonset pod: %s", pod.Spec.NodeName, resp.Status)
-			}
-			resp.Body.Close()
-		}
-	}()
-}

router/functionReferenceResolver.go

@@ -36,8 +36,6 @@ type (
 	// functionReferenceResolver provides a resolver to turn a function
 	// reference into a resolveResult
 	functionReferenceResolver struct {
-		fissionClient *crd.FissionClient
-
 		// FunctionReference -> function metadata
 		refCache *cache.Cache
 
@@ -68,28 +66,14 @@ const (
 	resolveResultSingleFunction = iota
 )
 
-func makeFunctionReferenceResolver(fissionClient *crd.FissionClient) *functionReferenceResolver {
+func makeFunctionReferenceResolver(store k8sCache.Store) *functionReferenceResolver {
 	frr := &functionReferenceResolver{
-		fissionClient: fissionClient,
+		refCache: cache.MakeCache(time.Minute, 0),
-		refCache:      cache.MakeCache(time.Minute, 0),
+		store:    store,
 	}
 	return frr
 }
 
-// Sync starts syncing crd function resources from k8s api server
-func (frr *functionReferenceResolver) Sync(crdClient *rest.RESTClient) {
-	stopCh := make(chan struct{})
-	store, controller := makeK8SCache(crdClient)
-	frr.stopCh = stopCh
-	frr.store = store
-	go controller.Run(stopCh)
-}
-
-// Stop stops crd resources syncing
-func (frr *functionReferenceResolver) Stop() {
-	frr.stopCh <- struct{}{}
-}
-
 func makeK8SCache(crdClient *rest.RESTClient) (k8sCache.Store, k8sCache.Controller) {
 	watchlist := k8sCache.NewListWatchFromClient(crdClient, "functions", metav1.NamespaceDefault, fields.Everything())
 	listWatch := &k8sCache.ListWatch{

router/httpTriggers.go

@@ -17,6 +17,7 @@ limitations under the License.
 package router
 
 import (
+	"context"
 	"log"
 	"net/http"
 	"time"
@@ -24,8 +25,6 @@ import (
 	"github.com/gorilla/mux"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/fields"
-	"k8s.io/apimachinery/pkg/runtime"
-	"k8s.io/apimachinery/pkg/watch"
 	"k8s.io/client-go/rest"
 	k8sCache "k8s.io/client-go/tools/cache"
 
@@ -37,30 +36,42 @@ import (
 type HTTPTriggerSet struct {
 	*functionServiceMap
 	*mutableRouter
-	fissionClient *crd.FissionClient
+	fissionClient     *crd.FissionClient
-	poolmgr       *poolmgrClient.Client
+	poolmgr           *poolmgrClient.Client
-	resolver      *functionReferenceResolver
+	resolver          *functionReferenceResolver
-	triggers      []crd.HTTPTrigger
+	crdClient         *rest.RESTClient
-	triggerStore  k8sCache.Store
+	triggers          []crd.HTTPTrigger
-	functions     []crd.Function
+	triggerStore      k8sCache.Store
-	funcStore     k8sCache.Store
+	triggerController k8sCache.Controller
-	crdClient     *rest.RESTClient
+	functions         []crd.Function
+	funcStore         k8sCache.Store
+	funcController    k8sCache.Controller
 }
 
 func makeHTTPTriggerSet(fmap *functionServiceMap, fissionClient *crd.FissionClient,
-	poolmgr *poolmgrClient.Client, resolver *functionReferenceResolver, crdClient *rest.RESTClient) *HTTPTriggerSet {
+	poolmgr *poolmgrClient.Client, crdClient *rest.RESTClient) (*HTTPTriggerSet, k8sCache.Store, k8sCache.Store) {
-	triggers := make([]crd.HTTPTrigger, 1)
+	httpTriggerSet := &HTTPTriggerSet{
-	return &HTTPTriggerSet{
 		functionServiceMap: fmap,
-		triggers:           triggers,
+		triggers:           []crd.HTTPTrigger{},
 		fissionClient:      fissionClient,
 		poolmgr:            poolmgr,
-		resolver:           resolver,
 		crdClient:          crdClient,
 	}
+	var tStore, fnStore k8sCache.Store
+	var tController, fnController k8sCache.Controller
+	if httpTriggerSet.crdClient != nil {
+		tStore, tController = httpTriggerSet.initTriggerController()
+		httpTriggerSet.triggerStore = tStore
+		httpTriggerSet.triggerController = tController
+		fnStore, fnController = httpTriggerSet.initFunctionController()
+		httpTriggerSet.funcStore = fnStore
+		httpTriggerSet.funcController = fnController
+	}
+	return httpTriggerSet, tStore, fnStore
 }
 
-func (ts *HTTPTriggerSet) subscribeRouter(mr *mutableRouter) {
+func (ts *HTTPTriggerSet) subscribeRouter(ctx context.Context, mr *mutableRouter, resolver *functionReferenceResolver) {
+	ts.resolver = resolver
 	ts.mutableRouter = mr
 	mr.updateRouter(ts.getRouter())
 
@@ -69,8 +80,8 @@ func (ts *HTTPTriggerSet) subscribeRouter(mr *mutableRouter) {
 		log.Printf("Skipping continuous trigger updates")
 		return
 	}
-	go ts.watchTriggers()
+	go ts.runWatcher(ctx, ts.funcController)
-	go ts.watchFunctions()
+	go ts.runWatcher(ctx, ts.triggerController)
 }
 
 func defaultHomeHandler(w http.ResponseWriter, r *http.Request) {
@@ -140,17 +151,9 @@ func (ts *HTTPTriggerSet) updateTriggerStatusFailed(ht *crd.HTTPTrigger, err err
 	// TODO
 }
 
-func (ts *HTTPTriggerSet) watchTriggers() {
+func (ts *HTTPTriggerSet) initTriggerController() (k8sCache.Store, k8sCache.Controller) {
-	watchlist := k8sCache.NewListWatchFromClient(ts.crdClient, "httptriggers", metav1.NamespaceDefault, fields.Everything())
-	listWatch := &k8sCache.ListWatch{
-		ListFunc: func(options metav1.ListOptions) (runtime.Object, error) {
-			return watchlist.List(options)
-		},
-		WatchFunc: func(options metav1.ListOptions) (watch.Interface, error) {
-			return watchlist.Watch(options)
-		},
-	}
 	resyncPeriod := 30 * time.Second
+	listWatch := k8sCache.NewListWatchFromClient(ts.crdClient, "httptriggers", metav1.NamespaceDefault, fields.Everything())
 	store, controller := k8sCache.NewInformer(listWatch, &crd.HTTPTrigger{}, resyncPeriod,
 		k8sCache.ResourceEventHandlerFuncs{
 			AddFunc: func(obj interface{}) {
@@ -163,25 +166,12 @@ func (ts *HTTPTriggerSet) watchTriggers() {
 				ts.syncTriggers()
 			},
 		})
-	ts.triggerStore = store
+	return store, controller
-	stop := make(chan struct{})
-	defer func() {
-		stop <- struct{}{}
-	}()
-	controller.Run(stop)
 }
 
-func (ts *HTTPTriggerSet) watchFunctions() {
+func (ts *HTTPTriggerSet) initFunctionController() (k8sCache.Store, k8sCache.Controller) {
-	watchlist := k8sCache.NewListWatchFromClient(ts.crdClient, "functions", metav1.NamespaceDefault, fields.Everything())
-	listWatch := &k8sCache.ListWatch{
-		ListFunc: func(options metav1.ListOptions) (runtime.Object, error) {
-			return watchlist.List(options)
-		},
-		WatchFunc: func(options metav1.ListOptions) (watch.Interface, error) {
-			return watchlist.Watch(options)
-		},
-	}
 	resyncPeriod := 30 * time.Second
+	listWatch := k8sCache.NewListWatchFromClient(ts.crdClient, "functions", metav1.NamespaceDefault, fields.Everything())
 	store, controller := k8sCache.NewInformer(listWatch, &crd.Function{}, resyncPeriod,
 		k8sCache.ResourceEventHandlerFuncs{
 			AddFunc: func(obj interface{}) {
@@ -206,12 +196,13 @@ func (ts *HTTPTriggerSet) watchFunctions() {
 				ts.syncTriggers()
 			},
 		})
-	ts.funcStore = store
+	return store, controller
-	stop := make(chan struct{})
+}
-	defer func() {
+
-		stop <- struct{}{}
+func (ts *HTTPTriggerSet) runWatcher(ctx context.Context, controller k8sCache.Controller) {
+	go func() {
+		controller.Run(ctx.Done())
 	}()
-	controller.Run(stop)
 }
 
 func (ts *HTTPTriggerSet) syncTriggers() {

router/router.go

@@ -40,6 +40,7 @@ Its job is to:
 package router
 
 import (
+	"context"
 	"fmt"
 	"log"
 	"net/http"
@@ -57,15 +58,15 @@ import (
 
 // request url ---[trigger]---> Function(name, deployment) ----[deployment]----> Function(name, uid) ----[pool mgr]---> k8s service url
 
-func router(httpTriggerSet *HTTPTriggerSet) *mutableRouter {
+func router(ctx context.Context, httpTriggerSet *HTTPTriggerSet, resolver *functionReferenceResolver) *mutableRouter {
 	muxRouter := mux.NewRouter()
 	mr := NewMutableRouter(muxRouter)
-	httpTriggerSet.subscribeRouter(mr)
+	httpTriggerSet.subscribeRouter(ctx, mr, resolver)
 	return mr
 }
 
-func serve(port int, httpTriggerSet *HTTPTriggerSet) {
+func serve(ctx context.Context, port int, httpTriggerSet *HTTPTriggerSet, resolver *functionReferenceResolver) {
-	mr := router(httpTriggerSet)
+	mr := router(ctx, httpTriggerSet, resolver)
 	url := fmt.Sprintf(":%v", port)
 	http.ListenAndServe(url, handlers.LoggingHandler(os.Stdout, mr))
 }
@@ -79,12 +80,10 @@ func Start(port int, poolmgrUrl string) {
 	}
 	restClient := fissionClient.GetCrdClient()
 	poolmgr := poolmgrClient.MakeClient(poolmgrUrl)
-	resolver := makeFunctionReferenceResolver(fissionClient)
+	triggers, _, fnStore := makeHTTPTriggerSet(fmap, fissionClient, poolmgr, restClient)
-	resolver.Sync(restClient)
+	resolver := makeFunctionReferenceResolver(fnStore)
-	defer func() {
-		resolver.Stop()
-	}()
-	triggers := makeHTTPTriggerSet(fmap, fissionClient, poolmgr, resolver, restClient)
 	log.Printf("Starting router at port %v\n", port)
-	serve(port, triggers)
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+	serve(ctx, port, triggers, resolver)
 }

router/router_test.go

@@ -17,6 +17,7 @@ limitations under the License.
 package router
 
 import (
+	"context"
 	"fmt"
 	"testing"
 	"time"
@@ -58,7 +59,7 @@ func TestRouter(t *testing.T) {
 	frr.refCache.Set(nfr, rr)
 
 	// HTTP trigger set with a trigger for this function
-	triggers := makeHTTPTriggerSet(fmap, nil, nil, frr, nil)
+	triggers, _, _ := makeHTTPTriggerSet(fmap, nil, nil, nil)
 	triggerUrl := "/foo"
 	triggers.triggers = append(triggers.triggers,
 		crd.HTTPTrigger{
@@ -75,7 +76,9 @@ func TestRouter(t *testing.T) {
 
 	// run the router
 	port := 4242
-	go serve(port, triggers)
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+	go serve(ctx, port, triggers, frr)
 	time.Sleep(100 * time.Millisecond)
 
 	// hit the router

test/build_and_test.sh

@@ -13,6 +13,7 @@ source $(dirname $0)/test_utils.sh
 REPO=gcr.io/fission-ci
 IMAGE=$REPO/fission-bundle
 FETCHER_IMAGE=$REPO/fetcher
+FLUENTD_IMAGE=gcr.io/fission-ci/fluentd
 TAG=test
 
 build_and_push_fission_bundle $IMAGE:$TAG
@@ -21,12 +22,14 @@ build_and_push_fetcher $FETCHER_IMAGE:$TAG
 
 build_builder
 
-ENV='python3'
+ENV='python'
 
 build_and_push_env_runtime $ENV $REPO/$ENV-env:$TAG
 
 build_and_push_env_builder $ENV $REPO/$ENV-env-builder:$TAG
 
+build_and_push_fluentd $FLUENTD_IMAGE:$TAG
+
 build_fission_cli
 
-install_and_test $IMAGE $TAG $FETCHER_IMAGE $TAG
+install_and_test $IMAGE $TAG $FETCHER_IMAGE $TAG $FLUENTD_IMAGE $TAG

test/test_utils.sh

@@ -56,6 +56,19 @@ build_builder() {
     popd
 }
 
+build_and_push_fluentd(){
+    image_tag=$1
+
+    pushd $ROOT/logger/fluentd
+    docker build -t $image_tag .
+
+    gcloud_login
+
+    gcloud docker -- push $image_tag
+    popd
+
+}
+
 build_and_push_env_runtime() {
     env=$1
     image_tag=$2
@@ -110,13 +123,17 @@ helm_install_fission() {
     fetcherImageTag=$5
     controllerNodeport=$6
     routerNodeport=$7
+    fluentdImage=$8
 
     ns=f-$id
     fns=f-func-$id
 
-    helmVars=image=$image,imageTag=$imageTag,fetcherImage=$fetcherImage,fetcherImageTag=$fetcherImageTag,functionNamespace=$fns,controllerPort=$controllerNodeport,routerPort=$routerNodeport,pullPolicy=Always,analytics=false
+    helmVars=image=$image,imageTag=$imageTag,fetcherImage=$fetcherImage,fetcherImageTag=$fetcherImageTag,functionNamespace=$fns,controllerPort=$controllerNodeport,routerPort=$routerNodeport,pullPolicy=Always,analytics=false,logger.fluentdImage=$fluentdImage
 
     helm_setup
+
+    echo "Deleting failed releases"
+    helm list --failed -q|xargs -I@ bash -c "helm delete @"
     
     echo "Installing fission"
     helm install		\
@@ -127,6 +144,8 @@ helm_install_fission() {
 	 --namespace $ns        \
 	 --debug                \
 	 $ROOT/charts/fission-all
+    
+    helm list
 }
 
 wait_for_service() {
@@ -275,6 +294,8 @@ install_and_test() {
     imageTag=$2
     fetcherImage=$3
     fetcherImageTag=$4
+    fluentdImage=$5
+    fluentdImageTag=$6
 
     controllerPort=31234
     routerPort=31235
@@ -283,7 +304,7 @@ install_and_test() {
 
     id=$(generate_test_id)
     trap "helm_uninstall_fission $id" EXIT
-    if ! helm_install_fission $id $image $imageTag $fetcherImage $fetcherImageTag $controllerPort $routerPort
+    if ! helm_install_fission $id $image $imageTag $fetcherImage $fetcherImageTag $controllerPort $routerPort $fluentdImage:$fluentdImageTag
     then
 	dump_logs $id
 	exit 1

test/tests/test_logging/log.js

@@ -0,0 +1,8 @@
+
+module.exports = async function(context) {
+    console.log("log test log test log test")
+    return {
+        status: 200,
+        body: "Log, test!\n"
+    };
+}

test/tests/test_logging/test_function_logs.sh

@@ -0,0 +1,58 @@
+#!/bin/bash
+
+set -euo pipefail
+
+ROOT=$(dirname $0)/../..
+
+fn=nodejs-logtest
+
+
+function cleanup {
+    echo "Cleanup route"
+    var=$(fission route list | grep $fn | awk '{print $1;}')
+    fission route delete --name $var
+    fission function delete --name $fn
+}
+
+# Create a hello world function in nodejs, test it with an http trigger
+echo "Pre-test cleanup"
+fission env delete --name nodejs || true
+
+echo "Creating nodejs env"
+fission env create --name nodejs --image fission/node-env
+trap "fission env delete --name nodejs" EXIT
+
+echo "Creating function"
+fission fn create --name $fn --env nodejs --code log.js
+trap "fission fn delete --name $fn" EXIT
+
+echo "Creating route"
+fission route create --function $fn --url /logtest --method GET
+
+echo "Waiting for router to catch up"
+sleep 3
+
+echo "Doing 4 HTTP GET on the function's route"
+curl http://$FISSION_ROUTER/logtest
+curl http://$FISSION_ROUTER/logtest
+curl http://$FISSION_ROUTER/logtest
+curl http://$FISSION_ROUTER/logtest
+
+
+echo "Grabbing logs, should have 4 calls in logs"
+
+sleep 15
+
+echo "woke up"
+logs=$(fission function logs --name $fn)
+num=$(echo "$logs" | grep 'log test' | wc -l)
+echo $num
+
+if [ $num -ne 4 ]
+then
+    echo "Test Failed"
+    trap cleanup EXIT
+fi
+cleanup
+ 
+echo "All done."