Disable dotnet20 build for now (issue #359)

Add a flag to the environment to control multiple specialization -- the max number of functions per container. This can be set to 1 or infinity. 

Add an api proxy to workflow apiserver from the controller.

Add function metadata to FunctionLoadRequest; every v2 environment now knows which function it's loading (but can ignore that information if it wants to).

Add function identity headers to router. This is useful for multiple specialization, so the router can disambiguate between different function calls. (If this turns out to be a non-trivial perf overhead, we could add these headers conditionally, but for now they are always added.)

Documentation/docs-site/content/install.md

@@ -48,7 +48,7 @@ First, you'll need the helm CLI:
 
 On __OS X__:
 ```
-$ curl -L https://storage.googleapis.com/kubernetes-helm/helm-v2.6.1-darwin-amd64.tar.gz
+$ curl -LO https://storage.googleapis.com/kubernetes-helm/helm-v2.6.1-darwin-amd64.tar.gz
 
 $ tar xzf helm-v2.6.1-darwin-amd64.tar.gz
 
@@ -57,7 +57,7 @@ $ mv darwin-amd64/helm /usr/local/bin
 
 On __Linux__:
 ```
-$ curl -L https://storage.googleapis.com/kubernetes-helm/helm-v2.6.1-linux-amd64.tar.gz
+$ curl -LO https://storage.googleapis.com/kubernetes-helm/helm-v2.6.1-linux-amd64.tar.gz
 
 $ tar xzf helm-v2.6.1-linux-amd64.tar.gz
 

INSTALL.md

@@ -1,221 +1,2 @@
-
-- [Running Fission on your Cluster](#running-fission-on-your-cluster)
-  * [Setup Kubernetes](#setup-kubernetes)
-    + [Install and start Kubernetes on OSX:](#install-and-start-kubernetes-on-osx)
-    + [Or, install and start Kubernetes on Linux:](#or-install-and-start-kubernetes-on-linux)
-  * [Verify access to the cluster](#verify-access-to-the-cluster)
-  * [Get and Run Fission: Minikube or Local cluster](#get-and-run-fission-minikube-or-local-cluster)
-  * [Get and Run Fission: GKE or other Cloud](#get-and-run-fission-gke-or-other-cloud)
-  * [Get and Run Fission: OpenShift](#get-and-run-fission-openshift)
-    + [Using Minishift or Local Cluster](#using-minishift-or-local-cluster)
-    + [Using other clouds](#using-other-clouds)
-  * [Install the client CLI](#install-the-client-cli)
-  * [Run an example](#run-an-example)
-  * [Enable Persistent Function Logs (Optional)](#enable-persistent-function-logs-optional)
-  * [Use the web based Fission-ui (Optional)](#use-the-web-based-fission-ui-optional)
-
-## Running Fission on your Cluster
-
-### Setup Kubernetes
-
-You can install Kubernetes on your laptop with [minikube](https://github.com/kubernetes/minikube):
-
-#### Install and start Kubernetes on OSX:
-```bash
-  $ curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/darwin/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin
-  $ curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.16.0/minikube-darwin-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/
-  $ minikube start
-```
-
-#### Or, install and start Kubernetes on Linux:
-```bash
-  $ curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin
-  $ curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.16.0/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/
-  $ minikube start
-```
-
-Or, you can use [Google Container Engine's](https://cloud.google.com/container-engine/) free trial to get a 3 node cluster.
-
-### Verify access to the cluster
-
-```
-  $ kubectl version
-```
-
-### Get and Run Fission: Minikube or Local cluster
-
-If you're using minikube or no cloud provider, use these commands to
-set up services with NodePort.  This exposes fission on ports 31313
-and 31314.
-
-```
-  $ kubectl create -f https://github.com/fission/fission/releases/download/nightly20170705/fission-rbac.yaml
-  $ kubectl create -f https://github.com/fission/fission/releases/download/nightly20170705/fission-nodeport.yaml
-```
-
-Set the FISSION_URL and FISSION_ROUTER environment variables.
-FISSION_URL is used by the fission CLI to find the server.
-FISSION_URL should be prefixed with a `http://`.  (FISSION_ROUTER is
-only needed for the examples below to work.)
-
-If you're using minikube, use these commands:
-
-```
-  $ export FISSION_URL=http://$(minikube ip):31313
-  $ export FISSION_ROUTER=$(minikube ip):31314
-```
-
-
-### Get and Run Fission: GKE or other Cloud
-
-If you're using GKE or any other cloud provider that supports the
-LoadBalancer service type, use these commands:
-
-```
-  $ kubectl create -f https://github.com/fission/fission/releases/download/nightly20170705/fission-rbac.yaml
-  $ kubectl create -f https://github.com/fission/fission/releases/download/nightly20170705/fission-cloud.yaml
-```
-
-Save the external IP addresses of controller and router services in
-FISSION_URL and FISSION_ROUTER, respectively.  Wait for services to
-get IP addresses (check this with ```kubectl --namespace fission get
-svc```).  Then:
-
-```
-  $ export FISSION_URL=http://$(kubectl --namespace fission get svc controller -o=jsonpath='{..ip}')
-  $ export FISSION_ROUTER=$(kubectl --namespace fission get svc router -o=jsonpath='{..ip}')
-```
-
-### Get and Run Fission: OpenShift
-
-If you're using OpenShift, it's possible to run Fission on it! The
-deployment template needs to be deployed as a user with cluster-admin
-permissions (like `system:admin`), as it needs to create a
-`ClusterRole` for deploying function containers from the `fission`
-namespace/project.
-
-Identically as with Kubernetes, you need to set the FISSION_URL and FISSION_ROUTER environment variables. If you're using minishift, use these commands:
-
-```
-  $ export FISSION_URL=http://$(minishift ip):31313¬
-  $ export FISSION_ROUTER=$(minishift ip):31314¬
-```
-#### Using Minishift or Local Cluster
-
-If you're using minishift or no cloud provider, use these commands to set up services with NodePort. This exposes fission on ports 31313 and 31314.
-
-```
-  $ oc login -u system:admin
-  $ oc create -f https://github.com/fission/fission/releases/download/nightly20170705/fission-openshift.yaml
-  $ oc create -f https://github.com/fission/fission/releases/download/nightly20170705/fission-nodeport.yaml
-```
-
-#### Using other clouds
-
-If you're using any cloud provider that supports the LoadBalancer service type, use these commands:
-
-```
-$ oc login -u system:admin
-$ oc create -f https://github.com/fission/fission/releases/download/nightly20170705/fission-openshift.yaml
-$ oc create -f https://github.com/fission/fission/releases/download/nightly20170705/fission-cloud.yaml
-```
-After these steps, you should be able to run fission client as with kubernetes.
-
-### Install the client CLI
-
-#### Mac OS
-
-Get the CLI binary for Mac:
-
-```
-  $ curl -Lo fission https://github.com/fission/fission/releases/download/nightly20170705/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
-```
-
-#### Linux
-
-```
-  $ curl -Lo fission https://github.com/fission/fission/releases/download/nightly20170705/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
-```
-
-#### Windows
-
-For Windows, you can use the linux binary on WSL. Or you can download
-this windows executable: [fission.exe](https://github.com/fission/fission/releases/download/nightly20170705/fission-cli-windows.exe)
-
-### Run an example
-
-Finally, you're ready to use Fission!
-
-```
-  $ fission env create --name nodejs --image fission/node-env
-
-  $ curl https://raw.githubusercontent.com/fission/fission/master/examples/nodejs/hello.js > hello.js
-
-  $ fission function create --name hello --env nodejs --code hello.js
-  
-  $ fission route create --method GET --url /hello --function hello
-  
-  $ curl http://$FISSION_ROUTER/hello
-  Hello, world!
-```
-
-
-### Enable Persistent Function Logs (Optional)
-
-Fission uses InfluxDB to store logs and fluentd to forward them from
-function pods into InfluxDB.  
-
-```
-  $ kubectl create -f https://github.com/fission/fission/releases/download/nightly20170705/fission-logger.yaml
-```
-
-That's it for the basic setup.  You can now use following command to view function logs:
-
-```
-  $ fission function logs --name hello
-```
-
-You can also list the all the pods that have hosted the function
-(including ones that aren't alive any more) and view logs for a
-particular pod:
-
-```
-  $ fission function pods --name hello
-
-  $ fission function logs --name hello --pod <pod name>
-```
-
-### Use the web based Fission-ui (Optional)
-
-[Fission-ui](https://github.com/fission/fission-ui) is the ui for fission maintained by the community.
-It allows users to observe and manage fission. It also provides a simple online development environment for serverless functions.
-
-To setup Fission-ui with fission in k8s is simple:
-
-```bash
-  # Run this after fission is deployed
-  $ kubectl create -f https://raw.githubusercontent.com/fission/fission-ui/master/docker/fission-ui.yaml
-```
-
-Then open `http://node-ip:31319` to use Fission-ui.
-
-For more infomation, please check out [Fission-ui Readme](https://github.com/fission/fission-ui/blob/master/README.md).
-
-### Install NATS for message-queue based triggers (Optional)
-
-Fission supports message queue triggers that allow you to invoke
-functions based on events in a queue.  For now, NATS-Streaming is the
-only supported message queue.
-
-You can install NATS Streaming on your Kubernetes cluster with:
-
-```
-  $ kubectl create -f https://github.com/fission/fission/releases/download/nightly20170705/fission-nats.yaml
-```
-
-You can subscribe to a NATS Streaming queue with a command like this:
-(See `fission mqtrigger --help` for details)
-
-```
-  $ fission mqtrigger create --name myQueueTrigger --function processEvent --topic "myQueue.request" 
-```
+These instructions are replaced by:
+http://fission.io/docs/v0.2.1/install/

README.md

@@ -81,7 +81,7 @@ See the [examples](examples) directory for more.
 Running Fission on your Cluster
 ===============================
 
-See the [installation guide](INSTALL.md).
+See the [installation guide](http://fission.io/docs/v0.2.1/install/).
 
 Compiling Fission
 =================

builder/builder.go

@@ -98,7 +98,12 @@ func (builder *Builder) Handler(w http.ResponseWriter, r *http.Request) {
 	srcPkgPath := filepath.Join(builder.sharedVolumePath, req.SrcPkgFilename)
 	deployPkgFilename := fmt.Sprintf("%v-%v", req.SrcPkgFilename, strings.ToLower(uniuri.NewLen(6)))
 	deployPkgPath := filepath.Join(builder.sharedVolumePath, deployPkgFilename)
-	buildLogs, err := builder.build(req.BuildCommand, srcPkgPath, deployPkgPath)
+	buildCmd := req.BuildCommand
+	if len(buildCmd) == 0 {
+		// use default build command
+		buildCmd = "/build"
+	}
+	buildLogs, err := builder.build(buildCmd, srcPkgPath, deployPkgPath)
 	if err != nil {
 		e := errors.New(fmt.Sprintf("Error building source package: %v", err))
 		http.Error(w, e.Error(), 500)

builder/client/client.go

@@ -19,6 +19,7 @@ package client
 import (
 	"bytes"
 	"encoding/json"
+	"io/ioutil"
 	"net/http"
 	"strings"
 
@@ -32,25 +33,37 @@ type (
 	}
 )
 
-func MakeClient(serverUrl string) *Client {
+func MakeClient(builderUrl string) *Client {
 	return &Client{
-		url: strings.TrimSuffix(serverUrl, "/"),
+		url: strings.TrimSuffix(builderUrl, "/"),
 	}
 }
 
-func (c *Client) Build(req *builder.PackageBuildRequest) error {
+func (c *Client) Build(req *builder.PackageBuildRequest) (*builder.PackageBuildResponse, error) {
 	body, err := json.Marshal(req)
 	if err != nil {
-		return err
+		return nil, err
 	}
 	resp, err := http.Post(c.url, "application/json", bytes.NewReader(body))
 	if err != nil {
-		return err
+		return nil, err
 	}
 	defer resp.Body.Close()
 
 	if resp.StatusCode != 200 {
-		return fission.MakeErrorFromHTTP(resp)
+		return nil, fission.MakeErrorFromHTTP(resp)
 	}
-	return nil
+
+	rBody, err := ioutil.ReadAll(resp.Body)
+	if err != nil {
+		return nil, err
+	}
+
+	pkgBuildResp := builder.PackageBuildResponse{}
+	err = json.Unmarshal([]byte(rBody), &pkgBuildResp)
+	if err != nil {
+		return nil, err
+	}
+
+	return &pkgBuildResp, nil
 }

builder/cmd/Dockerfile

@@ -1,5 +0,0 @@
-FROM alpine:3.4
-
-ADD builder /
-
-EXPOSE 8000

builder/cmd/build.sh

@@ -1,2 +1,3 @@
-#!/bin/sh
-GOOS=linux GOARCH=386 go build -o builder .
+#!/bin/bash
+CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o builder .
+

builder/cmd/main.go

@@ -38,5 +38,5 @@ func main() {
 	builder := builder.MakeBuilder(dir)
 	mux := http.NewServeMux()
 	mux.HandleFunc("/", builder.Handler)
-	http.ListenAndServe(":8000", mux)
+	http.ListenAndServe(":8001", mux)
 }

buildermgr/api.go

@@ -0,0 +1,108 @@
+/*
+Copyright 2017 The Fission Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package buildermgr
+
+import (
+	"encoding/json"
+	"fmt"
+	"io/ioutil"
+	"log"
+	"net/http"
+	"os"
+
+	"github.com/gorilla/handlers"
+	"github.com/gorilla/mux"
+	"k8s.io/client-go/1.5/kubernetes"
+	"k8s.io/client-go/1.5/pkg/api"
+
+	"github.com/fission/fission"
+	"github.com/fission/fission/tpr"
+)
+
+type (
+	BuildRequest struct {
+		Package api.ObjectMeta `json:"package"`
+	}
+
+	BuilderMgr struct {
+		fissionClient    *tpr.FissionClient
+		kubernetesClient *kubernetes.Clientset
+		storageSvcUrl    string
+		namespace        string
+	}
+)
+
+func MakeBuilderMgr(fissionClient *tpr.FissionClient,
+	kubernetesClient *kubernetes.Clientset, storageSvcUrl string,
+	envBuilderNamespace string) *BuilderMgr {
+
+	envWatcher := makeEnvironmentWatcher(fissionClient, kubernetesClient, envBuilderNamespace)
+	go envWatcher.watchEnvironments()
+
+	pkgWatcher := makePackageWatcher(fissionClient, kubernetesClient, envBuilderNamespace, storageSvcUrl)
+	go pkgWatcher.watchPackages()
+
+	return &BuilderMgr{
+		fissionClient:    fissionClient,
+		kubernetesClient: kubernetesClient,
+		storageSvcUrl:    storageSvcUrl,
+		namespace:        envBuilderNamespace,
+	}
+}
+
+func (builderMgr *BuilderMgr) build(w http.ResponseWriter, r *http.Request) {
+	body, err := ioutil.ReadAll(r.Body)
+	if err != nil {
+		e := fmt.Sprintf("Failed to read request: %v", err)
+		log.Println(e)
+		http.Error(w, e, 500)
+		return
+	}
+
+	buildReq := BuildRequest{}
+	err = json.Unmarshal([]byte(body), &buildReq)
+	if err != nil {
+		e := fmt.Sprintf("invalid request body: %v", err)
+		log.Println(e)
+		http.Error(w, e, 400)
+		return
+	}
+
+	buildLogs, err := buildPackage(builderMgr.fissionClient, builderMgr.kubernetesClient,
+		builderMgr.namespace, builderMgr.storageSvcUrl, buildReq)
+	if err != nil {
+		code, e := fission.GetHTTPError(err)
+		http.Error(w, e, code)
+		return
+	}
+
+	w.Header().Set("Content-Type", "text/plain")
+	_, err = w.Write([]byte(buildLogs))
+	if err != nil {
+		e := fmt.Sprintf("Failed to reply http request: %v", err)
+		log.Println(e)
+		http.Error(w, e, 500)
+	}
+}
+
+func (builderMgr *BuilderMgr) Serve(port int) {
+	r := mux.NewRouter()
+	r.HandleFunc("/v1/build", builderMgr.build).Methods("POST")
+	address := fmt.Sprintf(":%v", port)
+	log.Printf("Start buildermgr at port %v", address)
+	log.Fatal(http.ListenAndServe(address, handlers.LoggingHandler(os.Stdout, r)))
+}

buildermgr/buildermgr.go

@@ -0,0 +1,39 @@
+/*
+Copyright 2016 The Fission Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package buildermgr
+
+import (
+	"log"
+
+	"github.com/fission/fission/tpr"
+)
+
+// Start the buildermgr service.
+func Start(port int, storageSvcUrl string, envBuilderNamespace string) error {
+	fissionClient, kubernetesClient, err := tpr.MakeFissionClient()
+	if err != nil {
+		log.Printf("Failed to get kubernetes client: %v", err)
+		return err
+	}
+
+	api := MakeBuilderMgr(fissionClient, kubernetesClient,
+		storageSvcUrl, envBuilderNamespace)
+
+	go api.Serve(port)
+
+	return nil
+}

buildermgr/client/client.go

@@ -0,0 +1,61 @@
+/*
+Copyright 2017 The Fission Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package client
+
+import (
+	"bytes"
+	"encoding/json"
+	"io/ioutil"
+	"net/http"
+	"strings"
+
+	"github.com/fission/fission"
+	"github.com/fission/fission/buildermgr"
+)
+
+type (
+	Client struct {
+		url string
+	}
+)
+
+func MakeClient(builderUrl string) *Client {
+	return &Client{
+		url: strings.TrimSuffix(builderUrl, "/") + "/v1",
+	}
+}
+
+func (c *Client) PackageBuild(req *buildermgr.BuildRequest) ([]byte, error) {
+	body, err := json.Marshal(req)
+	if err != nil {
+		return nil, err
+	}
+	resp, err := http.Post(c.url+"/build", "application/json", bytes.NewReader(body))
+	if err != nil {
+		return nil, err
+	}
+	return c.handleResponse(resp)
+}
+
+func (c *Client) handleResponse(resp *http.Response) ([]byte, error) {
+	if resp.StatusCode != 200 {
+		return nil, fission.MakeErrorFromHTTP(resp)
+	}
+	defer resp.Body.Close()
+	body, err := ioutil.ReadAll(resp.Body)
+	return body, err
+}

buildermgr/common.go

@@ -0,0 +1,211 @@
+/*
+Copyright 2017 The Fission Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package buildermgr
+
+import (
+	"fmt"
+	"log"
+	"strings"
+
+	"k8s.io/client-go/1.5/kubernetes"
+	"k8s.io/client-go/1.5/pkg/api"
+
+	"github.com/dchest/uniuri"
+	"github.com/fission/fission"
+	"github.com/fission/fission/builder"
+	builderClient "github.com/fission/fission/builder/client"
+	"github.com/fission/fission/environments/fetcher"
+	fetcherClient "github.com/fission/fission/environments/fetcher/client"
+	"github.com/fission/fission/tpr"
+)
+
+// buildPackage helps to build source package into deployment package.
+// Following is steps buildPackage takes to complete the build process.
+// 1. Check package status
+// 2. Update package status to running state
+// 3. Send fetch request to fetcher to fetch source package.
+// 4. Send build request to builder to start a build.
+// 5. Send upload request to fetcher to upload deployment package.
+// 6. Update package status to succeed state
+// 7. Update package resource in package ref of functions that share the same package
+// *. Update package status to failed state,if any one of steps above failed
+func buildPackage(fissionClient *tpr.FissionClient, kubernetesClient *kubernetes.Clientset,
+	builderNamespace string, storageSvcUrl string, buildReq BuildRequest) (buildLogs string, err error) {
+
+	pkg, err := fissionClient.Packages(
+		buildReq.Package.Namespace).Get(buildReq.Package.Name)
+	if err != nil {
+		e := fmt.Sprintf("Error getting function TPR info: %v", err)
+		log.Println(e)
+		updatePackage(fissionClient, pkg, fission.BuildStatusFailed, e, nil)
+		return e, fission.MakeError(500, e)
+	}
+
+	// Only do build for pending packages
+	if pkg.Status.BuildStatus != fission.BuildStatusPending {
+		e := "package is not in pending state"
+		log.Println(e)
+		return e, fission.MakeError(400, e)
+	}
+
+	// update package status to running state, so that
+	// we can know what status a package is through cli.
+	_, err = updatePackage(fissionClient, pkg, fission.BuildStatusRunning, "", nil)
+	if err != nil {
+		e := fmt.Sprintf("Error setting package pending state: %v", err)
+		log.Println(e)
+		updatePackage(fissionClient, pkg, fission.BuildStatusFailed, e, nil)
+		return e, fission.MakeError(500, e)
+	}
+
+	env, err := fissionClient.Environments(api.NamespaceDefault).Get(pkg.Spec.Environment.Name)
+	if err != nil {
+		e := fmt.Sprintf("Error getting environment TPR info: %v", err)
+		log.Println(e)
+		updatePackage(fissionClient, pkg, fission.BuildStatusFailed, e, nil)
+		return e, fission.MakeError(500, e)
+	}
+
+	svcName := fmt.Sprintf("%v-%v.%v", env.Metadata.Name, env.Metadata.ResourceVersion, builderNamespace)
+	srcPkgFilename := fmt.Sprintf("%v-%v", pkg.Metadata.Name, strings.ToLower(uniuri.NewLen(6)))
+	fetcherC := fetcherClient.MakeClient(fmt.Sprintf("http://%v:8000", svcName))
+	builderC := builderClient.MakeClient(fmt.Sprintf("http://%v:8001", svcName))
+
+	fetchReq := &fetcher.FetchRequest{
+		FetchType: fetcher.FETCH_SOURCE,
+		Package:   pkg.Metadata,
+		Filename:  srcPkgFilename,
+	}
+
+	// send fetch request to fetcher
+	err = fetcherC.Fetch(fetchReq)
+	if err != nil {
+		e := fmt.Sprintf("Error fetching source package: %v", err)
+		log.Println(e)
+		updatePackage(fissionClient, pkg, fission.BuildStatusFailed, e, nil)
+		return e, fission.MakeError(500, e)
+	}
+
+	pkgBuildReq := &builder.PackageBuildRequest{
+		SrcPkgFilename: srcPkgFilename,
+		BuildCommand:   pkg.Spec.BuildCommand,
+	}
+
+	log.Printf("Start building with source package: %v", srcPkgFilename)
+	// send build request to builder
+	buildResp, err := builderC.Build(pkgBuildReq)
+	if err != nil {
+		e := fmt.Sprintf("Error building deployment package: %v", err)
+		log.Println(e)
+		updatePackage(fissionClient, pkg, fission.BuildStatusFailed, e, nil)
+		return e, fission.MakeError(500, e)
+	}
+
+	log.Printf("Build succeed, source package: %v, deployment package: %v", srcPkgFilename, buildResp.ArtifactFilename)
+
+	uploadReq := &fetcher.UploadRequest{
+		Filename:      buildResp.ArtifactFilename,
+		StorageSvcUrl: storageSvcUrl,
+	}
+
+	log.Printf("Start uploading deployment package: %v", buildResp.ArtifactFilename)
+	// ask fetcher to upload the deployment package
+	uploadResp, err := fetcherC.Upload(uploadReq)
+	if err != nil {
+		e := fmt.Sprintf("Error uploading deployment package: %v", err)
+		log.Println(e)
+		updatePackage(fissionClient, pkg, fission.BuildStatusFailed, e, nil)
+		return e, fission.MakeError(500, e)
+	}
+
+	log.Printf("Start updating info of package: %v", pkg.Metadata.Name)
+	// update package status and also build logs
+	newPkgRV, err := updatePackage(fissionClient, pkg,
+		fission.BuildStatusSucceeded, buildResp.BuildLogs, uploadResp)
+	if err != nil {
+		e := fmt.Sprintf("Error creating deployment package TPR resource: %v", err)
+		log.Println(e)
+		updatePackage(fissionClient, pkg, fission.BuildStatusFailed, e, nil)
+		return e, fission.MakeError(500, e)
+	}
+
+	fnList, err := fissionClient.
+		Functions(api.NamespaceDefault).List(api.ListOptions{})
+	if err != nil {
+		e := fmt.Sprintf("Error getting function list: %v", err)
+		log.Println(e)
+		updatePackage(fissionClient, pkg, fission.BuildStatusFailed, e, nil)
+		return e, fission.MakeError(500, e)
+	}
+
+	// A package may be used by multiple functions. Update
+	// functions with old package resource version
+	for _, fn := range fnList.Items {
+		if fn.Spec.Package.PackageRef.Name == pkg.Metadata.Name &&
+			fn.Spec.Package.PackageRef.Namespace == pkg.Metadata.Namespace &&
+			fn.Spec.Package.PackageRef.ResourceVersion != pkg.Metadata.ResourceVersion {
+			fn.Spec.Package.PackageRef.ResourceVersion = newPkgRV
+			// update TPR
+			_, err = fissionClient.Functions(fn.Metadata.Namespace).Update(&fn)
+			if err != nil {
+				e := fmt.Sprintf("Error updating function package resource version: %v", err)
+				log.Println(e)
+				updatePackage(fissionClient, pkg, fission.BuildStatusFailed, e, nil)
+				return e, fission.MakeError(500, e)
+			}
+		}
+	}
+
+	log.Printf("Completed build request for package: %v", pkg.Metadata.Name)
+
+	return buildResp.BuildLogs, nil
+}
+
+func updatePackage(fissionClient *tpr.FissionClient,
+	pkg *tpr.Package, status fission.BuildStatus, buildLogs string,
+	uploadResp *fetcher.UploadResponse) (string, error) {
+
+	// Kubernetes checks resource version before applying
+	// new resource config. The update operation will be
+	// rejected if the resource version in metadata is lower
+	// than the latest version. Set resource version empty
+	// string to skip resource version check.
+	pkg.Metadata.ResourceVersion = ""
+
+	pkg.Status = fission.PackageStatus{
+		BuildStatus: status,
+		BuildLog:    buildLogs,
+	}
+
+	if uploadResp != nil {
+		pkg.Spec.Deployment = fission.Archive{
+			Type:     fission.ArchiveTypeUrl,
+			URL:      uploadResp.ArchiveDownloadUrl,
+			Checksum: uploadResp.Checksum,
+		}
+	}
+
+	// update package spec
+	pkg, err := fissionClient.Packages(api.NamespaceDefault).Update(pkg)
+	if err != nil {
+		log.Printf("Error updating package: %v", err)
+		return "", err
+	}
+
+	// return resource version for function to update function package ref
+	return pkg.Metadata.ResourceVersion, nil
+}

buildermgr/envwatcher.go

@@ -0,0 +1,477 @@
+/*
+Copyright 2017 The Fission Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package buildermgr
+
+import (
+	"fmt"
+	"log"
+	"os"
+	"time"
+
+	"k8s.io/client-go/1.5/kubernetes"
+	"k8s.io/client-go/1.5/pkg/api"
+	"k8s.io/client-go/1.5/pkg/api/v1"
+	"k8s.io/client-go/1.5/pkg/apis/extensions/v1beta1"
+	"k8s.io/client-go/1.5/pkg/labels"
+	"k8s.io/client-go/1.5/pkg/util/intstr"
+	"k8s.io/client-go/1.5/pkg/watch"
+
+	"github.com/fission/fission/tpr"
+)
+
+type requestType int
+
+const (
+	GET_BUILDER requestType = iota
+	CLEANUP_BUILDERS
+
+	LABEL_ENV_NAME            = "envName"
+	LABEL_ENV_RESOURCEVERSION = "envResourceVersion"
+)
+
+type (
+	builderInfo struct {
+		envMetadata *api.ObjectMeta
+		deployment  *v1beta1.Deployment
+		service     *v1.Service
+	}
+
+	envwRequest struct {
+		requestType
+		env      *tpr.Environment
+		envList  []tpr.Environment
+		respChan chan envwResponse
+	}
+
+	envwResponse struct {
+		builderInfo *builderInfo
+		err         error
+	}
+
+	environmentWatcher struct {
+		cache                  map[string]*builderInfo
+		requestChan            chan envwRequest
+		builderNamespace       string
+		fissionClient          *tpr.FissionClient
+		kubernetesClient       *kubernetes.Clientset
+		fetcherImage           string
+		fetcherImagePullPolicy v1.PullPolicy
+	}
+)
+
+func makeEnvironmentWatcher(fissionClient *tpr.FissionClient,
+	kubernetesClient *kubernetes.Clientset, builderNamespace string) *environmentWatcher {
+
+	fetcherImage := os.Getenv("FETCHER_IMAGE")
+	if len(fetcherImage) == 0 {
+		fetcherImage = "fission/fetcher"
+	}
+
+	fetcherImagePullPolicy := os.Getenv("FETCHER_IMAGE_PULL_POLICY")
+	if len(fetcherImagePullPolicy) == 0 {
+		fetcherImagePullPolicy = "IfNotPresent"
+	}
+
+	var pullPolicy v1.PullPolicy
+	switch fetcherImagePullPolicy {
+	case "Always":
+		pullPolicy = v1.PullAlways
+	case "Never":
+		pullPolicy = v1.PullNever
+	default:
+		pullPolicy = v1.PullIfNotPresent
+	}
+
+	envWatcher := &environmentWatcher{
+		cache:                  make(map[string]*builderInfo),
+		requestChan:            make(chan envwRequest),
+		builderNamespace:       builderNamespace,
+		fissionClient:          fissionClient,
+		kubernetesClient:       kubernetesClient,
+		fetcherImage:           fetcherImage,
+		fetcherImagePullPolicy: pullPolicy,
+	}
+
+	go envWatcher.service()
+
+	return envWatcher
+}
+
+func (envw *environmentWatcher) getCacheKey(envName string, envResourceVersion string) string {
+	return fmt.Sprintf("%v-%v", envName, envResourceVersion)
+}
+
+func (envw *environmentWatcher) getLabels(envName string, envResourceVersion string) map[string]string {
+	return map[string]string{
+		LABEL_ENV_NAME:            envName,
+		LABEL_ENV_RESOURCEVERSION: envResourceVersion,
+	}
+}
+
+func (envw *environmentWatcher) watchEnvironments() {
+	rv := ""
+	for {
+		wi, err := envw.fissionClient.Environments(api.NamespaceAll).Watch(api.ListOptions{
+			ResourceVersion: rv,
+		})
+		if err != nil {
+			log.Fatalf("Error watching environment list: %v", err)
+		}
+
+		for {
+			ev, more := <-wi.ResultChan()
+			if !more {
+				// restart watch from last rv
+				break
+			}
+			if ev.Type == watch.Error {
+				// restart watch from the start
+				rv = ""
+				time.Sleep(time.Second)
+				break
+			}
+			env := ev.Object.(*tpr.Environment)
+			rv = env.Metadata.ResourceVersion
+			envw.sync()
+		}
+	}
+}
+
+func (envw *environmentWatcher) sync() {
+	envList, err := envw.fissionClient.Environments(api.NamespaceAll).List(api.ListOptions{})
+	if err != nil {
+		log.Fatalf("Error syncing environment TPR resources: %v", err)
+	}
+
+	// Create environment builders for all environments
+	for i := range envList.Items {
+		env := envList.Items[i]
+
+		if env.Spec.Version == 1 || // builder is not supported with v1 interface
+			len(env.Spec.Builder.Image) == 0 { // ignore env without builder image
+			continue
+		}
+		_, err := envw.getEnvBuilder(&env)
+		if err != nil {
+			log.Printf("Error creating builder for %v: %v", env.Metadata.Name, err)
+		}
+	}
+	envw.cleanupEnvBuilders(envList.Items)
+}
+
+func (envw *environmentWatcher) service() {
+	for {
+		req := <-envw.requestChan
+		switch req.requestType {
+		case GET_BUILDER:
+			key := envw.getCacheKey(req.env.Metadata.Name, req.env.Metadata.ResourceVersion)
+			builderInfo, ok := envw.cache[key]
+			if !ok {
+				builderInfo, err := envw.createBuilder(req.env)
+				if err != nil {
+					req.respChan <- envwResponse{err: err}
+					continue
+				}
+				envw.cache[key] = builderInfo
+			}
+			req.respChan <- envwResponse{builderInfo: builderInfo}
+
+		case CLEANUP_BUILDERS:
+			latestEnvList := make(map[string]*tpr.Environment)
+			for i := range req.envList {
+				env := req.envList[i]
+				key := envw.getCacheKey(env.Metadata.Name, env.Metadata.ResourceVersion)
+				latestEnvList[key] = &env
+			}
+
+			// If an environment is deleted when builder manager down,
+			// the builder belongs to the environment will be out-of-
+			// control (an orphan builder) since there is no record in
+			// cache and TPR. We need to iterate over the services &
+			// deployments to remove both normal and orphan builders.
+
+			svcList, err := envw.getBuilderServiceList(nil)
+			if err != nil {
+				log.Println(err.Error())
+			}
+			for _, svc := range svcList {
+				envName := svc.ObjectMeta.Labels[LABEL_ENV_NAME]
+				envResourceVersion := svc.ObjectMeta.Labels[LABEL_ENV_RESOURCEVERSION]
+				key := envw.getCacheKey(envName, envResourceVersion)
+				if _, ok := latestEnvList[key]; !ok {
+					err := envw.deleteBuilderService(svc.ObjectMeta.Labels)
+					if err != nil {
+						log.Printf("Error removing builder service: %v", err)
+					}
+				}
+				delete(envw.cache, svc.ObjectMeta.Name)
+			}
+
+			deployList, err := envw.getBuilderDeploymentList(nil)
+			if err != nil {
+				log.Printf(err.Error())
+			}
+			for _, deploy := range deployList {
+				envName := deploy.ObjectMeta.Labels[LABEL_ENV_NAME]
+				envResourceVersion := deploy.ObjectMeta.Labels[LABEL_ENV_RESOURCEVERSION]
+				key := envw.getCacheKey(envName, envResourceVersion)
+				if _, ok := latestEnvList[key]; !ok {
+					err := envw.deleteBuilderDeployment(deploy.ObjectMeta.Labels)
+					if err != nil {
+						log.Printf("Error removing builder deployment: %v", err)
+					}
+				}
+				delete(envw.cache, deploy.ObjectMeta.Name)
+			}
+		}
+	}
+}
+
+func (envw *environmentWatcher) getEnvBuilder(env *tpr.Environment) (*builderInfo, error) {
+	respChan := make(chan envwResponse)
+	envw.requestChan <- envwRequest{
+		requestType: GET_BUILDER,
+		env:         env,
+		respChan:    respChan,
+	}
+	resp := <-respChan
+	return resp.builderInfo, resp.err
+}
+
+func (envw *environmentWatcher) cleanupEnvBuilders(envs []tpr.Environment) {
+	envw.requestChan <- envwRequest{
+		requestType: CLEANUP_BUILDERS,
+		envList:     envs,
+	}
+}
+
+func (envw *environmentWatcher) createBuilder(env *tpr.Environment) (*builderInfo, error) {
+	var svc *v1.Service
+	var deploy *v1beta1.Deployment
+
+	sel := envw.getLabels(env.Metadata.Name, env.Metadata.ResourceVersion)
+
+	svcList, err := envw.getBuilderServiceList(sel)
+	if err != nil {
+		return nil, err
+	}
+	if len(svcList) == 0 {
+		svc, err = envw.createBuilderService(env)
+		if err != nil {
+			return nil, fmt.Errorf("Error creating builder service: %v", err)
+		}
+	}
+
+	deployList, err := envw.getBuilderDeploymentList(sel)
+	if err != nil {
+		return nil, err
+	}
+	if len(deployList) == 0 {
+		deploy, err = envw.createBuilderDeployment(env)
+		if err != nil {
+			return nil, fmt.Errorf("Error creating builder deployment: %v", err)
+		}
+	}
+
+	return &builderInfo{
+		envMetadata: &env.Metadata,
+		service:     svc,
+		deployment:  deploy,
+	}, nil
+}
+
+func (envw *environmentWatcher) deleteBuilderService(sel map[string]string) error {
+	svcList, err := envw.getBuilderServiceList(sel)
+	if err != nil {
+		return err
+	}
+	for _, svc := range svcList {
+		log.Printf("Removing builder service: %v", svc.ObjectMeta.Name)
+
+		// cascading deletion
+		// https://kubernetes.io/docs/concepts/workloads/controllers/garbage-collection/
+		falseVal := false
+		delOpt := &api.DeleteOptions{
+			OrphanDependents: &falseVal,
+		}
+
+		err = envw.kubernetesClient.
+			Services(envw.builderNamespace).
+			Delete(svc.ObjectMeta.Name, delOpt)
+		if err != nil {
+			return fmt.Errorf("Error deleting builder service: %v", err)
+		}
+	}
+	return nil
+}
+
+func (envw *environmentWatcher) deleteBuilderDeployment(sel map[string]string) error {
+	deployList, err := envw.getBuilderDeploymentList(sel)
+	if err != nil {
+		return err
+	}
+	for _, deploy := range deployList {
+		log.Printf("Removing builder deployment: %v", deploy.ObjectMeta.Name)
+
+		falseVal := false
+		delOpt := &api.DeleteOptions{
+			OrphanDependents: &falseVal,
+		}
+
+		err = envw.kubernetesClient.
+			Deployments(envw.builderNamespace).
+			Delete(deploy.ObjectMeta.Name, delOpt)
+		if err != nil {
+			return fmt.Errorf("Error deleteing builder deployment: %v", err)
+		}
+	}
+	return nil
+}
+
+func (envw *environmentWatcher) getBuilderServiceList(sel map[string]string) ([]v1.Service, error) {
+	svcList, err := envw.kubernetesClient.Services(envw.builderNamespace).List(
+		api.ListOptions{
+			LabelSelector: labels.Set(sel).AsSelector(),
+		})
+	if err != nil {
+		return nil, fmt.Errorf("Error getting builder service list: %v", err)
+	}
+	return svcList.Items, nil
+}
+
+func (envw *environmentWatcher) createBuilderService(env *tpr.Environment) (*v1.Service, error) {
+	name := envw.getCacheKey(env.Metadata.Name, env.Metadata.ResourceVersion)
+	sel := envw.getLabels(env.Metadata.Name, env.Metadata.ResourceVersion)
+	service := v1.Service{
+		ObjectMeta: v1.ObjectMeta{
+			Namespace: envw.builderNamespace,
+			Name:      name,
+			Labels:    sel,
+		},
+		Spec: v1.ServiceSpec{
+			Selector: sel,
+			Type:     v1.ServiceTypeClusterIP,
+			Ports: []v1.ServicePort{
+				{
+					Name:     "fetcher-port",
+					Protocol: v1.ProtocolTCP,
+					Port:     8000,
+					TargetPort: intstr.IntOrString{
+						Type:   intstr.Int,
+						IntVal: 8000,
+					},
+				},
+				{
+					Name:     "builder-port",
+					Protocol: v1.ProtocolTCP,
+					Port:     8001,
+					TargetPort: intstr.IntOrString{
+						Type:   intstr.Int,
+						IntVal: 8001,
+					},
+				},
+			},
+		},
+	}
+	log.Printf("Creating builder service: %v", name)
+	_, err := envw.kubernetesClient.Services(envw.builderNamespace).Create(&service)
+	if err != nil {
+		return nil, err
+	}
+	return &service, nil
+}
+
+func (envw *environmentWatcher) getBuilderDeploymentList(sel map[string]string) ([]v1beta1.Deployment, error) {
+	deployList, err := envw.kubernetesClient.Deployments(envw.builderNamespace).List(
+		api.ListOptions{
+			LabelSelector: labels.Set(sel).AsSelector(),
+		})
+	if err != nil {
+		return nil, fmt.Errorf("Error getting builder deployment list: %v", err)
+	}
+	return deployList.Items, nil
+}
+
+func (envw *environmentWatcher) createBuilderDeployment(env *tpr.Environment) (*v1beta1.Deployment, error) {
+	sharedMountPath := "/package"
+	name := envw.getCacheKey(env.Metadata.Name, env.Metadata.ResourceVersion)
+	sel := envw.getLabels(env.Metadata.Name, env.Metadata.ResourceVersion)
+	var replicas int32 = 1
+	deployment := &v1beta1.Deployment{
+		ObjectMeta: v1.ObjectMeta{
+			Namespace: envw.builderNamespace,
+			Name:      name,
+			Labels:    sel,
+		},
+		Spec: v1beta1.DeploymentSpec{
+			Replicas: &replicas,
+			Selector: &v1beta1.LabelSelector{
+				MatchLabels: sel,
+			},
+			Template: v1.PodTemplateSpec{
+				ObjectMeta: v1.ObjectMeta{
+					Labels: sel,
+				},
+				Spec: v1.PodSpec{
+					Volumes: []v1.Volume{
+						{
+							Name: "package",
+							VolumeSource: v1.VolumeSource{
+								EmptyDir: &v1.EmptyDirVolumeSource{},
+							},
+						},
+					},
+					Containers: []v1.Container{
+						{
+							Name:                   "builder",
+							Image:                  env.Spec.Builder.Image,
+							ImagePullPolicy:        v1.PullAlways,
+							TerminationMessagePath: "/dev/termination-log",
+							VolumeMounts: []v1.VolumeMount{
+								{
+									Name:      "package",
+									MountPath: sharedMountPath,
+								},
+							},
+							Command: []string{"/builder", sharedMountPath},
+						},
+						{
+							Name:                   "fetcher",
+							Image:                  envw.fetcherImage,
+							ImagePullPolicy:        envw.fetcherImagePullPolicy,
+							TerminationMessagePath: "/dev/termination-log",
+							VolumeMounts: []v1.VolumeMount{
+								{
+									Name:      "package",
+									MountPath: sharedMountPath,
+								},
+							},
+							Command: []string{"/fetcher", sharedMountPath},
+						},
+					},
+					ServiceAccountName: "fission-builder",
+				},
+			},
+		},
+	}
+	log.Printf("Creating builder deployment: %v", envw.getCacheKey(env.Metadata.Name, env.Metadata.ResourceVersion))
+	_, err := envw.kubernetesClient.Deployments(envw.builderNamespace).Create(deployment)
+	if err != nil {
+		return nil, err
+	}
+	return deployment, nil
+}

buildermgr/pkgwatcher.go

@@ -0,0 +1,91 @@
+/*
+Copyright 2017 The Fission Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package buildermgr
+
+import (
+	"log"
+	"time"
+
+	"k8s.io/client-go/1.5/kubernetes"
+	"k8s.io/client-go/1.5/pkg/api"
+	"k8s.io/client-go/1.5/pkg/watch"
+
+	"github.com/fission/fission"
+	"github.com/fission/fission/tpr"
+)
+
+type (
+	packageWatcher struct {
+		fissionClient    *tpr.FissionClient
+		kubernetesClient *kubernetes.Clientset
+		builderNamespace string
+		storageSvcUrl    string
+	}
+)
+
+func makePackageWatcher(fissionClient *tpr.FissionClient,
+	kubernetesClient *kubernetes.Clientset, builderNamespace string, storageSvcUrl string) *packageWatcher {
+	pkgw := &packageWatcher{
+		fissionClient:    fissionClient,
+		kubernetesClient: kubernetesClient,
+		builderNamespace: builderNamespace,
+		storageSvcUrl:    storageSvcUrl,
+	}
+	return pkgw
+}
+
+func (pkgw *packageWatcher) build(pkgMetadata api.ObjectMeta) {
+	buildReq := BuildRequest{
+		Package: pkgMetadata,
+	}
+	_, err := buildPackage(pkgw.fissionClient,
+		pkgw.kubernetesClient, pkgw.builderNamespace, pkgw.storageSvcUrl, buildReq)
+	if err != nil {
+		log.Printf("Error building package %v: %v", buildReq.Package.Name, err)
+	}
+}
+
+func (pkgw *packageWatcher) watchPackages() {
+	rv := ""
+	for {
+		wi, err := pkgw.fissionClient.Packages(api.NamespaceDefault).Watch(api.ListOptions{
+			ResourceVersion: rv,
+		})
+		if err != nil {
+			log.Fatalf("Error watching package TPR resources: %v", err)
+		}
+
+		for {
+			ev, more := <-wi.ResultChan()
+			if !more {
+				break
+			}
+			if ev.Type == watch.Error {
+				rv = ""
+				time.Sleep(time.Second)
+				break
+			}
+			pkg := ev.Object.(*tpr.Package)
+			rv = pkg.Metadata.ResourceVersion
+
+			// only do build for packages in pending state
+			if pkg.Status.BuildStatus == fission.BuildStatusPending {
+				go pkgw.build(pkg.Metadata)
+			}
+		}
+	}
+}

cache/cache.go

@@ -17,7 +17,6 @@ limitations under the License.
 package cache
 
 import (
-	"errors"
 	"fmt"
 	"time"
 
@@ -114,7 +113,7 @@ func (c *Cache) service() {
 				val := c.cache[req.key]
 				val.atime = time.Now()
 				resp.existingValue = val.value
-				resp.error = errors.New("value already exists")
+				resp.error = fission.MakeError(fission.ErrorNameExists, "key already exists")
 			} else {
 				c.cache[req.key] = &Value{
 					value: req.value,

charts/README.md

@@ -47,6 +47,7 @@ The following table lists the configurable parameters of the Fission chart and t
 | `controllerPort`    | Fission Controller Service Port            | `31313`                  |
 | `routerPort`        | Fission Router Service Port                | `31314`                  |
 | `functionNamespace` | Namespace for Fission functions            | `fission-function`       |
+| `builderNamespace`  | Namespace for Fission environment builders | `fission-builder`        |
 | `openshift`         | RBAC for openshift                         | `false`                  |
 
 

charts/fission-all/Chart.yaml

@@ -1,6 +1,6 @@
 apiVersion: v1
 name: fission-all
-version: 0.2.1
+version: 0.3.0-rc
 description: Fission is a fast serverless framework for Kubernetes.
 keywords:
 - fission
@@ -12,4 +12,4 @@ maintainers:
   - name: Ta Ching Chen
     email: contact@tachingchen.com
 engine: gotpl
-appVersion: 0.2.1
+appVersion: 0.3.0-rc

charts/fission-all/templates/NOTES.txt

@@ -1,13 +1,13 @@
 1. Install the client CLI.
 
 Mac:
-  $ curl -Lo fission https://github.com/fission/fission/releases/download/v0.2.1/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
+  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.3.0-rc/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
 
 Linux:
-  $ curl -Lo fission https://github.com/fission/fission/releases/download/v0.2.1/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
+  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.3.0-rc/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
 
 Windows:
-  For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/v0.2.1/fission-cli-windows.exe
+  For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/0.3.0-rc/fission-cli-windows.exe
 
 2. Set the FISSION_URL and FISSION_ROUTER environment variables.
 

charts/fission-all/templates/deployment.yaml

@@ -81,6 +81,15 @@ metadata:
     name: fission-function
     chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
 
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: {{ .Values.builderNamespace }}
+  labels:
+    name: fission-builder
+    chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
+
 ---
 apiVersion: v1
 kind: ServiceAccount
@@ -153,6 +162,27 @@ roleRef:
   name: cluster-admin
   apiGroup: rbac.authorization.k8s.io
 
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: fission-builder
+  namespace: {{ .Values.builderNamespace }}
+
+---
+kind: ClusterRoleBinding
+apiVersion: rbac.authorization.k8s.io/v1beta1
+metadata:
+  name: fission-builder-tpr
+subjects:
+- kind: ServiceAccount
+  name: fission-builder
+  namespace: {{ .Values.builderNamespace }}
+roleRef:
+  kind: ClusterRole
+  name: cluster-admin
+  apiGroup: rbac.authorization.k8s.io
+
 {{ end }}
 
 ---
@@ -240,6 +270,51 @@ spec:
           value: "{{ .Values.fetcherImage }}:{{ .Values.fetcherImageTag }}"
         - name: FETCHER_IMAGE_PULL_POLICY
           value: "{{ .Values.pullPolicy }}"
+        - name: RUNTIME_IMAGE_PULL_POLICY
+          value: "{{ .Values.pullPolicy }}"
+      serviceAccount: fission-svc
+
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: buildermgr
+  labels:
+    svc: buildermgr
+    chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
+spec:
+  type: ClusterIP
+  ports:
+  - port: 80
+    targetPort: 8889
+  selector:
+    svc: buildermgr
+
+---
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  name: buildermgr
+  labels:
+    chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
+spec:
+  replicas: 1
+  template:
+    metadata:
+      labels:
+        svc: buildermgr
+    spec:
+      containers:
+      - name: buildermgr
+        image: "{{ .Values.image }}:{{ .Values.imageTag }}"
+        imagePullPolicy: {{ .Values.pullPolicy }}
+        command: ["/fission-bundle"]
+        args: ["--builderMgrPort", "8889", "--storageSvcUrl", "http://storagesvc.{{ .Release.Namespace }}", "--envbuilder-namespace", "{{ .Values.builderNamespace }}"]
+        env:
+        - name: FETCHER_IMAGE
+          value: "{{ .Values.fetcherImage }}:{{ .Values.fetcherImageTag }}"
+        - name: FETCHER_IMAGE_PULL_POLICY
+          value: "{{ .Values.pullPolicy }}"
       serviceAccount: fission-svc
 
 ---

charts/fission-all/templates/post-install-job.yaml

@@ -0,0 +1,30 @@
+{{ if .Values.analytics }}
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: {{ template "fullname" . }}
+  labels:
+    # The "release" convention makes it easy to tie a release to all of the
+    # Kubernetes resources that were created as part of that release.
+    release: {{ .Release.Name }}
+    # This makes it easy to audit chart usage.
+    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    app: {{ template "name" . }}
+  annotations:
+    # This is what defines this resource as a hook. Without this line, the
+    # job is considered part of the release.
+    "helm.sh/hook": post-install
+spec:
+  template:
+    metadata:
+      name: {{ template "fullname" . }}
+      labels:
+        release: {{ .Release.Name }}
+        app: {{ template "name" . }}
+    spec:
+      restartPolicy: Never
+      containers:
+        - name: post-install-job
+          image: "fission/alpinecurl"
+          command: ["sh", "-c", "/usr/bin/curl -H \"Content-Type: application/json\" -X POST -d '{\"type\": \"helm-post-install\", \"chartName\": \"{{ .Chart.Name }}\", \"chartName\": \"{{ .Chart.Version }}\"}' https://g.fission.sh/analytics || true"]
+{{ end }}

charts/fission-all/values.yaml

@@ -13,13 +13,13 @@ image: fission/fission-bundle
 pullPolicy: IfNotPresent
 
 ## Fission image version
-imageTag: v0.2.1
+imageTag: 0.3.0-rc
 
 ## Fission fetcher repository
 fetcherImage: fission/fetcher
 
 ## Fission fetcher image version
-fetcherImageTag: v0.2.1
+fetcherImageTag: 0.3.0-rc
 
 ## Port at which Fission controller service should be exposed
 controllerPort: 31313
@@ -34,6 +34,10 @@ natsStreamingPort: 31316
 ## the release namespace)
 functionNamespace: fission-function
 
+## Namespace in which to run fission builders (this is different from
+## the release namespace)
+builderNamespace: fission-builder
+
 ## Set up openshift RBAC rule
 openshift: false
 
@@ -58,3 +62,7 @@ persistence:
   # storageClass:
   accessMode: ReadWriteOnce
   size: 8Gi
+
+## Analytics let us count how many people installed fission. Set to
+## false to disable analytics.
+analytics: true

charts/fission-core/Chart.yaml

@@ -1,6 +1,6 @@
 apiVersion: v1
 name: fission-core
-version: 0.2.1
+version: 0.3.0-rc
 description: Fission is a fast serverless framework for Kubernetes.
 keywords:
 - fission
@@ -12,4 +12,4 @@ maintainers:
   - name: Ta Ching Chen
     email: contact@tachingchen.com
 engine: gotpl
-appVersion: 0.2.1
+appVersion: 0.3.0-rc

charts/fission-core/templates/NOTES.txt

@@ -1,13 +1,13 @@
 1. Install the client CLI.
 
 Mac:
-  $ curl -Lo fission https://github.com/fission/fission/releases/download/v0.2.1/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
+  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.3.0-rc/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
 
 Linux:
-  $ curl -Lo fission https://github.com/fission/fission/releases/download/v0.2.1/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
+  $ curl -Lo fission https://github.com/fission/fission/releases/download/0.3.0-rc/fission-cli-linux && chmod +x fission && sudo mv fission /usr/local/bin/
 
 Windows:
-  For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/v0.2.1/fission-cli-windows.exe
+  For Windows, you can use the linux binary on WSL. Or you can download this windows executable: https://github.com/fission/fission/releases/download/0.3.0-rc/fission-cli-windows.exe
 
 2. Set the FISSION_URL and FISSION_ROUTER environment variables.
 

charts/fission-core/templates/deployment.yaml

@@ -81,6 +81,15 @@ metadata:
     name: fission-function
     chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
 
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: {{ .Values.builderNamespace }}
+  labels:
+    name: fission-builder
+    chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
+
 ---
 apiVersion: v1
 kind: ServiceAccount
@@ -153,6 +162,27 @@ roleRef:
   name: cluster-admin
   apiGroup: rbac.authorization.k8s.io
 
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: fission-builder
+  namespace: {{ .Values.builderNamespace }}
+
+---
+kind: ClusterRoleBinding
+apiVersion: rbac.authorization.k8s.io/v1beta1
+metadata:
+  name: fission-builder-tpr
+subjects:
+- kind: ServiceAccount
+  name: fission-builder
+  namespace: {{ .Values.builderNamespace }}
+roleRef:
+  kind: ClusterRole
+  name: cluster-admin
+  apiGroup: rbac.authorization.k8s.io
+
 {{ end }}
 
 ---
@@ -242,6 +272,49 @@ spec:
           value: "{{ .Values.pullPolicy }}"
       serviceAccount: fission-svc
 
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: buildermgr
+  labels:
+    svc: buildermgr
+    chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
+spec:
+  type: ClusterIP
+  ports:
+  - port: 80
+    targetPort: 8889
+  selector:
+    svc: buildermgr
+
+---
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  name: buildermgr
+  labels:
+    chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
+spec:
+  replicas: 1
+  template:
+    metadata:
+      labels:
+        svc: buildermgr
+    spec:
+      containers:
+      - name: buildermgr
+        image: "{{ .Values.image }}:{{ .Values.imageTag }}"
+        imagePullPolicy: {{ .Values.pullPolicy }}
+        command: ["/fission-bundle"]
+        args: ["--builderMgrPort", "8889", "--storageSvcUrl", "http://storagesvc.{{ .Release.Namespace }}", "--envbuilder-namespace", "{{ .Values.builderNamespace }}"]
+        env:
+        - name: FETCHER_IMAGE
+          value: "{{ .Values.fetcherImage }}:{{ .Values.fetcherImageTag }}"
+        - name: FETCHER_IMAGE_PULL_POLICY
+          value: "{{ .Values.pullPolicy }}"
+      serviceAccount: fission-svc
+
 ---
 apiVersion: extensions/v1beta1
 kind: Deployment

charts/fission-core/templates/post-install-job.yaml

@@ -0,0 +1,30 @@
+{{ if .Values.analytics }}
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: {{ template "fullname" . }}
+  labels:
+    # The "release" convention makes it easy to tie a release to all of the
+    # Kubernetes resources that were created as part of that release.
+    release: {{ .Release.Name }}
+    # This makes it easy to audit chart usage.
+    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    app: {{ template "name" . }}
+  annotations:
+    # This is what defines this resource as a hook. Without this line, the
+    # job is considered part of the release.
+    "helm.sh/hook": post-install
+spec:
+  template:
+    metadata:
+      name: {{ template "fullname" . }}
+      labels:
+        release: {{ .Release.Name }}
+        app: {{ template "name" . }}
+    spec:
+      restartPolicy: Never
+      containers:
+        - name: post-install-job
+          image: "fission/alpinecurl"
+          command: ["sh", "-c", "/usr/bin/curl -H \"Content-Type: application/json\" -X POST -d '{\"type\": \"helm-post-install\", \"chartName\": \"{{ .Chart.Name }}\", \"chartName\": \"{{ .Chart.Version }}\"}' https://g.fission.sh/analytics || true"]
+{{ end }}

charts/fission-core/values.yaml

@@ -10,7 +10,7 @@ serviceType: LoadBalancer
 image: fission/fission-bundle
 
 ## Fission image version
-imageTag: v0.2.1
+imageTag: 0.3.0-rc
 
 ## Image pull policy
 pullPolicy: IfNotPresent
@@ -19,7 +19,7 @@ pullPolicy: IfNotPresent
 fetcherImage: fission/fetcher
 
 ## Fission fetcher image version
-fetcherImageTag: v0.2.1
+fetcherImageTag: 0.3.0-rc
 
 ## Port at which Fission controller service should be exposed
 controllerPort: 31313
@@ -31,6 +31,10 @@ routerPort: 31314
 ## the release namespace)
 functionNamespace: fission-function
 
+## Namespace in which to run fission builders (this is different from
+## the release namespace)
+builderNamespace: fission-builder
+
 ## Set up openshift RBAC rule
 openshift: false
   
@@ -43,3 +47,7 @@ persistence:
   # storageClass:
   accessMode: ReadWriteOnce
   size: 8Gi
+
+## Analytics let us count how many people installed fission. Set to
+## false to disable analytics.
+analytics: true

controller/api.go

@@ -37,6 +37,8 @@ type (
 	API struct {
 		fissionClient     *tpr.FissionClient
 		storageServiceUrl string
+		builderManagerUrl string
+		workflowApiUrl    string
 	}
 
 	logDBConfig struct {
@@ -56,6 +58,20 @@ func MakeAPI() (*API, error) {
 		api.storageServiceUrl = "http://storagesvc"
 	}
 
+	u = os.Getenv("BUILDER_MANAGER_URL")
+	if len(u) > 0 {
+		api.builderManagerUrl = strings.TrimSuffix(u, "/")
+	} else {
+		api.builderManagerUrl = "http://buildermgr"
+	}
+
+	wfEnv := os.Getenv("WORKFLOW_API_URL")
+	if len(u) > 0 {
+		api.workflowApiUrl = strings.TrimSuffix(wfEnv, "/")
+	} else {
+		api.workflowApiUrl = "http://workflow-apiserver"
+	}
+
 	return api, err
 }
 
@@ -102,7 +118,7 @@ func (api *API) getLogDBConfig(dbType string) logDBConfig {
 
 func (api *API) HomeHandler(w http.ResponseWriter, r *http.Request) {
 	w.Header().Set("Content-Type", "application/json; charset=utf-8")
-	fmt.Fprintf(w, "{\"message\": \"Fission API\", \"version\": \"0.2.1\"}\n")
+	fmt.Fprintf(w, "{\"message\": \"Fission API\", \"version\": \"0.3.0-rc\"}\n")
 }
 
 func (api *API) ApiVersionMismatchHandler(w http.ResponseWriter, r *http.Request) {
@@ -160,6 +176,9 @@ func (api *API) Serve(port int) {
 
 	r.HandleFunc("/proxy/{dbType}", api.FunctionLogsApiPost).Methods("POST")
 	r.HandleFunc("/proxy/storage/v1/archive", api.StorageServiceProxy)
+	r.HandleFunc("/proxy/buildermgr/v1/build", api.BuilderManagerBuildProxy)
+	r.HandleFunc("/proxy/buildermgr/v1/builder", api.BuilderManagerEnvBuilderProxy)
+	r.HandleFunc("/proxy/workflow", api.WorkflowApiProxy)
 
 	address := fmt.Sprintf(":%v", port)
 

controller/buildermgr.go

@@ -0,0 +1,67 @@
+/*
+Copyright 2017 The Fission Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package controller
+
+import (
+	"fmt"
+	"log"
+	"net/http"
+	"net/http/httputil"
+	"net/url"
+)
+
+func (api *API) BuilderManagerBuildProxy(w http.ResponseWriter, r *http.Request) {
+	u := api.builderManagerUrl + "/v1/build"
+	proxy, err := api._getBuilderManagerProxy(u)
+	if err != nil {
+		msg := fmt.Sprintf("Failed to establish proxy server: %v", err)
+		log.Println(msg)
+		http.Error(w, msg, 500)
+		return
+	}
+	proxy.ServeHTTP(w, r)
+}
+
+func (api *API) BuilderManagerEnvBuilderProxy(w http.ResponseWriter, r *http.Request) {
+	u := api.builderManagerUrl + "/v1/builder"
+	proxy, err := api._getBuilderManagerProxy(u)
+	if err != nil {
+		msg := fmt.Sprintf("Failed to establish proxy server: %v", err)
+		log.Println(msg)
+		http.Error(w, msg, 500)
+		return
+	}
+	proxy.ServeHTTP(w, r)
+}
+
+func (api *API) _getBuilderManagerProxy(targetUrl string) (*httputil.ReverseProxy, error) {
+	svcUrl, err := url.Parse(targetUrl)
+	if err != nil {
+		return nil, err
+	}
+	// set up proxy server director
+	director := func(req *http.Request) {
+		// only replace url Scheme and Host to remote server
+		// and leave query string intact
+		req.URL.Scheme = svcUrl.Scheme
+		req.URL.Host = svcUrl.Host
+		req.URL.Path = svcUrl.Path
+	}
+	return &httputil.ReverseProxy{
+		Director: director,
+	}, nil
+}

controller/environmentApi.go

@@ -139,6 +139,7 @@ func (a *API) EnvironmentApiUpdate(w http.ResponseWriter, r *http.Request) {
 		a.respondWithError(w, err)
 		return
 	}
+
 	a.respondWithSuccess(w, resp)
 }
 

controller/workflowApiProxy.go

@@ -0,0 +1,29 @@
+package controller
+
+import (
+	"fmt"
+	"net/http"
+	"net/http/httputil"
+	"net/url"
+	"strings"
+)
+
+func (api *API) WorkflowApiProxy(w http.ResponseWriter, r *http.Request) {
+	u := api.storageServiceUrl
+	ssUrl, err := url.Parse(u)
+	if err != nil {
+		msg := fmt.Sprintf("Error parsing url %v: %v", u, err)
+		http.Error(w, msg, 500)
+		return
+	}
+	director := func(req *http.Request) {
+		req.URL.Scheme = ssUrl.Scheme
+		req.URL.Host = ssUrl.Host
+		req.URL.Path = strings.TrimPrefix(ssUrl.Path, "/proxy/workflow")
+		req.URL.RawQuery = ssUrl.RawQuery
+	}
+	proxy := &httputil.ReverseProxy{
+		Director: director,
+	}
+	proxy.ServeHTTP(w, r)
+}

environments/dotnet20/Dockerfile

@@ -0,0 +1,9 @@
+FROM microsoft/dotnet:2.0-runtime
+
+WORKDIR /fission-workdir
+COPY out .
+EXPOSE 8888
+
+ENTRYPOINT ["dotnet"]
+
+CMD ["fission-dotnet20.dll"]

environments/dotnet20/ExecutorModule.cs

@@ -0,0 +1,82 @@
+using Fission.DotNetCore.Compiler;
+using Fission.DotNetCore.Api;
+using System.Collections.Generic;
+using Nancy;
+using System.IO;
+using System;
+
+namespace Fission.DotNetCore
+{
+    public class ExecutorModule : NancyModule
+    {
+#if DEBUG
+        private const string CODE_PATH = "/tmp/func.cs";
+#else
+        private const string CODE_PATH = "/userfunc/user";
+#endif
+        private static Function _userFunc;
+        private static Logger _logger = new Logger();
+
+        public ExecutorModule()
+        {
+            Post("/specialize", args => Specialize());
+            Get("/", _ => Run());
+            Post("/", _ => Run());
+            Put("/", _ => Run());
+            Head("/", _ => Run());
+            Options("/", _ => Run());
+            Delete("/", _ => Run());
+        }
+
+        private object Specialize()
+        {
+            var errors = new List<string>();
+            if (File.Exists(CODE_PATH))
+            {
+                var code = File.ReadAllText(CODE_PATH);
+                _userFunc = FissionCompiler.Compile(code, out errors);
+                if (_userFunc == null)
+                {
+                    var errstr = string.Join(Environment.NewLine, errors);
+                    _logger.WriteError(errstr);
+                    var response = (Response)errstr;
+                    response.StatusCode = HttpStatusCode.InternalServerError;
+                    return response;
+
+                }
+                return null;
+            }
+            else
+            {
+                var errstr = $"Unable to locate code at '{CODE_PATH}'";
+                _logger.WriteError(errstr);
+                var response = (Response)errstr;
+                response.StatusCode = HttpStatusCode.InternalServerError;
+                return response;
+            }
+        }
+
+        private object Run()
+        {
+            if (_userFunc == null)
+            {
+                var response = (Response)"Generic container: no requests supported";
+                response.StatusCode = HttpStatusCode.InternalServerError;
+                return response;
+            }
+            
+            try
+            {
+                return _userFunc.Invoke(FissionContext.Build(Request, new Logger()));
+            }
+            catch (Exception e)
+            {
+                _logger.WriteError(e.ToString());
+                var response = (Response)e.Message;
+                response.StatusCode = HttpStatusCode.BadRequest;
+                return response;
+            }
+        }
+    }
+}
+

environments/dotnet20/FissionCompiler.cs

@@ -0,0 +1,75 @@
+using System.Collections.Generic;
+using System.IO;
+using System.Linq;
+using System.Reflection;
+using System.Runtime.Loader;
+using Microsoft.CodeAnalysis;
+using Microsoft.CodeAnalysis.CSharp;
+using Microsoft.CodeAnalysis.Emit;
+
+namespace Fission.DotNetCore.Compiler
+{
+    //adapted from this article http://www.tugberkugurlu.com/archive/compiling-c-sharp-code-into-memory-and-executing-it-with-roslyn
+    class FissionCompiler
+    {
+        public static Function Compile(string code, out List<string> errors)
+        {
+            errors = new List<string>();
+
+            SyntaxTree syntaxTree = CSharpSyntaxTree.ParseText(code);
+
+            string assemblyName = Path.GetRandomFileName();
+
+            var coreDir = Directory.GetParent(typeof(Enumerable).GetTypeInfo().Assembly.Location);      
+            
+            List<MetadataReference> references = new List<MetadataReference>
+            {
+                MetadataReference.CreateFromFile(coreDir.FullName + Path.DirectorySeparatorChar + "mscorlib.dll"),
+                MetadataReference.CreateFromFile(typeof(object).GetTypeInfo().Assembly.Location),
+                MetadataReference.CreateFromFile(Assembly.GetEntryAssembly().Location),
+                MetadataReference.CreateFromFile(typeof(System.Runtime.Serialization.Json.DataContractJsonSerializer).GetTypeInfo().Assembly.Location)
+            };
+
+            foreach (var referencedAssembly in Assembly.GetEntryAssembly().GetReferencedAssemblies())
+            {
+                var assembly = Assembly.Load(referencedAssembly);
+                references.Add(MetadataReference.CreateFromFile(assembly.Location));
+            }
+
+            CSharpCompilation compilation = CSharpCompilation.Create(
+                assemblyName,
+                syntaxTrees: new[] { syntaxTree },
+                references: references,
+                options: new CSharpCompilationOptions(
+                    OutputKind.DynamicallyLinkedLibrary,
+                    optimizationLevel: OptimizationLevel.Release));
+
+            using (var ms = new MemoryStream())
+            {
+                EmitResult result = compilation.Emit(ms);
+
+                if (!result.Success)
+                {
+                    IEnumerable<Diagnostic> failures = result.Diagnostics.Where(diagnostic =>
+                        diagnostic.IsWarningAsError ||
+                        diagnostic.Severity == DiagnosticSeverity.Error).ToList();
+
+                    foreach (Diagnostic diagnostic in failures)
+                    {
+                        errors.Add($"{diagnostic.Id}: {diagnostic.GetMessage()}");
+                    }
+                }
+                else
+                {
+                    ms.Seek(0, SeekOrigin.Begin);
+
+                    Assembly assembly = AssemblyLoadContext.Default.LoadFromStream(ms);
+                    var type = assembly.GetType("FissionFunction");
+                    var info = type.GetMember("Execute").First() as MethodInfo;
+                    return new Function(assembly, type, info);
+                }
+            }
+            return null;
+        }
+    }
+}

environments/dotnet20/FissionContext.cs

@@ -0,0 +1,114 @@
+using System;
+using System.Collections.Generic;
+using System.IO;
+using System.Security.Cryptography.X509Certificates;
+using System.Text;
+using Nancy;
+
+namespace Fission.DotNetCore.Api
+{
+    public class FissionContext
+    {
+        public FissionContext(Dictionary<string, object> args, Logger logger, FissionHttpRequest request)
+        {
+            if (args == null) throw new ArgumentNullException(nameof(args));
+            if (logger == null) throw new ArgumentNullException(nameof(logger));
+            if (request == null) throw new ArgumentNullException(nameof(request));
+            Arguments = args;
+            Logger = logger;
+            Request = request;
+        }
+
+        public Dictionary<string, object> Arguments { get; private set; }
+
+        public FissionHttpRequest Request { get; private set; }
+
+        public Logger Logger { get; private set; }
+
+        public static FissionContext Build(Request request, Logger logger)
+        {
+            return new FissionContext(((DynamicDictionary)request.Query).ToDictionary(),
+                                        logger,
+                                        new FissionHttpRequest(request));
+        }
+    }
+
+    public class Logger
+    {
+        public void Write(Severity severity, string format, params object[] args)
+        {
+            Console.WriteLine($"{DateTime.Now.ToString("MM/dd/yy H:mm:ss zzz")} {severity}: " + format, args);
+        }
+
+        public void WriteInfo(string format, params object[] args)
+        {
+            Write(Severity.Info, format, args);
+        }
+
+        public void WriteWarning(string format, params object[] args)
+        {
+            Write(Severity.Warning, format, args);
+        }
+
+        public void WriteError(string format, params object[] args)
+        {
+            Write(Severity.Error, format, args);
+        }
+
+        public void WriteCritical(string format, params object[] args)
+        {
+            Write(Severity.Critical, format, args);
+        }
+
+        public void WriteVerbose(string format, params object[] args)
+        {
+            Write(Severity.Verbose, format, args);
+        }
+    }
+
+    public enum Severity
+    {
+        Info,
+        Warning,
+        Error,
+        Critical,
+        Verbose
+    }
+
+    public class FissionHttpRequest
+    {
+        private readonly Request _request;
+        internal FissionHttpRequest(Request request)
+        {
+            if (request == null) throw new ArgumentNullException(nameof(request));
+            _request = request;
+        }
+
+        public Stream Body { get { return _request.Body; } }
+
+        public string BodyAsString()
+        {
+            int length = (int)_request.Body.Length;
+            byte[] data = new byte[length];
+            _request.Body.Read(data, 0, length);
+            return Encoding.UTF8.GetString(data);
+        }
+        
+        public Dictionary<string, IEnumerable<string>> Headers
+        {
+            get
+            {
+                var headers = new Dictionary<string, IEnumerable<string>>();
+                foreach (var kv in _request.Headers)
+                {
+                    headers.Add(kv.Key, kv.Value);
+                }
+                return headers;
+            }
+        }
+
+        public X509Certificate Certificate { get { return _request.ClientCertificate; } }
+        public string Url { get { return _request.Url.ToString(); } }
+        public string Method { get { return _request.Method; } }
+    }
+}

environments/dotnet20/Function.cs

@@ -0,0 +1,28 @@
+using System;
+using System.Reflection;
+using Fission.DotNetCore.Api;
+
+namespace Fission.DotNetCore.Compiler
+{
+    class Function
+    {
+        private readonly Assembly _assembly;
+        private readonly Type _type;
+        private readonly MethodInfo _info;
+
+        public Function(Assembly assembly, Type type, MethodInfo info)
+        {
+            if (info == null) throw new ArgumentNullException(nameof(info));
+            if (assembly == null) throw new ArgumentNullException(nameof(assembly));
+            if (type == null) throw new ArgumentNullException(nameof(type));
+            _assembly = assembly;
+            _type = type;
+            _info = info;
+        }
+
+        public object Invoke(FissionContext context)
+        {
+            return _info.Invoke(_assembly.CreateInstance(_type.FullName), new[] { context });
+        }
+    }
+}

environments/dotnet20/Program.cs

@@ -0,0 +1,21 @@
+using System.IO;
+using Microsoft.AspNetCore.Builder;
+using Microsoft.AspNetCore.Hosting;
+using Nancy.Owin;
+
+namespace Fission.DotNetCore
+{
+    public class Program
+    {
+        public static void Main(string[] args)
+        {
+            var host = new WebHostBuilder()
+               .UseContentRoot(Directory.GetCurrentDirectory())
+               .UseKestrel()
+               .UseUrls("http://*:8888")
+               .Configure(app => app.UseOwin(x => x.UseNancy()))
+               .Build();
+            host.Run();
+        }
+    }
+}

environments/dotnet20/README.md

@@ -0,0 +1,306 @@
+# Fission: dotnet 2.0 C# Environment
+
+This is a simple dotnet core 2.0 C# environment for Fission.
+
+It's a Docker image containing the dotnet 2.0.0 runtime. The image 
+uses Kestrel with Nancy to host the internal web server and uses 
+Roslyn to compile the uploaded code.
+
+The image supports compiling and running code with types defined in
+mscorlib and does not at present support other library references.
+One workaround for this would be to add the references to this project's
+project.json file and rebuild the container.
+
+The environment works via convention where you create a C# class
+called FissionFunction which has a  method named Execute taking a single
+parameter, a FissionContext object.
+
+The FissionContext object gives access to the arguments and other items 
+like logging. Please see FissionContext.cs for public API.
+
+Example of simplest possible class to be executed:
+
+```
+using System;
+using Fission.DotNetCore.Api;
+
+public class FissionFunction {
+    public string Execute(FissionContext context) {
+        return null;
+    }
+}
+```
+
+Please see examples below.
+
+## Rebuilding and pushing the image
+
+To rebuild the image you need either a computer with dotnet core 2.0.0
+installed or else you will have to map the source directory into a
+container containing the dotnet core 2.0.0 environment.
+
+### Locally installed Dotnet core 2.0.0
+
+Simply move to the source directory in a terminal and run the ./build.sh script.
+
+The script will restore dependencies, compile a release build and
+and build the container. If you need to change the name of the container
+simply change it in the script.
+
+After the build finishes push the new image to a Docker registry using the 
+standard procedure.
+
+### Build in a container
+
+Move to the directory containing the source and start the Docker container
+with dotnet and mount the current directory to a build location:
+
+```
+docker run -it --rm -v $PWD:/build microsoft/dotnet
+```
+
+Move to the build directory inside the container and restore the packages:
+
+```
+cd /build 
+dotnet restore
+log  : Restoring packages for /source/project.json...
+log  : Installing System.Net.WebSockets 4.0.0.
+log  : Installing runtime.native.System.IO.Compression 4.1.0.
+...
+```
+
+Compile and publish a release build of the source to the 'out' folder:
+
+```
+dotnet publish -c Release -o out
+Publishing source for .NETCoreApp,Version
+...
+```
+Exit the build container and build the Docker container on the local host:
+
+```
+exit
+docker build -t USER/dotnet20-env .
+``` 
+After the build finishes push the new image to a Docker registry using the 
+standard procedure.
+
+## Echo example
+
+### Setup fission environment
+First you need to setup the fission according to your cluster setup as 
+specified here: https://github.com/fission/fission
+
+
+### Create the class to run
+
+Secondly you need to create a file /tmp/func.cs containing the following code:
+
+```
+using System;
+using Fission.DotNetCore.Api;
+
+public class FissionFunction 
+{
+    public string Execute(FissionContext context){
+        context.Logger.WriteInfo("executing.. {0}", context.Arguments["text"]);
+        return (string)context.Arguments["text"];
+    }
+}
+``` 
+### Run the example
+
+Lastly to run the example:
+
+```
+$ fission env create --name dotnet --image fission/dotnet20-env
+
+$ fission function create --name echo --env dotnet --code /tmp/func.cs
+
+$ fission route create --method GET --url /echo --function echo
+
+$ curl http://$FISSION_ROUTER/echo?text=hello%20world!
+  hello world
+```
+
+## Addition service example
+
+### Setup fission environment
+First you need to setup the fission according to your cluster setup as 
+specified here: https://github.com/fission/fission
+
+
+### Create the class to run
+
+Secondly you need to create a file /tmp/func.cs containing the following code:
+
+```
+using System;
+using Fission.DotNetCore.Api;
+
+public class FissionFunction 
+{
+    public string Execute(FissionContext context){
+        var x = Convert.ToInt32(context.Arguments["x"]);
+        var y = Convert.ToInt32(context.Arguments["y"]);
+        return (x+y).ToString();
+    }
+}
+``` 
+### Run the example
+
+Lastly to run the example:
+
+```
+$ fission env create --name dotnet --image fission/dotnet20-env
+
+$ fission function create --name addition --env dotnet --code /tmp/func.cs
+
+$ fission route create --method GET --url /add --function addition
+
+$ curl "http://$FISSION_ROUTER/add?x=30&y=12"
+  42
+```
+
+## Accessing http request information example
+
+### Setup fission environment
+First you need to setup the fission according to your cluster setup as 
+specified here: https://github.com/fission/fission
+
+
+### Create the class to run
+
+Secondly you need to create a file /tmp/func.cs containing the following code:
+
+```
+using System;
+using Fission.DotNetCore.Api;
+
+public class FissionFunction
+{
+    public string Execute(FissionContext context){
+        var buffer = new System.Text.StringBuilder();
+        foreach(var header in context.Request.Headers){
+                buffer.AppendLine(header.Key);
+                foreach(var item in header.Value){
+                        buffer.AppendLine($"\t{item}");
+                }
+        }
+        buffer.AppendLine($"Url: {context.Request.Url}, method: {context.Request.Method}");
+        return buffer.ToString();
+    }
+}
+
+``` 
+### Run the example
+
+Lastly to run the example:
+
+```
+$ fission env create --name dotnet --image fission/dotnet20-env
+
+$ fission function create --name httpinfo --env dotnet --code /tmp/func.cs
+
+$ fission route create --method GET --url /http_info --function httpinfo
+
+$ curl "http://$FISSION_ROUTER/http_info"
+Accept
+	*/*;q=1
+Host
+	fissionserver:8888
+User-Agent
+	curl/7.47.0
+Url: http://fissionserver:8888, method: GET
+
+```
+
+## Accessing http request body example
+
+### Setup fission environment
+First you need to setup the fission according to your cluster setup as 
+specified here: https://github.com/fission/fission
+
+
+### Create the class to run
+
+Secondly you need to create a file /tmp/func.cs containing the following code:
+
+```
+using System.IO;
+using System.Runtime.Serialization.Json;
+using Fission.DotNetCore.Api;
+
+public class FissionFunction
+{
+    public string Execute(FissionContext context)
+    {
+        var person = Person.Deserialize(context.Request.Body);
+        return $"Hello, my name is {person.Name} and I am {person.Age} years old.";
+    }
+}
+
+public class Person
+{
+    public string Name { get; set; }
+    public int Age { get; set; }
+
+    public static Person Deserialize(Stream json)
+    {
+        var serializer = new DataContractJsonSerializer(typeof(Person));
+        return (Person)serializer.ReadObject(json);
+    }
+}
+
+``` 
+### Run the example
+
+Lastly to run the example:
+
+```
+$ fission env create --name dotnet --image fission/dotnet20-env
+
+$ fission function create --name httpbody --env dotnet --code /tmp/func.cs
+
+$ fission route create --method GET --url /http_body --function httpbody
+
+$ curl -XPOST "http://$FISSION_ROUTER/http_body" -d '{ "Name":"Arthur", "Age":42}'
+Hello, my name is Arthur and I am 42 years old.
+
+```
+
+
+## Developing/debugging the enviroment locally
+
+The easiest way to debug the environment is to open the directory in
+Visual Studio Code (VSCode) as that will setup debugger for you the
+first time.
+
+Remember to install the excellent extension 
+"C# for Visual Studio Code(powered by OmniSharp)" to get statement completion
+
+The class ExecutorModule contain preprocessor directive overriding where 
+the input code file should be found:
+
+```
+#if DEBUG
+        private const string CODE_PATH = "/tmp/func.cs";
+#else
+        private const string CODE_PATH = "/userfunc/user";
+#endif
+```
+
+So what you need to do is:
+1. Open the directory in VSCode. 
+This will prompt restore of packages and query is debugger setup is needed. Accept both prompts.
+2. Press F5 to start the web server. Set breakpoints etc..
+3. Add a code file containing valid C# at /tmp/func.cs 
+4. Specialize the service with curl via post
+```
+$ curl -XPOST http://localhost:8888/specialize
+```
+5. Call your function with curl
+```
+$ curl -XGET http://localhost:8888
+``` 

environments/dotnet20/build.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+dotnet restore
+dotnet publish -c Release -o out
+docker build -t fission/dotnet20-env .

environments/dotnet20/dotnet20.sln

@@ -0,0 +1,25 @@
+
+Microsoft Visual Studio Solution File, Format Version 12.00
+# Visual Studio 15
+VisualStudioVersion = 15.0.26730.12
+MinimumVisualStudioVersion = 10.0.40219.1
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "fission-dotnet20", "dotnet20\fission-dotnet20.csproj", "{3F044DE1-74E5-48F7-8D23-233C24AAA45C}"
+EndProject
+Global
+	GlobalSection(SolutionConfigurationPlatforms) = preSolution
+		Debug|Any CPU = Debug|Any CPU
+		Release|Any CPU = Release|Any CPU
+	EndGlobalSection
+	GlobalSection(ProjectConfigurationPlatforms) = postSolution
+		{3F044DE1-74E5-48F7-8D23-233C24AAA45C}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{3F044DE1-74E5-48F7-8D23-233C24AAA45C}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{3F044DE1-74E5-48F7-8D23-233C24AAA45C}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{3F044DE1-74E5-48F7-8D23-233C24AAA45C}.Release|Any CPU.Build.0 = Release|Any CPU
+	EndGlobalSection
+	GlobalSection(SolutionProperties) = preSolution
+		HideSolutionNode = FALSE
+	EndGlobalSection
+	GlobalSection(ExtensibilityGlobals) = postSolution
+		SolutionGuid = {272000F7-BBEC-4B61-8865-783EF7E5CEE9}
+	EndGlobalSection
+EndGlobal

environments/dotnet20/fission-dotnet20.csproj

@@ -0,0 +1,17 @@
+<Project Sdk="Microsoft.NET.Sdk">
+
+  <PropertyGroup>
+    <OutputType>Exe</OutputType>
+    <TargetFramework>netcoreapp2.0</TargetFramework>
+  </PropertyGroup>
+
+  <ItemGroup>
+    <PackageReference Include="Microsoft.AspNetCore.Server.Kestrel" Version="2.0.0" />
+    <PackageReference Include="Microsoft.AspNetCore.Owin" Version="2.0.0" />
+    <PackageReference Include="Nancy" Version="2.0.0-clinteastwood" />
+    <PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="2.3.2" />
+    <PackageReference Include="System.Runtime.Loader" Version="4.3.0" />
+    <PackageReference Include="System.Runtime.Serialization.Json" Version="4.3.0" />
+  </ItemGroup>
+
+</Project>

environments/fetcher/client/client.go

@@ -3,6 +3,8 @@ package client
 import (
 	"bytes"
 	"encoding/json"
+	"io/ioutil"
+	"log"
 	"net/http"
 	//"time"
 
@@ -11,7 +13,19 @@ import (
 	//"github.com/fission/fission/router"
 )
 
-func DoFetchRequest(fetcherUrl string, fr *fetcher.FetchRequest) error {
+type (
+	Client struct {
+		url string
+	}
+)
+
+func MakeClient(fetcherUrl string) *Client {
+	return &Client{
+		url: fetcherUrl,
+	}
+}
+
+func (c *Client) Fetch(fr *fetcher.FetchRequest) error {
 	body, err := json.Marshal(fr)
 	if err != nil {
 		return err
@@ -21,7 +35,7 @@ func DoFetchRequest(fetcherUrl string, fr *fetcher.FetchRequest) error {
 	// 	Transport: router.MakeRetryingRoundTripper(10, 50*time.Millisecond),
 	// }
 
-	resp, err := http.Post(fetcherUrl, "application/json", bytes.NewReader(body))
+	resp, err := http.Post(c.url, "application/json", bytes.NewReader(body))
 	if err != nil {
 		return err
 	}
@@ -33,3 +47,34 @@ func DoFetchRequest(fetcherUrl string, fr *fetcher.FetchRequest) error {
 
 	return nil
 }
+
+func (c *Client) Upload(fr *fetcher.UploadRequest) (*fetcher.UploadResponse, error) {
+	body, err := json.Marshal(fr)
+	if err != nil {
+		return nil, err
+	}
+	resp, err := http.Post(c.url+"/upload", "application/json", bytes.NewReader(body))
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode != 200 {
+		return nil, fission.MakeErrorFromHTTP(resp)
+	}
+
+	rBody, err := ioutil.ReadAll(resp.Body)
+	if err != nil {
+		return nil, err
+	}
+
+	log.Printf("Received upload response: %v", string(rBody))
+
+	uploadReq := fetcher.UploadResponse{}
+	err = json.Unmarshal([]byte(rBody), &uploadReq)
+	if err != nil {
+		return nil, err
+	}
+
+	return &uploadReq, nil
+}

environments/fetcher/cmd/main.go

@@ -21,6 +21,7 @@ func main() {
 	}
 	fetcher := fetcher.MakeFetcher(dir)
 	mux := http.NewServeMux()
-	mux.HandleFunc("/", fetcher.Handler)
+	mux.HandleFunc("/", fetcher.FetchHandler)
+	mux.HandleFunc("/upload", fetcher.UploadHandler)
 	http.ListenAndServe(":8000", mux)
 }

environments/fetcher/fetcher.go

@@ -20,6 +20,7 @@ import (
 	"k8s.io/client-go/1.5/pkg/api"
 
 	"github.com/fission/fission"
+	storageSvcClient "github.com/fission/fission/storagesvc/client"
 	"github.com/fission/fission/tpr"
 )
 
@@ -28,12 +29,26 @@ type (
 
 	FetchRequest struct {
 		FetchType     FetchRequestType `json:"fetchType"`
-		Function      api.ObjectMeta   `json:"function"`
+		Package       api.ObjectMeta   `json:"package"`
 		Url           string           `json:"url"`
 		StorageSvcUrl string           `json:"storagesvcurl"`
 		Filename      string           `json:"filename"`
 	}
 
+	// UploadRequest send from builder manager describes which
+	// deployment package should be upload to storage service.
+	UploadRequest struct {
+		Filename      string `json:"filename"`
+		StorageSvcUrl string `json:"storagesvcurl"`
+	}
+
+	// UploadResponse defines the download url of an archive and
+	// its checksum.
+	UploadResponse struct {
+		ArchiveDownloadUrl string           `json:"archiveDownloadUrl"`
+		Checksum           fission.Checksum `json:"checksum"`
+	}
+
 	Fetcher struct {
 		sharedVolumePath string
 		fissionClient    *tpr.FissionClient
@@ -79,33 +94,44 @@ func downloadUrl(url string, localPath string) error {
 	return nil
 }
 
-func verifyChecksum(path string, checksum *fission.Checksum) error {
-	if checksum.Type != fission.ChecksumTypeSHA256 {
-		return fission.MakeError(fission.ErrorInvalidArgument, "Unsupported checksum type")
-	}
-
+func getChecksum(path string) (*fission.Checksum, error) {
 	f, err := os.Open(path)
 	if err != nil {
-		return err
+		return nil, err
 	}
 	defer f.Close()
 
 	hasher := sha256.New()
 	_, err = io.Copy(hasher, f)
 	if err != nil {
-		return err
+		return nil, err
 	}
 
 	c := hex.EncodeToString(hasher.Sum(nil))
-	if c != checksum.Sum {
+
+	return &fission.Checksum{
+		Type: fission.ChecksumTypeSHA256,
+		Sum:  c,
+	}, nil
+}
+
+func verifyChecksum(path string, checksum *fission.Checksum) error {
+	if checksum.Type != fission.ChecksumTypeSHA256 {
+		return fission.MakeError(fission.ErrorInvalidArgument, "Unsupported checksum type")
+	}
+	c, err := getChecksum(path)
+	if err != nil {
+		return err
+	}
+	if c.Sum != checksum.Sum {
 		return fission.MakeError(fission.ErrorChecksumFail, "Checksum validation failed")
 	}
 	return nil
 }
 
-func (fetcher *Fetcher) Handler(w http.ResponseWriter, r *http.Request) {
+func (fetcher *Fetcher) FetchHandler(w http.ResponseWriter, r *http.Request) {
 	if r.Method != "POST" {
-		http.Error(w, "", 405)
+		http.Error(w, "only POST is supported on this endpoint", 405)
 		return
 	}
 
@@ -129,11 +155,13 @@ func (fetcher *Fetcher) Handler(w http.ResponseWriter, r *http.Request) {
 		http.Error(w, err.Error(), 400)
 		return
 	}
-	log.Printf("fetcher received request: %v", req)
+	log.Printf("fetcher received fetch request: %v", req)
 
 	tmpFile := req.Filename + ".tmp"
 	tmpPath := filepath.Join(fetcher.sharedVolumePath, tmpFile)
 
+	log.Printf("Start downloading...")
+
 	if req.FetchType == FETCH_URL {
 		// fetch the file and save it to the tmp path
 		err := downloadUrl(req.Url, tmpPath)
@@ -144,19 +172,8 @@ func (fetcher *Fetcher) Handler(w http.ResponseWriter, r *http.Request) {
 			return
 		}
 	} else {
-		// get function object
-		fn, err := fetcher.fissionClient.Functions(req.Function.Namespace).Get(req.Function.Name)
-		if err != nil {
-			e := fmt.Sprintf("Failed to get function: %v", err)
-			log.Printf(e)
-			http.Error(w, e, 500)
-			return
-		}
-
 		// get pkg
-		var pkg *tpr.Package
-		pkg, err = fetcher.fissionClient.
-			Packages(fn.Spec.Package.PackageRef.Namespace).Get(fn.Spec.Package.PackageRef.Name)
+		pkg, err := fetcher.fissionClient.Packages(req.Package.Namespace).Get(req.Package.Name)
 		if err != nil {
 			e := fmt.Sprintf("Failed to get package: %v", err)
 			log.Printf(e)
@@ -200,7 +217,6 @@ func (fetcher *Fetcher) Handler(w http.ResponseWriter, r *http.Request) {
 				return
 			}
 		}
-
 	}
 
 	// check file type here, if the file is a zip file unarchive it.
@@ -229,6 +245,85 @@ func (fetcher *Fetcher) Handler(w http.ResponseWriter, r *http.Request) {
 	w.WriteHeader(http.StatusOK)
 }
 
+func (fetcher *Fetcher) UploadHandler(w http.ResponseWriter, r *http.Request) {
+	if r.Method != "POST" {
+		http.Error(w, "only POST is supported on this endpoint", 405)
+		return
+	}
+
+	startTime := time.Now()
+	defer func() {
+		elapsed := time.Now().Sub(startTime)
+		log.Printf("elapsed time in upload request = %v", elapsed)
+	}()
+
+	// parse request
+	body, err := ioutil.ReadAll(r.Body)
+	if err != nil {
+		log.Printf("Error reading request body")
+		http.Error(w, err.Error(), 500)
+		return
+	}
+
+	var req UploadRequest
+	err = json.Unmarshal(body, &req)
+	if err != nil {
+		log.Printf("Error reading request body: %v", err)
+		http.Error(w, err.Error(), 400)
+		return
+	}
+	log.Printf("fetcher received upload request: %v", req)
+
+	zipFilename := req.Filename + ".zip"
+	srcFilepath := filepath.Join(fetcher.sharedVolumePath, req.Filename)
+	dstFilepath := filepath.Join(fetcher.sharedVolumePath, zipFilename)
+
+	err = fetcher.archive(srcFilepath, dstFilepath)
+	if err != nil {
+		e := fmt.Sprintf("Error archiving zip file: %v", err)
+		log.Println(e)
+		http.Error(w, e, 500)
+		return
+	}
+
+	log.Println("Start uploading...")
+	ssClient := storageSvcClient.MakeClient(req.StorageSvcUrl)
+
+	fileID, err := ssClient.Upload(dstFilepath, nil)
+	if err != nil {
+		e := fmt.Sprintf("Error uploading zip file: %v", err)
+		log.Println(e)
+		http.Error(w, e, 500)
+		return
+	}
+
+	sum, err := getChecksum(dstFilepath)
+	if err != nil {
+		e := fmt.Sprintf("Error calculating checksum of zip file: %v", err)
+		log.Println(e)
+		http.Error(w, e, 500)
+		return
+	}
+
+	resp := UploadResponse{
+		ArchiveDownloadUrl: ssClient.GetUrl(fileID),
+		Checksum:           *sum,
+	}
+
+	rBody, err := json.Marshal(resp)
+	if err != nil {
+		e := fmt.Sprintf("Error encoding upload response: %v", err)
+		log.Println(e)
+		http.Error(w, e, 500)
+		return
+	}
+
+	log.Println("Completed upload request")
+	w.Header().Add("Content-Type", "application/json")
+	w.Write(rBody)
+	w.WriteHeader(http.StatusOK)
+}
+
 func (fetcher *Fetcher) rename(src string, dst string) error {
 	err := os.Rename(src, dst)
 	if err != nil {
@@ -239,7 +334,21 @@ func (fetcher *Fetcher) rename(src string, dst string) error {
 
 // archive is a function that zips directory into a zip file
 func (fetcher *Fetcher) archive(src string, dst string) error {
-	return archiver.Zip.Make(dst, []string{src})
+	var files []string
+	target, err := os.Stat(src)
+	if err != nil {
+		return errors.New(fmt.Sprintf("Failed to zip file: %v", err))
+	}
+	if target.IsDir() {
+		// list all
+		fs, _ := ioutil.ReadDir(src)
+		for _, f := range fs {
+			files = append(files, filepath.Join(src, f.Name()))
+		}
+	} else {
+		files = append(files, src)
+	}
+	return archiver.Zip.Make(dst, files)
 }
 
 // unarchive is a function that unzips a zip file to destination

environments/nodejs/Dockerfile

@@ -1,6 +1,6 @@
 # A docker image for the func container.
 
-FROM node:7-onbuild
+FROM node:8-onbuild
 
 ADD server.js /usr/src/app/server.js
 

environments/nodejs/package.json

@@ -13,12 +13,12 @@
     "node": ">=7.6.0"
   },
   "dependencies": {
-    "body-parser": "",
+    "body-parser": "*",
     "co": "~4.6.0",
-    "express": "",
-    "minimist": "",
-    "morgan": "",
-    "mz": "~2.1.0",
+    "express": "*",
+    "minimist": "*",
+    "morgan": "*",
+    "mz": "~2.7.0",
     "request": "^2.81.0",
     "request-promise-native": "^1.0.3",
     "underscore": ">=1.8.3"

environments/nodejs/server.js

@@ -66,6 +66,7 @@ app.use(morgan('combined'))
 app.use(bodyParser.urlencoded({ extended: false }));
 app.use(bodyParser.json());
 app.use(bodyParser.raw());
+app.use(bodyParser.text({ type : "text/*" }));
 
 app.post('/specialize', specialize);
 

environments/python3/builder/Dockerfile

@@ -6,6 +6,6 @@ RUN pip3 install --upgrade pip
 RUN rm -r /root/.cache
 
 ADD defaultBuildCmd /usr/local/bin/build
-ADD builder /
+ADD builder /builder
 
-EXPOSE 8000
+EXPOSE 8001

environments/python3/builder/build.sh

@@ -1,11 +0,0 @@
-#!/bin/sh
-set -e
-builderDir=${GOPATH}/src/github.com/fission/fission/builder/cmd
-pushd ${builderDir}
-GOOS=linux GOARCH=386 go build -o builder .
-popd
-cp ${builderDir}/builder .
-docker build -t python-builder .
-docker tag python-builder fission/python-builder:$tag
-docker push fission/python-builder:$tag
-

environments/python3/server.py

@@ -3,21 +3,35 @@
 import logging
 import sys
 import imp
+import os
 
 from flask import Flask, request, abort, g
 
 app = Flask(__name__)
 
-codepath = '/userfunc/user'
-
 userfunc = None
 
 @app.route('/specialize', methods=['POST'])
 def load():
     global userfunc
+    # load user function from codepath
+    codepath = '/userfunc/user'
     userfunc = (imp.load_source('user', codepath)).main
     return ""
 
+@app.route('/v2/specialize', methods=['POST'])
+def loadv2():
+    global userfunc
+    body = request.get_json()
+    filepath = body['filepath']
+    functionName = body['functionName']
+    # add filepath into syspath for module import
+    sys.path.append(filepath)
+    fn, path, desc = imp.find_module('user', [filepath])
+    mod = imp.load_module('user', fn, path, desc)
+    userfunc = getattr(mod, functionName)
+    return ""
+
 @app.route('/', methods=['GET', 'POST', 'PUT', 'HEAD', 'OPTIONS', 'DELETE'])
 def f():
     if userfunc == None:

examples/dotnet20/arguments.cs

@@ -0,0 +1,11 @@
+using System;
+using Fission.DotNetCore.Api;
+
+public class FissionFunction 
+{
+    public string Execute(FissionContext context){
+        var x = Convert.ToInt32(context.Arguments["x"]);
+        var y = Convert.ToInt32(context.Arguments["y"]);
+        return (x+y).ToString();
+    }
+

examples/dotnet20/echo.cs

@@ -0,0 +1,10 @@
+using System;
+using Fission.DotNetCore.Api;
+
+public class FissionFunction 
+{
+    public string Execute(FissionContext context){
+        context.Logger.WriteInfo("executing.. {0}", context.Arguments["text"]);
+        return (string)context.Arguments["text"];
+    }
+}

examples/dotnet20/helloworld.cs

@@ -0,0 +1,9 @@
+using Fission.DotNetCore.Api;
+
+public class FissionFunction
+{
+    public string Execute(FissionContext context)
+    {
+        return "Hello World!";
+    }
+}

examples/dotnet20/requestbody.cs

@@ -0,0 +1,24 @@
+using System.IO;
+using System.Runtime.Serialization.Json;
+using Fission.DotNetCore.Api;
+
+public class FissionFunction
+{
+    public string Execute(FissionContext context)
+    {
+        var person = Person.Deserialize(context.Request.Body);
+        return $"Hello, my name is {person.Name} and I am {person.Age} years old.";
+    }
+}
+
+public class Person
+{
+    public string Name { get; set; }
+    public int Age { get; set; }
+
+    public static Person Deserialize(Stream json)
+    {
+        var serializer = new DataContractJsonSerializer(typeof(Person));
+        return (Person)serializer.ReadObject(json);
+    }
+}

examples/dotnet20/requestheaders.cs

@@ -0,0 +1,17 @@
+using System;
+using Fission.DotNetCore.Api;
+
+public class FissionFunction
+{
+    public string Execute(FissionContext context){
+        var buffer = new System.Text.StringBuilder();
+        foreach(var header in context.Request.Headers){
+                buffer.AppendLine(header.Key);
+                foreach(var item in header.Value){
+                        buffer.AppendLine($"\t{item}");
+                }
+        }
+        buffer.AppendLine($"Url: {context.Request.Url}, method: {context.Request.Method}");
+        return buffer.ToString();
+    }
+}

examples/python/sourcepkg/build.sh

@@ -0,0 +1,2 @@
+#!/bin/sh
+pip3 install -r ${SRC_PKG}/requirements.txt -t ${SRC_PKG} && cp -r ${SRC_PKG} ${DEPLOY_PKG}

examples/python/sourcepkg/requirements.txt

@@ -0,0 +1 @@
+pyyaml

examples/python/sourcepkg/user.py

@@ -0,0 +1,12 @@
+import sys
+import yaml
+
+document = """
+  a: 1
+  b:
+    c: 3
+    d: 4
+"""
+
+def main():
+    return yaml.dump(yaml.load(document))

fission-bundle/build.sh

@@ -1,2 +1,2 @@
 #!/bin/sh
-CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build 
+CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build

fission-bundle/main.go

@@ -6,6 +6,7 @@ import (
 	"strconv"
 
 	"github.com/docopt/docopt-go"
+	"github.com/fission/fission/buildermgr"
 	"github.com/fission/fission/controller"
 	"github.com/fission/fission/kubewatcher"
 	"github.com/fission/fission/logger"
@@ -68,6 +69,13 @@ func runStorageSvc(port int, filePath string) {
 		filePath, subdir, port)
 }
 
+func runBuilderMgr(port int, storageSvcUrl string, envBuilderNamespace string) {
+	err := buildermgr.Start(port, storageSvcUrl, envBuilderNamespace)
+	if err != nil {
+		log.Fatalf("Error starting buildermgr: %v", err)
+	}
+}
+
 func getPort(portArg interface{}) int {
 	portArgStr := portArg.(string)
 	port, err := strconv.Atoi(portArgStr)
@@ -113,23 +121,26 @@ Usage:
   fission-bundle --poolmgrPort=<port> [--namespace=<namespace>] [--fission-namespace=<namespace>]
   fission-bundle --kubewatcher [--routerUrl=<url>]
   fission-bundle --storageServicePort=<port> --filePath=<filePath>
+  fission-bundle --builderMgrPort=<port> [--storageSvcUrl=<url>] [--envbuilder-namespace=<namespace>]
   fission-bundle --logger
   fission-bundle --timer [--routerUrl=<url>]
   fission-bundle --mqt   [--routerUrl=<url>]
 Options:
-  --controllerPort=<port>     Port that the controller should listen on.
-  --routerPort=<port>         Port that the router should listen on.
-  --poolmgrPort=<port>        Port that the poolmgr should listen on.
-  --storageServicePort=<port> Port that the storage service should listen on.
-  --poolmgrUrl=<url>          Poolmgr URL. Not required if --poolmgrPort is specified.
-  --routerUrl=<url>           Router URL.
-  --etcdUrl=<etcdUrl>         Etcd URL.
-  --filePath=<filePath>       Directory to store functions in.
-  --namespace=<namespace>     Kubernetes namespace in which to run function containers. Defaults to 'fission-function'.
-  --kubewatcher               Start Kubernetes events watcher.
-  --logger                    Start logger.
-  --timer 		      Start Timer.
-  --mqt 		      Start message queue trigger.
+  --controllerPort=<port>         Port that the controller should listen on.
+  --routerPort=<port>             Port that the router should listen on.
+  --poolmgrPort=<port>            Port that the poolmgr should listen on.
+  --storageServicePort=<port>     Port that the storage service should listen on.
+  --builderMgrPort=<port>         Port that the buildermgr should listen on.
+  --poolmgrUrl=<url>              Poolmgr URL. Not required if --poolmgrPort is specified.
+  --routerUrl=<url>               Router URL.
+  --etcdUrl=<etcdUrl>             Etcd URL.
+  --storageSvcUrl=<url>           StorageService URL.
+  --filePath=<filePath>           Directory to store functions in.
+  --namespace=<namespace>         Kubernetes namespace in which to run function containers. Defaults to 'fission-function'.
+  --kubewatcher                   Start Kubernetes events watcher.
+  --logger                        Start logger.
+  --timer                         Start Timer.
+  --mqt                           Start message queue trigger.
 `
 	arguments, err := docopt.Parse(usage, nil, true, "fission-bundle", false)
 	if err != nil {
@@ -138,9 +149,11 @@ Options:
 
 	functionNs := getStringArgWithDefault(arguments["--namespace"], "fission-function")
 	fissionNs := getStringArgWithDefault(arguments["--fission-namespace"], "fission")
+	envBuilderNs := getStringArgWithDefault(arguments["--envbuilder-namespace"], "fission-builder")
 
 	poolmgrUrl := getStringArgWithDefault(arguments["--poolmgrUrl"], "http://poolmgr.fission")
 	routerUrl := getStringArgWithDefault(arguments["--routerUrl"], "http://router.fission")
+	storageSvcUrl := getStringArgWithDefault(arguments["--storageSvcUrl"], "http://storagesvc.fission")
 
 	if arguments["--controllerPort"] != nil {
 		port := getPort(arguments["--controllerPort"])
@@ -179,5 +192,10 @@ Options:
 		runStorageSvc(port, filePath)
 	}
 
+	if arguments["--builderMgrPort"] != nil {
+		port := getPort(arguments["--builderMgrPort"])
+		runBuilderMgr(port, storageSvcUrl, envBuilderNs)
+	}
+
 	select {}
 }

fission-cloud.yaml

@@ -1,69 +0,0 @@
-apiVersion: v1
-kind: Service
-metadata:
-  name: router
-  namespace: fission
-  labels:
-    svc: router
-spec:
-  type: LoadBalancer
-  ports:
-  - port: 80
-    targetPort: 8888
-    nodePort: 31314
-  selector:
-    svc: router
-
----
-apiVersion: v1
-kind: Service
-metadata:
-  name: controller
-  namespace: fission
-  labels:
-    svc: controller
-spec:
-  type: LoadBalancer
-  ports:
-  - port: 80
-    targetPort: 8888
-    nodePort: 31313
-  selector:
-    svc: controller
-
----
-# If you want to switch to external db service, please checkout 
-# sample Kubernetes Service config in fission-logger.yaml.
-apiVersion: v1
-kind: Service
-metadata:
-  name: influxdb
-  namespace: fission
-  labels:
-    svc: influxdb
-spec:
-  type: LoadBalancer
-  ports:
-  - port: 8086
-    targetPort: 8086
-    nodePort: 31315
-  selector:
-    svc: influxdb
-
----
-
-apiVersion: v1
-kind: Service
-metadata:
-  name: nats-streaming
-  namespace: fission
-  labels:
-    svc: nats-streaming
-spec:
-  type: LoadBalancer
-  ports:
-  - port: 4222
-    targetPort: 4222
-    nodePort: 31316
-  selector:
-    svc: nats-streaming

fission-logger.yaml

@@ -1,122 +0,0 @@
-# Sample Kubernetes Service config for external db service. 
-#
-# apiVersion: v1
-# kind: Service
-# metadata:
-#   name: influxdb
-#   namespace: fission
-# spec:
-#   ports:
-#   - name: "8086"
-#     port: 8086
-#     targetPort: 8086
-#     protocol: TCP
-
-# ---
-# apiVersion: v1
-# kind: Endpoints
-# metadata:
-#   name: influxdb
-#   namespace: fission
-# subsets:
-#   - addresses:
-#     - ip: replace.influxdb.host.here
-#     ports:
-#     - name: "8086"
-#       port: 8086
-#       protocol: TCP
-
-apiVersion: extensions/v1beta1
-kind: Deployment
-metadata:
-  name: influxdb
-  namespace: fission
-spec:
-  replicas: 1
-  template:
-    metadata:
-      labels:
-        svc: influxdb
-    spec:
-      containers:
-      - name: influxdb
-        image: tutum/influxdb
-        env:
-          - name: PRE_CREATE_DB
-            value: fissionFunctionLog
-          # Create a random username/password for InfluxDB here, and
-          # repeat it in the fluentd spec below.
-          - name: ADMIN_USER
-            value: ""
-          - name: INFLUXDB_INIT_PWD
-            value: ""
-
----
-apiVersion: extensions/v1beta1
-kind: DaemonSet
-metadata:
-  name: logger
-  namespace: fission
-spec:
-  template:
-    metadata:
-      labels:
-        svc: logger
-    spec:
-      containers:
-        - name: logger
-          image: fission/fission-bundle
-          imagePullPolicy: IfNotPresent
-          command: ["/fission-bundle"]
-          args: ["--logger"]
-          volumeMounts:
-            - name: container-log
-              mountPath: /var/log/containers
-              readOnly: true
-            - name: docker-log
-              mountPath: /var/lib/docker/containers
-              readOnly: true
-            - name: fission-log
-              mountPath: /var/log/fission
-              readOnly: false
-          ports:
-            - containerPort: 1234
-              hostPort: 1234
-              protocol: TCP
-        - name: fluentd
-          image: fission/fluentd
-          imagePullPolicy: IfNotPresent
-          env:
-            - name: INFLUXDB_ADDRESS
-              value: influxdb
-            - name: INFLUXDB_PORT
-              value: "8086"
-            - name: INFLUXDB_DBNAME
-              value: "fissionFunctionLog"
-            # Username/password for fluentd to push logs to
-            # influxdb. This should match the influxdb
-            # username/passowrd defined above.
-            - name: INFLUXDB_USERNAME
-              value: ""
-            - name: INFLUXDB_PASSWD
-              value: ""
-          volumeMounts:
-            - name: container-log
-              mountPath: /var/log/containers
-              readOnly: true
-            - name: docker-log
-              mountPath: /var/lib/docker/containers
-              readOnly: true
-            - name: fission-log
-              mountPath: /var/log/fission
-              readOnly: false
-      volumes:
-        - name: container-log
-          hostPath:
-              path: /var/log/containers
-        - name: docker-log
-          hostPath:
-              path: /var/lib/docker/containers
-        - name: fission-log
-          hostPath:
-              path: /var/log/fission

fission-nats.yaml

@@ -1,60 +0,0 @@
-# To enable nats authentication, please follow the instruction described in
-# http://nats.io/documentation/server/gnatsd-authentication/.
-# And dont forget to change the MESSAGE_QUEUE_URL in mqtrigger deployment.
-
-apiVersion: extensions/v1beta1
-kind: Deployment
-metadata:
-  name: mqtrigger
-  namespace: fission
-spec:
-  replicas: 1
-  template:
-    metadata:
-      labels:
-        svc: mqtrigger
-    spec:
-      containers:
-      - name: mqtrigger
-        image: fission/fission-bundle
-        command: ["/fission-bundle"]
-        args: ["--mqt"]
-        env:
-        - name: MESSAGE_QUEUE_TYPE
-          value: nats-streaming
-        - name: MESSAGE_QUEUE_URL
-          value: nats://nats-streaming:4222
-
----
-apiVersion: extensions/v1beta1
-kind: Deployment
-metadata:
-  labels:
-    svc: nats-streaming
-  name: nats-streaming
-  namespace: fission
-spec:
-  replicas: 1
-  selector:
-    matchLabels:
-      svc: nats-streaming
-  strategy:
-    rollingUpdate:
-      maxSurge: 1
-      maxUnavailable: 1
-    type: RollingUpdate
-  template:
-    metadata:
-      creationTimestamp: null
-      labels:
-        svc: nats-streaming
-    spec:
-      containers:
-      - name: nats-streaming
-        image: nats-streaming
-        args: ["--cluster_id", "fissionMQTrigger"]
-        ports:
-        - containerPort: 4222
-          hostPort: 4222
-          protocol: TCP
-

fission-nodeport.yaml

@@ -1,69 +0,0 @@
-apiVersion: v1
-kind: Service
-metadata:
-  name: router
-  namespace: fission
-  labels:
-    svc: router
-spec:
-  type: NodePort
-  ports:
-  - port: 80
-    targetPort: 8888
-    nodePort: 31314
-  selector:
-    svc: router
-
----
-apiVersion: v1
-kind: Service
-metadata:
-  name: controller
-  namespace: fission
-  labels:
-    svc: controller
-spec:
-  type: NodePort
-  ports:
-  - port: 80
-    targetPort: 8888
-    nodePort: 31313
-  selector:
-    svc: controller
-
----
-# If you want to switch to external db service, please checkout 
-# sample Kubernetes Service config in fission-logger.yaml.
-apiVersion: v1
-kind: Service
-metadata:
-  name: influxdb
-  namespace: fission
-  labels:
-    svc: influxdb
-spec:
-  type: NodePort
-  ports:
-  - port: 8086
-    targetPort: 8086
-    nodePort: 31315
-  selector:
-    svc: influxdb
-
----
-
-apiVersion: v1
-kind: Service
-metadata:
-  name: nats-streaming
-  namespace: fission
-  labels:
-    svc: nats-streaming
-spec:
-  type: NodePort
-  ports:
-  - port: 4222
-    targetPort: 4222
-    nodePort: 31316
-  selector:
-    svc: nats-streaming

fission-openshift.yaml

@@ -1,202 +0,0 @@
----
-apiVersion: v1
-kind: ProjectRequest
-metadata:
-  name: fission
-  labels:
-    name: fission
-
----
-apiVersion: v1
-kind: ProjectRequest
-metadata:
-  name: fission-function
-  labels:
-    name: fission-function
-
----
-apiVersion: v1
-kind: ServiceAccount
-metadata:
-  name: poolmgr
-  namespace: fission
-
----
-apiVersion: v1
-kind: ClusterRole
-metadata:
-  name: fission:poolmgr
-rules:
-- apiGroups:
-  - extensions
-  attributeRestrictions: null
-  resources:
-  - deployments
-  verbs:
-  - create
-  - get
-  - list
-  - update
-  - watch
-- apiGroups:
-  - ""
-  attributeRestrictions: null
-  resources:
-  - pods
-  verbs:
-  - get
-  - list
-  - update
-
----
-apiVersion: extensions/v1beta1
-kind: Deployment
-metadata:
-  name: controller
-  namespace: fission
-spec:
-  replicas: 1
-  template:
-    metadata:
-      labels:
-        svc: controller
-    spec:
-      volumes:
-      - name: controller-storage
-        emptyDir: {}
-      containers:
-      - name: controller
-        image: fission/fission-bundle
-        command: ["/fission-bundle"]
-        args: ["--controllerPort", "8888", "--filepath", "/filestore"]
-        volumeMounts:
-          - name: controller-storage
-            mountPath: /filestore
-
----
-apiVersion: extensions/v1beta1
-kind: Deployment
-metadata:
-  name: router
-  namespace: fission
-spec:
-  replicas: 1
-  template:
-    metadata:
-      labels:
-        svc: router
-    spec:
-      containers:
-      - name: router
-        image: fission/fission-bundle
-        command: ["/fission-bundle"]
-        args: ["--routerPort", "8888"]
-
----
-apiVersion: v1
-kind: Service
-metadata:
-  name: poolmgr
-  namespace: fission
-  labels:
-    svc: poolmgr
-spec:
-  ports:
-  - port: 80
-    targetPort: 8888
-  selector:
-    svc: poolmgr
-
----
-apiVersion: extensions/v1beta1
-kind: Deployment
-metadata:
-  name: poolmgr
-  namespace: fission
-spec:
-  replicas: 1
-  template:
-    metadata:
-      labels:
-        svc: poolmgr
-    spec:
-      containers:
-      - name: poolmgr
-        image: fission/fission-bundle
-        command: ["/fission-bundle"]
-        args: ["--poolmgrPort", "8888"]
-      serviceAccount: poolmgr
-
----
-apiVersion: extensions/v1beta1
-kind: Deployment
-metadata:
-  name: kubewatcher
-  namespace: fission
-spec:
-  replicas: 1
-  template:
-    metadata:
-      labels:
-        svc: kubewatcher
-    spec:
-      containers:
-      - name: kubewatcher
-        image: fission/fission-bundle
-        command: ["/fission-bundle"]
-        args: ["--kubewatcher"]
-
----
-apiVersion: v1
-kind: Service
-metadata:
-  name: etcd
-  namespace: fission
-  labels:
-    svc: etcd
-spec:
-  ports:
-  - port: 2379
-    targetPort: 2379
-  selector:
-    svc: etcd
-
----
-apiVersion: extensions/v1beta1
-kind: Deployment
-metadata:
-  name: etcd
-  namespace: fission
-spec:
-  replicas: 1
-  template:
-    metadata:
-      labels:
-        svc: etcd
-    spec:
-      containers:
-      - name: etcd
-        image: quay.io/coreos/etcd
-        env:
-        - name: ETCD_LISTEN_CLIENT_URLS
-          value: http://0.0.0.0:2379
-        - name: ETCD_ADVERTISE_CLIENT_URLS
-          value: http://etcd:2379
-        - name: ETCD_DATA_DIR
-          value: /tmp/default.etcd
-
----
-apiVersion: v1
-groupNames: null
-kind: RoleBinding
-metadata:
-  name: fission:poolmgr
-  namespace: fission-function
-roleRef:
-  name: fission:poolmgr
-subjects:
-- kind: ServiceAccount
-  name: poolmgr
-  namespace: fission
-userNames:
-- system:serviceaccount:fission:poolmgr

fission-rbac.yaml

@@ -1,159 +0,0 @@
-apiVersion: v1
-kind: Namespace
-metadata:
-  name: fission
-  labels:
-    name: fission
----
-apiVersion: v1
-kind: Namespace
-metadata:
-  name: fission-function
-  labels:
-    name: fission-function
----
-apiVersion: v1
-kind: ServiceAccount
-metadata:
-  name: fission-svc
-  namespace: fission
----
-kind: RoleBinding
-apiVersion: rbac.authorization.k8s.io/v1alpha1
-metadata:
-  name: fission-admin
-  namespace: fission
-subjects:
-  - kind: ServiceAccount
-    name: fission-svc
-    namespace: fission
-roleRef:
-  kind: ClusterRole
-  name: admin
-  apiGroup: rbac.authorization.k8s.io
----
-kind: RoleBinding
-apiVersion: rbac.authorization.k8s.io/v1alpha1
-metadata:
-  name: fission-function-admin
-  namespace: fission-function
-subjects:
-  - kind: ServiceAccount
-    name: fission-svc
-    namespace: fission
-roleRef:
-  kind: ClusterRole
-  name: admin
-  apiGroup: rbac.authorization.k8s.io
----
-apiVersion: v1
-kind: ServiceAccount
-metadata:
-  name: fission-fetcher
-  namespace: fission-function
----
-kind: ClusterRoleBinding
-apiVersion: rbac.authorization.k8s.io/v1beta1
-metadata:
-  name: fission-fetcher-tpr
-subjects:
-- kind: ServiceAccount
-  name: fission-fetcher
-  namespace: fission-function
-roleRef:
-  kind: ClusterRole
-  name: cluster-admin
-  apiGroup: rbac.authorization.k8s.io
----
-apiVersion: extensions/v1beta1
-kind: Deployment
-metadata:
-  name: controller
-  namespace: fission
-spec:
-  replicas: 1
-  template:
-    metadata:
-      labels:
-        svc: controller
-    spec:
-      containers:
-        - name: controller
-          image: fission/fission-bundle
-          command: ["/fission-bundle"]
-          args: ["--controllerPort", "8888"]
-      serviceAccountName: fission-svc
----
-apiVersion: extensions/v1beta1
-kind: Deployment
-metadata:
-  name: router
-  namespace: fission
-spec:
-  replicas: 1
-  template:
-    metadata:
-      labels:
-        svc: router
-    spec:
-      containers:
-      - name: router
-        image: fission/fission-bundle
-        command: ["/fission-bundle"]
-        args: ["--routerPort", "8888"]
-      serviceAccountName: fission-svc
----
-apiVersion: v1
-kind: Service
-metadata:
-  name: poolmgr
-  namespace: fission
-  labels:
-    svc: poolmgr
-spec:
-  ports:
-  - port: 80
-    targetPort: 8888
-  selector:
-    svc: poolmgr
----
-apiVersion: extensions/v1beta1
-kind: Deployment
-metadata:
-  name: poolmgr
-  namespace: fission
-spec:
-  replicas: 1
-  template:
-    metadata:
-      labels:
-        svc: poolmgr
-    spec:
-      containers:
-      - name: poolmgr
-        image: fission/fission-bundle
-        command: ["/fission-bundle"]
-        args: ["--poolmgrPort", "8888"]
-        env:
-        - name: FETCHER_IMAGE
-          value: fission/fetcher:v2
-      serviceAccountName: fission-svc
----
-apiVersion: extensions/v1beta1
-kind: Deployment
-metadata:
-  name: kubewatcher
-  namespace: fission
-spec:
-  replicas: 1
-  template:
-    metadata:
-      labels:
-        svc: kubewatcher
-    spec:
-      containers:
-      - name: kubewatcher
-        image: fission/fission-bundle
-        command: ["/fission-bundle"]
-        args: ["--kubewatcher"]
-      serviceAccountName: fission-svc

fission.yaml

@@ -1,127 +0,0 @@
-apiVersion: v1
-kind: Namespace
-metadata:
-  name: fission
-  labels:
-    name: fission
-
----
-apiVersion: v1
-kind: Namespace
-metadata:
-  name: fission-function
-  labels:
-    name: fission-function
-
----
-apiVersion: extensions/v1beta1
-kind: Deployment
-metadata:
-  name: controller
-  namespace: fission
-spec:
-  replicas: 1
-  template:
-    metadata:
-      labels:
-        svc: controller
-    spec:
-      containers:
-      - name: controller
-        image: fission/fission-bundle
-        command: ["/fission-bundle"]
-        args: ["--controllerPort", "8888"]
-
----
-apiVersion: extensions/v1beta1
-kind: Deployment
-metadata:
-  name: router
-  namespace: fission
-spec:
-  replicas: 1
-  template:
-    metadata:
-      labels:
-        svc: router
-    spec:
-      containers:
-      - name: router
-        image: fission/fission-bundle
-        command: ["/fission-bundle"]
-        args: ["--routerPort", "8888"]
-
----
-apiVersion: v1
-kind: Service
-metadata:
-  name: poolmgr
-  namespace: fission
-  labels:
-    svc: poolmgr
-spec:
-  ports:
-  - port: 80
-    targetPort: 8888
-  selector:
-    svc: poolmgr
-
----
-apiVersion: extensions/v1beta1
-kind: Deployment
-metadata:
-  name: poolmgr
-  namespace: fission
-spec:
-  replicas: 1
-  template:
-    metadata:
-      labels:
-        svc: poolmgr
-    spec:
-      containers:
-      - name: poolmgr
-        image: fission/fission-bundle
-        command: ["/fission-bundle"]
-        args: ["--poolmgrPort", "8888"]
-        env:
-        - name: FETCHER_IMAGE
-          value: fission/fetcher:v2
-
----
-apiVersion: extensions/v1beta1
-kind: Deployment
-metadata:
-  name: kubewatcher
-  namespace: fission
-spec:
-  replicas: 1
-  template:
-    metadata:
-      labels:
-        svc: kubewatcher
-    spec:
-      containers:
-      - name: kubewatcher
-        image: fission/fission-bundle
-        command: ["/fission-bundle"]
-        args: ["--kubewatcher"]
-
----
-apiVersion: extensions/v1beta1
-kind: Deployment
-metadata:
-  name: timer
-  namespace: fission
-spec:
-  replicas: 1
-  template:
-    metadata:
-      labels:
-        svc: timer
-    spec:
-      containers:
-      - name: timer
-        image: fission/fission-bundle
-        command: ["/fission-bundle"]
-        args: ["--timer"]

fission/environment.go

@@ -18,7 +18,6 @@ package main
 
 import (
 	"fmt"
-	"log"
 	"os"
 	"text/tabwriter"
 
@@ -42,12 +41,22 @@ func envCreate(c *cli.Context) error {
 		fatal("Need an image, use --image.")
 	}
 
+	envVersion := c.Int("version")
 	envBuilderImg := c.String("builder")
-
 	envBuildCmd := c.String("buildcmd")
-	if len(envBuilderImg) > 0 && len(envBuildCmd) == 0 {
-		log.Printf("No build command is specified, use the default build command.")
-		envBuildCmd = "build"
+
+	if len(envBuilderImg) > 0 {
+		envVersion = 2
+		if len(envBuildCmd) == 0 {
+			envBuildCmd = "build"
+		}
+	}
+
+	// Environment API interface version is not specified and
+	// builder image is empty, set default interface version
+	if envVersion == 0 {
+		fmt.Println("Use default environment v1 API interface")
+		envVersion = 1
 	}
 
 	env := &tpr.Environment{
@@ -56,7 +65,7 @@ func envCreate(c *cli.Context) error {
 			Namespace: api.NamespaceDefault,
 		},
 		Spec: fission.EnvironmentSpec{
-			Version: 1,
+			Version: envVersion,
 			Runtime: fission.Runtime{
 				Image: envImg,
 			},
@@ -121,6 +130,11 @@ func envUpdate(c *cli.Context) error {
 	if len(envImg) > 0 {
 		env.Spec.Runtime.Image = envImg
 	}
+
+	if env.Spec.Version == 1 && (len(envBuilderImg) > 0 || len(envBuildCmd) > 0) {
+		fatal("Environment v1 API interface doesn't supported environment builder.")
+	}
+
 	if len(envBuilderImg) > 0 {
 		env.Spec.Builder.Image = envBuilderImg
 	}

fission/function.go

@@ -18,14 +18,16 @@ package main
 
 import (
 	"context"
+	"crypto/sha256"
+	"encoding/hex"
 	"fmt"
+	"io"
 	"io/ioutil"
 	"os"
 	"strings"
 	"text/tabwriter"
 	"time"
 
-	"github.com/dchest/uniuri"
 	"github.com/satori/go.uuid"
 	"github.com/urfave/cli"
 	"k8s.io/client-go/1.5/pkg/api"
@@ -62,10 +64,67 @@ func createArchive(client *client.Client, fileName string) *fission.Archive {
 
 		archive.Type = fission.ArchiveTypeUrl
 		archive.URL = archiveUrl
+
+		f, err := os.Open(fileName)
+		if err != nil {
+			checkErr(err, fmt.Sprintf("find file %v", fileName))
+		}
+		defer f.Close()
+
+		h := sha256.New()
+		if _, err := io.Copy(h, f); err != nil {
+			checkErr(err, fmt.Sprintf("calculate checksum for file %v", fileName))
+		}
+
+		archive.Checksum = fission.Checksum{
+			Type: fission.ChecksumTypeSHA256,
+			Sum:  hex.EncodeToString(h.Sum(nil)),
+		}
 	}
 	return &archive
 }
 
+func createPackage(client *client.Client, envName, srcArchiveName, deployArchiveName, buildcmd string) *api.ObjectMeta {
+	pkgSpec := fission.PackageSpec{
+		Environment: fission.EnvironmentReference{
+			Namespace: api.NamespaceDefault,
+			Name:      envName,
+		},
+	}
+	var pkgStatus fission.BuildStatus = fission.BuildStatusSucceeded
+
+	if len(deployArchiveName) > 0 {
+		pkgSpec.Deployment = *createArchive(client, deployArchiveName)
+		if len(srcArchiveName) > 0 {
+			fmt.Println("Deployment may be overwritten by builder manager after source package compilation")
+		}
+	}
+	if len(srcArchiveName) > 0 {
+		pkgSpec.Source = *createArchive(client, srcArchiveName)
+		// set pending status to package
+		pkgStatus = fission.BuildStatusPending
+	}
+
+	if len(buildcmd) > 0 {
+		pkgSpec.BuildCommand = buildcmd
+	}
+
+	pkgName := strings.ToLower(uuid.NewV4().String())
+	pkg := &tpr.Package{
+		Metadata: api.ObjectMeta{
+			Name:      pkgName,
+			Namespace: api.NamespaceDefault,
+		},
+		Spec: pkgSpec,
+		Status: fission.PackageStatus{
+			BuildStatus: pkgStatus,
+		},
+	}
+	pkgMetadata, err := client.PackageCreate(pkg)
+	checkErr(err, "create package")
+	return pkgMetadata
+}
+
 func getContents(filePath string) []byte {
 	var code []byte
 	var err error
@@ -78,6 +137,10 @@ func getContents(filePath string) []byte {
 func fnCreate(c *cli.Context) error {
 	client := getClient(c.GlobalString("server"))
 
+	if len(c.String("package")) > 0 {
+		fatal("--package is deprecated, please use --deploy instead.")
+	}
+
 	fnName := c.String("name")
 	if len(fnName) == 0 {
 		fatal("Need --name argument.")
@@ -88,17 +151,24 @@ func fnCreate(c *cli.Context) error {
 		fatal("Need --env argument.")
 	}
 
-	srcPkgName := c.String("srcpkg")
-
-	deployPkgName := c.String("code")
-	if len(deployPkgName) == 0 {
-		deployPkgName = c.String("package")
+	srcArchiveName := c.String("src")
+	deployArchiveName := c.String("code")
+	if len(deployArchiveName) == 0 {
+		deployArchiveName = c.String("deploy")
 	}
 
-	if len(srcPkgName) == 0 && len(deployPkgName) == 0 {
-		fatal("Need --code or --package to specify deployment package, or use --srcpkg to specify source package.")
+	if len(srcArchiveName) == 0 && len(deployArchiveName) == 0 {
+		fatal("Need --code or --deploy to specify deployment archive, or use --src to specify source archive.")
 	}
 
+	entrypoint := c.String("entrypoint")
+	buildcmd := c.String("buildcmd")
+	if len(buildcmd) == 0 {
+		buildcmd = "/builder"
+	}
+
+	pkgMetadata := createPackage(client, envName, srcArchiveName, deployArchiveName, buildcmd)
+
 	function := &tpr.Function{
 		Metadata: api.ObjectMeta{
 			Name:      fnName,
@@ -109,35 +179,18 @@ func fnCreate(c *cli.Context) error {
 				Name:      envName,
 				Namespace: api.NamespaceDefault,
 			},
+			Package: fission.FunctionPackageRef{
+				FunctionName: entrypoint,
+				PackageRef: fission.PackageRef{
+					Namespace:       pkgMetadata.Namespace,
+					Name:            pkgMetadata.Name,
+					ResourceVersion: pkgMetadata.ResourceVersion,
+				},
+			},
 		},
 	}
 
-	var pkgSpec fission.PackageSpec
-	if len(srcPkgName) > 0 {
-		pkgSpec.Source = *createArchive(client, srcPkgName)
-	}
-	if len(deployPkgName) > 0 {
-		pkgSpec.Deployment = *createArchive(client, deployPkgName)
-	}
-	pkgName := fmt.Sprintf("%v-%v", fnName, strings.ToLower(uniuri.NewLen(6)))
-	pkg := &tpr.Package{
-		Metadata: api.ObjectMeta{
-			Name:      pkgName,
-			Namespace: api.NamespaceDefault,
-		},
-		Spec: pkgSpec,
-	}
-	newpkg, err := client.PackageCreate(pkg)
-	checkErr(err, "create package")
-
-	function.Spec.Package = fission.FunctionPackageRef{
-		PackageRef: fission.PackageRef{
-			Name:            newpkg.Name,
-			Namespace:       newpkg.Namespace,
-			ResourceVersion: newpkg.ResourceVersion,
-		},
-	}
-	_, err = client.FunctionCreate(function)
+	_, err := client.FunctionCreate(function)
 	checkErr(err, "create function")
 
 	fmt.Printf("function '%v' created\n", fnName)
@@ -229,6 +282,10 @@ func fnUpdate(c *cli.Context) error {
 		fatal("Need name of function, use --name")
 	}
 
+	if len(c.String("package")) > 0 {
+		fatal("--package is deprecated, please use --deploy instead.")
+	}
+
 	function, err := client.FunctionGet(&api.ObjectMeta{
 		Name:      fnName,
 		Namespace: api.NamespaceDefault,
@@ -236,42 +293,50 @@ func fnUpdate(c *cli.Context) error {
 	checkErr(err, fmt.Sprintf("read function '%v'", fnName))
 
 	envName := c.String("env")
-	deployPkgName := c.String("code")
-	if len(deployPkgName) == 0 {
-		deployPkgName = c.String("package")
+	deployArchiveName := c.String("code")
+	if len(deployArchiveName) == 0 {
+		deployArchiveName = c.String("deploy")
 	}
-	srcPkgName := c.String("srcpkg")
+	srcArchiveName := c.String("src")
 
-	if len(envName) == 0 && len(deployPkgName) == 0 && len(srcPkgName) == 0 {
-		fatal("Need --env or --code or --package or --srcpkg argument.")
-	}
-
-	if len(deployPkgName) != 0 || len(srcPkgName) != 0 {
-		// get existing package
-		pkg, err := client.PackageGet(&api.ObjectMeta{
-			Name:      function.Spec.Package.PackageRef.Name,
-			Namespace: function.Spec.Package.PackageRef.Namespace,
-		})
-		// update package spec
-		if len(srcPkgName) > 0 {
-			archive := createArchive(client, srcPkgName)
-			pkg.Spec.Source = *archive
-		}
-		if len(deployPkgName) > 0 {
-			archive := createArchive(client, deployPkgName)
-			pkg.Spec.Deployment = *archive
-		}
-		// updage package object
-		newpkg, err := client.PackageUpdate(pkg)
-		checkErr(err, "update package")
-		// update function spec with resource version
-		function.Spec.Package.PackageRef.ResourceVersion = newpkg.ResourceVersion
+	if len(envName) == 0 && len(deployArchiveName) == 0 && len(srcArchiveName) == 0 {
+		fatal("Need --env or --code or --package or --deploy argument.")
 	}
 
 	if len(envName) > 0 {
 		function.Spec.Environment.Name = envName
 	}
 
+	entrypoint := c.String("entrypoint")
+	if len(entrypoint) > 0 {
+		function.Spec.Package.FunctionName = entrypoint
+	}
+
+	pkg, err := client.PackageGet(&api.ObjectMeta{
+		Name:      function.Spec.Package.PackageRef.Name,
+		Namespace: function.Spec.Package.PackageRef.Namespace,
+	})
+	checkErr(err, fmt.Sprintf("read package '%v'", function.Spec.Package.PackageRef.Name))
+
+	buildcmd := c.String("buildcmd")
+	if len(buildcmd) == 0 {
+		// use previous build command if not specified.
+		buildcmd = pkg.Spec.BuildCommand
+	}
+
+	if len(deployArchiveName) > 0 || len(srcArchiveName) > 0 {
+		// create a new package for function
+		pkgMetadata := createPackage(client,
+			function.Spec.Environment.Name, srcArchiveName, deployArchiveName, buildcmd)
+
+		// update function spec with resource version
+		function.Spec.Package.PackageRef = fission.PackageRef{
+			Namespace:       pkgMetadata.Namespace,
+			Name:            pkgMetadata.Name,
+			ResourceVersion: pkgMetadata.ResourceVersion,
+		}
+	}
+
 	_, err = client.FunctionUpdate(function)
 	checkErr(err, "update function")
 

fission/main.go

@@ -26,7 +26,7 @@ func main() {
 	app := cli.NewApp()
 	app.Name = "fission"
 	app.Usage = "Serverless functions for Kubernetes"
-	app.Version = "0.2.1"
+	app.Version = "0.3.0-rc"
 
 	app.Flags = []cli.Flag{
 		cli.StringFlag{Name: "server", Usage: "Fission server URL", EnvVar: "FISSION_URL"},
@@ -40,17 +40,20 @@ func main() {
 	fnNameFlag := cli.StringFlag{Name: "name", Usage: "function name"}
 	fnEnvNameFlag := cli.StringFlag{Name: "env", Usage: "environment name for function"}
 	fnCodeFlag := cli.StringFlag{Name: "code", Usage: "local path or URL for source code"}
-	fnPackageFlag := cli.StringFlag{Name: "package", Usage: "local path or URL for binary package"}
-	fnSrcPackageFlag := cli.StringFlag{Name: "srcpkg", Usage: "local path or URL for source package"}
+	fnPackageFlag := cli.StringFlag{Name: "package", Usage: "(Deprecated) local path or URL for binary package"}
+	fnDeployArchiveFlag := cli.StringFlag{Name: "deployarchive, deploy", Usage: "local path or URL for deployment archive"}
+	fnSrcArchiveFlag := cli.StringFlag{Name: "sourcearchive, src", Usage: "local path or URL for source archive"}
 	fnPodFlag := cli.StringFlag{Name: "pod", Usage: "function pod name, optional (use latest if unspecified)"}
 	fnFollowFlag := cli.BoolFlag{Name: "follow, f", Usage: "specify if the logs should be streamed"}
 	fnDetailFlag := cli.BoolFlag{Name: "detail, d", Usage: "display detailed information"}
 	fnLogDBTypeFlag := cli.StringFlag{Name: "dbtype", Usage: "log database type, e.g. influxdb (currently only influxdb is supported)"}
+	fnEntryPointFlag := cli.StringFlag{Name: "entrypoint", Usage: "entry point for environment v2 to load with"}
+	fnBuildCmdFlag := cli.StringFlag{Name: "buildcmd", Usage: "build command for builder to run with"}
 	fnSubcommands := []cli.Command{
-		{Name: "create", Usage: "Create new function (and optionally, an HTTP route to it)", Flags: []cli.Flag{fnNameFlag, fnEnvNameFlag, fnCodeFlag, fnPackageFlag, fnSrcPackageFlag, htUrlFlag, htMethodFlag}, Action: fnCreate},
+		{Name: "create", Usage: "Create new function (and optionally, an HTTP route to it)", Flags: []cli.Flag{fnNameFlag, fnEnvNameFlag, fnCodeFlag, fnPackageFlag, fnSrcArchiveFlag, fnDeployArchiveFlag, fnEntryPointFlag, fnBuildCmdFlag, htUrlFlag, htMethodFlag}, Action: fnCreate},
 		{Name: "get", Usage: "Get function source code", Flags: []cli.Flag{fnNameFlag}, Action: fnGet},
 		{Name: "getmeta", Usage: "Get function metadata", Flags: []cli.Flag{fnNameFlag}, Action: fnGetMeta},
-		{Name: "update", Usage: "Update function source code", Flags: []cli.Flag{fnNameFlag, fnEnvNameFlag, fnCodeFlag, fnPackageFlag, fnSrcPackageFlag}, Action: fnUpdate},
+		{Name: "update", Usage: "Update function source code", Flags: []cli.Flag{fnNameFlag, fnEnvNameFlag, fnCodeFlag, fnPackageFlag, fnSrcArchiveFlag, fnDeployArchiveFlag, fnEntryPointFlag, fnBuildCmdFlag}, Action: fnUpdate},
 		{Name: "delete", Usage: "Delete function", Flags: []cli.Flag{fnNameFlag}, Action: fnDelete},
 		{Name: "list", Usage: "List all functions", Flags: []cli.Flag{}, Action: fnList},
 		{Name: "logs", Usage: "Display function logs", Flags: []cli.Flag{fnNameFlag, fnPodFlag, fnFollowFlag, fnDetailFlag, fnLogDBTypeFlag}, Action: fnLogs},
@@ -100,8 +103,9 @@ func main() {
 	envImageFlag := cli.StringFlag{Name: "image", Usage: "Environment image URL"}
 	envBuilderImageFlag := cli.StringFlag{Name: "builder", Usage: "Environment builder image URL (optional)"}
 	envBuildCmdFlag := cli.StringFlag{Name: "buildcmd", Usage: "Build command for environment builder to build source package (optional)"}
+	envVersionFlag := cli.IntFlag{Name: "version", Usage: "Environment API version: defaults to 1 (means v1 interface)"}
 	envSubcommands := []cli.Command{
-		{Name: "create", Aliases: []string{"add"}, Usage: "Add an environment", Flags: []cli.Flag{envNameFlag, envImageFlag, envBuilderImageFlag, envBuildCmdFlag}, Action: envCreate},
+		{Name: "create", Aliases: []string{"add"}, Usage: "Add an environment", Flags: []cli.Flag{envNameFlag, envImageFlag, envBuilderImageFlag, envBuildCmdFlag, envVersionFlag}, Action: envCreate},
 		{Name: "get", Usage: "Get environment details", Flags: []cli.Flag{envNameFlag}, Action: envGet},
 		{Name: "update", Usage: "Update environment", Flags: []cli.Flag{envNameFlag, envImageFlag, envBuilderImageFlag, envBuildCmdFlag}, Action: envUpdate},
 		{Name: "delete", Usage: "Delete environment", Flags: []cli.Flag{envNameFlag}, Action: envDelete},
@@ -125,7 +129,7 @@ func main() {
 	upgradeFileFlag := cli.StringFlag{Name: "file", Usage: "JSON file containing all fission state"}
 	upgradeSubCommands := []cli.Command{
 		{Name: "dump", Usage: "Dump all state from a v0.1 fission installation", Flags: []cli.Flag{upgradeFileFlag}, Action: upgradeDumpState},
-		{Name: "restore", Usage: "Restore state dumped from a v0.1 install into a v0.2 install", Flags: []cli.Flag{upgradeFileFlag}, Action: upgradeRestoreState},
+		{Name: "restore", Usage: "Restore state dumped from a v0.1 install into a v0.2+ install", Flags: []cli.Flag{upgradeFileFlag}, Action: upgradeRestoreState},
 	}
 	app.Commands = []cli.Command{
 		{Name: "function", Aliases: []string{"fn"}, Usage: "Create, update and manage functions", Subcommands: fnSubcommands},

hack/release.sh

@@ -106,7 +106,9 @@ build_and_push_env_image() {
        ./build.sh
     fi
     docker build -t fission/$imgname:$version .
+    docker tag  fission/$imgname:$version fission/$imgname:latest
     docker push fission/$imgname:$version
+    docker push fission/$imgname:latest
     popd
 }
 
@@ -116,6 +118,7 @@ build_and_push_all_envs() {
     build_and_push_env_image $version nodejs node-env
     build_and_push_env_image $version binary binary-env
     build_and_push_env_image $version dotnet dotnet-env
+    # (See #359) build_and_push_env_image $version dotnet20 dotnet20-env
     build_and_push_env_image $version go go-env
     build_and_push_env_image $version perl perl-env
     build_and_push_env_image $version php7 php-env

mqtrigger/messageQueue/messageQueue.go

@@ -165,8 +165,9 @@ func (mqt *MessageQueueTriggerManager) syncTriggers() {
 			log.Fatalf("Failed to read message queue trigger list: %v", err)
 		}
 		newTriggerMap := make(map[string]*tpr.Messagequeuetrigger)
-		for _, trigger := range newTriggers.Items {
-			newTriggerMap[tpr.CacheKey(&trigger.Metadata)] = &trigger
+		for index := range newTriggers.Items {
+			newTrigger := &newTriggers.Items[index]
+			newTriggerMap[tpr.CacheKey(&newTrigger.Metadata)] = newTrigger
 		}
 
 		// get current set of triggers

poolmgr/functionServiceCache.go

@@ -22,6 +22,7 @@ import (
 
 	"k8s.io/client-go/1.5/pkg/api"
 
+	"github.com/fission/fission"
 	"github.com/fission/fission/cache"
 	"github.com/fission/fission/tpr"
 )
@@ -134,6 +135,7 @@ func (fsc *functionServiceCache) GetByFunction(m *api.ObjectMeta) (*funcSvc, err
 	return &fsvcCopy, nil
 }
 
+// TODO: error should be second return
 func (fsc *functionServiceCache) Add(fsvc funcSvc) (error, *funcSvc) {
 	err, existing := fsc.byFunction.Set(tpr.CacheKey(fsvc.function), &fsvc)
 	if err != nil {
@@ -152,13 +154,25 @@ func (fsc *functionServiceCache) Add(fsvc funcSvc) (error, *funcSvc) {
 	fsvc.ctime = now
 	fsvc.atime = now
 
+	// Add to byAddress and byPod caches. Ignore NameExists errors
+	// because of multiple-specialization. See issue #331.
 	err, _ = fsc.byAddress.Set(fsvc.address, *fsvc.function)
 	if err != nil {
+		if fe, ok := err.(fission.Error); ok {
+			if fe.Code == fission.ErrorNameExists {
+				err = nil
+			}
+		}
 		log.Printf("error caching fsvc: %v", err)
 		return err, nil
 	}
 	err, _ = fsc.byPod.Set(fsvc.podName, *fsvc.function)
 	if err != nil {
+		if fe, ok := err.(fission.Error); ok {
+			if fe.Code == fission.ErrorNameExists {
+				err = nil
+			}
+		}
 		log.Printf("error caching fsvc: %v", err)
 		return err, nil
 	}

poolmgr/gp.go

@@ -27,6 +27,7 @@ import (
 	"net/http"
 	"net/url"
 	"os"
+	"path/filepath"
 	"strings"
 	"time"
 
@@ -61,10 +62,13 @@ type (
 		poolInstanceId         string                // small random string to uniquify pod names
 		fetcherImage           string
 		fetcherImagePullPolicy v1.PullPolicy
+		runtimeImagePullPolicy v1.PullPolicy // pull policy for generic pool to created env deployment
 		kubernetesClient       *kubernetes.Clientset
+		fissionClient          *tpr.FissionClient
 		instanceId             string // poolmgr instance id
 		labelsForPool          map[string]string
 		requestChannel         chan *choosePodRequest
+		sharedMountPath        string
 	}
 
 	// serialize the choosing of pods so that choices don't conflict
@@ -78,7 +82,19 @@ type (
 	}
 )
 
+func getImagePullPolicy(policy string) v1.PullPolicy {
+	switch policy {
+	case "Always":
+		return v1.PullAlways
+	case "Never":
+		return v1.PullNever
+	default:
+		return v1.PullIfNotPresent
+	}
+}
+
 func MakeGenericPool(
+	fissionClient *tpr.FissionClient,
 	kubernetesClient *kubernetes.Clientset,
 	env *tpr.Environment,
 	initialReplicas int32,
@@ -92,9 +108,13 @@ func MakeGenericPool(
 	if len(fetcherImage) == 0 {
 		fetcherImage = "fission/fetcher"
 	}
-	fetcherImagePullPolicyS := os.Getenv("FETCHER_IMAGE_PULL_POLICY")
-	if len(fetcherImagePullPolicyS) == 0 {
-		fetcherImagePullPolicyS = "IfNotPresent"
+	fetcherImagePullPolicy := os.Getenv("FETCHER_IMAGE_PULL_POLICY")
+	if len(fetcherImagePullPolicy) == 0 {
+		fetcherImagePullPolicy = "IfNotPresent"
+	}
+	runtimeImagePullPolicy := os.Getenv("RUNTIME_IMAGE_PULL_POLICY")
+	if len(runtimeImagePullPolicy) == 0 {
+		runtimeImagePullPolicy = "IfNotPresent"
 	}
 
 	// TODO: in general we need to provide the user a way to configure pools.  Initial
@@ -103,6 +123,7 @@ func MakeGenericPool(
 		env:              env,
 		replicas:         initialReplicas, // TODO make this an env param instead?
 		requestChannel:   make(chan *choosePodRequest),
+		fissionClient:    fissionClient,
 		kubernetesClient: kubernetesClient,
 		namespace:        namespace,
 		podReadyTimeout:  5 * time.Minute, // TODO make this an env param?
@@ -111,18 +132,13 @@ func MakeGenericPool(
 		poolInstanceId:   uniuri.NewLen(8),
 		instanceId:       instanceId,
 		fetcherImage:     fetcherImage,
-		useSvc:           false, // defaults off -- svc takes a second or more to become routable, slowing cold start
+		useSvc:           false,       // defaults off -- svc takes a second or more to become routable, slowing cold start
+		sharedMountPath:  "/userfunc", // used by generic pool when creating env deployment to specify the share volume path for fetcher & env
 	}
 
-	switch fetcherImagePullPolicyS {
-	case "Always":
-		gp.fetcherImagePullPolicy = v1.PullAlways
-	case "Never":
-		gp.fetcherImagePullPolicy = v1.PullNever
-	default:
-		gp.fetcherImagePullPolicy = v1.PullIfNotPresent
-	}
+	gp.runtimeImagePullPolicy = getImagePullPolicy(runtimeImagePullPolicy)
 
+	gp.fetcherImagePullPolicy = getImagePullPolicy(fetcherImagePullPolicy)
 	log.Printf("fetcher image: %v, pull policy: %v", gp.fetcherImage, gp.fetcherImagePullPolicy)
 
 	// Labels for generic deployment/RS/pods.
@@ -141,7 +157,10 @@ func MakeGenericPool(
 
 	go gp.choosePodService()
 
-	go gp.idlePodReaper()
+	// Unless specified otherwise, periodically cleanup inactive pods.
+	if env.Spec.AllowedFunctionsPerContainer != fission.AllowedFunctionsPerContainerInfinite {
+		go gp.idlePodReaper()
+	}
 
 	return gp, nil
 }
@@ -186,8 +205,7 @@ func (gp *GenericPool) _choosePod(newLabels map[string]string) (*v1.Pod, error)
 		// Get pods; filter the ones that are ready
 		podList, err := gp.kubernetesClient.Core().Pods(gp.namespace).List(
 			api.ListOptions{
-				LabelSelector: labels.Set(
-					gp.deployment.Spec.Selector.MatchLabels).AsSelector(),
+				LabelSelector: labels.Set(gp.deployment.Spec.Selector.MatchLabels).AsSelector(),
 			})
 		if err != nil {
 			return nil, err
@@ -212,8 +230,7 @@ func (gp *GenericPool) _choosePod(newLabels map[string]string) (*v1.Pod, error)
 				readyPods = append(readyPods, &pod)
 			}
 		}
-		log.Printf("[%v] found %v ready pods of %v total",
-			newLabels, len(readyPods), len(podList.Items))
+		log.Printf("[%v] found %v ready pods of %v total", newLabels, len(readyPods), len(podList.Items))
 
 		// If there are no ready pods, wait and retry.
 		if len(readyPods) == 0 {
@@ -229,15 +246,17 @@ func (gp *GenericPool) _choosePod(newLabels map[string]string) (*v1.Pod, error)
 		// and make a good scheduling decision.
 		chosenPod := readyPods[rand.Intn(len(readyPods))]
 
-		// Relabel.  If the pod already got picked and
-		// modified, this should fail; in that case just
-		// retry.
-		chosenPod.ObjectMeta.Labels = newLabels
-		log.Printf("relabeling pod: [%v]", chosenPod.ObjectMeta.Name)
-		_, err = gp.kubernetesClient.Core().Pods(gp.namespace).Update(chosenPod)
-		if err != nil {
-			log.Printf("failed to relabel pod [%v]: %v", chosenPod.ObjectMeta.Name, err)
-			continue
+		if gp.env.Spec.AllowedFunctionsPerContainer != fission.AllowedFunctionsPerContainerInfinite {
+			// Relabel.  If the pod already got picked and
+			// modified, this should fail; in that case just
+			// retry.
+			chosenPod.ObjectMeta.Labels = newLabels
+			log.Printf("relabeling pod: [%v]", chosenPod.ObjectMeta.Name)
+			_, err = gp.kubernetesClient.Core().Pods(gp.namespace).Update(chosenPod)
+			if err != nil {
+				log.Printf("failed to relabel pod [%v]: %v", chosenPod.ObjectMeta.Name, err)
+				continue
+			}
 		}
 		log.Printf("Chosen pod: %v (in %v)", chosenPod.ObjectMeta.Name, time.Now().Sub(startTime))
 		return chosenPod, nil
@@ -276,12 +295,15 @@ func (gp *GenericPool) getFetcherUrl(podIP string) string {
 	return fmt.Sprintf("http://%v:8000/", podIP)
 }
 
-func (gp *GenericPool) getSpecializeUrl(podIP string) string {
+func (gp *GenericPool) getSpecializeUrl(podIP string, version int) string {
 	u := os.Getenv("TEST_SPECIALIZE_URL")
 	if len(u) != 0 {
 		return u
 	}
-	return fmt.Sprintf("http://%v:8888/specialize", podIP)
+	if version == 1 {
+		return fmt.Sprintf("http://%v:8888/specialize", podIP)
+	}
+	return fmt.Sprintf("http://%v:8888/v%v/specialize", podIP, version)
 }
 
 // specializePod chooses a pod, copies the required user-defined function to that pod
@@ -297,10 +319,23 @@ func (gp *GenericPool) specializePod(pod *v1.Pod, metadata *api.ObjectMeta) erro
 	// tell fetcher to get the function.
 	fetcherUrl := gp.getFetcherUrl(podIP)
 	log.Printf("[%v] calling fetcher to copy function", metadata.Name)
-	err := fetcherClient.DoFetchRequest(fetcherUrl, &fetcher.FetchRequest{
+
+	fn, err := gp.fissionClient.
+		Functions(metadata.Namespace).
+		Get(metadata.Name)
+	if err != nil {
+		return err
+	}
+
+	targetFilename := "user"
+
+	err = fetcherClient.MakeClient(fetcherUrl).Fetch(&fetcher.FetchRequest{
 		FetchType: fetcher.FETCH_DEPLOYMENT,
-		Function:  *metadata,
-		Filename:  "user", // XXX use function id instead
+		Package: api.ObjectMeta{
+			Namespace: fn.Spec.Package.PackageRef.Namespace,
+			Name:      fn.Spec.Package.PackageRef.Name,
+		},
+		Filename: targetFilename, // XXX use function id instead
 	})
 	if err != nil {
 		return err
@@ -311,12 +346,30 @@ func (gp *GenericPool) specializePod(pod *v1.Pod, metadata *api.ObjectMeta) erro
 
 	// get function run container to specialize
 	log.Printf("[%v] specializing pod", metadata.Name)
-	specializeUrl := gp.getSpecializeUrl(podIP)
 
 	// retry the specialize call a few times in case the env server hasn't come up yet
 	maxRetries := 20
+
+	loadReq := fission.FunctionLoadRequest{
+		FilePath:         filepath.Join(gp.sharedMountPath, targetFilename),
+		FunctionName:     fn.Spec.Package.FunctionName,
+		FunctionMetadata: &fn.Metadata,
+	}
+
+	body, err := json.Marshal(loadReq)
+	if err != nil {
+		return err
+	}
+
 	for i := 0; i < maxRetries; i++ {
-		resp2, err := http.Post(specializeUrl, "text/plain", bytes.NewReader([]byte{}))
+		var resp2 *http.Response
+		if gp.env.Spec.Version == 2 {
+			specializeUrl := gp.getSpecializeUrl(podIP, 2)
+			resp2, err = http.Post(specializeUrl, "application/json", bytes.NewReader(body))
+		} else {
+			specializeUrl := gp.getSpecializeUrl(podIP, 1)
+			resp2, err = http.Post(specializeUrl, "text/plain", bytes.NewReader([]byte{}))
+		}
 		if err == nil && resp2.StatusCode < 300 {
 			// Success
 			resp2.Body.Close()
@@ -352,7 +405,6 @@ func (gp *GenericPool) createPool() error {
 	poolDeploymentName := fmt.Sprintf("%v-%v-%v",
 		gp.env.Metadata.Name, gp.env.Metadata.UID, strings.ToLower(gp.poolInstanceId))
 
-	sharedMountPath := "/userfunc"
 	deployment := &v1beta1.Deployment{
 		ObjectMeta: v1.ObjectMeta{
 			Name:   poolDeploymentName,
@@ -380,12 +432,12 @@ func (gp *GenericPool) createPool() error {
 						{
 							Name:                   gp.env.Metadata.Name,
 							Image:                  gp.env.Spec.Runtime.Image,
-							ImagePullPolicy:        v1.PullIfNotPresent,
+							ImagePullPolicy:        gp.runtimeImagePullPolicy,
 							TerminationMessagePath: "/dev/termination-log",
 							VolumeMounts: []v1.VolumeMount{
 								{
 									Name:      "userfunc",
-									MountPath: sharedMountPath,
+									MountPath: gp.sharedMountPath,
 								},
 							},
 						},
@@ -397,10 +449,10 @@ func (gp *GenericPool) createPool() error {
 							VolumeMounts: []v1.VolumeMount{
 								{
 									Name:      "userfunc",
-									MountPath: sharedMountPath,
+									MountPath: gp.sharedMountPath,
 								},
 							},
-							Command: []string{"/fetcher", sharedMountPath},
+							Command: []string{"/fetcher", gp.sharedMountPath},
 						},
 					},
 					ServiceAccountName: "fission-fetcher",
@@ -478,7 +530,7 @@ func (gp *GenericPool) GetFuncSvc(m *api.ObjectMeta) (*funcSvc, error) {
 	if gp.useSvc {
 		svcName := fmt.Sprintf("svc-%v", m.Name)
 		if len(m.UID) > 0 {
-			svcName += ("-" + string(m.UID))
+			svcName = fmt.Sprintf("%s-%v", svcName, m.UID)
 		}
 
 		labels := gp.labelsForFunction(m)
@@ -511,13 +563,18 @@ func (gp *GenericPool) GetFuncSvc(m *api.ObjectMeta) (*funcSvc, error) {
 
 	err, existingFsvc := gp.fsCache.Add(*fsvc)
 	if err != nil {
-		// Some other thread beat us to it -- return the other thread's fsvc and clean up
-		// our own.
-		log.Printf("func svc already exists: %v", existingFsvc.podName)
-		go func() {
-			gp.kubernetesClient.Core().Pods(gp.namespace).Delete(fsvc.podName, nil)
-		}()
-		return existingFsvc, nil
+		if fe, ok := err.(fission.Error); ok {
+			if fe.Code == fission.ErrorNameExists {
+				// Some other thread beat us to it -- return the other thread's fsvc and clean up
+				// our own.
+				log.Printf("func svc already exists: %v", existingFsvc.podName)
+				go func() {
+					gp.kubernetesClient.Core().Pods(gp.namespace).Delete(fsvc.podName, nil)
+				}()
+				return existingFsvc, nil
+			}
+		}
+		return nil, err
 	}
 	return fsvc, nil
 }

poolmgr/gpm.go

@@ -23,6 +23,7 @@ import (
 	"k8s.io/client-go/1.5/kubernetes"
 	"k8s.io/client-go/1.5/pkg/api"
 
+	"github.com/fission/fission"
 	"github.com/fission/fission/tpr"
 )
 
@@ -86,9 +87,14 @@ func (gpm *GenericPoolManager) service() {
 			var err error
 			pool, ok := gpm.pools[tpr.CacheKey(&req.env.Metadata)]
 			if !ok {
+				var poolSize int32 = 3 // TODO configurable/autoscalable
+				switch req.env.Spec.AllowedFunctionsPerContainer {
+				case fission.AllowedFunctionsPerContainerInfinite:
+					poolSize = 1
+				}
+
 				pool, err = MakeGenericPool(
-					gpm.kubernetesClient, req.env,
-					3, // TODO configurable/autoscalable
+					gpm.fissionClient, gpm.kubernetesClient, req.env, poolSize,
 					gpm.namespace, gpm.fsCache, gpm.instanceId)
 				if err != nil {
 					req.responseChannel <- &response{error: err}

router/functionHandler.go

@@ -98,6 +98,9 @@ func (fh *functionHandler) handler(responseWriter http.ResponseWriter, request *
 		request.Header.Add(fmt.Sprintf("X-Fission-Params-%v", k), v)
 	}
 
+	// System Params
+	MetadataToHeaders(HEADERS_FISSION_FUNCTION_PREFIX, fh.function, request)
+
 	// cache lookup
 	serviceUrl, err := fh.fmap.lookup(fh.function)
 	if err != nil {

router/httpTriggers.go

@@ -115,9 +115,10 @@ func (ts *HTTPTriggerSet) getRouter() *mux.Router {
 	// Internal triggers for each function by name. Non-http
 	// triggers route into these.
 	for _, function := range ts.functions {
+		m := function.Metadata
 		fh := &functionHandler{
 			fmap:     ts.functionServiceMap,
-			function: &function.Metadata,
+			function: &m,
 			poolmgr:  ts.poolmgr,
 		}
 		muxRouter.HandleFunc(fission.UrlForFunction(function.Metadata.Name), fh.handler)

router/util.go

@@ -0,0 +1,28 @@
+package router
+
+import (
+	"fmt"
+	"k8s.io/client-go/1.5/pkg/api"
+	"k8s.io/client-go/1.5/pkg/types"
+	"net/http"
+)
+
+const (
+	HEADERS_FISSION_FUNCTION_PREFIX = "Fission-Function"
+)
+
+func MetadataToHeaders(prefix string, meta *api.ObjectMeta, request *http.Request) {
+	request.Header.Add(fmt.Sprintf("X-%s-Uid", prefix), string(meta.UID))
+	request.Header.Add(fmt.Sprintf("X-%s-Name", prefix), meta.Name)
+	request.Header.Add(fmt.Sprintf("X-%s-Namespace", prefix), meta.Namespace)
+	request.Header.Add(fmt.Sprintf("X-%s-ResourceVersion", prefix), meta.ResourceVersion)
+}
+
+func HeadersToMetadata(prefix string, headers http.Header) *api.ObjectMeta {
+	return &api.ObjectMeta{
+		Name:            headers.Get(fmt.Sprintf("X-%s-Name", prefix)),
+		UID:             types.UID(headers.Get(fmt.Sprintf("X-%s-Uid", prefix))),
+		Namespace:       headers.Get(fmt.Sprintf("X-%s-Namespace", prefix)),
+		ResourceVersion: headers.Get(fmt.Sprintf("X-%s-ResourceVersion", prefix)),
+	}
+}

storagesvc/storagesvc.go

@@ -201,6 +201,21 @@ func MakeStorageService(sc *storageConfig) (*StorageService, error) {
 	ss.location = loc
 
 	con, err := loc.CreateContainer(sc.containerName)
+	if os.IsExist(err) {
+		var cons []stow.Container
+		var cursor string
+
+		// use location.Containers to find containers that match the prefix (container name)
+		cons, cursor, err = loc.Containers(sc.containerName, stow.CursorStart, 1)
+		if err == nil {
+			if !stow.IsCursorEnd(cursor) {
+				// Should only have one storage container
+				err = errors.New("Found more than one matched storage containers")
+			} else {
+				con = cons[0]
+			}
+		}
+	}
 	if err != nil {
 		log.Printf("Error initializing storage: %v", err)
 		return nil, err

test/build_and_test.sh

@@ -12,12 +12,18 @@ source $(dirname $0)/test_utils.sh
 
 IMAGE=gcr.io/fission-ci/fission-bundle
 FETCHER_IMAGE=gcr.io/fission-ci/fetcher
+PYTHON_RUNTIME_IMAGE=gcr.io/fission-ci/python-env
+PYTHON_BUILDER_IMAGE=gcr.io/fission-ci/python-env-builder
 TAG=test
 
 build_and_push_fission_bundle $IMAGE:$TAG
 
 build_and_push_fetcher $FETCHER_IMAGE:$TAG
 
+build_and_push_python_env_runtime $PYTHON_RUNTIME_IMAGE:$TAG
+
+build_and_push_python_env_builder $PYTHON_BUILDER_IMAGE:$TAG
+
 build_fission_cli
 
 install_and_test $IMAGE $TAG $FETCHER_IMAGE $TAG

test/test_utils.sh

@@ -50,6 +50,36 @@ build_and_push_fetcher() {
     popd
 }
 
+build_and_push_python_env_runtime() {
+    image_tag=$1
+
+    pushd $ROOT/environments/python3/
+    docker build -t $image_tag .
+
+    gcloud_login
+    
+    gcloud docker -- push $image_tag
+    popd
+}
+
+build_and_push_python_env_builder() {
+    image_tag=$1
+
+    pushd $ROOT/builder/cmd
+    ./build.sh
+    popd
+    pushd $ROOT/environments/python3/builder
+    builderDir=${GOPATH}/src/github.com/fission/fission/builder/cmd
+    cp ${builderDir}/builder .
+
+    docker build -t $image_tag .
+
+    gcloud_login
+    
+    gcloud docker -- push $image_tag
+    popd
+}
+
 
 build_fission_cli() {
     pushd $ROOT/fission
@@ -73,7 +103,7 @@ helm_install_fission() {
     ns=f-$id
     fns=f-func-$id
 
-    helmVars=image=$image,imageTag=$imageTag,fetcherImage=$fetcherImage,fetcherImageTag=$fetcherImageTag,functionNamespace=$fns,controllerPort=$controllerNodeport,routerPort=$routerNodeport,pullPolicy=Always
+    helmVars=image=$image,imageTag=$imageTag,fetcherImage=$fetcherImage,fetcherImageTag=$fetcherImageTag,functionNamespace=$fns,controllerPort=$controllerNodeport,routerPort=$routerNodeport,pullPolicy=Always,analytics=false
 
     helm_setup
     
@@ -171,6 +201,7 @@ dump_fission_tpr() {
 
 dump_fission_tprs() {
     dump_fission_tpr function.fission.io    
+    dump_fission_tpr package.fission.io    
     dump_fission_tpr httptrigger.fission.io    
     dump_fission_tpr environment.fission.io    
 }

test/tests/test_buildermgr.sh

@@ -0,0 +1,73 @@
+#!/bin/bash
+
+set -euo pipefail
+
+# Create a function with source package in python 
+# to test builder manger functionality. 
+# There are two ways to trigger the build
+# 1. manually trigger by http post 
+# 2. package watcher triggers the build if any changes to packages
+
+ROOT=$(dirname $0)/../..
+PYTHON_RUNTIME_IMAGE=gcr.io/fission-ci/python-env:test
+PYTHON_BUILDER_IMAGE=gcr.io/fission-ci/python-env-builder:test
+
+fn=python-srcbuild-$(date +%s)
+
+checkFunctionResponse() {
+    echo "Doing an HTTP GET on the function's route"
+    response=$(curl http://$FISSION_ROUTER/$1)
+
+    echo "Checking for valid response"
+    echo $response
+    echo $response | grep -i "a: 1 b: {c: 3, d: 4}"
+}
+
+echo "Pre-test cleanup"
+fission env delete --name python || true
+
+echo "Creating python env"
+fission env create --name python --image $PYTHON_RUNTIME_IMAGE --builder $PYTHON_BUILDER_IMAGE
+trap "fission env delete --name python" EXIT
+
+echo "Waiting for env builder to catch up"
+sleep 30
+
+echo "Creating source pacakage"
+zip -jr demo-src-pkg.zip $ROOT/examples/python/sourcepkg/
+
+echo "Creating function " $fn
+fission fn create --name $fn --env python --src demo-src-pkg.zip --entrypoint "main" --buildcmd "./build.sh"
+trap "fission fn delete --name $fn" EXIT
+
+echo "Creating route"
+fission route create --function $fn --url /$fn --method GET
+
+echo "Waiting for router to catch up"
+sleep 3
+
+echo "Doing an HTTP POST on the builder manager's route to start a build"
+pkg=$(kubectl --namespace default get functions $fn -o jsonpath='{.spec.package.packageref.name}')
+echo $pkg
+response=$(curl -X POST $FISSION_URL/proxy/buildermgr/v1/build \
+  -H 'content-type: application/json' \
+  -d "{\"package\": {\"namespace\": \"default\",\"name\": \"$pkg\"}}")
+
+echo "Waiting for builder manager to finish the build triggered by http request"
+sleep 30
+
+checkFunctionResponse $fn
+
+echo "Updating function " $fn
+fission fn update --name $fn --src demo-src-pkg.zip
+trap "fission fn delete --name $fn" EXIT
+
+echo "Waiting for builder manager to finish the build triggered by packageWatcher"
+sleep 30
+
+checkFunctionResponse $fn
+
+# crappy cleanup, improve this later
+kubectl get httptrigger -o name | tail -1 | cut -f2 -d'/' | xargs kubectl delete httptrigger
+
+echo "All done."

test/tests/test_internal_routes.sh

@@ -0,0 +1,46 @@
+#!/bin/bash
+
+#
+# Create two functions, make sure their internal http triggers invoke
+# them correctly.
+#
+
+set -euo pipefail
+
+ROOT=$(dirname $0)/../..
+
+echo "Pre-test cleanup"
+fission env delete --name nodejs || true
+
+echo "Creating nodejs env"
+fission env create --name nodejs --image fission/node-env
+trap "fission env delete --name nodejs" EXIT
+
+echo "Writing functions"
+f1=f1-$(date +%s)
+f2=f2-$(date +%s)
+echo $f1 $f2
+
+for f in $f1 $f2
+do
+    echo "module.exports = function(context, callback) { callback(200, \"$f\n\"); }" > $f.js
+done
+
+echo "Creating functions"
+for f in $f1 $f2
+do
+    fission fn create --name $f --env nodejs --code $f.js
+    trap "fission fn delete --name $f" EXIT
+done
+
+echo "Waiting for router to catch up"
+sleep 3
+
+echo "Testing internal routes"
+for f in $f1 $f2
+do
+    response=$(curl http://$FISSION_ROUTER/fission-function/$f)
+    echo $response | grep $f
+done
+
+echo "All done."

tpr/types.go

@@ -43,6 +43,8 @@ type (
 		unversioned.TypeMeta `json:",inline"`
 		Metadata             api.ObjectMeta      `json:"metadata"`
 		Spec                 fission.PackageSpec `json:"spec"`
+
+		Status fission.PackageStatus `json:"status"`
 	}
 	PackageList struct {
 		unversioned.TypeMeta `json:",inline"`

types.go

@@ -16,6 +16,8 @@ limitations under the License.
 
 package fission
 
+import "k8s.io/client-go/1.5/pkg/api"
+
 type (
 	//
 	// Functions and packages
@@ -65,9 +67,10 @@ type (
 	BuildStatus string
 
 	PackageSpec struct {
-		Environment EnvironmentReference `json:"environment"`
-		Source      Archive              `json:"source"`
-		Deployment  Archive              `json:"deployment"`
+		Environment  EnvironmentReference `json:"environment"`
+		Source       Archive              `json:"source"`
+		Deployment   Archive              `json:"deployment"`
+		BuildCommand string               `json:"buildcmd"`
 		// In the future, we can have a debug build here too
 	}
 	PackageStatus struct {
@@ -162,8 +165,14 @@ type (
 		// Optional, but strongly encouraged. Used to populate
 		// links from UI, CLI, etc.
 		DocumentationURL string `json:"documentationurl"`
+
+		// Optional
+		// Defaults to 'Single'
+		AllowedFunctionsPerContainer AllowedFunctionsPerContainer `json:"allowedFunctionsPerContainer"`
 	}
 
+	AllowedFunctionsPerContainer string
+
 	//
 	// Triggers
 	//
@@ -220,15 +229,18 @@ type (
 		// env-specific. Optional.
 		FilePath string `json:"filepath"`
 
-		// Entrypoint has an environment-specific meaning;
+		// FunctionName has an environment-specific meaning;
 		// usually, it defines a function within a module
 		// containing multiple functions. Optional; default is
 		// environment-specific.
-		EntryPoint string `json:"entrypoint"`
+		FunctionName string `json:"functionName"`
 
 		// URL to expose this function at. Optional; defaults
 		// to "/".
 		URL string `json:"url"`
+
+		// Metatdata
+		FunctionMetadata *api.ObjectMeta
 	}
 )
 
@@ -252,6 +264,11 @@ const (
 	BuildStatusFailed    = "failed"
 )
 
+const (
+	AllowedFunctionsPerContainerSingle   = "single"
+	AllowedFunctionsPerContainerInfinite = "infinite"
+)
+
 const (
 	// FunctionReferenceFunctionName means that the function
 	// reference is simply by function name.