Newdeploy backend (#387)

A newdeploy backend which uses new deployment to serve requests. This is the second phase of #193 and builds on top of changes in #384 .

* Executor layer added on top of pool manager

* Removed the external server for executor

* Minor changes to keep existing semantics as much possible

* Separating the executor vs. poolmgr backend functionality and associated data members

* Executor logic separated from Poolmgr backend completely, placeholder for new backend

* Changed references to poolmgr in tests

* Moved poolmgr to it's package, as a side effect moved Cache to its's package (was causing cyclical dependency) and had to make some data structures exposed outside package

* Rebased from master and changed references to tpr -> crd

* Executor layer added on top of pool manager

* Executor logic separated from Poolmgr backend completely, placeholder for new backend

* Changed podName to a generic objectReference in fscache (#391)

Changed podName to a generic objectReference in function service cache implementation.

* Moved poolmgr to it's package, as a side effect moved Cache to its's package (was causing cyclical dependency) and had to make some data structures exposed outside package

* Rebased from master and changed references to tpr -> crd

* Merged from master with latest changes

* Executor layer added on top of pool manager

* Removed the external server for executor

* Minor changes to keep existing semantics as much possible

* Separating the executor vs. poolmgr backend functionality and associated data members

* Executor logic separated from Poolmgr backend completely, placeholder for new backend

* Changed references to poolmgr in tests

* update compiling.md to use helm

* Compile instructions: changed pullPolicy to IfNotPresent (#378)

Containers will get stuck in ErrImagePull/ImagePullBackOff state otherwise

* Moved poolmgr to it's package, as a side effect moved Cache to its's package (was causing cyclical dependency) and had to make some data structures exposed outside package

* Fetcher called when pod is created for newDeploy backend but also supports older way, this is WIP and still needs pod specialization and creating & exposing a service so the URL can be hit by end user

* WIP Specializing the POD as part of startup along with fetching

* Working specialization of a new deployment. Needs some work on caching, cleanup etc.

* Switched to service based address instead of POD address

* Minor formating issue fixed

* Added logging to pods and a readiness check, the readiness check is flaky though ATM

* Fixed some rebase issues that were failing build

* Better names for K8S objects and methods

* Switched usage of FuncSvc in backends from pod to api.ObjectReference

* Adding retry to fetcher request, for now just using default retry client which might need tweaking in future

* Switching to plain old retry, some issue in getting retryablehttp with glide import

* Removed stale executor service & deployment from previous merge

* Addressed review comments, still testing some areas

* Added types in FunctionSpec

* Resolved conflicts due to merge from executor_abstraction branch

* Added backend type on EnvironmentSpec along with operations for create/list/update, the pools are created/destroyed based on change in backend type

* Backend from types and a minor err return issue fixed

* Draft version of CPU and memory parameters added to environment

* Added resourceReq to newDeploy, though it has some issues

* Issue with resourceName fixed, now newdeploy pods also pick up resources from the environment config

* Adding scale params, removing validation on CPU params for now

* Fixed a formatting issue

* Checking if slight more delay helps in the test which is currently failing for internal routes

* The resourceList newly added in Env can not be compared by compiler, hence must use breakdown comparison instead

* Added strategy selection on client side

* Added caching, informers, delete operations for newdeploy backend functions

* Deleted a stale directory

* A simple HPA based on scale parameters, testing still WIP

* Fixed a small issue in delete function, added HPA delete too when deleting a function

* Previous merge missed the pkg flag for update fn command somehow, fixed that

* Fixed comments from review

* Changed poolmgr cleanup to be generic cleanup and moved to executor, added instanceID labels to newdeploy so that cleanup works

* Moved instanceIdLabel to types to avoid cyclic dependency

* More review fixes

* Tweaking sleep to see results

* If user does not provide poolsize, then it should not default to zero

* Switched to naming convention for now, fixed default poolsize if not provided

* Changed error return behaviour in delete fn, also changed cleanup to look based on obj type though support for additional type will need more work

* Changed check location so avoid false logging

* Test for newdeploy backend

* Adding tests for poolmgr backend

* Fixed an issue with glide dependency version, already fixed in master

* Added instanceId for NewDeploy, Initial cleanup now cleans older objects of newdeploy backend, removed eagercreate flag and instead using minScale to drive eager creation

* Moved cleanup to executor layer with cleanup for newDeploy backend, changes to use the new Cache impl

* Cleaning up pod & rs along with deployment for newdeploy backend

* Enhanced fn and env listing to show min/maxscale and resuorces respectively

* Added conditional heapster deployment and fixed a small issue with resources for fetcher container in function pod

* Addressed review comments from previous change

* Addressed some more review comments - majorly create only on NotFoundError

* Added TargetCPU as an input for scaling

* Bumped target CPU to be greater than 0 and added a default value

* Min replicas should be 1 even if the minScale is 0 when creating deployment

* Changed name from 'backend' to executorType, added additional test for minscale 0 case, changed TargetCPU to TargetCPUPercent

fission/environment.go

@@ -19,10 +19,13 @@ package main
 import (
 	"fmt"
 	"os"
+	"strconv"
 	"text/tabwriter"
 
 	"github.com/urfave/cli"
+	"k8s.io/apimachinery/pkg/api/resource"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/client-go/pkg/api/v1"
 
 	"github.com/fission/fission"
 	"github.com/fission/fission/crd"
@@ -36,6 +39,13 @@ func envCreate(c *cli.Context) error {
 		fatal("Need a name, use --name.")
 	}
 
+	var poolsize int
+	if c.IsSet("poolsize") {
+		poolsize = c.Int("poolsize")
+	} else {
+		poolsize = 3
+	}
+
 	envImg := c.String("image")
 	if len(envImg) == 0 {
 		fatal("Need an image, use --image.")
@@ -52,6 +62,8 @@ func envCreate(c *cli.Context) error {
 		}
 	}
 
+	resourceReq := getResourceReq(c.Int("mincpu"), c.Int("maxcpu"), c.Int("minmemory"), c.Int("maxmemory"))
+
 	// Environment API interface version is not specified and
 	// builder image is empty, set default interface version
 	if envVersion == 0 {
@@ -72,6 +84,8 @@ func envCreate(c *cli.Context) error {
 				Image:   envBuilderImg,
 				Command: envBuildCmd,
 			},
+			Poolsize:  poolsize,
+			Resources: resourceReq,
 		},
 	}
 
@@ -112,6 +126,7 @@ func envUpdate(c *cli.Context) error {
 	if len(envName) == 0 {
 		fatal("Need a name, use --name.")
 	}
+
 	envImg := c.String("image")
 	envBuilderImg := c.String("builder")
 	envBuildCmd := c.String("buildcmd")
@@ -141,6 +156,10 @@ func envUpdate(c *cli.Context) error {
 		env.Spec.Builder.Command = envBuildCmd
 	}
 
+	if c.IsSet("poolsize") {
+		env.Spec.Poolsize = c.Int("poolsize")
+	}
+
 	_, err = client.EnvironmentUpdate(env)
 	checkErr(err, "update environment")
 
@@ -174,12 +193,60 @@ func envList(c *cli.Context) error {
 	checkErr(err, "list environments")
 
 	w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
-	fmt.Fprintf(w, "%v\t%v\t%v\n", "NAME", "UID", "IMAGE")
+	fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\n", "NAME", "UID", "IMAGE", "POOLSIZE", "MINCPU", "MAXCPU", "MINMEMORY", "MAXMEMORY")
 	for _, env := range envs {
-		fmt.Fprintf(w, "%v\t%v\t%v\n",
-			env.Metadata.Name, env.Metadata.UID, env.Spec.Runtime.Image)
+		fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\t%v\n",
+			env.Metadata.Name, env.Metadata.UID, env.Spec.Runtime.Image, env.Spec.Poolsize,
+			env.Spec.Resources.Requests.Cpu(), env.Spec.Resources.Limits.Cpu(),
+			env.Spec.Resources.Requests.Memory(), env.Spec.Resources.Limits.Memory())
 	}
 	w.Flush()
 
 	return nil
 }
+
+func getResourceReq(mincpu int, maxcpu int, minmem int, maxmem int) v1.ResourceRequirements {
+
+	requestResources := make(map[v1.ResourceName]resource.Quantity)
+
+	if mincpu != 0 {
+		cpuRequest, err := resource.ParseQuantity(strconv.Itoa(mincpu) + "m")
+		if err != nil {
+			fatal("Failed to parse mincpu")
+		}
+		requestResources[v1.ResourceCPU] = cpuRequest
+	}
+
+	if minmem != 0 {
+		memRequest, err := resource.ParseQuantity(strconv.Itoa(minmem) + "Mi")
+		if err != nil {
+			fatal("Failed to parse minmemory")
+		}
+		requestResources[v1.ResourceMemory] = memRequest
+	}
+
+	limitResources := make(map[v1.ResourceName]resource.Quantity)
+
+	if maxcpu != 0 {
+		cpuLimit, err := resource.ParseQuantity(strconv.Itoa(maxcpu) + "m")
+		if err != nil {
+			fatal("Failed to parse maxcpu")
+		}
+		limitResources[v1.ResourceCPU] = cpuLimit
+	}
+
+	if maxmem != 0 {
+		memLimit, err := resource.ParseQuantity(strconv.Itoa(maxmem) + "Mi")
+		if err != nil {
+			fatal("Failed to parse maxmemory")
+		}
+		limitResources[v1.ResourceMemory] = memLimit
+	}
+
+	resources := v1.ResourceRequirements{
+		Requests: requestResources,
+		Limits:   limitResources,
+	}
+
+	return resources
+}

fission/function.go

@@ -64,6 +64,42 @@ func printPodLogs(c *cli.Context) error {
 	return nil
 }
 
+func getInvokeStrategy(minScale int, maxScale int, executorType string, targetcpu int) fission.InvokeStrategy {
+
+	if maxScale == 0 {
+		maxScale = 1
+	}
+
+	if minScale > maxScale {
+		fatal("Maxscale must be higher than or equal to minscale")
+	}
+
+	var fnExecutor fission.ExecutorType
+	switch executorType {
+	case "":
+		fnExecutor = fission.ExecutorTypePoolmgr
+	case fission.ExecutorTypePoolmgr:
+		fnExecutor = fission.ExecutorTypePoolmgr
+	case fission.ExecutorTypeNewdeploy:
+		fnExecutor = fission.ExecutorTypeNewdeploy
+	default:
+		fatal("Executor type must be one of 'poolmgr' or 'newdeploy', defaults to 'poolmgr'")
+	}
+
+	// Right now a simple single case strategy implementation
+	// This will potentially get more sophisticated once we have more strategies in place
+	strategy := fission.InvokeStrategy{
+		StrategyType: fission.StrategyTypeExecution,
+		ExecutionStrategy: fission.ExecutionStrategy{
+			ExecutorType:     fnExecutor,
+			MinScale:         minScale,
+			MaxScale:         maxScale,
+			TargetCPUPercent: targetcpu,
+		},
+	}
+	return strategy
+}
+
 func fnCreate(c *cli.Context) error {
 	client := getClient(c.GlobalString("server"))
 
@@ -135,6 +171,21 @@ func fnCreate(c *cli.Context) error {
 		pkgMetadata = createPackage(client, envName, srcArchiveName, deployArchiveName, buildcmd)
 	}
 
+	//TODO Warn user about resources at fn level overriding the env resources
+	resourceReq := getResourceReq(c.Int("mincpu"), c.Int("maxcpu"), c.Int("minmemory"), c.Int("maxmemory"))
+
+	var targetCPU int
+	if c.IsSet("targetcpu") {
+		targetCPU = c.Int("targetcpu")
+		if targetCPU <= 0 || targetCPU > 100 {
+			fatal("TargetCPU must be a value between 1 - 100")
+		}
+	} else {
+		targetCPU = 80
+	}
+
+	invokeStrategy := getInvokeStrategy(c.Int("minscale"), c.Int("maxscale"), c.String("executortype"), targetCPU)
+
 	function := &crd.Function{
 		Metadata: metav1.ObjectMeta{
 			Name:      fnName,
@@ -153,6 +204,8 @@ func fnCreate(c *cli.Context) error {
 					ResourceVersion: pkgMetadata.ResourceVersion,
 				},
 			},
+			Resources:      resourceReq,
+			InvokeStrategy: invokeStrategy,
 		},
 	}
 
@@ -364,10 +417,14 @@ func fnList(c *cli.Context) error {
 
 	w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
 
-	fmt.Fprintf(w, "%v\t%v\t%v\n", "NAME", "UID", "ENV")
+	fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\n", "NAME", "UID", "ENV", "EXECUTORTYPE", "MINSCALE", "MAXSCALE", "TARGETCPU")
 	for _, f := range fns {
-		fmt.Fprintf(w, "%v\t%v\t%v\n",
-			f.Metadata.Name, f.Metadata.UID, f.Spec.Environment.Name)
+		fmt.Fprintf(w, "%v\t%v\t%v\t%v\t%v\t%v\t%v\n",
+			f.Metadata.Name, f.Metadata.UID, f.Spec.Environment.Name,
+			f.Spec.InvokeStrategy.ExecutionStrategy.ExecutorType,
+			f.Spec.InvokeStrategy.ExecutionStrategy.MinScale,
+			f.Spec.InvokeStrategy.ExecutionStrategy.MaxScale,
+			f.Spec.InvokeStrategy.ExecutionStrategy.TargetCPUPercent)
 	}
 	w.Flush()
 

fission/main.go

@@ -36,6 +36,15 @@ func main() {
 	htMethodFlag := cli.StringFlag{Name: "method", Usage: "HTTP Method: GET|POST|PUT|DELETE|HEAD; defaults to GET"}
 	htUrlFlag := cli.StringFlag{Name: "url", Usage: "URL pattern (See gorilla/mux supported patterns)"}
 
+	// Resource & scale related flags (Used in env and function)
+	minCpu := cli.StringFlag{Name: "mincpu", Usage: "Minimum CPU to be assigned to pod (In millicore, minimum 1)"}
+	maxCpu := cli.StringFlag{Name: "maxcpu", Usage: "Maximum CPU to be assigned to pod (In millicore, minimum 1)"}
+	minMem := cli.StringFlag{Name: "minmemory", Usage: "Minimum memory to be assigned to pod (In megabyte)"}
+	maxMem := cli.StringFlag{Name: "maxmemory", Usage: "Maximum memory to be assigned to pod (In megabyte)"}
+	minScale := cli.StringFlag{Name: "minscale", Usage: "Minmum number of pods (Uses resource inputs to configure HPA)"}
+	maxScale := cli.StringFlag{Name: "maxscale", Usage: "Maximum number of pods (Uses resource inputs to configure HPA)"}
+	targetcpu := cli.StringFlag{Name: "targetcpu", Usage: "Target average CPU across pods for scaling (In percentage, defaults to 80)"}
+
 	// functions
 	fnNameFlag := cli.StringFlag{Name: "name", Usage: "function name"}
 	fnEnvNameFlag := cli.StringFlag{Name: "env", Usage: "environment name for function"}
@@ -54,12 +63,13 @@ func main() {
 	fnBuildCmdFlag := cli.StringFlag{Name: "buildcmd", Usage: "build command for builder to run with"}
 	fnLogCountFlag := cli.StringFlag{Name: "recordcount", Usage: "the n most recent log records"}
 	fnForceFlag := cli.BoolFlag{Name: "force", Usage: "Force update a package even if it is used by one or more functions"}
+	fnExecutorTypeFlag := cli.StringFlag{Name: "executortype", Usage: "Executor type for execution; one of 'poolmgr', 'newdeploy' defaults to 'poolmgr'"}
 
 	fnSubcommands := []cli.Command{
-		{Name: "create", Usage: "Create new function (and optionally, an HTTP route to it)", Flags: []cli.Flag{fnNameFlag, fnEnvNameFlag, fnCodeFlag, fnPackageFlag, fnSrcArchiveFlag, fnDeployArchiveFlag, fnEntryPointFlag, fnBuildCmdFlag, fnPkgNameFlag, htUrlFlag, htMethodFlag}, Action: fnCreate},
+		{Name: "create", Usage: "Create new function (and optionally, an HTTP route to it)", Flags: []cli.Flag{fnNameFlag, fnEnvNameFlag, fnCodeFlag, fnPackageFlag, fnSrcArchiveFlag, fnDeployArchiveFlag, fnEntryPointFlag, fnBuildCmdFlag, fnPkgNameFlag, htUrlFlag, htMethodFlag, minCpu, maxCpu, minMem, maxMem, minScale, maxScale, fnExecutorTypeFlag, targetcpu}, Action: fnCreate},
 		{Name: "get", Usage: "Get function source code", Flags: []cli.Flag{fnNameFlag}, Action: fnGet},
 		{Name: "getmeta", Usage: "Get function metadata", Flags: []cli.Flag{fnNameFlag}, Action: fnGetMeta},
-		{Name: "update", Usage: "Update function", Flags: []cli.Flag{fnNameFlag, fnEnvNameFlag, fnCodeFlag, fnPackageFlag, fnSrcArchiveFlag, fnDeployArchiveFlag, fnEntryPointFlag, fnPkgNameFlag, fnBuildCmdFlag, fnForceFlag}, Action: fnUpdate},
+		{Name: "update", Usage: "Update function source code", Flags: []cli.Flag{fnNameFlag, fnEnvNameFlag, fnCodeFlag, fnPackageFlag, fnSrcArchiveFlag, fnDeployArchiveFlag, fnEntryPointFlag, fnPkgNameFlag, fnBuildCmdFlag, fnForceFlag, minCpu, maxCpu, minMem, maxMem, minScale, maxScale, fnExecutorTypeFlag, targetcpu}, Action: fnUpdate},
 		{Name: "delete", Usage: "Delete function", Flags: []cli.Flag{fnNameFlag}, Action: fnDelete},
 		{Name: "list", Usage: "List all functions", Flags: []cli.Flag{}, Action: fnList},
 		{Name: "logs", Usage: "Display function logs", Flags: []cli.Flag{fnNameFlag, fnPodFlag, fnFollowFlag, fnDetailFlag, fnLogDBTypeFlag, fnLogCountFlag}, Action: fnLogs},
@@ -107,14 +117,16 @@ func main() {
 
 	// environments
 	envNameFlag := cli.StringFlag{Name: "name", Usage: "Environment name"}
+	envPoolsizeFlag := cli.IntFlag{Name: "poolsize", Usage: "Size of the pool, if not specified defaults to 3"}
 	envImageFlag := cli.StringFlag{Name: "image", Usage: "Environment image URL"}
 	envBuilderImageFlag := cli.StringFlag{Name: "builder", Usage: "Environment builder image URL (optional)"}
 	envBuildCmdFlag := cli.StringFlag{Name: "buildcmd", Usage: "Build command for environment builder to build source package (optional)"}
+
 	envVersionFlag := cli.IntFlag{Name: "version", Usage: "Environment API version: defaults to 1 (means v1 interface)"}
 	envSubcommands := []cli.Command{
-		{Name: "create", Aliases: []string{"add"}, Usage: "Add an environment", Flags: []cli.Flag{envNameFlag, envImageFlag, envBuilderImageFlag, envBuildCmdFlag, envVersionFlag}, Action: envCreate},
+		{Name: "create", Aliases: []string{"add"}, Usage: "Add an environment", Flags: []cli.Flag{envNameFlag, envPoolsizeFlag, envImageFlag, envBuilderImageFlag, envBuildCmdFlag, minCpu, maxCpu, minMem, maxMem, envVersionFlag}, Action: envCreate},
 		{Name: "get", Usage: "Get environment details", Flags: []cli.Flag{envNameFlag}, Action: envGet},
-		{Name: "update", Usage: "Update environment", Flags: []cli.Flag{envNameFlag, envImageFlag, envBuilderImageFlag, envBuildCmdFlag}, Action: envUpdate},
+		{Name: "update", Usage: "Update environment", Flags: []cli.Flag{envNameFlag, envPoolsizeFlag, envImageFlag, envBuilderImageFlag, envBuildCmdFlag, minCpu, maxCpu, minMem, maxMem}, Action: envUpdate},
 		{Name: "delete", Usage: "Delete environment", Flags: []cli.Flag{envNameFlag}, Action: envDelete},
 		{Name: "list", Usage: "List all environments", Flags: []cli.Flag{}, Action: envList},
 	}