Feature request per pod (#1946)

This feature enables routing more than one request to a pod at the same time. This is the first draft of the work and might involve more optimizations later.
2021-03-31 12:35:30 +05:30
parent 3e6dae9616
commit 25acd7fd16
39 changed files with 511 additions and 264 deletions
@@ -353,6 +353,10 @@ type (
 		// Maximum number of pods to be specialized which will serve requests
 		// This is optional. If not specified default value will be taken as 5
 		Concurrency int `json:"concurrency,omitempty"`
+
+		// RequestsPerPod indicates the maximum number of concurrent requests that can be served by a specialized pod
+		// This is optional. If not specified default value will be taken as 1
+		RequestsPerPod int `json:"requestsPerPod,omitempty"`
 	}

 	// InvokeStrategy is a set of controls over how the function executes.