Optimize function latency when cache expired/invalid under high concurrency (#856)

* Optimize router response time by adding update lock

In 0.9.2, the router sends multiple GetServiceForFunction requests to executor
to get the service URL. However, the response time of executor will increase
under high-concurrency situation due to too many requests are waiting for
processing.

To solve the problem, an update lock was added to the router. All of goroutines belongs
to the same function need to grab the update lock before sending the request.
Only the first goroutine which gets the update lock is allowed to send request.
In this way, we reduce the burden of executor and lower the failure rate.

charts/fission-all/templates/deployment.yaml

@@ -200,6 +200,10 @@ spec:
             value: {{ .Values.routerRoundTripKeepAliveTime | default "30s" | quote }}
           - name: ROUTER_ROUND_TRIP_MAX_RETRIES
             value: {{ .Values.routerRoundTripMaxRetries | default 10 | quote }}
+          - name: ROUTER_ROUND_TRIP_SVC_ADDRESS_MAX_RETRIES
+            value: {{ .Values.routerRoundTripSvcAddressMaxRetries | default 5 | quote }}
+          - name: ROUTER_ROUND_TRIP_SVC_ADDRESS_UPDATE_TIMEOUT
+            value: {{ .Values.routerRoundTripSvcAddressUpdateTimeout | default 30 | quote }}
           - name: DEBUG_ENV
             value: {{ .Values.debugEnv | quote }}
         readinessProbe:

charts/fission-core/templates/deployment.yaml

@@ -197,6 +197,10 @@ spec:
             value: {{ .Values.routerRoundTripKeepAliveTime | default "30s" | quote }}
           - name: ROUTER_ROUND_TRIP_MAX_RETRIES
             value: {{ .Values.routerRoundTripMaxRetries | default 10 | quote }}
+          - name: ROUTER_ROUND_TRIP_SVC_ADDRESS_MAX_RETRIES
+            value: {{ .Values.routerRoundTripSvcAddressMaxRetries | default 5 | quote }}
+          - name: ROUTER_ROUND_TRIP_SVC_ADDRESS_UPDATE_TIMEOUT
+            value: {{ .Values.routerRoundTripSvcAddressUpdateTimeout | default 30 | quote }}
           - name: DEBUG_ENV
             value: {{ .Values.debugEnv | quote }}
         readinessProbe: