This PR adds an interface for controller API client, it allows us to
implement mock API client for unit testing with ease and we are
able to generate spec file without accessing the real Fission server.
The pod template is embedded inside the deployment. So if
the pod annotation contains instance-id, the deployment
will get updated and thus triggers a rolling update whenever
a new executor starts which is unwanted.
After this PR, poolmanager will patches instance-id when a
pod is chosen for a function.
For newdeploy, unlike poolmanager manages the lifecycle
of function pod directly, newdeploy is only responsible
to create the deployment so we append instance-id to top-
level controller (deployment) only.
We used to update timestamp in the deployment environment field
in order to trigger a rolling update when the function referenced
resources get updated. However, use timestamp means we are not
able to avoid triggering a rolling update when executor tries to adopt
orphaned deployment due to timestamp changed which is unwanted.
In order to let executor adopt deployment without triggering a rolling
update, we need an identical way to get a value that can reflect res-
ources changed without affecting by time.
To achieve this goal, the sum of the resource version of all referenced
resources is a good fit for our scenario since the sum of the resource
version is always the same as long as no resources changed.
When a new executor starts up, it adopts the orphan kubernetes resources created
by the old executor instance. However, the adopted resource won't reflect the changes
come with the new executor, for example, the fetcher image inside won't be changed.
To solve this, executor updates the resource spec (HPA/Deployment/Service) with the
latest resources spec. By doing this, we can prevent the inconsistency between resources
created by different executor instance, also minimizes the impact on users.
Previously, once the executor is deleted for reasons (like upgrade or cluster scale-in),
the new executor deletes all existing resources created by the old executor and creates
new one. This mechanism becomes a problem when there are requests connecting to the
existing pods. Also in the worst case, the cluster may not have enough resources to create
new pods and cause service downtime.
This PR let each executor type adopts existing resources before starting the executor
API services, and so the alive connections won't experience failure. However, the requests
send to the function that doesn't have alive function pods will still fail due to the
executor is in bootstrapping.
The pool manager keeps terminating function pod periodically even there are
traffic to the function. The root cause is that executor, poolmgr, newdeploy
manage their own functionServiceCache separately. And when router taps a
function, executor updates the access time of the function service entry in its
own cache without notifying executor types to do the update as well. Hence,
the access time of function service entry in poolmanager cache never gets updated.
Due to the access time never gets updated, the idle pod reaper in poolmanager
then thinks the function pod is in idle state and recycle it.
This PR removes the cache in executor itself, and when router tries to tap a function,
executor will call executor type to tap the function and update access time.
The router taps function service one by one which is inefficient and
increases the burden of executor. This PR aggregates all requests into
one to solve the problem mentioned above.
When a function is created before the creation of the environment it's used, the newdeploy will not be able to create kube objs. Hence no function service record is inserted into the cache.
When getFuncSvc is called, the newdeploy tries to find the record in service cache in order to create kube objs with the same name used in previous kubeobjs creation. However, due to no record in the cache, a NotFound error is returned directly and causes the problem. To solve this, we use fn meta UID to ensure we always get the same obj name instead of getting it from the cache.
If the context of request is closed before ReverseProxy finishing writing
a huge response body to the response writer, the client will only receive
a truncated response body.
To solve this, move the context cancel after ReverseProxy finished.
To keep archive creation implementation simple and prevent any
confusion, we decided to remove `--keepurl` flag and embed URL
directly without downloading the file from it . In this way, we can
ensure consistent behavior in either package creation or spec file
creation. Also, it increases the portability of spec file.
