“Naeel”
919e84396c
Problem
-------
The namespace reconciler (RunReconciler, added previously) retries namespaces
in NamespacePhaseFailed every 30s by calling DispatchResync. But the phase
could never actually reach NamespacePhaseFailed for the executor component
because the executor's NamespaceSubscriber always returned nil — swallowing
any SA-provisioning or informer-init errors. The reconciler was dead code for
the executor path.
Root cause chain
----------------
1. setupSAAndRoleBindings() — void, errors only logged internally.
2. EnsureNamespaceSA() — void, just called setupSAAndRoleBindings.
3. registerNamespace() — void, errors from both functions lost.
4. Executor AddFunc/ResyncFunc — always returned nil to dispatch().
5. dispatch() marks parts Active unconditionally → NamespacePhaseFailed
is never triggered for executor → RunReconciler never fires for executor.
Consequence: if EnsureNamespaceSA failed (transient k8s 503, RBAC webhook
timeout, etc.) the namespace appeared Active in the manager but the fetcher
ServiceAccount was missing. Pool pods would CrashLoopBackOff on every call
to that namespace until a full process restart.
Changes
-------
pkg/utils/serviceaccount.go
- setupSAAndRoleBindings: void → error. Returns the first k8s API error
so callers can decide whether to retry.
- runSACheck: ignores the error with _ = (same behaviour as before, it's
a periodic background loop that already logs internally).
- EnsureNamespaceSA: void → error, propagates setupSAAndRoleBindings.
Updated godoc to explain the retry contract.
pkg/executor/multitenant/ns_watcher.go
- registerNamespace: void → error.
* EnsureNamespaceSA error → wrapped as 'EnsureNamespaceSA: ...' and returned.
* registerExecutorTypes error → wrapped as 'registerExecutorTypes: ...' and returned.
* Success log line only emitted when both succeed.
- Added 'fmt' import for error wrapping.
pkg/executor/multitenant/namespace_subscriber.go
- AddFunc: return registerNamespace(...) instead of ignoring its error.
- ResyncFunc: same — plus a comment explaining why it is safe to call
registerNamespace again (SA creation is idempotent, executor-type
AddNamespace guards against duplicate informer creation).
pkg/utils/namespace_manager.go
- RunReconciler interface signature: added *zap.Logger parameter.
Callers pass the component logger so retries are visible in prod logs.
- RunReconciler implementation:
* Accepts logger; falls back to zap.NewNop() if nil.
* Skips the tick entirely when no failed namespaces are found (no log spam).
* Logs 'retrying failed namespaces' with count + list when found.
* Logs per-namespace 'dispatching resync'.
* Logs 'resync succeeded' or 'resync still failing, will retry' with error.
- RunManagedNamespaceWatcher: passes logger to RunReconciler.
End-to-end flow after this fix
-------------------------------
1. EnsureNamespaceSA fails (k8s 503).
2. registerNamespace returns error.
3. Executor AddFunc returns error.
4. dispatch() calls MarkPartFailed("executor") → deriveNamespacePhase →
NamespacePhaseFailed.
5. RunReconciler tick (30s) finds the namespace → DispatchResync →
registerNamespace called again → EnsureNamespaceSA (idempotent) →
if API recovered: success → MarkPartActive → NamespacePhaseActive.
6. Log line 'namespace reconciler: resync succeeded' confirms recovery.
Backward compatibility
----------------------
- NamespaceManager interface: RunReconciler gained a *zap.Logger param.
There is exactly one implementation (inMemoryNamespaceManager) and one
call site (RunManagedNamespaceWatcher). No external mocks.
- EnsureNamespaceSA: callers outside this codebase (if any) that ignore
the error will still compile (Go allows ignoring return values).
- All 26 affected tests pass: go test ./pkg/utils/... ./pkg/executor/...
./pkg/buildermgr/... ./pkg/router/...
Fission: Serverless Functions for Kubernetes
Fission is an open-source, Kubernetes-native serverless framework that simplifies the deployment of functions and applications on Kubernetes. With Fission, developers can easily create and deploy serverless functions that can be triggered by a variety of events, such as HTTP requests, messages from a message queue, or scheduled tasks.
Fission provides a simple, easy-to-use interface for developers to create serverless functions in their language of choice, without having to worry about the underlying infrastructure. The framework also offers automatic scaling, so functions can scale up or down based on demand, without any additional configuration.
Fission operates on just the code: Docker and Kubernetes are abstracted away under normal operation, though you can use both to extend Fission if you want to.
Fission is extensible to any language; the core is written in Go, and language-specific parts are isolated in something called environments (more below). Fission currently supports NodeJS, Python, Ruby, Go, PHP, Bash, and any Linux executable, with more languages coming soon.
Table of Contents
Performance: 100msec cold start
Fission maintains a pool of "warm" containers that each contain a small dynamic loader. When a function is first called, i.e. "cold-started", a running container is chosen and the function is loaded. This pool is what makes Fission fast: cold-start latencies are typically about 100msec.
Kubernetes is the right place for Serverless
We're built on Kubernetes because we think any non-trivial app will use a combination of serverless functions and more conventional microservices, and Kubernetes is a great framework to bring these together seamlessly.
Building on Kubernetes also means that anything you do for operations on your Kubernetes cluster — such as monitoring or log aggregation — also helps with ops on your Fission deployment.
Getting Started
# Add the stock NodeJS env to your Fission deployment
$ fission env create --name nodejs --image ghcr.io/fission/node-env
# Create a function with a javascript one-liner that prints "hello world"
$ fission function create --name hello --env nodejs --code https://raw.githubusercontent.com/fission/examples/master/nodejs/hello.js
# Run the function. This takes about 100msec the first time.
$ fission function test --name hello
Hello, world!
Learn More
- Understand Fission Concepts.
- See the installation guide for installing and running Fission.
- You can learn more about Fission and get started from Fission Docs.
- To see Fission in action, check out the Fission Examples Repo.
- See the troubleshooting guide for debugging your functions and Fission installation.
Contributing
Check out the contributing guide.
Who is using Fission?
- Fareye
- Apple
- iQuanti
- A large telecom CSP
- Gadget
- CinnamonAI
- Armo
- The Social Audience
- KubeML
- Unilever
- BD
- Biofourmis
- Babylon
Sponsors
The following companies, organizations, and individuals support Fission's ongoing maintenance and development. If you are using/contributing to Fission, we would be happy to list you here, please raise a Pull request.
License
Fission is licensed under the Apache License 2.0 - see the LICENSE file for details