Add function log aggregation and persistence using Fluentd and InfluxDB. Fluentd and a helper sidecar run as a daemonset. The poolmgr sets up logging for each function pod, using the helper sidecar. Fluentd forwards logs to InfluxDB, which is run as a deployment and service. The client CLI directly queries InfluxDB for logs. Fluentd supports many outputs besides InfluxDB, so we aren't very tied to InfluxDB. The setup is somewhat manual, which we should be able to improve by integrating this into the helm chart. Diagram of component interactions: https://cloud.githubusercontent.com/assets/202578/23100399/b0e3ea00-f6ba-11e6-8f2f-6588cfef2e84.png
This commit is contained in:
committed by
Soam Vasani
parent
1665235c14
commit
a13015e75a
@@ -110,4 +110,22 @@ be loaded with any function.
|
||||
When poolmgr needs to create a service for a function, it calls
|
||||
fetcher to fetch the function. Fetcher downloads the function into a
|
||||
volume shared between fetcher and this environment container. Poolmgr
|
||||
then requests the container to load the function.
|
||||
then requests the container to load the function.
|
||||
|
||||
Logger
|
||||
-----------
|
||||
|
||||
Logger helps to forward function logs to centralized db service for log
|
||||
persistence. Currently only influxdb is supported to store logs.
|
||||
Following is a diagram describe how log service works:
|
||||
|
||||

|
||||
|
||||
1. Pool manager choose a pod from pool to execute user function
|
||||
2. Pool manager makes a HTTP POST to logger helper, once helper receives
|
||||
the request it creates a symlink to container log for fluentd.
|
||||
3. Fluentd reads log from symlink and pipes to influxdb
|
||||
4. `fission function logs ...` retrieve event logs from influxdb with
|
||||
optional log filter
|
||||
5. Pool manager removes function pod from pool
|
||||
6. Pool manager asks logger helper to stop piping logs, logger removes symlink.
|
||||
Reference in New Issue
Block a user