Add function logs support (#53) (#131)

Add function log aggregation and persistence using Fluentd and InfluxDB.

Fluentd and a helper sidecar run as a daemonset.  The poolmgr sets up logging for each function pod, using the helper sidecar. Fluentd forwards logs to InfluxDB, which is run as a deployment and service. The client CLI directly queries InfluxDB for logs.

Fluentd supports many outputs besides InfluxDB, so we aren't very tied to InfluxDB.

The setup is somewhat manual, which we should be able to improve by integrating this into the helm chart.

Diagram of component interactions: https://cloud.githubusercontent.com/assets/202578/23100399/b0e3ea00-f6ba-11e6-8f2f-6588cfef2e84.png
This commit is contained in:
Ta-Ching Chen
2017-03-22 15:59:42 -07:00
committed by Soam Vasani
parent 1665235c14
commit a13015e75a
20 changed files with 1029 additions and 14 deletions
+44
View File
@@ -7,6 +7,7 @@
* [Get and Run Fission: GKE or other Cloud](#get-and-run-fission-gke-or-other-cloud)
* [Install the client CLI](#install-the-client-cli)
* [Run an example](#run-an-example)
* [Enable Persistent Function Logs (Optional)](#enable-persistent-function-logs)
## Running Fission on your Cluster
@@ -110,3 +111,46 @@ Finally, you're ready to use Fission!
$ curl http://$FISSION_ROUTER/hello
Hello, world!
```
### Enable Persistent Function Logs (Optional)
Fission uses InfluxDB to store logs and fluentd to forward them from
function pods into InfluxDB. To setup both InfluxDB and fluentd:
Edit `fission-logger.yaml` to add a username and password for the
Influxdb deployment. Then:
```
$ kubectl create -f fission-logger.yaml
```
On the client side,
If you're using minikube or a local cluster:
```
$ export FISSION_LOGDB=http://$(minikube ip):31315
```
If you're using GKE or other cloud:
```
$ export FISSION_LOGDB=http://$(kubectl --namespace fission get svc influxdb -o=jsonpath='{..ip}'):8086
```
That's it for setup. You can now use this to view function logs:
```
$ fission function logs --name hello
```
You can also list the all the pods that have hosted the function
(including ones that aren't alive any more) and view logs for a
particular pod:
```
$ fission function pods --name hello
$ fission function logs --name hello --pod <pod name>
```