Add function logs support (#53) (#131)

Add function log aggregation and persistence using Fluentd and InfluxDB.

Fluentd and a helper sidecar run as a daemonset.  The poolmgr sets up logging for each function pod, using the helper sidecar. Fluentd forwards logs to InfluxDB, which is run as a deployment and service. The client CLI directly queries InfluxDB for logs.

Fluentd supports many outputs besides InfluxDB, so we aren't very tied to InfluxDB.

The setup is somewhat manual, which we should be able to improve by integrating this into the helm chart.

Diagram of component interactions: https://cloud.githubusercontent.com/assets/202578/23100399/b0e3ea00-f6ba-11e6-8f2f-6588cfef2e84.png
This commit is contained in:
Ta-Ching Chen
2017-03-22 15:59:42 -07:00
committed by Soam Vasani
parent 1665235c14
commit a13015e75a
20 changed files with 1029 additions and 14 deletions
Generated
+12 -13
View File
@@ -1,5 +1,5 @@
hash: fb2f38693c5f2fade3db9434c266743d0f9ea9b5f878b44e79763e248136dcf2
updated: 2016-11-14T01:06:12.764919869-08:00
hash: 6b4c050c1cd5a7cbb99141348020c882d088b30a9fe9cf92b78a7f706e59da4c
updated: 2017-02-22T23:15:19.862789385+08:00
imports:
- name: github.com/blang/semver
version: 60ec3488bfea7cca02b021d106d9911120d25fe9
@@ -69,9 +69,15 @@ imports:
- name: github.com/gorilla/context
version: 08b5f424b9271eedf6f9f0ce86cb9396ed337a42
- name: github.com/gorilla/handlers
version: ee54c7b44cab12289237fb8631314790076e728b
version: 3a5767ca75ece5f7f1440b1d16975247f8d8b221
- name: github.com/gorilla/mux
version: 0eeaf8392f5b04950925b8a69fe70f110fa7cbfc
version: 392c28fe23e1c45ddba891b0320b3b5df220beea
- name: github.com/influxdata/influxdb
version: b7bb7e8359642b6e071735b50ae41f5eb343fd42
subpackages:
- client/v2
- models
- pkg/escape
- name: github.com/jonboulle/clockwork
version: 2eee05ed794112d45db504eb05aa693efd2b8b09
- name: github.com/juju/ratelimit
@@ -91,7 +97,7 @@ imports:
- name: github.com/satori/go.uuid
version: 879c5887cd475cd7864858769793b2ceb0d44feb
- name: github.com/Sirupsen/logrus
version: d26492970760ca5d33129d2d799e34be5c4782eb
version: c078b1e43f58d563c74cebe63c85789e76ddb627
- name: github.com/spf13/pflag
version: 08b1a584251b5b62f458943640fc8ebd4d50aaa5
- name: github.com/ugorji/go
@@ -99,7 +105,7 @@ imports:
subpackages:
- codec
- name: github.com/urfave/cli
version: a14d7d367bc02b1f57d88de97926727f2d936387
version: 0bdeddeeb0f650497d603c4ad7b20cfe685682f6
- name: golang.org/x/net
version: 6acef71eb69611914f7a30939ea9f6e194c78172
subpackages:
@@ -158,13 +164,6 @@ imports:
- name: k8s.io/client-go
version: 843f7c4f28b1f647f664f883697107d5c02c5acc
subpackages:
- 1.4/kubernetes
- 1.4/pkg/api
- 1.4/pkg/api/v1
- 1.4/pkg/apis/extensions/v1beta1
- 1.4/pkg/labels
- 1.4/pkg/util/intstr
- 1.4/rest
- 1.5/discovery
- 1.5/kubernetes
- 1.5/kubernetes/typed/apps/v1alpha1