These changes have specifically been made for Kafka connector. This will expose a new metric named `fission_mqt_message_lag`, which will show a number of messages lag per topic and partition.
We can use this metric in the auto-scaling of the pod for the new deploy type executor function. While creating a new deploy function we need to add hpa metrics of external type inside the function definition.
We are removing Fission deprecated Azure Storage Queue connector and
planning to adopt Keda going forward to have better
delegated functionality and more rich support.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
We are removing Fission deprecated Nats connector and
planning to adopt Keda going forward to have better
delegated functionality and more rich support.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
- Use mqtrigger watch instead of polling mqtrigger every 5 seconds
- Added metrics to monitor no of subscriptions, and no of messages per subscription
- Add standard go metrics exported by prometheus
- Enable prometheus discovery for mqtrigger pod
- Optimized mqtrigger manager cache
- Add unit tests for mqtrigger cache
Co-authored-by: Sanket Sudake <sanketsudake@gmail.com>
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* Retrieve pod metrics only if metrics server is running
Currently we query pod metrics every 30 sec which floods executor logs,
added check which confirms if metrics server is running then only we start
querying pod metrics for identifying CPU utilization.
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Fixed couple of typos and misspells with Go CI
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
* Remove unnecessary conversions with Go CI
Signed-off-by: Sanket Sudake <sanketsudake@gmail.com>
This PR removes not so useful logs and changes most of Info level
log to Debug/Error level in hot path while preserving some of them
that is helpful for troubleshooting.
