Documentation Revamp (#496)

Added a new documentation structure with details of concepts, tutorials and examples
This commit is contained in:
Vishal
2018-02-28 11:46:44 +05:30
committed by GitHub
parent 5e2f984136
commit 1b1c4f1380
25 changed files with 925 additions and 192 deletions
@@ -0,0 +1,10 @@
---
title: "Using Fission"
chapter: true
draft: false
weight: 40
---
# Using Fission
### Usage guides, tutorials and examples
@@ -0,0 +1,105 @@
---
title: "Accessing Secret/configmap in function"
draft: false
weight: 47
---
From fission v0.5.0 and later, functions are able to access [Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) and [ConfigMaps](https://kubernetes.io/docs/concepts/storage/volumes/#configmap) specified by users.
### Create Secret and ConfigMap
You can create Secret and ConfigMap with CLI.
``` bash
$ kubectl -n default create secret generic foo --from-literal=TEST_KEY="TESTVALUE"
$ kubectl -n default create configmap bar --from-literal=TEST_KEY=TESTVALUE
```
Or use `kubectl create -f <filename.yaml>` to create these from a YAML file.
``` yaml
apiVersion: v1
kind: Secret
metadata:
namespace: default
name: foo
data:
TEST_KEY: VEVTVFZBTFVF # value after base64 encode
type: Opaque
---
apiVersion: v1
kind: ConfigMap
metadata:
namespace: default
name: bar
data:
TEST_KEY: TESTVALUE
```
### Access Secret and ConfigMap
Since content of Secret and ConfigMap are key-value pairs, functions can access them with following paths:
``` bash
# Secret path
/secrets/<namespace>/<name>/<key>
# ConfigMap path
/configs/<namespace>/<name>/<key>
```
From the previous example, the paths are:
``` bash
# secret foo
/secrets/default/foo/TEST_KEY
# confimap bar
/configs/default/bar/TEST_KEY
```
Now, let's create a simple python function (leaker.py) that return value of Secret `foo` and ConfigMap `bar`.
``` python
# leaker.py
def main():
path = "/configs/default/bar/TEST_KEY"
f = open(path, "r")
config = f.read()
path = "/secrets/default/foo/TEST_KEY"
f = open(path, "r")
secret = f.read()
msg = "ConfigMap: %s\nSecret: %s" % (config, secret)
return msg, 200
```
Create environment, function and http trigger.
``` bash
# create python env
$ fission env create --name python --image fission/python-env
# create function named "leaker"
$ fission fn create --name leaker --env python --code leaker.py --secret foo --configmap bar
# create route(http trigger)
$ fission route create --function leaker --url /leaker --method GET
```
Try to access the function, the output should look like following.
``` bash
$ curl http://$FISSION_ROUTER/leaker
ConfigMap: TESTVALUE
Secret: TESTVALUE
```
Note: If the Secret or ConfigMap value is updated, the function may not get the updated value for some time; it may get a cached older value.
@@ -0,0 +1,37 @@
---
title: "Environment"
draft: false
weight: 42
---
### Create an environment
You can create an environment on your cluster from an image for that language. Optionally, you can specify CPU and memory resource limits. You can also specify the number of initially pre-warmed pods, which is called the poolsize.
```
fission env create --name node --image fission/node-env:0.4.0 --mincpu 40 --maxcpu 80 --minmemory 64 --maxmemory 128 --poolsize 4
```
In case of pool based executor, the resources specified for environment are used for function pod as well. In case of new deployment executor, you can override the resources when you create a function.
### Using a builder
When you create an environment, you can specify a builder image and builder command which will be used for building from source code. You can override the build command when creating a function. For more details on builder and packages you should check out examples in [Functions](../functions) and [packages](../package)
```
fission env create --name python --image fission/python-env:latest --builder fission/python-builder:latest
```
### Viweing environment information
You can list the environments or view information of an individual environment:
```
$ fission env list
NAME UID IMAGE POOLSIZE MINCPU MAXCPU MINMEMORY MAXMEMORY
node ac84d62e-001f-11e8-85c9-42010aa00010 fission/node-env:0.4.0 4 40m 80m 64Mi 128Mi
$
$ fission env get --name node
NAME UID IMAGE
node ac84d62e-001f-11e8-85c9-42010aa00010 fission/node-env:0.4.0
```
@@ -0,0 +1,86 @@
---
title: "Controlling Function Execution"
draft: false
weight: 43
---
### Autoscaling
Let's create a function to demonstrate the autoscaling behaviour in Fission. We create a simple function which outputs "Hello World" in using NodeJS. We have kept the CPU request and limit purposefully low to simulate the load and also kept the target CPU percent to 50%.
```
$ fission fn create --name hello --env node --code hello.js --mincpu 10 --maxcpu 40 --minmemory 64 --maxmemory 128 --minscale 1 --maxscale 6 --executortype newdeploy --targetcpu 50
function 'hello' created
```
Now let's use [hey](https://github.com/rakyll/hey) to generate the load with 250 concurrent and a total of 10000 requests:
```
$ hey -c 250 -n 10000 http://$FISSION_ROUTER/hello
Summary:
Total: 67.3535 secs
Slowest: 4.6192 secs
Fastest: 0.0177 secs
Average: 1.6464 secs
Requests/sec: 148.4704
Total data: 160000 bytes
Size/request: 16 bytes
Response time histogram:
0.018 [1] |
0.478 [486] |∎∎∎∎∎∎∎
0.938 [971] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎
1.398 [2686] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
1.858 [2326] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
2.318 [1641] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
2.779 [1157] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
3.239 [574] |∎∎∎∎∎∎∎∎∎
3.699 [120] |∎∎
4.159 [0] |
4.619 [38] |∎
Latency distribution:
10% in 0.7037 secs
25% in 1.1979 secs
50% in 1.5038 secs
75% in 2.1959 secs
90% in 2.6670 secs
95% in 2.8855 secs
99% in 3.4102 secs
Details (average, fastest, slowest):
DNS+dialup: 0.0058 secs, 0.0000 secs, 1.0853 secs
DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs
req write: 0.0000 secs, 0.0000 secs, 0.0026 secs
resp wait: 1.6405 secs, 0.0176 secs, 3.6144 secs
resp read: 0.0001 secs, 0.0000 secs, 0.0056 secs
Status code distribution:
[200] 10000 responses
```
While the load is being generated, we will watch the HorizontalPodAutoscaler and how it scales over period of time. As you can notice, the number of pods is scaled from 1 to 3 after the load rises from 8 - 103%. After the load generator stops, it takes a few iterations to scale down from 3 to 1 pod.
```
$ k -n fission-function get hpa -w
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
hello-qoxmothj Deployment/hello-qoxmothj 5% / 50% 1 6 1 3m
hello-qoxmothj Deployment/hello-qoxmothj 8% / 50% 1 6 1 3m
hello-qoxmothj Deployment/hello-qoxmothj 103% / 50% 1 6 1 4m
hello-qoxmothj Deployment/hello-qoxmothj 103% / 50% 1 6 3 5m
hello-qoxmothj Deployment/hello-qoxmothj 25% / 50% 1 6 3 5m
hello-qoxmothj Deployment/hello-qoxmothj 25% / 50% 1 6 3 6m
hello-qoxmothj Deployment/hello-qoxmothj 5% / 50% 1 6 3 6m
hello-qoxmothj Deployment/hello-qoxmothj 5% / 50% 1 6 3 7m
hello-qoxmothj Deployment/hello-qoxmothj 5% / 50% 1 6 3 7m
hello-qoxmothj Deployment/hello-qoxmothj 5% / 50% 1 6 3 8m
hello-qoxmothj Deployment/hello-qoxmothj 5% / 50% 1 6 3 8m
hello-qoxmothj Deployment/hello-qoxmothj 5% / 50% 1 6 3 9m
hello-qoxmothj Deployment/hello-qoxmothj 5% / 50% 1 6 3 9m
hello-qoxmothj Deployment/hello-qoxmothj 5% / 50% 1 6 3 10m
hello-qoxmothj Deployment/hello-qoxmothj 5% / 50% 1 6 3 10m
hello-qoxmothj Deployment/hello-qoxmothj 7% / 50% 1 6 1 11m
hello-qoxmothj Deployment/hello-qoxmothj 6% / 50% 1 6 1 11m
hello-qoxmothj Deployment/hello-qoxmothj 6% / 50% 1 6 1 12m
hello-qoxmothj Deployment/hello-qoxmothj 6% / 50% 1 6 1 12m
```
@@ -0,0 +1,238 @@
---
title: "Function"
draft: false
weight: 41
---
### Create a function
Before creating a function the environment should be created, we will assume that you have already created environment named `node`.
Let's create a simple code snippet in nodejs which will output Hello world:
```
module.exports = async function(context) {
return {
status: 200,
body: "Hello, world!\n"
};
}
```
Let's create a route for the function which can be used for making HTTP requests:
```
$ fission route create --function hello --url /hello
trigger '5327e9a7-6d87-4533-a4fb-c67f55b1e492' created
```
Let's create a function based on pool based executor.
```
fission fn create --name hello --code hello.js --env node --executortype poolmgr
```
When you hit this function's URL , you get a response:
```
$ curl http://$FISSION_ROUTER/hello
Hello, world!
```
Similarly you can create a new deployment executor type function and provide minmum and maximum scale for the function.
```
fission fn create --name hello --code hello.js --env node --minscale 1 --maxscale 5 --executortype newdeploy
```
### View & update function source code
You can look at the source code associated with given function:
```
$ fission fn get --name hello
module.exports = async function(context) {
return {
status: 200,
body: "Hello, world!\n"
};
}
```
Let's say you want to update the function to output "Hello Fission" instead of "Hello world", you can update the source file and update the source code for function:
```
$ fission fn update --name hello --code ../hello.js
package 'hello-js-ku9s' updated
function 'hello' updated
```
Let's verify that the function now respond with a different output than earlier:
```
$ curl http://$FISSION_ROUTER/hello
Hello, Fission!
```
### Test and debug function
You can directly test a function using test command. If the function call succeeds, it will output the function's response.
```
$ fission fn test --name hello
Hello, Fission!
```
But if there is an error in function execution then the logs of function execution are displayed:
```
$ fission fn test --name hello
Error calling function hello: 500 Internal server error (fission)
> fission-nodejs-runtime@0.1.0 start /usr/src/app
> node server.js
Codepath defaulting to /userfunc/user
Port defaulting to 8888
user code load error: SyntaxError: Unexpected token function
::ffff:10.8.1.181 - - [16/Feb/2018:08:44:33 +0000] "POST /specialize HTTP/1.1" 500 2 "-" "Go-http-client/1.1"
```
You can also look at function execution logs explicitly:
```
$ fission fn logs --name hello
[2018-02-16 08:41:43 +0000 UTC] 2018/02/16 08:41:43 fetcher received fetch request and started downloading: {1 {hello-js-rqew default 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] nil [] } user [] []}
[2018-02-16 08:41:43 +0000 UTC] 2018/02/16 08:41:43 Successfully placed at /userfunc/user
[2018-02-16 08:41:43 +0000 UTC] 2018/02/16 08:41:43 Checking secrets/cfgmaps
[2018-02-16 08:41:43 +0000 UTC] 2018/02/16 08:41:43 Completed fetch request
[2018-02-16 08:41:43 +0000 UTC] 2018/02/16 08:41:43 elapsed time in fetch request = 89.844653ms
[2018-02-16 08:41:43 +0000 UTC] user code loaded in 0sec 4.235593ms
[2018-02-16 08:41:43 +0000 UTC] ::ffff:10.8.1.181 - - [16/Feb/2018:08:41:43 +0000] "POST /specialize HTTP/1.1" 202 - "-" "Go-http-client/1.1"
[2018-02-16 08:41:43 +0000 UTC] ::ffff:10.8.1.182 - - [16/Feb/2018:08:41:43 +0000] "GET / HTTP/1.1" 200 16 "-" "curl/7.54.0"
```
### Fission builds & compiled artifacts
Most real world functions will require more than one source files. It is also easier to simply provide source files and let Fission take care of building from source files. Fission provides first class support for building from source as well as using compiled artifacts to create functions.
You can attach the source/deployment packages to a function or explicitly create packages and use them across functions. Check documentation for [package](../package) for more information.
#### Building function from source
Let's take a simple python function which has dependency on a python pyyaml module. We can specify the dependencies in requirements.txt and a simple command to build from source. The tree structure of directory looks like:
```
sourcepkg/
├── __init__.py
├── build.sh
├── requirements.txt
└── user.py
```
And the file contents:
```
$ cat user.py
import sys
import yaml
document = """
a: 1
b:
c: 3
d: 4
"""
def main():
return yaml.dump(yaml.load(document))
$ cat requirements.txt
pyyaml
$ cat build.sh
#!/bin/sh
pip3 install -r ${SRC_PKG}/requirements.txt -t ${SRC_PKG} && cp -r ${SRC_PKG} ${DEPLOY_PKG}
```
You first need to create an environment with environment image and python-builder image specified:
```
$fission env create --name python --image fission/python-env:latest --builder fission/python-builder:latest --mincpu 40 --maxcpu 80 --minmemory 64 --maxmemory 128 --poolsize 2
```
Now let's zip the directory containing the source files and create a function with source package:
```
$zip -jr demo-src-pkg.zip sourcepkg/
adding: __init__.py (stored 0%)
adding: build.sh (deflated 24%)
adding: requirements.txt (stored 0%)
adding: user.py (deflated 25%)
$ fission fn create --name hellopy --env python --src demo-src-pkg.zip --entrypoint "user.main" --buildcmd "./build.sh"
function 'hellopy' created
$ fission route create --function hellopy --url /hellopy
```
Once we create the function, the build process is started. You can check logs of the builder in fission-builder namespace:
```
$ k -n fission-builder logs -f py3-4214348-59555d9bd8-ks7m4 builder
2018/02/16 11:44:21 Builder received request: {demo-src-pkg-zip-ninf-djtswo ./build.sh}
2018/02/16 11:44:21 Starting build...
=== Build Logs ===command=./build.sh
env=[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=py3-4214348-59555d9bd8-ks7m4 PYTHON_4212095_PORT_8000_TCP_PROTO=tcp PY3_4214348_SERVICE_HOST=10.11.250.161 KUBERNETES_PORT=tcp://10.11.240.1:443 PYTHON_4212095_PORT=tcp://10.11.244.134:8000 PYTHON_4212095_PORT_8000_TCP=tcp://10.11.244.134:8000 PYTHON_4212095_PORT_8001_TCP_PROTO=tcp PYTHON_4212095_PORT_8001_TCP_ADDR=10.11.244.134 PY3_4214348_SERVICE_PORT=8000 PY3_4214348_SERVICE_PORT_BUILDER_PORT=8001 PY3_4214348_PORT_8001_TCP=tcp://10.11.250.161:8001 KUBERNETES_PORT_443_TCP_PORT=443 KUBERNETES_PORT_443_TCP_ADDR=10.11.240.1 PY3_4214348_SERVICE_PORT_FETCHER_PORT=8000 PY3_4214348_PORT_8000_TCP=tcp://10.11.250.161:8000 PY3_4214348_PORT_8001_TCP_PORT=8001 PYTHON_4212095_SERVICE_PORT_FETCHER_PORT=8000 PYTHON_4212095_PORT_8000_TCP_ADDR=10.11.244.134 KUBERNETES_SERVICE_HOST=10.11.240.1 PY3_4214348_PORT=tcp://10.11.250.161:8000 PYTHON_4212095_SERVICE_PORT_BUILDER_PORT=8001 PYTHON_4212095_PORT_8001_TCP=tcp://10.11.244.134:8001 PY3_4214348_PORT_8000_TCP_PROTO=tcp PY3_4214348_PORT_8000_TCP_PORT=8000 KUBERNETES_SERVICE_PORT_HTTPS=443 KUBERNETES_PORT_443_TCP=tcp://10.11.240.1:443 PYTHON_4212095_PORT_8001_TCP_PORT=8001 PY3_4214348_PORT_8000_TCP_ADDR=10.11.250.161 PY3_4214348_PORT_8001_TCP_PROTO=tcp KUBERNETES_SERVICE_PORT=443 PYTHON_4212095_SERVICE_PORT=8000 PYTHON_4212095_PORT_8000_TCP_PORT=8000 PY3_4214348_PORT_8001_TCP_ADDR=10.11.250.161 KUBERNETES_PORT_443_TCP_PROTO=tcp PYTHON_4212095_SERVICE_HOST=10.11.244.134 HOME=/root SRC_PKG=/packages/demo-src-pkg-zip-ninf-djtswo DEPLOY_PKG=/packages/demo-src-pkg-zip-ninf-djtswo-c40gfu]
Collecting pyyaml (from -r /packages/demo-src-pkg-zip-ninf-djtswo/requirements.txt (line 1))
Downloading PyYAML-3.12.tar.gz (253kB)
Installing collected packages: pyyaml
Running setup.py install for pyyaml: started
Running setup.py install for pyyaml: finished with status 'done'
Successfully installed pyyaml-3.12
==================
2018/02/16 11:44:24 elapsed time in build request = 3.460498847s
```
Once the build has succeeded, you can hit the function URL to test the function:
```
$curl http://$FISSION_ROUTER/hellopy
a: 1
b: {c: 3, d: 4}
```
#### Using compiled artifacts with Fission
In some cases you have a pre-built deployment package which you need to deploy to Fission. For this example let's use a simple python file as a deployment package but in practice it can be any other compiled package.
We will use a simple python file in a directory and turn it into a deployment package:
```
$ cat testDir/hello.py
def main():
return "Hello, world!"
$zip -jr demo-deploy-pkg.zip testDir/
```
Let's use the deployment package to create a function and route and then test it.
```
$ fission fn create --name hellopy --env python --deploy demo-deploy-pkg.zip --entrypoint "hello.main"
function 'hellopy' created
$ fission route create --function hellopy --url /hellopy
$ curl http://$FISSION_ROUTER/hellopy
Hello, world!
```
### View function information
You can retrieve metadata information of a single function or list all functions to look at basic information of functions:
```
$ fission fn getmeta --name hello
NAME UID ENV
hello 34234b50-12f5-11e8-85c9-42010aa00010 node
$ fission fn list
NAME UID ENV EXECUTORTYPE MINSCALE MAXSCALE TARGETCPU
hello 34234b50-12f5-11e8-85c9-42010aa00010 node poolmgr 0 1 80
hello2 e37a46e3-12f4-11e8-85c9-42010aa00010 node newdeploy 1 5 80
```
@@ -0,0 +1,130 @@
---
title: "Packaging source code"
draft: false
weight: 46
---
### Creating source package
Before you create a package, you need to create an environment with associated builder image:
```
$ fission env create --name pythonsrc --image fission/python-env:latest --builder fission/python-builder:latest --mincpu 40 --maxcpu 80 --minmemory 64 --maxmemory 128 --poolsize 2
environment 'pythonsrc' created
```
Let's take a simple python function which has dependency on a python pyyaml module. We can specify the dependencies in requirements.txt and a simple command to build from source. The tree structure of directory and contents of the file looks like:
```
sourcepkg/
├── __init__.py
├── build.sh
├── requirements.txt
└── user.py
```
And the file contents:
```
$ cat user.py
import sys
import yaml
document = """
a: 1
b:
c: 3
d: 4
"""
def main():
return yaml.dump(yaml.load(document))
$ cat requirements.txt
pyyaml
$ cat build.sh
#!/bin/sh
pip3 install -r ${SRC_PKG}/requirements.txt -t ${SRC_PKG} && cp -r ${SRC_PKG} ${DEPLOY_PKG}
$zip -jr demo-src-pkg.zip sourcepkg/
adding: __init__.py (stored 0%)
adding: build.sh (deflated 24%)
adding: requirements.txt (stored 0%)
adding: user.py (deflated 25%)
```
Using the source archive creared in previous step, you can create a package in Fission:
```
$ fission package create --sourcearchive demo-src-pkg.zip --env pythonsrc --buildcmd "./build.sh"
Package 'demo-src-pkg-zip-8lwt' created
```
Since we are working with source package, we provided the build command. Once you create the package, the build process will start and you can check the build logs by getting information of the package:
```
$ fission pkg info --name demo-src-pkg-zip-8lwt
Name: demo-src-pkg-zip-8lwt
Environment: pythonsrc
Status: succeeded
Build Logs:
Collecting pyyaml (from -r /packages/demo-src-pkg-zip-8lwt-v57qil/requirements.txt (line 1))
Using cached PyYAML-3.12.tar.gz
Installing collected packages: pyyaml
Running setup.py install for pyyaml: started
Running setup.py install for pyyaml: finished with status 'done'
Successfully installed pyyaml-3.12
```
Using the package above you can create the function. Since package already is associated with a source package, environment and build command, these will be ignored when creating a function. Only addition thing you will need to provide is the entrypoint. Assuming you hace created the route, the function should be reachable with successful output:
```
$ fission fn create --name srcpy --pkg demo-src-pkg-zip-8lwt --entrypoint "user.main"
function 'srcpy' created
$ curl http://$FISSION_ROUTER/srcpy
a: 1
b: {c: 3, d: 4}
```
### Creating deployment package
Before you create a package you need to create an environment with the builder image:
```
$ fission env create --name pythondeploy --image fission/python-env:latest --builder fission/python-builder:latest --mincpu 40 --maxcpu 80 --minmemory 64 --maxmemory 128 --poolsize 2
environment 'pythonsrc' created
```
We will use a simple Python example which outputs "Hello World!" in a directory to create a deployment archive:
```
$ cat testDir/hello.py
def main():
return "Hello, world!"
$zip -jr demo-deploy-pkg.zip testDir/
```
Using the archive and environments created previously, you can create a package:
```
$ fission package create --deployarchive demo-deploy-pkg.zip --env pythondeploy
Package 'demo-deploy-pkg-zip-whzl' created
```
Since it is a deployment archive, there is no need to build it, hence the build logs for the package will be empty:
```
$ fission package info --name demo-deploy-pkg-zip-whzl
Name: demo-deploy-pkg-zip-xlaw
Environment: pythondeploy2
Status: succeeded
Build Logs:
```
Finally you can create a function with the package and test the function:
```
$fission fn create --name deploypy --pkg demo-deploy-pkg-zip-whzl --entrypoint "hello.main"
$curl http://$FISSION_ROUTER/deploypy
Hello, world!
```
@@ -0,0 +1,48 @@
---
title: "Trigger"
draft: false
weight: 44
---
### Create a HTTP Trigger
You can create a HTTP trigger with default method (GET) for a function:
```
$ fission ht create --url /hello --function hello
trigger '94cd5163-30dd-4fb2-ab3c-794052f70841' created
```
### Create a Time Trigger
Time based triggers can be created with cron specifications:
```
$ fission tt create --name halfhourly --function hello --cron "0 30 * * *"
trigger 'halfhourly' created
```
Also a more friendly syntax such "every 1m" or "@hourly" can be used to create a time based trigger.
```
$ fission tt create --name minute --function hello --cron "@every 1m"
trigger 'minute' created
```
You can list time based triggers to inspect their associated function and cron specifications:
```
$ fission tt list
NAME CRON FUNCTION_NAME
halfhourly 0 30 * * * hello
minute @every 1m hello
```
### Create a MQ Trigger
For creating a MQ based trigger which will invoke the function when a new message arrives in newfile topic, you can use the syntax below. The response of the function execution will be sent to topic newfileresponse.
```
$ fission mqt create --name hellomsg --function hello --mqtype nats-streaming --topic newfile --resptopic newfileresponse
trigger 'hellomsg' created
```