Nail/fission-src
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fixed typos across fission repo (#1832)

Browse Source 
Co-authored-by: Vishal <vishal-biyani@users.noreply.github.com>
This commit is contained in:
Gaurav Gahlot
2020-10-16 17:25:11 +05:30
committed by GitHub
co-authored by Vishal
parent 6e3c42f238
commit fca0a60e5b
49 changed files with 157 additions and 172 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Documentation/wip/Multi-tenancy.md
+9 -9
View File
@@ -1,7 +1,7 @@
# Multi-tenancy in Fission
Multi-tenancy in Fission allows users to create Fission objects, i.e functions, packages, environments and triggers in different namespaces.
It mandates that a function reference secrets, configmaps and its package (if explicity referenced during function creation/updation) to be present in the same namespace as the function.
It mandates that a function reference secrets, configmaps and its package (if explicitly referenced during function create/update operation) to be present in the same namespace as the function.
This allows user separation and prevents in-advertent access to sensitive data of other users sharing the same cluster.
However, users are allowed and encouraged to share environments to ensure optimal utilization of cluster resources. To achieve this, users can create all the necessary environments in a ns, say ns1 and then go on to create functions in different namespaces and refer to env in ns1.
Users that prefer complete isolation can create their env, functions in the same ns.
@@ -27,20 +27,20 @@ This SA is created in every namespace that a user creates builder environments i
1. Package-getter-binding
Every time a user creates a package explicity in a namespace, this rolebinding is created in package's namespace (which is also function's namespace). This grants package-getter role to fission-fetcher SA present in the referenced environment's namespace.
If the package is a source package, then, fission-builder SA present in the environment namespace is also added to this rolebinding.
Every time a user creates a package explicitly in a namespace, this role binding is created in package's namespace (which is also function's namespace). This grants package-getter role to fission-fetcher SA present in the referenced environment's namespace.
If the package is a source package, then, fission-builder SA present in the environment namespace is also added to this role binding.
Next when the user creates a function in the same namespace, if the function's executor type is newdeploy, then, the fission-fetcher SA present in function namespace is also added to the same rolebinding.
Next when the user creates a function in the same namespace, if the function's executor type is newdeploy, then, the fission-fetcher SA present in function namespace is also added to the same role binding.
Note : For functions that have executor type poolmgr, the env pods are created in the namespace that env object is created. Whereas, for those functions that have executor type New deploy mgr, the function pods are created in the namespace that function object is created in.
This is because, poolmgr allows env sharing and optimal resource utilization. so generic env pools are created in a different namespace and all functions that prefer sharing this env pool can reference these pools.
If users require strict isolation, they can either create functions with new deploy backend, or, create env's in different namespaces and not share them across functions.
If users require strict isolation, they can either create functions with new deploy backend, or, create envs in different namespaces and not share them across functions.
2. Secret-Configmap-getter-binding
Every time a user creates a function in a namespace, Secret-Configmap-getter-binding is created in the same namespace, granting secret-configmap-getter role to fission-fetcher SA present in the referenced environment's namespace in case the executor type is poolmgr.
If the executor type is newdeploymgr, then the same rolebinding is created in the same namespace as the function, granting the same secret-configmap-getter role to fission-fetcher SA present in the function namespace.
If the executor type is newdeploymgr, then the same role binding is created in the same namespace as the function, granting the same secret-configmap-getter role to fission-fetcher SA present in the function namespace.
## Examples
@@ -51,7 +51,7 @@ $ fission env create --name python --image fission/python-env --envns ns1
$ fission function create --name func1 --env python --code hello.py --fns ns2
```
2. create a builder and runtime environment in ns3, a source pkg in ns3 and a function referring to this src pkg also in ns3. (for complete isolation, all objs are in ns3)
2. create a builder and runtime environment in ns3, a source pkg in ns3 and a function referring to this src pkg also in ns3. (for complete isolation, all objects are in ns3)
```bash
$ fission env create --name python-builder-env --builder fission/python-builder --image fission/python-env --ns3
@@ -62,5 +62,5 @@ $ fission fn create --name func3 --fns ns3 --pkg $pkg --entrypoint "user.main"
## Note
1. To maintain backward compatibility, fission objects that are created without the ns flags are created in default namespace. Also, the run time env pods in such a case will continue to live in fission-function ns and builder env pods in fission-builder ns
2. Since all envs in a namespace have the same fission-fetcher SA mounted in them, even though multiple envs are created in a namespace and referenced by functions in different namespaces, the SA will have previleges to view those function's secrets if any.
3. Similarly, if there are multiple functions in different namespaces but all sharing an env in one namespace, the fission-fetcher SA in that namespace will have previleges to see all of their secrets.
2. Since all envs in a namespace have the same fission-fetcher SA mounted in them, even though multiple envs are created in a namespace and referenced by functions in different namespaces, the SA will have privileges to view those function's secrets if any.
3. Similarly, if there are multiple functions in different namespaces but all sharing an env in one namespace, the fission-fetcher SA in that namespace will have privileges to see all of their secrets.
Documentation/wip/ProgrammingModel.md
+1 -1
View File
@@ -20,5 +20,5 @@ documentation on the context object in each environment.
## Kubernetes Watch Event Triggers
Kubernetes watches can be used to trigger functions. These functions
Kubernetes watches can be used to trigger functions. These functions
receive the Kubernetes watch.Event object in JSON-serialized form.
Documentation/wip/annotations_design.md
+2 -2
View File
@@ -48,7 +48,7 @@ So we can write a simple logic - to check if a annotation is applicable for an i
### Implementation 2
- One of side effects of this is that the annotations will still stay on the source CRD object - for example annotation will stay on the httptrigger as well as the ingress object. This can cause problems in certain cases where something like Prometheus uses annotations to scrape objects. So instead we wrap the annotations needed by an object into another annotation name. This also solves problem of having to guess which annotations to apply to which object.
- One of the side effects is that the annotations will still stay on the source CRD object - for example annotation will stay on the httptrigger as well as the ingress object. This can cause problems in certain cases where something like Prometheus uses annotations to scrape objects. So instead we wrap the annotations needed by an object into another annotation name. This also solves problem of having to guess which annotations to apply to which object.
```yaml
apiVersion: fission.io/v1
@@ -88,4 +88,4 @@ HTTPTriggerSpec struct {
## Final thoughts
- The implementation idea 2 & 3 look better than 1. The third option involves HTTPTrigger Spec change.
- For both (2) & (3) - if in future we have to implement annotations for Functions etc. we will have to consider the fact that a function will in turn create 3 objects (Service, Pod & HPA) and annotations for all three would need to be accommodated.
- For both (2) & (3) - if in future we have to implement annotations for Functions etc. we will have to consider the fact that a function will in turn create 3 objects (Service, Pod & HPA) and annotations for all three would need to be accommodated.
Documentation/wip/ci-cd-proposal.md
+3 -3
View File
@@ -57,7 +57,7 @@ If we look at this from CI/CD perspective this process requires:
3. Fission CLI
4. Kubernetes Config so that the apply command can be run
So if we build a container - which has the above requirements met as installed software (Ex. Fission & Kubectl CLI) or available as environment variable (Github pull token or Kubeconfig), the container can be used as part of CI workflow in any tool such as - Jenkins, Argo, Github Actions, Gitlab etc.
So if we build a container - which has the above requirements met as installed software (Ex. Fission & Kubectl CLI) or available as environment variable (Github pull token or Kubeconfig), the container can be used as part of CI workflow in any tool such as - Jenkins, Argo, Github Actions, GitLab etc.
The idea is to build a generic container with Fission CLI, Kubernetes CLI and a way to read Github token and Kubeconfig from env variable/mounted files and being able to run `fission spec apply` command.
@@ -90,7 +90,7 @@ Without changing anything in Fission spec it is possible to change these things
1. Generate and maintain specs for each environment. This is not a best practice as it leads to drift in code and configuration between environment over time.
2. Use placeholder variables (i.e. $DB_CONNECTION_VALUE) and replace them for each environment before deploying. This is better in the sense that you are combining changes specific to each environment with spec code but is still a work around sort of.
For environment specific configurations, it is possible to use some sort of templating or overlay mechanism. One of interesting projects using overlays is [Kustomize](https://github.com/kubernetes-sigs/kustomize). In any case as of today the fission spec command does not have a way to use template or modify values using overlay and it is worth exploring this approach for fission spec.
For environment specific configurations, it is possible to use some sort of templates or overlay mechanism. One of interesting projects using overlays is [Kustomize](https://github.com/kubernetes-sigs/kustomize). In any case as of today the fission spec command does not have a way to use template or modify values using overlay and it is worth exploring this approach for fission spec.
## 3 Promotion from one environment to another
@@ -98,4 +98,4 @@ This necessarily does not fall in the area of Fission per se but it would be fai
## Action Items
As a first step it would be good to build a simple container mentioned in (1) and use it in various tools to understand the value it adds and any unknowns. The next steps would be to build a full end to end pipeline from source to production.
As a first step it would be good to build a simple container mentioned in (1) and use it in various tools to understand the value it adds and any unknowns. The next steps would be to build a full end to end pipeline from source to production.
Documentation/wip/cli-extensibility.md
+1 -1
View File
@@ -125,4 +125,4 @@ Improving this is probably best left to future work.
The more heavyweight solution solves some of these limitations to an extent, but these do not way up to the increased
development and maintainance cost IMO. If needed we could explore this option (or some hybrid option) in the future.
development and maintenance cost IMO. If needed we could explore this option (or some hybrid option) in the future.
Documentation/wip/env-api.md
+2 -2
View File
@@ -46,8 +46,8 @@ directory (in case of deployment packages).
The load request may contain an EntryPoint. If it does, the loader
must interpret this; usually it's the name of a function in a module
or package containing multiple functions. If there is no entrypoint,
the environment must use a default; again, the value of thsi default
or package containing multiple functions. If there is no entry point,
the environment must use a default; again, the value of this default
is environment-specific.
The load request may contain a URL. If it does, requests to that URL
Documentation/wip/environments-v2.md
+5 -5
View File
@@ -52,11 +52,11 @@ container.
### Discussion
Early feedback shows that almost evey user ends up rebuilding images
to add some dependecies. Some sort of automated dependecy resolution
to add some dependencies. Some sort of automated dependency resolution
would be very nice to have and improve the development workflow. In
other words, just attach a package.json(nodejs) or
requirements.txt(python) with a function, and fission will do the
rest. There's also the possiblity of supporting buildpacks (simple
rest. There's also the possibility of supporting buildpacks (simple
zipfiles), a la AWS Lambda.
Though we can support compiled languages by doing the compilation
@@ -82,7 +82,7 @@ use environments, I've already packaged up my function, here it is".
Roughly in order of priority:
0. Retain the simplicity of the simple use cases. First user
experience shoud remain trivial -- write a function, map a URL,
experience should remain trivial -- write a function, map a URL,
done.
1. Support compiled languages. Support error reporting on function
@@ -91,7 +91,7 @@ Roughly in order of priority:
2. Support functions as a collection of files rather than just one
file.
3. Support automated environment-specific dependecy resolution.
3. Support automated environment-specific dependency resolution.
(#3 may end up having the same solution as #1. You could think of
gathering deps as a "compilation" of package.json,
@@ -297,7 +297,7 @@ affinity rule, but k8s services only support IP based affinity.]
#### Poolmgr
Poolmgr remains relatively unchanged. Instead of contructing URLs for
Poolmgr remains relatively unchanged. Instead of constructing URLs for
function metadata, it uses the deployment package URL in the function
object.
Documentation/wip/execution-backends.md
+2 -2
View File
@@ -9,7 +9,7 @@ A new layer - executor now sits between the router and actual backends are respo
# Backend
A backend is responsible for execution of a function - which can involve provisioning appropriate objects in Kubernetes. So with the new design Pool manager becomes one of backends. As of this writing there are two backends which are described as:
A backend is responsible for execution of a function - which can involve provisioning appropriate objects in Kubernetes. So with the new design Pool manager becomes one of the backends. As of this writing there are two backends which are described as:
### Pool Manager Backend
@@ -21,7 +21,7 @@ New Deploy backend create a Kubernetes deployment, a Kubernetes Service for a gi
### Execution Strategy
While this is still a WIP, parameters that affect execution behaviour of function are based on `InvokeStrategy`. A invoke strategy defines the `strategyType` and actual strategy parameters encapsulated in the strategy object.
While this is still a WIP, parameters that affect execution behavior of function are based on `InvokeStrategy`. A invoke strategy defines the `strategyType` and actual strategy parameters encapsulated in the strategy object.
```
InvokeStrategy struct {
Documentation/wip/java-env-design.md
+3 -3
View File
@@ -8,7 +8,7 @@ This document documents the design and thoughts that lead to design of Java envi
- Kotlin (Server side with Spring)
- In Java there are a few prominent frameworks which have a ecosystem of their own (See list below). How these framework fit in the environment will be detailed later, but it is important to understand their place in ecosystem and design for it.
- A large percentage of entertprise developers use [Spring framework](https://spring.io/) as has been shown by multiple surveys
- A large percentage of enterprise developers use [Spring framework](https://spring.io/) as has been shown by multiple surveys
- Reactive has taken up recently with data intensive operations [Reactive extensions for JVM](https://github.com/ReactiveX/RxJava)
- [Spark](http://sparkjava.com/) is a micro web framework
@@ -31,7 +31,7 @@ Now - the T & R could be different things and we discuss some options below:
From the early implementation in the branch mentioned above, the environment extracts the body and send it as a JSON string. The JSON then can be transformed into the appropriate object by the function.
This works well, but has one major limitation: the function does not get access to other things like headers etc. The same thing applies to the reponse: function can send the body but looses control over status code etc.
This works well, but has one major limitation: the function does not get access to other things like headers etc. The same thing applies to the response: function can send the body but looses control over status code etc.
### HttpServletRequest and HttpServletResponse
@@ -64,4 +64,4 @@ The Spring cloud function project also discusses the issue of not having access
JVM environment design is based on Spring boot and Spring MVC frameworks. The details can be found in branch, but here are some key points:
- All classes in the function and dependent classes are loaded into JVM. Which means the user should supply the uber/fat jar for execution.
- The entrypoint class is specified by the user as ```entrypoint``` flag on the class. The method is by convention (```apply``` as per the Function interface contract)
- The entrypoint class is specified by the user as ```entrypoint``` flag on the class. The method is by convention (```apply``` as per the Function interface contract)
Documentation/wip/testing-proposal.md
+9 -9
View File
@@ -9,7 +9,7 @@ Most of current integration tests are CLI driven. Fission CLI is used to test ex
This section only explains the problems/best practices without going into tooling and language used for implementation.
### Separating the test & data
Seperating the tests from test data has two aspects - one is separation of concerns and second is scaling the tests without touching the test logic. The test data is a simple data structure which holds all information and test can take data and execute the logic.
Separating the tests from test data has two aspects - one is separation of concerns and second is scaling the tests without touching the test logic. The test data is a simple data structure which holds all information and test can take data and execute the logic.
As an example today we test "Hello world" for nodejs environment with a simple hello.js like this:
@@ -36,7 +36,7 @@ And feed it with a dictionary which has all possible combination of tests:
node: {"fission/node-env", "test/hello.js", "/hellonode"},
python: {"fission/python-env", "test/hello.py", "/hellopy"},
golang: {"fission/go-env", "test/hello.go", "/hellogo"},
binray: {"fission/binary-env", "test/hello.sh", "/hellobinary"},
binary: {"fission/binary-env", "test/hello.sh", "/hellobinary"},
}
```
@@ -63,7 +63,7 @@ The ability to have clean and separate before and after blocks, apart from separ
- Running a suite of tests for same setup (See tagging for suite of tests)
### Tagging tests & running a selection
Over a period of time as tests grow, there will be unit, smkoe, integration, performance, soak tests and so on. Ability to run a perticular test suite only or a combination of them makes it easy to run for specific purpose.
Over a period of time as tests grow, there will be unit, smoke, integration, performance, soak tests and so on. Ability to run a particular test suite only or a combination of them makes it easy to run for specific purpose.
### Measuring test times
[Good to have, not a must] Measuring time for tests and reporting somewhere helps over time to monitor trends. Although this job is better done by performance/benchmark tests so it is not a strict requirement
@@ -74,7 +74,7 @@ It would be good to have cleaner/relevant logging as part of build & test. For e
It would be good to be able to run tests in parallel.
## Evalutaing the tools/alternatives
## Evaluating the tools/alternatives
### BATS
Bash Automated Testing System is like a enhanced version of bash with support for @test tags and before and after steps & ability to skip tests etc. While it enhances the bash to certain extent, the overall improvement is only marginal.
@@ -111,7 +111,7 @@ https://golang.org/pkg/os/exec
### Using CLI package
Currently we build a CLI and then execute the tests. The tests basically call one of functions from CLI package. If we decide to use a go lang based framework, then we can import the CLI package and then call those functions by providing them context. This is as good as calling the Fission from CLI, with added benefit of programmibility of Go langugage.
Currently we build a CLI and then execute the tests. The tests basically call one of functions from CLI package. If we decide to use a go lang based framework, then we can import the CLI package and then call those functions by providing them context. This is as good as calling the Fission from CLI, with added benefit of programmability of Go language.
```
func TestSomething(t *testing.T) {
@@ -135,7 +135,7 @@ Ginkgo is a BDD framework which works with Gomega matcher library. I will state
From Ginkgo:
- Global `BeforeSuite` and after `AfterSuite` can be used to have global setup and tear down phases
- For tests `BeforeEach` and `AfterEach` and more such varients to do before and after test tasks.
- For tests `BeforeEach` and `AfterEach` and more such variants to do before and after test tasks.
From Gomega:
@@ -180,7 +180,7 @@ Eventually(session.Out).Should(gbytes.Say("hello [A-Za-z], world"))
```
There are many more matchers which cane be found here: http://onsi.github.io/gomega/#provided-matchers
- We can build custom mathers in Go language for reusable logic.
- We can build custom matchers in Go language for reusable logic.
#### Links
Ginkgo: http://onsi.github.io/ginkgo/
@@ -199,11 +199,11 @@ Based on the discussion with team, here are current thoughts and next action ite
- How will migration of tests happen over time:
- Aim is to keep existing tests around so that enough validation is in place
- May be migrate one test at a time
- How much of current setup etc. will move into framework? For example it is clear that helm commands should be part of test framework as part of setup/teardown. But other sections may or may not be. A RCA needs to be done to analyze and come up with clear demarkation.
- How much of current setup etc. will move into framework? For example it is clear that helm commands should be part of test framework as part of setup/teardown. But other sections may or may not be. A RCA needs to be done to analyze and come up with clear demarcation.
## References
- EngineYard uses BATS: https://www.engineyard.com/blog/bats-test-command-line-tools
- AWS CLI Tests, written in Python (CLI itself is also in Python): https://github.com/aws/aws-cli/tree/develop/tests
- Kubernetes uses Ginkgo and Gomega extensively: https://github.com/kubernetes/kubernetes/search?l=Go&q=onsi&type=
- Hashicorp's Mitchell's talk on Advanced testing with go talks about some good patterns to use: https://www.youtube.com/watch?v=8hQG7QlcLBk
- Hashicorp's Mitchell's talk on Advanced testing with go talks about some good patterns to use: https://www.youtube.com/watch?v=8hQG7QlcLBk