Fixed typos across fission repo (#1832)
Co-authored-by: Vishal <vishal-biyani@users.noreply.github.com>
This commit is contained in:
Gaurav Gahlot
+7
-7
@@ -11,7 +11,7 @@ Table of Contents
|
||||
* [Choose something to work on](#choose-something-to-work-on)
|
||||
* [Get Help.](#get-help)
|
||||
* [Contributing - building & deploying](#contributing---building--deploying)
|
||||
* [Prequisite](#prequisite)
|
||||
* [Prerequisite](#prerequisite)
|
||||
* [Getting Started](#getting-started)
|
||||
* [Use Skaffold with Kind/K8S Cluster to build and deploy](#use-skaffold-with-kindk8s-cluster-to-build-and-deploy)
|
||||
* [Validating Installation](#validating-installation)
|
||||
@@ -42,7 +42,7 @@ Do reach out on Slack or Twitter and we are happy to help.
|
||||
|
||||
# Contributing - building & deploying
|
||||
|
||||
## Pre-requisite
|
||||
## Prerequisite
|
||||
|
||||
- You'll need the `go` compiler and tools installed. Currently version 1.12.x of Go is needed.
|
||||
|
||||
@@ -82,7 +82,7 @@ You should bring up Kind/Minikube cluster or if using a cloud provider cluster t
|
||||
* For building & deploying to Cloud Provider K8S cluster such as GKE/EKS/AKS:
|
||||
|
||||
```
|
||||
$ skaffold config set default-repo vishalbiyani // (vishalbiyani - should be your registry/Dockerhub handle)
|
||||
$ skaffold config set default-repo vishalbiyani // (vishalbiyani - should be your registry/Docker Hub handle)
|
||||
$ skaffold run
|
||||
```
|
||||
|
||||
@@ -106,7 +106,7 @@ fission fission 1 2020-05-19 16:31:46.947562 +0530 IST success fission-
|
||||
Also you should see the Fission services deployed and running:
|
||||
|
||||
```
|
||||
$ kubectl get pods -nfission
|
||||
$ kubectl get pods -n fission
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
buildermgr-6f778d4ff9-dqnq5 1/1 Running 0 6h9m
|
||||
controller-d44bd4f4d-5q4z5 1/1 Running 0 6h9m
|
||||
@@ -127,7 +127,7 @@ timer-7d85d9c9fb-knctw 1/1 Running
|
||||
|
||||
### cmd
|
||||
|
||||
Cmd package is entrypoint for all runtime components and also has Dockerfile for each component. The actual logic here will be pretty light and most of logic of each component is in `pkg` (Discussed later)
|
||||
`cmd` package is entry point for all runtime components and also has Dockerfile for each component. The actual logic here will be pretty light and most of logic of each component is in `pkg` (Discussed later)
|
||||
|
||||
| Component | Runtime Component |Used in|
|
||||
| :------------- |:------------- |:-|
|
||||
@@ -167,7 +167,7 @@ cmd
|
||||
/fission-bundle --kubewatcher --routerUrl http://router.fission # Runs Kubewatcher
|
||||
```
|
||||
|
||||
So most serverside components running on server side are fission-bundle binary wrapped in container and used with different arguments. Various arguments and environment variables are passed from manifests/helm chart
|
||||
So most server side components running on server side are fission-bundle binary wrapped in container and used with different arguments. Various arguments and environment variables are passed from manifests/helm chart
|
||||
|
||||
**fission-cli** : is the cli used by end user to interact Fission
|
||||
|
||||
@@ -208,7 +208,7 @@ Pkg is where most of core components and logic reside. The structure is fairly s
|
||||
|
||||
### Charts
|
||||
|
||||
Fission currently has two charts - and we reccommend using fission-all for development.
|
||||
Fission currently has two charts - and we recommend using fission-all for development.
|
||||
|
||||
```
|
||||
.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Multi-tenancy in Fission
|
||||
|
||||
Multi-tenancy in Fission allows users to create Fission objects, i.e functions, packages, environments and triggers in different namespaces.
|
||||
It mandates that a function reference secrets, configmaps and its package (if explicity referenced during function creation/updation) to be present in the same namespace as the function.
|
||||
It mandates that a function reference secrets, configmaps and its package (if explicitly referenced during function create/update operation) to be present in the same namespace as the function.
|
||||
This allows user separation and prevents in-advertent access to sensitive data of other users sharing the same cluster.
|
||||
However, users are allowed and encouraged to share environments to ensure optimal utilization of cluster resources. To achieve this, users can create all the necessary environments in a ns, say ns1 and then go on to create functions in different namespaces and refer to env in ns1.
|
||||
Users that prefer complete isolation can create their env, functions in the same ns.
|
||||
@@ -27,20 +27,20 @@ This SA is created in every namespace that a user creates builder environments i
|
||||
|
||||
1. Package-getter-binding
|
||||
|
||||
Every time a user creates a package explicity in a namespace, this rolebinding is created in package's namespace (which is also function's namespace). This grants package-getter role to fission-fetcher SA present in the referenced environment's namespace.
|
||||
If the package is a source package, then, fission-builder SA present in the environment namespace is also added to this rolebinding.
|
||||
Every time a user creates a package explicitly in a namespace, this role binding is created in package's namespace (which is also function's namespace). This grants package-getter role to fission-fetcher SA present in the referenced environment's namespace.
|
||||
If the package is a source package, then, fission-builder SA present in the environment namespace is also added to this role binding.
|
||||
|
||||
Next when the user creates a function in the same namespace, if the function's executor type is newdeploy, then, the fission-fetcher SA present in function namespace is also added to the same rolebinding.
|
||||
Next when the user creates a function in the same namespace, if the function's executor type is newdeploy, then, the fission-fetcher SA present in function namespace is also added to the same role binding.
|
||||
|
||||
Note : For functions that have executor type poolmgr, the env pods are created in the namespace that env object is created. Whereas, for those functions that have executor type New deploy mgr, the function pods are created in the namespace that function object is created in.
|
||||
|
||||
This is because, poolmgr allows env sharing and optimal resource utilization. so generic env pools are created in a different namespace and all functions that prefer sharing this env pool can reference these pools.
|
||||
If users require strict isolation, they can either create functions with new deploy backend, or, create env's in different namespaces and not share them across functions.
|
||||
If users require strict isolation, they can either create functions with new deploy backend, or, create envs in different namespaces and not share them across functions.
|
||||
|
||||
2. Secret-Configmap-getter-binding
|
||||
|
||||
Every time a user creates a function in a namespace, Secret-Configmap-getter-binding is created in the same namespace, granting secret-configmap-getter role to fission-fetcher SA present in the referenced environment's namespace in case the executor type is poolmgr.
|
||||
If the executor type is newdeploymgr, then the same rolebinding is created in the same namespace as the function, granting the same secret-configmap-getter role to fission-fetcher SA present in the function namespace.
|
||||
If the executor type is newdeploymgr, then the same role binding is created in the same namespace as the function, granting the same secret-configmap-getter role to fission-fetcher SA present in the function namespace.
|
||||
|
||||
## Examples
|
||||
|
||||
@@ -51,7 +51,7 @@ $ fission env create --name python --image fission/python-env --envns ns1
|
||||
$ fission function create --name func1 --env python --code hello.py --fns ns2
|
||||
```
|
||||
|
||||
2. create a builder and runtime environment in ns3, a source pkg in ns3 and a function referring to this src pkg also in ns3. (for complete isolation, all objs are in ns3)
|
||||
2. create a builder and runtime environment in ns3, a source pkg in ns3 and a function referring to this src pkg also in ns3. (for complete isolation, all objects are in ns3)
|
||||
|
||||
```bash
|
||||
$ fission env create --name python-builder-env --builder fission/python-builder --image fission/python-env --ns3
|
||||
@@ -62,5 +62,5 @@ $ fission fn create --name func3 --fns ns3 --pkg $pkg --entrypoint "user.main"
|
||||
## Note
|
||||
|
||||
1. To maintain backward compatibility, fission objects that are created without the ns flags are created in default namespace. Also, the run time env pods in such a case will continue to live in fission-function ns and builder env pods in fission-builder ns
|
||||
2. Since all envs in a namespace have the same fission-fetcher SA mounted in them, even though multiple envs are created in a namespace and referenced by functions in different namespaces, the SA will have previleges to view those function's secrets if any.
|
||||
3. Similarly, if there are multiple functions in different namespaces but all sharing an env in one namespace, the fission-fetcher SA in that namespace will have previleges to see all of their secrets.
|
||||
2. Since all envs in a namespace have the same fission-fetcher SA mounted in them, even though multiple envs are created in a namespace and referenced by functions in different namespaces, the SA will have privileges to view those function's secrets if any.
|
||||
3. Similarly, if there are multiple functions in different namespaces but all sharing an env in one namespace, the fission-fetcher SA in that namespace will have privileges to see all of their secrets.
|
||||
|
||||
@@ -20,5 +20,5 @@ documentation on the context object in each environment.
|
||||
|
||||
## Kubernetes Watch Event Triggers
|
||||
|
||||
Kubernetes watches can be used to trigger functions. These functions
|
||||
Kubernetes watches can be used to trigger functions. These functions
|
||||
receive the Kubernetes watch.Event object in JSON-serialized form.
|
||||
|
||||
@@ -48,7 +48,7 @@ So we can write a simple logic - to check if a annotation is applicable for an i
|
||||
|
||||
### Implementation 2
|
||||
|
||||
- One of side effects of this is that the annotations will still stay on the source CRD object - for example annotation will stay on the httptrigger as well as the ingress object. This can cause problems in certain cases where something like Prometheus uses annotations to scrape objects. So instead we wrap the annotations needed by an object into another annotation name. This also solves problem of having to guess which annotations to apply to which object.
|
||||
- One of the side effects is that the annotations will still stay on the source CRD object - for example annotation will stay on the httptrigger as well as the ingress object. This can cause problems in certain cases where something like Prometheus uses annotations to scrape objects. So instead we wrap the annotations needed by an object into another annotation name. This also solves problem of having to guess which annotations to apply to which object.
|
||||
|
||||
```yaml
|
||||
apiVersion: fission.io/v1
|
||||
@@ -88,4 +88,4 @@ HTTPTriggerSpec struct {
|
||||
## Final thoughts
|
||||
|
||||
- The implementation idea 2 & 3 look better than 1. The third option involves HTTPTrigger Spec change.
|
||||
- For both (2) & (3) - if in future we have to implement annotations for Functions etc. we will have to consider the fact that a function will in turn create 3 objects (Service, Pod & HPA) and annotations for all three would need to be accommodated.
|
||||
- For both (2) & (3) - if in future we have to implement annotations for Functions etc. we will have to consider the fact that a function will in turn create 3 objects (Service, Pod & HPA) and annotations for all three would need to be accommodated.
|
||||
|
||||
@@ -57,7 +57,7 @@ If we look at this from CI/CD perspective this process requires:
|
||||
3. Fission CLI
|
||||
4. Kubernetes Config so that the apply command can be run
|
||||
|
||||
So if we build a container - which has the above requirements met as installed software (Ex. Fission & Kubectl CLI) or available as environment variable (Github pull token or Kubeconfig), the container can be used as part of CI workflow in any tool such as - Jenkins, Argo, Github Actions, Gitlab etc.
|
||||
So if we build a container - which has the above requirements met as installed software (Ex. Fission & Kubectl CLI) or available as environment variable (Github pull token or Kubeconfig), the container can be used as part of CI workflow in any tool such as - Jenkins, Argo, Github Actions, GitLab etc.
|
||||
|
||||
The idea is to build a generic container with Fission CLI, Kubernetes CLI and a way to read Github token and Kubeconfig from env variable/mounted files and being able to run `fission spec apply` command.
|
||||
|
||||
@@ -90,7 +90,7 @@ Without changing anything in Fission spec it is possible to change these things
|
||||
1. Generate and maintain specs for each environment. This is not a best practice as it leads to drift in code and configuration between environment over time.
|
||||
2. Use placeholder variables (i.e. $DB_CONNECTION_VALUE) and replace them for each environment before deploying. This is better in the sense that you are combining changes specific to each environment with spec code but is still a work around sort of.
|
||||
|
||||
For environment specific configurations, it is possible to use some sort of templating or overlay mechanism. One of interesting projects using overlays is [Kustomize](https://github.com/kubernetes-sigs/kustomize). In any case as of today the fission spec command does not have a way to use template or modify values using overlay and it is worth exploring this approach for fission spec.
|
||||
For environment specific configurations, it is possible to use some sort of templates or overlay mechanism. One of interesting projects using overlays is [Kustomize](https://github.com/kubernetes-sigs/kustomize). In any case as of today the fission spec command does not have a way to use template or modify values using overlay and it is worth exploring this approach for fission spec.
|
||||
|
||||
## 3 Promotion from one environment to another
|
||||
|
||||
@@ -98,4 +98,4 @@ This necessarily does not fall in the area of Fission per se but it would be fai
|
||||
|
||||
## Action Items
|
||||
|
||||
As a first step it would be good to build a simple container mentioned in (1) and use it in various tools to understand the value it adds and any unknowns. The next steps would be to build a full end to end pipeline from source to production.
|
||||
As a first step it would be good to build a simple container mentioned in (1) and use it in various tools to understand the value it adds and any unknowns. The next steps would be to build a full end to end pipeline from source to production.
|
||||
|
||||
@@ -125,4 +125,4 @@ Improving this is probably best left to future work.
|
||||
|
||||
|
||||
The more heavyweight solution solves some of these limitations to an extent, but these do not way up to the increased
|
||||
development and maintainance cost IMO. If needed we could explore this option (or some hybrid option) in the future.
|
||||
development and maintenance cost IMO. If needed we could explore this option (or some hybrid option) in the future.
|
||||
|
||||
@@ -46,8 +46,8 @@ directory (in case of deployment packages).
|
||||
|
||||
The load request may contain an EntryPoint. If it does, the loader
|
||||
must interpret this; usually it's the name of a function in a module
|
||||
or package containing multiple functions. If there is no entrypoint,
|
||||
the environment must use a default; again, the value of thsi default
|
||||
or package containing multiple functions. If there is no entry point,
|
||||
the environment must use a default; again, the value of this default
|
||||
is environment-specific.
|
||||
|
||||
The load request may contain a URL. If it does, requests to that URL
|
||||
|
||||
@@ -52,11 +52,11 @@ container.
|
||||
### Discussion
|
||||
|
||||
Early feedback shows that almost evey user ends up rebuilding images
|
||||
to add some dependecies. Some sort of automated dependecy resolution
|
||||
to add some dependencies. Some sort of automated dependency resolution
|
||||
would be very nice to have and improve the development workflow. In
|
||||
other words, just attach a package.json(nodejs) or
|
||||
requirements.txt(python) with a function, and fission will do the
|
||||
rest. There's also the possiblity of supporting buildpacks (simple
|
||||
rest. There's also the possibility of supporting buildpacks (simple
|
||||
zipfiles), a la AWS Lambda.
|
||||
|
||||
Though we can support compiled languages by doing the compilation
|
||||
@@ -82,7 +82,7 @@ use environments, I've already packaged up my function, here it is".
|
||||
Roughly in order of priority:
|
||||
|
||||
0. Retain the simplicity of the simple use cases. First user
|
||||
experience shoud remain trivial -- write a function, map a URL,
|
||||
experience should remain trivial -- write a function, map a URL,
|
||||
done.
|
||||
|
||||
1. Support compiled languages. Support error reporting on function
|
||||
@@ -91,7 +91,7 @@ Roughly in order of priority:
|
||||
2. Support functions as a collection of files rather than just one
|
||||
file.
|
||||
|
||||
3. Support automated environment-specific dependecy resolution.
|
||||
3. Support automated environment-specific dependency resolution.
|
||||
|
||||
(#3 may end up having the same solution as #1. You could think of
|
||||
gathering deps as a "compilation" of package.json,
|
||||
@@ -297,7 +297,7 @@ affinity rule, but k8s services only support IP based affinity.]
|
||||
|
||||
#### Poolmgr
|
||||
|
||||
Poolmgr remains relatively unchanged. Instead of contructing URLs for
|
||||
Poolmgr remains relatively unchanged. Instead of constructing URLs for
|
||||
function metadata, it uses the deployment package URL in the function
|
||||
object.
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ A new layer - executor now sits between the router and actual backends are respo
|
||||
|
||||
# Backend
|
||||
|
||||
A backend is responsible for execution of a function - which can involve provisioning appropriate objects in Kubernetes. So with the new design Pool manager becomes one of backends. As of this writing there are two backends which are described as:
|
||||
A backend is responsible for execution of a function - which can involve provisioning appropriate objects in Kubernetes. So with the new design Pool manager becomes one of the backends. As of this writing there are two backends which are described as:
|
||||
|
||||
### Pool Manager Backend
|
||||
|
||||
@@ -21,7 +21,7 @@ New Deploy backend create a Kubernetes deployment, a Kubernetes Service for a gi
|
||||
|
||||
### Execution Strategy
|
||||
|
||||
While this is still a WIP, parameters that affect execution behaviour of function are based on `InvokeStrategy`. A invoke strategy defines the `strategyType` and actual strategy parameters encapsulated in the strategy object.
|
||||
While this is still a WIP, parameters that affect execution behavior of function are based on `InvokeStrategy`. A invoke strategy defines the `strategyType` and actual strategy parameters encapsulated in the strategy object.
|
||||
|
||||
```
|
||||
InvokeStrategy struct {
|
||||
|
||||
@@ -8,7 +8,7 @@ This document documents the design and thoughts that lead to design of Java envi
|
||||
- Kotlin (Server side with Spring)
|
||||
|
||||
- In Java there are a few prominent frameworks which have a ecosystem of their own (See list below). How these framework fit in the environment will be detailed later, but it is important to understand their place in ecosystem and design for it.
|
||||
- A large percentage of entertprise developers use [Spring framework](https://spring.io/) as has been shown by multiple surveys
|
||||
- A large percentage of enterprise developers use [Spring framework](https://spring.io/) as has been shown by multiple surveys
|
||||
- Reactive has taken up recently with data intensive operations [Reactive extensions for JVM](https://github.com/ReactiveX/RxJava)
|
||||
- [Spark](http://sparkjava.com/) is a micro web framework
|
||||
|
||||
@@ -31,7 +31,7 @@ Now - the T & R could be different things and we discuss some options below:
|
||||
|
||||
From the early implementation in the branch mentioned above, the environment extracts the body and send it as a JSON string. The JSON then can be transformed into the appropriate object by the function.
|
||||
|
||||
This works well, but has one major limitation: the function does not get access to other things like headers etc. The same thing applies to the reponse: function can send the body but looses control over status code etc.
|
||||
This works well, but has one major limitation: the function does not get access to other things like headers etc. The same thing applies to the response: function can send the body but looses control over status code etc.
|
||||
|
||||
### HttpServletRequest and HttpServletResponse
|
||||
|
||||
@@ -64,4 +64,4 @@ The Spring cloud function project also discusses the issue of not having access
|
||||
JVM environment design is based on Spring boot and Spring MVC frameworks. The details can be found in branch, but here are some key points:
|
||||
|
||||
- All classes in the function and dependent classes are loaded into JVM. Which means the user should supply the uber/fat jar for execution.
|
||||
- The entrypoint class is specified by the user as ```entrypoint``` flag on the class. The method is by convention (```apply``` as per the Function interface contract)
|
||||
- The entrypoint class is specified by the user as ```entrypoint``` flag on the class. The method is by convention (```apply``` as per the Function interface contract)
|
||||
|
||||
@@ -9,7 +9,7 @@ Most of current integration tests are CLI driven. Fission CLI is used to test ex
|
||||
This section only explains the problems/best practices without going into tooling and language used for implementation.
|
||||
### Separating the test & data
|
||||
|
||||
Seperating the tests from test data has two aspects - one is separation of concerns and second is scaling the tests without touching the test logic. The test data is a simple data structure which holds all information and test can take data and execute the logic.
|
||||
Separating the tests from test data has two aspects - one is separation of concerns and second is scaling the tests without touching the test logic. The test data is a simple data structure which holds all information and test can take data and execute the logic.
|
||||
|
||||
As an example today we test "Hello world" for nodejs environment with a simple hello.js like this:
|
||||
|
||||
@@ -36,7 +36,7 @@ And feed it with a dictionary which has all possible combination of tests:
|
||||
node: {"fission/node-env", "test/hello.js", "/hellonode"},
|
||||
python: {"fission/python-env", "test/hello.py", "/hellopy"},
|
||||
golang: {"fission/go-env", "test/hello.go", "/hellogo"},
|
||||
binray: {"fission/binary-env", "test/hello.sh", "/hellobinary"},
|
||||
binary: {"fission/binary-env", "test/hello.sh", "/hellobinary"},
|
||||
}
|
||||
|
||||
```
|
||||
@@ -63,7 +63,7 @@ The ability to have clean and separate before and after blocks, apart from separ
|
||||
- Running a suite of tests for same setup (See tagging for suite of tests)
|
||||
### Tagging tests & running a selection
|
||||
|
||||
Over a period of time as tests grow, there will be unit, smkoe, integration, performance, soak tests and so on. Ability to run a perticular test suite only or a combination of them makes it easy to run for specific purpose.
|
||||
Over a period of time as tests grow, there will be unit, smoke, integration, performance, soak tests and so on. Ability to run a particular test suite only or a combination of them makes it easy to run for specific purpose.
|
||||
### Measuring test times
|
||||
|
||||
[Good to have, not a must] Measuring time for tests and reporting somewhere helps over time to monitor trends. Although this job is better done by performance/benchmark tests so it is not a strict requirement
|
||||
@@ -74,7 +74,7 @@ It would be good to have cleaner/relevant logging as part of build & test. For e
|
||||
|
||||
It would be good to be able to run tests in parallel.
|
||||
|
||||
## Evalutaing the tools/alternatives
|
||||
## Evaluating the tools/alternatives
|
||||
### BATS
|
||||
|
||||
Bash Automated Testing System is like a enhanced version of bash with support for @test tags and before and after steps & ability to skip tests etc. While it enhances the bash to certain extent, the overall improvement is only marginal.
|
||||
@@ -111,7 +111,7 @@ https://golang.org/pkg/os/exec
|
||||
|
||||
### Using CLI package
|
||||
|
||||
Currently we build a CLI and then execute the tests. The tests basically call one of functions from CLI package. If we decide to use a go lang based framework, then we can import the CLI package and then call those functions by providing them context. This is as good as calling the Fission from CLI, with added benefit of programmibility of Go langugage.
|
||||
Currently we build a CLI and then execute the tests. The tests basically call one of functions from CLI package. If we decide to use a go lang based framework, then we can import the CLI package and then call those functions by providing them context. This is as good as calling the Fission from CLI, with added benefit of programmability of Go language.
|
||||
|
||||
```
|
||||
func TestSomething(t *testing.T) {
|
||||
@@ -135,7 +135,7 @@ Ginkgo is a BDD framework which works with Gomega matcher library. I will state
|
||||
From Ginkgo:
|
||||
|
||||
- Global `BeforeSuite` and after `AfterSuite` can be used to have global setup and tear down phases
|
||||
- For tests `BeforeEach` and `AfterEach` and more such varients to do before and after test tasks.
|
||||
- For tests `BeforeEach` and `AfterEach` and more such variants to do before and after test tasks.
|
||||
|
||||
From Gomega:
|
||||
|
||||
@@ -180,7 +180,7 @@ Eventually(session.Out).Should(gbytes.Say("hello [A-Za-z], world"))
|
||||
```
|
||||
There are many more matchers which cane be found here: http://onsi.github.io/gomega/#provided-matchers
|
||||
|
||||
- We can build custom mathers in Go language for reusable logic.
|
||||
- We can build custom matchers in Go language for reusable logic.
|
||||
|
||||
#### Links
|
||||
Ginkgo: http://onsi.github.io/ginkgo/
|
||||
@@ -199,11 +199,11 @@ Based on the discussion with team, here are current thoughts and next action ite
|
||||
- How will migration of tests happen over time:
|
||||
- Aim is to keep existing tests around so that enough validation is in place
|
||||
- May be migrate one test at a time
|
||||
- How much of current setup etc. will move into framework? For example it is clear that helm commands should be part of test framework as part of setup/teardown. But other sections may or may not be. A RCA needs to be done to analyze and come up with clear demarkation.
|
||||
- How much of current setup etc. will move into framework? For example it is clear that helm commands should be part of test framework as part of setup/teardown. But other sections may or may not be. A RCA needs to be done to analyze and come up with clear demarcation.
|
||||
|
||||
## References
|
||||
|
||||
- EngineYard uses BATS: https://www.engineyard.com/blog/bats-test-command-line-tools
|
||||
- AWS CLI Tests, written in Python (CLI itself is also in Python): https://github.com/aws/aws-cli/tree/develop/tests
|
||||
- Kubernetes uses Ginkgo and Gomega extensively: https://github.com/kubernetes/kubernetes/search?l=Go&q=onsi&type=
|
||||
- Hashicorp's Mitchell's talk on Advanced testing with go talks about some good patterns to use: https://www.youtube.com/watch?v=8hQG7QlcLBk
|
||||
- Hashicorp's Mitchell's talk on Advanced testing with go talks about some good patterns to use: https://www.youtube.com/watch?v=8hQG7QlcLBk
|
||||
|
||||
@@ -12,7 +12,7 @@ COPY go.* ./
|
||||
|
||||
RUN go mod download
|
||||
|
||||
From godep as builder
|
||||
FROM godep as builder
|
||||
|
||||
ARG GOPKG
|
||||
WORKDIR /go/src/${GOPKG}
|
||||
|
||||
@@ -12,7 +12,7 @@ COPY go.* ./
|
||||
|
||||
RUN go mod download
|
||||
|
||||
From godep as builder
|
||||
FROM godep as builder
|
||||
|
||||
ARG GOPKG
|
||||
WORKDIR /go/src/${GOPKG}
|
||||
|
||||
@@ -12,7 +12,7 @@ COPY go.* ./
|
||||
|
||||
RUN go mod download
|
||||
|
||||
From godep as builder
|
||||
FROM godep as builder
|
||||
|
||||
ARG GOPKG
|
||||
WORKDIR /go/src/${GOPKG}
|
||||
|
||||
@@ -12,7 +12,7 @@ COPY go.* ./
|
||||
|
||||
RUN go mod download
|
||||
|
||||
From godep as builder
|
||||
FROM godep as builder
|
||||
|
||||
ARG GOPKG
|
||||
WORKDIR /go/src/${GOPKG}
|
||||
|
||||
@@ -64,7 +64,7 @@ func makePreUpgradeTaskClient(logger *zap.Logger, fnPodNs, envBuilderNs string)
|
||||
}, nil
|
||||
}
|
||||
|
||||
// IsFissionReInstall checks if there is atleast one fission CRD, i.e. function in this case, on this cluster.
|
||||
// IsFissionReInstall checks if there is at least one fission CRD, i.e. function in this case, on this cluster.
|
||||
// We need this to find out if fission had been previously installed on this cluster
|
||||
func (client *PreUpgradeTaskClient) IsFissionReInstall() bool {
|
||||
for i := 0; i < maxRetries; i++ {
|
||||
@@ -158,10 +158,10 @@ func (client *PreUpgradeTaskClient) RemoveClusterAdminRolesForFissionSAs() {
|
||||
}
|
||||
}
|
||||
|
||||
client.logger.Info("femoved cluster admin privileges for fission-builder and fission-fetcher service accounts")
|
||||
client.logger.Info("removed cluster admin privileges for fission-builder and fission-fetcher service accounts")
|
||||
}
|
||||
|
||||
// NeedRoleBindings checks if there is atleast one package or function in default namespace.
|
||||
// NeedRoleBindings checks if there is at least one package or function in default namespace.
|
||||
// It is needed to find out if package-getter-rb and secret-configmap-getter-rb needs to be created for fission-fetcher
|
||||
// and fission-builder service accounts.
|
||||
// This is because, we just deleted the ClusterRoleBindings for these service accounts in the previous function and
|
||||
@@ -180,7 +180,7 @@ func (client *PreUpgradeTaskClient) NeedRoleBindings() bool {
|
||||
return false
|
||||
}
|
||||
|
||||
// Setup appropriate role bindings for fission-fetcher and fission-builder SAs
|
||||
// SetupRoleBindings sets appropriate role bindings for fission-fetcher and fission-builder SAs
|
||||
func (client *PreUpgradeTaskClient) SetupRoleBindings() {
|
||||
if !client.NeedRoleBindings() {
|
||||
client.logger.Info("no fission objects found, so no role-bindings to create")
|
||||
|
||||
@@ -10,8 +10,8 @@ Use Cases
|
||||
|
||||
⚠️ **Words of Caution** ⚠️
|
||||
|
||||
The environment runs on an alpine image with some additional utility commandline tools installed, such as 'grep'.
|
||||
However, in case you want to make use of more esoteric commandline tools, you should add the relevant apk to the
|
||||
The environment runs on an alpine image with some additional utility command line tools installed, such as 'grep'.
|
||||
However, in case you want to make use of more esoteric command line tools, you should add the relevant apk to the
|
||||
Dockerfile and build a new binary environment. See 'Compiling' for instructions.
|
||||
|
||||
When executing functions using binaries, **ensure that the executable is built for the right architecture**.
|
||||
|
||||
@@ -44,14 +44,14 @@ namespace Fission.DotNetCore
|
||||
var oinfo = new List<string>();
|
||||
var _request = Request;
|
||||
var _body = Request.Body;
|
||||
// Request.Body.Position = 0; use it only if requst has already been read before that
|
||||
// Request.Body.Position = 0; use it only if request has already been read before that
|
||||
var _requestBodystring = RequestStream.FromStream(Request.Body).AsString();
|
||||
Console.WriteLine($"Request received by endpoint from builder : {_requestBodystring}");
|
||||
BuilderRequest builderRequest = EnvironmentHelper.Instance.GetBuilderRequest(_requestBodystring);
|
||||
if (builderRequest == null)
|
||||
{
|
||||
Console.WriteLine("Error : Unbale to parse builder request!!");
|
||||
throw new Exception("Error : Unbale to parse builder request!!");
|
||||
Console.WriteLine("Error : Unable to parse builder request!!");
|
||||
throw new Exception("Error : Unable to parse builder request!!");
|
||||
}
|
||||
|
||||
string functionPath = string.Empty;
|
||||
@@ -113,8 +113,8 @@ namespace Fission.DotNetCore
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
Console.WriteLine($"Exception occured {ex.Message} | {ex.StackTrace}");
|
||||
var errstr = $"Exception occured {ex.Message} | {ex.StackTrace}";
|
||||
Console.WriteLine($"Exception occurred {ex.Message} | {ex.StackTrace}");
|
||||
var errstr = $"Exception occurred {ex.Message} | {ex.StackTrace}";
|
||||
_logger.WriteError(errstr);
|
||||
var response = (Response)errstr;
|
||||
response.StatusCode = HttpStatusCode.InternalServerError;
|
||||
|
||||
@@ -87,14 +87,14 @@ namespace Fission.DotNetCore.Compiler
|
||||
errors = new List<string>();
|
||||
oinfo = new List<string>();
|
||||
|
||||
#region syntext tree and default refrence build
|
||||
#region syntext tree and default reference build
|
||||
SyntaxTree syntaxTree = CSharpSyntaxTree.ParseText(code);
|
||||
|
||||
string assemblyName = Path.GetRandomFileName();
|
||||
|
||||
var coreDir = Directory.GetParent(typeof(Enumerable).GetTypeInfo().Assembly.Location);
|
||||
|
||||
Console.WriteLine("Adding core refrences !!");
|
||||
Console.WriteLine("Adding core references !!");
|
||||
List<MetadataReference> references = new List<MetadataReference>
|
||||
{
|
||||
MetadataReference.CreateFromFile(coreDir.FullName + Path.DirectorySeparatorChar + "mscorlib.dll"),
|
||||
@@ -104,16 +104,13 @@ namespace Fission.DotNetCore.Compiler
|
||||
MetadataReference.CreateFromFile(typeof(System.Runtime.Serialization.Json.DataContractJsonSerializer).GetTypeInfo().Assembly.Location)
|
||||
};
|
||||
|
||||
Console.WriteLine("Adding parent assembaly based refrences !!");
|
||||
Console.WriteLine("Adding parent assembly based references !!");
|
||||
foreach (var referencedAssembly in Assembly.GetEntryAssembly().GetReferencedAssemblies())
|
||||
{
|
||||
var assembly = Assembly.Load(referencedAssembly);
|
||||
references.Add(MetadataReference.CreateFromFile(assembly.Location));
|
||||
|
||||
references.Add(MetadataReference.CreateFromFile(assembly.Location));
|
||||
}
|
||||
|
||||
|
||||
|
||||
#endregion
|
||||
|
||||
#region load function specs based dlls
|
||||
@@ -128,14 +125,14 @@ namespace Fission.DotNetCore.Compiler
|
||||
{
|
||||
string dllCompletePath = Path.Combine(packagepath, library.path).GetrelevantPathAsPerOS();
|
||||
references.Add(MetadataReference.CreateFromFile(dllCompletePath));
|
||||
Console.WriteLine($"refered folder based dll : {dllCompletePath} from package {library.nugetPackage}");
|
||||
Console.WriteLine($"referred folder based dll : {dllCompletePath} from package {library.nugetPackage}");
|
||||
}
|
||||
Console.WriteLine($"refered all available dlls!!");
|
||||
oinfo.Add("refered all available dlls!!");
|
||||
Console.WriteLine($"referred all available dlls!!");
|
||||
oinfo.Add("referred all available dlls!!");
|
||||
|
||||
#endregion
|
||||
|
||||
#region dynamic resolve handeler registration
|
||||
#region dynamic resolve handler registration
|
||||
AppDomain currentDomain = AppDomain.CurrentDomain;
|
||||
currentDomain.AssemblyResolve += CurrentDomain_AssemblyResolve;
|
||||
|
||||
@@ -188,9 +185,6 @@ namespace Fission.DotNetCore.Compiler
|
||||
return null;
|
||||
|
||||
#endregion
|
||||
|
||||
|
||||
|
||||
}
|
||||
|
||||
private Assembly CurrentDomain_AssemblyResolve(object sender, ResolveEventArgs args)
|
||||
@@ -213,7 +207,7 @@ namespace Fission.DotNetCore.Compiler
|
||||
{
|
||||
strTempAssmbPath_relative = functionSpecification.libraries.Where(x => x.name.ToLower() == (args.Name.Substring(0, args.Name.IndexOf(",")).ToString() + ".dll").ToLower()).FirstOrDefault().path;
|
||||
strTempAssmbPath_absolute = Path.Combine(packagepath, strTempAssmbPath_relative);
|
||||
Console.WriteLine($"loading dll in parent assembaly :{strTempAssmbPath_absolute.GetrelevantPathAsPerOS()}");
|
||||
Console.WriteLine($"loading dll in parent assembly :{strTempAssmbPath_absolute.GetrelevantPathAsPerOS()}");
|
||||
//Load the assembly from the specified path.
|
||||
MyAssembly = Assembly.LoadFile(strTempAssmbPath_absolute.GetrelevantPathAsPerOS());
|
||||
Console.WriteLine($"Load success for :{strTempAssmbPath_absolute.GetrelevantPathAsPerOS()}");
|
||||
|
||||
@@ -47,7 +47,6 @@ namespace Fission.DotNetCore.Api
|
||||
{
|
||||
return File.ReadAllText(Path.Combine(this.PackagePath, relativePath));
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
public class Logger
|
||||
@@ -128,4 +127,4 @@ namespace Fission.DotNetCore.Api
|
||||
public string Url { get { return _request.Url.ToString(); } }
|
||||
public string Method { get { return _request.Method; } }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -235,7 +235,7 @@ Hello, my name is Arthur and I am 42 years old.
|
||||
```
|
||||
|
||||
|
||||
## Developing/debugging the enviroment locally
|
||||
## Developing/debugging the environment locally
|
||||
|
||||
The easiest way to debug the environment is to open the directory in
|
||||
Visual Studio Code (VSCode) as that will setup debugger for you the
|
||||
@@ -269,7 +269,7 @@ $ curl -XPOST http://localhost:8888/specialize
|
||||
$ curl -XGET http://localhost:8888
|
||||
```
|
||||
|
||||
## Few Aditional Features
|
||||
## Few Additional Features
|
||||
|
||||
**1. NameSpace support :**
|
||||
|
||||
@@ -283,16 +283,17 @@ Now , You can use namespace for Fission function class and have many other class
|
||||
public class FissionFunction
|
||||
{
|
||||
public string Execute(FissionContext context){
|
||||
//orignal logic
|
||||
//original logic
|
||||
}
|
||||
public string AnotherClass(string myval){
|
||||
//do something
|
||||
public string AnotherClass(string myVal){
|
||||
//do something
|
||||
}
|
||||
}
|
||||
```
|
||||
**2. Aditional **setting/configuration file** support :**
|
||||
|
||||
**2. Additional **setting/configuration file** support :**
|
||||
|
||||
Now , with Fission V2 end point with builder , in source package you can have aditional setting
|
||||
Now , with Fission V2 end point with builder , in source package you can have additional setting
|
||||
files which can be read by fission function .
|
||||
Lets say you are writing a function and you need some configurable option and setting to be available in function and thus you want to use some additional configuration file , then you can also achieve the same by having a JSON based configuration file and a corresponding POCO Class for the same.
|
||||
|
||||
@@ -302,7 +303,7 @@ Here is an example of a such file which we want to use in function , lets say y
|
||||
|
||||
```
|
||||
Source Package zip :
|
||||
--soruce.zip
|
||||
--source.zip
|
||||
|--Func.cs
|
||||
|--nuget.txt
|
||||
|--exclude.txt
|
||||
@@ -335,13 +336,13 @@ namespace FuncNameSpace
|
||||
public class FissionFunction
|
||||
{
|
||||
public string Execute(FissionContext context){
|
||||
string respo="initial value";
|
||||
string res="initial value";
|
||||
context.Logger.WriteInfo("Staring..... ");
|
||||
var settings =context.GetSettings<SendGridSettings>("mysetting.json");
|
||||
context.Logger.WriteInfo($"SendGridEndPoint port : {settings.SendGridEndPoints[0].port} ..... ");
|
||||
respo=settings.SendGridEndPoints[0].port;
|
||||
res=settings.SendGridEndPoints[0].port;
|
||||
context.Logger.WriteInfo("Done!!");
|
||||
return respo;
|
||||
return res;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -8,18 +8,13 @@ namespace Fission.DotNetCore.Utilty
|
||||
{
|
||||
public sealed class ObjectConverter
|
||||
{
|
||||
|
||||
private static readonly Lazy<ObjectConverter> lazy =
|
||||
new Lazy<ObjectConverter>(() => new ObjectConverter());
|
||||
|
||||
public static ObjectConverter Instance { get { return lazy.Value; } }
|
||||
|
||||
private ObjectConverter()
|
||||
{
|
||||
private ObjectConverter() {}
|
||||
|
||||
}
|
||||
|
||||
|
||||
public EnvironmentSettings GetWatcherSettingsFromJson(string json)
|
||||
{
|
||||
return JsonConvert.DeserializeObject<EnvironmentSettings>(json);
|
||||
@@ -29,8 +24,5 @@ namespace Fission.DotNetCore.Utilty
|
||||
{
|
||||
return JsonConvert.DeserializeObject<FunctionSpecification>(json);
|
||||
}
|
||||
|
||||
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
@@ -41,7 +41,7 @@ namespace Builder
|
||||
}
|
||||
catch(Exception childEx)
|
||||
{
|
||||
//do nothing , just log orignal exception
|
||||
//do nothing , just log original exception
|
||||
Console.WriteLine($"{Environment.NewLine} Exception During Build :{ex.Message} |{Environment.NewLine} {ex.StackTrace} {Environment.NewLine} ");
|
||||
}
|
||||
|
||||
|
||||
@@ -40,18 +40,18 @@ namespace Builder.Engine
|
||||
await BuildDllInfo();
|
||||
|
||||
Console.WriteLine("DLL Info Gathered!!");
|
||||
// try to compile the function and if compilation succedd ,then create func spec file
|
||||
// try to compile the function and if compilation succeed ,then create func spec file
|
||||
//this enables us to find compilation issues during package creation itself thus saving time
|
||||
// however this feature impose that the function file name should be func.cs
|
||||
//if we dont want it , we can comment the TryCompile() logic
|
||||
Console.WriteLine("Trying to compile it during build itslef !!");
|
||||
//if we don't want it , we can comment the TryCompile() logic
|
||||
Console.WriteLine("Trying to compile it during build itself !!");
|
||||
bool compiled =await TryCompile();
|
||||
Console.WriteLine($"Compilation result Gathered as : {compiled}!!");
|
||||
if (compiled)
|
||||
{
|
||||
|
||||
//nowwhatever has been done so far and all files which are generated are in /app folder where dll resides
|
||||
//thus copy relavant thing in SRC_PKG as it is ,but lest skip it here , we shall do it in build.sh
|
||||
//thus copy relevant thing in SRC_PKG as it is ,but lest skip it here , we shall do it in build.sh
|
||||
CopyToSourceDir();
|
||||
Console.WriteLine($"Copy to Source Done!!");
|
||||
//build the function specs
|
||||
@@ -98,7 +98,7 @@ namespace Builder.Engine
|
||||
|
||||
public async Task<bool> TryCompile()
|
||||
{
|
||||
bool issuccess = false;
|
||||
bool isSuccess = false;
|
||||
|
||||
string CODE_PATH = Path.Combine(SRC_PKG, BuilderHelper.Instance.builderSettings.functionBodyFileName);
|
||||
if (!File.Exists(CODE_PATH))
|
||||
@@ -107,20 +107,20 @@ namespace Builder.Engine
|
||||
$" to use TryCompile() in Builder, make sure , your main function file name is " +
|
||||
$"{BuilderHelper.Instance.builderSettings.functionBodyFileName} and " +
|
||||
$"it is located at root of zip!!" );
|
||||
return issuccess;
|
||||
return isSuccess;
|
||||
}
|
||||
|
||||
var code = File.ReadAllText(CODE_PATH);
|
||||
issuccess = await Compile(code);
|
||||
isSuccess = await Compile(code);
|
||||
|
||||
return issuccess;
|
||||
return isSuccess;
|
||||
}
|
||||
|
||||
public async Task<bool> Compile(string code)
|
||||
{
|
||||
bool issuccess = false;
|
||||
bool isSuccess = false;
|
||||
|
||||
#region assymbaly init and parent dll refrences
|
||||
#region assembly init and parent dll references
|
||||
SyntaxTree syntaxTree = CSharpSyntaxTree.ParseText(code);
|
||||
string assemblyName = Path.GetRandomFileName();
|
||||
|
||||
@@ -139,13 +139,13 @@ namespace Builder.Engine
|
||||
{
|
||||
var assembly = Assembly.Load(referencedAssembly);
|
||||
references.Add(MetadataReference.CreateFromFile(assembly.Location));
|
||||
BuilderHelper.Instance.logger.Log($"Refering assembaly based dls : {assembly.Location}");
|
||||
BuilderHelper.Instance.logger.Log($"Refering assembly based dls : {assembly.Location}");
|
||||
}
|
||||
|
||||
#endregion
|
||||
|
||||
#region handler registration for runtime resolution
|
||||
//now add handeler for missing dlls for parent app domain as same assembalies should be needed
|
||||
//now add handler for missing dlls for parent app domain as same assemblies should be needed
|
||||
//for parent , thus refering from https://support.microsoft.com/en-in/help/837908/how-to-load-an-assembly-at-runtime-that-is-located-in-a-folder-that-is
|
||||
AppDomain currentDomain = AppDomain.CurrentDomain;
|
||||
currentDomain.AssemblyResolve += CurrentDomain_AssemblyResolve;
|
||||
@@ -153,8 +153,8 @@ namespace Builder.Engine
|
||||
#endregion
|
||||
|
||||
BuilderHelper.Instance.logger.Log($"dynamic handlar registered!!");
|
||||
#region nuget dll refrence add
|
||||
//now add those dll refrence
|
||||
#region nuget dll reference add
|
||||
//now add those dll reference
|
||||
foreach (var dll in dllInfos)
|
||||
{
|
||||
BuilderHelper.Instance.logger.Log($"refering nuget based dll : {dll.path}");
|
||||
@@ -193,12 +193,12 @@ namespace Builder.Engine
|
||||
else
|
||||
{
|
||||
BuilderHelper.Instance.logger.Log("Compile Success!!",true);
|
||||
issuccess = true;
|
||||
isSuccess = true;
|
||||
}
|
||||
}
|
||||
|
||||
#endregion
|
||||
return issuccess;
|
||||
return isSuccess;
|
||||
|
||||
}
|
||||
|
||||
@@ -223,7 +223,7 @@ namespace Builder.Engine
|
||||
functionSpecification.libraries.Add(library);
|
||||
}
|
||||
|
||||
//serilize that object to save it in json file
|
||||
//serialize that object to save it in json file
|
||||
string funcMetaJson= JsonConvert.SerializeObject(functionSpecification);
|
||||
|
||||
string funcMetaFile = Path.Combine(this.SRC_PKG, BuilderHelper.Instance.builderSettings.functionSpecFileName);
|
||||
@@ -247,7 +247,7 @@ namespace Builder.Engine
|
||||
dllInfos.AddRange(nugetEngine.dllInfos);
|
||||
}
|
||||
|
||||
//now do a distinct of all dlls paths as multiple packaed might have added same dll
|
||||
//now do a distinct of all dlls paths as multiple packaged might have added same dll
|
||||
dllInfos = dllInfos.DistinctBy(x => x.path).ToList();
|
||||
|
||||
#if DEBUG
|
||||
|
||||
@@ -21,12 +21,12 @@ FROM microsoft/dotnet:aspnetcore-runtime
|
||||
WORKDIR /app
|
||||
COPY --from=builderimage /app/out .
|
||||
|
||||
#this builder is actually compilation from : https://github.com/fission/fission/tree/master/builder/cmd and renamed cmd.exe to builder
|
||||
# this builder is actually compilation from : https://github.com/fission/fission/tree/master/builder/cmd and renamed cmd.exe to builder
|
||||
# make sure to compile it in linux only else you will get exec execute error as binary was compiled in windows and running on linux
|
||||
|
||||
COPY --from=fission-builder /builder /builder
|
||||
|
||||
#ADD builder /builder
|
||||
# ADD builder /builder
|
||||
|
||||
ADD build.sh /usr/local/bin/build
|
||||
RUN chmod +x /usr/local/bin/build
|
||||
@@ -34,4 +34,4 @@ RUN chmod +x /usr/local/bin/build
|
||||
ADD build.sh /bin/build
|
||||
RUN chmod +x /bin/build
|
||||
|
||||
EXPOSE 8001
|
||||
EXPOSE 8001
|
||||
|
||||
@@ -34,22 +34,22 @@ namespace Fission.DotNetCore.Api
|
||||
new FissionHttpRequest(request));
|
||||
}
|
||||
|
||||
//this are curruntly dummy , not being implemented, just to pass compilation
|
||||
//this is a dummy, not being implemented, just to pass compilation
|
||||
//actual execution is written in environment to use the app settings as there we need it
|
||||
public T GetSettings<T>(string relativePath)
|
||||
{
|
||||
//intentionaly doing it as these are just dummy methods not being called
|
||||
//intentionally doing it as these are just dummy methods not being called
|
||||
//but if tomorrow if we decide to give implementation for execution in build then we
|
||||
//need to implement it
|
||||
throw new NotImplementedException();
|
||||
}
|
||||
|
||||
//this are curruntly dummy , not being implemented, just to pass compilation
|
||||
//this is a dummy, not being implemented, just to pass compilation
|
||||
//actual execution is written in environment to use the app settings as there we need it
|
||||
private string GetSettingsJson(string relativePath)
|
||||
{
|
||||
//intentionaly doing it as these are just dummy methods not being called
|
||||
//but if tomorrow if we decide to give implementation for execution in build then we
|
||||
//intentionally doing it as these are just dummy methods not being called
|
||||
//but tomorrow if we decide to give implementation for execution in build then we
|
||||
//need to implement it
|
||||
throw new NotImplementedException();
|
||||
}
|
||||
@@ -134,4 +134,4 @@ namespace Fission.DotNetCore.Api
|
||||
public string Url { get { return _request.Url.ToString(); } }
|
||||
public string Method { get { return _request.Method; } }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4,7 +4,7 @@ This is a simple dotnet core 2.0 C# environment builder for Fission.
|
||||
|
||||
It's a docker image containing the dotnet 2.0.0 (core) run-time builder. This image read the source package and uses
|
||||
*roslyn* to compile the source package code and creates deployment package out of it.
|
||||
This enables using nuget packages as part of function and thus user can use extended functionality in fission functions via nuget.
|
||||
This enables using nuget packages as part of function and thus user can use extended functionality in fission functions via nuget.
|
||||
|
||||
During build , builder also does a pre-compile to prevent any compilation issues during function environment pod specialization.
|
||||
Thus we get the function compilation issues during builder phase in package info's build logs itself.
|
||||
@@ -25,7 +25,7 @@ The source package structure in zip file :
|
||||
|
||||
```
|
||||
Source Package zip :
|
||||
--soruce.zip
|
||||
--source.zip
|
||||
|--func.cs
|
||||
|--nuget.txt
|
||||
|--exclude.txt
|
||||
@@ -63,8 +63,8 @@ this should match the following regex as mentions in builderSetting.json
|
||||
```
|
||||
"ExcludeDllRegEx": "\\:?\\s*(?<package>[^:\\n]*)(?:\\:)?(?<dll>.*)?",
|
||||
```
|
||||
From above , builder will create a deployment package with all dlls in a folder and one functionspecification file :
|
||||
Deployement Package zip :
|
||||
From above , builder will create a deployment package with all dlls in a folder and one function specification file :
|
||||
Deployment Package zip :
|
||||
|
||||
```
|
||||
--Deploye.zip
|
||||
@@ -77,7 +77,7 @@ this should match the following regex as mentions in builderSetting.json
|
||||
|--csvhelper.dll
|
||||
|--logs()
|
||||
|-->logFileName
|
||||
|--func.meta.json // this is the functionspecific file
|
||||
|--func.meta.json // this is the function specific file
|
||||
|--....MiscFiles(optional)
|
||||
|--....MiscFiles(optional)
|
||||
```
|
||||
@@ -93,19 +93,19 @@ using Fission.DotNetCore.Api;
|
||||
public class FissionFunction
|
||||
{
|
||||
public string Execute(FissionContext context){
|
||||
string respo="initial value";
|
||||
string res="initial value";
|
||||
try
|
||||
{
|
||||
context.Logger.WriteInfo("Staring..... ");
|
||||
respo=$" sample object by getting Enum of CsvHelper nuget dll: { CsvHelper.Caches.NamedIndex.ToString()}";
|
||||
res=$" sample object by getting Enum of CsvHelper nuget dll: { CsvHelper.Caches.NamedIndex.ToString()}";
|
||||
}
|
||||
catch(Exception ex)
|
||||
{
|
||||
context.Logger.WriteError(ex.Message);
|
||||
respo = ex.Message;
|
||||
res = ex.Message;
|
||||
}
|
||||
context.Logger.WriteInfo("Done!");
|
||||
return respo;
|
||||
return res;
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -116,16 +116,16 @@ CsvHelper
|
||||
```
|
||||
**Content of exclude.txt**
|
||||
|
||||
As we dont want to exclude any specific dll thus we shall leave it as empty.
|
||||
As we don't want to exclude any specific dll thus we shall leave it as empty.
|
||||
|
||||
Now check name of existing environments & functions as we want to create a unique environment for this dotnetcore if not already present
|
||||
Now check name of existing environments & functions as we want to create a unique environment for this .Net Core if not already present
|
||||
|
||||
```
|
||||
fission env list
|
||||
fission fn list
|
||||
```
|
||||
Create Environment with builder (choose a unique which doesn't exist , here we have chosen : dotnetcorewithnuget )
|
||||
also suppose the builder image name is fissiondotnet20-builder and hosted on dockerhub as fission/dotnet20-builder
|
||||
also suppose the builder image name is fissiondotnet20-builder and hosted on Docker Hub as fission/dotnet20-builder
|
||||
```
|
||||
fission environment create --name dotnetcorewithnuget --image fission/dotnet20-env --builder fission/dotnet20-builder
|
||||
```
|
||||
|
||||
@@ -77,7 +77,7 @@ public class JerseyServer {
|
||||
Enumeration<JarEntry> e = jarFile.entries();
|
||||
URL[] urls = { new URL("jar:file:" + file + "!/") };
|
||||
|
||||
// TODO Check if the classloading can be improved for ex. use something like:
|
||||
// TODO Check if the class loading can be improved for ex. use something like:
|
||||
// Thread.currentThread().setContextClassLoader(cl);
|
||||
if (this.getClass().getClassLoader() == null) {
|
||||
cl = URLClassLoader.newInstance(urls);
|
||||
@@ -86,8 +86,7 @@ public class JerseyServer {
|
||||
}
|
||||
|
||||
if (cl == null) {
|
||||
|
||||
return Response.status(Response.Status.BAD_REQUEST).entity("Failed to initialize the classloader")
|
||||
return Response.status(Response.Status.BAD_REQUEST).entity("Failed to initialize the class loader")
|
||||
.build();
|
||||
}
|
||||
|
||||
|
||||
@@ -61,7 +61,7 @@ public class Server {
|
||||
Enumeration<JarEntry> e = jarFile.entries();
|
||||
URL[] urls = { new URL("jar:file:" + file + "!/") };
|
||||
|
||||
// TODO Check if the classloading can be improved for ex. use something like:
|
||||
// TODO Check if the class loading can be improved for ex. use something like:
|
||||
// Thread.currentThread().setContextClassLoader(cl);
|
||||
if (this.getClass().getClassLoader() == null) {
|
||||
cl = URLClassLoader.newInstance(urls);
|
||||
@@ -70,7 +70,7 @@ public class Server {
|
||||
}
|
||||
|
||||
if (cl == null) {
|
||||
return ResponseEntity.status(500).body("Failed to initialize the classloader");
|
||||
return ResponseEntity.status(500).body("Failed to initialize the class loader");
|
||||
}
|
||||
|
||||
// Load all dependent classes from libraries etc.
|
||||
@@ -119,5 +119,4 @@ public class Server {
|
||||
public static void main(String[] args) throws Exception {
|
||||
SpringApplication.run(Server.class, args);
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
@@ -54,5 +54,5 @@ Or, if you already have an environment, you can update its image:
|
||||
fission env update --name php7 --image USER/php7-env
|
||||
```
|
||||
|
||||
After this, fission functions that have the env parmeter set to the
|
||||
After this, fission functions that have the env parameter set to the
|
||||
same environment name as this command will use this environment.
|
||||
|
||||
@@ -222,7 +222,7 @@ type (
|
||||
URL string `json:"url,omitempty"`
|
||||
|
||||
// Checksum ensures the integrity of packages
|
||||
// refereced by URL. Ignored for literals.
|
||||
// referenced by URL. Ignored for literals.
|
||||
Checksum Checksum `json:"checksum,omitempty"`
|
||||
}
|
||||
|
||||
@@ -377,7 +377,7 @@ type (
|
||||
// ExecutionStrategy specifies low-level parameters for function execution,
|
||||
// such as the number of instances.
|
||||
//
|
||||
// MinScale affects the cold start behaviour for a function. If MinScale is 0 then the
|
||||
// MinScale affects the cold start behavior for a function. If MinScale is 0 then the
|
||||
// deployment is created on first invocation of function and is good for requests of
|
||||
// asynchronous nature. If MinScale is greater than 0 then MinScale number of pods are
|
||||
// created at the time of creation of function. This ensures faster response during first
|
||||
@@ -679,7 +679,7 @@ type (
|
||||
MqtKind string `json:"mqtkind,omitempty"`
|
||||
}
|
||||
|
||||
// TimeTrigger invokes the specific function at a time or
|
||||
// TimeTriggerSpec invokes the specific function at a time or
|
||||
// times specified by a cron string.
|
||||
TimeTriggerSpec struct {
|
||||
// Cron schedule
|
||||
@@ -691,7 +691,7 @@ type (
|
||||
|
||||
FailureType string
|
||||
|
||||
// Canary Config Spec
|
||||
// CanaryConfigSpec defines the canary configuration spec
|
||||
CanaryConfigSpec struct {
|
||||
// HTTP trigger that this config references
|
||||
Trigger string `json:"trigger"`
|
||||
@@ -713,7 +713,7 @@ type (
|
||||
FailureType FailureType `json:"failureType"`
|
||||
}
|
||||
|
||||
// CanaryConfig Status
|
||||
// CanaryConfigStatus represents canary config status
|
||||
CanaryConfigStatus struct {
|
||||
Status string `json:"status"`
|
||||
}
|
||||
|
||||
@@ -153,7 +153,7 @@ func (pkgw *packageWatcher) build(buildCache *cache.Cache, srcpkg *fv1.Package)
|
||||
}
|
||||
|
||||
// Add the package getter rolebinding to builder sa
|
||||
// we continue here if role binding was not setup succeesffully. this is because without this, the fetcher wont be able to fetch the source pkg into the container and
|
||||
// we continue here if role binding was not setup successfully. this is because without this, the fetcher wont be able to fetch the source pkg into the container and
|
||||
// the build will fail eventually
|
||||
err := utils.SetupRoleBinding(pkgw.logger, pkgw.k8sClient, fv1.PackageGetterRB, pkg.ObjectMeta.Namespace, fv1.PackageGetterCR, fv1.ClusterRole, fv1.FissionBuilderSA, builderNs)
|
||||
if err != nil {
|
||||
@@ -264,7 +264,7 @@ func (pkgw *packageWatcher) watchPackages() {
|
||||
// TODO: Once enable "/status", check generation for spec changed instead.
|
||||
// Before "/status" is enabled, the generation and resource version will be changed
|
||||
// if we update the status of a package, hence we are not able to differentiate
|
||||
// the spec change or status change. So we only build package which's status
|
||||
// the spec change or status change. So we only build package which has status
|
||||
// us "pending" and user have to use "kubectl replace" to update a package.
|
||||
if oldPkg.ResourceVersion == pkg.ResourceVersion &&
|
||||
pkg.Status.BuildStatus != fv1.BuildStatusPending {
|
||||
|
||||
@@ -337,7 +337,7 @@ func (canaryCfgMgr *canaryConfigMgr) RollForwardOrBack(canaryConfig *fv1.CanaryC
|
||||
|
||||
if doneProcessingCanaryConfig {
|
||||
ticker.Stop()
|
||||
// update the status of canary config as done processing, we dont care if we arent able to update because
|
||||
// update the status of canary config as done processing, we don't care if we aren't able to update because
|
||||
// resync takes care of the update
|
||||
err = canaryCfgMgr.updateCanaryConfigStatusWithRetries(canaryConfig.ObjectMeta.Name, canaryConfig.ObjectMeta.Namespace,
|
||||
fv1.CanaryConfigStatusSucceeded)
|
||||
|
||||
@@ -130,7 +130,7 @@ func (api *API) extractQueryParamFromRequest(r *http.Request, queryParam string)
|
||||
// check if namespace exists, if not create it.
|
||||
func (api *API) createNsIfNotExists(ns string) error {
|
||||
if ns == metav1.NamespaceDefault {
|
||||
// we dont have to create default ns
|
||||
// we don't have to create default ns
|
||||
return nil
|
||||
}
|
||||
|
||||
|
||||
@@ -19,6 +19,7 @@ package v1
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
|
||||
"github.com/fission/fission/pkg/controller/client/rest"
|
||||
|
||||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
@@ -105,7 +106,7 @@ func (c *KubeWatcher) Get(m *metav1.ObjectMeta) (*fv1.KubernetesWatchTrigger, er
|
||||
}
|
||||
|
||||
func (c *KubeWatcher) Update(w *fv1.KubernetesWatchTrigger) (*metav1.ObjectMeta, error) {
|
||||
return nil, ferror.MakeError(ferror.ErrorNotImplmented, "watch update not implemented")
|
||||
return nil, ferror.MakeError(ferror.ErrorNotImplemented, "watch update not implemented")
|
||||
}
|
||||
|
||||
func (c *KubeWatcher) Delete(m *metav1.ObjectMeta) error {
|
||||
|
||||
@@ -182,7 +182,7 @@ func (a *API) WatchApiGet(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
|
||||
func (a *API) WatchApiUpdate(w http.ResponseWriter, r *http.Request) {
|
||||
a.respondWithError(w, ferror.MakeError(ferror.ErrorNotImplmented,
|
||||
a.respondWithError(w, ferror.MakeError(ferror.ErrorNotImplemented,
|
||||
"Not implemented"))
|
||||
}
|
||||
|
||||
|
||||
@@ -131,7 +131,7 @@ const (
|
||||
ErrorNameExists
|
||||
ErrorInvalidArgument
|
||||
ErrorNoSpace
|
||||
ErrorNotImplmented
|
||||
ErrorNotImplemented
|
||||
ErrorChecksumFail
|
||||
ErrorSizeLimitExceeded
|
||||
ErrorRequestTimeout
|
||||
|
||||
+1
-1
@@ -78,7 +78,7 @@ func (executor *Executor) getServiceForFunctionApi(w http.ResponseWriter, r *htt
|
||||
}
|
||||
if t == fv1.ExecutorTypePoolmgr && et.GetTotalAvailable(fn) >= conncurrency {
|
||||
errMsg := fmt.Sprintf("max concurrency reached for %v. All %v instance are active", fn.ObjectMeta.Name, fn.Spec.Concurrency)
|
||||
executor.logger.Error("error occured", zap.String("error", errMsg))
|
||||
executor.logger.Error("error occurred", zap.String("error", errMsg))
|
||||
http.Error(w, errMsg, http.StatusTooManyRequests)
|
||||
return
|
||||
}
|
||||
|
||||
@@ -558,8 +558,8 @@ func (deploy *NewDeploy) cleanupNewdeploy(ns string, name string) error {
|
||||
|
||||
// referencedResourcesRVSum returns the sum of resource version of all resources the function references to.
|
||||
// We used to update timestamp in the deployment environment field in order to trigger a rolling update when
|
||||
// the function referenced resources get updated. However, use timestamp means we are not able to avoid tri-
|
||||
// ggering a rolling update when executor tries to adopt orphaned deployment due to timestamp changed which
|
||||
// the function referenced resources get updated. However, use timestamp means we are not able to avoid
|
||||
// triggering a rolling update when executor tries to adopt orphaned deployment due to timestamp changed which
|
||||
// is unwanted. In order to let executor adopt deployment without triggering a rolling update, we need an
|
||||
// identical way to get a value that can reflect resources changed without affecting by the time.
|
||||
// To achieve this goal, the sum of the resource version of all referenced resources is a good fit for our
|
||||
|
||||
@@ -70,7 +70,7 @@ func (gpm *GenericPoolManager) makeFuncController(fissionClient *crd.FissionClie
|
||||
}
|
||||
|
||||
// TODO : Just bring to your attention during review :
|
||||
// setup rolebinding is tried, if it fails, we dont return. we just log an error and move on, because :
|
||||
// setup rolebinding is tried, if it fails, we don't return. we just log an error and move on, because :
|
||||
// 1. not all functions have secrets and/or configmaps, so things will work without this rolebinding in that case.
|
||||
// 2. on the contrary, when the route is tried, the env fetcher logs will show a 403 forbidden message and same will be relayed to executor.
|
||||
err := utils.SetupRoleBinding(gpm.logger, kubernetesClient, fv1.SecretConfigMapGetterRB, fn.ObjectMeta.Namespace, fv1.SecretConfigMapGetterCR, fv1.ClusterRole, fv1.FissionFetcherSA, envNs)
|
||||
|
||||
@@ -253,7 +253,7 @@ func (fetcher *Fetcher) Fetch(ctx context.Context, pkg *fv1.Package, req Functio
|
||||
}
|
||||
archive = &pkg.Spec.Deployment
|
||||
} else {
|
||||
return http.StatusBadRequest, fmt.Errorf("unkonwn fetch type: %v", req.FetchType)
|
||||
return http.StatusBadRequest, fmt.Errorf("unknown fetch type: %v", req.FetchType)
|
||||
}
|
||||
|
||||
// get package data as literal or by url
|
||||
@@ -297,7 +297,7 @@ func (fetcher *Fetcher) Fetch(ctx context.Context, pkg *fv1.Package, req Functio
|
||||
tmpUnarchivePath := filepath.Join(fetcher.sharedVolumePath, uuid.NewV4().String())
|
||||
err := fetcher.unarchive(tmpPath, tmpUnarchivePath)
|
||||
if err != nil {
|
||||
fetcher.logger.Error("error unarchiving",
|
||||
fetcher.logger.Error("error unarchive",
|
||||
zap.Error(err),
|
||||
zap.String("archive_location", tmpPath),
|
||||
zap.String("target_location", tmpUnarchivePath))
|
||||
|
||||
@@ -64,7 +64,7 @@ func (opts *UpdateSubCommand) complete(input cli.Input) error {
|
||||
envName := input.String(flagkey.FnEnvironmentName)
|
||||
envNamespace := input.String(flagkey.NamespaceEnvironment)
|
||||
// if the new env specified is the same as the old one, no need to update package
|
||||
// same is true for all update parameters, but, for now, we dont check all of them - because, its ok to
|
||||
// same is true for all update parameters, but, for now, we don't check all of them - because, its ok to
|
||||
// re-write the object with same old values, we just end up getting a new resource version for the object.
|
||||
if len(envName) > 0 && envName == function.Spec.Environment.Name {
|
||||
envName = ""
|
||||
|
||||
@@ -53,7 +53,7 @@ func (opts *ListSubCommand) complete(input cli.Input) error {
|
||||
func (opts *ListSubCommand) run(input cli.Input) error {
|
||||
ws, err := opts.Client().V1().KubeWatcher().List(opts.namespace)
|
||||
if err != nil {
|
||||
return errors.Wrap(err, "error listing kubewatches")
|
||||
return errors.Wrap(err, "error listing kubewatchers")
|
||||
}
|
||||
|
||||
w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
|
||||
|
||||
@@ -88,7 +88,7 @@ func (influx InfluxDB) GetLogs(filter LogFilter) ([]LogEntry, error) {
|
||||
for _, r := range response.Results {
|
||||
for _, series := range r.Series {
|
||||
|
||||
//create map of columns to row indeces
|
||||
//create map of columns to row indices
|
||||
indexMap := makeIndexMap(series.Columns)
|
||||
|
||||
// TODO: Remove fallback indexes. Some of index's name changed in fluent-bit, here we add extra fallbackIndexes to address compatibility problem.
|
||||
|
||||
@@ -96,7 +96,7 @@ func (c *Cache) service() {
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
resp.error = ferror.MakeError(ferror.ErrorNotFound, fmt.Sprintf("funtion '%v' No inactive function found", req.function))
|
||||
resp.error = ferror.MakeError(ferror.ErrorNotFound, fmt.Sprintf("function '%v' No inactive function found", req.function))
|
||||
}
|
||||
req.responseChannel <- resp
|
||||
case listAvailableValue:
|
||||
|
||||
@@ -123,7 +123,7 @@ func functionCallCompleted(f *functionLabels, h *httpLabels, overhead, duration
|
||||
|
||||
l := labelsToStrings(f, h)
|
||||
|
||||
// overhead: time from request ingress into router upto proxing into function pod
|
||||
// overhead: time from request ingress into router up to proxing into function pod
|
||||
functionCallOverhead.WithLabelValues(l...).Observe(float64(overhead.Nanoseconds()) / 1e9)
|
||||
|
||||
// total function call counter
|
||||
|
||||
@@ -29,7 +29,7 @@ const (
|
||||
HEADERS_FISSION_FUNCTION_PREFIX = "Fission-Function"
|
||||
)
|
||||
|
||||
// setFunctionMetadataToHeaders set function metadatas to request header
|
||||
// setFunctionMetadataToHeaders set function metadata to request header
|
||||
func setFunctionMetadataToHeader(meta *metav1.ObjectMeta, request *http.Request) {
|
||||
request.Header.Set(fmt.Sprintf("X-%s-Uid", HEADERS_FISSION_FUNCTION_PREFIX), string(meta.UID))
|
||||
request.Header.Set(fmt.Sprintf("X-%s-Name", HEADERS_FISSION_FUNCTION_PREFIX), meta.Name)
|
||||
|
||||
@@ -115,7 +115,7 @@ func TestS3StorageService(t *testing.T) {
|
||||
}
|
||||
|
||||
// This is to ensure container is up. Just getting minioClient
|
||||
// isn't suffcient to assume container is up.
|
||||
// isn't sufficient to assume container is up.
|
||||
_, err = minioClient.ListBuckets()
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -148,7 +148,7 @@ func TestS3StorageService(t *testing.T) {
|
||||
|
||||
time.Sleep(10 * time.Second)
|
||||
|
||||
// Retrive file trhough minioClient
|
||||
// Retrive file through minioClient
|
||||
reader, err := minioClient.GetObject(bucketName, fileID, minio.GetObjectOptions{})
|
||||
panicIf(err)
|
||||
defer reader.Close()
|
||||
|
||||
@@ -84,7 +84,7 @@ func makeRoleBindingObj(roleBinding, roleBindingNs, role, roleKind, sa, saNamesp
|
||||
}
|
||||
}
|
||||
|
||||
// isSAInRoleBinding checkis if a service account is present in the rolebinding object
|
||||
// isSAInRoleBinding checks if a service account is present in the rolebinding object
|
||||
func isSAInRoleBinding(rbObj *rbac.RoleBinding, sa, ns string) bool {
|
||||
for _, subject := range rbObj.Subjects {
|
||||
if subject.Name == sa && subject.Namespace == ns {
|
||||
@@ -281,7 +281,7 @@ func SetupRoleBinding(logger *zap.Logger, k8sClient *kubernetes.Clientset, roleB
|
||||
// returns silently.
|
||||
func DeleteRoleBinding(k8sClient *kubernetes.Clientset, roleBinding, roleBindingNs string) error {
|
||||
// if deleteRoleBinding is invoked by 2 fission services at the same time for the same rolebinding,
|
||||
// the first call will succeed while the 2nd will fail with isNotFound. but we dont want to error out then.
|
||||
// the first call will succeed while the 2nd will fail with isNotFound. but we don't want to error out then.
|
||||
err := k8sClient.RbacV1beta1().RoleBindings(roleBindingNs).Delete(roleBinding, &metav1.DeleteOptions{})
|
||||
if err == nil || k8serrors.IsNotFound(err) {
|
||||
return nil
|
||||
|
||||
Reference in New Issue
Block a user