V2 types and TPR (#266)

This changes the core fission function, environment and trigger types. It also changes Fission's storage to use ThirdPartyResources.

 - Functions are now specified by packages. Functions can also have both source and deployment packages. A package can be specified by a literal, or by a URL.
 - Environments have a build and runtime component.
 - Triggers reference functions by a FunctionReference. This is a layer of indirection between triggers and functions, and will allow things like incremental function upgrades in future releases.

See Documentation/wip/env-v2.md for design discussion about points 1 and 2.

Changes:

* V2 Types

All types now have a spec, following the pattern of K8s objects.

Functions now have source and deployment packages. A Package can be
specified by literal, or by URL.

Environments now have a builder and runtime component.

All triggers use a new FunctionReference to specify the function. This
for now only uses a function name, but in the future can be extended
to be more flexible.

A new FunctionLoadRequest type is added for specialization requests to
the environment runtime.

* TPR types, TPR init code, and a "fission client"

Implements TPR types using the spec types in fission/types.go.

Adds code for adding creating TPR types, and convenient types for crud
operations on each of our resource types.

Adds code for connecting to K8s API and configuring a REST client with
fission types set up.

* Change old stateful controller into a thin apiserver

This apiserver is now simply a stateless api layer on top of the TPR
types. At the moment it doesn't do anything that couldn't be done by
simply talking to the TPR types. In the future we can have better
validation and potentially some higher level APIs (like versioning for
example) in here.

* Split controller client into files and update for v2 types.

* Update CLI for v2 types.

As far as possible we keep the CLI flags the same. We'll have to add
flags for source/deploy packages and builder/runtime
environments. That will come in the next change.

* Update poolmgr and fetcher for v2 types.

Also adds a poolmgr_test.

* Update router for new types.

Also adds a function reference resolver, which separates out the job
of resolving a FunctionReference to a function.

* Update kubewatcher and timer for v2 types.

* Update Message Queue trigger type for v2 types.

* Minor odds and ends.

* Fission bundle CLI updates

Remove controllerUrl flag, since we don't need it any more.

* Remove etcd deployment (replaced by storing state in TPR)

Also update the poolmgr commandline, and use an env var for the
fetcher image URL.

* Explicit ChecksumType and consts

* Clarify separation of environment interface types
This commit is contained in:
Soam Vasani
2017-08-05 01:18:30 -07:00
committed by GitHub
parent 37aa266a4d
commit e238776bf7
81 changed files with 4649 additions and 3588 deletions
+198
View File
@@ -0,0 +1,198 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package tpr
import (
"errors"
"os"
"time"
"k8s.io/client-go/1.5/kubernetes"
"k8s.io/client-go/1.5/pkg/api"
"k8s.io/client-go/1.5/pkg/api/unversioned"
"k8s.io/client-go/1.5/pkg/runtime"
"k8s.io/client-go/1.5/pkg/runtime/serializer"
"k8s.io/client-go/1.5/rest"
"k8s.io/client-go/1.5/tools/clientcmd"
)
type (
FissionClient struct {
tprClient *rest.RESTClient
}
)
// Get a kubernetes client using the kubeconfig file at the
// environment var $KUBECONFIG, or an in-cluster config if that's
// undefined.
func GetKubernetesClient() (*rest.Config, *kubernetes.Clientset, error) {
var config *rest.Config
var err error
// get the config, either from kubeconfig or using our
// in-cluster service account
kubeConfig := os.Getenv("KUBECONFIG")
if len(kubeConfig) != 0 {
config, err = clientcmd.BuildConfigFromFlags("", kubeConfig)
if err != nil {
return nil, nil, err
}
} else {
config, err = rest.InClusterConfig()
if err != nil {
return nil, nil, err
}
}
// creates the clientset
clientset, err := kubernetes.NewForConfig(config)
if err != nil {
return nil, nil, err
}
return config, clientset, nil
}
// GetTprClient gets a TPR client config
func GetTprClient(config *rest.Config) (*rest.RESTClient, error) {
// mutate config to add our types
configureClient(config)
// make a REST client with that config
return rest.RESTClientFor(config)
}
// configureClient sets up a REST client for Fission TPR types.
//
// This is copied from the client-go TPR example. (I don't understand
// all of it completely.) It registers our types with the global API
// "scheme" (api.Scheme), which keeps a directory of types [I guess so
// it can use the string in the Kind field to make a Go object?]. It
// also puts the fission TPR types under a "group version" which we
// create for our TPRs types.
func configureClient(config *rest.Config) {
groupversion := unversioned.GroupVersion{
Group: "fission.io",
Version: "v1",
}
config.GroupVersion = &groupversion
config.APIPath = "/apis"
config.ContentType = runtime.ContentTypeJSON
config.NegotiatedSerializer = serializer.DirectCodecFactory{CodecFactory: api.Codecs}
schemeBuilder := runtime.NewSchemeBuilder(
func(scheme *runtime.Scheme) error {
scheme.AddKnownTypes(
groupversion,
&Function{},
&FunctionList{},
&api.ListOptions{},
&api.DeleteOptions{},
)
scheme.AddKnownTypes(
groupversion,
&Environment{},
&EnvironmentList{},
&api.ListOptions{},
&api.DeleteOptions{},
)
scheme.AddKnownTypes(
groupversion,
&Httptrigger{},
&HttptriggerList{},
&api.ListOptions{},
&api.DeleteOptions{},
)
scheme.AddKnownTypes(
groupversion,
&Kuberneteswatchtrigger{},
&KuberneteswatchtriggerList{},
&api.ListOptions{},
&api.DeleteOptions{},
)
scheme.AddKnownTypes(
groupversion,
&Timetrigger{},
&TimetriggerList{},
&api.ListOptions{},
&api.DeleteOptions{},
)
scheme.AddKnownTypes(
groupversion,
&Messagequeuetrigger{},
&MessagequeuetriggerList{},
&api.ListOptions{},
&api.DeleteOptions{},
)
return nil
})
schemeBuilder.AddToScheme(api.Scheme)
}
func waitForTPRs(tprClient *rest.RESTClient) error {
start := time.Now()
for {
fi := MakeFunctionInterface(tprClient, api.NamespaceDefault)
_, err := fi.List(api.ListOptions{})
if err != nil {
time.Sleep(100 * time.Millisecond)
} else {
return nil
}
if time.Now().Sub(start) > 30*time.Second {
return errors.New("timeout waiting for TPRs")
}
}
}
func MakeFissionClient() (*FissionClient, *kubernetes.Clientset, error) {
config, kubeClient, err := GetKubernetesClient()
if err != nil {
return nil, nil, err
}
tprClient, err := GetTprClient(config)
if err != nil {
return nil, nil, err
}
fc := &FissionClient{
tprClient: tprClient,
}
return fc, kubeClient, nil
}
func (fc *FissionClient) Functions(ns string) FunctionInterface {
return MakeFunctionInterface(fc.tprClient, ns)
}
func (fc *FissionClient) Environments(ns string) EnvironmentInterface {
return MakeEnvironmentInterface(fc.tprClient, ns)
}
func (fc *FissionClient) Httptriggers(ns string) HttptriggerInterface {
return MakeHttptriggerInterface(fc.tprClient, ns)
}
func (fc *FissionClient) Kuberneteswatchtriggers(ns string) KuberneteswatchtriggerInterface {
return MakeKuberneteswatchtriggerInterface(fc.tprClient, ns)
}
func (fc *FissionClient) Timetriggers(ns string) TimetriggerInterface {
return MakeTimetriggerInterface(fc.tprClient, ns)
}
func (fc *FissionClient) Messagequeuetriggers(ns string) MessagequeuetriggerInterface {
return MakeMessagequeuetriggerInterface(fc.tprClient, ns)
}
func (fc *FissionClient) WaitForTPRs() {
waitForTPRs(fc.tprClient)
}
+119
View File
@@ -0,0 +1,119 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package tpr
import (
"k8s.io/client-go/1.5/pkg/api"
"k8s.io/client-go/1.5/pkg/watch"
"k8s.io/client-go/1.5/rest"
)
type (
EnvironmentInterface interface {
Create(*Environment) (*Environment, error)
Get(name string) (*Environment, error)
Update(*Environment) (*Environment, error)
Delete(name string, options *api.DeleteOptions) error
List(opts api.ListOptions) (*EnvironmentList, error)
Watch(opts api.ListOptions) (watch.Interface, error)
}
environmentClient struct {
client *rest.RESTClient
namespace string
}
)
func MakeEnvironmentInterface(tprClient *rest.RESTClient, namespace string) EnvironmentInterface {
return &environmentClient{
client: tprClient,
namespace: namespace,
}
}
func (ec *environmentClient) Create(e *Environment) (*Environment, error) {
var result Environment
err := ec.client.Post().
Resource("environments").
Namespace(ec.namespace).
Body(e).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (ec *environmentClient) Get(name string) (*Environment, error) {
var result Environment
err := ec.client.Get().
Resource("environments").
Namespace(ec.namespace).
Name(name).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (ec *environmentClient) Update(e *Environment) (*Environment, error) {
var result Environment
err := ec.client.Put().
Resource("environments").
Namespace(ec.namespace).
Name(e.Metadata.Name).
Body(e).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (ec *environmentClient) Delete(name string, opts *api.DeleteOptions) error {
return ec.client.Delete().
Namespace(ec.namespace).
Resource("environments").
Name(name).
Body(opts).
Do().
Error()
}
func (ec *environmentClient) List(opts api.ListOptions) (*EnvironmentList, error) {
var result EnvironmentList
err := ec.client.Get().
Namespace(ec.namespace).
Resource("environments").
VersionedParams(&opts, api.ParameterCodec).
Do().
Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (ec *environmentClient) Watch(opts api.ListOptions) (watch.Interface, error) {
return ec.client.Get().
Prefix("watch").
Namespace(ec.namespace).
Resource("environments").
VersionedParams(&opts, api.ParameterCodec).
Watch()
}
+119
View File
@@ -0,0 +1,119 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package tpr
import (
"k8s.io/client-go/1.5/pkg/api"
"k8s.io/client-go/1.5/pkg/watch"
"k8s.io/client-go/1.5/rest"
)
type (
FunctionInterface interface {
Create(*Function) (*Function, error)
Get(name string) (*Function, error)
Update(*Function) (*Function, error)
Delete(name string, options *api.DeleteOptions) error
List(opts api.ListOptions) (*FunctionList, error)
Watch(opts api.ListOptions) (watch.Interface, error)
}
functionClient struct {
client *rest.RESTClient
namespace string
}
)
func MakeFunctionInterface(tprClient *rest.RESTClient, namespace string) FunctionInterface {
return &functionClient{
client: tprClient,
namespace: namespace,
}
}
func (fc *functionClient) Create(f *Function) (*Function, error) {
var result Function
err := fc.client.Post().
Resource("functions").
Namespace(fc.namespace).
Body(f).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (fc *functionClient) Get(name string) (*Function, error) {
var result Function
err := fc.client.Get().
Resource("functions").
Namespace(fc.namespace).
Name(name).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (fc *functionClient) Update(f *Function) (*Function, error) {
var result Function
err := fc.client.Put().
Resource("functions").
Namespace(fc.namespace).
Name(f.Metadata.Name).
Body(f).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (fc *functionClient) Delete(name string, opts *api.DeleteOptions) error {
return fc.client.Delete().
Namespace(fc.namespace).
Resource("functions").
Name(name).
Body(opts).
Do().
Error()
}
func (fc *functionClient) List(opts api.ListOptions) (*FunctionList, error) {
var result FunctionList
err := fc.client.Get().
Namespace(fc.namespace).
Resource("functions").
VersionedParams(&opts, api.ParameterCodec).
Do().
Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (fc *functionClient) Watch(opts api.ListOptions) (watch.Interface, error) {
return fc.client.Get().
Prefix("watch").
Namespace(fc.namespace).
Resource("functions").
VersionedParams(&opts, api.ParameterCodec).
Watch()
}
+119
View File
@@ -0,0 +1,119 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package tpr
import (
"k8s.io/client-go/1.5/pkg/api"
"k8s.io/client-go/1.5/pkg/watch"
"k8s.io/client-go/1.5/rest"
)
type (
HttptriggerInterface interface {
Create(*Httptrigger) (*Httptrigger, error)
Get(name string) (*Httptrigger, error)
Update(*Httptrigger) (*Httptrigger, error)
Delete(name string, options *api.DeleteOptions) error
List(opts api.ListOptions) (*HttptriggerList, error)
Watch(opts api.ListOptions) (watch.Interface, error)
}
httpTriggerClient struct {
client *rest.RESTClient
namespace string
}
)
func MakeHttptriggerInterface(tprClient *rest.RESTClient, namespace string) HttptriggerInterface {
return &httpTriggerClient{
client: tprClient,
namespace: namespace,
}
}
func (c *httpTriggerClient) Create(obj *Httptrigger) (*Httptrigger, error) {
var result Httptrigger
err := c.client.Post().
Resource("httptriggers").
Namespace(c.namespace).
Body(obj).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (c *httpTriggerClient) Get(name string) (*Httptrigger, error) {
var result Httptrigger
err := c.client.Get().
Resource("httptriggers").
Namespace(c.namespace).
Name(name).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (c *httpTriggerClient) Update(obj *Httptrigger) (*Httptrigger, error) {
var result Httptrigger
err := c.client.Put().
Resource("httptriggers").
Namespace(c.namespace).
Name(obj.Metadata.Name).
Body(obj).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (c *httpTriggerClient) Delete(name string, opts *api.DeleteOptions) error {
return c.client.Delete().
Namespace(c.namespace).
Resource("httptriggers").
Name(name).
Body(opts).
Do().
Error()
}
func (c *httpTriggerClient) List(opts api.ListOptions) (*HttptriggerList, error) {
var result HttptriggerList
err := c.client.Get().
Namespace(c.namespace).
Resource("httptriggers").
VersionedParams(&opts, api.ParameterCodec).
Do().
Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (c *httpTriggerClient) Watch(opts api.ListOptions) (watch.Interface, error) {
return c.client.Get().
Prefix("watch").
Namespace(c.namespace).
Resource("httptriggers").
VersionedParams(&opts, api.ParameterCodec).
Watch()
}
+33
View File
@@ -0,0 +1,33 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package tpr
import (
"fmt"
"k8s.io/client-go/1.5/pkg/api"
)
// Given metadata, create a key that uniquely identifies the contents
// of the object. Since resourceVersion changes on every update and
// UIDs are unique, uid+resourceVersion identifies the
// content. (ResourceVersion may also update on status updates, so
// this will result in some unnecessary cache misses. That should be
// ok.)
func CacheKey(metadata *api.ObjectMeta) string {
return fmt.Sprintf("%v_%v", metadata.UID, metadata.ResourceVersion)
}
+119
View File
@@ -0,0 +1,119 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package tpr
import (
"k8s.io/client-go/1.5/pkg/api"
"k8s.io/client-go/1.5/pkg/watch"
"k8s.io/client-go/1.5/rest"
)
type (
KuberneteswatchtriggerInterface interface {
Create(*Kuberneteswatchtrigger) (*Kuberneteswatchtrigger, error)
Get(name string) (*Kuberneteswatchtrigger, error)
Update(*Kuberneteswatchtrigger) (*Kuberneteswatchtrigger, error)
Delete(name string, options *api.DeleteOptions) error
List(opts api.ListOptions) (*KuberneteswatchtriggerList, error)
Watch(opts api.ListOptions) (watch.Interface, error)
}
kubernetesWatchTriggerClient struct {
client *rest.RESTClient
namespace string
}
)
func MakeKuberneteswatchtriggerInterface(tprClient *rest.RESTClient, namespace string) KuberneteswatchtriggerInterface {
return &kubernetesWatchTriggerClient{
client: tprClient,
namespace: namespace,
}
}
func (c *kubernetesWatchTriggerClient) Create(obj *Kuberneteswatchtrigger) (*Kuberneteswatchtrigger, error) {
var result Kuberneteswatchtrigger
err := c.client.Post().
Resource("kuberneteswatchtriggers").
Namespace(c.namespace).
Body(obj).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (c *kubernetesWatchTriggerClient) Get(name string) (*Kuberneteswatchtrigger, error) {
var result Kuberneteswatchtrigger
err := c.client.Get().
Resource("kuberneteswatchtriggers").
Namespace(c.namespace).
Name(name).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (c *kubernetesWatchTriggerClient) Update(obj *Kuberneteswatchtrigger) (*Kuberneteswatchtrigger, error) {
var result Kuberneteswatchtrigger
err := c.client.Put().
Resource("kuberneteswatchtriggers").
Namespace(c.namespace).
Name(obj.Metadata.Name).
Body(obj).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (c *kubernetesWatchTriggerClient) Delete(name string, opts *api.DeleteOptions) error {
return c.client.Delete().
Namespace(c.namespace).
Resource("kuberneteswatchtriggers").
Name(name).
Body(opts).
Do().
Error()
}
func (c *kubernetesWatchTriggerClient) List(opts api.ListOptions) (*KuberneteswatchtriggerList, error) {
var result KuberneteswatchtriggerList
err := c.client.Get().
Namespace(c.namespace).
Resource("kuberneteswatchtriggers").
VersionedParams(&opts, api.ParameterCodec).
Do().
Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (c *kubernetesWatchTriggerClient) Watch(opts api.ListOptions) (watch.Interface, error) {
return c.client.Get().
Prefix("watch").
Namespace(c.namespace).
Resource("kuberneteswatchtriggers").
VersionedParams(&opts, api.ParameterCodec).
Watch()
}
+119
View File
@@ -0,0 +1,119 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package tpr
import (
"k8s.io/client-go/1.5/pkg/api"
"k8s.io/client-go/1.5/pkg/watch"
"k8s.io/client-go/1.5/rest"
)
type (
MessagequeuetriggerInterface interface {
Create(*Messagequeuetrigger) (*Messagequeuetrigger, error)
Get(name string) (*Messagequeuetrigger, error)
Update(*Messagequeuetrigger) (*Messagequeuetrigger, error)
Delete(name string, options *api.DeleteOptions) error
List(opts api.ListOptions) (*MessagequeuetriggerList, error)
Watch(opts api.ListOptions) (watch.Interface, error)
}
messagequeuetriggerClient struct {
client *rest.RESTClient
namespace string
}
)
func MakeMessagequeuetriggerInterface(tprClient *rest.RESTClient, namespace string) MessagequeuetriggerInterface {
return &messagequeuetriggerClient{
client: tprClient,
namespace: namespace,
}
}
func (fc *messagequeuetriggerClient) Create(f *Messagequeuetrigger) (*Messagequeuetrigger, error) {
var result Messagequeuetrigger
err := fc.client.Post().
Resource("messagequeuetriggers").
Namespace(fc.namespace).
Body(f).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (fc *messagequeuetriggerClient) Get(name string) (*Messagequeuetrigger, error) {
var result Messagequeuetrigger
err := fc.client.Get().
Resource("messagequeuetriggers").
Namespace(fc.namespace).
Name(name).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (fc *messagequeuetriggerClient) Update(f *Messagequeuetrigger) (*Messagequeuetrigger, error) {
var result Messagequeuetrigger
err := fc.client.Put().
Resource("messagequeuetriggers").
Namespace(fc.namespace).
Name(f.Metadata.Name).
Body(f).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (fc *messagequeuetriggerClient) Delete(name string, opts *api.DeleteOptions) error {
return fc.client.Delete().
Namespace(fc.namespace).
Resource("messagequeuetriggers").
Name(name).
Body(opts).
Do().
Error()
}
func (fc *messagequeuetriggerClient) List(opts api.ListOptions) (*MessagequeuetriggerList, error) {
var result MessagequeuetriggerList
err := fc.client.Get().
Namespace(fc.namespace).
Resource("messagequeuetriggers").
VersionedParams(&opts, api.ParameterCodec).
Do().
Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (fc *messagequeuetriggerClient) Watch(opts api.ListOptions) (watch.Interface, error) {
return fc.client.Get().
Prefix("watch").
Namespace(fc.namespace).
Resource("messagequeuetriggers").
VersionedParams(&opts, api.ParameterCodec).
Watch()
}
+119
View File
@@ -0,0 +1,119 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package tpr
import (
"k8s.io/client-go/1.5/pkg/api"
"k8s.io/client-go/1.5/pkg/watch"
"k8s.io/client-go/1.5/rest"
)
type (
TimetriggerInterface interface {
Create(*Timetrigger) (*Timetrigger, error)
Get(name string) (*Timetrigger, error)
Update(*Timetrigger) (*Timetrigger, error)
Delete(name string, options *api.DeleteOptions) error
List(opts api.ListOptions) (*TimetriggerList, error)
Watch(opts api.ListOptions) (watch.Interface, error)
}
timetriggerClient struct {
client *rest.RESTClient
namespace string
}
)
func MakeTimetriggerInterface(tprClient *rest.RESTClient, namespace string) TimetriggerInterface {
return &timetriggerClient{
client: tprClient,
namespace: namespace,
}
}
func (fc *timetriggerClient) Create(f *Timetrigger) (*Timetrigger, error) {
var result Timetrigger
err := fc.client.Post().
Resource("timetriggers").
Namespace(fc.namespace).
Body(f).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (fc *timetriggerClient) Get(name string) (*Timetrigger, error) {
var result Timetrigger
err := fc.client.Get().
Resource("timetriggers").
Namespace(fc.namespace).
Name(name).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (fc *timetriggerClient) Update(f *Timetrigger) (*Timetrigger, error) {
var result Timetrigger
err := fc.client.Put().
Resource("timetriggers").
Namespace(fc.namespace).
Name(f.Metadata.Name).
Body(f).
Do().Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (fc *timetriggerClient) Delete(name string, opts *api.DeleteOptions) error {
return fc.client.Delete().
Namespace(fc.namespace).
Resource("timetriggers").
Name(name).
Body(opts).
Do().
Error()
}
func (fc *timetriggerClient) List(opts api.ListOptions) (*TimetriggerList, error) {
var result TimetriggerList
err := fc.client.Get().
Namespace(fc.namespace).
Resource("timetriggers").
VersionedParams(&opts, api.ParameterCodec).
Do().
Into(&result)
if err != nil {
return nil, err
}
return &result, nil
}
func (fc *timetriggerClient) Watch(opts api.ListOptions) (watch.Interface, error) {
return fc.client.Get().
Prefix("watch").
Namespace(fc.namespace).
Resource("timetriggers").
VersionedParams(&opts, api.ParameterCodec).
Watch()
}
+106
View File
@@ -0,0 +1,106 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package tpr
import (
"k8s.io/client-go/1.5/kubernetes"
"k8s.io/client-go/1.5/pkg/api/errors"
"k8s.io/client-go/1.5/pkg/api/v1"
"k8s.io/client-go/1.5/pkg/apis/extensions/v1beta1"
)
// ensureTPR checks if the given TPR type exists, and creates it if
// needed. (Note that this creates the TPR type; it doesn't create any
// _instances_ of that type.)
func ensureTPR(clientset *kubernetes.Clientset, tpr *v1beta1.ThirdPartyResource) error {
_, err := clientset.Extensions().ThirdPartyResources().Get(tpr.ObjectMeta.Name)
if err != nil {
if errors.IsNotFound(err) {
_, err := clientset.Extensions().ThirdPartyResources().Create(tpr)
if err != nil {
return err
}
}
}
return nil
}
func EnsureFissionTPRs(clientset *kubernetes.Clientset) error {
tprs := []v1beta1.ThirdPartyResource{
{
ObjectMeta: v1.ObjectMeta{
Name: "function.fission.io",
},
Versions: []v1beta1.APIVersion{
{Name: "v1"},
},
Description: "Functions",
},
{
ObjectMeta: v1.ObjectMeta{
Name: "environment.fission.io",
},
Versions: []v1beta1.APIVersion{
{Name: "v1"},
},
Description: "Environments (function containers)",
},
{
ObjectMeta: v1.ObjectMeta{
Name: "httptrigger.fission.io",
},
Versions: []v1beta1.APIVersion{
{Name: "v1"},
},
Description: "HTTP triggers for functions",
},
{
ObjectMeta: v1.ObjectMeta{
Name: "kuberneteswatchtrigger.fission.io",
},
Versions: []v1beta1.APIVersion{
{Name: "v1"},
},
Description: "Kubernetes watch triggers for functions",
},
{
ObjectMeta: v1.ObjectMeta{
Name: "timetrigger.fission.io",
},
Versions: []v1beta1.APIVersion{
{Name: "v1"},
},
Description: "Time-based triggers for functions",
},
{
ObjectMeta: v1.ObjectMeta{
Name: "messagequeuetrigger.fission.io",
},
Versions: []v1beta1.APIVersion{
{Name: "v1"},
},
Description: "Messqge queue triggers for functions",
},
}
for _, tpr := range tprs {
err := ensureTPR(clientset, &tpr)
if err != nil {
return err
}
}
return nil
}
+429
View File
@@ -0,0 +1,429 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package tpr
import (
"log"
"os"
"testing"
"time"
"k8s.io/client-go/1.5/pkg/api"
"k8s.io/client-go/1.5/pkg/api/unversioned"
"k8s.io/client-go/1.5/rest"
"github.com/fission/fission"
)
func panicIf(err error) {
if err != nil {
log.Panicf("err: %v", err)
}
}
func functionTests(tprClient *rest.RESTClient) {
// sample function object
function := &Function{
TypeMeta: unversioned.TypeMeta{
Kind: "Function",
APIVersion: "fission.io/v1",
},
Metadata: api.ObjectMeta{
Name: "hello",
},
Spec: fission.FunctionSpec{
Source: fission.Package{},
Deployment: fission.Package{
Literal: []byte("hi"),
},
EnvironmentName: "xxx",
},
}
// Test function CRUD
fi := MakeFunctionInterface(tprClient, api.NamespaceDefault)
// cleanup from old crashed tests, ignore errors
fi.Delete(function.Metadata.Name, nil)
// create
f, err := fi.Create(function)
panicIf(err)
if f.Metadata.Name != function.Metadata.Name {
log.Panicf("Bad result from create: %v", f)
}
// read
f, err = fi.Get(function.Metadata.Name)
panicIf(err)
if len(f.Spec.Deployment.Literal) != len(function.Spec.Deployment.Literal) {
log.Panicf("Bad result from Get: %v", f)
}
log.Printf("f.Metadata = %#v", f.Metadata)
// update
function.Spec.EnvironmentName = "yyy"
f, err = fi.Update(function)
panicIf(err)
log.Printf("f.Metadata = %#v", f.Metadata)
// list
fl, err := fi.List(api.ListOptions{})
panicIf(err)
if len(fl.Items) != 1 {
log.Panicf("wrong count from list: %v", fl)
}
if fl.Items[0].Spec.EnvironmentName != function.Spec.EnvironmentName {
log.Panicf("bad object from list: %v", fl.Items[0])
}
// delete
err = fi.Delete(f.Metadata.Name, nil)
panicIf(err)
// start a watch
wi, err := fi.Watch(api.ListOptions{})
panicIf(err)
start := time.Now()
f, err = fi.Create(function)
panicIf(err)
defer fi.Delete(f.Metadata.Name, nil)
// assert that we get a watch event for the new function
recvd := false
select {
case <-time.NewTimer(1 * time.Second).C:
if !recvd {
log.Panicf("Didn't get watch event")
}
case ev := <-wi.ResultChan():
wf, ok := ev.Object.(*Function)
if !ok {
log.Panicf("Can't cast to Function")
}
if wf.Spec.EnvironmentName != function.Spec.EnvironmentName {
log.Panicf("Bad object from watch: %#v", wf)
}
log.Printf("watch event took %v", time.Now().Sub(start))
recvd = true
}
}
func environmentTests(tprClient *rest.RESTClient) {
// sample environment object
environment := &Environment{
TypeMeta: unversioned.TypeMeta{
Kind: "Environment",
APIVersion: "fission.io/v1",
},
Metadata: api.ObjectMeta{
Name: "hello",
},
Spec: fission.EnvironmentSpec{
Runtime: fission.Runtime{
Image: "xxx",
},
Builder: fission.Builder{
Image: "yyy",
Command: "zzz",
},
},
}
// Test environment CRUD
ei := MakeEnvironmentInterface(tprClient, api.NamespaceDefault)
// cleanup from old crashed tests, ignore errors
ei.Delete(environment.Metadata.Name, nil)
// create
e, err := ei.Create(environment)
panicIf(err)
if e.Metadata.Name != environment.Metadata.Name {
log.Panicf("Bad result from create: %v", e)
}
// read
e, err = ei.Get(environment.Metadata.Name)
panicIf(err)
if len(e.Spec.Runtime.Image) != len(environment.Spec.Runtime.Image) {
log.Panicf("Bad result from Get: %#v", e)
}
// update
environment.Spec.Runtime.Image = "www"
e, err = ei.Update(environment)
panicIf(err)
// list
el, err := ei.List(api.ListOptions{})
panicIf(err)
if len(el.Items) != 1 {
log.Panicf("wrong count from list: %v", el)
}
if el.Items[0].Spec.Runtime.Image != environment.Spec.Runtime.Image {
log.Panicf("bad object from list: %v", el.Items[0])
}
// delete
err = ei.Delete(e.Metadata.Name, nil)
panicIf(err)
// start a watch
wi, err := ei.Watch(api.ListOptions{})
panicIf(err)
start := time.Now()
e, err = ei.Create(environment)
panicIf(err)
defer ei.Delete(e.Metadata.Name, nil)
// assert that we get a watch event for the new environment
recvd := false
select {
case <-time.NewTimer(1 * time.Second).C:
if !recvd {
log.Panicf("Didn't get watch event")
}
case ev := <-wi.ResultChan():
obj, ok := ev.Object.(*Environment)
if !ok {
log.Panicf("Can't cast to Environment")
}
if obj.Spec.Runtime.Image != environment.Spec.Runtime.Image {
log.Panicf("Bad object from watch: %#v", obj)
}
log.Printf("watch event took %v", time.Now().Sub(start))
recvd = true
}
}
func httpTriggerTests(tprClient *rest.RESTClient) {
// sample httpTrigger object
httpTrigger := &Httptrigger{
TypeMeta: unversioned.TypeMeta{
Kind: "Httptrigger",
APIVersion: "fission.io/v1",
},
Metadata: api.ObjectMeta{
Name: "hello",
},
Spec: fission.HTTPTriggerSpec{
RelativeURL: "/hi",
Method: "GET",
FunctionReference: fission.FunctionReference{
Type: fission.FunctionReferenceTypeFunctionName,
Name: "hello",
},
},
}
// Test httpTrigger CRUD
ei := MakeHttptriggerInterface(tprClient, api.NamespaceDefault)
// cleanup from old crashed tests, ignore errors
ei.Delete(httpTrigger.Metadata.Name, nil)
// create
e, err := ei.Create(httpTrigger)
panicIf(err)
if e.Metadata.Name != httpTrigger.Metadata.Name {
log.Panicf("Bad result from create: %v", e)
}
// read
e, err = ei.Get(httpTrigger.Metadata.Name)
panicIf(err)
if len(e.Spec.Method) != len(httpTrigger.Spec.Method) {
log.Panicf("Bad result from Get: %#v", e)
}
// update
httpTrigger.Spec.Method = "POST"
e, err = ei.Update(httpTrigger)
panicIf(err)
// list
el, err := ei.List(api.ListOptions{})
panicIf(err)
if len(el.Items) != 1 {
log.Panicf("wrong count from list: %v", el)
}
if el.Items[0].Spec.Method != httpTrigger.Spec.Method {
log.Panicf("bad object from list: %v", el.Items[0])
}
// delete
err = ei.Delete(e.Metadata.Name, nil)
panicIf(err)
// start a watch
wi, err := ei.Watch(api.ListOptions{})
panicIf(err)
start := time.Now()
e, err = ei.Create(httpTrigger)
panicIf(err)
defer ei.Delete(e.Metadata.Name, nil)
// assert that we get a watch event for the new httpTrigger
recvd := false
select {
case <-time.NewTimer(1 * time.Second).C:
if !recvd {
log.Panicf("Didn't get watch event")
}
case ev := <-wi.ResultChan():
obj, ok := ev.Object.(*Httptrigger)
if !ok {
log.Panicf("Can't cast to Httptrigger")
}
if obj.Spec.Method != httpTrigger.Spec.Method {
log.Panicf("Bad object from watch: %#v", obj)
}
log.Printf("watch event took %v", time.Now().Sub(start))
recvd = true
}
}
func kubernetesWatchTriggerTests(tprClient *rest.RESTClient) {
// sample kubernetesWatchTrigger object
kubernetesWatchTrigger := &Kuberneteswatchtrigger{
TypeMeta: unversioned.TypeMeta{
Kind: "Kuberneteswatchtrigger",
APIVersion: "fission.io/v1",
},
Metadata: api.ObjectMeta{
Name: "hello",
},
Spec: fission.KubernetesWatchTriggerSpec{
Namespace: "foo",
Type: "pod",
LabelSelector: map[string]string{
"x": "y",
},
FunctionReference: fission.FunctionReference{
Type: fission.FunctionReferenceTypeFunctionName,
Name: "foo",
},
},
}
// Test kubernetesWatchTrigger CRUD
ei := MakeKuberneteswatchtriggerInterface(tprClient, api.NamespaceDefault)
// cleanup from old crashed tests, ignore errors
ei.Delete(kubernetesWatchTrigger.Metadata.Name, nil)
// create
e, err := ei.Create(kubernetesWatchTrigger)
panicIf(err)
if e.Metadata.Name != kubernetesWatchTrigger.Metadata.Name {
log.Panicf("Bad result from create: %v", e)
}
// read
e, err = ei.Get(kubernetesWatchTrigger.Metadata.Name)
panicIf(err)
if e.Spec.Type != kubernetesWatchTrigger.Spec.Type {
log.Panicf("Bad result from Get: %#v", e)
}
// update
kubernetesWatchTrigger.Spec.Type = "service"
e, err = ei.Update(kubernetesWatchTrigger)
panicIf(err)
// list
el, err := ei.List(api.ListOptions{})
panicIf(err)
if len(el.Items) != 1 {
log.Panicf("wrong count from list: %v", el)
}
if el.Items[0].Spec.Type != kubernetesWatchTrigger.Spec.Type {
log.Panicf("bad object from list: %v", el.Items[0])
}
// delete
err = ei.Delete(e.Metadata.Name, nil)
panicIf(err)
// start a watch
wi, err := ei.Watch(api.ListOptions{})
panicIf(err)
start := time.Now()
e, err = ei.Create(kubernetesWatchTrigger)
panicIf(err)
defer ei.Delete(e.Metadata.Name, nil)
// assert that we get a watch event for the new kubernetesWatchTrigger
recvd := false
select {
case <-time.NewTimer(1 * time.Second).C:
if !recvd {
log.Panicf("Didn't get watch event")
}
case ev := <-wi.ResultChan():
obj, ok := ev.Object.(*Kuberneteswatchtrigger)
if !ok {
log.Panicf("Can't cast to Kuberneteswatchtrigger")
}
if obj.Spec.Type != kubernetesWatchTrigger.Spec.Type {
log.Panicf("Bad object from watch: %#v", obj)
}
log.Printf("watch event took %v", time.Now().Sub(start))
recvd = true
}
}
func TestTpr(t *testing.T) {
// skip test if no cluster available for testing
kubeconfig := os.Getenv("KUBECONFIG")
if len(kubeconfig) == 0 {
log.Println("Skipping test, no kubernetes cluster")
return
}
// Create the client config. Needs the KUBECONFIG env var to
// point at a valid kubeconfig.
config, clientset, err := GetKubernetesClient()
panicIf(err)
// init our types
err = EnsureFissionTPRs(clientset)
panicIf(err)
// rest client with knowledge about our tpr types
tprClient, err := GetTprClient(config)
panicIf(err)
err = waitForTPRs(tprClient)
panicIf(err)
functionTests(tprClient)
environmentTests(tprClient)
httpTriggerTests(tprClient)
kubernetesWatchTriggerTests(tprClient)
}
+208
View File
@@ -0,0 +1,208 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package tpr
import (
"k8s.io/client-go/1.5/pkg/api"
"k8s.io/client-go/1.5/pkg/api/meta"
"k8s.io/client-go/1.5/pkg/api/unversioned"
"github.com/fission/fission"
)
//
// To add a Fission TPR type:
// 1. Create a "spec" type, for everything in the type except metadata
// 2. Create the type with metadata + the spec
// 3. Create a list type (for example see FunctionList and Function, below)
// 4. Add methods at the bottom of this file for satisfying Object and List interfaces
// 5. Add the type to configureClient in client.go
// 6. Add the type to EnsureFissionTPRs in tpr.go
// 7. Add tests to tpr_test.go
//
type (
// Functions.
Function struct {
unversioned.TypeMeta `json:",inline"`
Metadata api.ObjectMeta `json:"metadata"`
Spec fission.FunctionSpec `json:"spec"`
}
FunctionList struct {
unversioned.TypeMeta `json:",inline"`
Metadata unversioned.ListMeta `json:"metadata"`
Items []Function `json:"items"`
}
// Environments.
Environment struct {
unversioned.TypeMeta `json:",inline"`
Metadata api.ObjectMeta `json:"metadata"`
Spec fission.EnvironmentSpec `json:"spec"`
}
EnvironmentList struct {
unversioned.TypeMeta `json:",inline"`
Metadata unversioned.ListMeta `json:"metadata"`
Items []Environment `json:"items"`
}
// HTTP Triggers. (Something in the TPR reflection stuff wants
// it to be spelled "Httptrigger" not "HTTPTrigger" or even
// "HttpTrigger". Bleh.)
Httptrigger struct {
unversioned.TypeMeta `json:",inline"`
Metadata api.ObjectMeta `json:"metadata"`
Spec fission.HTTPTriggerSpec `json:"spec"`
}
HttptriggerList struct {
unversioned.TypeMeta `json:",inline"`
Metadata unversioned.ListMeta `json:"metadata"`
Items []Httptrigger `json:"items"`
}
// Kubernetes Watches as triggers
Kuberneteswatchtrigger struct {
unversioned.TypeMeta `json:",inline"`
Metadata api.ObjectMeta `json:"metadata"`
Spec fission.KubernetesWatchTriggerSpec `json:"spec"`
}
KuberneteswatchtriggerList struct {
unversioned.TypeMeta `json:",inline"`
Metadata unversioned.ListMeta `json:"metadata"`
Items []Kuberneteswatchtrigger `json:"items"`
}
// Time triggers
Timetrigger struct {
unversioned.TypeMeta `json:",inline"`
Metadata api.ObjectMeta `json:"metadata"`
Spec fission.TimeTriggerSpec `json:"spec"`
}
TimetriggerList struct {
unversioned.TypeMeta `json:",inline"`
Metadata unversioned.ListMeta `json:"metadata"`
Items []Timetrigger `json:"items"`
}
// Message Queue triggers
Messagequeuetrigger struct {
unversioned.TypeMeta `json:",inline"`
Metadata api.ObjectMeta `json:"metadata"`
Spec fission.MessageQueueTriggerSpec `json:"spec"`
}
MessagequeuetriggerList struct {
unversioned.TypeMeta `json:",inline"`
Metadata unversioned.ListMeta `json:"metadata"`
Items []Messagequeuetrigger `json:"items"`
}
)
// Each TPR type needs:
// GetObjectKind (to satisfy the Object interface)
//
// In addition, each singular TPR type needs:
// GetObjectMeta (to satisfy the ObjectMetaAccessor interface)
//
// And each list TPR type needs:
// GetListMeta (to satisfy the ListMetaAccessor interface)
func (f *Function) GetObjectKind() unversioned.ObjectKind {
return &f.TypeMeta
}
func (e *Environment) GetObjectKind() unversioned.ObjectKind {
return &e.TypeMeta
}
func (ht *Httptrigger) GetObjectKind() unversioned.ObjectKind {
return &ht.TypeMeta
}
func (w *Kuberneteswatchtrigger) GetObjectKind() unversioned.ObjectKind {
return &w.TypeMeta
}
func (w *Timetrigger) GetObjectKind() unversioned.ObjectKind {
return &w.TypeMeta
}
func (w *Messagequeuetrigger) GetObjectKind() unversioned.ObjectKind {
return &w.TypeMeta
}
func (f *Function) GetObjectMeta() meta.Object {
return &f.Metadata
}
func (e *Environment) GetObjectMeta() meta.Object {
return &e.Metadata
}
func (ht *Httptrigger) GetObjectMeta() meta.Object {
return &ht.Metadata
}
func (w *Kuberneteswatchtrigger) GetObjectMeta() meta.Object {
return &w.Metadata
}
func (w *Timetrigger) GetObjectMeta() meta.Object {
return &w.Metadata
}
func (w *Messagequeuetrigger) GetObjectMeta() meta.Object {
return &w.Metadata
}
func (fl *FunctionList) GetObjectKind() unversioned.ObjectKind {
return &fl.TypeMeta
}
func (el *EnvironmentList) GetObjectKind() unversioned.ObjectKind {
return &el.TypeMeta
}
func (hl *HttptriggerList) GetObjectKind() unversioned.ObjectKind {
return &hl.TypeMeta
}
func (wl *KuberneteswatchtriggerList) GetObjectKind() unversioned.ObjectKind {
return &wl.TypeMeta
}
func (wl *TimetriggerList) GetObjectKind() unversioned.ObjectKind {
return &wl.TypeMeta
}
func (wl *MessagequeuetriggerList) GetObjectKind() unversioned.ObjectKind {
return &wl.TypeMeta
}
func (fl *FunctionList) GetListMeta() unversioned.List {
return &fl.Metadata
}
func (el *EnvironmentList) GetListMeta() unversioned.List {
return &el.Metadata
}
func (hl *HttptriggerList) GetListMeta() unversioned.List {
return &hl.Metadata
}
func (wl *KuberneteswatchtriggerList) GetListMeta() unversioned.List {
return &wl.Metadata
}
func (wl *TimetriggerList) GetListMeta() unversioned.List {
return &wl.Metadata
}
func (wl *MessagequeuetriggerList) GetListMeta() unversioned.List {
return &wl.Metadata
}
// In the client-go TPR example, UnmarshalJSON is defined here for the
// singular and list types. That's supposed to be a workaround for
// some ugorji bug. But we don't seem to need it, and all our tests
// pass without it, so we don't define any UnmarshalJSON methods.