Nail/fission-src
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Extensible Fission CLI (#743)

Browse Source 
This PR introduces extensibility to the Fission CLI. The principle behind the design of this extensibility is taken from the approach git uses: the main binary (fission) calls other binaries (fission-workflows) - which it discovers using simple prefix-rules - based on specific commands (fission workflow <command>).
This commit is contained in:
Erwin van Eyk
2018-08-17 15:20:46 +08:00
committed by Ta-Ching Chen
parent ad28922871
commit 5c1d8bc90c
7 changed files with 606 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Documentation/wip/cli-extensibility.md
+128
View File
@@ -0,0 +1,128 @@
# Fission CLI Extensibility
### Approach
To sum up the approach: git-style plugins.
Plugins are named with the fission prefix `fission-*`. When fission is invoked with an undefined/non-core subcommand
is called (like `fission foo`). Fission will look in the PATH
**Installing Fission**
```bash
# The same as before
$ curl -Lo fission https://github.com/fission/fission/releases/download/0.7.2/fission-cli-osx && chmod +x fission && sudo mv fission /usr/local/bin/
```
**Installing Fission Workflows**
```bash
# The same process as Fission cli itself
$ curl -Lo fission https://github.com/fission/fission-workflows/releases/download/0.4.0/fission-workflows-osx && chmod +x fission-workflows && sudo mv fission-workflows /usr/local/bin/
```
**Invoking Fission Workflows**
```bash
$ fission workflows invocation get b1278e802a
# Which is equivalent to:
$ fission-workflows invocation get b1278e802a
```
General flow:
1. fission does not recognize `workflows` subcommand
2. fission checks the path for a binary called `fission-workflows`
3. fission finds the binary.
4. fission invokes the `fission-workflows`, passing the remainder of the arguments.
**Discoverability: fission --help**
```bash
$ fission --help
USAGE:
fission [global options] command [command options] [arguments...]
VERSION:
0.6.0
COMMANDS:
function, fn Create, update and manage functions
httptrigger, ht, route Manage HTTP triggers (routes) for functions
timetrigger, tt, timer Manage Time triggers (timers) for functions
mqtrigger, mqt, messagequeue Manage message queue triggers for functions
environment, env Manage environments
watch, w Manage watches
package, pkg Manage packages
spec, specs Manage a declarative app specification
upgrade Upgrade tool from fission v0.1
tpr2crd Migrate tool for TPR to CRD
help, h Shows a list of commands or help for one command
PLUGINS:
workflows, wf Inspect and manage workflow executions
ui Start the user interface
GLOBAL OPTIONS:
--server value Fission server URL (default: "http://127.0.0.1:65356")
--help, -h show help
--version, -v print the version
```
Of course Fission needs to be able to find all plugins for this. There are several ways in which we can provide discoverability. The simplest one is for Fission to look in the path for all binaries starting with the `fission-*` prefix. Optionally, fission could invoke a specific command on the subcommand to get info about the plugin (such as version, help text, aliases...)
With Fission Workflows this info would look something like this:
```bash
$ fission-workflows --plugin
name: workflows
version: 0.4.0
help: Inspect and manage workflow executions
```
The idea is that this plugin info is all completely optional.
If it is not available, we simply degrade the results to user.
This way users/we can easily prototype or add plugins without having to worry about adhering to some interface.
**List version**
```bash
$ fission --version
client:
fission: 0.8.0
fission-workflows: 0.4.0
server:
fission: 0.8.1
fission-workflows: 0.3.0
```
Again, versioning info for fission-workflows is taken from the plugin info of the commands.
Note: a related issue is to have some more formalized plugin support/discoverability on the server-side,
but that is out of the scope of this issue.
### Other (optional) extensions and notes
- Like git we could setup a preferred binary path, where fission looks first when searching for the subcommand.
This could optionally be defined with a `FISSION_EXEC_PATH`.
- With the current approach we cannot have aliases for commands---fission will not be able to find fission-workflows
when the user calls `fission wf`. This might be UX issue, with these long path names. One option is let the user fix
it themselves by symlinking `fission-wf` to `fission-workflows`; using the plugin info Fission can recognize and
merge aliases together.
- To help detect versioning conflicts (old version of fission, too new version of fission workflows). We could add
a `requires` field to the fission-workflows plugin info. Then we could throw a warning or error, when two out of sync
versions are being used.
- To avoid unhelpful errors to the user when they have not installed a plugin, we could add a heuristic to check
`https://github.com/fission/SUBCOMMAND` to see if the subcommand might be an uninstalled plugin.
OR, we could lookup a simple text file that contains common plugins `https://github.com/fission/fission/plugins.txt`
and list them as suggestions to the user. OR we could of course just default to a bit help text that says something
like `unknown subcommand 'foo'. If this is a plugin, ensure that it is present on your PATH`.
---
### Motivation
The proposed approach is to use the git-based plugin system for now. Reasons for this approach over a sophisticated,
integrated plugin-based approach:
- It is low effort to implement.
- It is easy to extend with minimal to no required interface.
- The binaries remain standalone, allowing users to separate them if needed and make independent development on the
binaries easy.
Limitations of the proposed approach:
- The user still has to do some work, adding binaries to the PATH; ensuring that permissions are correct; ensuring
that the binary is executable; how to deal with duplicate binaries on the PATH. All this makes this approach assume
basic/intermediate knowledge of the OS from the user.
- I have to admit: I am not entirely sure if this approach requires any changes for Windows. Probably not.
- Upgrading fission with many plugins could be cumbersome, as you would need to upgrade each binary one by one.
Improving this is probably best left to future work.
The more heavyweight solution solves some of these limitations to an extent, but these do not way up to the increased
development and maintainance cost IMO. If needed we could explore this option (or some hybrid option) in the future.
fission/main.go
+114 -13
View File
@@ -20,12 +20,16 @@ import (
"fmt"
"os"
"path/filepath"
"strings"
"github.com/ghodss/yaml"
"github.com/urfave/cli"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/fission/fission"
"github.com/fission/fission/fission/log"
"github.com/fission/fission/fission/plugin"
"github.com/fission/fission/fission/portforward"
"github.com/urfave/cli"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func getFissionNamespace() string {
@@ -77,17 +81,15 @@ func main() {
app.Name = "fission"
app.Usage = "Serverless functions for Kubernetes"
app.Version = fission.Version
cli.VersionPrinter = func(c *cli.Context) {
clientVer := fission.BuildInfo().String()
fmt.Printf("Client Version: %v\n", clientVer)
serverInfo, err := getClient(getServerUrl()).ServerInfo()
if err != nil {
fmt.Printf("Error getting Fission API version: %v", err)
} else {
serverVer := serverInfo.Build.String()
fmt.Printf("Server Version: %v\n", serverVer)
cli.VersionPrinter = versionPrinter
app.CustomAppHelpTemplate = helpTemplate
app.ExtraInfo = func() map[string]string {
info := map[string]string{}
for _, pmd := range plugin.FindAll() {
names := strings.Join(append([]string{pmd.Name}, pmd.Aliases...), ", ")
info[names] = pmd.Usage
}
return info
}
app.Flags = []cli.Flag{
@@ -310,8 +312,107 @@ func main() {
{Name: "package", Aliases: []string{"pkg"}, Usage: "Manage packages", Subcommands: pkgSubCommands},
{Name: "spec", Aliases: []string{"specs"}, Usage: "Manage a declarative app specification", Subcommands: specSubCommands},
{Name: "upgrade", Aliases: []string{}, Usage: "Upgrade tool from fission v0.1", Subcommands: upgradeSubCommands},
cmdPlugin,
}
app.Before = cliHook
app.CommandNotFound = handleCommandNotFound
app.Run(os.Args)
}
func handleCommandNotFound(ctx *cli.Context, subCommand string) {
pmd, err := plugin.Find(subCommand)
if err != nil {
switch err {
case plugin.ErrPluginNotFound:
url, ok := plugin.SearchRegistries(subCommand)
if !ok {
log.Fatal("No help topic for '" + subCommand + "'")
}
log.Fatal(fmt.Sprintf(`Command '%v' is not installed.
It is available to download at '%v'.
To install it for your local Fission CLI:
1. Download the plugin binary for your OS from the URL
2. Ensure that the plugin binary is executable: chmod +x <binary>
2. Add the plugin binary to your $PATH: mv <binary> /usr/local/bin/fission-%v`, subCommand, url, subCommand))
default:
log.Fatal("Error occurred when invoking " + subCommand + ": " + err.Error())
}
os.Exit(1)
}
// Rebuild global arguments string (urfave/cli does not have an option to get the raw input of the global flags)
var globalArgs []string
for _, globalFlagName := range ctx.GlobalFlagNames() {
val := fmt.Sprintf("%v", ctx.GlobalGeneric(globalFlagName))
if len(val) > 0 {
globalArgs = append(globalArgs, fmt.Sprintf("--%v", globalFlagName), val)
}
}
args := append(globalArgs, ctx.Args().Tail()...)
err = plugin.Exec(pmd, args)
if err != nil {
os.Exit(1)
}
}
// Versions is a container of versions of the client (and its plugins) and server (and its plugins).
type Versions struct {
Client map[string]fission.BuildMeta `json:"client"`
Server map[string]fission.BuildMeta `json:"server"`
}
func versionPrinter(_ *cli.Context) {
serverInfo, err := getClient(getServerUrl()).ServerInfo()
if err != nil {
log.Warn(fmt.Sprintf("Error getting Fission API version: %v", err))
}
// Fetch client versions
versions := Versions{
Client: map[string]fission.BuildMeta{
"fission/core": fission.BuildInfo(),
},
}
for _, pmd := range plugin.FindAll() {
versions.Client[pmd.Name] = fission.BuildMeta{
Version: pmd.Version,
}
}
// Fetch server versions
versions.Server = map[string]fission.BuildMeta{
"fission/core": serverInfo.Build,
}
// FUTURE: fetch versions of plugins server-side
bs, err := yaml.Marshal(versions)
if err != nil {
log.Fatal("Failed to format versions: " + err.Error())
}
fmt.Print(string(bs))
}
var helpTemplate = `NAME:
{{.Name}}{{if .Usage}} - {{.Usage}}{{end}}
USAGE:
{{if .UsageText}}{{.UsageText}}{{else}}{{.HelpName}} {{if .VisibleFlags}}[global options]{{end}}{{if .Commands}} command [command options]{{end}} {{if .ArgsUsage}}{{.ArgsUsage}}{{else}}[arguments...]{{end}}{{end}}{{if .Version}}{{if not .HideVersion}}
VERSION:
{{.Version}}{{end}}{{end}}{{if .Description}}
DESCRIPTION:
{{.Description}}{{end}}{{if .VisibleCommands}}
COMMANDS:{{range .VisibleCategories}}{{if .Name}}
{{.Name}}:{{end}}{{range .VisibleCommands}}
{{join .Names ", "}}{{"\t"}}{{.Usage}}{{end}}{{end}}{{end}}{{if .VisibleFlags}}
PLUGIN COMMANDS:{{ range $name, $usage := ExtraInfo }}
{{$name}}{{"\t"}}{{$usage}}{{end}}
GLOBAL OPTIONS:
{{range $index, $option := .VisibleFlags}}{{if $index}}
{{end}}{{$option}}{{end}}{{end}}
`
fission/plugin.go
+50
View File
@@ -0,0 +1,50 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"fmt"
"os"
"text/tabwriter"
"github.com/urfave/cli"
"github.com/fission/fission/fission/plugin"
)
var cmdPlugin = cli.Command{
Name: "plugin",
Aliases: []string{"plugins"},
Usage: "Manage Fission CLI plugins",
Subcommands: []cli.Command{
{
Name: "list",
Usage: "List installed client plugins",
Action: pluginList,
},
},
}
func pluginList(_ *cli.Context) error {
w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
fmt.Fprintln(w, "NAME\tVERSION\tPATH")
for _, p := range plugin.FindAll() {
fmt.Fprintf(w, "%v\t%v\t%v\n", p.Name, p.Version, p.Path)
}
w.Flush()
return nil
}
fission/plugin/plugin.go
+176
View File
@@ -0,0 +1,176 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
// Package plugins provides support for creating extensible CLIs
package plugin
import (
"bytes"
"context"
"encoding/json"
"errors"
"io/ioutil"
"os"
"os/exec"
"path"
"strings"
"time"
)
const (
cmdTimeout = 5 * time.Second
cmdMetadataArgs = "--plugin"
)
var (
ErrPluginNotFound = errors.New("plugin not found")
ErrPluginInvalid = errors.New("invalid plugin")
Prefix = "fission-"
)
// Metadata contains the metadata of a plugin.
// The only metadata that is guaranteed to be non-empty is the path and Name. All other fields are considered optional.
type Metadata struct {
Name string `json:"name,omitempty"`
Version string `json:"version,omitempty"`
Aliases []string `json:"aliases,omitempty"`
Usage string `json:"usage,omitempty"`
Path string `json:"path,omitempty"`
}
func (md *Metadata) AddAlias(alias string) {
if alias != md.Name && !md.HasAlias(alias) {
md.Aliases = append(md.Aliases, alias)
}
}
func (md *Metadata) HasAlias(needle string) bool {
for _, alias := range md.Aliases {
if alias == needle {
return true
}
}
return false
}
// Find searches the machine for the given plugin, returning the metadata of the plugin.
// The only metadata that is guaranteed to be non-empty is the path and Name. All other fields are considered optional.
// If found it returns the plugin, otherwise it returns ErrPluginNotFound if the plugin was not found.
func Find(pluginName string) (*Metadata, error) {
// Search PATH for plugin as command-name
// To check if plugin is actually there still.
pluginPath, err := findPluginOnPath(pluginName)
if err != nil {
// Fallback: Search for alias in each command
mds := FindAll()
for _, md := range mds {
if md.HasAlias(pluginName) {
return md, nil
}
}
return nil, ErrPluginNotFound
}
md, err := fetchPluginMetadata(pluginPath)
if err != nil {
return nil, err
}
return md, nil
}
// Exec executes the plugin using the provided args.
// All input and output is redirected to stdin, stdout, and stderr.
func Exec(md *Metadata, args []string) error {
cmd := exec.Command(md.Path, args...)
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
return cmd.Run()
}
// FindAll searches the machine for all plugins currently present.
func FindAll() map[string]*Metadata {
plugins := map[string]*Metadata{}
dirs := strings.Split(os.Getenv("PATH"), ":")
for _, dir := range dirs {
fs, err := ioutil.ReadDir(dir)
if err != nil {
continue
}
for _, f := range fs {
if !strings.HasPrefix(f.Name(), Prefix) {
continue
}
fp := path.Join(dir, f.Name())
md, err := fetchPluginMetadata(fp)
if err != nil {
continue
}
if existing, ok := plugins[md.Name]; ok {
for _, alias := range existing.Aliases {
md.AddAlias(alias)
}
}
plugins[md.Name] = md
}
}
return plugins
}
func findPluginOnPath(pluginName string) (path string, err error) {
binaryName := Prefix + pluginName
path, err = exec.LookPath(binaryName)
if err != nil || len(path) == 0 {
return "", ErrPluginNotFound
}
return path, nil
}
// fetchPluginMetadata attempts to fetch the plugin metadata given the plugin path.
func fetchPluginMetadata(pluginPath string) (*Metadata, error) {
d, err := os.Stat(pluginPath)
if err != nil {
return nil, ErrPluginNotFound
}
if m := d.Mode(); m.IsDir() || m&0111 == 0 {
return nil, ErrPluginInvalid
}
// Fetch the metadata from the plugin itself.
buf := bytes.NewBuffer(nil)
ctx, cancel := context.WithTimeout(context.Background(), cmdTimeout)
defer cancel()
cmd := exec.CommandContext(ctx, pluginPath, cmdMetadataArgs) // Note: issue can occur with signal propagation
cmd.Stdout = buf
err = cmd.Run()
if err != nil {
return nil, err
}
// Parse metadata if possible
pluginName := strings.TrimPrefix(path.Base(pluginPath), Prefix)
md := &Metadata{}
err = json.Unmarshal(buf.Bytes(), md)
if err != nil {
md.Name = pluginName
}
md.Path = pluginPath
md.AddAlias(pluginName)
return md, nil
}
fission/plugin/plugin_test.go
+104
View File
@@ -0,0 +1,104 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package plugin
import (
"encoding/json"
"fmt"
"io/ioutil"
"os"
"path"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
func TestFind(t *testing.T) {
os.Clearenv()
testDir := path.Join(os.TempDir(), fmt.Sprintf("fission-test-plugins-%v", time.Now().UnixNano()))
err := os.MkdirAll(testDir, os.ModePerm)
if err != nil {
t.FailNow()
}
defer os.RemoveAll(testDir)
testBinary := path.Join(testDir, "foo")
md := &Metadata{
Name: "foo",
Version: "1.0.1",
Usage: "Usage help",
Aliases: []string{"bar"},
}
jsonMd, err := json.Marshal(md)
if err != nil {
t.FailNow()
}
err = ioutil.WriteFile(testBinary, []byte(fmt.Sprintf("#!/bin/sh\necho '%v'", string(jsonMd))), os.ModePerm)
if err != nil {
t.FailNow()
}
err = os.Setenv("PATH", testDir)
if err != nil {
t.FailNow()
}
Prefix = ""
found, err := Find(md.Name)
os.RemoveAll(testDir)
assert.NoError(t, err)
assert.NotEmpty(t, found)
assert.Equal(t, md.Name, found.Name)
assert.Equal(t, path.Join(testDir, md.Name), found.Path)
assert.Equal(t, md.Aliases, found.Aliases)
assert.Equal(t, md.Usage, found.Usage)
assert.Equal(t, md.Version, found.Version)
}
func TestExec(t *testing.T) {
os.Clearenv()
testDir := path.Join(os.TempDir(), fmt.Sprintf("fission-test-plugins-%v", time.Now().UnixNano()))
err := os.MkdirAll(testDir, os.ModePerm)
if err != nil {
t.FailNow()
}
defer os.RemoveAll(testDir)
testBinary := path.Join(testDir, "foo")
md := &Metadata{
Name: "foo",
Version: "1.0.1",
Usage: "Usage help",
Aliases: []string{"bar"},
Path: path.Join(testBinary),
}
jsonMd, err := json.Marshal(md)
if err != nil {
t.FailNow()
}
err = ioutil.WriteFile(testBinary, []byte(fmt.Sprintf("#!/bin/sh\necho '%v'", string(jsonMd))), os.ModePerm)
if err != nil {
t.FailNow()
}
err = os.Setenv("PATH", testDir)
if err != nil {
t.FailNow()
}
Prefix = ""
err = Exec(md, nil)
os.RemoveAll(testDir)
assert.NoError(t, err)
}
fission/plugin/registry.go
+29
View File
@@ -0,0 +1,29 @@
/*
Copyright 2016 The Fission Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package plugin
// builtinRegistry consists of a map of plugin names along with the relevant url.
var builtinRegistry = map[string]string{
"workflows": "https://github.com/fission/fission-workflows/releases",
}
// SearchRegistries will search (remote) registries for the presence of the command.
// For now we only use the builtinRegistry
func SearchRegistries(cmd string) (string, bool) {
url, ok := builtinRegistry[cmd]
return url, ok
}
plugins.yaml
+5
View File
@@ -0,0 +1,5 @@
# plugins.yaml contains a basic registry of plugins for the CLI
workflows:
version: 0.4.0
url: https://github.com/fission/fission-workflows/releases/tag/0.4.0
usage: Inspect and manage workflow executions