Merge pull request #54 from grafana/53-simplify-responsibility

simplify responsibility
grafana · Sep 3, 2024 · f5c3a2d · f5c3a2d
2 parents 7692fb3 + dea5c70
commit f5c3a2d
Show file tree

Hide file tree

Showing 39 changed files with 212 additions and 1,314 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -36,19 +36,18 @@ eget golangci/golangci-lint
 eget goreleaser/goreleaser
 ```
 
-## schema - Contribute to the JSON schema
+## schema - Generate code due to schema change
 
-The JSON schema of the registry can be found in the [registry.schema.yaml] file, after modification of which the schema in JSON format ([registry.schema.json]) and the golang data model ([registry_gen.go]) must be regenerated.
+After modifying registry schema ([registry.schema.json]), the [registry_gen.go] file must be regenerated.
 
 ```bash
-yq -o=json -P docs/registry.schema.yaml > docs/registry.schema.json
-go-jsonschema --capitalization URL --capitalization OSS -p k6registry --only-models -o registry_gen.go docs/registry.schema.yaml
+go-jsonschema --capitalization URL --capitalization OSS -p k6registry --only-models -o registry_gen.go docs/registry.schema.json
 ```
 
 [registry.schema.json]: docs/registry.schema.json
 [registry_gen.go]: registry_gen.go
 
-## example - Contribute to the example
+## example - Update documentation due to example changes
 
 The example registry can be found in [example.yaml] file, the documentation ([registry.md], [README.md]) must be updated after modification.
 
@@ -139,14 +138,14 @@ docker build -t k6registry -f Dockerfile.goreleaser .
 rm -rf build
 ```
 
-## all - Update everything
+### update - Update everything
 
 The most robust thing is to update everything (both the schema and the example) after modifying the source.
 
 Requires
 : schema, example, readme
 
-## legacy - Convert legacy registry
+### legacy - Convert legacy registry
 
 ```bash
  go run ./cmd/k6registry . --legacy | yq '.[]|= pick(["module","description","tier","products","imports","outputs","repo","categories"])|sort_by(.module)' > ./docs/legacy.yaml

diff --git a/README.md b/README.md
@@ -1,8 +1,14 @@
 <h1 name="title">k6registry</h1>
 
-**Data model and tooling for the k6 extension registry**
+**k6 Extension Registry/Catalog Generator**
 
-This repository contains the [JSON schema](docs/registry.schema.json) of the k6 extension registry and the [`k6registry`](#k6registry) command line tool for generating registry from source. The command line tool can also be used as a [GitHub Action](#github-action).
+k6registry is a CLI tool and a GitHub Action that enables the generation of the k6 extension registry. The generation source is a YAML (or JSON) file that contains the most important properties of extensions. The generator generates the missing properties from the repository metadata. Repository metadata is collected using the repository manager APIs. GitHub and GitLab APIs are currently supported.
+
+The generator also performs static analysis of extensions. The result of the analysis is the level of compliance with best practices (0-100%). A compliance grade (A-F) is calculated from the compliance level. The compliance level and grade are stored in the registry for each extension. Based on the compliance grade, an SVG compliance badge is created for each extension. Example badge:
+
+![xk6-sql](https://registry.k6.io/module/github.com/grafana/xk6-sql/badge.svg)
+
+The k6 Extension Catalog is an alternative representation of the k6 Extension Registry. The output of the generation can be in k6 Extension Catalog format. This format is optimized to resolve extensions as dependencies.
 
 Check [k6 Extension Registry Concept](docs/registry.md) for information on design considerations.
 
@@ -82,7 +88,7 @@ Registry generated from the source above.
       "name": "xk6-dashboard",
       "owner": "grafana",
       "public": true,
-      "stars": 327,
+      "stars": 328,
       "timestamp": 1719907965,
       "topics": [
         "xk6",
@@ -455,15 +461,7 @@ go install github.com/grafana/k6registry/cmd/k6registry@latest
 
 ## GitHub Action
 
-`grafana/k6registry` is a GitHub Action that enables k6 extension registry processing and the generation of customized JSON output for different applications. Processing is based on popular `jq` expressions using an embedded `jq` implementation.
-
-The jq filter expression can be specified in the `filter` parameter.
-
-The extension registry is read from the YAML format file specified in the `in` parameter.
-
-Repository metadata is collected using the repository manager APIs. Currently only the GitHub API is supported.
-
-The output of the processing will be written to the standard output by default. The output can be saved to a file using the `out` parameter.
+`grafana/k6registry` is a GitHub Action that enables k6 Extension Registry and Catalog generation.
 
 **Inputs**
 
@@ -472,10 +470,11 @@ name   | reqired | default | description
 in     |   yes   |         | input file name
 out    |    no   |  stdout | output file name
 api    |    no   |         | output directory name
-mute   |    no   | `false` | no output, only validation
+quiet  |    no   | `false` | no output, only validation
 loose  |    no   | `false` | skip JSON schema validation
 lint   |    no   | `false` | enable built-in linter
 compact|    no   | `false` | compact instead of pretty-printed output
+catalog|    no   | `false` | generate catalog instead of registry
 ref    |    no   |         | reference output URL for change detection
 
 In GitHub action mode, the change can be indicated by comparing the output to a reference output. The reference output URL can be passed in the `ref` action parameter. The `changed` output variable will be `true` or `false` depending on whether the output has changed or not compared to the reference output.
@@ -491,8 +490,8 @@ changed | `true` if the output has changed compared to `ref`, otherwise `false`
 **Example usage**
 
 ```yaml
-- name: Generate registry in JSON format
-  uses: grafana/[email protected].10
+- name: Generate extension registry
+  uses: grafana/[email protected].18
   with:
     in: "registry.yaml"
     out: "registry.json"
@@ -504,22 +503,23 @@ changed | `true` if the output has changed compared to `ref`, otherwise `false`
 <!-- #region cli -->
 ## k6registry
 
-k6 extension registry generator
+k6 Extension Registry/Catalog Generator
 
 ### Synopsis
 
-Command line k6 extension registry generator.
+Generate k6 extension registry/catalog from source.
 
-The source of the extension registry contains only the most important properties of the extensions. The rest of the properties are collected by k6registry using the API of the extensions' git repository managers.
+The generation source is a YAML (or JSON) file that contains the most important properties of extensions. The generator generates the missing properties from the repository metadata. Repository metadata is collected using the repository manager APIs. GitHub and GitLab APIs are currently supported.
 
-The source of the extension registry is read from the YAML (or JSON) format file specified as command line argument. If it is missing, the source is read from the standard input.
+The generator also performs static analysis of extensions. The result of the analysis is the level of compliance with best practices (0-100%). A compliance grade (A-F) is calculated from the compliance level. The compliance level and grade are stored in the registry for each extension. Based on the compliance grade, an SVG compliance badge is created for each extension. 
 
-Repository metadata is collected using the API of the extensions' git repository managers. Currently only the GitHub API is supported.
+The source is read from file specified as command line argument. If it is missing, the source is read from the standard input.
 
 The output of the generation will be written to the standard output by default. The output can be saved to a file using the `-o/--out` flag.
 
 The `--api` flag can be used to specify a directory to which the outputs will be written. The `registry.json` file is placed in the root directory. The `extension.json` file and the `badge.svg` file (if the `--lint` flag is used) are placed in a directory with the same name as the extension's go module path.
 
+
 ```
 k6registry [flags] [source-file]
 ```
@@ -551,9 +551,7 @@ By using the `--api` flag, files are created with relative paths in a base direc
 ```ascii file=docs/example-api.txt
 docs/example-api
 ├── catalog.json
-├── openapi.yaml
 ├── registry.json
-├── registry.schema.json
 ├── category
 │   ├── authentication.json
 │   ├── browser.json

diff --git a/action.yml b/action.yml
@@ -19,7 +19,7 @@ inputs:
     description: api directory name
     required: false
 
-  mute:
+  quiet:
     description: no output, only validation
     required: false
 
@@ -35,6 +35,10 @@ inputs:
     description: compact instead of pretty-printed output
     required: false
 
+  catalog:
+    description: generate catalog instead of registry
+    required: false
+
   ref:
     description: reference output URL for change detection
     required: false

diff --git a/cmd/api.go b/cmd/api.go
@@ -46,14 +46,6 @@ func writeJSON(filename string, source interface{}) error {
 }
 
 func writeAPIGroupGlobal(registry k6registry.Registry, target string) error {
-	if err := writeData(filepath.Join(target, "registry.schema.json"), k6registry.Schema); err != nil {
-		return err
-	}
-
-	if err := writeData(filepath.Join(target, "openapi.yaml"), k6registry.OpenAPI); err != nil {
-		return err
-	}
-
 	if err := writeJSON(filepath.Join(target, "registry.json"), registry); err != nil {
 		return err
 	}

diff --git a/cmd/cmd.go b/cmd/cmd.go
@@ -35,7 +35,7 @@ func New() (*cobra.Command, error) {
 
 	root := &cobra.Command{
 		Use:               "k6registry [flags] [source-file]",
-		Short:             "k6 extension registry generator",
+		Short:             "k6 Extension Registry/Catalog Generator",
 		Long:              help,
 		SilenceUsage:      true,
 		SilenceErrors:     true,

diff --git a/cmd/help.md b/cmd/help.md
@@ -1,11 +1,11 @@
-Command line k6 extension registry generator.
+Generate k6 extension registry/catalog from source.
 
-The source of the extension registry contains only the most important properties of the extensions. The rest of the properties are collected by k6registry using the API of the extensions' git repository managers.
+The generation source is a YAML (or JSON) file that contains the most important properties of extensions. The generator generates the missing properties from the repository metadata. Repository metadata is collected using the repository manager APIs. GitHub and GitLab APIs are currently supported.
 
-The source of the extension registry is read from the YAML (or JSON) format file specified as command line argument. If it is missing, the source is read from the standard input.
+The generator also performs static analysis of extensions. The result of the analysis is the level of compliance with best practices (0-100%). A compliance grade (A-F) is calculated from the compliance level. The compliance level and grade are stored in the registry for each extension. Based on the compliance grade, an SVG compliance badge is created for each extension. 
 
-Repository metadata is collected using the API of the extensions' git repository managers. Currently only the GitHub API is supported.
+The source is read from file specified as command line argument. If it is missing, the source is read from the standard input.
 
 The output of the generation will be written to the standard output by default. The output can be saved to a file using the `-o/--out` flag.
 
-The `--api` flag can be used to specify a directory to which the outputs will be written. The `registry.json` file is placed in the root directory. The `extension.json` file and the `badge.svg` file (if the `--lint` flag is used) are placed in a directory with the same name as the extension's go module path.
+The `--api` flag can be used to specify a directory to which the outputs will be written. The `registry.json` file is placed in the root directory. The `extension.json` file and the `badge.svg` file (if the `--lint` flag is used) are placed in a directory with the same name as the extension's go module path.
diff --git a/cmd/k6registry/main.go b/cmd/k6registry/main.go
@@ -55,8 +55,8 @@ func getArgs() []string {
 
 	var args []string
 
-	if getenv("INPUT_MUTE", "false") == "true" {
-		args = append(args, "--mute")
+	if getenv("INPUT_QUIET", "false") == "true" {
+		args = append(args, "--quiet")
 	}
 
 	if getenv("INPUT_LOOSE", "false") == "true" {
@@ -71,6 +71,10 @@ func getArgs() []string {
 		args = append(args, "--compact")
 	}
 
+	if getenv("INPUT_CATALOG", "false") == "true" {
+		args = append(args, "--catalog")
+	}
+
 	if api := getenv("INPUT_API", ""); len(api) != 0 {
 		args = append(args, "--api", api)
 	}

diff --git a/cmd/legacy.go b/cmd/legacy.go
@@ -80,13 +80,14 @@ func legacyConvert(ctx context.Context) error {
 
 		legacyPatch(ext)
 
-		repo, _, err := loadRepository(ctx, ext.Module)
+		repo, tags, err := loadRepository(ctx, ext.Module)
 		if err != nil {
 			return err
 		}
 
 		tmp := *ext
 		tmp.Repo = repo
+		tmp.Versions = filterVersions(tags)
 
 		if ok, _ := lintExtension(tmp); ok {
 			reg = append(reg, ext)

diff --git a/docs/example-api.txt b/docs/example-api.txt
@@ -1,8 +1,6 @@
 docs/example-api
 ├── catalog.json
-├── openapi.yaml
 ├── registry.json
-├── registry.schema.json
 ├── category
 │   ├── authentication.json
 │   ├── browser.json

diff --git a/docs/example-api/catalog.json b/docs/example-api/catalog.json
@@ -24,7 +24,7 @@
       "name": "xk6-dashboard",
       "owner": "grafana",
       "public": true,
-      "stars": 327,
+      "stars": 328,
       "timestamp": 1719907965,
       "topics": [
         "xk6",

diff --git a/docs/example-api/category/observability.json b/docs/example-api/category/observability.json
@@ -24,7 +24,7 @@
       "name": "xk6-dashboard",
       "owner": "grafana",
       "public": true,
-      "stars": 327,
+      "stars": 328,
       "timestamp": 1719907965,
       "topics": [
         "xk6",

diff --git a/docs/example-api/category/reporting.json b/docs/example-api/category/reporting.json
@@ -24,7 +24,7 @@
       "name": "xk6-dashboard",
       "owner": "grafana",
       "public": true,
-      "stars": 327,
+      "stars": 328,
       "timestamp": 1719907965,
       "topics": [
         "xk6",

diff --git a/docs/example-api/grade/C.json b/docs/example-api/grade/C.json
@@ -24,7 +24,7 @@
       "name": "xk6-dashboard",
       "owner": "grafana",
       "public": true,
-      "stars": 327,
+      "stars": 328,
       "timestamp": 1719907965,
       "topics": [
         "xk6",

diff --git a/docs/example-api/grade/at-least/C.json b/docs/example-api/grade/at-least/C.json
@@ -24,7 +24,7 @@
       "name": "xk6-dashboard",
       "owner": "grafana",
       "public": true,
-      "stars": 327,
+      "stars": 328,
       "timestamp": 1719907965,
       "topics": [
         "xk6",

diff --git a/docs/example-api/grade/at-least/D.json b/docs/example-api/grade/at-least/D.json
@@ -24,7 +24,7 @@
       "name": "xk6-dashboard",
       "owner": "grafana",
       "public": true,
-      "stars": 327,
+      "stars": 328,
       "timestamp": 1719907965,
       "topics": [
         "xk6",

diff --git a/docs/example-api/grade/at-least/E.json b/docs/example-api/grade/at-least/E.json
@@ -24,7 +24,7 @@
       "name": "xk6-dashboard",
       "owner": "grafana",
       "public": true,
-      "stars": 327,
+      "stars": 328,
       "timestamp": 1719907965,
       "topics": [
         "xk6",

diff --git a/docs/example-api/grade/at-least/F.json b/docs/example-api/grade/at-least/F.json
@@ -24,7 +24,7 @@
       "name": "xk6-dashboard",
       "owner": "grafana",
       "public": true,
-      "stars": 327,
+      "stars": 328,
       "timestamp": 1719907965,
       "topics": [
         "xk6",

diff --git a/docs/example-api/module/github.com/grafana/xk6-dashboard/extension.json b/docs/example-api/module/github.com/grafana/xk6-dashboard/extension.json
@@ -23,7 +23,7 @@
     "name": "xk6-dashboard",
     "owner": "grafana",
     "public": true,
-    "stars": 327,
+    "stars": 328,
     "timestamp": 1719907965,
     "topics": [
       "xk6",