Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add ghes broker docs #1441

Merged
merged 3 commits into from
Dec 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
59 changes: 37 additions & 22 deletions content/docs/catalog/location-management/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,45 +6,42 @@ description: How to configure locations autodiscovery

## Introduction

The Backstage catalog uses location entities to point to external sources which define entities. Roadie will update your
entities in Backstage whenever one of the repositories defined in a location is updated.
The Backstage catalog uses location entities to point to external sources which define entities. Roadie will update your
entities in Backstage whenever one of the repositories defined in a location is updated.

Autodiscovery is the mechanism to define patterns pointing to filenames in URLs that should be queried for entity files.

For more information about locations and troubleshooting Catalog ingestion issues you can take a look at the [catalog troubleshooting documentation](/docs/details/troubleshooting-the-catalog)


## Setting up autodiscovery

Roadie supports autodiscovery for integrations configured in the application. The configuration of an autodiscovery entry can be done at the same time as integration is configured to start automatic polling of entities. You can navigate to `https://<your-tenant>.roadie.so/administration/settings/` to find configuration options for each supported autodiscovery target

### GitHub Autodiscovery and Organization Autodiscovery

GitHub integration for Roadie provides two autodiscovery targets, entities and teams/groups.
GitHub integration for Roadie provides two autodiscovery targets, entities and teams/groups.

![GitHub Autodiscovery of Catalog Files](./gh_autodiscovery.webp)

The GitHub *autodiscovery* integration can be configured to automatically discover catalog entities from a provided configured path pointing to GitHub.
The GitHub _autodiscovery_ integration can be configured to automatically discover catalog entities from a provided configured path pointing to GitHub.

More information about possible patterns and examples of URLs can be found from the [Backstage GitHub Discovery documentation.](https://backstage.io/docs/integrations/github/discovery#configuration)

![GitHub organization autodiscovery](gh_org_ingestion.webp)

The GitHub *organization* integration provides the possibility to automatically import GitHub users and teams/groups from an organization. You can configure this discovery type by providing the name of the GitHub organization. Note that the GitHub app needs to be installed for that organization and needs to have permissions to access user info.
The GitHub _organization_ integration provides the possibility to automatically import GitHub users and teams/groups from an organization. You can configure this discovery type by providing the name of the GitHub organization. Note that the GitHub app needs to be installed for that organization and needs to have permissions to access user info.

### AWS S3 Autodiscovery


![AWS S3 autodiscovery target](./s3_entry.webp)

AWS S3 autodiscovery can be configured to download catalog files from an S3 bucket. For this to be functionality the [integration configuration for AWS S3](/docs/integrations/aws-s3) must be done. S3 autodiscovery does not support wildcard path, it must point to a specific prefix of an S3 bucket.

### Azure DevOps


![Azure DevOps autodiscovery target](./azure-autodiscovery.webp)

Azure DevOps autodiscovery can be configured to download catalog files from Azure DevOps SCM. For this to be functionality the [integration configuration for Azure Devops](/docs/integrations/azure-devops-provider) must be done.
Azure DevOps autodiscovery can be configured to download catalog files from Azure DevOps SCM. For this to be functionality the [integration configuration for Azure Devops](/docs/integrations/azure-devops-provider) must be done.

### Bitbucket Autodiscovery

Expand Down Expand Up @@ -75,17 +72,17 @@ describes the url format for Bitbucket Cloud (reproduced from Backstage.io):
[See here for the specification](https://developer.atlassian.com/bitbucket/api/2/reference/meta/filtering)
for the query argument (will be passed as the `q` query parameter).
- (Optional) The `search=true` query argument to activate the mode utilizing code search.
- Is mutually exclusive to the `q` query argument.
- Allows providing values at `catalogPath` for finding catalog files as allowed by the `path` filter/modifier
[at Bitbucket Cloud's code search](https://confluence.atlassian.com/bitbucket/code-search-in-bitbucket-873876782.html#Search-Pathmodifier).
- `catalogPath=/catalog-info.yaml`
- `catalogPath=catalog-info.yaml` (anywhere in the repository)
- `catalogPath=/path/catalog-info.yaml`
- `catalogPath=path/catalog-info.yaml`
- `catalogPath=/path/*/catalog-info.yaml`
- `catalogPath=path/*/catalog-info.yaml`
- Supports multiple catalog files per repository depending on the `catalogPath` value.
- Registers `Location` entities for existing files only vs all matching repositories.
- Is mutually exclusive to the `q` query argument.
- Allows providing values at `catalogPath` for finding catalog files as allowed by the `path` filter/modifier
[at Bitbucket Cloud's code search](https://confluence.atlassian.com/bitbucket/code-search-in-bitbucket-873876782.html#Search-Pathmodifier).
- `catalogPath=/catalog-info.yaml`
- `catalogPath=catalog-info.yaml` (anywhere in the repository)
- `catalogPath=/path/catalog-info.yaml`
- `catalogPath=path/catalog-info.yaml`
- `catalogPath=/path/*/catalog-info.yaml`
- `catalogPath=path/*/catalog-info.yaml`
- Supports multiple catalog files per repository depending on the `catalogPath` value.
- Registers `Location` entities for existing files only vs all matching repositories.

Examples:

Expand Down Expand Up @@ -136,13 +133,31 @@ The format is slightly different for Bitbucket server. The url is composed of fo
will result in:
`https://bitbucket.mycompany.com/projects/my-project/repos/service-a/catalog-info.yaml`.

### GitHub Enterprise Server Autodiscovery

The GitHub autodiscovery integration can be configured to automatically discover catalog entities from a provided configured path pointing to GitHub.

#### GHES Broker Autodiscovery

You can also connect to your GitHub Enterprise server via the broker connection.
![GHES broker configuration](./broker-ghes-configuration.webp)

To enable autodiscovery of entities via the broker, you can add entries to the autodiscovery section of the settings page.
Click the + sign next to the `Configure GHES via the Broker` to open the input dialog to configure the GHES target URL via the broker.
Within the opened dialog you can add the URL of your GitHub organization or repository directly, as it is referenced in the main configuration section. If the broker connection is configured, these URLs will be automatically translated to use the broker protocol. Note that this configuration only affects the GHES when it is configured via a broker and won't affect discovery if broker configuration is not present.

Once you have added the information about your GHES instance and organization, you can see your existing configured target URLs in the list below
![GHES broker autodiscovery](./broker-ghes-discovery.webp).

The autodiscovery kicks in shortly after the save action and automatically refreshes discovered entities periodically

## Managing Locations

A list of all configured locations is located in URL `https://<your-tenant>.roadie.so/administration/locations/management`.
A list of all configured locations is located in URL `https://<your-tenant>.roadie.so/administration/locations/management`.

![Location management table](./location_mgmt.webp)

The list contains an entry for all locations added as an autodiscovery configuration within each integration, as well as locations imported using the Catalog Import functionality within `https://<your-tenant>.roadie.so/import/entity`.
The list contains an entry for all locations added as an autodiscovery configuration within each integration, as well as locations imported using the Catalog Import functionality within `https://<your-tenant>.roadie.so/import/entity`.

Note that deleting a location via the administration UI (either at the location management page or at the individual integration page) does not automatically remove individual entities that have been found from that location. If the desired effect is to remove all items from a particular location, it is better to find that location via the catalog and[ unregister it from there.](/docs/details/unregister-components)

Expand Down
Binary file not shown.
23 changes: 18 additions & 5 deletions content/docs/integrations/ghe/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Configuring GitHub Enterprise Server in Roadie
publishedDate: '2024-10-01T16:03:23.0Z'
description: How to configure a GitHub Enterprise Server on your Roadie instance

humanName: GitHub Enterprise Service
humanName: GitHub Enterprise Server
logoImage: '../../../assets/logos/github/mark/official/PNG/GitHub-Mark-120px-plus.webp'
integrationType: Catalog data source
---
Expand All @@ -12,6 +12,8 @@ integrationType: Catalog data source

Roadie integrates with both cloud and self-hosted GitHub installations. This guide instructs how to set up self-hosted GitHub Enterprise Server to work with your Roadie instance. With this integration, it is possible to ingest entities and connect and use GitHub plugins.

Roadie supports GitHub Enterprise Servers via both public access via the Internet and a via the [secured broker access](/docs/integrations/broker/).


## Steps

Expand All @@ -22,11 +24,16 @@ Roadie integrates with both cloud and self-hosted GitHub installations. This gui
![ghes-token.png](ghes-token.png)


### Step 2. Store GitHub Enterprise Server token into Roadie
### Step 2a. Store GitHub Enterprise Server token into Roadie (public GHES)

1. Navigate to your Roadie instance URL `https://<your-roadie-tenant>.roadie.so/administration/integrations/github-enterprise`
2. Input the GHES token from step 1 as a value to secret `GHE_TOKEN`


### Step 2b. Store GitHub Enterprise Server token into Roadie (GHES behind broker)

1. Modify the environment variables of your running broker client to reference the API token created in step 1.

### Step 3. Create a new GitHub Enterprise Server OAuth Client

1. Create a new OAuth app within GitHub Enterprise Server developer settings
Expand All @@ -44,13 +51,19 @@ Roadie integrates with both cloud and self-hosted GitHub installations. This gui
2. Input the client id from step 3 as a value to secret `GHE_CLIENT_ID`
2. Input the generated client secret from step 3 as a value to secret `GHE_CLIENT_SECRET`

### Step 5. Configure GitHub Enterprise Server on Roadie
### Step 5a. Configure GitHub Enterprise Server on Roadie (public GHES)

1. Configure you GitHub Enterprise Server URL within `https://<your-roadie-tenant>.roadie.so/administration/integrations/github-enterprise`
* The URL should have a value like `https://<your-roadie-tenant>.enginehouse.io`
* The URL should have a value like `https://<your-roadie-tenant>.mydomain.dev`


![ghes-roadie-settings.png](ghes-roadie-settings.png)
### Step 5b. Configure GitHub Enterprise Server on Roadie (GHES behind broker)
1. Configure you GitHub Enterprise Server URL within `https://<your-roadie-tenant>.roadie.so/administration/integrations/github-enterprise`
* The URL should have a value like `https://<your-roadie-tenant>.mydomain.dev`
2. Enable the broker connection and configured your broker connection path.
* The path should have a value like `broker://my-ghes-broker-token`

![ghes-settings.webp](ghes-settings.webp)

## Next Steps

Expand Down
Loading