Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 102 additions & 0 deletions docs/internal-developer-portal/catalog/catalog-yaml.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,108 @@ If you're using Harness IDP 2.0, please ensure you have reviewed the [IDP 2.0 Ov

Harness IDP 2.0 marks a significant evolution of the Internal Developer Portal, introducing a Harness-native data model tailored for enterprise-scale environments and strong access control. As we transition to this new model, **legacy Backstage YAML** will no longer be supported. Going forward, only the **Harness-native data model** schema will be used and referenced. This guide walks you through the new **Harness-native YAML schema** and outlines the key changes from the previous schema.

## Quick Reference: Entity YAML Examples

<details>
<summary>Component YAML Example</summary>

```yaml
apiVersion: harness.io/v1
kind: Component
name: anomaly-detection
identifier: anomaly-detection
type: service
owner: group:ccmplayacc
spec:
lifecycle: production
dependsOn:
- component:ng-manager
metadata:
description: CCM anomaly detection backend service
annotations:
backstage.io/kubernetes-label-selector: app=anomaly-detection
github.com/project-slug: wings-software/ce-anomalyDetection
pagerduty.com/service-id: PFVOX97
jira/project-key: CCM
backstage.io/source-location: url:https://github.com/wings-software/ce-anomalyDetection/tree/main
links:
- title: repo
url: https://github.com/wings-software/ce-anomalyDetection
tags:
- python
```
</details>

<details>
<summary>API YAML Example</summary>

```yaml
apiVersion: harness.io/v1
kind: API
type: openapi
identifier: cenextgen
name: cenextgen
owner: johndoe
spec:
lifecycle: production
definition:
$text: https://github.com/OAI/OpenAPI-Specification/blob/main/examples/v2.0/json/api-with-examples.json
metadata:
description: The official CE NEXTGEN service REST APIs
```
</details>

<details>
<summary>System YAML Example</summary>

```yaml
apiVersion: harness.io/v1
kind: System
name: Payment System
identifier: paymentsystem
type: domain
owner: team-payment
metadata:
description: This system groups services and libraries related to payment processing.
tags:
- rest
- java
```
</details>

<details>
<summary>Resource YAML Example</summary>

```yaml
apiVersion: harness.io/v1
kind: Resource
name: Payment Database
identifier: payment-database
type: database
owner: group:ccmplayacc
spec:
system:
- system:account/ccm_platform
dependsOn:
- resource:account/ccm-backup-storage
metadata:
description: MongoDB database for CCM platform data storage
annotations:
harness.io/db-instance: mongodb-ccm-prod-01
harness.io/db-region: us-east-1
harness.io/backup-schedule: daily-2am-UTC
links:
- title: Database Dashboard
url: https://harness-monitoring.grafana.net/d/mongodb-ccm
- title: Backup Policy
url: https://harness.atlassian.net/wiki/spaces/CCM/pages/123456/Database+Backup+Policy
tags:
- mongodb
- database
- production
```
</details>

:::info
Please ensure that **no entity YAML files** are stored in **Git in IDP 2.0** until the [Git Experience](/docs/internal-developer-portal/idp-2o-overview/2-0-overview-and-upgrade-path.md#native-harness-git-experience) feature is released. You can track its release and other updates in the **[IDP 2.0 Features Status](/docs/internal-developer-portal/idp-2o-overview/2-0-overview-and-upgrade-path.md)** table
:::
Expand Down
110 changes: 95 additions & 15 deletions docs/internal-developer-portal/catalog/manage-catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: Manage Catalog
description: Learn more about creating and managing your Catalog entities.
sidebar_position: 2
redirect_from: /docs/internal-developer-portal/catalog/register-software-component.md
redirect_from: /docs/internal-developer-portal/catalog/register-software-component
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
Expand All @@ -11,7 +11,7 @@ import TabItem from '@theme/TabItem';
<TabItem value="IDP 2.0 (New)" label="IDP 2.0 (New)">

:::tip For IDP 2.0 Customers
If you're using Harness IDP 2.0, please ensure you have reviewed the [IDP 2.0 Overview guide](/docs/internal-developer-portal/idp-2o-overview/2-0-overview-and-upgrade-path.md) and are familiar with the key steps for [upgrading to IDP 2.0](/docs/internal-developer-portal/idp-2o-overview/migrating-idp-2o.md). To enable IDP 2.0, you must raise a support ticket to activate the `IDP_2_0` feature flag for your account.
If you're using Harness IDP 2.0, please ensure you have reviewed the [IDP 2.0 Overview guide](/docs/internal-developer-portal/idp-2o-overview/2-0-overview-and-upgrade-path) and are familiar with the key steps for [upgrading to IDP 2.0](/docs/internal-developer-portal/idp-2o-overview/migrating-idp-2o). To enable IDP 2.0, you must raise a support ticket to activate the `IDP_2_0` feature flag for your account.
:::

The **Software Catalog** is a centralized registry for all your software assets — including components, services, websites, libraries, data pipelines, and more.
Expand All @@ -27,7 +27,7 @@ Harness IDP 2.0 represents a major evolution of the Internal Developer Portal, b
- **Revamped Catalog UI:** A fresh user experience featuring an updated Catalog table with filters, enhanced navigation, and a built-in entity creation interface.

## Creating Entities [IDP 2.0]
With IDP 2.0, we are moving towards a Harness-native Data Model designed for enterprise scale and access control. There are different entities within the Harness-native Data Model. To learn more about the Harness IDP entities, go to [Data Model](/docs/internal-developer-portal/catalog/data-model.md#harness-idp-entities)
With IDP 2.0, we are moving towards a Harness-native Data Model designed for enterprise scale and access control. There are different entities within the Harness-native Data Model. To learn more about the Harness IDP entities, go to [Data Model](/docs/internal-developer-portal/catalog/data-model#harness-idp-entities)

There are two ways to add and create a new entity in your catalog:
- **Create an entity via the Harness IDP UI**:
Expand All @@ -47,23 +47,52 @@ To create a new entity, navigate to the Harness IDP portal and click on **“Cre
![](./static/create-entity-2.png)
3. Define the **entity scope** — choose whether the entity should reside at the Account, Project, or Organization level. Read more about Catalog RBAC.
![](./static/scope-entity.png)
4. Next, choose how you want to manage the entity:

4. **Link to Source Code Repository**
Configure the source code repository associated with this component. This link enables several key capabilities, such as:

* Automatically configuring plugins like **Scorecards**, **TechDocs**, and **STO**
* Displaying the **View Source** option in the UI

![](./static/source-code-link-ui.png)

You can connect repositories from multiple supported providers:
* **Harness Code Repository** – Native Harness Git management
* **GitHub** – Cloud or enterprise GitHub instances
* **GitLab** – Cloud or self-managed GitLab instances
* **Bitbucket** – Cloud or server-hosted Bitbucket repositories
* **Azure Repos** – Repositories from Azure DevOps Services or Server

You also get the option to connect mono repository (monorepo) is a single repository that contains multiple projects or services, often organized in subdirectories.
![](./static/source-code-mono.png)
* **Yes** – Select this if the component's code is located inside a specific subdirectory of a larger repository. You must also provide the **Subdirectory Path** (for example `/harness/java-service`).
* **No** – Select this if the repository is dedicated to a single component or service.


This field is **optional**, but strongly recommended if your component is tied to a Git-based workflow or needs source-aware plugins.
For Harness Code Repo, note that the source code repository link is scoped to the same level as the entity itself (Account, Organization, or Project).
:::info
When configuring Link to Source Code, ensure that the selected Git connector has permissions matching the scope at which the entity is created (Account, Organization, or Project). For example, if the entity is created at the Project level, the Git connector must have access to all three scopes. The same applies to the Harness Code Repository — it should be configured with access rights that align with the selected scope.
:::
> Harness IDP also auto-generates the legacy `backstage.io/source-location` annotation for backwards compatibility.

6. Choose how you want to manage the entity:
* **Inline (default):** Manage the entity YAML directly within Harness.
* **Remote:** Choose to store your entity YAML in a Git repository for version control, collaboration, and change tracking.
You can either use a **Harness Code Repository** or connect to a **Third-party Git provider** like GitHub or GitLab by selecting a Git connector, repository, branch, and YAML path.
![](./static/catalog-git.png)
> The Git Experience is ideal for teams who prefer to manage entities as code. Learn more in the [Git Experience Journey](/docs/internal-developer-portal/git-experience/gitx-journey.md).
5. Click on **“Review YAML”** to view the auto-generated YAML. Since there's a live sync between the Visual and YAML views, changes in one will reflect in the other.
> The Git Experience is ideal for teams who prefer to manage entities as code. Learn more in the [Git Experience Journey](/docs/internal-developer-portal/git-experience/gitx-journey).
7. Click on **“Review YAML”** to view the auto-generated YAML. Since there's a live sync between the Visual and YAML views, changes in one will reflect in the other.

:::info
**YAML validation** is performed to ensure compatibility with the **Harness-native Catalog YAML** model. Any errors will be shown in the Validation logs.
Ensure your `identifier` follows [naming rules](https://developer.harness.io/docs/platform/references/entity-identifier-reference/#identifier-naming-rules). Invalid identifiers may lead to entity registration errors.
![](./static/yaml-validation.png)
:::

6. If needed, **configure a plugin** by referring to its documentation and adding the required annotations in the Catalog YAML.
8. If needed, **configure a plugin** by referring to its documentation and adding the required annotations in the Catalog YAML.
![](./static/plugins-entity.png)
7. Once everything is set, click **“Create Component”** to finalize and create the entity.
9. Once everything is set, click **“Create Component”** to finalize and create the entity.
![](./static/yaml-view.png)

### Catalog with Git Experience (GitX)
Expand All @@ -86,7 +115,7 @@ This makes it easy to track, version, and collaborate on entity definitions as c
While you can view and edit entity YAML across multiple Git branches, the entity registered in the Software Catalog will always reflect the YAML from the repository’s **default branch** (e.g., `main`). Ensure that any final changes are merged to the default branch to be considered active in IDP.
:::
### Catalog YAML
You can also use the [Catalog YAML](/docs/internal-developer-portal/catalog/catalog-yaml.md) to create entities in Harness IDP. With IDP 2.0, you are required to follow the new Harness-native Data Model and structure when defining entities in Catalog YAML. If you have existing entities defined using **legacy Backstage YAML**, you can still use them—Harness will automatically convert them into the Harness-native Data Model format. Learn more about the new data model here. To create a new entity, navigate to the Harness IDP portal and click **“Create”** from the side-bar menu.
You can also use the [Catalog YAML](/docs/internal-developer-portal/catalog/catalog-yaml) to create entities in Harness IDP. With IDP 2.0, you are required to follow the new Harness-native Data Model and structure when defining entities in Catalog YAML. If you have existing entities defined using **legacy Backstage YAML**, you can still use them—Harness will automatically convert them into the Harness-native Data Model format. Learn more about the new data model here. To create a new entity, navigate to the Harness IDP portal and click **“Create”** from the side-bar menu.

:::info
If you have a **legacy Backstage YAML**, you can still use it to create an entity. Harness will automatically convert it into the native **Catalog Entity Model** format.
Expand All @@ -102,17 +131,68 @@ Note: **YAML validation** is automatically performed to ensure compatibility wit
![](./static/yaml-validation.png)
:::

3. You can define the **scope** of the entity in two ways: either switch to the Visual View and select the desired scope, or specify the **[projectIdentifier](/docs/internal-developer-portal/catalog/catalog-yaml.md#projectidentifier)** or **[orgIdentifier](/docs/internal-developer-portal/catalog/catalog-yaml.md#orgidentifier)** directly in the YAML to set the project or organization scope.
3. You can define the **scope** of the entity in two ways: either switch to the Visual View and select the desired scope, or specify the **[projectIdentifier](/docs/internal-developer-portal/catalog/catalog-yaml#projectidentifier)** or **[orgIdentifier](/docs/internal-developer-portal/catalog/catalog-yaml#orgidentifier)** directly in the YAML to set the project or organization scope.
![](./static/scope-entity.png)
4. Choose how you want to manage the entity:

4. Define **Link to Source Code Repository** to configure the source code repository associated with this component. This link enables several key capabilities, such as, Automatically configuring plugins and Displaying the **View Source** option in the UI
This field is **optional**, but strongly recommended if your component is tied to a Git-based workflow or needs source-aware plugins.

```yaml
spec:
sourceCode:
monoRepo: false
provider: Github
repoName: java-service_svc
connectorRef: account.ShibamDhar // Empty in case of Harness Code Repository
```

#### Supported Repository Providers in YAML

The `provider` field defines the Git-based source hosting service. Harness IDP supports:

```yaml
provider: Harness # Harness Code Repository
provider: Github # GitHub Cloud or Enterprise
provider: Gitlab # GitLab Cloud or Self-Managed
provider: Bitbucket # Bitbucket Cloud or Server
provider: AzureRepo # Azure DevOps Repositories
```

You should select the correct provider according to where your code is hosted. The `connectorRef` should point to a valid Harness Connector for that provider.

:::info
When configuring Link to Source Code, ensure that the selected Git connector has permissions matching the scope at which the entity is created (Account, Organization, or Project). For example, if the entity is created at the Project level, the Git connector must have access to all three scopes. The same applies to the Harness Code Repository — it should be configured with access rights that align with the selected scope.
:::
#### Mono Repository Setup in YAML

A **mono repository (monorepo)** contains multiple projects or services in separate subdirectories within the same repository. This is useful for organizations managing many services in a unified repository.

When `monoRepo` is set to `true`, you must also define the `monoReposubDirectoryPath` to indicate the folder containing this component’s source code.

Example for a **mono repository** setup:

```yaml
spec:
sourceCode:
monoRepo: true
provider: Github
repoName: java-service_svc
connectorRef: account.ShibamDhar
monoReposubDirectoryPath: /harness
```

> Harness IDP also auto-generates the legacy `backstage.io/source-location` annotation for backwards compatibility.
> Currently, Scorecard computation uses the GitX connector and the Git Integration connector. An upcoming enhancement, available behind the `USE_LOCAL_GIT_CONNECTOR_FOR_SCORE_COMPUTATION` feature flag, will enable Scorecard computation to directly use the connector from Link to Source Code.

5. Choose how you want to manage the entity:
* **Inline (default):** Manage the entity YAML directly within Harness.
* **Remote:** Choose to store your entity YAML in a Git repository for version control, collaboration, and change tracking.
You can either use a **Harness Code Repository** or connect to a **Third-party Git provider** like GitHub or GitLab by selecting a Git connector, repository, branch, and YAML path.
![](./static/catalog-git.png)
> The Git Experience is ideal for teams who prefer to manage entities as code. Learn more in the [Git Experience Journey](/docs/internal-developer-portal/git-experience/gitx-journey.md).
> The Git Experience is ideal for teams who prefer to manage entities as code. Learn more in the [Git Experience Journey](/docs/internal-developer-portal/git-experience/gitx-journey).

5. If needed, **configure a plugin** by referring to the plugin’s documentation and adding the appropriate **annotations** in the Catalog YAML.
6. Once all details are complete, click **“Create Component”** to finalize and register your entity in the catalog.
6. If needed, **configure a plugin** by referring to the plugin’s documentation and adding the required annotations in the Catalog YAML.
7. Once all details are complete, click **“Create Component”** to finalize and register your entity in the catalog.

## Editing Entities [IDP 2.0]
You can now modify your entities directly from the **Harness IDP UI**, removing the dependency on manually editing the Catalog YAML file in your Git repository. This streamlines the update process and makes entity management much easier.
Expand Down Expand Up @@ -170,7 +250,7 @@ Although it's possible to name catalog entity descriptor files however you wish,

:::

Follow the instructions [here](/docs/internal-developer-portal/catalog/catalog-yaml.md) to understand more on how to write a `catalog-info.yaml`
Follow the instructions [here](/docs/internal-developer-portal/catalog/catalog-yaml) to understand more on how to write a `catalog-info.yaml`

### Save the File

Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading