This is not an official Google product.
This implementation is not an official Google product, nor is it part of an official Google product. Support is available on a best-effort basis via GitHub.
This repository provides a concrete example of a custom API Hub Curation process, built using Google Application Integration. This process specifically ingests API metadata from a GitHub repository.
API Hub Curation involves transforming and enriching API metadata that has been ingested by Apigee API hub plugins. For a comprehensive overview, refer to the Apigee API hub Curation documentation.
The curation logic implemented here performs the following steps for each ingested API:
- Checks for API Specification File: It verifies if an API specification file is available in the GitHub repository.
- If file not available, the API is ingested without any additional enrichment.
- If file available, it extracts:
- API Version Metadata from the API specification file.
- API Metadata from an API configuration file, using OpenAPI extensions.
API Metadata and API Version Metadata are API hub System Attributes
- API Renaming: The API's name is standardized by removing any versioning information (if present) to consolidate all versions under a single, consistent name.
Example: if we have 2 Apigee proxies named proxyName.v1 and proxyName.v2, we don't want to have two APIs in API hub (proxyName.v1 and proxyName.v2) but only ONE API (proxyName) with TWO versions (v1 and v2).
To implement this sample, you'll need a GCP account with both Apigee API hub and Application Integration activated. If needed refer to Google Documentation Provision API huband Set up Application Integration.
You'll also need a GitHub account with privileges that allow setting up external access to your repositories, during Github Integraition Connector setup.
To begin, you'll need a GitHub repository to store your configuration and API specification files. If you haven't already, here are some essential GitHub tasks you'll need to complete, depending on your setup:
-
Create a GitHub account: If you're new to GitHub, start by creating an account.
-
Create new repositories: You'll need to create a new repository to host your files.
If you want to test the Integration using sample default data, you can copy files specfile.yaml and apiHub.cfg in your repository.
-
Create a GitHub App to authenticate connexion from Integration Connexion. See Github documentation create a GitHub App.
ℹ️ One Github App. can allow access to one or many repositories : you can configure it in Github App. permissions.
ℹ️ the Client ID and Client Secret generated during this process are crucial for setting up your GitHub Integration Connection. You'll also need the Callback URL from your Integration Connection to finalize the GitHub App creation. For more details, refer to the GitHub Integration Connection documentation.
Application Integration leverages a GitHub Integration Connection to retrieve files from your GitHub repository. Therefore, your initial step is to create this connection by following the instructions provided in the Configure the GitHub connector documentation. Please also refer to the preceding note regarding GitHub App parameters (credentials and call back URL).
To upload, do the following steps:
- Clone the repo
https://github.com/g-lalevee/apihub-curation-github.git-
In the Google Cloud console, go to the Application Integration page
-
In the navigation menu, click Integrations. The Integrations List page appears.
-
Select an existing integration or create a new integration by clicking Create integration. If you are creating a new integration:
- Enter a name
apiHubCurationGithub-v1and description in the Create Integration dialog. - Select a Region for the integration from the list of supported regions.
- Click Create.
This opens the integration in the integration designer.
- Enter a name
-
In the integration designer, click
Upload/download menuand then selectUpload integration. -
In the file browser dialog, select
apiHubCurationGithub-v1.json, and then click Open. A new version of the integration is created using the uploaded file. -
In the variable panel, update default values for config variables
CONFIG_githubConnectionName: the name of the Integration Connection created during step Application Integration Setup.CONFIG_githubRepoName: the name of the Github repository used to store API specification files and configuration files.CONFIG_githubRepoOwnerName: the name of the Github owner name of the repository.
-
In the integration designer, click Deploy.
Now you'll create a new custom Curation within Apigee API Hub and then associate it with an ingestion plugin.
First, let's create the custom Curation:
- In the Google Cloud console, navigate to the Apigee API Hub page.
- In the left navigation menu, click Settings.
- Select the Curations sub-menu, then click Setup a new Curation.
- Enter a Curation name and, from the dropdown list, select your Integration (apiHubCurationGithub-v1) and the api_trigger/curation trigger.
- Click Create curation.
If you need more detailed instructions, you can refer to the API Hub documentation on creating custom curations.
Next, you need to associate this new curation with your ingestion plugin. Currently, API Hub doesn't support editing existing plugin instances. This means if you want to change a plugin instance's configuration (like its curation logic), you'll need to delete the existing one and create a new one. For further information, see the API Hub documentation on managing plugin instances.
To associate your new curation:
- Select the Plugin sub-menu, then click Setup a new Plugin.
- Delete the existing plugin instance.
- Create a new plugin instance, providing the following:
- Display name: A name for your plugin instance.
- Sync frequency: How often the plugin instance should sync with its data source (default is Hourly).
- Curation: Select the curation you created in the steps above.
To effectively test this integration with the provided sample files (ApiHub.cfg and cl-AudienceAnalyse-v1.yaml), you'll need to configure specific API Hub attributes. These attributes are referenced by the integration's default apiData values.
Here's a breakdown of the attributes and their corresponding values:
-
ApiHub.cfgcontains:api_targetUser="internal" api_team="testing" api_businessUnit="none" api_maturityLevel="level-1" api_apiStyle="rest"
-
cl-AudienceAnalyse-v1.yamlcontains:openapi: 3.0.0 info: title: Contextual Analyzis API description:... . . . x-data-classification: publicaopenadata x-lifecycle: develop
See OpenAPI Extensions "x-" on the root level of the API spec and in info section.
You'll need to configure the following API Hub system attributes with their corresponding IDs and values:
| System Attributes | Resource | Value | ID | Description |
|---|---|---|---|---|
| Target Users | API | internal | Internal target user | |
| Team | API | Testing | testing | Apigee Testing Team |
| Business Unit | API | None | none | None |
| Maturity Level | API | Level 1 | level 1 | Resource |
| API Style | API | REST | REST | |
| Lifecycle | Version | Develop | develop | Development |
| Compliance | Version | Public_Open_Data | publicaopenadata | Publicly available data |
For detailed instructions on how to update API Hub system attributes via the interface or API, refer to the API Hub System Attributes documentation.
You can perform a quick unit test directly within Application Integration:
- Open the
apiHubCurationGithub-v1Integration in the Application Integration designer. - Ensure the Integration is deployed.
- Click the Test button and validate the default values. This will run the integration and display the execution results in the "Test Integration" dialog.
{
"apiMetadataList":{
"apiMetadata":[
{
"api":{
"name":"projects/apigee.org/locations/europe-west1/apis/apigee.org-cl-AudienceAnalyse-v1",
"displayName":"cl-AudienceAnalyse-v1",
"fingerprint":"cl-audienceanalyse-v1"
},
"versions":[
{
"version":{
"name":"projects/apigee.org/locations/europe-west1/apis/apigee.org-cl-AudienceAnalyse-v1/versions/version-1",
"displayName":"version-1"
},
"deployments":[
{
"deployment":{
"displayName":"cl-AudienceAnalyse-v1",
"deploymentType":{
"enumValues":{
"values":[
{
"id":"apigee"
}
]
},
"attribute":"projects/apigee.org/locations/europe-west1/attributes/system-deployment-type"
},
"resourceUri":"organizations/apigee.org/apis/cl-AudienceAnalyse-v1/revisions/24/environments/default-dev",
"endpoints":[
"https://34-149-164-96.nip.io//v1/audience"
],
"attributes":{
"projects/apigee.org/locations/europe-west1/attributes/plugin-system-apigee-x-and-hybrid-environment":{
"stringValues":{
"values":[
"default-dev"
]
},
"attribute":"projects/apigee.org/locations/europe-west1/attributes/plugin-system-apigee-x-and-hybrid-environment"
},
"projects/apigee.org/locations/europe-west1/attributes/plugin-system-apigee-x-and-hybrid-organization":{
"stringValues":{
"values":[
"apigee.org"
]
},
"attribute":"projects/apigee.org/locations/europe-west1/attributes/plugin-system-apigee-x-and-hybrid-organization"
}
}
},
"originalId":"apis/cl-AudienceAnalyse-v1/revisions/24/environments/default-dev",
"originalCreateTime":"2024-08-29T16:28:54.137Z",
"originalUpdateTime":"2024-08-29T16:28:54.137Z"
}
],
"originalId":"apis/cl-AudienceAnalyse-v1/revisions/24",
"originalCreateTime":"2024-08-29T16:27:47.136Z",
"originalUpdateTime":"2024-08-29T16:27:47.136Z"
}
],
"originalId":"apis/cl-AudienceAnalyse-v1",
"originalUpdateTime":"2024-08-29T16:27:47.136Z"
}
]
}
}
{
"apiMetadataList":{
"apiMetadata":[
{
"api":{
"name":"projects/apigee.org/locations/europe-west1/apis/cl-AudienceAnalyse",
"displayName":"cl-AudienceAnalyse",
"fingerprint":"cl-AudienceAnalyse",
- ADDED --> "targetUser":{
| "enumValues":{
| "values":[
| {
| "id":"internal"
| }
| ]
| }
| },
| "team":{
| "enumValues":{
| "values":[
| {
| "id":"testing"
| }
| ]
| }
| },
| "businessUnit":{
| "enumValues":{
| "values":[
| {
| "id":"none"
| }
| ]
| }
| },
| "maturityLevel":{
| "enumValues":{
| "values":[
| {
| "id":"level-1"
| }
| ]
| }
| },
| "apiStyle":{
| "enumValues":{
| "values":[
| {
| "id":"rest"
| }
| ]
| }
| ---> }
},
"versions":[
{
"version":{
"name":"projects/apigee.org/locations/europe-west1/apis/cl-AudienceAnalyse/versions/v1",
"displayName":"v1",
- ADDED --> "compliance":{
| "enumValues":{
| "values":[
| {
| "id":"publicaopenadata"
| }
| ]
| }
| },
| "lifecycle":{
| "enumValues":{
| "values":[
| {
| "id":"develop"
| }
| ]
| }
| ---> }
},
"deployments":[
{
"deployment":{
"displayName":"cl-AudienceAnalyse-v1",
"deploymentType":{
"enumValues":{
"values":[
{
"id":"apigee"
}
]
},
"attribute":"projects/apigee.org/locations/europe-west1/attributes/system-deployment-type"
},
"resourceUri":"organizations/apigee.org/apis/cl-AudienceAnalyse-v1/revisions/24/environments/default-dev",
"endpoints":[
"https://34-149-164-96.nip.io//v1/audience"
],
"attributes":{
"projects/apigee.org/locations/europe-west1/attributes/plugin-system-apigee-x-and-hybrid-environment":{
"stringValues":{
"values":[
"default-dev"
]
},
"attribute":"projects/apigee.org/locations/europe-west1/attributes/plugin-system-apigee-x-and-hybrid-environment"
},
"projects/apigee.org/locations/europe-west1/attributes/plugin-system-apigee-x-and-hybrid-organization":{
"stringValues":{
"values":[
"apigee.org"
]
},
"attribute":"projects/apigee.org/locations/europe-west1/attributes/plugin-system-apigee-x-and-hybrid-organization"
}
}
},
"originalId":"apis/cl-AudienceAnalyse-v1/revisions/24/environments/default-dev",
"originalCreateTime":"2024-08-29T16:28:54.137Z",
"originalUpdateTime":"2024-08-29T16:28:54.137Z"
}
],
"originalId":"apis/cl-AudienceAnalyse-v1/revisions/24",
"originalCreateTime":"2024-08-29T16:27:47.136Z",
"originalUpdateTime":"2024-08-29T16:27:47.136Z",
"specs":[
{
"spec":{
"displayName":"cl-AudienceAnalyse-v1",
"specType":{
"enumValues":{
"values":[
{
"id":"openapi"
}
]
}
},
"contents":{
"contents":"<BASE64 content cut>",
"mimeType":"application/yaml"
}
},
"originalUpdateTime":"2025-06-22T23:27:34.075Z"
}
]
}
],
"originalId":"apis/cl-AudienceAnalyse-v1",
"originalUpdateTime":"2024-08-29T16:27:47.136Z"
}
]
}
}
⚠️ Important Note on Running Curation:When you run the Curation process from your API Hub, it will apply the curation logic to ALL your APIs, with slightly different behaviors depending on whether an API has a corresponding specification file in your GitHub repository:
- Renaming Only: If an API does not have a corresponding API specification file in the GitHub repository, only the renaming part of the process will be applied.
- All Steps Applied: If an API does have a corresponding API specification file in the GitHub repository, all defined curation steps (including enrichment and renaming) will be applied.
To see the curation in action within API Hub:
- In the Google Cloud console, navigate to the API Hub interface.
- In the left navigation menu, click Settings.
- Select the Plugins sub-menu, then click the "3 dots" menu to the right of your configured plugin.
- Select Run.
- After a few minutes, click APIs in the left navigation menu.
- Validate how your APIs have been imported and enriched based on the curation logic.