-
Notifications
You must be signed in to change notification settings - Fork 17
8.4. Caching
The APIM-CLI is using the official API-Manager REST-API in order to load the actual state of an API, App, Org, etc.
Based on the number of APIs, Applications, Organizations and how they relate to each other (e.g. how many subscriptions to an API), this might result in a massive amount of requests and significantly reduces the performance of the CLI, especially when doing a massive export of APIs or Applications.
To avoid requesting the same entities over and over again, the CLI is using a cache (based on Ehcache), which persists the requested information to disk and it is reusing next time the CLI is started.
The APIM CLI distinguishes between Export-Actions, like apim api get and Import-Actions like apim api import.
Export actions use all available caches by default, import actions do not. As the import of entities is much more sensitive operation than the export, because changes are made in the API-Manager. Therefore, the cache for the import is disabled by default.
👉 Please read the following carefully. It requires your understanding of how the cache works in the context of your system & data and what effects it might have.
Since version 1.7.0 it is possible to enable even for Import-Actions selected caches to improve the import performance. This can be necessary, for example, if an API is subscribed by more than 2000 applications. When updating such an API the APIM-CLI is forced to load the actual API-State incl. all subscribed applications, if necessary the configured quota for each application. This leads to more than 4000 API requests and might take a significant amount of time.
However, the cache is not activated by default for Import-Actions to avoid that the APIM CLI loads an actual state that is not fully up-to-date and then uses it in the further process to update an existing entity. Use the option as with caution and monitor closely whether the behavior works as required.
Assume you enable the cache for application quotas. This cache contains which quotas are configured for which applications. If the APIM-CLI needs to know the application quota, it calls the following endpoint for each application: /applications/<appId>/quota.
If you now change the quota for an application which has been cached, for example using the API-Manager UI, then the existing cache entry will of course remain unchanged.
If you import an API, which has to be created (e.g. because the API-Specification has changed), then the APIM-CLI will load actual state incl. that application, which the old Quota-Information from the cache. After the new API is created it will configure this application with the previously configured quota.
This is the reason why for Import-Actions not all caches are activated and instead only individual caches.
- only enable caches that actually have an impact on performance
- consider how dynamically the corresponding data changes in the API Manager
- it makes much more sense to enable caches for resources that are called in a loop and you have a lot of entities
- Good examples are
- the quotas for application if you change them rarely
- the application credentials if you change them rarely
- To figure out, which REST-API-Calls are causing a problem you may enable Traffic-Debug for the API-Manager port 8075 and monitor API-Requests when using the CLI
To enable selected caches for an Import-Action please add the following parameter:
-enabledCaches applicationsQuotaCache
You can also use wildcards for this:
-enabledCaches applicationsQuotaCache,*API*
Please note: If entities in the API Manager are changed by the APIM CLI, for example if an application quota is changed or an application is deleted, the cache is updated accordingly.
Of course, this only applies if subsequent APIM-CLI instances access the same cache. If the APIM CLI runs in the CI/CD pipeline, make sure if possible that the CLI is always executed on the same worker having access to the same cache.
If you wish, you can disable the cache completely and ensure that the API Manager REST API is accessed for any necessary information. You can use the toggle: -ignoreCache to skip using the caches at all. For import actions it is not possible to disable individual caches.
If you would like to clear one, more or all caches you can provide the option: -clearCache to a command:
-clearCache ALL -- All caches are cleared and will be filled up with fresh data
-clearCache "*API*" -- All caches named with API. This is NOT case sensitive.
-clearCache "organizationCache,*API*" -- Clears a combination of combination of caches
The get a list of all available caches, just provide an invalid to the clearCache option (-clearCache XXXX)
The following table represents all known caches and in which context they are used.
| Cache | Cached entity | REST-Endpoint | Comment |
|---|---|---|---|
| applicationAPIAccessCache | Apps subscribed to API | /applications/<appId>/apis |
Since API-Mgr version 7.7 not used anymore. |
| organizationAPIAccessCache | Orgs access to an API | /organizations/<orgId>/apis |
Which APIs the organization has granted access. |
| oauthClientProviderCache | List of OAuth-Profiles | /oauthclientprofiles |
In Policy-Studio configured OAuth-Client profiles |
| applicationsCache | Application details | /applications/<appId> |
The details of an application |
| applicationsSubscriptionCache | Subscribed apps for an API | /proxies/<apiId>/applications |
Applications subscribed a specific API |
| applicationsQuotaCache | Application-Quotas | /applications/<appId>/quota |
Configured quotas for an application |
| applicationsCredentialCache | Application-Credentials | /applications/<appId>/<type> |
Configured credentials for an application. Type might be oauth |
| organizationCache | Organization details | /organizations/<orgId> |
The details of an organization |
| userCache | User details | /users/<userId> |
The details of a user |
The CLI is delivered with a pre-configured set of entities and you can configure the caches time-to-live yourselves in the configuration file: lib/cacheConfig.xml.
Also, the configuration specifies where the cache should be persisted to disk. The default is ${java.io.tmpdir}/apim-cli-cache.
Please note that only one running apim-cli can initialize the cache at a time. If you run the APIM CLI as part of the CI/CD pipeline and jobs are running at the same time, you may see the following message:
"Error initiliazing cache - Perhaps another APIM-CLI is running that locks the cache. Retry again in 3 seconds. Attempts: 5/100
The CLI will check every 3 seconds for a maximum of 5 minutes to see if the cache can be exclusively initialized. These values are not configurable as of today.