extension/proxmox: add console access for instances #11601

shwstppr · 2025-09-09T13:43:59Z

Description

This PR introduces console access support for instances deployed using Orchestrator Extensions, available via either VNC or a direct URL.

CloudStack queries the extension using the getconsole action.
For VNC-based access, the extension must return host/port/ticket details. CloudStack then forwards these to the Console Proxy VM (CPVM) in the instance’s zone. It is assumed that the CPVM can reach the specified host and port.
For direct URL access, the extension returns a console URL with the protocol set to direct. The URL is then provided directly to the user.
The built-in Proxmox Orchestrator Extension now supports console access via VNC. The extension calls the Proxmox API to fetch console details and returns them in the required format.

Also, adds changes to send caller details to the extension payload.

# cat /var/lib/cloudstack/management/extensions/Proxmox/02b650f6-bb98-49cb-8cac-82b7a78f43a2.json | jq
{
  "caller": {
    "roleid": "6b86674b-7e61-11f0-ba77-1e00c8000158",
    "rolename": "Root Admin",
    "name": "admin",
    "roletype": "Admin",
    "id": "93567ed9-7e61-11f0-ba77-1e00c8000158",
    "type": "ADMIN"
  },
  "virtualmachineid": "126f4562-1f0f-4313-875e-6150cabeb72f",
  ...

Documentation PR: apache/cloudstack-documentation#560

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

proxmox-console.mp4

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Add console access support for the instances deployed using inbuilt Proxmox Orchestrator extension. Underlying CloudStack queries Proxmox API for console and then passes the ticket and host to the CPVM for the zone. During the flow it is assumed CPVM will be able to access the instance (Proxmox) host. Signed-off-by: Abhishek Kumar <[email protected]>

shwstppr · 2025-09-09T13:47:33Z

@blueorangutan package

blueorangutan · 2025-09-09T13:48:04Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

codecov · 2025-09-09T13:52:31Z

Codecov Report

❌ Patch coverage is 66.98113% with 140 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.50%. Comparing base (0526ec0) to head (1e911a9).
⚠️ Report is 23 commits behind head on main.

Files with missing lines	Patch %	Lines
...com/cloud/agent/api/GetExternalConsoleCommand.java	0.00%	21 Missing ⚠️
.../com/cloud/consoleproxy/vnc/network/NioSocket.java	0.00%	16 Missing ⚠️
.../com/cloud/agent/api/GetExternalConsoleAnswer.java	48.27%	15 Missing ⚠️
...udstack/consoleproxy/ConsoleAccessManagerImpl.java	92.81%	9 Missing and 4 partials ⚠️
...al/provisioner/ExternalPathPayloadProvisioner.java	83.56%	7 Missing and 5 partials ⚠️
...work/extensions/manager/ExtensionsManagerImpl.java	75.55%	2 Missing and 9 partials ⚠️
...om/cloud/consoleproxy/ConsoleProxyClientParam.java	0.00%	9 Missing ⚠️
...in/java/com/cloud/servlet/ConsoleProxyServlet.java	0.00%	8 Missing ⚠️
...om/cloud/consoleproxy/ConsoleProxyNoVncClient.java	0.00%	8 Missing ⚠️
...hypervisor/external/resource/ExternalResource.java	0.00%	7 Missing ⚠️
... and 7 more

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #11601      +/-   ##
============================================
+ Coverage     17.39%   17.50%   +0.10%     
- Complexity    15285    15425     +140     
============================================
  Files          5889     5894       +5     
  Lines        526183   526836     +653     
  Branches      64242    64331      +89     
============================================
+ Hits          91542    92228     +686     
+ Misses       424297   424231      -66     
- Partials      10344    10377      +33

Flag	Coverage Δ
uitests	`3.61% <ø> (-0.01%)`	⬇️
unittests	`18.56% <66.98%> (+0.11%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

blueorangutan · 2025-09-09T15:18:35Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 14921

Signed-off-by: Abhishek Kumar <[email protected]>

Copilot

Pull Request Overview

Add console access support for instances deployed using the inbuilt Proxmox Orchestrator extension. CloudStack queries the Proxmox API for console credentials and passes them to the CPVM for the zone.

Enables console access for External hypervisor instances by removing restriction
Implements console command handling for External/Proxmox instances
Adds proper authentication bypass for External hypervisors

Reviewed Changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
ui/src/components/view/ActionButton.vue	Removes External hypervisor restriction from console buttons
server/src/main/java/org/apache/cloudstack/consoleproxy/ConsoleAccessManagerImpl.java	Adds console connection details handling for External hypervisors with proper fallback logic
server/src/main/java/com/cloud/servlet/ConsoleProxyServlet.java	Adds authentication bypass for External hypervisor instances
server/src/main/java/com/cloud/consoleproxy/AgentHookBase.java	Adds authentication bypass for External hypervisor instances
server/src/main/java/com/cloud/server/ManagementServerImpl.java	Implements getExternalVmConsole method
extensions/Proxmox/proxmox.sh	Adds getconsole action for Proxmox VNC proxy support
Multiple test files	Comprehensive test coverage for new console functionality
Multiple API/framework files	New command classes and framework integration

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

server/src/test/java/org/apache/cloudstack/consoleproxy/ConsoleAccessManagerImplTest.java

server/src/main/java/org/apache/cloudstack/consoleproxy/ConsoleAccessManagerImpl.java

Signed-off-by: Abhishek Kumar <[email protected]>

shwstppr · 2025-09-10T10:57:28Z

@blueorangutan package

blueorangutan · 2025-09-10T10:58:04Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-09-10T12:15:24Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 14936

nvazquez

Thanks @shwstppr LGTM - left only some minor comments.

It would worth documenting what CloudStack expects from external extensions in order to display the console correctly.

.../src/main/java/org/apache/cloudstack/framework/extensions/manager/ExtensionsManagerImpl.java

...va/org/apache/cloudstack/hypervisor/external/provisioner/ExternalPathPayloadProvisioner.java

Related: apache/cloudstack#11601 Signed-off-by: Abhishek Kumar <[email protected]>

Signed-off-by: Abhishek Kumar <[email protected]>

kiranchavala · 2025-09-24T09:44:56Z

@blueorangutan test

blueorangutan · 2025-09-24T09:46:04Z

@kiranchavala a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

kiranchavala

LGTM

Test Case Execution	Result
Test the console access of a proxmox vm from the UI	Pass
Test the listConsoleSessions api call	Pass
Test the api createConsoleEndpoint	Pass
Test the Direct Url based console	Pass
Test from multiple management servers enabled environment	Pass
Test from a ssl enabled cloudstack-management server	Pass

blueorangutan · 2025-09-25T00:54:58Z

[SF] Trillian test result (tid-14431)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 51538 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11601-t14431-kvm-ol8.zip
Smoke tests completed. 147 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

rohityadavcloud

Changes requested:

Improve spec & documentation on getconsole access types, vnc and url. This does not make sense to me: " the extension returns a console URL with the protocol set to direct, url, or link. The URL is then provided directly to the user." Please also update https://github.com/apache/cloudstack-documentation/pull/560/files
In the getconsole, can we pass some kind of account or role type information, which can be used by Extension authors to return different URLs for different types of roles. For example, return an internal URL if the caller is a root admin versus a normal/public URL if the caller is a public tenant.

Signed-off-by: Abhishek Kumar <[email protected]>

shwstppr · 2025-09-25T07:22:12Z

@blueorangutan package

blueorangutan · 2025-09-25T07:24:04Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

shwstppr · 2025-09-25T07:33:24Z

@rohityadavcloud @harikrishna-patnala addressed comments. Now all extension payloads will contain caller field in externaldetails which will give basic details of the caller. Please see if that works for you

# cat /var/lib/cloudstack/management/extensions/Proxmox/5811778e-bded-4576-96bb-3d529d8c0a33.json | jq
{
  "virtualmachineid": "126f4562-1f0f-4313-875e-6150cabeb72f",
  "cloudstack.vm.details": {
    "id": 45,
    "name": "i-2-45-VM",
    "state": "Running",
    "type": "User",
    "cpus": 1,
    "minSpeed": 500,
    "maxSpeed": 500,
    "minRam": 536870912,
    "maxRam": 536870912,
    "arch": "x86_64",
    "bootArgs": "",
    "enableHA": false,
    "limitCpuUse": false,
    "enableDynamicallyScaleVm": false,
    "vncPassword": "uHQHQ4niy8f4ogwCM1k-rw",
    "details": {
      "proxmox_vmid": "103",
      "cpuOvercommitRatio": "2.0",
      "External:template_type": "template",
      "External:template_id": "101",
      "Message.ReservedCapacityFreed.Flag": "false"
    },
    "uuid": "126f4562-1f0f-4313-875e-6150cabeb72f",
    "enterHardwareSetup": false,
    "disks": [],
    "nics": [],
    "configDriveLocation": "SECONDARY",
    "guestOsDetails": {},
    "extraConfig": {},
    "networkIdToNetworkNameMap": {}
  },
  "virtualmachinename": "i-2-45-VM",
  "externaldetails": {
    "caller": {
      "roleid": "6b86674b-7e61-11f0-ba77-1e00c8000158",
      "rolename": "Root Admin",
      "name": "admin",
      "roletype": "Admin",
      "id": "93567ed9-7e61-11f0-ba77-1e00c8000158",
      "type": "ADMIN"
    },
    "host": {
      "node": "ref-trl-9283-k-mr8-abhishek-kumar-proxmox1",
      "network_bridge": "cloudbr1",
      "secret": "c0541327-d325-4a0f-8e14-3baf9eeb2da0",
      "user": "root@pam",
      "verify_tls_certificate": "false",
      "url": "10.0.35.44",
      "token": "token1"
    },
    "virtualmachine": {
      "template_type": "template",
      "template_id": "101"
    }
  }
}

blueorangutan · 2025-09-25T08:59:24Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 15161

rohityadavcloud

LGTM, didn't test the changes but good to see my remarks have been addressed. Thanks Abhishek, great work.

shwstppr · 2025-09-25T09:31:46Z

On a second thought, I feel caller field shouldn't be a child of externaldetails. I would make changes to move it to the root element.

Signed-off-by: Abhishek Kumar <[email protected]>

shwstppr · 2025-09-25T10:48:29Z

@blueorangutan package

blueorangutan · 2025-09-25T10:50:04Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-09-25T12:30:28Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15165

shwstppr · 2025-09-25T13:02:29Z

@blueorangutan test ol8 vmware-80u3

blueorangutan · 2025-09-25T13:04:05Z

@shwstppr a [SL] Trillian-Jenkins test job (ol8 mgmt + vmware-80u3) has been kicked to run smoke tests

blueorangutan · 2025-09-26T11:53:15Z

[SF] Trillian test result (tid-14439)
Environment: vmware-80u3 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 78614 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11601-t14439-vmware-80u3.zip
Smoke tests completed. 144 look OK, 3 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_08_upgrade_kubernetes_ha_cluster	`Failure`	3681.92	test_kubernetes_clusters.py
test_01_ssl_offloading_isolated_network	`Failure`	895.58	test_ssl_offloading.py
test_02_ssl_offloading_project_vpc	`Failure`	764.59	test_ssl_offloading.py
test_01_vpn_usage	`Error`	1.11	test_usage.py

shwstppr · 2025-09-27T03:22:57Z

Some VMware test failures unrelated. Similar results seen here in healthcheck, #11523 (comment)
Merging based on reviews and test results

Notes about console access for Orchestrator extension instances. Related: apache/cloudstack#11601 --------- Signed-off-by: Abhishek Kumar <[email protected]>

boring-cyborg bot added component:api component:management-server component:UI labels Sep 9, 2025

shwstppr added this to the 4.22.0 milestone Sep 9, 2025

harikrishna-patnala added this to Apache CloudStack 4.22.0 Sep 10, 2025

harikrishna-patnala moved this to In Progress in Apache CloudStack 4.22.0 Sep 10, 2025

shwstppr added 2 commits September 10, 2025 14:15

match response

c5a727e

Signed-off-by: Abhishek Kumar <[email protected]>

tests and changes

85deee8

Signed-off-by: Abhishek Kumar <[email protected]>

shwstppr requested a review from Copilot September 10, 2025 10:09

Copilot AI reviewed Sep 10, 2025

View reviewed changes

fix typo

0617ed4

Signed-off-by: Abhishek Kumar <[email protected]>

shwstppr requested review from abh1sar and nvazquez September 10, 2025 11:31

shwstppr added component:extensions component:console-proxy labels Sep 10, 2025

nvazquez approved these changes Sep 11, 2025

View reviewed changes

.../src/main/java/org/apache/cloudstack/framework/extensions/manager/ExtensionsManagerImpl.java Outdated Show resolved Hide resolved

...va/org/apache/cloudstack/hypervisor/external/provisioner/ExternalPathPayloadProvisioner.java Outdated Show resolved Hide resolved

shwstppr added a commit to shapeblue/cloudstack-documentation that referenced this pull request Sep 11, 2025

extensions: vnc console access

f095fb3

Related: apache/cloudstack#11601 Signed-off-by: Abhishek Kumar <[email protected]>

shwstppr mentioned this pull request Sep 11, 2025

extensions: instance console access apache/cloudstack-documentation#560

Merged

shwstppr added 2 commits September 11, 2025 11:29

improve log, error

d5c642a

Signed-off-by: Abhishek Kumar <[email protected]>

Merge remote-tracking branch 'apache/main' into proxmox-console

2f6234b

rohityadavcloud requested a review from harikrishna-patnala September 11, 2025 07:32

server,extension: allow returning direct console url

63f5f94

Signed-off-by: Abhishek Kumar <[email protected]>

kiranchavala approved these changes Sep 24, 2025

View reviewed changes

shwstppr marked this pull request as ready for review September 25, 2025 05:05

rohityadavcloud requested changes Sep 25, 2025

View reviewed changes

shwstppr added 2 commits September 25, 2025 12:31

review changes

ff4ab2d

Signed-off-by: Abhishek Kumar <[email protected]>

fix test

e06cbf9

Signed-off-by: Abhishek Kumar <[email protected]>

apache deleted a comment from blueorangutan Sep 25, 2025

rohityadavcloud approved these changes Sep 25, 2025

View reviewed changes

shwstppr marked this pull request as draft September 25, 2025 09:30

change

1e911a9

Signed-off-by: Abhishek Kumar <[email protected]>

shwstppr marked this pull request as ready for review September 26, 2025 13:16

shwstppr added status:ready-for-merge and removed status:ready-for-merge labels Sep 26, 2025

shwstppr merged commit 928972f into apache:main Sep 27, 2025
44 of 46 checks passed

shwstppr deleted the proxmox-console branch September 27, 2025 03:24

extension/proxmox: add console access for instances #11601

extension/proxmox: add console access for instances #11601

Uh oh!

Conversation

shwstppr commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

shwstppr commented Sep 9, 2025

Uh oh!

blueorangutan commented Sep 9, 2025

Uh oh!

codecov bot commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

blueorangutan commented Sep 9, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shwstppr commented Sep 10, 2025

Uh oh!

blueorangutan commented Sep 10, 2025

Uh oh!

blueorangutan commented Sep 10, 2025

Uh oh!

nvazquez left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

kiranchavala commented Sep 24, 2025

Uh oh!

blueorangutan commented Sep 24, 2025

Uh oh!

kiranchavala left a comment

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Sep 25, 2025

Uh oh!

rohityadavcloud left a comment

Choose a reason for hiding this comment

Uh oh!

shwstppr commented Sep 25, 2025

Uh oh!

blueorangutan commented Sep 25, 2025

Uh oh!

shwstppr commented Sep 25, 2025

Uh oh!

blueorangutan commented Sep 25, 2025

Uh oh!

rohityadavcloud left a comment

Choose a reason for hiding this comment

Uh oh!

shwstppr commented Sep 25, 2025

Uh oh!

shwstppr commented Sep 25, 2025

Uh oh!

blueorangutan commented Sep 25, 2025

Uh oh!

blueorangutan commented Sep 25, 2025

Uh oh!

shwstppr commented Sep 25, 2025

shwstppr commented Sep 9, 2025 •

edited

Loading

codecov bot commented Sep 9, 2025 •

edited

Loading