[WIP] Add support for collection attributes (attributes=vms.name) by jrafanie · Pull Request #1320 · ManageIQ/manageiq-api

jrafanie · 2026-05-08T21:34:08Z

Enables requesting specific attributes on has_many associations using dot notation.

Before: attributes=vms.name returned {"vms": {"name": "Vm"}}
After: Returns array of VMs with requested attributes plus href and id

Uses eager loading to prevent N+1 queries.

Fixes #871

Fryguy · 2026-05-08T21:47:17Z

+      # Check for ActiveRecord collections (CollectionProxy, Relation) without triggering queries
+      # Both have .klass method, plain Arrays don't
+      def collection_association?(obj)
+        obj.respond_to?(:klass)
+      end


This is interesting - I feel like this could be done a better way or there should be a helper method elsewhere that already does this.

def collection_association?(obj) obj.respond_to?(:loaded?) end

This seems more intention revealing and uses a more public API... What do you think?

Note, an inline obj.respond_to?(:loaded?):

if related_obj.respond_to?(:loaded?) && @req.collection_attributes_for(base).any?

seems worse than:

if collection_association?(related_obj) && @req.collection_attributes_for(base).any?

should be done, ptal

Yeah, I was more surprised that we didn't have this method already implemented. I agree on the helper method being more intention revealing.

jrafanie · 2026-05-13T16:49:49Z

+          return fetch_collection_attributes(type, resource, base, related_obj)
+        end
+
+        # Standard virtual attribute handling for single attributes (e.g., hardware.host.name)


Note, this method should possibly be renamed as we are now fetching collection_associations or indirect virtual attributes. The comments make this path clearer but it might make sense to make these method calls for clearer intent.

jrafanie · 2026-05-13T16:52:30Z

+
+        # TODO: Virtual attributes not yet supported - only physical attributes (database columns)
+        # Consider: Support virtual attributes on associations (vms.v_total_snapshots) or expose
+        # aggregated virtual attributes on primary collection where they can be properly eager loaded?


This is a big one... this adds only vms.name type support, it doesn't add virtual column/attributes on the has many association (so not, vms.v_total_snapshots)

It makes me think we might have to weigh the pros/cons of exposing virtual attributes at the primary collection side (which may not make sense), or adding support for pulling back virtual attribute/columns from these has many associations.

This is a big one... this adds only vms.name type support, it doesn't add virtual column/attributes on the has many association (so not, vms.v_total_snapshots)

It makes me think we might have to weigh the pros/cons of exposing virtual attributes at the primary collection side (which may not make sense), or adding support for pulling back virtual attribute/columns from these has many associations.

in other words, I don't know that things like

GET /providers?attributes=vms.v_total_snapshots...

would even make sense.

Or even worse, a virtual column on providers themselves:

GET /providers?attributes=v_total_snapshots...

jrafanie · 2026-05-14T20:41:53Z

+        if collection_association?(related_obj) && @req.collection_attributes_for(base).any?
+          fetch_collection_attributes(type, resource, base, related_obj)
+        else
+          fetch_standard_virtual_attribute(related_obj, base, attr)


I split out the logic into two methods for the conditionals so hopefully it's easier to follow

jrafanie · 2026-05-15T13:55:13Z

      def determine_include_for_find(klass)
-        attrs = virtual_attributes_for(klass) do |type, attr_name, attr_base|
+        attrs = determine_include_for_find_vattrs(klass)
+        collections = determine_include_for_find_collections(klass)


This method added the collections to the list of things we need to build the includes for. Since we now have virtuals and collections doing similar things, each was added or moved to methods that return what needs to be included and then loops over them and builds the includes.

Fryguy · 2026-05-19T18:06:13Z

I think this got conflicted by #1324

Endpoint was not registered in api.yml, so collection_config.name_for_klass returned nil for the Endpoint class. normalize_hash skips href generation when the type is nil, causing endpoints returned via attributes=endpoints,authentications to be missing href and id fields. Fix by registering Endpoint as a read-only collection/subcollection in api.yml in its correct alphabetical position (between :disks and :enterprises), which allows name_for_klass to map Endpoint -> :endpoints so hrefs resolve correctly. Also guard collection_config.klass and get_reftype against unregistered collection names returning nil, which would otherwise crash with NoMethodError when attribute names like 'compliance_details' or 'custom_action_buttons' are not in api.yml. See also ManageIQ#741

Enables requesting specific attributes on has_many associations using dot notation in the attributes parameter (e.g., attributes=vms.name,vms.vendor). Returns each association as an array of resource hashes with href, id, and the requested attributes. The association is eager-loaded to prevent N+1 queries. When rendering a resource, dot-notation attributes are parsed into a map of association -> sub-attributes by RequestAdapter. The association is added to the eager-load set so it is fetched in one query. When the renderer encounters a has_many collection for that association, it calls normalize_hash on each item with only the requested attributes, rather than the usual single-object virtual attribute path. Only physical attributes are supported on association members; virtual attributes would cause N+1 queries even with eager loading. Fixes ManageIQ#871

The three numbered cases were really two paths (direct vs nested) with the direct path split across two consecutive if/elsif branches. Flip attr_base.blank? to be the outer condition so direct and nested are the primary split. The two direct sub-cases (virtual attribute and association) become inner if/elsif with no need for numbered labels.

determine_include_for_find was doing three things: building the virtual attribute include list, building the collection association include list, and merging them into the final hash. Extract the first two into their own methods so each piece has a single responsibility and determine_include_for_find just combines and converts them.

Replace needs_eager_load? with two focused predicates that name what they actually test. virtual_with_includes? covers virtual attributes whose value requires preloading associated data (has virtual_includes and is not SQL-backed). real_association? covers real AR associations. The real_association? name intentionally signals that virtual associations without uses: are excluded.

Covers cases where the eager-load decision could silently go wrong: has_one was the only real AR macro not directly exercised; SQL-computable virtual columns must not add a join even though they have virtual_includes; and virtual associations without uses: must not add a join, documenting that a switch to reflection_with_virtual would be a deliberate behavior change rather than an invisible one.

miq-bot · 2026-07-20T18:36:28Z

Checked commits jrafanie/manageiq-api@ad1c65a~...646a2b0 with ruby 3.3.10, rubocop 1.86.0, haml-lint 0.73.0, and yamllint 1.37.1
6 files checked, 4 offenses detected

app/controllers/api/base_controller/renderer.rb

❗ - Line 321, Col 11 - Layout/EmptyLineAfterGuardClause - Add empty line after guard clause.

spec/requests/vms_spec.rb

❗ - Line 389, Col 14 - Style/BlockDelimiters - Avoid using {...} for multi-line blocks.
❗ - Line 405, Col 14 - Style/BlockDelimiters - Avoid using {...} for multi-line blocks.
❗ - Line 424, Col 14 - Style/BlockDelimiters - Avoid using {...} for multi-line blocks.

jrafanie requested a review from bdunne as a code owner May 8, 2026 21:34

jrafanie added the enhancement label May 8, 2026

miq-bot added the wip label May 8, 2026

Fryguy reviewed May 8, 2026

View reviewed changes

Comment thread app/controllers/api/base_controller/renderer.rb Outdated

Fryguy reviewed May 8, 2026

View reviewed changes

Comment thread lib/api/request_adapter.rb Outdated

jrafanie force-pushed the 871-support-attributes-on-has-many branch 3 times, most recently from cd6ff1c to 4a1783a Compare May 13, 2026 16:47

jrafanie commented May 13, 2026

View reviewed changes

jrafanie force-pushed the 871-support-attributes-on-has-many branch from 4a1783a to 5eb6e3d Compare May 14, 2026 20:40

jrafanie commented May 14, 2026

View reviewed changes

jrafanie force-pushed the 871-support-attributes-on-has-many branch 4 times, most recently from 6d730b3 to 2b8ffe5 Compare May 14, 2026 21:30

jrafanie commented May 15, 2026

View reviewed changes

miq-bot added the unmergeable label May 19, 2026

jrafanie added the ai assisted label Jun 10, 2026

jrafanie force-pushed the 871-support-attributes-on-has-many branch from 2b8ffe5 to 57e5737 Compare July 16, 2026 23:00

miq-bot removed the unmergeable label Jul 17, 2026

jrafanie force-pushed the 871-support-attributes-on-has-many branch 2 times, most recently from bd5a636 to 6acca3d Compare July 17, 2026 17:25

jrafanie added 5 commits July 20, 2026 14:35

jrafanie force-pushed the 871-support-attributes-on-has-many branch from 6acca3d to 646a2b0 Compare July 20, 2026 18:36

Navigation Menu

Uh oh!

[WIP] Add support for collection attributes (attributes=vms.name) - #1320

[WIP] Add support for collection attributes (attributes=vms.name)#1320
jrafanie wants to merge 6 commits into
ManageIQ:masterfrom
jrafanie:871-support-attributes-on-has-many

jrafanie commented May 8, 2026

Uh oh!

Uh oh!

Fryguy May 8, 2026

Uh oh!

jrafanie May 13, 2026

Uh oh!

jrafanie May 13, 2026

Uh oh!

Fryguy May 13, 2026

Uh oh!

Uh oh!

jrafanie May 13, 2026

Uh oh!

jrafanie May 13, 2026

Uh oh!

jrafanie May 14, 2026

Uh oh!

jrafanie May 14, 2026

Uh oh!

jrafanie May 15, 2026

Uh oh!

Fryguy commented May 19, 2026

Uh oh!

miq-bot commented Jul 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

jrafanie commented May 8, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Fryguy commented May 19, 2026

Uh oh!

miq-bot commented Jul 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants