Skip to content

Don't scrape pods via HTTP if the activator in path. #7324

Open

Description

@markusthoemmes

When the activator is in the routing path, scraping the pods is kinda useless as their information is effectively zeroed out, because all requests are considered "proxied", thus the activator metrics take precedence. We can use that to make our system more efficient in these cases!

I propose two steps to go about this:

  1. Never scrape if TBC=-1. The activator will always be in path in this case, so we never need to scrape really.
  2. Start/Stop scraping if activator is in path or not. This is a little more tricky as we need to make sure that we start scraping timely once we notice the activator will be taken off the path.

As the PodAutoscaler owns all of these decisions, this should be doable for us. As it's a good-first-issue, I'd love to see it land in two separate pieces as laid out though.

Note: I'm happy to mentor first-time contributors here. The changes needed should be fairly small in terms of number of files to touch.

Activity

  1. knative-prow-robot commented on Mar 19, 2020

    @knative-prow-robot
    Contributor

    @markusthoemmes:
    This request has been marked as suitable for new contributors.

    Please ensure the request meets the requirements listed here.

    If this request no longer meets these requirements, the label can be removed
    by commenting with the /remove-good-first-issue command.

    Details

    In response to this:

    When the activator is in the routing path, scraping the pods is kinda useless as their information is effectively zeroed out, because all requests are considered "proxied", thus the activator metrics take precedence. We can use that to make our system more efficient in these cases!

    I propose two steps to go about this:

    1. Never scrape if TBC=-1. The activator will always be in path in this case, so we never need to scrape really.
    2. Start/Stop scraping if activator is in path or not. This is a little more tricky as we need to make sure that we start scraping timely once we notice the activator will be taken off the path.

    As the PodAutoscaler owns all of these decisions, this should be doable for us. As it's a good-first-issue, I'd love to see it land in two separate pieces as laid out though.

    Note: I'm happy to mentor first-time contributors here. The changes needed should be fairly small in terms of number of files to touch.

    Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

  2. added
    kind/featureWell-understood/specified features, ready for coding.
    good first issueDenotes an issue ready for a new contributor, according to the "help wanted" guidelines.
    help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.
    on Mar 19, 2020
  3. dsimansk commented on Mar 19, 2020

    @dsimansk
    Contributor

    I'd like to give it a try to look into Serving internals a bit more.
    /assign dsimansk

  4. vagababov commented on Mar 19, 2020

    @vagababov
    Contributor

    /cc @taragu
    BTW, this is also one of the ways you can improve GSD, if we extend the activator reporting capabilities (not necessarily a good idea, but something to consider).

  5. added this to the Serving 0.15.x milestone on Apr 8, 2020
  6. removed
    good first issueDenotes an issue ready for a new contributor, according to the "help wanted" guidelines.
    help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.
    on Jun 29, 2020
  7. 12 remaining items

  8. nader-ziada commented on Apr 29, 2022

    @nader-ziada
    Member

    /assign

  9. dprotaso commented on Jun 29, 2022

    @dprotaso
    Member

    Related PR: #13027

    If there are no ready activators but ready pod endpoints requests will bypass the activator even if TBC=-1

  10. dprotaso commented on Jan 11, 2024

    @dprotaso
    Member

    /unassign @nader-ziada

  11. yenniechen commented on May 28, 2024

    @yenniechen

    Kindly to ask if this issue still open?If is, I’m glad to try.

    @dprotaso @markusthoemmes

  12. yenniechen commented on May 28, 2024

    @yenniechen

    /assign

  13. Alexander-Kita commented on Nov 13, 2025

    @Alexander-Kita
    Contributor

    Does this issue still need work? I would be glad to attempt a solution @markusthoemmes

  14. evankanderson commented on Nov 15, 2025

    @evankanderson
    Member

    @dprotaso or @skonto would probably be better able to speak to the current situation.

  15. Alexander-Kita commented on Nov 20, 2025

    @Alexander-Kita
    Contributor

    /assign

  16. modified the milestones: Icebox, v1.22.0 on Jan 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

area/autoscalegood first issueDenotes an issue ready for a new contributor, according to the "help wanted" guidelines.kind/featureWell-understood/specified features, ready for coding.lifecycle/frozenIndicates that an issue or PR should not be auto-closed due to staleness.triage/acceptedIssues which should be fixed (post-triage)

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions