Fix/minimise logs

[peshence]

Description

Contributor Declaration

By opening this pull request, I affirm the following:

• All authors agree to the Contributor License Agreement.
• The code follows the project's coding standards.
• I have performed self-review and added comments where needed.
• I have added or updated tests to verify that my changes are effective and functional.
• I have run all existing tests and confirmed they pass.

[Copilot]

Pull request overview

This pull request reduces logging verbosity by changing numerous logging.info() calls to logging.debug() for routine operational messages, removes some verbose logging statements entirely, and introduces security improvements by obfuscating API keys and tokens in logs. It also refactors subprocess output handling to use buffering instead of immediate line-by-line logging, and adds a caching decorator to the authentication method to improve performance.

Changes:

• Downgraded logging levels from info to debug for routine operational status updates across worker, datasource, request, staging, and metric components
• Added API key obfuscation when logging environment variables and MARS credentials
• Refactored subprocess output handling to use buffering with new helper methods
• Added 60-second caching to authotron.authenticate method
• Added optional logging parameter to RequestAccepted response handler
• Removed unused check_extra_roles method from polytope datasource

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
polytope_server/worker/worker.py	Changed logging levels to debug and removed verbose worker status logging block
polytope_server/frontend/common/flask_decorators.py	Added optional info_log parameter to RequestAccepted to allow debug-level logging
polytope_server/frontend/common/data_transfer.py	Used new optional logging parameter to reduce verbosity for query requests
polytope_server/common/user.py	Changed authorization success logging to debug level
polytope_server/common/subprocess.py	Major refactoring: added buffering for output, obfuscated API keys in env logging, fixed env parameter handling
polytope_server/common/staging/s3_staging.py	Changed bucket initialization logging to debug level
polytope_server/common/request.py	Changed request status update logging to debug level
polytope_server/common/metric_calculator/mongo.py	Changed metric calculator initialization and index creation logging to debug level
polytope_server/common/logging.py	Changed logging initialization message to debug level
polytope_server/common/io/fifo.py	Changed FIFO creation logging to debug level
polytope_server/common/datasource/polytope.py	Removed unused check_extra_roles method and changed result retrieval logging to debug
polytope_server/common/datasource/mars.py	Changed various MARS operation logs to debug and added API key obfuscation
polytope_server/common/datasource/datasource.py	Changed datasource initialization logging to debug level
polytope_server/common/authotron.py	Added cache decorator and changed authentication logging to debug level

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.