Skip to content

Conversation

jamesbraza
Copy link
Collaborator

This PR clears up some fuzziness in our evidence accounting:

  • Removes irrelevant evidence, instead of continually filtering them out
    • Comes with the benefit of much smaller PQASession
  • Updates Best evidence(s) message for clarity
    • Including the 'for the current question', to make it clear the best evidence varies per question
    • Using a bulleted list over a numbered list, since order is defined by score (not element index)

@jamesbraza jamesbraza self-assigned this Sep 27, 2025
@Copilot Copilot AI review requested due to automatic review settings September 27, 2025 00:07
@jamesbraza jamesbraza added the bug Something isn't working label Sep 27, 2025
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. enhancement New feature or request labels Sep 27, 2025
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR improves evidence management in the PQA (Paper Question Answering) system by filtering out irrelevant evidence contexts early and updating the evidence display format for better clarity.

  • Filters contexts with score ≤ 0 at collection time instead of repeatedly filtering during processing
  • Updates evidence display to use bullet points instead of numbered lists and clarifies scope with "for the current question"
  • Removes test code that artificially manipulated context scores with random values

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
src/paperqa/docs.py Adds filtering to exclude contexts with score ≤ 0 during evidence collection
src/paperqa/agents/tools.py Updates evidence display format and removes score filtering (now handled upstream)
tests/test_paperqa.py Removes random import and updates tests to work with new evidence filtering behavior
tests/test_agents.py Updates test assertions to expect bullet point format instead of numbered lists

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Copy link

dosubot bot commented Sep 27, 2025

Related Documentation

Checked 1 published document(s). No updates required.

How did I do? Any feedback?  Join Discord

top_contexts = "\n".join(
top_contexts = "\n\n".join(
[
f"{n + 1}. {sc.context}\n"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note there was trailing whitespace here previously, now there's not

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Sep 27, 2025
@jamesbraza jamesbraza merged commit 4ef5c87 into main Sep 27, 2025
5 checks passed
@jamesbraza jamesbraza deleted the fixing-context-gap branch September 27, 2025 03:23
Copy link

dosubot bot commented Sep 27, 2025

Documentation Updates

Checked 1 published document(s). No updates required.

How did I do? Any feedback?  Join Discord

@jamesbraza jamesbraza mentioned this pull request Oct 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request lgtm This PR has been approved by a maintainer size:M This PR changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants