Skip to content

store and load model connection config #119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
kaushik-himself merged 12 commits into from
Jun 2, 2025
Merged

store and load model connection config #119

kaushik-himself merged 12 commits into from
Jun 2, 2025

Conversation

@kaushik-himself
Copy link
Contributor

@kaushik-himself kaushik-himself commented May 31, 2025 •
edited
Loading

Description

  • What is this PR about?

when the user sets up the config, the config is saved to a local filesystem instance of tinydb.

this PR creates the following:

  • add tinydb and instantiate it
  • DB CRUD
  • integration into dashboard code:
  • save, update and run tests against model config stored in the DB
  • store and see past run reports

support for cli is now deprecated except to start the dashboard.
compliant-llm test command has not been tested and its associated test has been disabled.

README and docs have been updated. they instruct the user to use the dashboard to configure models and run tests.

Screenshots/Results

  • Add a screenshot of the changes or code responses

Reference Links

  • Links to JIRA issue, links, Slack discussions, etc.

Checklist

This PR includes the following (tick all that apply):

  • Tests added/updated for new/changed functionality
  • Bug Fix (explain the bug in the description)
  • Refactoring or optimizations (no functional changes)
  • Documentation updated (README, docstrings, etc.)
  • Build or deployment related changes
  • Dependency updates (requirements.txt/pyproject.toml)
  • Setup change (update README/setup instructions if required)
  • Code style checks passed (PEP8, flake8, black, etc.)
  • Type hints added/updated (if applicable)
  • Pre-commit hooks run (if configured)

kaushik-himself
kaushik-himself commented Jun 2, 2025
View reviewed changes
ui/dashboard.py Outdated
for i, report_path in enumerate(selected_profile['past_runs']):
try:
# Load report data
print("Rport Path", report_path)
Copy link
Contributor Author

@kaushik-himself kaushik-himself Jun 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove print

@kaushik-himself
fix provider name
bcea970 
the provider name was incorrectly set to the model name after the config changes.
updates this to take the provider_name/model_name as the provider param for litellm.

fixes bug: when the llm response is `None`, a few attacks are failing due to unsafe
handling of `None` value.

removes excessive or repetitive logging
@kaushik-himself
fix tests
962f243 
update requirements-lock.txt with the updated dependencies.
skip the cli test: will remove docs for the test run using the cli.
cli run is broken and won't be supported anymore.

fixes the broken test runner.
@kaushik-himself
add pytest-asyncio to pyproject.toml
8ed547b 
to fix build error on github actions
@kaushik-himself
update README and docs
398aa88
@kaushik-himself kaushik-himself changed the title add tinydb and a model config store store and load model connection config Jun 2, 2025
@kaushik-himself kaushik-himself merged commit b9b15e4 into main Jun 2, 2025
3 checks passed
@kaushik-himself kaushik-himself deleted the feature/tinydb branch June 2, 2025 23:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@kaushik-himself @vinikatyal