Improve OpenRouter Support #144

sirwolfgang · 2025-04-29T23:38:53Z

This fixes issues with OpenRouter support by splitting the media handling off of OpenAI's implementation, which is a little different. As part of that it also improves the way Media is loaded by Chat to ensure each provider is running it's own version, allowing for each extension by others.

Currently this works with either, making this default a URI option.

assistant.ask "Summarize this document", with: { pdf: pdf_path }

assistant.ask "Summarize this document", with: { pdf: pdf_url }

This interface does limit us from being able to pass in contents itself. This interface should likely be expanded or made more explicit to support raw files or already base64 encoded files.

Additionally this PR exposes the ability to set OpenRouter's Header settings and Provider Settings:

  # --- Open Router Configuration ---
  config.openrouter_referer                     = 'https://rubyllm.com'
  config.openrouter_title                       = 'RubyLLM'
  config.openrouter_provider_order              = nil
  config.openrouter_provider_allow_fallbacks    = true
  config.openrouter_provider_require_parameters = false
  config.openrouter_provider_data_collection    = 'allow'
  config.openrouter_provider_ignore             = nil
  config.openrouter_provider_quantizations      = nil
  config.openrouter_provider_sort               = nil

sirwolfgang · 2025-04-30T16:57:47Z

@crmne What would you like to see to get this merged in? It improves OpenRouter support a lot, and I want to prevent someone else from duplicating work

crmne

Why does this PR changes anthropic, bedrock, deepseek, gemini, ollama, and openai too?

sirwolfgang · 2025-05-12T17:23:53Z

@crmne Because the current setup doesn't totally allow code reuse through extension, this tweaks it to allow that to work as needed while cleaning up some of the usage that was patched in.

We can remove these tweaks. My expectations is that you'd like to continue to be able to share code across providers, and continue to use this module_function style. At the intersection of these, when Provider A makes use of Provider B module it moves into B's namespace; Which means when it needs to call back to Provider A's version of something, it's unavailable and will call Provider B's version. This is likely going to lead to some unexpected bugs.

These changes copy Provider's B implantation into Provider A namespace, so that calls can be directed back to Provider A implantation when it has been defined.

crmne · 2025-05-14T13:14:39Z

@sirwolfgang could you please move these improvements in another PR? I'd like to merge the improved OpenRouter support but don't want to change half of the library for it.

sirwolfgang · 2025-05-14T15:41:21Z

@crmne I can move some of them, but not all of them. I made this changes because I was simply unable to extend the OpenAI code and have it work within the OpenRouter namespace. So I can remove the self:: and the deepseek stub, but that would leave part of the code fixed and part of the code base buggy.

Do you still wish for me to remove the improvements from the other files?

crmne · 2025-05-20T20:53:36Z

Yes.

sirwolfgang · 2025-05-20T21:21:58Z

Updated

crmne · 2025-07-16T18:05:38Z

lib/ruby_llm/providers/openai/chat.rb

-              content: Media.format_content(msg.content),
+              content: self::Media.format_content(msg.content),


Was this necessary?

Yes, as previously explained, the module-based system has scoping issues. This file is deeply called by the other modules, so in order to have this file call the correct version of the Media file, which is not the OpenAI version, it needs to be scoped to the current runtime.

I attempted to make this fix universal, so that other people don't run into this bug; you asked me to roll it back where it wasn't necessary. So I did, but I can not roll it back in this location if I want the correct version of the Media object to be invoked at runtime.

Thanks for the explanation, makes sense!

crmne · 2025-07-16T18:08:41Z

lib/ruby_llm/providers/openrouter/chat.rb

Why you needed to implement all this? Check the Ollama provider, it only implements the messages that override what OpenAI already provides

crmne · 2025-07-16T18:09:27Z

lib/ruby_llm/providers/openrouter/media.rb

Why you needed to implement this as well? Check the Ollama provider, it only implements the messages that override what OpenAI already provides. And here there's nothing regarding the other provider options.

OpenRouter uses a different format than OpenAI in order to support additional functionality. This is to support that.

We have tests that work for OpenRouter media, including images, PDFs, etc.

crmne · 2025-07-16T18:10:19Z

spec/ruby_llm/chat_pdf_spec.rb

Why there is this test? Shouldn't there be simply a test with the options that you are implementing in this PR?

My implementation in this PR affects the interface of usage. This test is to ensure that both this PR and future PRs do not break functionality. Generally, increasing test coverage is considered desirable.

What exactly is affected in the interface? Again, there are already comprehensive tests for PDFs and other file formats already.

crmne · 2025-07-16T18:10:26Z

..._paths_behaves_like_pdf_models_anthropic_claude-3-5-haiku-20241022_handles_multiple_pdfs.yml

crmne · 2025-07-16T18:10:37Z

...at_pdf_model_with_paths_behaves_like_pdf_models_gemini_gemini-2_0-flash_understands_pdfs.yml

crmne · 2025-07-16T18:10:40Z

...t_pdf_model_with_paths_behaves_like_pdf_models_openai_gpt-4_1-nano_handles_multiple_pdfs.yml

crmne · 2025-07-16T18:10:44Z

...s/chat_pdf_model_with_paths_behaves_like_pdf_models_openai_gpt-4_1-nano_understands_pdfs.yml

crmne · 2025-07-16T18:10:48Z

...aths_behaves_like_pdf_models_openrouter_anthropic_claude-3_5-haiku_handles_multiple_pdfs.yml

crmne · 2025-07-16T18:10:52Z

...ith_paths_behaves_like_pdf_models_openrouter_anthropic_claude-3_5-haiku_understands_pdfs.yml

sirwolfgang marked this pull request as ready for review April 29, 2025 23:40

sirwolfgang force-pushed the main branch 4 times, most recently from 696e779 to 186998a Compare April 30, 2025 16:38

sirwolfgang changed the title ~~Improve OpenRouter Media Support~~ Improve OpenRouter Support Apr 30, 2025

sirwolfgang force-pushed the main branch from 186998a to cc232a1 Compare April 30, 2025 16:59

sirwolfgang mentioned this pull request May 2, 2025

Structured Output & JSON mode response support #131

Closed

crmne requested changes May 10, 2025

View reviewed changes

sirwolfgang force-pushed the main branch from cc232a1 to 5186300 Compare May 12, 2025 17:16

sirwolfgang requested a review from crmne May 12, 2025 17:18

sirwolfgang force-pushed the main branch from 5186300 to 3b5fac5 Compare May 12, 2025 17:57

sirwolfgang added 4 commits May 20, 2025 14:19

Improve OpenRouter Media Support

c7ff2c9

Add the ability to set OpenRouter headers

3f24270

Add the ability to configure OpenRouter provider settings

6e68ca4

Rollback module interoperability fixes

73c7784

sirwolfgang force-pushed the main branch from 3b5fac5 to 73c7784 Compare May 20, 2025 21:21

Merge branch 'main' into main

540d102

crmne requested changes Jul 16, 2025

View reviewed changes

crmne added the enhancement New feature or request label Jul 16, 2025

sirwolfgang mentioned this pull request Aug 5, 2025

replace specific model wrappers with RubyLLM gem activeagents/activeagent#154

Open

		content: Media.format_content(msg.content),
		content: self::Media.format_content(msg.content),

Uh oh!

Improve OpenRouter Support #144

Are you sure you want to change the base?

Improve OpenRouter Support #144

Conversation

sirwolfgang commented Apr 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sirwolfgang commented Apr 30, 2025

Uh oh!

crmne left a comment

Choose a reason for hiding this comment

Uh oh!

sirwolfgang commented May 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

crmne commented May 14, 2025

Uh oh!

sirwolfgang commented May 14, 2025

Uh oh!

crmne commented May 20, 2025

Uh oh!

sirwolfgang commented May 20, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sirwolfgang commented Apr 29, 2025 •

edited

Loading

sirwolfgang commented May 12, 2025 •

edited

Loading