Skip to content

feat(lua): add true parallel LLM execution API for concurrent scripting workflows #1282

Open

Description

@EugenesDad

Context & Problem

Currently, Lua scripting in RisuAI effectively runs LLM calls sequentially (e.g., the LLM and axLLM patterns). For advanced workflows, scripts need to trigger multiple model calls simultaneously and merge the results. Relying on sequential calls significantly increases latency and blocks these advanced use cases.

Goal

Add a new Lua API for concurrent LLM execution, allowing multiple requests to run in parallel and return their results together.

Use Cases

I have developed an advanced preset and found I need this feature.

  • The Workflow: The script triggers multiple axLLM calls to extract and distill key information into variables. This creates a much richer and more accurate context for the primary LLM’s final response.
  • The Bottleneck: Currently, these extraction calls must run one by one, causing a noticeable delay before the user sees a response.
  • Another Function I hope: PR #1042: fix upsertLocalLoreBook. I want to provide the user with visible data in Lorebook.

Since vibe coding is not accepted on PR, I choose to submit as issues.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions