Skip to content

Commit

Permalink
Allow generative models with the document QA widget #298 (#354)
Browse files Browse the repository at this point in the history
Fixes #298 

This PR enhances generative QA widget support. By making the score field
optional in the output structure, it improves flexibility for scenarios
without scores. The validation and parsing functions align with the
updated structure, ensuring accuracy. Additionally, the user interface
now displays scores only when available, enhancing the user experience.
These changes provide a more adaptable and user-friendly widget.

Test screenshots attached

`impira/layoutlm-document-qa`
<img width="499" alt="impira:layoutlm-document-qa"
src="https://github.com/huggingface/huggingface.js/assets/22408440/8922301e-f3f1-475d-8bae-c4023052c81c">


`naver-clova-ix/donut-base-finetuned-docvqa`
<img width="484" alt="naver-clova-ix:donut-base-finetuned-docvga"
src="https://github.com/huggingface/huggingface.js/assets/22408440/26d501f5-4cd6-4a0b-8f35-b34a14be9b4e">

---------

Co-authored-by: Eliott C <[email protected]>
  • Loading branch information
kohsheen1234 and coyotte508 authored Nov 28, 2023
1 parent cf06145 commit dc0ce01
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 9 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,13 @@ from-yellow-400 to-yellow-200 dark:from-yellow-400 dark:to-yellow-600
type LabelField = "label" | "answer";
export let labelField: LabelField = "label";
export let output: Array<
{ label: string; score: number; color?: string } | { answer: string; score: number; color?: string }
{ label: string; score?: number; color?: string } | { answer: string; score?: number; color?: string }
> = [];
export let highlightIndex = -1;
export let mouseover: (index: number) => void = () => {};
export let mouseout: () => void = () => {};
$: scoreMax = Math.max(0, ...output.map((x) => x.score));
$: scoreMax = Math.max(0, ...output.map((x) => x.score ?? 0));
function text(outputItem: (typeof output)[0]) {
if (labelField in outputItem) {
Expand Down Expand Up @@ -54,11 +54,13 @@ from-yellow-400 to-yellow-200 dark:from-yellow-400 dark:to-yellow-600
to-{color ?? defaultBarColor}-200
dark:from-{color ?? defaultBarColor}-400
dark:to-{color ?? defaultBarColor}-600"
style={`width: ${Math.ceil((score / scoreMax) * 100 * 0.8)}%;`}
style={`width: ${score ? Math.ceil((score / scoreMax) * 100 * 0.8) : 0}%;`}
/>
<span class="leading-snug">{text(output[index])}</span>
</div>
<span class="pl-2">{score.toFixed(3)}</span>
{#if typeof score === "number"}
<span class="pl-2">{score.toFixed(3)}</span>
{/if}
</div>
{/each}
</div>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
isLoading: false,
estimatedTime: 0,
};
let output: Array<{ answer: string; score: number }> | null = [];
let output: Array<{ answer: string; score?: number }> | null = [];
let outputJson: string;
let question = "";
let imgSrc = "";
Expand Down Expand Up @@ -55,15 +55,18 @@
});
}
function isValidOutput(arg: any): arg is { answer: string; score: number }[] {
return Array.isArray(arg) && arg.every((x) => typeof x.answer === "string" && typeof x.score === "number");
function isValidOutput(arg: any): arg is { answer: string; score?: number }[] {
return (
Array.isArray(arg) &&
arg.every((x) => typeof x.answer === "string" && (typeof x.score === "number" || x.score === undefined))
);
}
function parseOutput(body: unknown): Array<{ answer: string; score: number }> {
function parseOutput(body: unknown): Array<{ answer: string; score?: number }> {
if (isValidOutput(body)) {
return body;
}
throw new TypeError("Invalid output: output must be of type Array<answer: string, score:number>");
throw new TypeError("Invalid output: output must be of type Array<{ answer: string, score?: number }>");
}
async function applyInputSample(sample: WidgetExampleAssetAndTextInput, opts: ExampleRunOpts = {}) {
Expand Down

0 comments on commit dc0ce01

Please sign in to comment.