Integrate Nscale-cloud into HuggingFace inference

[nbarr07]

Integrate Nscale provider for HuggingFace Inference

This PR adds support for Nscale inference to the HuggingFace inference API.

Note that our inference service is not publicly live yet but will be soon - this draft PR is for review and preparation.

• Implemented standard provider integration for nscale
• Added support for text-generation, conversational, and text-to-image tasks
• Included tests following the established patterns

The tests were all passing when I tried. Any feedback are welcomed!

[Wauplin]

Hey @nbarr07 , thanks for your PR :) Even though we are looking forward to integrate new providers, we'll have to hold on for a bit on this one. Integrating a new provider takes time and effort on our side to make things right so for now we've decided to focus on the existing ones. Will let you know when we are ready to move forward! 🤗

[julien-c]

Hi @nbarr07 we are currently finishing a refactoring of Inference Providers integration code in #1315, this should be merged soon, but we will need to rewrite part of your implementation (should be even simpler to integrate), will ping again after it's been merged.

[nbarr07]

Hi @julien-c , thanks for the heads-up about the refactoring, if there's anything we can do on our side let me know!

[hanouticelina]

Hi @nbarr07,
We've merged a refactoring for Inference Provider integration into main, which should make adding new providers much easier.
Could you merge main into your branch and update your PR accordingly? it should be relatively straightforward with the new structure:
1 - You have to update the PROVIDERS mapping here: inference/src/lib/getProviderHelper.ts#L49 and add nscale-cloud (let's ensure we respect the alphabetical order) :

import * as NScaleCloud from "../providers/nscale-cloud";
...
export const PROVIDERS: Record<InferenceProvider, Partial<Record<InferenceTask, TaskProviderHelper>>> = {
	...
	"nscale-cloud": {
		"text-to-image": new NScaleCloud.NScaleCloudTextToImageTask(),
		"conversational": new NScaleCloud.NScaleCloudConversationalTask(),
		"text-generation": new NScaleCloud.NScaleCloudTextGenerationTask(),
	},
	},
        ...

2 - Update packages/inference/src/providers/nscale-cloud.ts to implement the classes:

import { BaseConversationalTask, BaseTextGenerationTask } from "./providerHelper";

const NSCALE_API_BASE_URL = "https://inference.api.nscale.com";

export class NScaleCloudConversationalTask extends BaseConversationalTask {
	constructor() {
		super("nscale-cloud", NSCALE_API_BASE_URL);
	}
}

export class NScaleCloudTextGenerationTask extends BaseTextGenerationTask {
	constructor() {
		super("nscale-cloud", NSCALE_API_BASE_URL);
	}

	override makeRoute(): string {
		return "v1/chat/completions";
	}
}

export class NScaleCloudTextToImageTask extends TaskProviderHelper implements TextToImageTaskHelper {
	constructor() {
		super("nscale-cloud", NSCALE_API_BASE_URL);
	}

	makeRoute(): string {
		return "v1/images/generations";
	}

	preparePayload(params: BodyParams): Record<string, unknown> {
		// refer to Nebius implementation of preparePayload
	}
         
         getResponse(
		response: unknown,
		url?: string,
		headers?: HeadersInit,
		outputType?: "url" | "blob"
	): Promise<string | Blob>{
		// Refer to Nebius implementation of getResponse
	}
}

You can check Nebius implementation of text-to-image, it looks like it's the same payload and response format as NScale-cloud.
let us know if you need any help! you can find more details in the documentation : https://huggingface.co/docs/inference-providers/register-as-a-provider#2-js-client-integration.

[nbarr07]

Hey @hanouticelina,
thanks for the heads up, I've merged the latest changes into my branch and updated the PR. One question I have is does the provider name in inference need to march the organisation username on Huggingface or can they be different?
Let me know if there's any adjustments needed.

[julien-c]

it's better if they're the same, do you want us to check if we can move your org to nscale namespace?

[nbarr07]

yes that would be great, thanks!

[SBrandeis]

Hi there - thank you for your conribution!

I have two minor comments - otherwise the PR looks good !

Ping @hanouticelina for a quick validation before merging

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[hanouticelina]

looks good, thank you @nbarr07!

[nbarr07]

@hanouticelina @SBrandeis thank you for the review and changes!
I've just made a last change in the readme (the namespace had to be modified). Lmk when we can merge it, thanks!

[nbarr07]

Hey @hanouticelina @SBrandeis , just to get an idea, when could we expect to merge the PR?

[hanouticelina]

let's merge this, thank you @nbarr07 again for the contribution!