transformerlab-client

Python client and callbacks for Transformer Lab.

Install

pip install transformerlab-client

Prerequisites

• Python 3.10 or newer
• A running instance of the Transformer Lab API server
  • By default, the client connects to http://localhost:8338
  • Make sure the Transformer Lab application is running before using this client

Usage

from transformers import TrainingArguments, Trainer
from transformerlab_client.client import TransformerLabClient
from transformerlab_client.callbacks.hf_callback import TLabProgressCallback

# Initialize client and register job
client = TransformerLabClient(server_url="<ENTER YOUR TRANSFORMER LAB API URL>")
job_id = client.start(your_config)

# Set up Hugging Face trainer with TLabProgressCallback
training_args = TrainingArguments(
    output_dir="./output",
    # other arguments...
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    # other arguments...
    callbacks=[TLabProgressCallback(client)]  # Add Transformer Lab callback
)

# Train the model
trainer.train()

# Complete the job
client.complete()

Full Training Example

See the examples directory for a complete training script that demonstrates how to use the client for a full training workflow with Hugging Face Transformers.

API Reference

TransformerLabClient

Main client for communicating with Transformer Lab.

• start(config): Register a training job and get a job ID
• report_progress(progress, metrics=None): Report training progress (0-100) and any metrics in json format
• complete(message="Training completed successfully"): Mark job as complete
• stop(message="Training stopped"): Mark job as stopped
• save_model(saved_model_path): Save model to specified path
• log_info(message): Log info message
• log_error(message): Log error message
• log_warning(message): Log warning message
• log_debug(message): Log debug message

TLabProgressCallback

Callback for Hugging Face Transformers Trainer that reports progress to Transformer Lab.