fix/feat: P0 bug fixes, type annotations, docstrings, and more tests

* fix/feat: P0 bug fixes, type annotations, docstrings, new tests, CI coverage

P0 fixes:
- prompts/gen_text2image.py: fix nanochart -> igenbench import
- eval_engine.py: guard response with isinstance(dict) before .get()
- vis_item.py: correct generation field type Optional[dict] -> Dict[str,Any]
- utils/io.py: rename senmantic->semantic, add boundary check with ValueError

P1 improvements:
- pyproject.toml: add authors, keywords, classifiers, project URLs
- pyproject.toml: add pytest-cov to dev deps
- CLI functions: add -> None return type annotations
- gen_engine.py: add PILImage return type to text2image()
- caller_registry.py: add docstring + correct return type annotation
- llm_caller.py: add docstrings to all three LLMCaller abstract methods
- client.py: add docstrings to call_text/image_generation/understanding
- ci.yml: add --cov flag and upload coverage artifact

New tests (41 total, up from 17):
- tests/test_io.py: 9 tests for extract_from_markdown + split_semantic
- tests/test_eval_engine.py: 6 tests for EvalEngine (mocked LLMClient)
- tests/test_caller_registry.py: 4 tests for @register_caller decorator
- tests/test_cli_score.py: 5 tests for igenbench score command

P2:
- AGENTS.md: expand deprecated Question -> EvalEntry migration guide

* fix: address Gemini review — defensive guards for None/empty LLM responses

- client.py: return '' early when provider returns None/empty in
  call_text_generation and call_image_understanding
- io.py: raise ValueError early when prompt is None or empty in
  split_semantic_and_data_in_t2i_prompt
- eval_engine.py: avoid literal 'None' string by checking response is
  not None before str() conversion
- test_io.py: add tests for empty string and None prompt inputs

---------

Co-authored-by: Claude <noreply@anthropic.com>

Author: MisterBrookT

.github/workflows/ci.yml

@@ -46,4 +46,11 @@ jobs:
         run: uv sync --dev
 
       - name: Run unit tests
-        run: uv run pytest tests/ -v --tb=short
+        run: uv run pytest tests/ -v --tb=short --cov=igenbench --cov-report=term-missing --cov-report=xml
+
+      - name: Upload coverage report
+        if: matrix.python-version == '3.12'
+        uses: actions/upload-artifact@v4
+        with:
+          name: coverage-report
+          path: coverage.xml

AGENTS.md

@@ -146,5 +146,5 @@ outputs/
 - **Never** add API keys to source files. Use environment variables (see `.env.example`).
 - `VISItem.from_dict(path)` takes a **file path**, not a dict.
 - `EvalEntry.judgments` is a list — multiple (gen_model, eval_model) pairs can coexist on the same question (multi-model comparison).
-- The `Question` dataclass in `vis_item.py` is **deprecated**; use `EvalEntry` instead.
+- The `Question` dataclass in `vis_item.py` is **deprecated** and will be removed in a future version. Migrate to `EvalEntry`: replace `q` → `question`, `q_ground` → `ground`, `q_type` → `question_type`, and use `VISItem.evaluation: list[EvalEntry]` instead of a separate JSONL questions file.
 - When editing CLI commands, register them via `igenbench/cli/main.py` imports (side-effect import pattern).

igenbench/cli/batch_cli.py

@@ -27,7 +27,7 @@ def cmd_batch_gen(
         "--resume/--no-resume",
         help="Resume from existing state, skipping already generated images",
     ),
-):
+) -> None:
     """Batch generate infographic images for all VISItem JSON files in a directory."""
     json_files = sorted(Path(data_dir).glob("*.json"))
 
@@ -79,7 +79,7 @@ def cmd_batch_eval(
         "--resume/--no-resume",
         help="Resume from existing state, skipping already evaluated questions",
     ),
-):
+) -> None:
     """Batch evaluate generated images for all VISItem JSON files in a directory."""
     json_files = sorted(Path(data_dir).glob("*.json"))
 

igenbench/cli/eval_cli.py

@@ -33,7 +33,7 @@ def cmd_run_evaluation(
         "--resume",
         help="Resume from output directory to skip already processed items",
     ),
-):
+) -> None:
     """Run evaluation on a generated image using pre-generated questions.
 
     Evaluates the generated image on all questions in the VISItem.

igenbench/cli/gen_cli.py

@@ -25,7 +25,7 @@ def cmd_gen(
         "--resume",
         help="Resume from output directory to skip already generated images",
     ),
-):
+) -> None:
     """Generate image from text prompt using text-to-image model.
 
     Reads the VISItem and generates an image based on the t2i_prompt field.

igenbench/cli/main.py

@@ -11,7 +11,7 @@
 import igenbench.cli.score_cli  # noqa: E402, F401
 
 
-def main():
+def main() -> None:
     """Entry point for the IGenBench CLI."""
     app()
 

igenbench/cli/score_cli.py

@@ -32,7 +32,7 @@ def cmd_score(
         "--by-type/--no-by-type",
         help="Show accuracy breakdown by question type",
     ),
-):
+) -> None:
     """Aggregate and display evaluation accuracy scores from the output directory."""
     output_path = Path(output_dir)
 

igenbench/engine/eval_engine.py

@@ -49,11 +49,18 @@ def judge_entry(
             model=eval_model, image_path=image_path, prompt=question_judgment_prompt
         )
 
+        if isinstance(response, dict):
+            analysis = response.get("analysis", "")
+            answer = response.get("answer", "")
+        else:
+            analysis = str(response) if response is not None else ""
+            answer = ""
+
         judgment = Judgment(
             eval_model=eval_model,
             gen_model=gen_model,
-            analysis=response.get("analysis", ""),
-            answer=response.get("answer", ""),
+            analysis=analysis,
+            answer=answer,
         )
 
         return judgment

igenbench/engine/gen_engine.py

@@ -1,3 +1,5 @@
+from PIL.Image import Image as PILImage
+
 from igenbench.engine.base_engine import BaseEngine
 from igenbench.utils.llm.client import LLMClient
 from igenbench.vis_item import VISItem
@@ -9,7 +11,7 @@ class GenEngine(BaseEngine):
     def __init__(self, llm_client: LLMClient, model: str):
         super().__init__(llm_client, model)
 
-    def text2image(self, item: VISItem):
+    def text2image(self, item: VISItem) -> PILImage:
         """Generate image from text prompt.
 
         Args:

igenbench/utils/io.py

@@ -100,8 +100,22 @@ def get_model_name_from_image_path(image_path: Path | str) -> str:
     return stem.split("_")[-1]
 
 
-def split_senmantic_and_data_in_t2i_prompt(t2i_prompt: str) -> tuple[str, str]:
-    """
-    Split the semantic and data parts of the T2I prompt.
+def split_semantic_and_data_in_t2i_prompt(t2i_prompt: str) -> tuple[str, str]:
+    """Split a T2I prompt into semantic description and data sections.
+
+    Expects the prompt to contain the separator "The given data is:".
+    Returns a (semantic_part, data_part) tuple.
+
+    Raises:
+        ValueError: If the separator is not found in the prompt.
     """
-    return t2i_prompt.split("The given data is:")
+    if not t2i_prompt:
+        raise ValueError("Prompt cannot be empty or None.")
+    separator = "The given data is:"
+    if separator not in t2i_prompt:
+        raise ValueError(
+            f"Prompt does not contain expected separator '{separator}'. "
+            "Cannot split semantic and data sections."
+        )
+    parts = t2i_prompt.split(separator, maxsplit=1)
+    return parts[0], parts[1]