Merge branch 'develop' v0.3.2

Author: Bob Strahan

CHANGELOG.md

@@ -5,6 +5,26 @@ SPDX-License-Identifier: MIT-0
 
 ## [Unreleased]
 
+## [0.3.2]
+
+### Added
+
+- **Cost Estimator UI Feature for Context Grouping and Subtotals**
+  - Added context grouping functionality to organize cost estimates by logical categories (e.g. OCR, Classification, etc.)
+  - Implemented subtotal calculations for better cost breakdown visualization
+
+- **DynamoDB Caching for Resilient Classification**
+  - Added optional DynamoDB caching to the multimodal page-level classification service to improve efficiency and resilience
+  - Cache successful page classification results to avoid redundant processing during retries when some pages fail due to throttling
+  - Exception-safe caching preserves successful work even when individual threads or the overall process fails
+  - Configurable via `cache_table` parameter or `CLASSIFICATION_CACHE_TABLE` environment variable
+  - Cache entries scoped to document ID and workflow execution ARN with automatic TTL cleanup (24 hours)
+  - Significant cost reduction and improved retry performance for large multi-page documents
+
+### Fixed
+- "Use as Evaluation Baseline" incorrectly sets document status back to QUEUED. It should remain as COMPLETED.
+
+
 ## [0.3.1]
 
 ### Added

VERSION

@@ -1 +1 @@
-0.3.1
+0.3.2

config_library/pattern-2/few_shot_example_with_multimodal_page_classification/config.yaml

@@ -74,7 +74,7 @@ classes:
               "signature": "Will E. Clark",
               "cc": null,
               "reference_number": "TNJB 0008497"
-        imagePath: config_library/pattern-2/few_shot_example/example-images/letter1.jpg
+        imagePath: config_library/pattern-2/few_shot_example_with_multimodal_page_classification/example-images/letter1.jpg
       - classPrompt: This is an example of the class 'letter'
         name: Letter2
         attributesPrompt: |-
@@ -89,7 +89,7 @@ classes:
               "signature": "Bill",
               "cc": null,
               "reference_number": null
-        imagePath: config_library/pattern-2/few_shot_example/example-images/letter2.png
+        imagePath: config_library/pattern-2/few_shot_example_with_multimodal_page_classification/example-images/letter2.png
   - name: form
     description: >-
       A structured document with labeled fields, checkboxes, or blanks requiring
@@ -464,7 +464,7 @@ classes:
               "priority": null,
               "thread_id": null,
               "message_id": null
-        imagePath: config_library/pattern-2/few_shot_example/example-images/email1.jpg
+        imagePath: config_library/pattern-2/few_shot_example_with_multimodal_page_classification/example-images/email1.jpg
   - name: questionnaire
     description: >-
       A survey instrument containing numbered questions with multiple choice,
@@ -636,7 +636,7 @@ classes:
               "account_name": ["Checking", "Savings"],
               "account_number": ["003525801543","352580154336"],
               "transactions": [{"Date": "2/6/2020", "Description": "Food Purchase - AnyCompany Restaurant - 1194989245", "Amount": "-171"}]
-        imagePath: config_library/pattern-2/few_shot_example/example-images/bank-statement-pages/
+        imagePath: config_library/pattern-2/few_shot_example_with_multimodal_page_classification/example-images/bank-statement-pages/
 
 classification:
   classificationMethod: multimodalPageLevelClassification
@@ -672,11 +672,11 @@ classification:
     {CLASS_NAMES_AND_DESCRIPTIONS}
 
 
-    <few_shot_examples> 
+    <few_shot_example_with_multimodal_page_classifications> 
 
-    {FEW_SHOT_EXAMPLES}
+    {few_shot_example_with_multimodal_page_classificationS}
 
-    </few_shot_examples>
+    </few_shot_example_with_multimodal_page_classifications>
 
 
     <<CACHEPOINT>>

lib/idp_common_pkg/idp_common/bedrock/client.py

@@ -78,7 +78,8 @@ def __call__(
         top_k: Optional[Union[float, str]] = None,
         top_p: Optional[Union[float, str]] = None,
         max_tokens: Optional[Union[int, str]] = None,
-        max_retries: Optional[int] = None
+        max_retries: Optional[int] = None,
+        context: str = "Unspecified"
     ) -> Dict[str, Any]:
         """
         Make the instance callable with the same signature as the original function.
@@ -109,7 +110,8 @@ def __call__(
             top_k=top_k,
             top_p=top_p,
             max_tokens=max_tokens,
-            max_retries=effective_max_retries
+            max_retries=effective_max_retries,
+            context=context
         )
 
     def _preprocess_content_for_cachepoint(self, content: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
@@ -177,7 +179,8 @@ def invoke_model(
         top_k: Optional[Union[float, str]] = 5,
         top_p: Optional[Union[float, str]] = 0.1,
         max_tokens: Optional[Union[int, str]] = None,
-        max_retries: Optional[int] = None
+        max_retries: Optional[int] = None,
+        context: str = "Unspecified"
     ) -> Dict[str, Any]:
         """
         Invoke a Bedrock model with retry logic.
@@ -335,7 +338,8 @@ def invoke_model(
             converse_params=converse_params,
             retry_count=0,
             max_retries=effective_max_retries,
-            request_start_time=request_start_time
+            request_start_time=request_start_time,
+            context=context
         )
 
         return result
@@ -346,7 +350,8 @@ def _invoke_with_retry(
         retry_count: int,
         max_retries: int,
         request_start_time: float,
-        last_exception: Exception = None
+        last_exception: Exception = None,
+        context: str = "Unspecified"
     ) -> Dict[str, Any]:
         """
         Recursive helper method to handle retries for Bedrock invocation.
@@ -424,7 +429,7 @@ def _invoke_with_retry(
             response_with_metering = {
                 "response": response,
                 "metering": {
-                    f"bedrock/{converse_params['modelId']}": {
+                    f"{context}/bedrock/{converse_params['modelId']}": {
                         **usage
                     }
                 }
@@ -470,7 +475,8 @@ def _invoke_with_retry(
                     retry_count=retry_count + 1,
                     max_retries=max_retries,
                     request_start_time=request_start_time,
-                    last_exception=e
+                    last_exception=e,
+                    context=context
                 )
             else:
                 logger.error(f"Non-retryable Bedrock error: {error_code} - {error_message}")
@@ -838,6 +844,7 @@ def _sanitize_response_for_logging(self, response: Dict[str, Any]) -> Dict[str,
     top_p: Optional top_p parameter (float or string)
     max_tokens: Optional max_tokens parameter (int or string)
     max_retries: Optional override for the instance's max_retries setting
+    context: Context prefix for metering key (default: "Unspecified")
     
 Returns:
     Bedrock response object with metering information

lib/idp_common_pkg/idp_common/classification/README.md

@@ -16,6 +16,7 @@ This module provides document classification capabilities for the IDP Accelerato
 - Structured data models for results
 - Grouping of pages into sections by classification
 - Comprehensive error handling and retry mechanisms
+- **DynamoDB caching for resilient page-level classification**
 
 ## Usage Example
 
@@ -226,6 +227,136 @@ def handler(event, context):
 - `ClassificationResult`: Overall result of a classification operation
 - `Document`: Core document data model used throughout the IDP pipeline
 
+## DynamoDB Caching for Resilient Classification
+
+The classification service now supports optional DynamoDB caching to improve efficiency and resilience when processing documents with multiple pages. This feature addresses throttling scenarios where some pages succeed while others fail, avoiding the need to reclassify already successful pages on retry.
+
+### How It Works
+
+1. **Cache Check**: Before processing, the service checks for cached classification results for the document
+2. **Selective Processing**: Only pages without cached results are classified
+3. **Exception-Safe Caching**: Successful page results are cached even when other pages fail
+4. **Retry Efficiency**: Subsequent retries only process previously failed pages
+
+### Configuration
+
+#### Via Constructor Parameter
+```python
+from idp_common import classification, get_config
+
+config = get_config()
+service = classification.ClassificationService(
+    region="us-east-1",
+    config=config,
+    backend="bedrock",
+    cache_table="classification-cache-table"  # Enable caching
+)
+```
+
+#### Via Environment Variable
+```bash
+export CLASSIFICATION_CACHE_TABLE=classification-cache-table
+```
+
+```python
+# Cache table will be automatically detected from environment
+service = classification.ClassificationService(
+    region="us-east-1",
+    config=config,
+    backend="bedrock"
+)
+```
+
+### DynamoDB Table Schema
+
+The cache uses the following DynamoDB table structure:
+
+- **Primary Key (PK)**: `classcache#{document_id}#{workflow_execution_arn}`
+- **Sort Key (SK)**: `none`
+- **Attributes**:
+  - `page_classifications` (String): JSON-encoded successful page results
+  - `cached_at` (String): Unix timestamp of cache creation
+  - `document_id` (String): Document identifier
+  - `workflow_execution_arn` (String): Workflow execution ARN
+  - `ExpiresAfter` (Number): TTL attribute for automatic cleanup (24 hours)
+
+#### Example DynamoDB Item
+```json
+{
+  "PK": "classcache#doc-123#arn:aws:states:us-east-1:123456789012:execution:MyWorkflow:abc-123",
+  "SK": "none",
+  "page_classifications": "{\"1\":{\"doc_type\":\"invoice\",\"confidence\":1.0,\"metadata\":{\"metering\":{...}},\"image_uri\":\"s3://...\",\"text_uri\":\"s3://...\",\"raw_text_uri\":\"s3://...\"},\"2\":{...}}",
+  "cached_at": "1672531200",
+  "document_id": "doc-123",
+  "workflow_execution_arn": "arn:aws:states:us-east-1:123456789012:execution:MyWorkflow:abc-123",
+  "ExpiresAfter": 1672617600
+}
+```
+
+### Benefits
+
+- **Cost Reduction**: Avoids redundant API calls to Bedrock/SageMaker for already-classified pages
+- **Improved Resilience**: Handles partial failures gracefully during concurrent processing
+- **Faster Retries**: Subsequent attempts only process failed pages, not the entire document
+- **Automatic Cleanup**: TTL ensures cache entries don't accumulate indefinitely
+- **Thread Safety**: Safe for concurrent page processing within the same document
+
+### Example: Resilient Processing Flow
+
+```python
+from idp_common import classification, get_config
+from idp_common.models import Document
+
+config = get_config()
+service = classification.ClassificationService(
+    region="us-east-1",
+    config=config,
+    backend="bedrock",
+    cache_table="classification-cache-table"
+)
+
+# Create document with 5 pages
+document = Document(
+    id="doc-123",
+    workflow_execution_arn="arn:aws:states:us-east-1:123456789012:execution:MyWorkflow:abc-123",
+    pages={
+        "1": {...},
+        "2": {...},
+        "3": {...},
+        "4": {...},
+        "5": {...}
+    }
+)
+
+try:
+    # First attempt: pages 1,2,4 succeed, pages 3,5 fail due to throttling
+    document = service.classify_document(document)
+except Exception as e:
+    # Pages 1,2,4 are cached automatically before exception is raised
+    print(f"Classification failed: {e}")
+
+try:
+    # Retry: only pages 3,5 are processed (1,2,4 loaded from cache)
+    document = service.classify_document(document)
+    print("Document classified successfully on retry")
+except Exception as e:
+    print(f"Retry failed: {e}")
+```
+
+### Cache Lifecycle
+
+1. **Creation**: Cache entries are created when `classify_document()` completes successfully or encounters exceptions
+2. **Retrieval**: Cache is checked at the start of each `classify_document()` call
+3. **Update**: Cache entries are updated with new successful results from each processing attempt
+4. **Expiration**: Entries automatically expire after 24 hours via DynamoDB TTL
+
+### Important Notes
+
+- Caching only applies to the `classify_document()` method, not individual `classify_page()` calls
+- Cache entries are scoped to specific document and workflow execution combinations
+- Only successful page classifications (without errors in metadata) are cached
+- The cache is transparent - existing code continues to work without modifications
+
 ## Backend Options
 
 ### Bedrock Backend