Semantic Search w/Chunking example for 2.17?

[howdymatt]

The tutorial laid out in docs/tutorials/ml_inference/semantic_search/semantic_search_for_long_document.md only supports 2.19 and later. I am using AWS OS 2.17 (their latest available version) and I am unable to run this example.

Can someone help me with a 2.17-compatible version that produces the same query responses for chunked documents? Right now I'm using a hugging face model but will move to a bedrock connector, so I'm looking for an example that could run using either model.

[yuye-aws]

Hi @howdymatt . Can you please provide more details? Like which step you are getting what kind of error? And are you using Opensource OpenSearch or Amazon OpenSearch Service?

[howdymatt]

When simulating a doc ingest using the pipeline created in step 2. Here is my call (I've removed the IDs here)

PUT _ingest/pipeline/text-chunking-embedding-ingest-pipeline
{
  "description": "Text chunking embedding ingest pipeline",
  "processors": [
    {
      "text_chunking": {
        "algorithm": {
          "fixed_token_length": {
            "token_limit": 512,
            "overlap_rate": 0.2,
            "tokenizer": "standard"
          }
        },
        "field_map": {
          "passage_text": "passage_chunk"
        }
      }
    },
    {
      "foreach": {
        "field": "passage_chunk",
        "processor": {
          "text_embedding": {
            "model_id": "MODEL_ID_HERE",
            "field_map": {
              "passage_chunk": "passage_chunk_embedding"
            }
          }
        }
      }
    }
  ]
}

POST _ingest/pipeline/text-chunking-embedding-ingest-pipeline/_simulate
{
  "docs": [
    {
      "_index": "testindex",
      "_id": "1",
      "_source":{
         "passage_text": "This is an example document to be chunked. The document contains a single paragraph, two sentences and 24 tokens by standard tokenizer in OpenSearch."
      }
    }
  ]
}

returns:

{
  "docs": [
    {
      "error": {
        "root_cause": [
          {
            "type": "class_cast_exception",
            "reason": "class_cast_exception: class org.opensearch.ml.common.dataset.remote.RemoteInferenceInputDataSet cannot be cast to class org.opensearch.ml.common.dataset.TextDocsInputDataSet (org.opensearch.ml.common.dataset.remote.RemoteInferenceInputDataSet and org.opensearch.ml.common.dataset.TextDocsInputDataSet are in unnamed module of loader java.net.FactoryURLClassLoader @2b98c310)"
          }
        ],
        "type": "m_l_exception",
        "reason": "m_l_exception: Failed to inference TEXT_EMBEDDING model: MODEL_ID_HERE",
        "caused_by": {
          "type": "class_cast_exception",
          "reason": "class_cast_exception: class org.opensearch.ml.common.dataset.remote.RemoteInferenceInputDataSet cannot be cast to class org.opensearch.ml.common.dataset.TextDocsInputDataSet (org.opensearch.ml.common.dataset.remote.RemoteInferenceInputDataSet and org.opensearch.ml.common.dataset.TextDocsInputDataSet are in unnamed module of loader java.net.FactoryURLClassLoader @2b98c310)"
        }
      }
    }
  ]
}

I'm trying to use one of the hugging face encoders (I'm still resolving some role access so I wasn't able to use the Bedrock connector yet).

POST /_plugins/_ml/model_groups/_register
{
  "name": "local_model_group",
  "description": "A model group for local models"
}

POST /_plugins/_ml/models/_register
{
  "name": "huggingface/sentence-transformers/msmarco-distilbert-base-tas-b",
  "version": "1.0.2",
  "model_group_id": "GROUP_ID_HERE",
  "model_format": "TORCH_SCRIPT"
}

[ylwu-amzn]

You should use your correct model id when create ingest pipeline

"model_id": "MODEL_ID_HERE",

MODEL_ID_HERE is not correct model id.

[howdymatt]

yes sorry I removed the ids when pasting the commands

[howdymatt]

I did have actual IDs in my original request -- I've removed them in this discussion. The error occurs when the original IDs are used.