QueryOperationConfig with index fails in SDK V4 unless property is annotated with DynamoDBGlobalSecondaryIndexHashKey

[FinzRiki]

Describe the bug

After upgrading to version 4 of the AWS SDK for .NET (DynamoDB), a previously working query using a global secondary index now throws the following exception:

System.InvalidOperationException: 'Unable to locate index [myIndex] on table [myobject]'

This issue did not exist in version 3.x.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

In version 3, the query worked without needing to explicitly define index attributes DynamoDBGlobalSecondaryIndexHashKey on the model class. The index was identified through QueryOperationConfig.IndexName.

No mention of this breaking change is found in the changelog or documentation for SDK v4.

Current Behavior

Message:
System.InvalidOperationException: 'Unable to locate index [myIndex] on table [myobject]'

StackTrace:

at Amazon.DynamoDBv2.DocumentModel.Search.IsKeyAttribute(Table table, String indexName, String attributeName)
at Amazon.DynamoDBv2.DocumentModel.Search.SplitQueryFilter(Filter filter, Table targetTable, String indexName, Dictionary2& keyConditions, Dictionary2& filterConditions)
at Amazon.DynamoDBv2.DocumentModel.Search.d__80.MoveNext()
at Amazon.DynamoDBv2.DocumentModel.Search.d__82.MoveNext()
at Amazon.DynamoDBv2.DataModel.AsyncSearch`1.d__23.MoveNext()

Reproduction Steps

Given the following class:

public class MyObject
{
    [DynamoDBProperty("name")]
    public string Name { get; set; }

    [DynamoDBProperty("address ")]
    public string Address { get; set; }
}

And the following code to query with an index:

QueryFilter queryFilter = new("name", QueryOperator.Equal, "value");

QueryOperationConfig queryOperationConfig = new()
{
    IndexName = "myIndex",
    Filter = queryFilter
};

IAsyncSearch<MyObject> search = Context.FromQueryAsync<MyObject>(
    queryOperationConfig,
    new FromQueryConfig() { OverrideTableName = "MyObject" }
);

List<MyObject> result = await search.GetRemainingAsync();

Possible Solution

By explicitly annotating the property with the DynamoDBGlobalSecondaryIndexHashKey attribute, the query works again:

public class MyObject
{
    [DynamoDBProperty("name")]
    [DynamoDBGlobalSecondaryIndexHashKey("myIndex")]
    public string Name { get; set; }

    [DynamoDBProperty("address ")]
    public string Address { get; set; }
}

Additional Information/Context

However, if I use the following methods, I don't find any problems with the index:

var queryReq = new QueryRequest("MyObject")
{
    ExpressionAttributeValues = new()
    {
        [":value"] = new("test"),
    },
    ExpressionAttributeNames = new()
    {
        ["#name"] = "name"
    },
    IndexName = "MyIndex",
    KeyConditionExpression = "#name = :value",
};
var client = new AmazonDynamoDBClient();
QueryResponse responses = await client.QueryAsync(queryReq);

Is this new requirement to explicitly annotate properties with index attributes (DynamoDBGlobalSecondaryIndexHashKey) expected behavior in SDK v4?

• If yes, could this be clearly documented in the migration guide or changelog?

• If not, is this a regression or a bug in the object persistence model?

AWS .NET SDK and/or Package version used

AWSSDK.DynamoDBv2 4.0.1
AWSSDK.SSO 4.0.0.4
AWSSDK.SSOOIDC 4.0.0.4

Targeted .NET Platform

.NET 8

Operating System and version

Windows 11

[GarrettBeatty]

Full steps to reproduce issue

1. Create new DDB table in AWS console
   partitionKey: name

Global Secondary Index

1. partitionKey: name
2. Index name: myIndex

using Amazon;
using Amazon.DynamoDBv2;
using Amazon.DynamoDBv2.DataModel;
using Amazon.DynamoDBv2.DocumentModel;
using System.Diagnostics;

class Program
{
    static async Task Main(string[] args)
    {

        var client = new AmazonDynamoDBClient();
        DynamoDBContext context = new DynamoDBContextBuilder()
                    // Optional call to provide a specific instance of IAmazonDynamoDB
                    .WithDynamoDBClient(() => client)
                    .Build();        


        QueryFilter queryFilter = new("name", QueryOperator.Equal, "value");

        QueryOperationConfig queryOperationConfig = new()
        {
            IndexName = "myIndex",
            Filter = queryFilter
        };

        IAsyncSearch<MyObject> search = context.FromQueryAsync<MyObject>(
            queryOperationConfig,
            new FromQueryConfig() { OverrideTableName = "MyObject" }
        );

        List<MyObject> result = await search.GetRemainingAsync();
    }
}


public class MyObject
{
    [DynamoDBProperty("name")]
    public string Name { get; set; }

    [DynamoDBProperty("address ")]
    public string Address { get; set; }
}

[ashishdhingra]

@FinzRiki Good morning. Thanks for opening the issue. You are explicitly specifying IndexName as myIndex in QueryOperationConfig. Due to this, you are most likely getting exception here for AWS SDK for .NET V4.

Based on code search, the same exception is thrown in 3.x version of SDK per logic here.

Somehow, the below code works for both V3 and V4 version of SDK:

using Amazon.DynamoDBv2.Model;
using Amazon.DynamoDBv2.DataModel;
using Amazon.DynamoDBv2;
using Amazon.DynamoDBv2.DocumentModel;

var client = new AmazonDynamoDBClient();
DynamoDBContext context = new DynamoDBContext(client);

QueryFilter queryFilter = new("name", QueryOperator.Equal, "value");

QueryOperationConfig queryOperationConfig = new()
{
    IndexName = "myIndex",
    Filter = queryFilter
};

var search = context.FromQueryAsync<MyObject>(
    queryOperationConfig,
    new DynamoDBOperationConfig() { OverrideTableName = "MyObject" }
);

List<MyObject> result = await search.GetRemainingAsync();

public class MyObject
{
    [DynamoDBProperty("name")]
    //[DynamoDBGlobalSecondaryIndexHashKey("myIndex")]
    public string Name { get; set; }

    [DynamoDBProperty("address")]
    public string Address { get; set; }
}

While debugging, the code here was able to find specified GSI myIndex that I had created for DynamoDB table.

Could you please confirm the following:

• You have defined Global Secondary Index for the specified DynamoDB table?
• If yes, could you please re-confirm if the issue doesn't exist in 3.x version of AWS SDK for .NET?

Thanks,
Ashish

[GarrettBeatty]

i tried with @ashishdhingra 's code and it worked in v3 and v4. but i then modified the context to be

 DynamoDBContext context = new DynamoDBContextBuilder()
              .WithDynamoDBClient(() => client)
              .Build();

and got the same error 'Unable to locate index [myIndex] on table [myobject]'

Can you please share the complete code to verify our findings @FinzRiki ?

[GarrettBeatty]

potentially related #3849