Feedback requests: Conflict resolution

[adrianhall]

I am currently adding conflict resolution to the client library and would like feedback on the API and functionality.

Conflict resolution ONLY affects the push operation.

During the OnDatasyncInitialization() method, you can add a conflict resolver to the entity. For example:

protected override void OnDatasyncInitialization(DatasyncOfflineOptionsBuilder optionsBuilder)
{
    optionsBuilder.UseEndpoint(new Uri("https://MYENDPOINT.azurewebsites.net"));

    optionsBuilder.Entity<Movie>(cfg => {
        cfg.ClientName = "movies";
        cfg.ConflictResolver = new ServerWinsConflictResolver(),
        cfg.Endpoint = new Uri("/api/movies", UriKind.Relative),
        cfg.Query.Where(x => x.Rating != MovieRating.R)
    });
}

There are two default conflict resolvers - ClientWinsConflictResolver and ServerWinsConflictResolver. There is also an interface you can implement: IConflictResolver and an AbstractConflictResolver<T> that implements a typed conflict resolver instead. Here is the client-wins conflict resolver for the typed version:

public class ClientWinsConflictResolver<T> : AbstractConflictResolver<T>
{
  public Task<T?> ResolveConflictAsync(T? clientObject, T? serverObject, CancellationToken ct = default)
    => Task.FromResult(clientObject);
}

When the push operation gets a 409 or 412, then conflict resolver is called. If the conflict resolver is null or the conflict resolver returns null then the old behavior (just log an error) happens. If, however, the response from the conflict resolver is an object, the object is serialized and sent to the server using the "force" option - the operation is the same (delete, replace, add). The response from this operation is what is recorded (so if this fails, the whole thing fails as before).

There is no "default" conflict resolver. Would that be useful? i.e. optionsBuilder.UseConflictResolver(new ServerWinsConflictResolver())

You can find the updated library code here: https://github.com/adrianhall/CommunityToolkit-Datasync/tree/issues/298

You will have to compile the library code to test this out. I may be updating the code during the week as I do more testing.

I'm not going to create the PR for another week (next Friday) to allow folks to give feedback on the conflict resolver code. Is it useful? Is the API sensible? Does it work for you?

If you use it (even if you like it), please drop me a note here. If you have comments, bugs, etc. then please let me know here as well.

[atexinspect]

Hi Adrian,
A default (Server Wins) conflict resolver would suit my personal needs if possible.

Thanks
Paul.

[adrianhall]

I've included ServerWinsConflictResolver as a class to meet that requirement.

[Vroomer]

Personally I think it's too limited as it only handles conflict errors. In my experience there can be other types of issues that have to be handled. Just to name few: Gone, NotFound, Unauthorized, BadRequest, ServiceUnavailable... I think it would be a lot more useful if you allow it to handle all unsuccessful responses.

[adrianhall]

Most of those are handled through the failed requests from the pull. Conflict resolution is a specific and targeted problem that needs something a little different from standard error handling.

[adrianhall]

That being said:

• Gone and NotFound are good candidates for alternative processing. Might be able to do something with those. File an issue with what you would like to see.
• Unauthorized is already handled
• ServiceUnavailable will be handled on the next push - basically, it's a transient error.
• BadRequest can't be handled - we have no way of knowing what is wrong with the request - just that it was not accepted. The only thing I can do here is push it into the failed requests queue - which is exactly what happens today.

[Vroomer]

• Unauthorized is already handled

How? I'm speaking about situations where the user is authenticated, but fails the authorization in IAccessControlProvider. I feel like that's something that should be handled by users.

Alright, I understand your points, but can we atleast get this public, so that FailedRequests get a bit more useful?

internal Type? GetSynchronizableEntityType(string fullName)
        => this._entityMap.TryGetValue(fullName, out Type? entityType) ? entityType : null;

Also I have a small question looking at the code:

        if (response.IsConflictStatusCode && options.ConflictResolver is not null)
        {
            object? serverEntity = JsonSerializer.Deserialize(response.ContentStream, entityType, DatasyncSerializer.JsonSerializerOptions);
            object? clientEntity = JsonSerializer.Deserialize(operation.Item, entityType, DatasyncSerializer.JsonSerializerOptions);
            object? resolvedEntity = await options.ConflictResolver.ResolveConflictAsync(clientEntity, serverEntity, cancellationToken).ConfigureAwait(false);
            if (resolvedEntity is not null)
            {
                lock (this.pushlock)
                {
                    operation.Item = JsonSerializer.Serialize(resolvedEntity, entityType, DatasyncSerializer.JsonSerializerOptions);
                    operation.State = OperationState.Pending;
                    operation.LastAttempt = DateTimeOffset.UtcNow;
                    operation.HttpStatusCode = response.StatusCode;
                    operation.EntityVersion = string.Empty; // Force the push
                    operation.Version++;
                    _ = this._context.Update(operation);
                }

                ExecutableOperation resolveOperation = await ExecutableOperation.CreateAsync(operation, cancellationToken).ConfigureAwait(false);
                response = await resolveOperation.ExecuteAsync(options, cancellationToken).ConfigureAwait(false);
            }
        }

Do I understand correctly, that if we set the resolved item to server copy, we then push the exact same item back to the server? And in that case doesn't it update the server db and reset the UpdatedAt field which then allows it to get pulled once again to the client?

[adrianhall]

I saw the same thing (the wonderful thing about developing in public!) The latest version fixes that problem.

[jerinantony11]

I have observed an issue "Precondition Failed" while pushing data to server as below . Will the conflict resolver help to come out of it ?
Currently I cant find any way to proceed.

[adrianhall]

Yes - that's exactly what it's for. Precondition failed is another name for "Conflict"

[adrianhall]

The PR for the conflict resolver has now been merged. We can keep discussing here until I write the documentation for it!

[jerinantony11]

Do you know when the nugget package version (9.0.2) will be released?

[adrianhall]

When I (or another contributor) completes the work for it. I'm on vacation right now, so I am not checking in and doing work on it until I get back.

There are some easy wins anyone can do - like updating versions of packages and running the tests. Feel free to contribute if you want it to go faster.

[FrederikFBrandt]

I seem to only have an issue with this when the client that originally sent the entity to the server tries to synchronize a second time. If I were to guess why, it would be because the user shuts down the app during synchronization as it is the last step in his work process. My guess is that the server picks up on one of the sent entities -> saves it to the database -> the client shuts down and does not have a confirmation of the entity being sent successfully and when the app is opened again, the synchronization fails with a conflict because the server received it already and the client is once again trying to send the same entity?

If that is the case, would it then be more of a "client wins" scenario for my usecase?

Currently I delete the corresponding row in the database table manually so that the client once again is able to synchronize and then the problem goes away for a while - until it reappears with a new entity a week or two later.

[adrianhall]

I would do a "last-write wins" policy in that case. In your conflict handler, you can compare the UpdateAt from the server with the UpdatedAt from the client and choose whichever is later.

[FrederikFBrandt]

Hmm could you dumb it down a bit, I don't follow the theory. Isn't UpdatedAt set server side using a SQL trigger? The client wouldn't know the new UpdatedAt until the server correctly processed the item and returned it - and it wouldn't change UpdatedAt on its own?

[FrederikFBrandt]

I can see this has now been raised as an issue as well: #360

[adrianhall]

#360 references a different issue.

The last-write-wins requires your entity has a "ClientUpdatedAt" or similar property that is controlled by the client rather than the server. You can then compare the client-write date to the server-write update. I'll also note that the queue entry (stored in your local DB) has the date the record was written and saved, so you can use that instead if you prefer to not handle the client-side write date and the problems with sending or not sending that to the server.

[FrederikFBrandt]

Then I haven't explained myself correctly. What he experienced in the issue he raised is exactly what we see as well. My comment here is regarding how to cope with such a case.

[FrederikFBrandt]

Would it make sense with a default conflict resolver on the DatasyncOfflineOptionsBuilder so that we only have to handle specific entities where needed?

    protected override void OnDatasyncInitialization(DatasyncOfflineOptionsBuilder optionsBuilder)
    {
        optionsBuilder.DefaultConflictResolver = new ClientWinsConflictResolver();
        optionsBuilder.Entity<Movie>(cfg =>
        {
            cfg.ConflictResolver = new ServerWinsConflictResolver();
        });
    }

[adrianhall]

Great idea! Want to file the issue?