Migrating from old SDK: No events received

[remcoros]

Describe the bug

We are migrating from the old SDK to the new one, but run into an issue where no events are received anymore.

I followed the migration guide and created a unit test to see if events are coming into the source, but (using the Debugger view on the source in Segment dashboard), no events are received.

To Reproduce

New code:

    [Fact]
    public async Task Can_track2()
    {
        var configuration = new Segment.Analytics.Configuration(
            writeKey: "xxxxx",
            flushAt: 1,
            flushInterval: 1,
            storageProvider: new InMemoryStorageProvider(),
            useSynchronizeDispatcher: true
        );

        var analytics = new Segment.Analytics.Analytics(configuration);

        analytics.Identify("user");
        analytics.Track("TestEvent");
        
        analytics.Flush();

        await Task.Delay(15000);
    }

Old code:

    [Fact]
    public async Task Can_track()
    {
        Segment.Analytics.Initialize("xxxxxx", new Config());

        Segment.Analytics.Client.Track("user", "TestEvent");

        await Segment.Analytics.Client.FlushAsync();
        Segment.Analytics.Dispose();
    }

Expected behavior

I see these events in the debugger view of the source, but it doesn't.

The 'old code' works, but using the new SDK, nothing happens.

Platform (please complete the following information):

• Library Version in use: 2.5.1
• Platform being tested: .NET 9 on Windows

[remcoros]

I did some debugging, and created a ProxyHttpClientProvider and noticed that SegmentUrl() for the ingestion endpoint, and "DoPost" are never called.

What am I missing? I've read through the documentation and migration guides multiple times now, read most of the Github issues, but nothing points to me doing something wrong?

[wenxi-zeng]

hi @remcoros thanks for reporting this. I did some investigation. looks like our SynchronizeDispatcher does not work well with the System.Net.Http HttpClient. the executor does not await for SendAsync to fetch settings, and resumes to update settings with empty values, which causes segment destination plugin to be disabled.

as a workaround, if you only use useSynchronizeDispatcher for unit tests purpose, you can add a defaultSettings in config as below, so analytics will enable segment destination plugin even if fetch settings from remote failed:

            var configuration = new Segment.Analytics.Configuration(
                writeKey: "YOUR WRITE KEY",
                flushAt: 1,
                flushInterval: 1,
                storageProvider: new InMemoryStorageProvider(),
                useSynchronizeDispatcher: true,
                defaultSettings: new Settings
                {
                    Integrations = new JsonObject
                    {
                        ["Segment.io"] = new JsonObject
                        {
                            ["apiKey"] = "YOUR WRITE KEY"
                        }
                    }
                }
            );

you should be able see events get into your segment debugger after adding the default settings. if you only want it for unit tests but do not need to see it in segment debugger, you can follow this unit test to setup a mockHttpClient.

if you want to use useSynchronizeDispatcher in production, I'd suggest against it, since this SDK is architectured to be async. events are written and flushed in a fire and forget manner. however, we do have customers want a sync flush especially for server use cases. you can do so by providing eventPipelineProvider: new SyncEventPipelineProvider() in the config to convert flush from async to sync. in fact, if you use SyncEventPipelineProvider, you no longer need await Task.Delay(15000); in your unit test.

we are updating our documentations, and will be sure to include the above information in the migration guide.

[remcoros]

@wenxi-zeng unfortunately, that did not work.

I tried with your example, with and without 'useSynchronizeDispatcher', with and without SyncEventPipelineProvider. But nothing I try seems to work.

In all cases, just as before, the "DoPost" method of the ProxyHttpHandler is never called, so it looks like it is never flushing.

Even if I use the default configuration (useSynchronizeDispatcher: false, and all other defaults as documented). And use "await Task.Delay(1000000)" in the unit test to keep it running for a while, events are never sent.

Actually it did work, I forgot to remove "autoAddSegmentDestination: false".

Thanks for the workaround!

[wenxi-zeng]

thanks for confirming @remcoros! I have created an internal ticket for SynchronizeDispatcher issue. we will take a look at the root cause of SynchronizeDispatcher at a later time.

[remcoros]

@wenxi-zeng Still having issues.

I'm now trying this in a live environment (without synchronous dispatcher), but events are not uploaded again.

This is the configuration:

        services.AddSingleton(sp =>
        {
            Analytics.Logger = sp.GetRequiredService<SegmentLogger>();
            var configuration = new global::Segment.Analytics.Configuration(
                writeKey,
                storageProvider: new InMemoryStorageProvider(),
                httpClientProvider: new ProxyHttpClientProvider(sp.GetRequiredService<ILogger<ProxyHttpClient>>()),
            {
                AnalyticsErrorHandler = sp.GetRequiredService<SegmentErrorHandler>()
            };

            return configuration;
        });

        services.AddScoped<Analytics>(sp =>
        {
            var configuration = sp.GetRequiredService<global::Segment.Analytics.Configuration>();
            return new Analytics(configuration);
        });

Here is the full log (replaced writekey with 'xxxxxxxx'):

16:16:43 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Information: No settings loaded from storage. Switch to default settings provided through configuration.
16:16:43 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = False
16:16:43 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Information: No settings loaded from storage. Switch to default settings provided through configuration.
16:16:43 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = False
16:16:43 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = False
16:16:43 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = False
16:16:43 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Segment.io performing flush
16:16:43 dbug: CADEngine.Analytics.Segment.ProxyHttpClient[0] SegmentURL(cdn-settings.segment.com/v1, /projects/xxxxxxxxxx/settings) = https://cdn-settings.segment.com/v1/projects/xxxxxxxxxx/settings
16:16:43 dbug: CADEngine.Analytics.Segment.ProxyHttpClient[0] SegmentURL(cdn-settings.segment.com/v1, /projects/xxxxxxxxxx/settings) = https://cdn-settings.segment.com/v1/projects/xxxxxxxxxx/settings
16:16:43 dbug: CADEngine.Analytics.Segment.ProxyHttpClient[0] DoGet(https://cdn-settings.segment.com/v1/projects/xxxxxxxxxx/settings)
16:16:43 dbug: CADEngine.Analytics.Segment.ProxyHttpClient[0] DoGet(https://cdn-settings.segment.com/v1/projects/xxxxxxxxxx/settings)
16:16:43 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = False
16:16:43 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = False
16:16:43 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = False
16:16:43 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = False
16:16:43 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = True
16:16:43 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = True
16:16:57 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Information: No settings loaded from storage. Switch to default settings provided through configuration.
16:16:57 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = False
16:16:57 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = False
16:16:57 dbug: CADEngine.Analytics.Segment.ProxyHttpClient[0] SegmentURL(cdn-settings.segment.com/v1, /projects/xxxxxxxxxx/settings) = https://cdn-settings.segment.com/v1/projects/xxxxxxxxxx/settings
16:16:57 dbug: CADEngine.Analytics.Segment.ProxyHttpClient[0] DoGet(https://cdn-settings.segment.com/v1/projects/xxxxxxxxxx/settings)
16:16:57 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: SegmentStartupQueue queueing event
16:16:57 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = False
16:16:57 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = False
16:16:57 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = True
16:16:57 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Information: No settings loaded from storage. Switch to default settings provided through configuration.
16:16:57 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = False
16:16:57 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = False
16:16:57 dbug: CADEngine.Analytics.Segment.ProxyHttpClient[0] SegmentURL(cdn-settings.segment.com/v1, /projects/xxxxxxxxxx/settings) = https://cdn-settings.segment.com/v1/projects/xxxxxxxxxx/settings
16:16:57 dbug: CADEngine.Analytics.Segment.ProxyHttpClient[0] DoGet(https://cdn-settings.segment.com/v1/projects/xxxxxxxxxx/settings)
16:16:57 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: SegmentStartupQueue queueing event
16:16:57 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Segment.io running {"type":"track","event":"PowernestPlanCreated","properties":{"UserId": "Unknown","Name": "PowernestPlanCreated","NumberOfPhysicalCuttingPlans": 1,"EventType": "PowernestPlanCreated","ProjectId": "01c97fa4-218c-43ce-a67a-31fae9e1a35b","Success": true,"IsSystem": false,"Category": "Nesting","NumberOfLogicalCuttingPlans": 1,"PartyId": "9d040b97-fc78-4323-a348-effd0f644d7f","userId": "Unknown"},"anonymousId":"69b484af-ffde-488d-be01-e071843462e5","messageId":"b221b47c-a612-4a10-9a31-d00f16c4430c","userId":null,"timestamp":"2025-03-12T15:16:57.3438489Z","context":{"os": "Microsoft Windows 10.0.26100","platform": ".NET","library": {"name": "Analytics-CSharp","version": "2.5.1"}},"integrations":{},"metrics":null,"_metadata":{"bundled":[],"unbundled":[],"bundledIds":[]}}
16:16:57 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = False
16:16:57 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = False
16:16:57 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Analytics starting = True
16:16:57 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Segment.io running {"type":"track","event":"PowernestPlanCreated","properties":{"UserId": "Unknown","Name": "PowernestPlanCreated","NumberOfPhysicalCuttingPlans": 1,"EventType": "PowernestPlanCreated","ProjectId": "01c97fa4-218c-43ce-a67a-31fae9e1a35b","Success": true,"IsSystem": false,"Category": "Nesting","NumberOfLogicalCuttingPlans": 1,"PartyId": "9d040b97-fc78-4323-a348-effd0f644d7f","userId": "Unknown"},"anonymousId":"c37e5948-d0dc-4250-9bb3-c00f5b960293","messageId":"130c0d66-c47d-4a62-a5f1-8fb199b92322","userId":null,"timestamp":"2025-03-12T15:16:57.3649320Z","context":{"os": "Microsoft Windows 10.0.26100","platform": ".NET","library": {"name": "Analytics-CSharp","version": "2.5.1"}},"integrations":{},"metrics":null,"_metadata":{"bundled":[],"unbundled":[],"bundledIds":[]}}
16:17:13 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Segment.io performing flush
16:17:43 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Segment.io performing flush
16:18:13 dbug: CADEngine.Analytics.Segment.SegmentLogger[0] Debug: Segment.io performing flush

I noticed a difference with my earlier unit test, where I got some message about uploaded events, that are missing now.

Also, adding the "defaultSettings" work-around seems to not help in this case.

[remcoros]

Also, I really don't understand the design of the Analytics class. According to the documentation, it should be a scoped service. But I looked at the source, and it looks like all things like the store, dispatchers, etc. are created new each time the Analytics class is created.

That means, for every http request, it creates a new instance and initializes everything new. Looking at the logs, it looks like this is indeed the case (all the "Analytics starting = False/True" logs). It also does a request to 'cdn-settings.segment.com' multiple times (for each http request?).

Isn't that extremely inefficient to do for every http request (especially under high load)?

[remcoros]

I'm going back to use the old SDK. I've spent way too much time on this (twice now), and I feel the design of the new SDK is completely broken and inefficient for HTTP API / backend scenarios.

[wenxi-zeng]

@remcoros you're right that this library should be used as singleton. we have realized the discrepancies on the docs and how misleading it is, and is actively working on the docs. the updated docs should be live in 2 weeks.

for the issue you're currently experiencing, it looks like you're using the default value of flushAt and flushInterval (see the default values here). by default, it flushes at every 20 events or every 30 seconds. if these threshold are not met, no flush would happen. you can set the flushAt = 1 to flush at a per event basis, or write your own flush policy to flush at the time you desired, or use analytics.Flush() to force a flush (need to provide sync event pipeline if you want to wait for it complete).

[remcoros]

@remcoros you're right that this library should be used as singleton. we have realized the discrepancies on the docs and how misleading it is, and is actively working on the docs. the updated docs should be live in 2 weeks.

for the issue you're currently experiencing, it looks like you're using the default value of flushAt and flushInterval (see the default values here). by default, it flushes at every 20 events or every 30 seconds. if these threshold are not met, no flush would happen. you can set the flushAt = 1 to flush at a per event basis, or write your own flush policy to flush at the time you desired, or use analytics.Flush() to force a flush (need to provide sync event pipeline if you want to wait for it complete).

I've let the project run for several minutes, and according to the logs it does try to flush every 30 seconds, but no events are sent.

I've given up for now. This was the 2nd attempt of trying to migrate to the new SDK to no avail and have no time left to dive deep or do testing anymore for this. Maybe in the future with updated documentation and samples we can give it another try. But for now we'll be using the old SDK with our current (and already working) solution of pushing tracking to a background queue.

Thanks for your help!