-
Notifications
You must be signed in to change notification settings - Fork 49
Description
I am using EnyimMemcachedCore
version 3.3.0
, which is connected to single memcached instance.
memcached version is 1.6.8
. it's running on windows machine as a windows service. It's running with configuration: -m 10144 -c 2048 -t 8
My application uses memcached exstensively, I have a lot of reads and writes.
I have about 7-8K Get
requests per second and about 400 Store
requests per second.
But after application has started and is running for about 15 minutes, Store
requests start to get failing, with the error code Unable to Locate Node
I dig though the code and find out that when there is single memcached node library uses SIngleNodeLocator
that has just one private field _node
and if it's null Operation will fail with such error. I am using PerformStore
method and if node locator returns null
it will fail with such error:
result.Fail("Unable to locate node");
return result;
I am using single static MemcachedClient
instance for both get
and store
requests.
here's my memcached configuration:
var loggerFactory = new LoggerFactory().AddSerilog();
var memcachedClientOptions = new MemcachedClientOptions()
{
Protocol = MemcachedProtocol.Binary,
SocketPool = new SocketPoolOptions()
{
},
Servers = new List<Server>()
{
new Server()
{
Address = address,
Port = port
}
},
Transcoder = "MessagePackTranscoder",
};
var memcachedClientConfiguration = new MemcachedClientConfiguration(loggerFactory, memcachedClientOptions)
{
NodeLocator = typeof(SingleNodeLocator),
};
_memcachedClient = new MemcachedClient(loggerFactory, memcachedClientConfiguration);
Also almost everywhere I have classic caching logic:
- Get from Cache
- If cache entry is not present Get from DB
- Store the result in cache
due to the fact that most Store requests are failing, number of store requests becomes more frequent and goes up to about 3k.
Is there any way I can debug this or some options in configuration I can try to pinpoint the issue? Also if there is specific scenarios when _node
field in SingleNode
locator can be null that might help me to investigate.
On development environment application has far less get/store
requests and there are no issues, but production load is much higher and maybe that is the issue.
Anyways will appreciate any help :)