Redis cache InvalidOperationException: Reading is not allowed after reader was completed

Question

I have one webjob that is continously inserting data into redis I have another time trigger function to read data from the same redis cache after every 5 minutes.

  public static void ProcessData([TimerTrigger("0 */5 * * * *")] TimerInfo myTimer)
        {
            var newCacheclient = new RedisCacheClient(Program.spRedisCacheConnectionPoolMgr, Program.serializer, Program.redisConfiguration).Db0;

            var cachedData = newCacheclient.GetAsync<List<MyObject>>("mydata").Result;

after executing 10-15minutes I'm getting below error from time trigger function .anyone know how to tackle this?

InvalidOperationException: Reading is not allowed after reader was completed.

I raised the same issue with github as well.

nuget version details -

Error Stack - Inner Exception 1:

InvalidOperationException: Reading is not allowed after reader was completed.


   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at MyData.Functions.ExecuteTrade(TimerInfo myTimer) in C:\Users\\Functions.cs:line 27
   at Microsoft.Azure.WebJobs.Host.Executors.VoidMethodInvoker`1.InvokeAsync(TReflected instance, Object[] arguments)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker`1.<InvokeAsync>d__0.MoveNext()
   
   at System.IO.Pipelines.ThrowHelper.ThrowInvalidOperationException_NoReadingAllowed()
   at System.IO.Pipelines.Pipe.AdvanceReader(SequencePosition& consumed, SequencePosition& examined)
   at System.IO.Pipelines.Pipe.DefaultPipeReader.AdvanceTo(SequencePosition consumed, SequencePosition examined)
   at StackExchange.Redis.PhysicalConnection.<ReadFromPipe>d__110.MoveNext() in /_/src/StackExchange.Redis/PhysicalConnection.cs:line 1495
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at StackExchange.Redis.Extensions.Core.Implementations.RedisDatabase.<GetAsync>d__14`1.MoveNext()

Answer 1

I see couple of problems in the code:

1. You should reuse same redis client instance to avoid ending up with too many tcp connections to redis server.
2. Do not use .Result to avoid blocking current thread. Update the method to be async and use await on GetAsync.
3. Make sure you do not store a large object in a cache entry (in this case List<MyObject> should not be a large list)

Below is the updated code:

        private static readonly IRedisCacheClient redisCacheClient = new RedisCacheClient(Program.spRedisCacheConnectionPoolMgr, Program.serializer, Program.redisConfiguration);
        
        public static async Task ProcessData([TimerTrigger("0 */5 * * * *")] TimerInfo myTimer)
        {
            var redisDB = redisCacheClient.Db0;
            var cachedData = await redisDB.GetAsync<List<MyObject>>("mydata").ConfigureAwait(false);
        }

Answer 2

You should apply what @krishg has said first. In addition to that, there is a Redis client called TheUniversalCity.RedisClient which prevents that error when thousands of GetAsync methods are called.

The code block of the RedisClientParallelCall method which belongs to TheUniversalCity library that you can see at the benchmark comparison above:

[Fact]
[Trait("Category", "RedisClientBenchmark")]
public async Task RedisClientParallelCall()
{
    var sw = new Stopwatch();

    var redisClient1 = await RedisClient.CreateClientAsync("localhost,allowAdmin=true,clientCache=false");
    var taskList = new List<Task>();

    sw.Start();

    var cts = new CancellationTokenSource(8000);

    for (int i = 0; i < 100000; i++)
    {
        taskList.Add(redisClient1.GetAsync(TEST_KEY, cts.Token));
    }

    await Task.WhenAll(taskList);

    sw.Stop();

    Console.WriteLine($"Time elapsed : {sw.ElapsedMilliseconds}");
}

Here is the code block of the method belongs to the StackExchange.Redis library that has the same purpose with the method above:

[Fact]
[Trait("Category", "StackExchangeBenchmark")]
public async Task StackExchangeParallelCall()
{
    var sw = new Stopwatch();

    var connection = StackExchange.Redis.ConnectionMultiplexer.Connect("localhost,allowAdmin=true");
    var db = connection.GetDatabase();
    var taskList = new List<Task>();

    sw.Start();

    for (int i = 0; i < 100000; i++)
    {
        taskList.Add(db.StringGetAsync(TEST_KEY));
    }

    await Task.WhenAll(taskList);

    sw.Stop();

    Console.WriteLine($"Time elapsed : {sw.ElapsedMilliseconds}");
}

I have executed these methods via XUnit Test Provider on Visual Studio 2019 and, as you can see at the benchmark comparison link, the same error that you encountered been occurred in the method of the Stack Exchange library while it hasn't been in the TheUniversalCity one.

Also, you can have a look at the source codes of TheUniversalCity.Redis client library.