I have a query I am running against my database using EF Core 3.1 with Postgresql that can sometimes return a large result set depending on the parameters requested. I am using Async Enumerable to stream the results, which can sometimes take over a minute to complete:
try
{
IAsyncEnumerable<Entity> ae = ctxt.Entity.Where(e => e.Value > 5).AsAsyncEnumerable();
await foreach(Entity e in ae.ConfigureAwait(false))
{
//Do some work on each one here that could potentially throw an exception
}
}
catch(Exception ex)
{
//This code can take over a minute after the exception is thrown to begin executing
}
I then got rid of the await foreach
loop to get a better idea what was happening:
IAsyncEnumerable<Entity> ae = ctxt.Entity.Where(e => e.Value > 5).AsAsyncEnumerable();
var enumerator = ae.GetAsyncEnumerator();
await enumerator.MoveNextAsync();
await enumerator.DisposeAsync();
Console.WriteLline("Done Disposing");
And found that the DisposeAsync
call was where the time was being spent. I went back to the await foreach
and was able to confirm that using break
inside the loop exhibited the same behavior.
Next, I tried creating a non-ef IAsyncEnumerable
to see if this behavior was isolated to Entity Frameowork or not.
private async IAsyncEnumerable<int> IntEnumerator()
{
await Task.Delay(0).ConfigureAwait(false);
for(int i = 0; i < int.MaxValue; i++)
{
yield return i;
}
}
Using this code, the Dispose was instant.
IAsyncEnumerable<int> ae = IntEnumerator();
var enumerator = ae.GetAsyncEnumerator();
await enumerator.MoveNextAsync();
await enumerator.DisposeAsync();
Console.WriteLline("Done Disposing");
This leads me to believe the issue is specific to EF core, but I am not sure if it is a bug in their implementation, or I am doing something wrong. Has anyone else seen something like this and found a solution?
I wanted to check here before opening an issue on the EF github page.
It looks like this is actually an issue with postgresql. See the issue I opened on the Npgsql github page. According to the maintainer, it is not resolvable.