connection is already in state 'Executing' when inserting lots of objects [Using Dapper]
See original GitHub issueI submitted a report on the dapper repo for this issue as well. https://github.com/StackExchange/Dapper/issues/919
Steps to reproduce
Essentially, I have a bit of code that inserts rows into 2 tables. The second table has a Foreign Key to the first.
using (TransactionScope scope = new TransactionScope(TransactionScopeOption.Required, TransactionScopeAsyncFlowOption.Enabled))
{
const string query = @"WITH updated as (
INSERT INTO public.schedule_row
(id, schedule_id, petraid, ssnit, staffid, fullname, salary, total, created_by, validation_messages)
VALUES
(@Id, @ScheduleId, @PetraId, @Ssnit, @StaffId, @Fullname, @Salary, @Total, @ModifiedBy, @ValidationMessage)
)
UPDATE public.schedule
SET
is_valid = false
, updated_by = @ModifiedBy
, updated_at = now()
, validation_processed_at = NULL
, validation_reminders_count = 0
, validation_reminder_last_sent_at = NULL
WHERE id = @ScheduleId;
";
var rows_ = rows.Select(sc =>
{
var total = sc.Contributions.Sum(c => string.IsNullOrWhiteSpace(c.Amount) ? 0 : Decimal.Parse(c.Amount));
var salary = string.IsNullOrWhiteSpace(sc.Salary) ? 0 : Decimal.Parse(sc.Salary);
var petraId = sc.PetraId?.ToUpper();
petraId = petraId?.StartsWith("H1") ?? false ? petraId.Replace("H1", "HI") : petraId;
return new
{
Id = NewId.NextGuid(),
ScheduleId = scheduleId,
PetraId = petraId,
Ssnit = sc.Ssnit,
StaffId = sc.StaffId,
Fullname = sc.Fullname,
Salary = salary,
Total = total,
ModifiedBy = modifiedBy,
Contributions = sc.Contributions,
ValidationMessage = new String[] { "Not validated yet" }
};
}).ToArray();
var count = await conn.ExecuteAsync(query, rows_).ConfigureAwait(false);
if (count > 0)
{
const string contributionsQuery = @"
INSERT INTO public.schedule_row_contribution
(schedule_row_id, deal_contribution_type, amount)
VALUES
(@ScheduleRowId, @DealContributionType, @Amount)
ON CONFLICT (schedule_row_id, deal_contribution_type)
DO NOTHING
";
var _rowContributions = rows_.SelectMany(r => r.Contributions.Select(c => new
{
ScheduleRowId = r.Id,
DealContributionType = c.DealContributionType,
Amount = string.IsNullOrWhiteSpace(c.Amount) ? 0 : Decimal.Parse(c.Amount),
})).ToArray();
await conn.ExecuteAsync(contributionsQuery, _rowContributions).ConfigureAwait(false);
}
}
The issue
The code works find and we haven’t had any issues.
I noticed however that an exception is thrown when the number of rows being inserted into both tables is more than approximately 20k total. In my case, I had about 11244 items in each list.
I made sure all calls are awaited so I do not get promoted to a distributed transaction since this is net core 2.0.
I run tests where I split the rows into batches of 5000 in an attempt to test. The exception gets thrown at around the 3rd or 4th iteration.
I am totally stumped here. Any pointers?
Exception message: The connection is already in state 'Executing'
Stack trace:
Npgsql.NpgsqlOperationInProgressException: The connection is already in state 'Executing'
at Npgsql.NpgsqlConnector.StartUserAction(ConnectorState newState, NpgsqlCommand command)
at Npgsql.NpgsqlCommand.<ExecuteNonQuery>d__84.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Npgsql.NpgsqlCommand.<>c__DisplayClass83_0.<<ExecuteNonQueryAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Dapper.SqlMapper.<ExecuteMultiImplAsync>d__36.MoveNext() in C:\projects\dapper\Dapper\SqlMapper.Async.cs:line 622
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
Further technical details
Npgsql version: 3.2.6 PostgreSQL version: 9.6 Operating system: Windows 10
Other details about my project setup: Net Core 2.0
Issue Analytics
- State:
- Created 6 years ago
- Comments:18 (9 by maintainers)
Let me anonymize it, I’ll send you some code to repro
Closing since no repro was posted back.