[Bug]: Post sqlite -> postgres migration invoice generation failure

What is your BTCPay version?

Migrating from v1.7.7 -> 1.8.1, on an instance that was running sqlite.

How did you deploy BTCPay Server?

embassyOS v0.3.3

What happened?

Post successful database migration, new invoices cannot be generated on a store with a bitcoin wallet setup and using an internal LND node.

How did you encounter this bug?

1. Install btcpay v1.7.7 on embassyOS v0.3.3 (which is using a sqlite database)
2. Update btcpay to v1.8.1
3. Notice successful update and database migration
4. Setup a store with a bitcoin wallet, LND internal lightning node, and LNURL enabled for standard invoices
5. Attempt to generate an invoice and receive the below error in the logs

Relevant log output

2023-03-06T10:12:50-07:00  2023-03-06 17:12:50.137 UTC [283] ERROR:  duplicate key value violates unique constraint "PK_InvoiceSearches"
2023-03-06T10:12:50-07:00  2023-03-06 17:12:50.137 UTC [283] DETAIL:  Key ("Id")=(1) already exists.
2023-03-06T10:12:50-07:00  2023-03-06 17:12:50.137 UTC [283] STATEMENT:  INSERT INTO "InvoiceSearches" ("InvoiceDataId", "Value")
2023-03-06T10:12:50-07:00      VALUES ($1, $2)
2023-03-06T10:12:50-07:00      RETURNING "Id"
2023-03-06T10:12:50-07:00  fail: Microsoft.AspNetCore.Server.Kestrel: Connection id "0HMOUCNOC50BM", Request id "0HMOUCNOC50BM:00000002": An unhandled exception was thrown by the application.
2023-03-06T10:12:50-07:00  Microsoft.EntityFrameworkCore.DbUpdateException: An error occurred while saving the entity changes. See the inner exception for details.
2023-03-06T10:12:50-07:00   ---> Npgsql.PostgresException (0x80004005): 23505: duplicate key value violates unique constraint "PK_InvoiceSearches"
2023-03-06T10:12:50-07:00  
2023-03-06T10:12:50-07:00  DETAIL: Detail redacted as it may contain sensitive data. Specify 'Include Error Detail' in the connection string to include this information.
2023-03-06T10:12:50-07:00     at Npgsql.Internal.NpgsqlConnector.<ReadMessage>g__ReadMessageLong|215_0(NpgsqlConnector connector, Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage)
2023-03-06T10:12:50-07:00     at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)
2023-03-06T10:12:50-07:00     at Npgsql.NpgsqlCommand.ExecuteReader(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)
2023-03-06T10:12:50-07:00     at Npgsql.NpgsqlCommand.ExecuteReader(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)
2023-03-06T10:12:50-07:00     at Npgsql.NpgsqlCommand.ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken)
2023-03-06T10:12:50-07:00     at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
2023-03-06T10:12:50-07:00     at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
2023-03-06T10:12:50-07:00     at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.ExecuteAsync(IRelationalConnection connection, CancellationToken cancellationToken)
2023-03-06T10:12:50-07:00    Exception data:
2023-03-06T10:12:50-07:00      Severity: ERROR
2023-03-06T10:12:50-07:00      SqlState: 23505
2023-03-06T10:12:50-07:00      MessageText: duplicate key value violates unique constraint "PK_InvoiceSearches"
2023-03-06T10:12:50-07:00      Detail: Detail redacted as it may contain sensitive data. Specify 'Include Error Detail' in the connection string to include this information.
2023-03-06T10:12:50-07:00      SchemaName: public
2023-03-06T10:12:50-07:00      TableName: InvoiceSearches
2023-03-06T10:12:50-07:00      ConstraintName: PK_InvoiceSearches
2023-03-06T10:12:50-07:00      File: nbtinsert.c
2023-03-06T10:12:50-07:00      Line: 649
2023-03-06T10:12:50-07:00      Routine: _bt_check_unique
2023-03-06T10:12:50-07:00     --- End of inner exception stack trace ---
2023-03-06T10:12:50-07:00     at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.ExecuteAsync(IRelationalConnection connection, CancellationToken cancellationToken)
2023-03-06T10:12:50-07:00     at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.ExecuteAsync(IEnumerable`1 commandBatches, IRelationalConnection connection, CancellationToken cancellationToken)
2023-03-06T10:12:50-07:00     at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.ExecuteAsync(IEnumerable`1 commandBatches, IRelationalConnection connection, CancellationToken cancellationToken)
2023-03-06T10:12:50-07:00     at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.ExecuteAsync(IEnumerable`1 commandBatches, IRelationalConnection connection, CancellationToken cancellationToken)
2023-03-06T10:12:50-07:00     at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(IList`1 entriesToSave, CancellationToken cancellationToken)
2023-03-06T10:12:50-07:00     at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(StateManager stateManager, Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
2023-03-06T10:12:50-07:00     at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.<>c__DisplayClass33_0`2.<<ExecuteAsync>b__0>d.MoveNext()
2023-03-06T10:12:50-07:00  --- End of stack trace from previous location ---
2023-03-06T10:12:50-07:00     at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4 operation, Func`4 verifySucceeded, TState state, CancellationToken cancellationToken)
2023-03-06T10:12:50-07:00     at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4 operation, Func`4 verifySucceeded, TState state, CancellationToken cancellationToken)
2023-03-06T10:12:50-07:00     at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken)
2023-03-06T10:12:50-07:00     at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
2023-03-06T10:12:50-07:00     at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
2023-03-06T10:12:50-07:00     at BTCPayServer.Services.Invoices.InvoiceRepository.CreateInvoiceAsync(String storeId, InvoiceEntity invoice, String[] additionalSearchTerms) in /source/BTCPayServer/Services/Invoices/InvoiceRepository.cs:line 239
2023-03-06T10:12:50-07:00     at BTCPayServer.Controllers.UIInvoiceController.CreateInvoiceCoreRaw(InvoiceEntity entity, StoreData store, IPaymentFilter invoicePaymentMethodFilter, String[] additionalSearchTerms, CancellationToken cancellationToken, Action`1 entityManipulator) in /source/BTCPayServer/Controllers/UIInvoiceController.cs:line 379
2023-03-06T10:12:50-07:00     at BTCPayServer.Controllers.UIInvoiceController.CreateInvoiceCoreRaw(BitpayCreateInvoiceRequest invoice, StoreData store, String serverUrl, List`1 additionalTags, CancellationToken cancellationToken, Action`1 entityManipulator) in /source/BTCPayServer/Controllers/UIInvoiceController.cs:line 180
2023-03-06T10:12:50-07:00     at BTCPayServer.Controllers.UIInvoiceController.CreateInvoiceCore(BitpayCreateInvoiceRequest invoice, StoreData store, String serverUrl, List`1 additionalTags, CancellationToken cancellationToken, Action`1 entityManipulator) in /source/BTCPayServer/Controllers/UIInvoiceController.cs:line 105
2023-03-06T10:12:50-07:00     at BTCPayServer.Controllers.UIInvoiceController.CreateInvoice(CreateInvoiceModel model, CancellationToken cancellationToken) in /source/BTCPayServer/Controllers/UIInvoiceController.UI.cs:line 1162
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.TaskOfIActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask`1 actionResultValueTask)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextExceptionFilterAsync>g__Awaited|26_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ExceptionContextSealed context)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Authorization.Policy.AuthorizationMiddlewareResultHandler.HandleAsync(RequestDelegate next, HttpContext context, AuthorizationPolicy policy, PolicyAuthorizationResult authorizeResult)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
2023-03-06T10:12:50-07:00     at BTCPayServer.Hosting.BTCPayMiddleware.Invoke(HttpContext httpContext) in /source/BTCPayServer/Hosting/BTCpayMiddleware.cs:line 101
2023-03-06T10:12:50-07:00     at BTCPayServer.Hosting.GreenfieldMiddleware.Invoke(HttpContext httpContext) in /source/BTCPayServer/Hosting/GreenfieldMiddleware.cs:line 50
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context)
2023-03-06T10:12:50-07:00     at BTCPayServer.Hosting.HeadersOverrideMiddleware.Invoke(HttpContext httpContext) in /source/BTCPayServer/Hosting/HeadersOverrideMiddleware.cs:line 30
2023-03-06T10:12:50-07:00     at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

What browser do you use?

No response

Additional information

This problem does not occur on a fresh install of v1.8.1.

The sqlite database had invoices that were migrated.

Are you sure this is a bug report?

• I confirm this is a bug report

[elvece]

Yes I’ll get the update out. We haven’t yet released the version with the migration to production, still in beta testing so no urgent worries.

[NicolasDorier]

You can run

SELECT SETVAL('"InvoiceSearches_Id_seq"', (SELECT max("Id") FROM "InvoiceSearches"));

on your DB to fix it, will fix the migration to not require this.