question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Sync tables with GENERATE ALWAYS property

See original GitHub issue

I am trying to sync 2 database with this table

CREATE TABLE [dbo].[User](
	[UserId] [int] IDENTITY(1,1) NOT NULL,
	[Username] [varchar](100) NULL,
	[FirstName] [nvarchar](100) NOT NULL,
	[LastName] [nvarchar](100) NOT NULL,
	[Emailaddress] [varchar](100) NOT NULL,
	[AuditAt] [datetime2](7) GENERATED ALWAYS AS ROW START NOT NULL,
	[AuditTo] [datetime2](7) GENERATED ALWAYS AS ROW END NOT NULL,
	[VeritasTemplateId] [int] NULL,
 CONSTRAINT [PKUser_UserId] PRIMARY KEY CLUSTERED 
(
	[UserId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY],
 CONSTRAINT [UKUser_Username] UNIQUE NONCLUSTERED 
(
	[Username] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY],
	PERIOD FOR SYSTEM_TIME ([AuditAt], [AuditTo])
) ON [PRIMARY]
WITH
(
SYSTEM_VERSIONING = ON (HISTORY_TABLE = [dbo].[UserHistory])
)

And I am running this code after SaveChangesAsync()

foreach (var entry in ChangeTracker.Entries().ToList())
{
    var attribute = Attribute.GetCustomAttribute(entry.Entity.GetType(), typeof(SyncEntityAttribute)); 
    if (attribute == null) continue;

    try
    {
        var modelName = entry.Entity.GetType().Name;

        SqlSyncProvider gatewayProvider = new SqlSyncProvider(config.GetSection("ConnectionStrings").GetSection("DefaultConnection").Value);
        SqlSyncProvider stagingProvider = new SqlSyncProvider(config.GetSection("ConnectionStrings").GetSection("StagingConnection").Value);

        var setup = new SyncSetup(modelName);
        SyncAgent agent = new SyncAgent(stagingProvider, gatewayProvider);

        var syncContext = await agent.SynchronizeAsync(setup);
    }
    catch (Exception ex)
    {
        var t = ex.Message;
    }
}

But when executing agent.SynchronizeAsync(), it will throw an exception:

Cannot insert an explicit value into a GENERATED ALWAYS column in table 'dbo.User'. Use INSERT with a column list to exclude the GENERATED ALWAYS column, or insert a DEFAULT into GENERATED ALWAYS column.

Is there a way to ignore or filter the properties that are GENERATE ALWAYS or temporal tables? I don’t want to set which property I am going to sync as I am doing this also for other entities (but not all).

Issue Analytics

  • State:closed
  • Created 10 months ago
  • Comments:8 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
Mimetiscommented, Nov 11, 2022

DMS is agnostic to the database engine, because it should work with SQL Server, MySql, SQLite, MariaDB (and sooner or later Oracle, PostgreSQL)

That means it can’t work with specific features, like the great Temporal Tables option from SQL Server.

That being said, what you can do, maybe, is to remove the temporal columns ([AuditAt] and [AuditTo]) from the Setup instance from your config. And be sure you don’t let DMS creates the client table for you

Here is the script I’ve used:

USE [master]
GO 

-- Server database
if (exists (select * from sys.databases where name = 'TemporalServer'))
Begin
	ALTER DATABASE [TemporalServer] SET  SINGLE_USER WITH ROLLBACK IMMEDIATE;
	DROP DATABASE [TemporalServer]
End
Create database [TemporalServer]
Go
-- Client database.
if (exists (select * from sys.databases where name = 'TemporalClient'))
Begin
	ALTER DATABASE [TemporalClient] SET  SINGLE_USER WITH ROLLBACK IMMEDIATE;
	DROP DATABASE [TemporalClient]
End
Create database [TemporalClient]
Go
Use [TemporalServer];

CREATE TABLE [dbo].[User](
	[UserId] [int] IDENTITY(1,1) NOT NULL PRIMARY KEY,
	[Username] [varchar](100) NULL,
	[FirstName] [nvarchar](100) NOT NULL,
	[LastName] [nvarchar](100) NOT NULL,
	[Emailaddress] [varchar](100) NOT NULL,
	[AuditAt] [datetime2](7) GENERATED ALWAYS AS ROW START NOT NULL,
	[AuditTo] [datetime2](7) GENERATED ALWAYS AS ROW END NOT NULL,
	[VeritasTemplateId] [int] NULL,
 CONSTRAINT [UKUser_Username] UNIQUE NONCLUSTERED ([Username] ASC),
 PERIOD FOR SYSTEM_TIME ([AuditAt], [AuditTo]))
WITH ( SYSTEM_VERSIONING = ON (HISTORY_TABLE = [dbo].[UserHistory]))

INSERT INTO [User] (Username, FirstName, LastName, Emailaddress, VeritasTemplateId) Values ('Jdoe', 'John', 'Doe', 'jdoe@contoso.com', 1);
INSERT INTO [User] (Username, FirstName, LastName, Emailaddress, VeritasTemplateId) Values ('bclay', 'Ben', 'Clay', 'bclay@contoso.com', 2);
UPDATE [User] Set VeritasTemplateId = 3 where UserName = 'bclay';

Use [TemporalClient];

CREATE TABLE [dbo].[User](
	[UserId] [int] IDENTITY(1,1) NOT NULL PRIMARY KEY,
	[Username] [varchar](100) NULL,
	[FirstName] [nvarchar](100) NOT NULL,
	[LastName] [nvarchar](100) NOT NULL,
	[Emailaddress] [varchar](100) NOT NULL,
	[AuditAt] [datetime2](7) GENERATED ALWAYS AS ROW START NOT NULL,
	[AuditTo] [datetime2](7) GENERATED ALWAYS AS ROW END NOT NULL,
	[VeritasTemplateId] [int] NULL,
 CONSTRAINT [UKUser_Username] UNIQUE NONCLUSTERED ([Username] ASC),
 PERIOD FOR SYSTEM_TIME ([AuditAt], [AuditTo]))
WITH ( SYSTEM_VERSIONING = ON (HISTORY_TABLE = [dbo].[UserHistory]))

And the code, where I specify the columns to sync:

var serverProvider = new SqlSyncProvider(DBHelper.GetDatabaseConnectionString("TemporalServer"));
var clientProvider = new SqlSyncProvider(DBHelper.GetDatabaseConnectionString("TemporalClient"));
var setup = new SyncSetup("User");
setup.Tables["User"].Columns.AddRange("UserId", "Username", "FirstName", "LastName", "Emailaddress", "VeritasTemplateId");

var progress = new SynchronousProgress<ProgressArgs>(s =>
    Console.WriteLine($"{s.ProgressPercentage:p}:  \t[{s?.Source[..Math.Min(4, s.Source.Length)]}] {s.TypeName}: {s.Message}"));

var agent = new SyncAgent(clientProvider, serverProvider);

do
{
    try
    {
        Console.Clear();
        Console.ForegroundColor = ConsoleColor.Green;
        var s = await agent.SynchronizeAsync(setup, progress: progress);
        Console.ResetColor();
        Console.WriteLine(s);

    }
    catch (SyncException e)
    {
        Console.ResetColor();
        Console.WriteLine(e.Message);
    }
    catch (Exception e)
    {
        Console.ResetColor();
        Console.WriteLine("UNKNOW EXCEPTION : " + e.Message);
    }
    Console.WriteLine("Sync Ended. Press a key to start again, or Escapte to end");
} while (Console.ReadKey().Key != ConsoleKey.Escape);

That’s being said, the history table will NOT be updated on the client if you are making multiple changes on a row, between 2 syncs.

0reactions
Mimetiscommented, Nov 14, 2022

You can use DropAllAsync is you dont know how to remove DMS metadata manually

Read more comments on GitHub >

github_iconTop Results From Across the Web

4.4.1 Examples of Synchronizing
Example 4-4 Synchronizing Graphs Using CREATE PROPERTY GRAPH Statement ... CREATE TABLE persons ( person_id NUMBER GENERATED ALWAYS AS IDENTITY (START WITH ...
Read more >
Auto increment even and odd for two databases ...
Here is a very simple solution, but it will work only for two servers. It can't be easily extended for more servers.
Read more >
How to keep a table in sync with a view - SQL Server Forums
To create a job go to Object Explorer | <your server> | sql server Agent | Jobs. right click "jobs" and select "new...
Read more >
Enable edits on sync tables - Coda Pack SDK
Sync tables allow you to pull in data from external applications and sources, but by default these tables are read-only. You can make...
Read more >
Add sync tables - Coda Pack SDK
Create tables of data that automatically sync in records from an external data source. ... Properties of the schema designated as featuredProperties are ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found