[FEATURE REQ] Allow specific properties to be ignored on ITableEntity implementation

Library or service name. Azure.Data.Tables

Is your feature request related to a problem? Please describe. Currently all public instance properties on an ITableEntity implementation are serialized to Azure Table Storage. https://github.com/Azure/azure-sdk-for-net/blob/533ee2530f783f081d04c4bbcb79d483d9ef16a6/sdk/tables/Azure.Data.Tables/src/Extensions/TableEntityExtensions.cs#L24

For my scenario, I have several read-only (no setter), convenience properties that I do not want serialized to Table Storage. Also, I have other properties that have both getters and setters than I do not want serialized. This latter case are nullable enum properties that are workarounds for https://github.com/Azure/azure-sdk-for-net/issues/19769.

In both { get; } and { get; set; } cases, I want to be able to mark specific properties as ignored.

In WindowsAzure.Storage, you can do this with an [IgnoreProperty] attribute. There are also analogies in Newtonsoft.Json, System.Text.Json, etc.

One question is whether { get; } only properties should ever be serialized considering they fail round-tripping with this exception:

Result Message:	System.ArgumentException : Property set method not found.
Result StackTrace:	
at System.Reflection.RuntimePropertyInfo.SetValue(Object obj, Object value, BindingFlags invokeAttr, Binder binder, Object[] index, CultureInfo culture)
   at System.Reflection.RuntimePropertyInfo.SetValue(Object obj, Object value, Object[] index)
   at Azure.Data.Tables.DictionaryTableExtensions.ToTableEntity[T](IDictionary`2 entity, PropertyInfo[] properties)
   at Azure.Data.Tables.DictionaryTableExtensions.ToTableEntityList[T](IReadOnlyList`1 entityList)
   at Azure.Data.Tables.TableClient.<>c__DisplayClass33_0`1.<<QueryAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Azure.Core.PageableHelpers.FuncAsyncPageable`1.AsPages(String continuationToken, Nullable`1 pageSizeHint)+MoveNext()
   at Azure.AsyncPageable`1.GetAsyncEnumerator(CancellationToken cancellationToken)+MoveNext()
   at Azure.AsyncPageable`1.GetAsyncEnumerator(CancellationToken cancellationToken)+MoveNext()
   at Azure.AsyncPageable`1.GetAsyncEnumerator(CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at System.Linq.AsyncEnumerable.<ToListAsync>g__Core|620_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToList.cs:line 36
   at System.Linq.AsyncEnumerable.<ToListAsync>g__Core|620_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToList.cs:line 36
   at Knapcode.ExplorePackages.Worker.BaseCatalogScanIntegrationTest.AssertEntityOutputAsync[T](TableClient table, String dir, Action`1 cleanEntity) in C:\z\Git\joelverhagen\ExplorePackages\test\ExplorePackages.Worker.Logic.Test\TestSupport\BaseCatalogScanIntegrationTest.cs:line 90
   at Knapcode.ExplorePackages.Worker.FindLatestCatalogLeafScan.FindLatestCatalogLeafScanIntegrationTest.AssertOutputAsync(String dir) in C:\z\Git\joelverhagen\ExplorePackages\test\ExplorePackages.Worker.Logic.Test\CatalogScan\Drivers\Internal_FindLatestCatalogLeafScan\FindLatestCatalogLeafScanIntegrationTest.cs:line 118
   at Knapcode.ExplorePackages.Worker.FindLatestCatalogLeafScan.FindLatestCatalogLeafScanIntegrationTest.Internal_FindLatestCatalogLeafScan.Execute() in C:\z\Git\joelverhagen\ExplorePackages\test\ExplorePackages.Worker.Logic.Test\CatalogScan\Drivers\Internal_FindLatestCatalogLeafScan\FindLatestCatalogLeafScanIntegrationTest.cs:line 35
--- End of stack trace from previous location where exception was thrown ---

Workarounds:

• Implement these properties as GetFoo() or SetFoo(...) methods for ignored properties.
• Implement an IDictionary<string, object> instead of ITableEntity and manage the dictionary building yourself.

[jmelkins]

The difference between code examples is that the Json.NET serializer, unlike the one from System.Text.Json, recognizes the [IgnoreDataMember] leading to a clash.

The problem could be resolved by switching to System.Text.Json but if you want to stay with Json.NET then perhaps a suitable workaround would be to make properties opt-in for JSON serialization, which is supported by Json.NET?

using System;
using System.Runtime.Serialization;
using Newtonsoft.Json;

var person = new PersonTable
{
    PersonJson = "{\"FirstName\":\"John\",\"LastName\":\"Doe\"}",
    OtherProperty = "OtherValue"
};

Console.WriteLine($"FirstName: {person.Person.FirstName}");
var serialized = JsonConvert.SerializeObject(person);
Console.WriteLine($"Serialized: {serialized}");

public class Person {
	public string FirstName { get; set; }
	public string LastName { get; set; }
}

[JsonObject(MemberSerialization.OptIn)] // Makes properties opt-in for JSON rather than the default opt-out
public class PersonTable {

	public string PersonJson { get; set; }

    private Person _person;
    
	[JsonProperty]  // Opt-in to JSON serialization
	[IgnoreDataMember]
    public Person Person {
        get { 
            if (_person == null)
            {
                 _person = JsonConvert.DeserializeObject<Person>(PersonJson);
            }
            return _person;
		}
        set { 
            _person = value;
			PersonJson = JsonConvert.SerializeObject(_person);
	    }
	}
	
	[JsonProperty]  // Opt-in to JSON serialization
	public string OtherProperty { get; set; }
}

https://dotnetfiddle.net/hr4sre

Or taking the first code example posted:

[JsonObject(MemberSerialization.OptIn)]
public class MyPerson : TableEntity {
	//	...
	//	... Other properties
	//	...

	// This is the value that I want to persist in my Azure Table
	public string FullName { get; set; }

	// This is the property that I want to serialize into my JSON
        [JsonProperty]
        [IgnoreDataMember]
	public Person Person {
		get {
			var data = FullName.Split(' ');
			return new Person() { FirstName = data[0], LastName = data[1] };
		}
	}
}

You would need to add [JsonProperty] to all other properties that you want to appear in the JSON. Perhaps this can help?

[peter-bozovic]

Thanks @jmelkins, this is exactly what I needed ! I was sure there must be some way but was not aware of the MemberSerialization.OptIn option …

On the subject of System.Text.Json, I had the inverse problem in another project where I needed it to take into account the [IgnoreDataMember] (defined in another class I don’t have control over), but currently it doesn’t . Apparantly it’s planned for future version or maybe already inplemented in dotnet 6.

System.Text.Json has it’s own specific [JsonIgnore] and I think it was good to have also specific [IgnoreProperty] for Azure Tables. That allowed to fine tune which properties should be included for which logic. If tomorrow everyone uses just [IgnoreDataMember] it will not be possible to do such distinction anymore …