NC keys: When copying page nested content does not update properly

This issue description has been updated with a propsed solution and has been marked as up for grabs. Please read below about the original issue and the proposed solution. You can also scroll way down to read the entire discussion and history.

Proposed solution

Taking some inspiration from the media tracking branch and looking at what we already have in core:

• Nested content (and any other property editor) can handle the IDataValueEditor.FromEditor to re-populate any missing keys. Nested content already overrides this method so it would be just adding the logic to check for empty keys. Empty keys may be sent up if the client side clears them when copying them with the clipboard service. Nested content calls FromEditor recursively so if there’s an NC inside an NC this will still work.
• Create a new interface: IDataValueCopying which can be implemented by any IDataValueEditor (like like IDataValueReference for media tracking). It could have one method like: object Copy(object input);
• We create an external component to handle ContentService.Copying, the event handler will use the current IContent along with the PropertyEditorCollection to figure out if any of the IDataValueEditors being used for any of the properties implement IDataValueCopying and if so, will invoke the Copy method and set the new property value to the result.
• the NestedContentPropertyValueEditor then implements this interface and it will have to deal with having nested nested contents but that shouldn’t be too difficult.

Original submitted issue

When we copy pages with Nested Content and edit a published element the copied page still shows values from the original content. For example an image block where we changed the image still gives us the original image when checking the strongly typed model. The source value of the property does update properly, as demonstrated in the image below:

Reproduction

• Create a page with nested content
• copy the page
• change a property of a block in the nested content
• check the return values.

This bug appears in Umbraco version 8.0.2

---

This item has been added to our backlog AB#6684

[enkelmedia]

Thanks @creativesuspects! Still think that we need to address this in the core, we should not expect the average developer or beginner to find this issue here and apply the fix. Why not apply the fix in core until the underlying problem is solved? @Shazwazza what do you say?

[creativesuspects]

@nul800sebastiaan @rbottema Okay, so that seems to work. Also, I just realized I had previously been reloading Memory Cache before reloading the Database Cache, so that might have been an issue as well. Like I said, it was very late.

I have only tested it on a very small site with only a few content nodes, but I’m going to do some more testing on a larger site with multiple language variants to make sure it works.

So to sum it all up, here’s what I did:

1. I’ve confirmed that there were duplicate nested content keys in the database. These nested content values are showing the same output when loaded onto a page even though the content is different. But obviously that’s because Umbraco caches the content based on the keys which are duplicates.
2. I’ve created a simple API controller that replaces all the nested content keys in the umbracoPropertyData table using the logic from PR #8177, see code below. So you’d only have to execute this once: /umbraco/backoffice/api/ResetNestedContentKeys/GetItDone
3. Settings -> Published Status -> Rebuild Database Cache
4. Settings -> Published Status -> Rebuild Memory Cache
5. Reload affected page and you should see unique content for each nested content item

using System;
using System.Linq;
using System.Web.Mvc;
using Newtonsoft.Json.Linq;
using NPoco;
using Umbraco.Core;
using Umbraco.Core.Persistence;
using Umbraco.Core.Scoping;
using Umbraco.Web.WebApi;

namespace Application.Core.Controllers
{
    public class ResetNestedContentKeysController : UmbracoAuthorizedApiController
    {
        private readonly IScopeProvider _scopeProvider;

        public ResetNestedContentKeysController(IScopeProvider scopeProvider)
        {
            _scopeProvider = scopeProvider;
        }

        [HttpGet]
        public HttpStatusCodeResult GetItDone()
        {
            using (var scope = _scopeProvider.CreateScope(autoComplete: true))
            {
                var selectSql = scope.SqlContext.Sql(@"
                    SELECT id, textValue FROM umbracoPropertyData WHERE propertyTypeId IN (
                        SELECT id FROM cmsPropertyType WHERE dataTypeId IN (
                            SELECT nodeId FROM umbracoDataType WHERE propertyEditorAlias = 'Umbraco.NestedContent'
                        )
                    )
                ");
                var records = scope.Database.Fetch<dynamic>(selectSql);

                foreach (var record in records)
                {
                    var r = (PocoExpando)record;
                    var id = Convert.ToInt32(r.Values.First());
                    var text = r.Values.Last().ToString();

                    if (!string.IsNullOrEmpty(text))
                    {
                        var updatedText = CreateNestedContentKeys(text, false);
                        scope.Database.Execute("UPDATE umbracoPropertyData SET textValue = @0 WHERE id = @1", new object[] { updatedText, id });
                    }
                }
            }

            return new HttpStatusCodeResult(200);
        }

        private string CreateNestedContentKeys(string rawJson, bool onlyMissingKeys, Func<Guid> createGuid = null)
        {
            // used so we can test nicely
            if (createGuid == null)
                createGuid = () => Guid.NewGuid();

            if (string.IsNullOrWhiteSpace(rawJson) || !rawJson.DetectIsJson())
                return rawJson;

            // Parse JSON
            var complexEditorValue = JToken.Parse(rawJson);

            UpdateNestedContentKeysRecursively(complexEditorValue, onlyMissingKeys, createGuid);

            return complexEditorValue.ToString();
        }

        private void UpdateNestedContentKeysRecursively(JToken json, bool onlyMissingKeys, Func<Guid> createGuid)
        {
            // check if this is NC
            var isNestedContent = json.SelectTokens($"$..['ncContentTypeAlias']", false).Any();

            // select all values (flatten)
            var allProperties = json.SelectTokens("$..*").OfType<JValue>().Select(x => x.Parent as JProperty).WhereNotNull().ToList();
            foreach (var prop in allProperties)
            {
                if (prop.Name == "ncContentTypeAlias")
                {
                    // get it's sibling 'key' property
                    var ncKeyVal = prop.Parent["key"] as JValue;
                    // TODO: This bool seems odd, if the key is null, shouldn't we fill it in regardless of onlyMissingKeys?
                    if ((onlyMissingKeys && ncKeyVal == null) || (!onlyMissingKeys && ncKeyVal != null))
                    {
                        // create or replace
                        prop.Parent["key"] = createGuid().ToString();
                    }
                }
                else if (!isNestedContent || prop.Name != "key")
                {
                    // this is an arbitrary property that could contain a nested complex editor
                    var propVal = prop.Value?.ToString();
                    // check if this might contain a nested NC
                    if (!propVal.IsNullOrWhiteSpace() && propVal.DetectIsJson() && propVal.InvariantContains("ncContentTypeAlias"))
                    {
                        // recurse
                        var parsed = JToken.Parse(propVal);
                        UpdateNestedContentKeysRecursively(parsed, onlyMissingKeys, createGuid);
                        // set the value to the updated one
                        prop.Value = parsed.ToString();
                    }
                }
            }
        }
    }
}