Concurrent lookups of instances on a class sometimes result in NullReferenceException

Prerequisites

The issue tracker is used to report bugs and request new features, NOT to ask questions.

Questions should be posted to the users mailing list which can be accessed at https://ironpython.groups.io/g/users.

• [Y] Are you running the latest version?
• [Y] Are you reporting to the correct repository?
• [Y] Did you perform a cursory search?

Description

When object instances are defined on a class, concurrent lookups of that instance sometimes produce a NullReferenceException in PythonTypeUserDescriptorSlot.GetValue.

Steps to Reproduce

Run the following in C# Interactive. It eventually fails with a NullReferenceException.

#r "Microsoft.Scripting"
#r "IronPython"

using System;
using IronPython.Hosting;
using IronPython.Runtime;
using IronPython;
using Microsoft.Scripting.Hosting;
using Microsoft.Scripting;

var engine = Python.CreateEngine();
var scope = engine.CreateScope();
engine.SetSearchPaths(new[] { @"C:\Program Files\IronPython 2.7\Lib\" });

string script = @"
import threading
from multiprocessing.pool import ThreadPool


class Member(object):
    alpha = 0
    beta = 1


class Thing(object):
    member = Member
    member_instance = Member()


test_thing = Thing()


def query(trigger_event):
    trigger_event.wait()
    return [test_thing.member_instance for _ in range(10)][-1]


def single_test(number_of_threads):
    trigger = threading.Event()

    pool = ThreadPool(number_of_threads)
    results = [pool.apply_async(query, [trigger]) for i in range(number_of_threads)]
    trigger.set()
    [r.get() for r in results]
    pool.close()
    pool.join()


# Including the next line prevents NullReferenceExceptions in the threaded queries.
# print test_thing.member_instance

single_test(16)
";

while (true) {
    engine.Execute(script);
}

Expected behavior: Concurrent lookups each return the object instance, always.

Actual behavior: Concurrent lookups occasionally produce a NullReferenceException.

System.NullReferenceException: Object reference not set to an instance of an object.
  + Microsoft.Scripting.Interpreter.ThrowInstruction.Run(Microsoft.Scripting.Interpreter.InterpretedFrame)
  + Microsoft.Scripting.Interpreter.Interpreter.Run(Microsoft.Scripting.Interpreter.InterpretedFrame)
  + Microsoft.Scripting.Interpreter.LightLambda.Run3<T0, T1, T2, TRet>(T0, T1, T2)
  + System.Dynamic.UpdateDelegates.UpdateAndExecute3<T0, T1, T2, TRet>(System.Runtime.CompilerServices.CallSite, T0, T1, T2)
  + IronPython.Runtime.Method.MethodBinding.SelfTarget(System.Runtime.CompilerServices.CallSite, IronPython.Runtime.CodeContext, object)
  + System.Dynamic.UpdateDelegates.UpdateAndExecute2<T0, T1, TRet>(System.Runtime.CompilerServices.CallSite, T0, T1)
  + IronPython.Compiler.Ast.CallExpression.Invoke0Instruction.Run(Microsoft.Scripting.Interpreter.InterpretedFrame)
  + Microsoft.Scripting.Interpreter.Interpreter.Run(Microsoft.Scripting.Interpreter.InterpretedFrame)
  + Microsoft.Scripting.Interpreter.LightLambda.Run2<T0, T1, TRet>(T0, T1)
  + System.Dynamic.UpdateDelegates.UpdateAndExecute3<T0, T1, T2, TRet>(System.Runtime.CompilerServices.CallSite, T0, T1, T2)
  + IronPython.Compiler.Ast.CallExpression.Invoke1Instruction.Run(Microsoft.Scripting.Interpreter.InterpretedFrame)
  + Microsoft.Scripting.Interpreter.Interpreter.Run(Microsoft.Scripting.Interpreter.InterpretedFrame)
  + Microsoft.Scripting.Interpreter.LightLambda.Run1<T0, TRet>(T0)
  + IronPython.Compiler.RuntimeScriptCode.InvokeTarget(Microsoft.Scripting.Runtime.Scope)
  + Microsoft.Scripting.Hosting.ScriptSource.Execute()

Versions

2.7.11 (2.7.11.1000) [.NETFramework,Version=v4.5 on .NET Framework 4.8.4300.0 (64-bit)]

[slozier]

Hmm, I haven’t been able to reproduce the failure with 4.8 over here, but I guess it was still possible to end up in inconsistent state. Anyway, I pushed an update to the fix.

[mickp]

@slozier Thanks for the quick response. #772:

• does not resolve the issue on .Net Framework 4.8;
• does appear to fix it on .Net Core 3.1.