Proposal: Ref Returns and Locals

(Note: this proposal was briefly discussed in dotnet/roslyn#98, the C# design notes for Jan 21, 2015. It has not been updated based on the discussion that’s already occurred on that thread.)

Background

Since the first release of C#, the language has supported passing parameters by reference using the ‘ref’ keyword, This is built on top of direct support in the runtime for passing parameters by reference.

Problem

Interestingly, that support in the CLR is actually a more general mechanism for passing around safe references to heap memory and stack locations; that could be used to implement support for ref return values and ref locals, but C# historically has not provided any mechanism for doing this in safe code. Instead, developers that want to pass around structured blocks of memory are often forced to do so with pointers to pinned memory, which is both unsafe and often inefficient.

Solution: ref returns

The language should support the ability to declare ref locals and ref return values. We could, for example, now declare a function like the following, which not only accepts ‘ref’ parameters but which also has a ref return value:

public static ref TValue Choose<TValue>(
    Func<bool> condition, ref TValue left, ref TValue right)
{
    return condition() ? ref left : ref right;
}

With a method like that, one can now write code that passes two values by reference, with one of them being returned based on some condition:

Matrix3D left = …, right = …;
Choose(chooser, ref left, ref right).M20 = 1.0;

Based on the function that gets passed in here, a reference to either ‘left’ or ‘right’ will be returned, and the M20 field of it will be set. Since we’re trading in references, the value contained in either ‘left’ or ‘right’ is updated, rather than a temporary copy being updated, and rather than needing to pass around big structures, necessitating big copies.

If we don’t want the returned reference to be writable, we could apply ‘readonly’ just as we were able to do earlier with ‘ref’ on parameters (extending the proposal mentioned in dotnet/roslyn#115 to also support return refs):

public static readonly ref TValue Choose<TValue>(
    Func<bool> condition, ref TValue left, ref TValue right)
{
    return condition() ? ref left : ref right;
}
…
Matrix3D left = …, right = …;
Choose(chooser, ref left, ref right) = new Matrix3D(...); // Error: returned reference is read-only

Note that when referencing the ‘left’ and ‘right’ ref arguments in the Choose method’s implementation, we used the ‘ref’ keyword. This would be required by the language, just as it’s required to use the ‘ref’ keyword when passing a value to a ‘ref’ parameter.

Solution: ref locals

Once you have the ability to receive ‘ref’ parameters and to return ‘ref’ return values, it’s very handy to be able to define ‘ref’ locals as well. A ‘ref’ local can be set to anything that’s safe to return as a ‘ref’ return, which includes references to variables on the heap, ‘ref’ parameters, ‘ref’ values returned from a call to another method where all ‘ref’ arguments to that method were safe to return, and other ‘ref’ locals.

public static ref int Max(ref int first, ref int second, ref int third)
{
    ref int max = first > second ? ref first : ref second;
    return max > third ? ref max : ref third;
}
…
int a = 1, b = 2, c = 3;
Max(ref a, ref b, ref c) = 4;
Debug.Assert(a == 1); // true
Debug.Assert(b == 2); // true
Debug.Assert(c == 4); // true

We could also use ‘readonly’ with ref on locals (again, see dotnet/roslyn#115), to ensure that the ref variables don’t change. This would work not only with ref parameters, but also with ref locals and ref returns:

public static readonly ref int Max(
    readonly ref int first, readonly ref int second, readonly ref int third)
{
    readonly ref int max = first > second ? ref first : ref second;
    return max > third ? ref max : ref third;
}

[MgSam]

If I recall, Eric Lippert blogged about this some years back and the response in the comments was largely negative.

I do not like this feature for C#. The resulting code is like an uglier version of C++, and code written with it takes longer to reason about and understand. The use-cases are not particularly compelling, and I have never run into a situation where I wished I had ref locals or return values.

[xen2]

Disclaimer: I work on game engine, so I am probably not the typical user.

One use case this could really help us is this one:

MyHugeStruct[] data; // we use a struct to improve data locality and reduce GC pressure
// Ideally, we would like to be able to use List<T>, but we can't take ref then
for (int i = 0; i < data.Length; ++i)
{
   // Option 1: make a local copy (slow)
   var item = data[i];

   // Option2: To avoid making a stack copy of MyHugeStruct,
   // we have to defer to a inner loop function
   MyLoopBody(ref data[i]);

   // Option3: using new proposal, that would be much better:
   ref MyHugeStruct = data[i];
}

We end up making separate function for loop body, and in case of tight loop this can end up being quite bad:

• Have to forward all parameters
• Sometimes we found out with VTune that inner loop stack “initlocals” was taking up most (80%+) of the time if inner loop body happened to have a several locals (even if only 0 or 1 was used due to branching). This would not happened if the locals were contained and memzeroed once in the function containing the “for” loop.
• not inlined in simple cases

Nice to have:

• ref this[] operator(?) so that List<> and other collections can be used (vs being forced to use arrays)
• a ++ operator on ref to be able to loop by incrementing pointer instead of indice multiplication (but probably unsafe).

Extra (probably impossible without changing BCL):

• Lot of struct copy could also be avoided in EqualityComparer (Dictionary) if ref could be used when large structs are being used as key.