Strange assert, unmoved heapptr was selected for finalization

Were there any major changes to garbage collection?

Take this with a grain of salt since usually the bugs I present is some weird thing I messed up in my side, but this was a strange one.

In my app I have certain indices in the stash reserved for some object pools. The pools get allocated at the beginning as arrays, and they are left there. I simply grab the heapptr and add/remove objects to that array. I haven’t had any issues with it ever, suddenly today I got an assert triggered when I pushed the stashed pool’s heapptr:

/* One particular problem case is where an object has been
		 * queued for finalization but the finalizer hasn't been
		 * executed.
		 */
		duk_heaphdr *curr;
		for (curr = thr->heap->finalize_list;
		     curr != NULL;
		     curr = DUK_HEAPHDR_GET_NEXT(thr->heap, curr)) {
			DUK_ASSERT(curr != (duk_heaphdr *) ptr);
		}

Again, I might have corrupted something somewhere, who knows, just looking for some info in case there were some changes that might have caused this? This portion of my code (with the pool) hasn’t been touched in a very long time…

The heapptr which is an array inside an array inside the heap stash is only ever grabbed from the stash index once at the beginning, the pushed into the stack only 3 places in the code. The stash is not messed with again other than in initializing those arrays at the beginning.

If it helps, the objects that are stored in that pool (the heapptr that caused the assert) are revived when being finalized. That is, the pool is full of preallocated Ecmascirpt objects that I grab from the pool return them when I need one, then nullify that object’s index. When the object get’s finalized I keep it alive by storing it back in the pool.

[svaarala]

I think I might have found the issue – if it turns out to be correct, there’s no functional problem, and the finalize_list assertions are a bit bogus when running mark-and-sweep finalizers.

In more detail:

The mark-and-sweep finalizer execution code walks heap->finalize_list and queues objects back to heap_allocated after running the finalizers. The finalize_list is walked in its entirety, and once all objects have been queued to heap_allocated (they all are, regardless of what happens with the finalizer), the finalize_list pointer is set to NULL for the next round. At this point all the heap objects are again correctly accounted for.

But this means that after the first finalizer has executed, heap->finalize_list still points to that first object. So when running the second finalizer, heap->finalize_list points to that first object which has now been queued back to heap_allocated – and so the finalize_list is essentially pointing to heap_allocated!

This is not a functional issue, because while we run the finalizers, no-one is operating on the finalize_list pointer. So it doesn’t matter it’s “out of sync”, and the code is intentionally written that way because it’s not an issue. Or rather, it wasn’t an issue until the assert was added.

If this is indeed the case, the easy fix is to keep heap->finalize_list valid during finalization.

For this to trigger, one needs:

• Assertions enabled.
• Finalizer executed by mark-and-sweep, not refcounting. So either refcounts disabled, debugger pause causing normally refcount finalized objects becoming mark-and-sweep finalized, objects in circular references, etc.
• At least two objects in finalize_list.
• Finalizer using duk_push_heapptr().

These weren’t covered simultaneously by current test cases 😃

[fatcerberus]

@harold-b If it helps you manage heap pointers any better, feel free to use these ref/unref functions I came up with. Just replace your duk_get_heapptr() calls with duk_ref_heapptr() and you get a pointer that’s guaranteed to remain reachable until you explicitly unref it (note: it’s refcounted too). This way you don’t have to manage the stash manually.

duk_ref_heapptr(): https://github.com/fatcerberus/minisphere/blob/master/src/engine/utility.c#L157-L200

duk_unref_heapptr(): https://github.com/fatcerberus/minisphere/blob/master/src/engine/utility.c#L235-L271