Slowness on single item transaction, but with a huge number of other items

Goal

This is not the first time I’m opening issue about speed, but I think this case is probably hopeless.

My goal would be to get some kind of advice on how should I be using Realm, in order to avoid or circumvent a slowness issue when creating executing single transactions on a single item, but there are a lot of other items.

Expected Results

Transactions on single items should be able to be handled in the UI thread.

Actual Results

Transactions on single items when there is a huge number of other items are slow and, apparently, shouldn’t be handled in the UI thread.

Steps & Code to Reproduce

Describe your current debugging efforts.

I’m developing an Android application where I have a similar schema as the following example (but more complex):

public class Project extends RealmObject {

    @PrimaryKey
    public Long id;

    public RealmList<Item> items;
}

public class Item extends RealmObject {

    @PrimaryKey
    public long id;

    public boolean isActive = false;

    public RealmList<Note> notes;
}

public class Note extends RealmObject {

    @PrimaryKey
    public long id;

    public String text;
}

Summary: Project has a list of Items who has a list of Notes.

I display the project’s items in a RecyclerView, and I add a RealmChangeListener to the items RealmList, to listen for changes and call notifyDataSetChanged().

My issue is as follows:

• each project can have a huge number of items, like 20K
• when there is a huge number of items in the project, simple transactions like a single item.isActive = !item.isActive, start taking too much time
• I stop being able to run this transactions in the UI thread, because of breaks
• the item won’t update immediately if I make the transaction in the background

I made a test project to repeat the following transaction 1000 times and calculate the average time spent in transaction.

realm.executeTransaction(new Realm.Transaction() {
    @Override
    public void execute(@NonNull Realm realm) {
        // item is always the same.
        item.isActive = !item.isActive;
    }
});

From my testing, these are the average times:

• first run:
  • 1 project
  • 1 item in project
  • 4 notes per item (4 in total)
  • average time: 5.415ms
• second run:
  • 1 project
  • 200 items in project
  • 4 notes per item (800 in total)
  • average time: 6.219ms
• third run:
  • 1 project
  • 20000 items in project
  • 4 notes per item (80000 in total)
  • average time: 76.023ms -> that’s a noticeable break

The tests were run on a Xiaomi Mi A1 device.

This times are already not perfect, but my issues is even worse, since the Schema is naturally more complex. There are times when I’m spending 200ms to update a single item.

Should I just take an optimistic approach on actions (like when activating or deactivating an item), and use executeTransactionAsync?

Version of Realm and tooling

Realm version: 4.1.0

Which Android version and device: 7.1.2 and Xiaomi Mi A1

Follow-up on @cmelchior

@cmelchior: Do you have any RealmListeners registered when doing those transactions? Because most likely you are running into calculating changesets. This is normally only a problem when you have schemas with circular references, which does not appear to be the case?

Since I opened #4544, I knew about the issues regarding schemas with circular references, and while my Schema does have circular dependencies between 2 classes (Project has Items, and Items may have a Project), the first thing I tested for was removing those, but the result was the same (I was not testing with average times, so I can’t tell for sure). However, as you can see in the Schema above, there are no circular dependencies and the results are not very good.

I do have 2 or 3 RealmChangeListeners in my main application. In the example above I only have one. I tried to remove it and the times are much better, indeed, like with a single item:

• fourth run:
  • 1 project
  • 20000 items in project
  • 4 notes per item (80000 in total)
  • no RealmChangeListener attached to project.items
  • average time: 5.126ms

So, should I remove all listeners before doing any transactions?

Note: this issue was originally posted in https://forums.realm.io/t/slowness-on-single-item-transaction-but-with-a-huge-number-of-other-items/684

[beeender]

realm-cocoa has a method commitWriteTransactionWithoutNotifying()

https://realm.io/docs/objc/3.0.2/api/Classes/RLMRealm.html#/c:objc(cs)RLMRealm(im)commitWriteTransactionWithoutNotifying:error:

Commits all write operations in the current write transaction, without notifying specific notification blocks of the changes. After saving the changes, all notification blocks registered on this specific RLMRealm instance are invoked synchronously. Notification blocks registered on other threads or on collections are scheduled to be invoked asynchronously. You can skip notifiying specific notification blocks about the changes made in this write transaction by passing in their associated notification tokens. This is primarily useful when the write transaction is saving changes already made in the UI and you do not want to have the notification block attempt to re-apply the same changes. The tokens passed to this method must be for notifications for this specific RLMRealm instance. Notifications for different threads cannot be skipped using this method. This method can fail if there is insufficient disk space available to save the writes made, or due to unexpected i/o errors.

But it is actually for other purposes which is only useful for cocoa. I would think use async transaction will be a better solution for this case.

The possibility is we might be able to supply a similar API like Realm.commitTransactionWithoutNotifyingListener(RealmChangeListener listener) to skip the change set population once. But there are quite a lot of side effect of this API and it is quite confusing for users i think.

I am closing this issue now. Hope the discussion cleared your doubt about this. Thanks a lot for the feedback!!

[jpmcosta]

@beeender no and I plan to use executeTransactionAsync. However, I was pushing the discussion a bit, because there are cases where I need to create objects, and on those cases I will still need to wait for them.

In addition to using executeTransactionAsync I will try to improve the Schema the best I can and try to manage the listeners better.

Thanks for all your help!