Performance impact with huge data insertions

We have an API paginated over 5000 records with 500 per page. Each page’s response is of 550KB in size. As soon as a page’s response is obtained, we write the data to the realm DB using realm.create() method. What we have observed is, when the Realm table has 2000 records inserted, the insertions from 2001-5000 are taking 2seconds per insertion. This is again causing an issue to the UI in our react native app.

Also, we have another API with 50000 records paginated with 500 records per page. Each page’s response is of 50KB size. The insertions of 50000 records took nearly 2 minutes 50 seconds. But for 5000 records, it took 24 minutes, surprisingly.

Our app’s UI was unresponsive for almost 24 minutes due to such slow insertions. Is there a way, we can optimise the insertions of such a huge data?

Version of Realm and Tooling

• Realm JS SDK Version: 2.21.1
• Node or React Native: node-8.12.0, RN-0.57
• Client OS & Version: Android Oreo (8.0)
• Which debugger for React Native: Samsung Galaxy Tab S3 device, running Android Oreo.

[kmvkrish]

@bmunkholm We have updated out business logic to handle the sorting and finding uniques rows after the insertions are finished. This gave us a huge performance improvement. Thanks.

[bmunkholm]

You should ensure you have a primary key for the properties you sort on.

But also find a way where you don’t have to sort 10000 records 5000 times. Can’t you do that after you have inserted all records?