Realm create running out of memory and crashing

Goals

download and insert 19000+ records into the database paginated by 200 per request

Expected Results

Insert 19000+ records into database

Actual Results

After about 9000 records, the app crashes, and if i check XCode, it says terminated due to memory issues

There are no leaks, that i know of, i just leave it to download and eventually it crashes. Another thing to note is that each time create is called it gradually get’s longer and longer every 200 records. it starts at about 0.8s for 200 then right before it crashes it get’s up to about 9.0s, i once deleted most of the UI elements to purely let it download and it got up to 14000 records total, and the last 200 it tried to insert took 50.0+ seconds.

Another curiosity is that upon booting the project back up, it loaded those 14000 records in barely anytime at all and the memory consumption barely moved. My current findings seems to be that calling create periodically is causing the issue.

14000 recrods over 70 fetch requests.

Steps to Reproduce

TBD I don’t have a surefire way as of yet, i will try and create a RN project and emulate it as best i can to see if i can get it happen in a more isolated environment.

Code Sample

currently i have a singleton realm.js file that should be instantiated once it can be found here

My api call uses RXJS to do paginated calls like so

static fetchAllAttendees = (path: string, authToken: string) => {
    return from(AttendeeFetcher._fetchAllAttendees(path, 1, authToken)).pipe(
      expand(({ meta }) => {
        return meta.pagination_info.next_page
          ? AttendeeFetcher._fetchAllAttendees(
              path,
              meta.pagination_info.next_page,
              authToken
            )
          : empty()
      }),
      map((x: any) => plainToClass(Attendee, <any[]>x.attendees)),
      catchError(err => throwError(err))
    )
  }

and i use class-transformer lib to transform plain JSON into a typed class with plainToClass

and here is where i subscribe to get the attendees

networkRouter
  .get(NetworkResource.Attendee, NetworkPath.getPaginated, {
    id: this.party.id,
    authToken: user.authToken
  })
  .subscribe({
    next: attendees => {
      reactotron.logImportant!(attendees)
      let bench = reactotron.benchmark!("attendee write")
      EXRealm.shared.write(() => {
        let i = 0
        let length = attendees.length
        for(; i< length; i++) {
          EXRealm.shared.create(Attendee.schema.name, attendees[i], true)
        }
        bench.step("created/updated attendees")
      })
      bench.stop("attendee write done")
    },
    error: reactotron.logImportant!
  })

i also create an observable out of the realm listener to response to updates

this.attendeeListener = EXRealm.shared
  .objects<Attendee>(Attendee.schema.name)
  .filtered(`partyId = ${this.party.id}`)
  .sorted("name", false)

this.realmAttendees$ = new Observable<AttendeeChanges>(observer => {
  const handler = (
    attendees: Collection<Attendee & Realm.Object>,
    changes: CollectionChangeSet
  ) => observer.next([attendees,changes])
  
  this.attendeeListener.addListener(handler)
  return () => this.attendeeListener.removeListener(handler)
})

Version of Realm and Tooling

• Realm JS SDK Version: 3.1.0
• Node or React Native: React Native
• Client OS & Version: iOS 12
• Which debugger for React Native: Reactotron

Im not certain if related or not but i did come across #2499 I will continue to tinker with it to see if it’s perhaps something I’m doing wrong and will update accordingly, but thought i should at least get this posted to see if anyone has any insight?

[Genhain]

@kneth I also changed from RXJS to just a recursive function, my initial worry about why the write times where getting larger per iteration was because React Native JS (or in my case TS) is not tail call optimised. But as I said the growing write times were solved when I removed the relationships from the schema.

It could also be interesting to know how large the Realm file is after 1000 or 10000 inserts.

I never actually checked the realm file size itself, i was just checking the memory usage of the device. The memory usage still grows and does not shrink, But until more tests are done i suspect that is more my own doing than Realm, potentially both. Either way ill try calling Realm.compact() and let you know what i find.

[kneth]

I assume that Realm.compact() has helped. The configuration option shouldCompactOnLaunch can also be useful in these situations.