If placeholders are disabled on PagedList, BoundaryCallback gets called abnormally often

Update. I figured the problem was that I disabled placeholders. To prove it, I created a fork of the android-architecture-components repo and made a commit in PagingWithNetworkSample, in which I disabled placeholders (https://github.com/Svechnikov/android-architecture-components/commit/b578dc565375036e4b16ded91a4f39c444e798e5). After that I saw the same problem in the official app: start scrolling down and after 10 pages of data loading goes crazy and starts loading page after page until you close the app.

Here are logs from the official app with disabled placeholders:

01-31 10:53:46.999  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?limit=10
01-31 10:53:47.747  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_aljui4&limit=10
01-31 10:53:48.041  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_alc2st&limit=10
01-31 10:53:48.325  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_al2mft&limit=10
01-31 10:53:55.781  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_al19eo&limit=10
01-31 10:53:56.233  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_al0vch&limit=10
01-31 10:54:00.124  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_aklj8v&limit=10
01-31 10:54:01.587  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_akmawy&limit=10
01-31 10:54:03.287  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_akd9ok&limit=10
01-31 10:54:05.830  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_ajyqmk&limit=10
01-31 10:54:07.864  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_ak1g9n&limit=10
01-31 10:54:08.161  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_ak5f6a&limit=10
01-31 10:54:08.447  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_ajux7n&limit=10
01-31 10:54:08.778  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_ajk3w7&limit=10
01-31 10:54:09.080  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_ajmfyt&limit=10
01-31 10:54:09.423  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_ajo9oz&limit=10
01-31 10:54:09.707  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_ajddat&limit=10
01-31 10:54:09.987  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_ajicu3&limit=10
01-31 10:54:10.297  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_ajdtv0&limit=10
01-31 10:54:10.658  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_aiwp8m&limit=10
01-31 10:54:10.943  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_aj7sx9&limit=10
01-31 10:54:11.224  8404  8456 D API     : --> GET https://www.reddit.com/r/androiddev/hot.json?after=t3_aj612t&limit=10

Note the spike in loading after 10:54:08.161.

I’m gonna describe my situation:

I have an app where a user scrolls a RecyclerView list with infinite scroll. RecyclerView is backed by PagedListAdapter. which gets PagedLists through the PagedList.submitList method by observing LiveData<PagedList<MyView>>. I have a BoundaryCallback which triggers loading from network and inserts data to my db. Everything is almost the same as in the PagingWithNetworkSample.

However, after some fixed number of pages scrolled my BoundaryCallback start being triggered constantly as if after each load of data I instantly scroll to the end of the list so the load has to be performed again (but I don’t even scroll the list). At the same time the contents of the list change with every load (but I repeat there’s no scrolling). It continues until there’s no more data to load. After that if I close and return to the app, everything works fine (no network calls, items are queried from the db).

I figured out the following condition in PagedList is true when it has to be false (which leads to the problem)(https://github.com/xcesco/kripton/blob/90de2c0523d39b99e81b8d38aa996898762f594a/kripton-arch-integration/src/main/java/android/arch/paging/PagedList.java#L468):

mHighestIndexAccessed >= size() - 1 - mConfig.prefetchDistance;

After some number of page loads the size gets smaller (although I specified maxSize as PagedList.Config.MAX_SIZE_UNBOUNDED), at this moment the mHighestIndexAccessed variable doesn’t reflect the real situation (it has index much bigger than the item on the screen). From now on the condition mHighestIndexAccessed >= size() - 1 - mConfig.prefetchDistance is always true and BoundaryCallback gets notified every few hundred of milliseconds.

I wrote down values of mHighestIndexAccessed and size() before each dispatchBoundaryCallbacks.

For example, with pageSize = 30 (the first value is mHighestIndexAccessed and the second is size()):

39 - 69 (difference is 30, as it should be with pageSize=30)
49 - 79
59 - 89
68 - 98
48 - 78
78 - 88 (note how 48 became 78)

With pageSize = 10:

10 - 20
20 - 30
19 - 29
28 - 29 (again, we were 10 items away from the end of the list and suddenly we are 1 item away)

Unfortunately I cannot prepare a sample right now.

I have a feeling I’m missing something. Maybe somebody was faced with the same problem?

[aftabsikander]

I have the same issue. Even If I don’t scroll onItemAtEndLoaded keeps going crazy and kept calling network API until I close the application.

[dlam]

The old Paging2 helpers for request deduplication can be found here: https://github.com/android/architecture-components-samples/blob/paging2/PagingWithNetworkSample/app/src/main/java/androidx/paging/PagingRequestHelper.java

The starting point for request de-duplication for Paging3 is here - which is more of a Coroutines approach that you might find helpful for reference is here: https://source.corp.google.com/aosp-androidx/paging/common/src/main/kotlin/androidx/paging/RemoteMediatorAccessor.kt;l=44?q=RemoteMediatorAccessor&sq=package:android

Unfortunately request de-duplication is not built into paging2 so you’ll need to turn to these kind of third-party helpers or by building something yourself 😕

Gonna close this out, but happy to continue replying here in case anyone has questions. But mostly, this is built into paging3 now so another reason to upgrade!