Better documentation on the revalidation behavior of the infinite hook

Bug report

Description / Observed Behavior

I have created a simple component that fetches a list of posts via GraphQL requests (cursor-based) using useSWRInfinite. Apparently everything works fine, i.e. the pages are all fetched correctly, in the correct order, etc. but when inspecting the ‘Network’ tab I can see that every time I load a new page, the first page is also fetched again. So, when I click ‘Load more’ two requests are made, one for the next page, the other for the first page (unnecessary).

I’m reporting it as a bug because I’ve seen it happen with the official infinite loading example, although only occasionally (in my app it always happens). I think the problem has something to do with the fact that the ‘index’ that is passed to ‘getKey’ correctly increases by 1 when loading more, but on re-render it resets to 0 (this always happen in the official example, too), so when the component re-renders and executes ‘getKey’ it fetches the first page again because ‘index’ is 0 again.

Expected Behavior

I would expect that only the request for the next page is made.

Repro Steps / Code Example

Here’s my component:

// Index.js 

const Index = ( { posts } ) => {	

  const getKey = ( index, previousData ) => {

    console.log( "index", index ) // 'index' increases by 1 after loading more, then resets to 0
    if( index === 0 ) return [ `/libs/api/posts`, "" ]
    
    return [ `/libs/api/posts`, previousData?.pageInfo?.endCursor ]
  
  }

  const { 
    data, error, mutate, size, setSize, isValidating 
  } = useSWRInfinite( getKey, getAllPosts, { initialData: [ posts ] } )

  return(
    <div>
      <LogInOutLink />
      {
        data.map( ( page, index ) => {
          return(
            <div key={ index }>
              <h3>Page n. { index }</h3>
              <ul>{ page.edges.map( ( { node } ) => <li key={ node.id }>{ node.title }</li> ) }</ul>
            </div>
          )
        })
      }
      <button onClick={ () => setSize( size + 1 ) }>Load more</button>
    </div>
  )	

}

export const getStaticProps = async () => {

  const posts = await getAllPosts( "/libs/api/posts" )

  return { props: { posts } }

}

export default Index

And here’s my fetcher:

const getAllPosts = async ( key, cursor = "" ) => {
   
   const query = `
      query getPosts( 
         $after: String, 
         $first: Int = 8 	
      ) {
         postsRoot: contentNodes(	
            after: $after, 
            first: $first, 
            where: { contentTypes: [ POST, NEWS ] } 
         ) {
            ${PAGINATION_FIELDS}
            ${POSTS_LIST_FIELDS}
         }
      }
   `
   
   const variables = cursor ? { after: cursor } : {}

   const data = await fetchAPI( query, variables )
 
   return data?.postsRoot

}

export default getAllPosts

export const fetchAPI = async ( query, variables = {} ) => {
 
   const res = await fetch( `${process.env.NEXT_PUBLIC_BACKEND_BASE_URL}/graphql`, {
      method: "POST",
      headers: {
         "Content-Type": "application/json"
      },
      body: JSON.stringify({
         query,
         variables
      })
   })
 
   const json = await res.json()
   
   if( json.errors ) {
     console.error( json.errors )
     throw new Error('Failed to fetch API')
   }
   
   return json.data

}

Additional Context

I encountered the problem with the latest version (0.3.9), so I also tried with 0.3.0 but to no luck.

[yume-chan]

Because most apps with the infinite loading design work like a timeline (think about FB and Twitter), new changes will mostly come up at the top.

However, in a typical timeline there are also things like comments, like count, etc. that need to be updated. useSWRInfinite only updates the first page (if no new post returned), that sounds strange.

I know the existence of { revalidateAll: true}, which I believe is never a good idea. Due to useSWRInfinite only loads pages in sequence (or due to cursor-based pagination can only load pages in sequence), when a user already has tens or hundreds of pages loaded, I can’t imagine how long it will take to load the next page. And due to useSWRInfinite only returns data after all pages are loaded, it’s basically same as no pagination (or even worse due to extra requests).

If I understand correctly, when there are new items in the first page, even with { revalidateAll: false } (which is the default) it will still reload all pages. I think I don’t need to repeat the result.

I’m prototyping a Twitter-like SNS, so I recently checked how Twitter works. It has two cursors, “top” and “bottom”, so it can load new tweets from both ends. It also uses server-side events to update like/reshare/comment counts, instead of reloading each pages.

Anyway, I don’t expect useSWRInfinite to fit all usages because there are infinite different methods to load an infinite list. But the behavior now is especially strange and not useful.

[shuding]

Thanks @yume-chan, very helpful feedback! I think overall infinite loading is a tricky problem, like you said a solid API implementation like Twitter needs bi-directional cursoring and some pushing mechanism because we can’t afford reloading every page, but every page can be changed. { revalidateAll: false } is good for the case that list changes will usually be popped up to the top, and smaller changes for each page such as comments/likes are controlled by useSWR hooks inside that item.

That said, I’d love to do more research for a better API design that:

• Supports the Twitter-like complex use case
• Has the ability to revalidate/mutate a specific page
• Easier to write