Pagination idea: type-specific refetching plugins

In some sense, handling pagination in a smart way is just a special case of “when you ask for data, treat certain fields in a special way because we know that the arguments are meaningful.” So if we’re looking at a Relay spec paginated list:

// first fetch
{
  user {
    id
    name
    friends(first: 10) {
      edges {
        cursor
        node {
          id
          name
        }
      }
      pageInfo {
        hasNextPage
      }
    }
  }
}

// new query
... same stuff
friends(first: 20)

// second fetch, doesn't fetch all 20 but uses the existing information
... same stuff
friends(first: 10, after: "lastCursor")

Notice how easy it is for us as humans to imagine what data needs to be fetched to satisfy the new query.

Here’s a set of hypotheses:

1. The initial fetch doesn’t need any up-front information about types or pagination - it can look at the query result to know what to do, as long as we inject __typename fields where necessary.
2. The re-fetch can determine the new query by the information in the store, which now has type annotations and the arguments from the new query
3. The transformation from (2) can be written as a pure function that can be injected into the apollo client and associated with certain type names

Basically, you could write a function and tell the apollo client:

“When refetching any field which refers to an object with a type matching this regular expression, give me the query you were going to fetch, and the contents of the store, and I’ll give you a new query that says how to fetch the missing data.”

So, for Relay pagination, you’d say:

client.registerPaginationPlugin(/.+Connection/, ({ state, id, selectionSet }) => {
  ... do work ...

  return {
    // as much of the result as can be found
    result,

    // if empty, then the cache is sufficient and result contains the data.
    // otherwise, an array of queries that need to be fetched
    missingSelectionSets, 
  }
});

Ideally this will allow plugging in to different pagination systems, as long as the paginated fields have predictable type names, for example *PaginatedList or Paginated*.

If we can do this, it will achieve some really nice effects:

1. You don’t necessarily need to use the Relay pagination spec, which can be hard to translate to some REST APIs
2. The store can avoid being concerned with pagination if the API above is indeed sufficient

This is just a very ambitious idea, and I’m eager to determine if this very simple model can actually handle all cases of pagination and Relay connections in particular. More analysis to come soon.

@jbaxleyiii @helfer curious what you think about this.

[stubailo]

We’re going to start working on pagination full-time soon, I’ll take a deeper look at this then!

[deoqc]

Any updates in pagination? I’m using a custom solution, without a very good result.

Really hoping to have an Apollo solution to this.

---

Pagination (see bellow) is as high-order element that initialize/fetch data on mount, delete on unmount, and expose 2 props to its children:

• data with the query result
• addFetch to call when needing to fetch more data (I use it in react native ListView in onEndReached).

Problems:

• Duplicated data: I actually have a reducer pagination in my redux store and aggregate the results of pagination fetching. This is only temporary since I destroy on unmount and trust the apollo client query to memoize results.
• Performance: I’m feeling its pain and believe its because of the duplicate the data fetching redux logic (first apollo query, then I use the result and aggregate in pagination.[config.name] branch in store. I’m using immutablejs in the store.

---

Here is the complete gist, and bellow the main parts and some explanation.

const query = `
  query doSomeQuery(..., $first: Int!, after: String) { # --> must accept these two parameters (after needs to allow null)
    viewer {
      nest {
         myPaginated($first: Int!, after: String) {
           edges { # --> data array must be in edges
             node {
               ...
             }
           }
           pageInfo {  #  --> must have a pageInfo with at least these two fields
             hasNextPage
             endCursor
           }
         }
       }
    }
  }
`

Now I can abstract the pagination to a custom connector:

const variables = (state, ownProps) => ({
  // custom variables: will fixed after first call
});

const config = {
  name: 'myUniquePaginationName',
  query,
  path: ['viewer', 'nest', 'myPagination'], // need to now where the pagination part is so I aggregate data
  initCount: 10, // $first parameter in initial fetching
  addCount: 10, // $first parameter in adicional fetching
  variables,
};

const ComponentWithPagination = Pagination(config)(Component);