Idea Hub draft post ideas requested on every page load

Bug Description

This is a follow-up bug for #3733.

See https://github.com/google/site-kit-wp/issues/3733#issuecomment-886938896: Currently the request to get draft post ideas in the dashboard happens on every page load, which is undesirable for performance.

In addition, the getNewIdeas and getSavedIdeas selectors also consider draft posts for ideas (in order to exclude those from the responses), but the original fix doesn’t address that. Essentially, whatever mechanism to invalidate the cache based on Idea Hub post changes getDraftPostIdeas gets, we also need to do for getNewIdeas and getSavedIdeas.

---

Do not alter or remove anything below. The following sections will be managed by moderators only.

Acceptance criteria

• Whenever an Idea Hub-based post has its post status changed or is deleted, that time should be stored in a transient for an hour, essentially as a “when was any Idea Hub post last changed” value.
• On the client-side, Idea Hub caches should be invalidated for any already cached results that are older than that stored timestamp (if there is any). This should apply to any Idea Hub endpoint caches to get ideas.
• The server-side timestamp should be in a transient with one hour expiration, since it wouldn’t be meaningful after more than an hour anyway, as the client-side cache expiration is also one hour.

Implementation Brief

Generate last changed time for Idea Hub posts

In includes/Modules/Idea_Hub.php:

• Add a new const IDEA_HUB_LAST_IDEA_POST_UPDATED_AT = googlesitekit_idea_hub_last_idea_post_updated_at
• In $this->is_connected() ) function, add a hook for transition_post_status action (https://codex.wordpress.org/Post_Status_Transitions )
  • Return if the post type is not an idea hub post (see is_idea_post())
  • Update the transient IDEA_HUB_LAST_IDEA_POST_UPDATED_AT with the output of time() using the set method of the Transient class instance with $expiration param set to 0.
• Add a new instance of the Script_Data class to the Idea Hub assets returned from the setup_assets method. It should use the googlesitekit-idea-hub-data handler and the following options passed as the second argument to its constructor:
  • The global property should be _googlesitekitIdeaHub
  • The data_callback property should be a function that returns an array with the lastIdeaPostUpdatedAt property.

<del>Create an endpoint for last updated param for Idea Hub posts</del>

• <del>Create a new endpoint in get_datapoint_definitions() and create_data_request() for GET:last-changed-timestamp to return the IDEA_HUB_LAST_CHANGED transient value.</del>
• <del>In $this->is_connected() ) function, add a hook for googlesitekit_apifetch_preload_paths filter to preload the url. See the following for example: https://github.com/google/site-kit-wp/blob/bbc5d84de6869937294a5009a9c783c4ca5ae8d5/includes/Core/Feature_Tours/REST_Feature_Tours_Controller.php#L62-L71</del>

Add <del>a new fetch datastore and</del> a new selector to the Idea Hub module that will return the timestamp

• <del>Create a new data store via createFetchStore in assets/js/modules/idea-hub/datastore/last-changed-timestamp.js</del>
  • <del>const fetchGetLastChangedTimestamp</del>
  • <del>basename getLastChangedTimestamp
  • <del>return API.get( ‘modules’, ‘idea-hub’, ‘last-changed-timestamp’, undefined, { useCache: false, } );</del>
  • <del>In baseInitialState add lastChangedTimestamp: undefined</del>
  • <del>In baseResolvers and baseSelectors add getLastChangedTimestamp</del>
  • <del>Export the store.</del>
• Add a new action receiveIdeaHubData that receives an object with custom data for the Idea Hub module and returns an action with RECEIVE_IDEA_HUB_DATA type and the incoming data as the action payload object.
• Add a new case to the reducer for the new action that will add Idea Hub data to the state.
• Add a new resolver getLastIdeaPostUpdatedAt that calls the receiveIdeaHubData action and passes the _googlesitekitIdeaHub object into it.
• Add a new selector getLastIdeaPostUpdatedAt that returns the lastIdeaPostUpdatedAt value from the Idea Hub data state.

See how site info datastore works. It uses the same approach to read global variables and add them to the store state.

Update exsiting datastores to use last changed timestamp

Update the following three places in a similar manner:

• assets/js/modules/idea-hub/datastore/draft-post-ideas.js
• assets/js/modules/idea-hub/datastore/new-ideas.js
• assets/js/modules/idea-hub/datastore/saved-ideas.js

 // new-ideas.js

 const fetchGetNewIdeasStore = createFetchStore( {
 	baseName: 'getNewIdeas',
-	controlCallback: () => {
- 		return API.get( 'modules', 'idea-hub', 'new-ideas' );
+	controlCallback: ( { timestamp } ) => {
+		return API.get( 'modules', 'idea-hub', 'new-ideas', { timestamp } );
 	},
 	reducerCallback: ( state, newIdeas ) => {
 		return {
 			...state,
 			newIdeas,
 		};
 	},
+	argsToParams( { timestamp } ) {
+		return { timestamp };
+	},
 } );

 ...

 const baseResolvers = {
 	*getNewIdeas() {
 		const registry = yield commonActions.getRegistry();
 		const newIdeas = registry.select( MODULES_IDEA_HUB ).getNewIdeas();
 
 		// If there are already ideas in state, don't make an API request.
 		if ( newIdeas === undefined ) {
-			yield fetchGetNewIdeasStore.actions.fetchGetNewIdeas();
+			const timestamp = registry.select( MODULES_IDEA_HUB ).getLastIdeaPostUpdatedAt();
+			yield fetchGetNewIdeasStore.actions.fetchGetNewIdeas( { timestamp } );
 		}
 	},
 };

 ...

Test Coverage

• <del>Create a new test file in assets/js/modules/idea-hub/datastore/last-changed-timestamp.test.js</del>

• Update the following tests to handle cached data:

  • assets/js/modules/idea-hub/datastore/new-ideas.test.js
  • assets/js/modules/idea-hub/datastore/saved-ideas.test.js
  • assets/js/modules/idea-hub/datastore/draft-post-ideas.test.js

Visual Regression Changes

• N/A

QA Brief

Overview

This shouldn’t be QA:eng as it is user-facing, although you will need to do some slightly technical checks (looking at network requests) to completely verify the ACs have been met.

Set up

• You will need Idea Hub enabled and connected.
• Open two browser windows:
  • one with the Site Kit dashboard
  • the other showing the WordPress Posts page (/wp-admin/edit.php)
• On the tab showing the dashboard, open Chrome dev tools and select the Network tab. Select XHR along the top and using the ‘filter’ input, type ‘idea’ to filter for idea hub requests (see image below).

Requests are made to the endpoints on page load.

• Delete the application cache by going to the Application tab in dev tools and using the ‘clear all’ (🚫 ) button.
• Switch back to the network tab (see setup instructions and image above).
• Then refresh the browser.
• There should be 3 new Idea Hub requests to three different endpoints:new-ideas, saved-ideas and draft-post-ideas.

Subsequent page loads should then use the cache (no new requests).

• Fresh requests should only be made when something has changed, so refresh the Site Kit dashboard without interacting with the Idea Hub widget.
• There should be no new Idea Hub requests in the network tab (because the cache is being used).

Creating a draft idea post should invalidate the cache.

• Create a draft using the ‘create a draft’ button.
• The number in the ‘Drafts’ tab should increment by one.
• This should make a fresh request to the create-idea-draft-post endpoint and then wipe the cache.
• This means that if you refresh again, you should expect to see fresh requests made to the new-ideas, saved-ideas and draft-post-ideas endpoints.

Changing an Idea Hub post state should force fresh requests

• Having created a draft post using the widget, now switch to the other browser window (showing your WordPress posts) and refresh to see the draft post appear.
• Change the state of the post (for example, trash it).
• Now switch back to the browser window showing the Site Kit dashboard and refresh
• Fresh requests should have been made to all 3 endpoints, and the number in the ‘Drafts’ tab should be adjusted accordingly (e.g. if you trashed the post, it should decrement).

Changing a non-Idea Hub post state should not force fresh requests

• Switch back to the browser window showing the posts page.
• Change the state of a non-Idea Hub post.
• As before, switch back to the dashboard and refresh.
• No new requests should have been made.

Changelog entry

• Update Idea Hub new, saved, and draft ideas requests to invalidate the cache when a post created for an idea changes its status.

[felixarntz]

@tofumatt @johnPhillips I’ve opened this one to follow up on #3733 - let’s continue discussing potential approaches here.

[ivankruchkoff]

Update: Now that the API is working again: QA ✅ @eugene-manuilov yes, I can confirm that we’re invalidating the cache for the three endpoints.

If we don’t want to change the functionality, perhaps we should change lastIdeaPostUpdatedAt to better reflect that it doesn’t account for new draft posts being created, and also that it’s initialised with the value false rather than when the idea hub module is enabled.