Tell search engines about your new pages

Every day at 07:00, your sitemap is read, the pages that are new or changed since the last run are picked out, and they are submitted for indexing through IndexNow, the shared protocol behind Bing, Yandex, Naver, Seznam and Yep. Google does not take part in IndexNow and still finds pages by crawling your sitemap, so this speeds up the other engines rather than all of them.

Your Google Sheet is the memory. An 'Index submissions' tab is created on the first run and each submitted page becomes one row: the date, the page address, its last modified date, where it was sent and the result. A page is only sent again when its last modified date in the sitemap changes, so nothing is submitted twice for the same version. Pages are submitted before anything is recorded, so a failure part way through means the next run offers the same pages again rather than losing them.

You need an IndexNow key saved in a text file at the root of your site, and a sitemap of pages rather than a sitemap index.

Run every day at 07:00.Run every day at 07:00.

Set sitemapUrl to blank.Set sitemapUrl to blank.

Set indexNowKey to blank.Set indexNowKey to blank.

Set spreadsheetUrl to blank.Set spreadsheetUrl to blank.

Set sheetTab to Index submissions.Set sheetTab to Index submissions.

Set maxPagesPerRun to 25.Set maxPagesPerRun to 25.

Set keyFileUrl to blank.Set keyFileUrl to blank.

Set indexNowEndpoint to https://api.indexnow.org/indexnow.Set indexNowEndpoint to https://api.indexnow.org/indexnow.

Set pageLimit to maxPagesPerRun multiplied by 1.Set pageLimit to maxPagesPerRun multiplied by 1.

If pageLimit is greater than or equal to 1 is noIf pageLimit is greater than or equal to 1 is no

Yes

Stop and report a problem: Set the 'maxPagesPerRun' step to how many pages may be submitted in one run, as a whole number - for example 25. It is currently: maxPagesPerRun.Stop and report a problem: Set the 'maxPagesPerRun' step to how many pages may be submitted in one run, as a whole number - for example 25. It is currently: maxPagesPerRun.

No

No steps

If pageLimit is greater than 500If pageLimit is greater than 500

Yes

Stop and report a problem: The 'maxPagesPerRun' step is set higher than 500. Search engines throttle and then ignore bulk submissions, so keep it at 500 or fewer. It is currently: maxPagesPerRun.Stop and report a problem: The 'maxPagesPerRun' step is set higher than 500. Search engines throttle and then ignore bulk submissions, so keep it at 500 or fewer. It is currently: maxPagesPerRun.

No

No steps

Trim indexNowKey and save it as submissionKey.Trim indexNowKey and save it as submissionKey.

Get the first item of the last item of sitemapUrl split by :// split by / and save it as siteHost.Get the first item of the last item of sitemapUrl split by :// split by / and save it as siteHost.

If siteHost is emptyIf siteHost is empty

Yes

Stop and report a problem: Could not work out which site the 'sitemapUrl' step belongs to. It should look like https://example.com/sitemap.xml, not: sitemapUrl.Stop and report a problem: Could not work out which site the 'sitemapUrl' step belongs to. It should look like https://example.com/sitemap.xml, not: sitemapUrl.

No

No steps

Trim keyFileUrl and save it as keyLocation.Trim keyFileUrl and save it as keyLocation.

If keyLocation is emptyIf keyLocation is empty

Yes

Change keyLocation to https://@siteHost/@submissionKey.txt.Change keyLocation to https://@siteHost/@submissionKey.txt.

No

No steps

Set sitemapText to blank.Set sitemapText to blank.

If a step below fails, save the error as sitemapReadErrorIf a step below fails, save the error as sitemapReadError

Try

Fetch sitemapUrl and save it as sitemapResponse.Fetch sitemapUrl and save it as sitemapResponse.

Trim sitemapResponse.body and save it as sitemapText.Trim sitemapResponse.body and save it as sitemapText.

On error

Stop and report a problem: Could not read the sitemap in the 'sitemapUrl' step. sitemapReadError.message with sitemapUrl replaced by the sitemap address.Stop and report a problem: Could not read the sitemap in the 'sitemapUrl' step. sitemapReadError.message with sitemapUrl replaced by the sitemap address.

If sitemapText contains <sitemapindexIf sitemapText contains <sitemapindex

Yes

Stop and report a problem: That address is a sitemap index - a list of other sitemaps - rather than a sitemap of pages. Point the 'sitemapUrl' step at one of the sitemaps it lists, or add this workflow once per sitemap.Stop and report a problem: That address is a sitemap index - a list of other sitemaps - rather than a sitemap of pages. Point the 'sitemapUrl' step at one of the sitemaps it lists, or add this workflow once per sitemap.

No

No steps

If sitemapText doesn't contain <loc>If sitemapText doesn't contain <loc>

Yes

Stop and report a problem: Nothing at the address in the 'sitemapUrl' step looks like a sitemap: it contains no <loc> entries. Check the address opens a sitemap in a browser.Stop and report a problem: Nothing at the address in the 'sitemapUrl' step looks like a sitemap: it contains no <loc> entries. Check the address opens a sitemap in a browser.

No

No steps

Replace & with & in sitemapText and save it as sitemapText.Replace & with & in sitemapText and save it as sitemapText.

Take sitemapText split by <loc> from item 2 on and save it as sitemapEntries.Take sitemapText split by <loc> from item 2 on and save it as sitemapEntries.

Set tabIsNew to false.Set tabIsNew to false.

If a step below fails, save the error as createTabErrorIf a step below fails, save the error as createTabError

Try

Google SheetsCreate the tab sheetTab in spreadsheetUrl.Create the tab sheetTab in spreadsheetUrl.

Change tabIsNew to true.Change tabIsNew to true.

On error

Show in the output: Recording this run on the existing '@sheetTab' tab.Show in the output: Recording this run on the existing '@sheetTab' tab.

If tabIsNew is yesIf tabIsNew is yes

Yes

Google SheetsAdd a row to sheetTab in spreadsheetUrl.Add a row to sheetTab in spreadsheetUrl.

No

No steps

Start an empty list and save it as alreadySubmitted.Start an empty list and save it as alreadySubmitted.

If a step below fails, save the error as historyReadErrorIf a step below fails, save the error as historyReadError

Try

Google SheetsRead the rows of spreadsheetUrl. Save it as history.Read the rows of spreadsheetUrl. Save it as history.

For each loggedRow in history.rows:For each loggedRow in history.rows:

Set loggedKey to blank. Set loggedKey to blank.

If a step below fails, save the error as loggedRowError If a step below fails, save the error as loggedRowError

Try

If loggedRow["Page URL"] isn't blankIf loggedRow["Page URL"] isn't blank

Yes

Change loggedKey to loggedRow["Page URL"] @ loggedRow["Last modified"].Change loggedKey to loggedRow["Page URL"] @ loggedRow["Last modified"].

No

No steps

On error

Show in the output: Skipping a record row that has no 'Page URL' and 'Last modified' value..Show in the output: Skipping a record row that has no 'Page URL' and 'Last modified' value..

If loggedKey isn't empty If loggedKey isn't empty

Yes

Add loggedKey to the end of alreadySubmitted and save it as alreadySubmitted.Add loggedKey to the end of alreadySubmitted and save it as alreadySubmitted.

No

No steps

On error

Show in the output: Could not read what has been submitted before, so every page in the sitemap is being treated as new. historyReadError.message.Show in the output: Could not read what has been submitted before, so every page in the sitemap is being treated as new. historyReadError.message.

Start an empty list and save it as pendingPages.Start an empty list and save it as pendingPages.

Start an empty list and save it as pendingKeys.Start an empty list and save it as pendingKeys.

Set unchangedCount to 0.Set unchangedCount to 0.

For each sitemapEntry in sitemapEntries:For each sitemapEntry in sitemapEntries:

If the length of pendingPages is greater than or equal to pageLimitIf the length of pendingPages is greater than or equal to pageLimit

Yes

Stop repeating.Stop repeating.

No

No steps

If sitemapEntry doesn't contain </loc>If sitemapEntry doesn't contain </loc>

Yes

Skip to the next item.Skip to the next item.

No

No steps

Extract the first web address from the first item of sitemapEntry split by </loc> and save it as pageUrl.Extract the first web address from the first item of sitemapEntry split by </loc> and save it as pageUrl.

If pageUrl is emptyIf pageUrl is empty

Yes

Skip to the next item.Skip to the next item.

No

No steps

Set lastModified to blank.Set lastModified to blank.

If sitemapEntry contains <lastmod>If sitemapEntry contains <lastmod>

Yes

Trim the first item of the last item of sitemapEntry split by <lastmod> split by </lastmod> and save it as lastModified.Trim the first item of the last item of sitemapEntry split by <lastmod> split by </lastmod> and save it as lastModified.

No

No steps

Set pageKey to pageUrl @ lastModified.Set pageKey to pageUrl @ lastModified.

If alreadySubmitted contains pageKeyIf alreadySubmitted contains pageKey

Yes

Add 1 to unchangedCount.Add 1 to unchangedCount.

Skip to the next item.Skip to the next item.

No

No steps

If pendingKeys contains pageKeyIf pendingKeys contains pageKey

Yes

Skip to the next item.Skip to the next item.

No

No steps

Add a record of url and lastModified to the end of pendingPages and save it as pendingPages.Add a record of url and lastModified to the end of pendingPages and save it as pendingPages.

Add pageKey to the end of pendingKeys and save it as pendingKeys.Add pageKey to the end of pendingKeys and save it as pendingKeys.

If pendingPages is emptyIf pendingPages is empty

Yes

Finish the run: Nothing to submit: all unchangedCount page(s) in the sitemap have already been submitted at their current last-modified date.Finish the run: Nothing to submit: all unchangedCount page(s) in the sitemap have already been submitted at their current last-modified date.

No

No steps

Start an empty list and save it as submissionUrls.Start an empty list and save it as submissionUrls.

For each pendingPage in pendingPages:For each pendingPage in pendingPages:

Add pendingPage.url to the end of submissionUrls and save it as submissionUrls.Add pendingPage.url to the end of submissionUrls and save it as submissionUrls.

Set submissionStatus to 0.Set submissionStatus to 0.

Set submissionAccepted to false.Set submissionAccepted to false.

If a step below fails, save the error as submissionErrorIf a step below fails, save the error as submissionError

Try

Send data to indexNowEndpoint and save it as submission.Send data to indexNowEndpoint and save it as submission.

Change submissionStatus to submission.status.Change submissionStatus to submission.status.

Change submissionAccepted to submission.ok.Change submissionAccepted to submission.ok.

On error

Stop and report a problem: Could not reach the search engine indexing endpoint. submissionError.message with indexNowEndpoint replaced by the indexing endpoint Nothing was recorded, so the next run will offer these pages again.Stop and report a problem: Could not reach the search engine indexing endpoint. submissionError.message with indexNowEndpoint replaced by the indexing endpoint Nothing was recorded, so the next run will offer these pages again.

If submissionAccepted is noIf submissionAccepted is no

Yes

Stop and report a problem: The indexing endpoint refused the submission (HTTP submissionStatus). Check that keyLocation opens in a browser and contains exactly the key from the 'indexNowKey' step. Nothing was recorded, so the next run will offer these pages again.Stop and report a problem: The indexing endpoint refused the submission (HTTP submissionStatus). Check that keyLocation opens in a browser and contains exactly the key from the 'indexNowKey' step. Nothing was recorded, so the next run will offer these pages again.

No

No steps

Get today's date and save it as submittedOn.Get today's date and save it as submittedOn.

Set submissionResult to Accepted (HTTP submissionStatus).Set submissionResult to Accepted (HTTP submissionStatus).

For each submittedPage in pendingPages:For each submittedPage in pendingPages:

If a step below fails, save the error as recordErrorIf a step below fails, save the error as recordError

Try

Google SheetsAdd a row to sheetTab in spreadsheetUrl.Add a row to sheetTab in spreadsheetUrl.

On error

Show in the output: Submitted but could not record submittedPage.url, so the next run will offer it again. recordError.message.Show in the output: Submitted but could not record submittedPage.url, so the next run will offer it again. recordError.message.

Show in the output: Submitted the length of pendingPages new or changed page(s) for indexing and left unchangedCount unchanged page(s) alone.Show in the output: Submitted the length of pendingPages new or changed page(s) for indexing and left unchangedCount unchanged page(s) alone.