Each week, records the title, summary and search terms of each page you list in Airtable, using only words found on the page.
Uses
Airtable
OpenAI
See which search terms each page of your site targets
Every Monday at 07:00, each web page on your list is fetched and read, and one row per page is added to your Airtable table holding the page's title, a one-sentence description and the search terms it targets. Over a few weeks that becomes a keyword table for your whole site.
Every term recorded has to appear on the page itself: each suggestion the AI makes is checked against the page's own wording and dropped if it cannot be found, and a page where nothing survives that check gets no row at all. No search volume, difficulty or ranking is recorded, because none of that is measured here. Airtable is asked first whether each address is already in the table, so no page is recorded twice; if that check fails the page waits for the next run rather than being written blind. A page that cannot be fetched is reported and skipped while the rest still run.
You need an Airtable table with four text fields named 'Page URL', 'Page Title', 'Summary' and 'Keywords'.
Run every Monday at 07:00.Run every Monday at 07:00.
Set pageUrls to blank.Set pageUrls to blank.Set airtableBaseId to blank.Set airtableBaseId to blank.Set airtableTableName to blank.Set airtableTableName to blank.Set maxPages to 8.Set maxPages to 8.
Trim airtableBaseId and save it as baseId.Trim airtableBaseId and save it as baseId.Trim airtableTableName and save it as tableName.Trim airtableTableName and save it as tableName.
Set pageLimit to maxPages multiplied by 1.Set pageLimit to maxPages multiplied by 1.
If pageLimit is greater than 25If pageLimit is greater than 25Yes
Change pageLimit to 25.Change pageLimit to 25.No
No steps
If pageLimit is less than 1If pageLimit is less than 1Yes
Change pageLimit to 1.Change pageLimit to 1.No
No steps
Set knownUrls to an empty list.Set knownUrls to an empty list.Set added to 0.Set added to 0.Set skipped to 0.Set skipped to 0.
For each rawUrl in pageUrls split by a new line:For each rawUrl in pageUrls split by a new line:If added is greater than or equal to pageLimitIf added is greater than or equal to pageLimitYes
Stop repeating.Stop repeating.No
No steps
Trim rawUrl and save it as pageUrl.Trim rawUrl and save it as pageUrl.If pageUrl is emptyIf pageUrl is emptyYes
Skip to the next item.Skip to the next item.No
No steps
If not pageUrl starts with https:// (ignoring case) or pageUrl starts with…If not pageUrl starts with https:// (ignoring case) or pageUrl starts with…Yes
Add 1 to skipped.Add 1 to skipped.
Show in the output: Skipped a line that is not a full web address: pageUrl.Show in the output: Skipped a line that is not a full web address: pageUrl.Skip to the next item.Skip to the next item.No
No steps
If pageUrl contains pageUrl contains Yes
Add 1 to skipped.Add 1 to skipped.Show in the output: Skipped an address carrying a quote mark: pageUrl.Show in the output: Skipped an address carrying a quote mark: pageUrl.Skip to the next item.Skip to the next item.No
No steps
If knownUrls contains pageUrlIf knownUrls contains pageUrlYes
Skip to the next item.Skip to the next item.No
No steps
Set wasChecked to false.Set wasChecked to false.Set alreadyRecorded to false.Set alreadyRecorded to false.
If a step below fails, save the error as readErrorIf a step below fails, save the error as readErrorTry
AirtableList records in tableName. Save it as found.List records in tableName. Save it as found.If found.records isn't emptyIf found.records isn't emptyYes
Change alreadyRecorded to true.Change alreadyRecorded to true.No
No steps
Change wasChecked to true.Change wasChecked to true.On error
Show in the output: Could not check whether pageUrl is already recorded, so it was left for the next run.Show in the output: Could not check whether pageUrl is already recorded, so it was left for the next run.If wasChecked is noIf wasChecked is noYes
Add 1 to skipped.Add 1 to skipped.Skip to the next item.Skip to the next item.No
No steps
If alreadyRecorded is yesIf alreadyRecorded is yesYes
Add pageUrl to the end of knownUrls and save it as knownUrls.Add pageUrl to the end of knownUrls and save it as knownUrls.Skip to the next item.Skip to the next item.No
No steps
Fetch pageUrl and save it as fetch.Fetch pageUrl and save it as fetch.Set pageSource to fetch.body.Set pageSource to fetch.body.If the length of pageSource is less than 400If the length of pageSource is less than 400Yes
Add 1 to skipped.Add 1 to skipped.Show in the output: Could not read enough of pageUrl to describe it, so no row was added for it.Show in the output: Could not read enough of pageUrl to describe it, so no row was added for it.Skip to the next item.Skip to the next item.No
No steps
Shorten pageSource to 14000 characters and save it as pageText.Shorten pageSource to 14000 characters and save it as pageText.Set pagePrompt to You are reading the raw source of one web page and recording what it is about, for a keyword table.Set pagePrompt to You are reading the raw source of one web page and recording what it is about, for a keyword table.
Answer with exactly three parts separated by ||| and nothing else, in this order:
the page's own title ||| one sentence saying what the page is about ||| a comma-separated list of up to ten search terms the page targets
Rules you must follow:
- Every search term must be copied word for word from the visible wording of the page below. Do not invent terms, and do not add terms you think the page ought to target.
- Ignore HTML tags, scripts and styles. Read only the wording a visitor would see.
- Do not mention search volume, difficulty, rankings or competitors - none of that was measured here.
- Write plain text. No markdown, no numbering, no extra commentary.
Page address: pageUrl.Change pagePrompt to pagePromptChange pagePrompt to pagePrompt
Page source:
pageText.Set answer to blank.Set answer to blank.If a step below fails, save the error as modelErrorIf a step below fails, save the error as modelErrorTry
Trim the response text from the OpenAI “Ask OpenAI” step and save it as answer.Trim the response text from the OpenAI “Ask OpenAI” step and save it as answer.On error
Show in the output: Could not read pageUrl with the model.Show in the output: Could not read pageUrl with the model.If answer is emptyIf answer is emptyYes
Add 1 to skipped.Add 1 to skipped.Show in the output: No description came back for pageUrl.Show in the output: No description came back for pageUrl.Skip to the next item.Skip to the next item.No
No steps
Split answer by ||| and save it as parts.Split answer by ||| and save it as parts.If the length of parts is less than 3If the length of parts is less than 3Yes
Add 1 to skipped.Add 1 to skipped.Show in the output: The answer for pageUrl did not carry a title, a summary and keywords, so no row was added for it.Show in the output: The answer for pageUrl did not carry a title, a summary and keywords, so no row was added for it.Skip to the next item.Skip to the next item.No
No steps
Trim parts[0] and save it as pageTitle.Trim parts[0] and save it as pageTitle.Trim parts[1] and save it as pageSummary.Trim parts[1] and save it as pageSummary.Trim parts[2] and save it as rawKeywordList.Trim parts[2] and save it as rawKeywordList.If pageTitle is empty or pageSummary is emptyIf pageTitle is empty or pageSummary is emptyYes
Add 1 to skipped.Add 1 to skipped.Show in the output: No title or summary came back for pageUrl.Show in the output: No title or summary came back for pageUrl.Skip to the next item.Skip to the next item.No
No steps
Set keptKeywords to an empty list.Set keptKeywords to an empty list.Set keywordCount to 0.Set keywordCount to 0.Set droppedKeywords to 0.Set droppedKeywords to 0.For each rawKeyword in rawKeywordList split by ,:For each rawKeyword in rawKeywordList split by ,:If keywordCount is greater than or equal to 12If keywordCount is greater than or equal to 12Yes
Stop repeating.Stop repeating.No
No steps
Trim rawKeyword and save it as keyword.Trim rawKeyword and save it as keyword.If the length of keyword is less than 3If the length of keyword is less than 3Yes
Skip to the next item.Skip to the next item.No
No steps
If pageText doesn't contain keyword (ignoring case)If pageText doesn't contain keyword (ignoring case)Yes
Add 1 to droppedKeywords.Add 1 to droppedKeywords.Skip to the next item.Skip to the next item.No
No steps
If keptKeywords contains keywordIf keptKeywords contains keywordYes
Skip to the next item.Skip to the next item.No
No steps
Add keyword to the end of keptKeywords and save it as keptKeywords.Add keyword to the end of keptKeywords and save it as keptKeywords.Add 1 to keywordCount.Add 1 to keywordCount.If keywordCount is 0If keywordCount is 0Yes
Add 1 to skipped.Add 1 to skipped.Show in the output: None of the search terms suggested for pageUrl appear on the page itself, so no row was added for it.Show in the output: None of the search terms suggested for pageUrl appear on the page itself, so no row was added for it.Skip to the next item.Skip to the next item.No
No steps
If droppedKeywords is greater than 0If droppedKeywords is greater than 0Yes
Show in the output: Dropped droppedKeywords suggested term(s) for pageUrl because they do not appear on the page.Show in the output: Dropped droppedKeywords suggested term(s) for pageUrl because they do not appear on the page.No
No steps
If a step below fails, save the error as writeErrorIf a step below fails, save the error as writeErrorTry
AirtableAdd a record to tableName.Add a record to tableName.Add pageUrl to the end of knownUrls and save it as knownUrls.Add pageUrl to the end of knownUrls and save it as knownUrls.Add 1 to added.Add 1 to added.On error
Add 1 to skipped.Add 1 to skipped.Show in the output: Airtable would not accept the row for pageUrl, so it was left for the next run.Show in the output: Airtable would not accept the row for pageUrl, so it was left for the next run.Show in the output: Added added page(s) to your keyword table. Skipped: skipped.Show in the output: Added added page(s) to your keyword table. Skipped: skipped.