Read details off web pages that won't share their text

Every Monday at 07:00, a full-page picture is taken of each web address you list, with adverts and cookie banners hidden, the details you asked for are read off the picture itself, and one row per page is added to your Google Sheet. It suits pages where a plain fetch gets nothing useful.

Each row carries the date, the page address, one cell per detail you asked for and a note. Every page you list gets exactly one row, including the ones that go wrong: a bad address, a page that could not be photographed or a picture with none of your details each produce a row with the detail cells empty and the reason in the note, so an empty cell never looks like a successful read. Nothing is guessed: only what is legible in the picture is written. Rows are only ever appended, so nothing already on the tab is edited or removed.

Before the first run your tab needs a header row of 'Captured on', 'Page', your own field names in the order you list them, then 'Note'.

Run every Monday at 07:00.Run every Monday at 07:00.

Set pageUrls to blank.Set pageUrls to blank.

Set fieldsToRead to blank.Set fieldsToRead to blank.

Set spreadsheetUrl to blank.Set spreadsheetUrl to blank.

Set sheetName to blank.Set sheetName to blank.

Set maxPages to 10.Set maxPages to 10.

Set fieldNames to an empty list.Set fieldNames to an empty list.

For each rawFieldName in fieldsToRead split by ,:For each rawFieldName in fieldsToRead split by ,:

Trim rawFieldName and save it as fieldName.Trim rawFieldName and save it as fieldName.

If fieldName isn't emptyIf fieldName isn't empty

Yes

Add fieldName to the end of fieldNames and save it as fieldNames.Add fieldName to the end of fieldNames and save it as fieldNames.

No

No steps

If fieldNames is emptyIf fieldNames is empty

Yes

Stop and report a problem: The 'fieldsToRead' step has no field names in it. Write the details you want off each page separated by commas - for example 'Price, Availability, Delivery time' - then run it again.Stop and report a problem: The 'fieldsToRead' step has no field names in it. Write the details you want off each page separated by commas - for example 'Price, Availability, Delivery time' - then run it again.

No

No steps

Set blankCells to an empty list.Set blankCells to an empty list.

For each unreadFieldName in fieldNames:For each unreadFieldName in fieldNames:

Add to the end of blankCells and save it as blankCells.Add to the end of blankCells and save it as blankCells.

Get today's date and save it as capturedOn.Get today's date and save it as capturedOn.

Set pagesCaptured to 0.Set pagesCaptured to 0.

Set rowsWritten to 0.Set rowsWritten to 0.

For each rawPageUrl in pageUrls with "For each rawPageUrl in pageUrls with replaced by , split by ,:

Trim rawPageUrl and save it as pageUrl.Trim rawPageUrl and save it as pageUrl.

If pageUrl is emptyIf pageUrl is empty

Yes

Skip to the next item.Skip to the next item.

No

No steps

If pagesCaptured is greater than or equal to maxPages multiplied by 1If pagesCaptured is greater than or equal to maxPages multiplied by 1

Yes

Stop repeating.Stop repeating.

No

No steps

Add 1 to pagesCaptured.Add 1 to pagesCaptured.

Set problemNote to blank.Set problemNote to blank.

Set pictureRef to blank.Set pictureRef to blank.

If pageUrl doesn't start with https:// and pageUrl doesn't start with http://If pageUrl doesn't start with https:// and pageUrl doesn't start with http://

Yes

Change problemNote to This is not a web address. Give the page as a full address starting with https://..Change problemNote to This is not a web address. Give the page as a full address starting with https://..

No

If a step below fails, save the error as captureErrorIf a step below fails, save the error as captureError

Try

Trim the document ref from the ScreenshotOne “Take a screenshot” step and save it as pictureRef.Trim the document ref from the ScreenshotOne “Take a screenshot” step and save it as pictureRef.

On error

Show in the output: Could not photograph pageUrl: captureError as text.Show in the output: Could not photograph pageUrl: captureError as text.

Change pictureRef to blank.Change pictureRef to blank.

If pictureRef doesn't start with doc_If pictureRef doesn't start with doc_

Yes

Change problemNote to No picture of this page could be taken. Check the address opens in a browser without signing in..Change problemNote to No picture of this page could be taken. Check the address opens in a browser without signing in..

No

No steps

Set readFields to an empty list.Set readFields to an empty list.

If problemNote is emptyIf problemNote is empty

Yes

If a step below fails, save the error as readErrorIf a step below fails, save the error as readError

Try

Run an AI agent: Read the attached picture of a web page and pull out each of these fields, in this order:… and save it as pictureReading.Run an AI agent: Read the attached picture of a web page and pull out each of these fields, in this order:… and save it as pictureReading.

Change readFields to pictureReading.output.fields.Change readFields to pictureReading.output.fields.

On error

Show in the output: Could not read the picture of pageUrl: readError as text.Show in the output: Could not read the picture of pageUrl: readError as text.

Change readFields to an empty list.Change readFields to an empty list.

Change problemNote to The picture was taken but nothing could be read off it..Change problemNote to The picture was taken but nothing could be read off it..

No

No steps

Set rowValues to [capturedOn, pageUrl].Set rowValues to [capturedOn, pageUrl].

Set filledCells to 0.Set filledCells to 0.

For each wantedField in fieldNames:For each wantedField in fieldNames:

Set cellValue to blank.Set cellValue to blank.

For each readField in readFields:For each readField in readFields:

Set readFieldName to blank.Set readFieldName to blank.

If a step below fails, save the error as unnamedFieldIf a step below fails, save the error as unnamedField

Try

Trim readField.name and save it as readFieldName.Trim readField.name and save it as readFieldName.

On error

Change readFieldName to blank.Change readFieldName to blank.

If readFieldName is wantedFieldIf readFieldName is wantedField

Yes

If a step below fails, save the error as valueIsAListIf a step below fails, save the error as valueIsAList

Try

Trim readField.value and save it as cellValue.Trim readField.value and save it as cellValue.

On error

Join readField.value with , and save it as cellValue.Join readField.value with , and save it as cellValue.

No

No steps

If cellValue isn't emptyIf cellValue isn't empty

Yes

Add 1 to filledCells.Add 1 to filledCells.

No

No steps

Add cellValue to the end of rowValues and save it as rowValues.Add cellValue to the end of rowValues and save it as rowValues.

If problemNote is empty and filledCells is 0If problemNote is empty and filledCells is 0

Yes

Change problemNote to The picture was read but none of the details you asked for are on this page..Change problemNote to The picture was read but none of the details you asked for are on this page..

No

No steps

Add problemNote to the end of rowValues and save it as rowValues.Add problemNote to the end of rowValues and save it as rowValues.

Google SheetsAdd a row to sheetName in spreadsheetUrl.Add a row to sheetName in spreadsheetUrl.

Add 1 to rowsWritten.Add 1 to rowsWritten.

Show in the output: Rows written: rowsWritten as text of pagesCaptured as text pages looked at.Show in the output: Rows written: rowsWritten as text of pagesCaptured as text pages looked at.