How to Loop Through HTML Pages

A robot often needs to loop through pages. For example, many web sites that present the results of a search request will do so over several pages, each containing e.g. 20 results from the search. To get the search results, you need to loop through the pages and process one page at a time. This section explains how to do this.