How to page through results from the Webz.io API
When you call the API, Webz.io returns a list of posts. The list is a subset of the total number of posts that are relevant to the query. To get the total number of available results, access the 'totalResults' field.
The following example shows the 'totalResults' field that a query returned.
To page through the available posts, call the value in the 'next' field.
Each request may return up to 100 posts matching your query. To consume all the data, make sure you keep on calling the URL string value that appears in the "next" field found in the output of each request.
This pagination method derives from the statelessness of HTTP GET requests.
So for example if the total number of results is 1,000, you will need to call the output value of the "next" field 10 times. Make sure you call the new value of the output of each request.
Note the "ts" (timestamp) parameter in the "next" URL. The &ts parameter is telling the system to return results that were crawled after this timestamp (Unix Timestamp in milliseconds). Every time you call the "next" URL, the &ts parameter will contain a time stamp closer to the present.