You can Use the Archive to get access to data older than 30 days.
Simply use the 'language:' query filter.
For example: (language:german OR language:Chinese) Will limit the search for posts in both German & Chinese.
By default (when the sort parameter isn't specified) the results are sorted by the recommended order of crawl date. You can however change the sort order by using the following values:
- relevancy
- pubilshed
- social.facebook.likes
- social.facebook.shares
- social.facebook.comments
- social.vk.shares
- replies_count
- participants_count
- performance_score
- domain_rank
- ord_in_thread
- rating
For example, the following call, will return posts ordered by the number of likes:
https://webz.io/filterWebContent?token=XXXX-XXXX-XXXX&format=json&q=*&sort=social.facebook.likesWe do filter out duplicates. You may get the same article link multiple times, if your query matches multiple comments for the same article. webz.io searches at the post level, so results include each post that matched your query. Each post also contains information about its containing thread. One of the properties of the thread is the article link. For that reason you might see the same link multiple times. If you want to search only for the first post (i.e only the article and no comments) add is_first:true to your query. For example:
opera is_first:true
Will return only articles (i.e no comments) containing the word "opera".
Yes. You can search by person, location or organization on news or blog posts in English. For example, organization:apple will return news or blog posts mentioning Apple the company and not the fruit.
You should page through results by calling the URL on the "next" field, to retrieve the rest of the data matching your search query.
To extract an entire thread, use the "thread.url" filter. This will return all the posts belonging to the thread URL provided. Example:
thread.url:http://domain.com/param=val
(note that you must escape the http:// part of the URL like so: http\:\/\/).
Yes. Just add highlight=true as a parameter to your call.
Boolean expressions can be nested in as many levels as you want.
For example: (exp1 AND exp2 AND exp3) OR (exp4 AND (exp5 OR exp6)) -(exp7 AND (exp8 OR exp9))
The maximum length of a query is 4,000 characters.
Yes. Just append the dollar sign ($) to the end of the keyword. For example, searching for the keyword "simplicity" will also return hits for the word "simple" since we index the stemmed version of the word, but if you want to find documents that contain "simplicity" and nothing else, search for "simplivity$". Stemmed searches are currently supported for English, Spanish, Arabic and Russian.
Yes. There are actually multiple ways to get better quality posts either from popular websites, or even popular posts. The first way would be to use the domain_rank filter. The domain rank filter specifies how popular a domain is (by monthly traffic), so if you want to search for posts from the top 1,000 sites, use the following:
domain_rank:<1000
The second option would be to either use the performance_score, or social signals to filter for posts that were either viral, or were shared/liked many times on social networks.