80legs, a Customizable Web Scraping Service
80legs is a web scraping service with customizable features that allows users to create and run web crawls through its SaaS platform. 80legs employs technology built on top of a distributed grid computing network which consists of about 50,000 individual computers located worldwide. These computers can crawl up to 2 billion different web pages daily.
When using 80legs, users can:
- choose websites they want to crawl
- specify what kind of data to extract
- choose between a general and directed web crawlers
- choose how many pages they want to crawl
- specify certain files to be analyzed
80legs offers such products as Custom Web Crawling, Datafiniti, and Giant Web Crawl.
With Custom Web Crawling, users can select one of the default 80legs apps and customize them to scrape and crawl public web pages, specifying what type of data to scrape and which links to follow from each URL. Users can also build their own crawling apps using 80legs app templates and run them on the cloud platform.
With Giant Web Crawl option, users can determine specifications to track web content relevant to their needs. The company’s customer solutions team developed customized scraping logic to extract data for each client when scraping the entire web. Giant Web Crawl applies this customized scraping logic to the HTML content when crawling about 1 million URLs every day and sends relevant results to the customers’ 80legs accounts. This option can be used to search for the specific keywords, to get contact information about target companies, and to identify websites with different HTML codes.
Datafiniti database constantly crawls the web and makes comprehensive data verticals for the specific types of data that include businesses, products, and properties. Each record can include dozens of various attributes. Users can integrate with Datafiniti database through RESTful API and get instant access to web data.