Web Scraper: Web Browser
Web Scraper Windows
|
Web Browser |
(1) Web Address/File Location Bar - Use this field to enter a web address and navigate to any web page. Additionally, enter the location of files on a local machine. The dropdown is already populated with a handful of power data web pages. Once selected, the web site or file will display in the window below.
(2) Download Buttons - Use the buttons to download the data.
-
Current Page will scrape the entire page.
-
Next Click will scrape data from a file accessed by the next mouse click. For example, in the screenshot below, Current Page will scrape exactly what you see on that page and not the data tables stored in various file types; however, Next Click will pull the data from the file selected by the next mouse click. In this case the user may select either .csv, .html, or .pdf file types by clicking on the icon links in the lower right-hand corner.
-
Custom Link allows a more customized approach for users comfortable with html language. It will launch a new window that is described below.
Next, go to the Web Tables window to see the downloaded data.
Custom Link
The Web Scraper is designed with some incredible flexibility. With the complexity of today's web site designs, the user may need to go beyond the options on the main form to consistently access the latest data. The Custom Link form allows even more control over how the scraper gets the data.
For instance, a site may post files daily but they include an hour time stamp in the file name. The hour changes everyday so the link address will never be know. However, if the most current file is always the first one on the page, use the Always grab the first link option shown below.
If the link moves around the page, i.e. further down in an ever changing list, then the next option to Find the first link containing the following text will search the page for the specific text entered in the box.
The advanced option to Enter a regular expression pattern is for those comfortable with html code and know exactly what expression they want to find.
Click the Convert link to Web Table to continue the process as described on the Web Tables window.
Web Browser
