Let's say we just wanted to extract all the HTML content in the, we could re-write the CSS selector (see section above) simply to 'head' and use the Inner HTML option to extract the HTML content: This option extracts the HTML content within the selector, including any other HTML elements. Typically, this would fall more under 'developer work' than 'SEO work', and you'd expect to do some post-processing of the data following the extraction. In this section we will examine the options for 'Inner HTML' and 'Outer HTML'. These options allow you to grab all the HTML content within the selector, which you might want to do if you were trying to grab code snippets (e.g. Selecting from the dropdown 'Extract Data Using' gives a variety of different options for what you can extract. When setting up your rules, the default 'Text' option is great if you want to. tracking codes or meta data).ĭeveloping a better understanding of CSS selectors will help you figure out the correct selector to use, Mozilla have a decent primer as a starting point. If you are familiar with content extraction on other tools, you are probably also familiar with using Chrome DevTools (or Firefox) and the 'Inspect Element' feature in order to find CSS selectors:įor visual elements, this method is often not better than Sitebulb's point-and-click method, however it does enable you to identify selectors that represent elements which are not visible on the page (e.g. You can test the selector by trying different example URLs and checking if the selector and/or the test results change. So you may need to adjust the selector to make it more robust. This means that it may extract the data on the example URL you test it with, but not on any of the other URLs you want to scrape. You may find that if the selector is too specific, it will change from page to page. However, you can extract the same datapoint with a much simpler selector: As an example, the following selector can be used to grab the phone number off this directory page:ĭiv.row-spaced:nth-child(1) > div:nth-child(2) > div:nth-child(1) > ul:nth-child(2) > li:nth-child(2) > span:nth-child(2) You may also find that the point-and-click method ends up choosing a selector that is too specific. We'll use this same technique in some of the examples below. You can see how the visual selector and 'test' results change as you re-write the selector: All I have done is switch from grabbing a single element within a content block, to grabbing the entire block. Here is a quick example of rewriting a selector from our homepage. You could re-write or adjust the selector yourself, or simply write a completely new one. However, you don't need to use this method. You can jump to a specific area of the guide using the jumplinks below:Īs you should already be aware, Sitebulb offers a point-and-click method for selecting a CSS selector: This guide covers the more advanced settings options for content extraction, including several examples. This guide assumes you know the basics about setting up content extraction in Sitebulb, and covers some of the more advanced use cases. We have a separate guide in the documentation which covers the basic process for adding content extractors - so if this is new to you, please head there first. If you have any questions or issues, we are always there to answer.Sitebulb allows you to configure the crawler to collect additional, custom data points as it crawls (in addition to all the 'normal' data like h1, title tag, meta description etc.). We also have daily Office Hours where we screen share with customers and answer questions. You also get access to our outstanding support. No other tool can match this functionality. We have over 1 million of them for many websites in the world. The recipes are user generated and shared for others to use. Using one of the existing extraction recipes, you can convert most of the popular websites to csv with just one click. Beyond our free plan, we have paid plans with more features. This way you can see how it works and what you can export with no risk. You will get 500 free page credits per month. You can use DataMiner for FREE in our starter subscription plan. With this tool you can export web pages into XLS, CSV, XLSX or TSV files (.xls. You can extract tables and lists from any page and upload them to Google Sheets or Microsoft Excel. Data Scraper extracts data out of HTML web pages and imports it into Microsoft Excel spreadsheets DataMiner is a data extraction tool that lets you scrape any HTML web page.
0 Comments
Leave a Reply. |