Screen scraping or web data extraction is basically a computer software method used for years in extracting information from various websites. Sometimes, a screen scraper application is used to simulate the discovery of information on the Web by humans, either through implementing low level HTTP or Hypertext Transfer Protocol or by embedding well established web browsers like Mozilla or Internet Explorer for Windows. The process for screen scraping is almost similar to the ones for web indexing which basically indexes all web content with the use of a bot and is a universal method which has been adopted and used by most search engines today. But as opposed to the typical web indexing technique, screen scraper software directs most of its focus on the transformation of web content that is unstructured yet and most of them are usually in HTML format.
This content is then transformed into a more structured format which can be analyzed and stored either in a spreadsheet or a central local databank. Screen scraping can also be associated with web automation wherein human browsing is simulated by utilizing specifically designed computer software. This may include, weather data monitoring, web research, change detection among websites, price comparison, integration of data from the web, or for web content mash up. When you look at screen scraper applications at a very simplified level, there are usually two basic stages that are involved and these are data extraction and data discovery. The data extraction basically deals with the process of actually pulling off data from web pages while data discovery usually deals with navigating a particular web site to locate and get into the page the researcher wants.
Usually, when people heard about screen scraping, they tend to shift their focus only on the process of data extraction although one of the most crucial stages is the data discovery itself. Using screen scraper applications, the stage for data discovery may be as simple as asking for a URL. For instance, it may be as simple as visiting a particular website and extracting out the relevant information you need. However, the process for data discovery may involve several complex tasks such as logging into a secured site, getting through a maze of supplemental pages just so you can obtain the required cookies, and getting though thousands of search results before finally tracking all the details so you can technically obtain the relevant data you are after.
The process itself seems so daunting and without a good screen scraping application, you have a very slim chance of getting what you really want. But with data extraction, this phase means you already got yourself into the page where the information you are after is located and all you need to do now is to extract the HTML. This process can be further simplified by a well trusted screen scraper programmed to go after the essential data that will be relevant to your search. Although this may entail complex programming to get finely tuned results, still there are web applications that are guaranteed to work just fine. With the help of this type of service, you will be able to simplify your hunt for information and make everything a whole lot easier on your part.
Source: http://www.fetch.com/screen-scraper-articles/
This content is then transformed into a more structured format which can be analyzed and stored either in a spreadsheet or a central local databank. Screen scraping can also be associated with web automation wherein human browsing is simulated by utilizing specifically designed computer software. This may include, weather data monitoring, web research, change detection among websites, price comparison, integration of data from the web, or for web content mash up. When you look at screen scraper applications at a very simplified level, there are usually two basic stages that are involved and these are data extraction and data discovery. The data extraction basically deals with the process of actually pulling off data from web pages while data discovery usually deals with navigating a particular web site to locate and get into the page the researcher wants.
Usually, when people heard about screen scraping, they tend to shift their focus only on the process of data extraction although one of the most crucial stages is the data discovery itself. Using screen scraper applications, the stage for data discovery may be as simple as asking for a URL. For instance, it may be as simple as visiting a particular website and extracting out the relevant information you need. However, the process for data discovery may involve several complex tasks such as logging into a secured site, getting through a maze of supplemental pages just so you can obtain the required cookies, and getting though thousands of search results before finally tracking all the details so you can technically obtain the relevant data you are after.
The process itself seems so daunting and without a good screen scraping application, you have a very slim chance of getting what you really want. But with data extraction, this phase means you already got yourself into the page where the information you are after is located and all you need to do now is to extract the HTML. This process can be further simplified by a well trusted screen scraper programmed to go after the essential data that will be relevant to your search. Although this may entail complex programming to get finely tuned results, still there are web applications that are guaranteed to work just fine. With the help of this type of service, you will be able to simplify your hunt for information and make everything a whole lot easier on your part.
Source: http://www.fetch.com/screen-scraper-articles/
No comments:
Post a Comment