Examples of com.ontometrics.scraper.extraction.HtmlExtractor

com.ontometrics.scraper.extraction.HtmlExtractor

Provides a means of collecting {@link Manipulator}s and performing progressive harvesting of html from an original source. This is done through an implementation of the Chain of Responsibility Pattern: the manipulators are held in a LinkedList and when new ones are added, they are bolted on to the end, then, when the source is requested, the first {@link Manipulator} isinvoked, setting off the chain of operations. Then the resulting source is extracted. Extractor is fetching content from {@link URL} using {@link UrlContentProvider}. Default implementation is {@link UrlConnectionContentProvider}, but it can be changed, see {@link #urlContentProvider(UrlContentProvider)} @author Rob

      public URL getBaseUrl() {
        // TODO Auto-generated method stub
        return PagedListingFolder.getUrl();
      }
    };
    HtmlExtractor htmlExtractor = html().url(PagedListingTable.getUrl());


    List<Record> records = new ListingDetailScraper().setConvertURLs(true)
        .iterator(pageIterator)
        .listing(new LinkExtractor().source(htmlExtractor))
        .details(new DefaultFieldExtractor().source(htmlExtractor))

Related Classes of com.ontometrics.scraper.extraction.HtmlExtractor

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.