Package net.vidageek.crawler.component

Examples of net.vidageek.crawler.component.PageCrawlerExecutor


    final ExecutorCounter counter = new ExecutorCounter();

    try {
      executor
          .execute(new PageCrawlerExecutor(new Url(config.beginUrl(), 0), executor, counter, config
              .downloader(), config.normalizer(), new DoesNotFollowVisitedUrlVisitor(config.beginUrl(),
              visitor)));

      while (counter.value() != 0) {
        log.debug("executors that finished: " + executor.getCompletedTaskCount());
View Full Code Here

TOP

Related Classes of net.vidageek.crawler.component.PageCrawlerExecutor

Copyright © 2018 www.massapicom. All rights reserved.
All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.