Package net.vidageek.crawler.visitor

Examples of net.vidageek.crawler.visitor.DoesNotFollowVisitedUrlVisitor

52
53
54
55
56
57
58
59
60
61
62
    final ExecutorCounter counter = new ExecutorCounter();

    try {
      executor
          .execute(new PageCrawlerExecutor(new Url(config.beginUrl(), 0), executor, counter, config
              .downloader(), config.normalizer(), new DoesNotFollowVisitedUrlVisitor(config.beginUrl(),
              visitor)));

      while (counter.value() != 0) {
        log.debug("executors that finished: " + executor.getCompletedTaskCount());
        log.debug("Number of Executors alive: " + counter.value());
View Full Code Here
TOP

Related Classes of net.vidageek.crawler.visitor.DoesNotFollowVisitedUrlVisitor

  • net.vidageek.crawler.PageCrawler
  • net.vidageek.crawler.Url
    • Copyright © 2018 www.massapicom. All rights reserved.
    All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.