Examples of ClueWarcDocnoMapping

edu.umd.cloud9.collection.clue.ClueWarcDocnoMapping

Object that maps between WARC-TREC-IDs (String identifiers) to docnos (sequentially-numbered ints). This object provides mappings for the Clue Web English collection; the docnos are numbered from part 1 all the way through part 10.

Note that this class needs the data file docno.mapping, loaded via the {@link #loadMapping(Path,FileSystem)} method.
@author Jimmy Lin

Examples of edu.umd.cloud9.collection.clue.ClueWarcDocnoMapping

        "-" + DocnoMapping.BuilderUtils.COLLECTION_OPTION + "=" + collectionPathRepacked,
        "-" + DocnoMapping.BuilderUtils.MAPPING_OPTION + "=" + mappingFile };


    IntegrationUtils.exec(Joiner.on(" ").join(args));


    ClueWarcDocnoMapping mapping = new ClueWarcDocnoMapping();
    mapping.loadMapping(new Path(mappingFile), fs);


    assertEquals("clueweb09-en0000-00-00000", mapping.getDocid(1));
    assertEquals("clueweb09-en0000-29-13313", mapping.getDocid(1000000));


    assertEquals(1, mapping.getDocno("clueweb09-en0000-00-00000"));
    assertEquals(1000000, mapping.getDocno("clueweb09-en0000-29-13313"));
  }

View Full Code Here

Examples of edu.umd.cloud9.collection.clue.ClueWarcDocnoMapping

      in.close();


      LOG.info("index class: " + indexClass);


      try {
        sForwardIndex = new IndexableAnchorTextForwardIndex(new ClueWarcDocnoMapping());
        sForwardIndex.loadIndex(new Path(indexFile), new Path(mappingFile), FileSystem.get(conf));
      } catch (Exception e) {
        e.printStackTrace();
        throw new RuntimeException("Error initializing forward index!");
      }

View Full Code Here

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.