Examples of eu.stratosphere.core.io.StringRecord

eu.stratosphere.core.io.StringRecord
This class stores text using standard UTF8 encoding. It provides methods to serialize, deserialize, and compare texts at byte level. The type of length is integer and is serialized using zero-compressed format.
In addition, it provides methods for string traversal without converting the byte array to a string.
Also includes utilities for serializing/deserialing a string, coding/decoding a string, checking if a byte array contains valid UTF8 code, calculating the length of an encoded string.

  public void invoke() throws Exception {
    this.output.initializeSerializers();


    while (this.input.hasNext()) {


      StringRecord s = input.next();
      this.output.emit(s);
    }


    this.output.flush();
  }

View Full Code Here

    dist.put(1, 1); dist.put(2, 2); dist.put(3, 3);
    Assert.assertEquals(dist, res.getAccumulatorResult("words-per-line"));
    
    // Test distinct words (custom accumulator)
    Set<StringRecord> distinctWords = Sets.newHashSet();
    distinctWords.add(new StringRecord("one"));
    distinctWords.add(new StringRecord("two"));
    distinctWords.add(new StringRecord("three"));
    Assert.assertEquals(distinctWords, res.getAccumulatorResult("distinct-words"));
  }

View Full Code Here

      int wordsPerLine = 0;
      this.word = new StringValue();
      while (tokenizer.next(this.word))
      {
        // Use custom counter
        distinctWords.add(new StringRecord(this.word.getValue()));
  
        this.outputRecord.setField(0, this.word);
        this.outputRecord.setField(1, this.one);
        collector.collect(this.outputRecord);
        ++ wordsPerLine;

View Full Code Here

0 1

TOP

Related Classes of eu.stratosphere.core.io.StringRecord

eu.stratosphere.nephele.executiongraph.ForwardTask1Input1Output

eu.stratosphere.nephele.executiongraph.ForwardTask1Input2Outputs

eu.stratosphere.nephele.executiongraph.ForwardTask2Inputs1Output

eu.stratosphere.nephele.jobmanager.DoubleSourceTask

eu.stratosphere.nephele.jobmanager.DoubleTargetTask

eu.stratosphere.nephele.jobmanager.ForwardTask

eu.stratosphere.nephele.types.StringRecordTest

eu.stratosphere.nephele.util.FileLineReader

eu.stratosphere.nephele.util.FileLineWriter

eu.stratosphere.runtime.io.DefaultChannelSelectorTest

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.