Examples of CodecFileNamingStrategy

org.springframework.data.hadoop.store.strategy.naming.CodecFileNamingStrategy
A {@code FileNamingStrategy} which adds suffix based on known codec. @author Janne Valkealahti

Examples of org.springframework.data.hadoop.store.strategy.naming.CodecFileNamingStrategy

  @Test
  public void testWriteReadManyLinesWithGzipWithCodecNaming() throws IOException {
    TextFileWriter writer = new TextFileWriter(getConfiguration(), testDefaultPath,
        Codecs.GZIP.getCodecInfo());
    ChainedFileNamingStrategy fileNamingStrategy = new ChainedFileNamingStrategy();
    fileNamingStrategy.register(new CodecFileNamingStrategy());
    fileNamingStrategy.register(new StaticFileNamingStrategy());
    writer.setFileNamingStrategy(fileNamingStrategy);
    TestUtils.writeData(writer, DATA09ARRAY);


    TextFileReader reader = new TextFileReader(getConfiguration(), new Path(testDefaultPath, "data.gzip"),

View Full Code Here

Examples of org.springframework.data.hadoop.store.strategy.naming.CodecFileNamingStrategy

    TextFileWriter writer = new TextFileWriter(getConfiguration(), testDefaultPath, Codecs.GZIP.getCodecInfo());


    ChainedFileNamingStrategy fileNamingStrategy = new ChainedFileNamingStrategy();
    fileNamingStrategy.register(new StaticFileNamingStrategy("data"));
    fileNamingStrategy.register(new RollingFileNamingStrategy());
    fileNamingStrategy.register(new CodecFileNamingStrategy());
    writer.setFileNamingStrategy(fileNamingStrategy);
    writer.setInWritingSuffix(".tmp");
    writer.afterPropertiesSet();
    TestUtils.writeData(writer, dataArray);


    Thread.sleep(1000);


    writer = new TextFileWriter(getConfiguration(), testDefaultPath, Codecs.GZIP.getCodecInfo());
    fileNamingStrategy = new ChainedFileNamingStrategy();
    fileNamingStrategy.register(new StaticFileNamingStrategy("data"));
    fileNamingStrategy.register(new RollingFileNamingStrategy());
    fileNamingStrategy.register(new CodecFileNamingStrategy());
    writer.setFileNamingStrategy(fileNamingStrategy);
    writer.setInWritingSuffix(".tmp");
    writer.afterPropertiesSet();
    TestUtils.writeData(writer, dataArray);


    Thread.sleep(1000);


    writer = new TextFileWriter(getConfiguration(), testDefaultPath, Codecs.GZIP.getCodecInfo());
    fileNamingStrategy = new ChainedFileNamingStrategy();
    fileNamingStrategy.register(new StaticFileNamingStrategy("data"));
    fileNamingStrategy.register(new RollingFileNamingStrategy());
    fileNamingStrategy.register(new CodecFileNamingStrategy());
    writer.setFileNamingStrategy(fileNamingStrategy);
    writer.setInWritingSuffix(".tmp");
    writer.afterPropertiesSet();
    TestUtils.writeData(writer, dataArray);

View Full Code Here

Examples of org.springframework.data.hadoop.store.strategy.naming.CodecFileNamingStrategy

    ByteArrayInputStream stream = new ByteArrayInputStream(buf.toString().getBytes());
    OutputStreamWriter writer = new OutputStreamWriter(getConfiguration(), testDefaultPath, Codecs.GZIP.getCodecInfo());


    ChainedFileNamingStrategy fileNamingStrategy = new ChainedFileNamingStrategy();
    fileNamingStrategy.register(new StaticFileNamingStrategy("data"));
    fileNamingStrategy.register(new CodecFileNamingStrategy());
    writer.setFileNamingStrategy(fileNamingStrategy);


    doWithInputStream(stream, writer);


    TextFileReader reader = new TextFileReader(getConfiguration(), new Path(testDefaultPath, "data.gzip"), Codecs.GZIP.getCodecInfo());

View Full Code Here

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.