List of readCsvFile() Examples

Examples of readCsvFile()

adlzanchetta.chronoDataIO.csv.ChronoDataCsvImporter.readCsvFile()
eu.stratosphere.api.java.ExecutionEnvironment.readCsvFile()
Creates a CSV reader to read a comma separated value (CSV) file. The reader has options to define parameters and field types and will eventually produce the DataSet that corresponds to the read and parsed CSV input. @param filePath The path of the CSV file. @return A CsvReader that can be used to configure the CSV input.
org.apache.flink.api.java.ExecutionEnvironment.readCsvFile()
Creates a CSV reader to read a comma separated value (CSV) file. The reader has options to define parameters and field types and will eventually produce the DataSet that corresponds to the read and parsed CSV input. @param filePath The path of the CSV file. @return A CsvReader that can be used to configure the CSV input.

Examples of adlzanchetta.chronoDataIO.csv.ChronoDataCsvImporter.readCsvFile()

        csvImporter.setDateColumn(0);
        csvImporter.setDataColumn(1);
        
        // read file
        try{
            dataSeq = csvImporter.readCsvFile();
        } catch (IOException exp) {
            System.err.println("IOException: " + exp.getMessage());
            return;
        } catch (ParseException exp) {
            System.err.println("ParseException: " + exp.getMessage());

View Full Code Here

Examples of adlzanchetta.chronoDataIO.csv.ChronoDataCsvImporter.readCsvFile()

        csvImporter.setDateColumn(0);
        csvImporter.setDataColumn(1);
        
        // read file
        try{
            dataSeq = csvImporter.readCsvFile();
        } catch (IOException exp) {
            System.err.println("IOException: " + exp.getMessage());
            return;
        } catch (ParseException exp) {
            System.err.println("ParseException: " + exp.getMessage());

View Full Code Here

Examples of eu.stratosphere.api.java.ExecutionEnvironment.readCsvFile()

  public void testGroupedReduceWithFieldPositionKey() {
    try {
      ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
      env.setDegreeOfParallelism(8);
      
      DataSet<Tuple2<String, Double>> data = env.readCsvFile("file:///will/never/be/read").types(String.class, Double.class)
        .name("source").setParallelism(6);
      
      data
        .groupBy(1)
        .reduce(new ReduceFunction<Tuple2<String,Double>>() {

View Full Code Here

Examples of eu.stratosphere.api.java.ExecutionEnvironment.readCsvFile()

  public void testGroupedReduceWithSelectorFunctionKey() {
    try {
      ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
      env.setDegreeOfParallelism(8);
      
      DataSet<Tuple2<String, Double>> data = env.readCsvFile("file:///will/never/be/read").types(String.class, Double.class)
        .name("source").setParallelism(6);
      
      data
        .groupBy(new KeySelector<Tuple2<String,Double>, String>() { 
          public String getKey(Tuple2<String, Double> value) { return value.f0; }

View Full Code Here

Examples of eu.stratosphere.api.java.ExecutionEnvironment.readCsvFile()

  public void testGroupedReduceWithFieldPositionKeyNonCombinable() {
    try {
      ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
      env.setDegreeOfParallelism(8);
      
      DataSet<Tuple2<String, Double>> data = env.readCsvFile("file:///will/never/be/read").types(String.class, Double.class)
        .name("source").setParallelism(6);
      
      data
        .groupBy(1)
        .reduceGroup(new GroupReduceFunction<Tuple2<String, Double>, Tuple2<String, Double>>() {

View Full Code Here

Examples of eu.stratosphere.api.java.ExecutionEnvironment.readCsvFile()

  public void testGroupedReduceWithFieldPositionKeyCombinable() {
    try {
      ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
      env.setDegreeOfParallelism(8);
      
      DataSet<Tuple2<String, Double>> data = env.readCsvFile("file:///will/never/be/read").types(String.class, Double.class)
        .name("source").setParallelism(6);
      
      ReduceGroupOperator<Tuple2<String, Double>, Tuple2<String, Double>> reduced = data
          .groupBy(1)
          .reduceGroup(new GroupReduceFunction<Tuple2<String, Double>, Tuple2<String, Double>>() {

View Full Code Here

Examples of eu.stratosphere.api.java.ExecutionEnvironment.readCsvFile()

  public void testGroupedReduceWithSelectorFunctionKeyNoncombinable() {
    try {
      ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
      env.setDegreeOfParallelism(8);
      
      DataSet<Tuple2<String, Double>> data = env.readCsvFile("file:///will/never/be/read").types(String.class, Double.class)
        .name("source").setParallelism(6);
      
      data
        .groupBy(new KeySelector<Tuple2<String,Double>, String>() { 
          public String getKey(Tuple2<String, Double> value) { return value.f0; }

View Full Code Here

Examples of eu.stratosphere.api.java.ExecutionEnvironment.readCsvFile()

  public void testGroupedReduceWithSelectorFunctionKeyCombinable() {
    try {
      ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
      env.setDegreeOfParallelism(8);
      
      DataSet<Tuple2<String, Double>> data = env.readCsvFile("file:///will/never/be/read").types(String.class, Double.class)
        .name("source").setParallelism(6);
      
      ReduceGroupOperator<Tuple2<String, Double>, Tuple2<String, Double>> reduced = data
        .groupBy(new KeySelector<Tuple2<String,Double>, String>() { 
          public String getKey(Tuple2<String, Double> value) { return value.f0; }

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.readCsvFile()

  private Plan getTestPlanRightStatic(String strategy) {
    
    ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    env.setDegreeOfParallelism(DEFAULT_PARALLELISM);
    
    DataSet<Tuple3<Long, Long, Long>> bigInput = env.readCsvFile("file://bigFile").types(Long.class, Long.class, Long.class).name("bigFile");
    
    DataSet<Tuple3<Long, Long, Long>> smallInput = env.readCsvFile("file://smallFile").types(Long.class, Long.class, Long.class).name("smallFile");
    
    IterativeDataSet<Tuple3<Long, Long, Long>> iteration = bigInput.iterate(10);

View Full Code Here

Examples of org.apache.flink.api.java.ExecutionEnvironment.readCsvFile()

    ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    env.setDegreeOfParallelism(DEFAULT_PARALLELISM);
    
    DataSet<Tuple3<Long, Long, Long>> bigInput = env.readCsvFile("file://bigFile").types(Long.class, Long.class, Long.class).name("bigFile");
    
    DataSet<Tuple3<Long, Long, Long>> smallInput = env.readCsvFile("file://smallFile").types(Long.class, Long.class, Long.class).name("smallFile");
    
    IterativeDataSet<Tuple3<Long, Long, Long>> iteration = bigInput.iterate(10);
    
    Configuration joinStrategy = new Configuration();
    joinStrategy.setString(PactCompiler.HINT_SHIP_STRATEGY, PactCompiler.HINT_SHIP_STRATEGY_REPARTITION_HASH);

View Full Code Here

0 1 2

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.