Examples of PartitionStrategy

com.cloudera.cdk.data.PartitionStrategy

The strategy used to determine how a dataset is partitioned.

A {@code PartitionStrategy} is configured with one or more{@link FieldPartitioner}s upon creation. When a {@link Dataset} is configuredwith a partition strategy, we say that data is partitioned. Any entities written to a partitioned dataset are evaluated with its {@code PartitionStrategy} which, in turn, produces a {@link PartitionKey}that is used by the dataset implementation to select the proper partition.

Users should use the inner {@link Builder} to create new instances.

@see FieldPartitioner @see PartitionKey @see DatasetDescriptor @see Dataset

org.kitesdk.data.PartitionStrategy

The strategy used to determine how a dataset is partitioned.

When a {@link Dataset} is configuredwith a partition strategy, that data is considered partitioned. Any entities written to a partitioned dataset are evaluated with its {@code PartitionStrategy} to determine which partition to write to.

You should use the inner {@link Builder} to create new instances.

@see DatasetDescriptor @see Dataset

    } catch (IOException e) {
      throw new DatasetIOException("Unable to locate or create dataset partition directory " + partitionDirectory, e);
    }


    int partitionDepth = key.getLength();
    PartitionStrategy subpartitionStrategy = Accessor.getDefault()
        .getSubpartitionStrategy(partitionStrategy, partitionDepth);


    return new FileSystemDataset.Builder<E>()
        .namespace(namespace)
        .name(name)

    }


    for (FileStatus stat : fileStatuses) {
      Path p = fileSystem.makeQualified(stat.getPath());
      PartitionKey key = keyFromDirectory(p.getName());
      PartitionStrategy subPartitionStrategy = Accessor.getDefault()
          .getSubpartitionStrategy(partitionStrategy, 1);
      Builder<E> builder = new FileSystemDataset.Builder<E>()
          .namespace(namespace)
          .name(name)
          .fileSystem(fileSystem)

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.

Examples of PartitionStrategy

Examples of org.kitesdk.data.PartitionStrategy

Examples of org.kitesdk.data.PartitionStrategy