List of shard() Examples

Examples of shard()

edu.cmu.graphchi.preprocessing.FastSharder.shard()
Execute sharding by reading edges from a inputstream @param inputStream @param format graph input format @throws IOException
org.elasticsearch.cluster.routing.IndexRoutingTable.shard()
org.elasticsearch.index.service.IndexService.shard()
org.elasticsearch.search.internal.InternalSearchHit.shard()

Examples of edu.cmu.graphchi.preprocessing.FastSharder.shard()

            String fileType = (cmdLine.hasOption("filetype") ? cmdLine.getOptionValue("filetype") : null);


            /* Create shards */
            if (baseFilename.equals("pipein")) {     // Allow piping graph in
                FastSharder sharder = createSharder(baseFilename, nShards);
                sharder.shard(System.in, fileType);
            } else {
                FastSharder sharder = createSharder(baseFilename, nShards);
                if (!new File(ChiFilenames.getFilenameIntervals(baseFilename, nShards)).exists()) {
                    sharder.shard(new FileInputStream(new File(baseFilename)), fileType);
                } else {

View Full Code Here

Examples of edu.cmu.graphchi.preprocessing.FastSharder.shard()

                FastSharder sharder = createSharder(baseFilename, nShards);
                sharder.shard(System.in, fileType);
            } else {
                FastSharder sharder = createSharder(baseFilename, nShards);
                if (!new File(ChiFilenames.getFilenameIntervals(baseFilename, nShards)).exists()) {
                    sharder.shard(new FileInputStream(new File(baseFilename)), fileType);
                } else {
                    logger.info("Found shards -- no need to pre-process");
                }
            }

View Full Code Here

Examples of edu.cmu.graphchi.preprocessing.FastSharder.shard()

    public static ALSMatrixFactorization computeALS(String baseFilename, int nShards, int D, int iterations) throws IOException {
        /* Run sharding (preprocessing) if the files do not exist yet */
        FastSharder sharder = createSharder(baseFilename, nShards);
        if (!new File(ChiFilenames.getFilenameIntervals(baseFilename, nShards)).exists() ||
                !new File(baseFilename + ".matrixinfo").exists()) {
            sharder.shard(new FileInputStream(new File(baseFilename)), FastSharder.GraphInputFormat.MATRIXMARKET);
        } else {
            logger.info("Found shards -- no need to preprocess");
        }


        /* Init */

View Full Code Here

Examples of edu.cmu.graphchi.preprocessing.FastSharder.shard()

        CompressedIO.disableCompression();


        /* Create shards */
        FastSharder sharder = createSharder(baseFilename, nShards);
        if (baseFilename.equals("pipein")) {     // Allow piping graph in
            sharder.shard(System.in, fileType);
        } else {
            if (!new File(ChiFilenames.getFilenameIntervals(baseFilename, nShards)).exists()) {
                sharder.shard(new FileInputStream(new File(baseFilename)), fileType);
            } else {
                logger.info("Found shards -- no need to preprocess");

View Full Code Here

Examples of edu.cmu.graphchi.preprocessing.FastSharder.shard()

        FastSharder sharder = createSharder(baseFilename, nShards);
        if (baseFilename.equals("pipein")) {     // Allow piping graph in
            sharder.shard(System.in, fileType);
        } else {
            if (!new File(ChiFilenames.getFilenameIntervals(baseFilename, nShards)).exists()) {
                sharder.shard(new FileInputStream(new File(baseFilename)), fileType);
            } else {
                logger.info("Found shards -- no need to preprocess");
            }
        }

View Full Code Here

Examples of edu.cmu.graphchi.preprocessing.FastSharder.shard()




        /* Create shards */
        FastSharder sharder = createSharder(baseFilename, nShards);
        if (baseFilename.equals("pipein")) {     // Allow piping graph in
            sharder.shard(System.in, fileType);
        } else {
            if (!new File(ChiFilenames.getFilenameIntervals(baseFilename, nShards)).exists()) {
                sharder.shard(new FileInputStream(new File(baseFilename)), fileType);
            } else {
                logger.info("Found shards -- no need to preprocess");

View Full Code Here

Examples of edu.cmu.graphchi.preprocessing.FastSharder.shard()

        FastSharder sharder = createSharder(baseFilename, nShards);
        if (baseFilename.equals("pipein")) {     // Allow piping graph in
            sharder.shard(System.in, fileType);
        } else {
            if (!new File(ChiFilenames.getFilenameIntervals(baseFilename, nShards)).exists()) {
                sharder.shard(new FileInputStream(new File(baseFilename)), fileType);
            } else {
                logger.info("Found shards -- no need to preprocess");
            }
        }

View Full Code Here

Examples of edu.cmu.graphchi.preprocessing.FastSharder.shard()

  public static void convert_matrix_market() throws IOException{
      /* Run sharding (preprocessing) if the files do not exist yet */
        FastSharder sharder = createSharder(ProblemSetup.training, ProblemSetup.nShards);
        if (!new File(ChiFilenames.getFilenameIntervals(ProblemSetup.training, ProblemSetup.nShards)).exists() ||
                !new File(ProblemSetup.training + ".matrixinfo").exists()) {
            sharder.shard(new FileInputStream(new File(ProblemSetup.training)), FastSharder.GraphInputFormat.MATRIXMARKET);
        } else {
            ProblemSetup.logger.info("Found shards -- no need to preprocess");
        }
  }

View Full Code Here

Examples of edu.cmu.graphchi.preprocessing.FastSharder.shard()

        
        /* Preprocessing graph : Making shards */


        FastSharder sharder = createSharder(inputFilePath, nShards);
        if (inputFilePath.equals("pipein")) {     // Allow piping graph in
            sharder.shard(System.in, fileType);
        } else {
            if (!new File(ChiFilenames.getFilenameIntervals(inputFilePath, nShards)).exists()) {
                sharder.shard(new FileInputStream(new File(inputFilePath)), fileType);
            } else {
                logger.info("Found shards -- no need to preprocess");

View Full Code Here

Examples of edu.cmu.graphchi.preprocessing.FastSharder.shard()

        FastSharder sharder = createSharder(inputFilePath, nShards);
        if (inputFilePath.equals("pipein")) {     // Allow piping graph in
            sharder.shard(System.in, fileType);
        } else {
            if (!new File(ChiFilenames.getFilenameIntervals(inputFilePath, nShards)).exists()) {
                sharder.shard(new FileInputStream(new File(inputFilePath)), fileType);
            } else {
                logger.info("Found shards -- no need to preprocess");
            }
        }

View Full Code Here

0 1 2 3 4

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.