List of bulkLoadHFiles() Examples

Examples of bulkLoadHFiles()

org.apache.hadoop.hbase.client.coprocessor.SecureBulkLoadClient.bulkLoadHFiles()
org.apache.hadoop.hbase.coprocessor.SecureBulkLoadClient.bulkLoadHFiles()
org.apache.hadoop.hbase.ipc.HRegionInterface.bulkLoadHFiles()
Atomically bulk load multiple HFiles (say from different column families) into an open region. @param familyPaths List of (family, hfile path) pairs @param regionName name of region to load hfiles into @return true if successful, false if failed recoverably @throws IOException if fails unrecoverably
org.apache.hadoop.hbase.regionserver.HRegion.bulkLoadHFiles()
Attempts to atomically load a group of hfiles. This is critical for loading rows with multiple column families atomically. @param familyPaths List of Pair @return true if successful, false if failed recoverably @throws IOException if failed unrecoverably.

Examples of org.apache.hadoop.hbase.coprocessor.SecureBulkLoadClient.bulkLoadHFiles()

          if(!useSecure) {
             success = server.bulkLoadHFiles(famPaths, regionName, assignSeqIds);
          } else {
            HTable table = new HTable(conn.getConfiguration(), tableName);
            secureClient = new SecureBulkLoadClient(table, location.getRegionInfo().getStartKey());
            success = secureClient.bulkLoadHFiles(famPaths, userToken, bulkToken, assignSeqIds);
          }
          return success;
        } finally {
          //Best effort copying of files that might not have been imported
          //from the staging directory back to original location

View Full Code Here

Examples of org.apache.hadoop.hbase.ipc.HRegionInterface.bulkLoadHFiles()

        (byte[]) Mockito.any(), Mockito.anyBoolean())).
      thenReturn(loc);
    Mockito.when(c.locateRegion((byte[]) Mockito.any(), (byte[]) Mockito.any())).
      thenReturn(loc);
    HRegionInterface hri = Mockito.mock(HRegionInterface.class);
    Mockito.when(hri.bulkLoadHFiles(Mockito.anyList(), (byte [])Mockito.any(),
      Mockito.anyBoolean())).thenThrow(new IOException("injecting bulk load error"));
    Mockito.when(c.getHRegionConnection(Mockito.anyString(), Mockito.anyInt())).
      thenReturn(hri);
    return c;
  }

View Full Code Here

Examples of org.apache.hadoop.hbase.ipc.HRegionInterface.bulkLoadHFiles()

        (byte[]) Mockito.any(), Mockito.anyBoolean())).
      thenReturn(loc);
    Mockito.when(c.locateRegion((byte[]) Mockito.any(), (byte[]) Mockito.any())).
      thenReturn(loc);
    HRegionInterface hri = Mockito.mock(HRegionInterface.class);
    Mockito.when(hri.bulkLoadHFiles(Mockito.anyList(), (byte [])Mockito.any())).
      thenThrow(new IOException("injecting bulk load error"));
    Mockito.when(c.getHRegionConnection(Mockito.anyString(), Mockito.anyInt())).
      thenReturn(hri);
    return c;
  }

View Full Code Here

Examples of org.apache.hadoop.hbase.ipc.HRegionInterface.bulkLoadHFiles()

        (byte[]) Mockito.any(), Mockito.anyBoolean())).
      thenReturn(loc);
    Mockito.when(c.locateRegion((byte[]) Mockito.any(), (byte[]) Mockito.any())).
      thenReturn(loc);
    HRegionInterface hri = Mockito.mock(HRegionInterface.class);
    Mockito.when(hri.bulkLoadHFiles(Mockito.anyList(), (byte [])Mockito.any())).
      thenThrow(new IOException("injecting bulk load error"));
    Mockito.when(c.getHRegionConnection(Mockito.anyString(), Mockito.anyInt())).
      thenReturn(hri);
    return c;
  }

View Full Code Here

Examples of org.apache.hadoop.hbase.ipc.HRegionInterface.bulkLoadHFiles()

        (byte[]) Mockito.any(), Mockito.anyBoolean())).
      thenReturn(loc);
    Mockito.when(c.locateRegion((byte[]) Mockito.any(), (byte[]) Mockito.any())).
      thenReturn(loc);
    HRegionInterface hri = Mockito.mock(HRegionInterface.class);
    Mockito.when(hri.bulkLoadHFiles(Mockito.anyList(), (byte [])Mockito.any(),
      Mockito.anyBoolean())).thenThrow(new IOException("injecting bulk load error"));
    Mockito.when(c.getHRegionConnection(Mockito.anyString(), Mockito.anyInt())).
      thenReturn(hri);
    return c;
  }

View Full Code Here

Examples of org.apache.hadoop.hbase.ipc.HRegionInterface.bulkLoadHFiles()

        (byte[]) Mockito.any(), Mockito.anyBoolean())).
      thenReturn(loc);
    Mockito.when(c.locateRegion((byte[]) Mockito.any(), (byte[]) Mockito.any())).
      thenReturn(loc);
    HRegionInterface hri = Mockito.mock(HRegionInterface.class);
    Mockito.when(hri.bulkLoadHFiles(Mockito.anyList(), (byte [])Mockito.any(),
      Mockito.anyBoolean())).thenThrow(new IOException("injecting bulk load error"));
    Mockito.when(c.getHRegionConnection(Mockito.anyString(), Mockito.anyInt())).
      thenReturn(hri);
    return c;
  }

View Full Code Here

Examples of org.apache.hadoop.hbase.regionserver.HRegion.bulkLoadHFiles()

    Path f =  new Path(basedir, "hfile");
    HFileTestUtil.createHFile(this.conf, fs, f, family, family, Bytes.toBytes(""),
        Bytes.toBytes("z"), 10);
    List <Pair<byte[],String>>  hfs= new ArrayList<Pair<byte[],String>>(1);
    hfs.add(Pair.newPair(family, f.toString()));
    region.bulkLoadHFiles(hfs, true);


    // Add an edit so something in the WAL
    byte [] row = Bytes.toBytes(tableNameStr);
    region.put((new Put(row)).add(family, family, family));
    wal.sync();

View Full Code Here

Examples of org.apache.hadoop.hbase.regionserver.HRegion.bulkLoadHFiles()

      Path f = new Path(basedir, "hfile"+i);
      HFileTestUtil.createHFile(this.conf, fs, f, family, family, Bytes.toBytes(i + "00"),
          Bytes.toBytes(i + "50"), 10);
      hfs.add(Pair.newPair(family, f.toString()));
    }
    region.bulkLoadHFiles(hfs, true);
    final int rowsInsertedCount = 31;
    assertEquals(rowsInsertedCount, getScannedCount(region.getScanner(new Scan())));


    // major compact to turn all the bulk loaded files into one normal file
    region.compactStores(true);

View Full Code Here

Examples of org.apache.hadoop.hbase.regionserver.HRegion.bulkLoadHFiles()

    byte [] row = Bytes.toBytes(tableNameStr);
    writer.append(new KeyValue(row, family, family, row));
    writer.close();
    List <Pair<byte[],String>>  hfs= new ArrayList<Pair<byte[],String>>(1);
    hfs.add(Pair.newPair(family, f.toString()));
    region.bulkLoadHFiles(hfs);
    // Add an edit so something in the WAL
    region.put((new Put(row)).add(family, family, family));
    wal.sync();


    // Now 'crash' the region by stealing its wal

View Full Code Here

Examples of org.apache.hadoop.hbase.regionserver.HRegion.bulkLoadHFiles()

    byte [] row = Bytes.toBytes(tableNameStr);
    writer.append(new KeyValue(row, family, family, row));
    writer.close();
    List <Pair<byte[],String>>  hfs= new ArrayList<Pair<byte[],String>>(1);
    hfs.add(Pair.newPair(family, f.toString()));
    region.bulkLoadHFiles(hfs);
    // Add an edit so something in the WAL
    region.put((new Put(row)).add(family, family, family));
    wal.sync();


    // Now 'crash' the region by stealing its wal

View Full Code Here

0 1 2 3 4

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.