Examples of CBZip2OutputStream

at.molindo.thirdparty.org.apache.tools.bzip2.CBZip2OutputStream

An output stream that compresses into the BZip2 format (without the file header chars) into another stream.

The compression requires large amounts of memory. Thus you should call the {@link #close() close()} method as soon as possible, to forceCBZip2OutputStream to release the allocated memory.

You can shrink the amount of allocated memory and maybe raise the compression speed by choosing a lower blocksize, which in turn may cause a lower compression ratio. You can avoid unnecessary memory allocation by avoiding using a blocksize which is bigger than the size of the input.

You can compute the memory usage for compressing by the following formula:

 <code>400k + (9 * blocksize)</code>.

To get the memory required for decompression by {@link CBZip2InputStream CBZip2InputStream} use

 <code>65k + (5 * blocksize)</code>.

Memory usage by blocksize
Blocksize	Compression memory usage	Decompression memory usage
100k	1300k	565k
200k	2200k	1065k
300k	3100k	1565k
400k	4000k	2065k
500k	4900k	2565k
600k	5800k	3065k
700k	6700k	3565k
800k	7600k	4065k
900k	8500k	4565k

For decompression CBZip2InputStream allocates less memory if the bzipped input is smaller than one block.

Instances of this class are not threadsafe.

TODO: Update to BZip2 1.0.1

org.apache.commons.compress.bzip2.CBZip2OutputStream

An output stream that compresses into the BZip2 format (without the file header chars) into another stream.

The compression requires large amounts of memory. Thus you should call the {@link #close() close()} method as soon as possible, to forceCBZip2OutputStream to release the allocated memory.

You can compute the memory usage for compressing by the following formula:

 <code>400k + (9 * blocksize)</code>.

To get the memory required for decompression by {@link CBZip2InputStream CBZip2InputStream} use

 <code>65k + (5 * blocksize)</code>.

Memory usage by blocksize
Blocksize	Compression memory usage	Decompression memory usage
100k	1300k	565k
200k	2200k	1065k
300k	3100k	1565k
400k	4000k	2065k
500k	4900k	2565k
600k	5800k	3065k
700k	6700k	3565k
800k	7600k	4065k
900k	8500k	4565k

For decompression CBZip2InputStream allocates less memory if the bzipped input is smaller than one block.

Instances of this class are not threadsafe.

TODO: Update to BZip2 1.0.1

org.apache.hadoop.io.compress.bzip2.CBZip2OutputStream

An output stream that compresses into the BZip2 format (without the file header chars) into another stream.

The compression requires large amounts of memory. Thus you should call the {@link #close() close()} method as soon as possible, to forceCBZip2OutputStream to release the allocated memory.

You can compute the memory usage for compressing by the following formula:

 <code>400k + (9 * blocksize)</code>.

To get the memory required for decompression by {@link CBZip2InputStream CBZip2InputStream} use

 <code>65k + (5 * blocksize)</code>.

Memory usage by blocksize
Blocksize	Compression memory usage	Decompression memory usage
100k	1300k	565k
200k	2200k	1065k
300k	3100k	1565k
400k	4000k	2065k
500k	4900k	2565k
600k	5800k	3065k
700k	6700k	3565k
800k	7600k	4065k
900k	8500k	4565k

For decompression CBZip2InputStream allocates less memory if the bzipped input is smaller than one block.

Instances of this class are not threadsafe.

TODO: Update to BZip2 1.0.1

org.apache.tools.bzip2.CBZip2OutputStream
An output stream that compresses into the BZip2 format (without the file header chars) into another stream. TODO: Update to BZip2 1.0.1
org.apache.tools.bzip2r.CBZip2OutputStream
An output stream that compresses into the BZip2 format (without the file header chars) into another stream. @author Keiron LiddleTODO: Update to BZip2 1.0.1
org.bouncycastle.apache.bzip2.CBZip2OutputStream
An output stream that compresses into the BZip2 format (with the file header chars) into another stream. @author Keiron LiddleTODO: Update to BZip2 1.0.1 NB: note this class has been modified to add a leading BZ to the start of the BZIP2 stream to make it compatible with other PGP programs.
org.bouncycastle2.apache.bzip2.CBZip2OutputStream
An output stream that compresses into the BZip2 format (with the file header chars) into another stream. @author Keiron LiddleTODO: Update to BZip2 1.0.1 NB: note this class has been modified to add a leading BZ to the start of the BZIP2 stream to make it compatible with other PGP programs.
org.codehaus.plexus.archiver.bzip2.CBZip2OutputStream
An output stream that compresses into the BZip2 format (without the file header chars) into another stream.
TODO: Update to BZip2 1.0.1 @version $Revision$ $Date$from org.apache.ant.tools.bzip2.CBZip2OutputStream v1.22
org.eclipse.php.internal.core.tar.CBZip2OutputStream
An output stream that compresses into the BZip2 format (without the file header chars) into another stream. TODO: Update to BZip2 1.0.1

Examples of org.apache.tools.bzip2r.CBZip2OutputStream

            File testFile = File.createTempFile("testPigLineRecordReader",
                    ".txt.bz2");
            String text = "This is a text";
            
            PrintStream ps = new PrintStream( 
                    new CBZip2OutputStream( new FileOutputStream( testFile )) );
            for( int i = 0; i < LOOP_COUNT; i++ ) {
                String counter = new Integer(i).toString();
                String output = counter.concat( text.substring(counter.length() ) );
                ps.println( output );
            }

View Full Code Here

Examples of org.apache.tools.bzip2r.CBZip2OutputStream

    public void testEmptyLineBZFile() {
        try {
            File testFile = File.createTempFile("testPigLineRecordReader", ".txt.bz2");
            String text = "This is a text";


            PrintStream ps = new PrintStream( new CBZip2OutputStream( new FileOutputStream( testFile )) );
            for( int i = 0; i < LOOP_COUNT; i++ ) {
                ps.println( text );
                // Add an empty line
                ps.println("");
            }

View Full Code Here

Examples of org.apache.tools.bzip2r.CBZip2OutputStream

            this.sfunc = sfunc;
            fs.delete(file, true);
            this.os = fs.create(file);
            String name = file.getName();
            if (name.endsWith(".bz") || name.endsWith(".bz2")) {
                os = new CBZip2OutputStream(os);
            }
            this.sfunc.bindTo(os);
        }

View Full Code Here

Examples of org.apache.tools.bzip2r.CBZip2OutputStream

        
        File out = File.createTempFile("junit", ".bz2");
        out.delete();
        String clusterOutput = Util.removeColon(out.getAbsolutePath());
               
        CBZip2OutputStream cos = 
            new CBZip2OutputStream(new FileOutputStream(in));
        for (int i = 1; i < 100; i++) {
            StringBuffer sb = new StringBuffer();
            sb.append(i).append("\n").append(-i).append("\n");
            byte bytes[] = sb.toString().getBytes();
            cos.write(bytes);
        }
        cos.close();
                       
        pig.registerQuery("AA = load '"
                + Util.generateURI(Util.encodeEscape(in.getAbsolutePath()), pig.getPigContext())
                + "';");
        pig.registerQuery("A = foreach (group (filter AA by $0 > 0) all) generate flatten($1);");

View Full Code Here

Examples of org.apache.tools.bzip2r.CBZip2OutputStream

        
        File out = File.createTempFile("junit", ".bz2");
        out.delete();
        String clusterOutput = Util.removeColon(out.getAbsolutePath());
               
        CBZip2OutputStream cos = 
            new CBZip2OutputStream(new FileOutputStream(in));
        for (int i = 1; i < 100; i++) {
            StringBuffer sb = new StringBuffer();
            sb.append(i).append("\n").append(-i).append("\n");
            byte bytes[] = sb.toString().getBytes();
            cos.write(bytes);
        }
        cos.close();
                       
        pig.registerQuery("AA = load '"
                + Util.generateURI(in.getAbsolutePath(), pig.getPigContext())
                + "';");
        pig.registerQuery("A = foreach (group (filter AA by $0 > 0) all) generate flatten($1);");

View Full Code Here

Examples of org.apache.tools.bzip2r.CBZip2OutputStream

        String compressedInputFileName = in.getAbsolutePath();
        String clusterCompressedFilePath = Util.removeColon(compressedInputFileName);
        in.deleteOnExit();
        
        try {
            CBZip2OutputStream cos = 
                new CBZip2OutputStream(new FileOutputStream(in));
            for (int i = 0; i < inputData.length; i++) {
                StringBuffer sb = new StringBuffer();
                sb.append(inputData[i]).append("\n");
                byte bytes[] = sb.toString().getBytes();
                cos.write(bytes);
            }
            cos.close();
            
            Util.copyFromLocalToCluster(cluster, compressedInputFileName,
                clusterCompressedFilePath);
            
            // pig script to read compressed input

View Full Code Here

Examples of org.apache.tools.bzip2r.CBZip2OutputStream


        String unCompressedInputFileName = "testRecordDelims-uncomp.txt";
        Util.createInputFile(cluster, unCompressedInputFileName, inputData);
        
        try {
            CBZip2OutputStream cos = 
                new CBZip2OutputStream(new FileOutputStream(in));
            for (int i = 0; i < inputData.length; i++) {
                StringBuffer sb = new StringBuffer();
                sb.append(inputData[i]).append("\n");
                byte bytes[] = sb.toString().getBytes();
                cos.write(bytes);
            }
            cos.close();
            
            Util.copyFromLocalToCluster(cluster, compressedInputFileName,
                    clusterCompressedFilePath);
            
            // pig script to read uncompressed input

View Full Code Here

Examples of org.apache.tools.bzip2r.CBZip2OutputStream

     */
    @Test
    public void testEmptyBzip() throws Exception {
        File tmp = File.createTempFile("junit", ".tmp");
        tmp.deleteOnExit();
        CBZip2OutputStream cos = new CBZip2OutputStream(new FileOutputStream(
                tmp));
        cos.close();
        assertNotSame(0, tmp.length());
        FileSystem fs = FileSystem.getLocal(new Configuration(false));
        CBZip2InputStream cis = new CBZip2InputStream(
                fs.open(new Path(tmp.getAbsolutePath())), -1, tmp.length());
        assertEquals(-1, cis.read(new byte[100]));

View Full Code Here

Examples of org.apache.tools.bzip2r.CBZip2OutputStream


        String unCompressedInputFileName = "testRecordDelims-uncomp.txt";
        Util.createInputFile(cluster, unCompressedInputFileName, inputDataMerged);
        
        try {
            CBZip2OutputStream cos = 
                new CBZip2OutputStream(new FileOutputStream(in1));
            for (int i = 0; i < inputData1.length; i++) {
                StringBuffer sb = new StringBuffer();
                sb.append(inputData1[i]).append("\n");
                byte bytes[] = sb.toString().getBytes();
                cos.write(bytes);
            }
            cos.close();
            
            CBZip2OutputStream cos2 = 
                new CBZip2OutputStream(new FileOutputStream(in2));
            for (int i = 0; i < inputData2.length; i++) {
                StringBuffer sb = new StringBuffer();
                sb.append(inputData2[i]).append("\n");
                byte bytes[] = sb.toString().getBytes();
                cos2.write(bytes);
            }
            cos2.close();


            // cat
            catInto(compressedInputFileName2, compressedInputFileName1);
            Util.copyFromLocalToCluster(cluster, compressedInputFileName1,
                    compressedInputFileName1);

View Full Code Here

Examples of org.apache.tools.bzip2r.CBZip2OutputStream

            this.sfunc = sfunc;
            fs.delete(file, true);
            this.os = fs.create(file);
            String name = file.getName();
            if (name.endsWith(".bz") || name.endsWith(".bz2")) {
                os = new CBZip2OutputStream(os);
            }
            this.sfunc.bindTo(os);
        }

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.