Examples of org.apache.cassandra.utils.obs.OpenBitSet

org.apache.cassandra.utils.obs.OpenBitSet
An "open" BitSet implementation that allows direct access to the arrays of words storing the bits. Derived from Lucene's OpenBitSet, but with a paged backing array (see bits delaration, below).
Unlike java.util.bitset, the fact that bits are packed into an array of longs is part of the interface. This allows efficient implementation of other algorithms by someone other than the author. It also allows one to efficiently implement alternate serialization or interchange formats.
OpenBitSet is faster than java.util.BitSet in most operations and *much* faster at calculating cardinality of sets and results of set operations. It can also handle sets of larger cardinality (up to 64 * 2**32-1)
The goals of OpenBitSet are the fastest implementation possible, and maximum code reuse. Extra safety and encapsulation may always be built on top, but if that's built in, the cost can never be removed (and hence people re-implement their own version in order to get better performance). If you want a "safe", totally encapsulated (and slower and limited) BitSet class, use java.util.BitSet.


    public BloomFilter deserialize(DataInput dis) throws IOException
    {
        int hashes = dis.readInt();
        long bitLength = dis.readInt();
        OpenBitSet bs = new OpenBitSet(bitLength << 6);
        int pageSize = bs.getPageSize();
        int pageCount = bs.getPageCount();


        for (int p = 0; p < pageCount; p++)
        {
            long[] bits = bs.getPage(p);
            for (int i = 0; i < pageSize && bitLength-- > 0; i++)
                bits[i] = dis.readLong();
        }


        return createFilter(hashes, bs);

View Full Code Here

    public final OpenBitSet bitset;


    BloomFilter(int hashes, long numElements, int bucketsPer)
    {
        hashCount = hashes;
        bitset = new OpenBitSet(numElements * bucketsPer + EXCESS);
    }

View Full Code Here


    public BloomFilter deserialize(DataInput dis) throws IOException
    {
        int hashes = dis.readInt();
        long bitLength = dis.readInt();
        OpenBitSet bs = new OpenBitSet(bitLength << 6);
        int pageSize = bs.getPageSize();
        int pageCount = bs.getPageCount();


        for (int p = 0; p < pageCount; p++)
        {
            long[] bits = bs.getPage(p);
            for (int i = 0; i < pageSize && bitLength-- > 0; i++)
                bits[i] = dis.readLong();
        }


        return new BloomFilter(hashes, bs);

View Full Code Here

        return serializer_;
    }


    private static OpenBitSet bucketsFor(long numElements, int bucketsPer)
    {
        return new OpenBitSet(numElements * bucketsPer + EXCESS);
    }

View Full Code Here

        int hashes = dis.readInt();
        int bitLength = dis.readInt();
        long[] bits = new long[bitLength];
        for (int i = 0; i < bitLength; i++)
            bits[i] = dis.readLong();
        OpenBitSet bs = new OpenBitSet(bits, bitLength);
        return new BloomFilter(hashes, bs);
    }

View Full Code Here

        return serializer_;
    }


    private static OpenBitSet bucketsFor(long numElements, int bucketsPer)
    {
        return new OpenBitSet(numElements * bucketsPer + EXCESS);
    }

View Full Code Here

    }


    @Test
    public void testOffHeapCompatibility() throws IOException
    {
        OpenBitSet bs = new OpenBitSet(100000);
        populateAndReserialize(bs);
    }

View Full Code Here

    }


    private static IFilter createFilter(int hash, long numElements, int bucketsPer, boolean offheap)
    {
        long numBits = (numElements * bucketsPer) + BITSET_EXCESS;
        IBitSet bitset = offheap ? new OffHeapBitSet(numBits) : new OpenBitSet(numBits);
        return new Murmur3BloomFilter(hash, bitset);
    }

View Full Code Here

    @Test
    public void testExpectedCompatablity() throws IOException
    {
        DataInputStream dis = new DataInputStream(new FileInputStream(new File(LEGACY_SST_FILE)));
        dis.readInt(); // bloom filter hash count
        OpenBitSet bs = OpenBitSet.deserialize(dis);


        dis = new DataInputStream(new FileInputStream(new File(LEGACY_SST_FILE)));
        dis.readInt(); // bloom filter hash count
        OffHeapBitSet obs = OffHeapBitSet.deserialize(dis);

View Full Code Here

    }


    @Test
    public void testOffHeapCompatibility() throws IOException
    {
        OpenBitSet bs = new OpenBitSet(100000);
        populateAndReserialize(bs);
    }

View Full Code Here

0 1 2 3

TOP

Related Classes of org.apache.cassandra.utils.obs.OpenBitSet

org.apache.cassandra.utils.BitSetTest

org.apache.cassandra.utils.BloomFilter

org.apache.cassandra.utils.BloomFilterSerializer

org.apache.cassandra.utils.FilterFactory

org.apache.cassandra.utils.LongBitSetTest

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.