Examples of com.sleepycat.je.tree.IN

com.sleepycat.je.tree.IN
An IN represents an Internal Node in the JE tree. Explanation of KD (KnownDeleted) and PD (PendingDelete) entry flags =================================================================== PD: set for all LN entries that are deleted, even before the LN is committed. Is used as an authoritative (transactionally correct) indication that an LN is deleted. PD will be cleared if the txn for the deleted LN is aborted. KD: set under special conditions for entries containing LNs which are known to be obsolete. Not used for entries in an active/uncommitted transaction. First notice that IN.fetchTarget will allow a FileNotFoundException when the PD or KD flag is set on the entry. And it will allow a NULL_LSN when the KD flag is set. KD was implemented first, and was originally used when the cleaner attempts to migrate an LN and discovers it is deleted (see Cleaner.migrateLN). We need KD because the INCompressor may not have run, and may not have compressed the BIN. There's the danger that we'll try to fetch that entry, and that the file was deleted by the cleaner. KD was used more recently when an unexpected exception occurs while logging an LN, after inserting the entry. Rather than delete the entry to clean up, we mark the entry KD so it won't cause a fetch error later. In this case the entry LSN is NULL_LSN. See Tree.insertNewSlot. PD is closely related to the first use of KD above (for cleaned deleted LNs) and came about because of a cleaner optimization we make. The cleaner considers all deleted LN log entries to be obsolete, without doing a tree lookup, and without any record of an obsolete offset. This makes the cost of cleaning of deleted LNs very low. For example, if the log looks like this: 100 LNA 200 delete of LNA then LSN 200 will be considered obsolete when this file is processed by the cleaner. After all, only two things can happen: (1) the txn commits, and we don't need LSN 200, because we can wipe this LN out of the tree, or (2) the txn aborts, and we don't need LSN 200, because we are going to revert to LSN 100/LNA. We set PD for the entry of a deleted LN at the time of the operation, and we clear PD if the transaction aborts. There is no real danger that this log entry will be processed by the cleaner before it's committed, because cleaning can only happen after the first active LSN. Just as in the first use of KD above, setting PD is necessary to avoid a fetch error, when the file is deleted by the cleaner but the entry containing the deleted LN has not been deleted by the INCompressor. PD is also set in replication rollback, when LNs are marked as invisible. When LSN locking was implemented (see CursorImpl.lockLN), the PD flag took on additional meaning. PD is used to determine whether an LN is deleted without fetching it, and therefore is relied on to be transactionally correct. In addition to the setting and use of the KD/PD flags described above, the situation is complicated by the fact that we must restore the state of these flags during abort, recovery, and set them properly during slot reuse. We have been meaning to consider whether PD and KD can be consolidated into one flag: simply the Deleted flag. The Deleted flag would be set in the same way as PD is currently set, as well as the second use of KD described above (when the LSN is NULL_LSN after an insertion error). The use of KD and PD for invisible entries and recovery rollback should also be considered. If we consolidate the two flags and set the Deleted flag during a delete operation (like PD), we'll have to remove optimizations (in CursorImpl for example) that consider a slot deleted when KD is set. Since KD is rarely set currently, this shouldn't have a noticeable performance impact.

        INList inList = DbInternal.envGetEnvironmentImpl(env).getInMemoryINs();
        inList.latchMajor();
        try {
            Iterator iter = inList.iterator();
            while (iter.hasNext()) {
                IN in = (IN) iter.next();
                if (in instanceof BIN) {
                    BIN bin = (BIN) in;
                    bin.latch();
                    assertTrue(bin.evictLNs() > 0);
                    bin.releaseLatch();

View Full Code Here

        INList inList = envImpl.getInMemoryINs();
        inList.latchMajor();
        try {
            Iterator iter = inList.iterator();
            while (iter.hasNext()) {
                IN in = (IN) iter.next();
                System.out.println("in nodeId=" + in.getNodeId());
            }
        } finally {
            inList.releaseMajorLatch();
        }
    }

View Full Code Here

        }


        public IN doWork(ChildReference root) 
            throws DatabaseException {


            IN rootIN = (IN) root.fetchTarget(database, null);
            readyToSplit = rootIN.needsSplitting();
            return null;
        }

View Full Code Here


        inList.latchMajor();
        try {
            Iterator iter = inList.iterator();
            while (iter.hasNext()) {
                IN in = (IN) iter.next();
                long size = in.getInMemorySize();
                totalSize += size;
            }
        } finally {
            inList.releaseMajorLatch();
        }

View Full Code Here

            /* consolidate the INList first. */
            inList.latchMinorAndDumpAddedINs();


      Iterator iter = inList.iterator();
      while (iter.hasNext()) {
    IN thisIN = (IN) iter.next();
    if (thisIN.getDatabase() == dbImpl) {
        foundSome = true;
        if (removeINsFromINList) {
      iter.remove();
                        memoryChange += (thisIN.getAccumulatedDelta() -
                                         thisIN.getInMemorySize());
                        thisIN.setInListResident(false);
        }
                    foundSet.add(thisIN);
    }
      }
        } catch (DatabaseException e) {
            /* Update the memory budget with any changes. */
            mb.updateTreeMemoryUsage(memoryChange);
            throw e;
  } finally {
      inList.releaseMajorLatch();
  }


        /* 
         * Do processing outside of INList latch in order to reduce lockout
         * of checkpointing and eviction. 
         */
        if (foundSome) {
            Iterator iter = foundSet.iterator();
            while (iter.hasNext()) {
                IN thisIN = (IN) iter.next();
                accumulateLSNs(thisIN);
            }
        }


        /*

View Full Code Here


    protected void walkInternal()
  throws DatabaseException {


  INList inList = envImpl.getInMemoryINs();
  IN root = null;
  if (!extractINsForDb(inList)) {
      if (rootLsn == DbLsn.NULL_LSN) {
    return;
      }

View Full Code Here

    private BIN searchForBIN(DatabaseImpl db, byte[] mainKey, byte[] dupKey)
        throws DatabaseException {


        /* Search for this IN */
        Tree tree = db.getTree();
        IN in = tree.search
            (mainKey, SearchType.NORMAL, -1, null, false /*updateGeneration*/);


        /* Couldn't find a bin, return null */
        if (in == null) {
            return null;

View Full Code Here

        }


        public IN doWork(ChildReference root) 
            throws DatabaseException {
            
            IN rootIN = (IN) root.fetchTarget(db, null);
            rootLevel = rootIN.getLevel();
            return null;
        }

View Full Code Here

             * list.
             */
            while ((evictBytes < requiredEvictBytes) &&
                   (nNodesScannedThisRun <= inListStartSize)) {


                IN target = selectIN(inList, scanIter);


                if (target == null) {
                    break;
                } else {
                    assert evictProfile.count(target);//intentional side effect

View Full Code Here

     */
    private IN selectIN(INList inList, ScanIterator scanIter)
        throws DatabaseException {


        /* Find the best target in the next <nodesPerScan> nodes. */
        IN target = null;
        long targetGeneration = Long.MAX_VALUE;
        int targetLevel = Integer.MAX_VALUE;
        boolean targetDirty = true;
  boolean envIsReadOnly = envImpl.isReadOnly();
        int scanned = 0;
        boolean wrapped = false;
        while (scanned < nodesPerScan) {
            if (scanIter.hasNext()) {
                IN in = scanIter.next();
                nNodesScannedThisRun++;


                DatabaseImpl db = in.getDatabase();


                /* 
                 * We don't expect to see an IN with a database that has
                 * finished delete processing, because it would have been
                 * removed from the inlist during post-delete cleanup.
                 */
                if (db == null || db.isDeleteFinished()) {
                    String inInfo = " IN type=" + in.getLogType() + " id=" +
                        in.getNodeId() + " not expected on INList";
                    String errMsg = (db == null) ? inInfo :
                        "Database " + db.getDebugName() + " id=" + db.getId() +
                        inInfo;
                    throw new DatabaseException(errMsg);
                }


                /* Ignore if the db is in the middle of delete processing. */
                if (db.isDeleted()) {
                    continue;
                }


                /* 
                 * Don't evict the DatabaseImpl Id Mapping Tree (db 0), both
                 * for object identity reasons and because the id mapping tree
                 * should stay cached.
                 */
                if (db.getId().equals(DbTree.ID_DB_ID)) {
                    continue;
                }


                /* 
                 * If this is a read only database and we have at least one
                 * target, skip any dirty INs (recovery dirties INs even in a
                 * read-only environment). We take at least one target so we
                 * don't loop endlessly if everything is dirty.
                 */
                if (envIsReadOnly && (target != null) && in.getDirty()) {
                    continue;
                }


                /*
                 * Only scan evictable or strippable INs.  This prevents higher
                 * level INs from being selected for eviction, unless they are
                 * part of an unused tree.
                 */
                int evictType = in.getEvictionType();
                if (evictType == IN.MAY_NOT_EVICT) {
                    continue;
                }


                /*
                 * This node is in the scanned node set.  Select according to
                 * the configured eviction policy.
                 */
                if (evictByLruOnly) {


                    /*
                     * Select the node with the lowest generation number,
                     * irrespective of tree level or dirtyness.
                     */
                    if (targetGeneration > in.getGeneration()) {
                        targetGeneration = in.getGeneration();
                        target = in;
                    }
                } else {


                    /*
                     * Select first by tree level, then by dirtyness, then by
                     * generation/LRU.
                     */
                    int level = normalizeLevel(in, evictType);
                    if (targetLevel != level) {
                        if (targetLevel > level) {
                            targetLevel = level;
                            targetDirty = in.getDirty();
                            targetGeneration = in.getGeneration();
                            target = in;
                        }
                    } else if (targetDirty != in.getDirty()) {
                        if (targetDirty) {
                            targetDirty = false;
                            targetGeneration = in.getGeneration();
                            target = in;
                        }
                    } else {
                        if (targetGeneration > in.getGeneration()) {
                            targetGeneration = in.getGeneration();
                            target = in;
                        }
                    }
                }
                scanned++;

View Full Code Here

0 1 2 3 4 5 6 7 8 9

TOP

Related Classes of com.sleepycat.je.tree.IN

com.sleepycat.je.cleaner.Cleaner

com.sleepycat.je.cleaner.Cleaner$RootDoWork

com.sleepycat.je.cleaner.FileProcessor

com.sleepycat.je.cleaner.FileProcessor$RootDoWork

com.sleepycat.je.cleaner.INUtilizationTest

com.sleepycat.je.cleaner.PackedObsoleteInfo

com.sleepycat.je.dbi.CursorImpl

com.sleepycat.je.dbi.DatabaseId

com.sleepycat.je.dbi.DatabaseImpl

com.sleepycat.je.dbi.DatabaseImpl$PreloadLSNTreeWalker

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.