HDFS Active Namenode synchronizes editlog process to journalnode

FSEditLog

For the classes used in the active namenode process, execute logsync() to synchronize the latest editlog segment to the journalnode

Call procedure

FSEditLog.logSync(long mytxid)
->EditLogOutputStream.flush()
->EditLogOutputStream.flush(boolean durable)
->EditLogOutputStream.flushAndSync(boolean durable)
->JournalSet.flushAndSync(boolean durable)
->JournalSet.mapJournalsAndReportErrors(
      JournalClosure closure, String status)

The final calling function of this procedure is in the journalset

/**
   * Apply the given operation across all of the journal managers, disabling
   * any for which the closure throws an IOException.
   * @param closure {@link JournalClosure} object encapsulating the operation.
   * @param status message used for logging errors (e.g. "opening journal")
   * @throws IOException If the operation fails on all the journals.
   */
  private void mapJournalsAndReportErrors(
      JournalClosure closure, String status) throws IOException{

    List<JournalAndStream> badJAS = Lists.newLinkedList();
    for (JournalAndStream jas : journals) {
      try {
        closure.apply(jas);
      } catch (Throwable t) {
        if (jas.isRequired()) {
          final String msg = "Error: " + status + " failed for required journal ("
            + jas + ")";
          LOG.fatal(msg, t);
          // If we fail on *any* of the required journals, then we must not
          // continue on any of the other journals. Abort them to ensure that
          // retry behavior doesn't allow them to keep going in any way.
          abortAllJournals();
          // the current policy is to shutdown the NN on errors to shared edits
          // dir. There are many code paths to shared edits failures - syncs,
          // roll of edits etc. All of them go through this common function 
          // where the isRequired() check is made. Applying exit policy here 
          // to catch all code paths.
          terminate(1, msg);
        } else {
          LOG.error("Error: " + status + " failed for (journal " + jas + ")", t);
          badJAS.add(jas);          
        }
      }
    }
    disableAndReportErrorOnJournals(badJAS);
    if (!NameNodeResourcePolicy.areResourcesAvailable(journals,
        minimumRedundantJournals)) {
      String message = status + " failed for too many journals";
      LOG.error("Error: " + message);
      throw new IOException(message);
    }
  }

In mapjournalsandreporterrors, if the number of successful journals does not reach minimumredundantjournals, namenode will actively shut down.

Similar Posts: