Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.0.0.M2
    • Fix Version/s: None
    • Component/s: MR
    • Labels:
      None

      Description

      It seems that there is a classloader leak caused by the LeaseChecker thread(used by DFSClient). It's unclear at this point what's the code path, but it seems for each Job submitted, a LeaseChecker thread leaks causing the context class loader to remain in memory - meaning new classloaders created for job creation leak as well.

      Some sort of cleanup mechanism (though ideally this should be in Hadoop) needs to be put in place, to prevent this from happening.

        Activity

        Hide
        Costin Leau added a comment -

        Some more info - the daemon is started only when something writes to the FS. This has some unexpected effects as the thread will inherit the context class loader of the running thread. However there should be only one DFSClient which does not seems to be the case in the forum post.

        Show
        Costin Leau added a comment - Some more info - the daemon is started only when something writes to the FS. This has some unexpected effects as the thread will inherit the context class loader of the running thread. However there should be only one DFSClient which does not seems to be the case in the forum post.
        Show
        Costin Leau added a comment - See https://issues.apache.org/jira/browse/HADOOP-8632 and https://issues.apache.org/jira/browse/HADOOP-8605
        Hide
        Costin Leau added a comment -

        HADOOP-8632 has been fixed and hopefully the updates will be pushed upstream.

        Show
        Costin Leau added a comment - HADOOP-8632 has been fixed and hopefully the updates will be pushed upstream.

          People

          • Assignee:
            Costin Leau
            Reporter:
            Costin Leau
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated: