Uploaded image for project: 'Spring XD'
  1. Spring XD
  2. XD-3079

Create a new Kerberos ticket instead of renew the current one

This issue belongs to an archived project. You can view it, but you can't modify it. Learn more

    XMLWordPrintable

Details

    • 5
    • 9223372036854775807
    • Hide

      Spring XD should be able to generate a new Kerberos ticket when start running a new job instead of renew the current one, because the current one could be expired or the renew time could be finished.

      Show
      Spring XD should be able to generate a new Kerberos ticket when start running a new job instead of renew the current one, because the current one could be expired or the renew time could be finished.
    • Sprint 50

    Description

      Running Spring-XD singlenode with a kerberized hadoop cluster on CDH 5.3.2. with JDK 1.7 and JCE 1.7.
      The kerberos ticket policies are:

      • expiration: 24 hours
      • renew: 7 days

      I need to keep the Spring XD server running constantly because my flows are always waiting for incoming files to be ingested into the HDFS, but the kerberos session expires if there aren't jobs to run before the expiration date. The expiration policies can't be changed due internal company policies.

      Is there a way which Spring XD can generate a new ticket instead of renew the current one when a job or stream start executing?

      The Spring XD server has configured the hadoop.properties like:

      1. Use servers.yml to change URI for namenode
      2. You can add additional properties in this file
        dfs.namenode.kerberos.principal=hdfs/[email protected]
        yarn.resourcemanager.principal=yarn/[email protected]

      yarn.application.classpath=/opt/cloudera/parcels/CDH/lib/hadoop/,/opt/cloudera/parcels/CDH/lib/hadoop/lib/,/opt/cloudera/parcels/CDH/lib/hadoop-hdfs/,/opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/,/opt/cloudera/parcels/CDH/lib/hadoop-yarn/,/opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/,/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/,/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/

      hadoop.security.authorization=true
      hadoop.security.authentication=kerberos

      spring.hadoop.userKeytab=file:///export/home/user/user.keytab
      [email protected]

      #Connecting to Kerberized Hadoop (Spring XD doc configuration Appendix D)
      spring.hadoop.security.authMethod=kerberos
      spring.hadoop.security.userKeytab=/export/home/user/user.keytab
      [email protected]
      spring.hadoop.security.namenodePrincipal=hdfs/[email protected]
      spring.hadoop.security.rmManagerPrincipal=yarn/[email protected]

      Attachments

        1. kerberos.png
          kerberos.png
          173 kB
        2. kerberos2.png
          kerberos2.png
          128 kB

        Activity

          People

            jvalkeal Janne Valkealahti
            cgiha Cristian Giha
            Archiver:
            tmarshall Trevor Marshall

            Dates

              Created:
              Updated:
              Resolved:
              Archived: