Uploaded image for project: 'Spring XD'
  1. Spring XD
  2. XD-3079

Create a new Kerberos ticket instead of renew the current one

    XMLWordPrintable

    Details

    • Story Points:
      5
    • Rank (Obsolete):
      9223372036854775807
    • Acceptance Criteria:
      Hide

      Spring XD should be able to generate a new Kerberos ticket when start running a new job instead of renew the current one, because the current one could be expired or the renew time could be finished.

      Show
      Spring XD should be able to generate a new Kerberos ticket when start running a new job instead of renew the current one, because the current one could be expired or the renew time could be finished.
    • Sprint:
      Sprint 50

      Description

      Running Spring-XD singlenode with a kerberized hadoop cluster on CDH 5.3.2. with JDK 1.7 and JCE 1.7.
      The kerberos ticket policies are:

      • expiration: 24 hours
      • renew: 7 days

      I need to keep the Spring XD server running constantly because my flows are always waiting for incoming files to be ingested into the HDFS, but the kerberos session expires if there aren't jobs to run before the expiration date. The expiration policies can't be changed due internal company policies.

      Is there a way which Spring XD can generate a new ticket instead of renew the current one when a job or stream start executing?

      The Spring XD server has configured the hadoop.properties like:

      1. Use servers.yml to change URI for namenode
      2. You can add additional properties in this file
        dfs.namenode.kerberos.principal=hdfs/[email protected]
        yarn.resourcemanager.principal=yarn/[email protected]

      yarn.application.classpath=/opt/cloudera/parcels/CDH/lib/hadoop/,/opt/cloudera/parcels/CDH/lib/hadoop/lib/,/opt/cloudera/parcels/CDH/lib/hadoop-hdfs/,/opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/,/opt/cloudera/parcels/CDH/lib/hadoop-yarn/,/opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/,/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/,/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/

      hadoop.security.authorization=true
      hadoop.security.authentication=kerberos

      spring.hadoop.userKeytab=file:///export/home/user/user.keytab
      [email protected]

      #Connecting to Kerberized Hadoop (Spring XD doc configuration Appendix D)
      spring.hadoop.security.authMethod=kerberos
      spring.hadoop.security.userKeytab=/export/home/user/user.keytab
      [email protected]
      spring.hadoop.security.namenodePrincipal=hdfs/[email protected]
      spring.hadoop.security.rmManagerPrincipal=yarn/[email protected]

        Attachments

        1. kerberos.png
          kerberos.png
          173 kB
        2. kerberos2.png
          kerberos2.png
          128 kB

          Activity

            People

            Assignee:
            jvalkeal Janne Valkealahti
            Reporter:
            cgiha Cristian Giha
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: