Kubernetes Connector 1.0.0.0

The Kubernetes Connector lets you find, run, import, and delete Kubernetes jobs.

Note: Kubernetes jobs are defined in YAML format.

Note: Kubernetes job names must be unique. To avoid name collisions, use the YAML generateName attribute.

Prerequisites

  • RunMyJobs 9.2.9 or higher.
  • Connection Management Extension 1.0.0.4 or later. This Extension will be installed or updated automatically if necessary when you install the Kubernetes Connector.
  • Privileges Required to Use the Kubernetes Connector.
  • A Kubernetes API Server that can be reached via outbound HTTP from the Secure Gateway (for SaaS customers) or the RunMyJobs Scheduler (for on-site installations).
  • Token or Client Certificate authorization must be configured on the Kubernetes API Server.

Contents of the Component

The Kubernetes Connector consists of the following objects.

Object Type Name Description
Application GLOBAL.Redwood.REDWOOD.Kubernetes Integration connector with a Kubernetes cluster
Constraint Definition REDWOOD.Redwood_KubernetesConnectionConstraint Constraint for Kubernetes Connection fields
Constraint Definition REDWOOD.Redwood_KubernetesNamespaceConstraint Constraint for Kubernetes Namespaces fields
Extension Point REDWOOD.Redwood_KubernetesConnection Kubernetes Connection
Process Definition REDWOOD.Redwood_Kubernetes_DeleteJobs Delete one or more Kubernetes Jobs
Process Definition REDWOOD.Redwood_Kubernetes_ImportJob Import a job from Kubernetes
Process Definition REDWOOD.Redwood_Kubernetes_RunExistingJob Run an existing job in Kubernetes
Process Definition REDWOOD.Redwood_Kubernetes_RunExistingJob_Template Template definition to run an existing job in Kubernetes
Process Definition REDWOOD.Redwood_Kubernetes_RunJob Run a job in Kubernetes
Process Definition REDWOOD.Redwood_Kubernetes_RunJob_Template Template definition to run a job in Kubernetes
Process Definition REDWOOD.Redwood_Kubernetes_ShowJobs List all existing jobs in a Kubernetes cluster
Process Definition Type REDWOOD.Redwood_Kubernetes Kubernetes Connector
Library REDWOOD.Redwood_Kubernetes Library for Kubernetes connector

Redwood_Kubernetes_DeleteJobs

Deletes one or more Kubernetes Jobs.

Parameters

Tab Name Description Documentation Data Type Direction Default Expression Values
Parameters connection Connection The Connection object that defines the connection to the Kubernetes cluster. String In

Parameters namespace Namespace The Namespace at the Kubernetes cluster. String In

Parameters nameFilter Job Name Filter Name filter to specify the jobs to delete. Provide an exact name, or wildcards * and ? are supported. String In

Parameters olderThan Older Than (Days) Only delete jobs that are older than (by completion date/time) the specified number of days. Number In 10

Parameters propagationPolicy Propagation Policy How to handle dependent objects. Orphan: Orphan the dependents. Background: The garbage collector will delete the dependents in the background. Foreground: Delete all dependents in the foreground. String In Background Background, Foreground, Orphan

Redwood_Kubernetes_ImportJob

Imports one or more jobs from Kubernetes.

Parameters

Tab Name Description Documentation Data Type Direction Default Expression Values
Parameters connection Connection The Connection object that defines the connection to the Kubernetes cluster. String In

Parameters namespace Namespace The Namespace at the Kubernetes cluster. String In

Parameters filter Job Name Filter This filter can be used to limit the amount of jobs returned to those which name matches the filter. Valid regular expressions are supported in this field. String In

Parameters overwrite Overwrite Existing Definition When set to Yes, if a definition already exists with the same name as the name generated for the imported object, it will be overwritten with the new import.
When set to No, the import for that template will be skipped if a definition with the same name already exists.		 String In N Y, N
Generation Settings targetPartition Partition The Partition to create the new definitions in. String In

QueryFilter:User.Redwood System.Partition.Partition%2e;all
Generation Settings targetApplication Application The Application to create the new definitions in. String In

QueryFilter:User.Redwood System.Application.Application%2e;all
Generation Settings targetQueue Default Queue The default Queue to assign to the generated definitions. String In

QueryFilter:User.Redwood System.Queue.Queue%2e;all
Generation Settings targetPrefix Definition Name Prefix The prefix to add onto the name of the imported Kubernetes Job to create the definition name. String In CUS_K8S_

Redwood_Kubernetes_RunExistingJob

Runs an existing job in Kubernetes.

Parameters

Tab Name Description Documentation Data Type Direction Default Expression Values
Parameters connection Connection The Connection object that defines the connection to the Kubernetes cluster. String In

Parameters namespace Namespace Namespace to run the job in String In

Parameters name Job name to run The name of the Kubernetes job to re-run. This job must already exist on the server. String In

Parameters downloadLogs Download Logs Download log files from the containers that ran as part of this job and attach them as job files. String In No No, Failed Pods Only, All
Parameters controllerUid Kubernetes Controller UID The Controller UID of the executed Job on the Kubernetes side String Out

Redwood_Kubernetes_RunExistingJob_Template

Template definition to run an existing job in Kubernetes.

Parameters

Tab Name Description Documentation Data Type Direction Default Expression Values
Parameters connection Connection The Connection object that defines the connection to the Kubernetes cluster. String In

Parameters namespace Namespace Namespace to run the job in String In

Parameters name Job name to run The name of the Kubernetes job to re-run. This job must already exist on the server. String In

Parameters downloadLogs Download Logs Download log files from the containers that ran as part of this job and attach them as job files. String In No No, Failed Pods Only, All
Parameters controllerUid Kubernetes Controller UID The Controller UID of the executed Job on the Kubernetes side String Out

Redwood_Kubernetes_RunJob

Runs a job in Kubernetes.

Parameters

Tab Name Description Documentation Data Type Direction Default Expression Values
Parameters connection Connection The Connection object that defines the connection to the Kubernetes cluster. String In

Parameters namespace Namespace Namespace to run the job in String In

Parameters yamlFile YAML File The YAML file defining the Job Properties File In

Parameters downloadLogs Download Logs Download log files from the containers that ran as part of this job and attach them as job files. String In No No, Failed Pods Only, All
Parameters jobName Kubernetes Job Name The Name of the executed Job on the Kubernetes side String Out

Parameters controllerUid Kubernetes Controller UID The Controller UID of the executed Job on the Kubernetes side String Out

Redwood_Kubernetes_RunJob_Template

Template definition to run a job in Kubernetes.

Parameters

Tab Name Description Documentation Data Type Direction Default Expression Values
Parameters connection Connection The Connection object that defines the connection to the Kubernetes cluster. String In

Parameters namespace Namespace Namespace to run the job in String In

Parameters downloadLogs Download Logs Download log files from the containers that ran as part of this job and attach them as job files. String In No No, Failed Pods Only, All
Parameters jobName Kubernetes Job Name The Name of the executed Job on the Kubernetes side String Out

Parameters controllerUid Kubernetes Controller UID The Controller UID of the executed Job on the Kubernetes side String Out

Redwood_Kubernetes_ShowJobs

Lists all existing jobs in a Kubernetes cluster.

Parameters

Tab Name Description Documentation Data Type Direction
Parameters connection Connection The Connection object that defines the connection to the Kubernetes cluster. String In
Parameters filter Job Name Filter This filter can be used to limit the amount of jobs returned to those which name matches the filter. Wildcards * and ? are supported String In
Parameters namespace Namespace The Namespace at the Kubernetes cluster. String In
Parameters listing Job listing The listing of all jobs available that match the input filter (or any if no input filter was provided) Table Out

Setup

  1. Locate the Kubernetes component in the Catalog and install it.
  2. Navigate to Custom > Connections.
  3. Click .

  4. Click the Kubernetes connection type.

  5. Click Next or Basic Properties, then create a Queue and Process Server for the connector. All required settings will be configured automatically.

  6. Click Next or Security, then click to specify which roles can access the connection information. It is recommended to grant the role at least the following additional privileges: View on the Kubernetes Connector Process Server, View Processes on the Kubernetes Connector Jobs Queue, View on library REDWOOD.Redwood_EnterpriseOne, and Submit on any Process Definitions that users with this role will submit.
  7. Click Next or Kubernetes Connection Properties. You have two options for authenticating with Kubernetes.

    • Kubernetes Certificate. Enter the URL for your Kubernetes instance. Use the Test Connection button to make sure the connection is working.

      Note: If you choose this option, you will need to create an X509_TrustedCertificate credential.

    • Kubernetes Token Authentication. If you choose this option, enter the URL and Access Token for your Kubernetes API Server.

  8. Click Save & Close.
  9. Navigate to Environment > Process Server, locate your Kubernetes Connector Process Server, start it, and make sure it reaches status Running.

Running a Kubernetes Job from a YAML File

To run a Kubernetes job from a YAML file:

  1. In the Kubernetes Application, submit Redwood_Kubernetes_RunJob.

  2. Choose the Connection.

  3. Choose a Namespace.

  4. Click Upload next to the YAML File field and upload the YAML file that defines the job you want to run.

  5. Choose an option from the Download Logs drop-down list.

  6. Submit the Process Definition.

Finding Existing Kubernetes Jobs

To retrieve the list of existing Kubernetes jobs (that is, Kubernetes jobs that have already been submitted at least once):

  1. Navigate to Applications > Redwood > Kubernetes, and submit Redwood_Kubernetes_ShowJobs.

  2. Choose the Connection.

  3. To specify a search string, enter a value in the Job Name Filter field. Wildcards * and ? are supported.

  4. Choose a Namespace.

  5. Once the process has finished, right-click it and choose Monitor Related Processes, then select the process run and click listing.rtx in the Detail View. Sample output is shown below.

Running an Existing Kubernetes Job

To run a Kubernetes job that has already run:

  1. In the Kubernetes Application, submit Redwood_Kubernetes_RunExistingJob.

  2. Choose the Connection.

  3. Choose a Namespace.

  4. Enter the name of the job you want to run.

  5. Choose an option from the Download Logs drop-down list.

  6. Submit the Process Definition.

Running a Kubernetes Job with a Template

To create a customized Process Definition, optionally with default values, for a Kubernetes job defined in a YAML file:

  1. Right-click the Redwood_Kubernetes_RunExistingJob_Template Process Definition and choose New (from Template) from the context menu. The New Process Definition pop-up window displays.

  2. Choose a Partition.

  3. Enter a Name.

  4. Delete the default Application value (if any) and substitute your own Application name if desired.

  5. Paste the YAML that defines the job you want to run in the Source field.

    Note: If you want to use custom parameters in your YAML script, insert them using the format ${parameterName}.

  6. In the Parameters tab, enter any Default Expressions you want to use.

    • When specifying the Connection value, use the format EXTCONNECTION:<partition>.<connection name>.

  7. If you included parameters in your YAML, create an additional parameter in the Parameters tab for each of them. For example, if you used ${jobName} in the YAML, create a parameter named jobName (and optionally specify a Default Expression). Supported parameter types are String and Number.

    Note: If you use a parameter in the YAML, but you do not create a corresponding parameter in the Parameters tab, the parameter reference will be submitted as-is (for example, as "${jobName}"). If you use a parameter in the YAML, but do not specify a value for it, it will be submitted as an empty string.

    There are no specific requirements for parameter groups.

  8. Save and then submit the new Process Definition.

Running an Existing Kubernetes Job with a Template

To create a customized Process Definition, optionally with default values, for an existing Kubernetes job:

  1. Right-click the Redwood_Kubernetes_RunExistingJob_Template Process Definition and choose New (from Template) from the context menu. The New Process Definition pop-up window displays.

  2. Choose a Partition.

  3. Enter a Name.

  4. Delete the default Application value (if any) and substitute your own Application name if desired.

  5. In the Parameters tab, enter any Default Expressions you want to use.

    • When specifying the Connection value, use the format EXTCONNECTION:<partition>.<connection name>.

  6. Save and then submit the new Process Definition.

Importing a Kubernetes Job

To import an existing Kubernetes job (that is, a Kubernetes job that has already been submitted at least once):

  1. Navigate to Applications > Redwood > Kubernetes, and submit Redwood_Kubernetes_ImportJob.

  2. On the Parameters tab, do this:

    1. Choose the Connection.

    2. Choose a Namespace.

    3. To specify a search string for the job name, enter a value in the Job Name Filter field. Regular expressions are supported.

    4. Choose an option from the Overwrite Existing Definition drop-down list.

  3. On the Generation Settings tab, do this:

    1. Optionally specify a Partition, Application, and/or Default Queue.

    2. In the Definition Name Prefix field, enter a prefix to add onto the name of the imported Kubernetes Job when creating the name of the Process Definition.

  4. Click Submit.

Deleting Kubernetes Jobs

To delete existing Kubernetes jobs (that is, Kubernetes jobs that have already been submitted at least once) from the Kubernetes instance:

  1. Navigate to Applications > Redwood > Kubernetes, and submit Redwood_Kubernetes_DeleteJobs.

  2. On the Parameters tab, do this:

    1. Choose the Connection.

    2. Choose a Namespace.

    3. To specify a search string for the job name, enter a value in the Job Name Filter field. Wildcards * and ? are supported.

    4. To delete only jobs that are older than a given number of days, enter that number in the Older Than (Days) field.

    5. To specify how to handle dependent objects for the job(s), choose an option from the Propagation Policy drop-down list.

      • Orphan: Orphan the dependents.

      • Background: The garbage collector will delete the dependents in the background.

      • Foreground: Delete all dependents in the foreground.

  3. Click Submit.

Determining Final Job Status

RunMyJobs monitors the status of a Kubernetes job by watching the job's active count. If the active count is not updated for some reason, RunMyJobs fetches the status of each pod and uses it to determine whether any pods are still active.

Once a Kubernetes job has completed, RunMyJobs determines the final status of the process that ran it based on the following criteria.

  • If the Kubernetes job’s status conditions are present and the Succeeded condition is true, the process goes into Completed status. Otherwise the process goes into Error status.

  • If the Kubernetes job's status conditions are not present, RunMyJobs considers the final status of its pods. If any pods have Failed or Unknown status, the RunMyJobs process goes into Error status. If all pods reach Succeeded status, the RunMyJobs process goes into Completed status.