Keep Clauses and Retention Strategies

Process Definitions and Chain Definitions that have already executed leave behind process objects, generally referred to as processes. You can view these processes by navigating to Monitoring > Processes. This displays the Processes tab, which is commonly referred to as the Process Monitor.

It is generally a good idea to delete old processes to prevent overloading your system. A keep clause defines how many processes are kept in the system and/or how long processes will be kept in the system. You can retain processes per parameter value combination, system, user, or date. You can further limit processes retention by status. The default setting is to keep all processes.

You can specify a keep clause for a specific Process Definition or Chain Definition using the Retention tab of the Process Definition pop-up window. In addition, you can set the DefaultKeepStatus registry key to define a keep clause for all objects with a particular final status.

Your retention strategy is the way you set up your keep clauses.

Warning: Incorrect or inefficient retention strategies can cause process accumulation, which can lead to poor performance and/or file system saturation. Contact Redwood Support Services if you are in doubt about the efficiency of your retention strategy.

Process Deletion

Processes are not deleted immediately. Rather, they are deleted by system processes that run in the background. Process objects are deleted by System_ProcessKeepClauses Process Definition, and output files are removed by the System_DeleteJobFiles Process Definition. By default, these Process Definitions are executed every 15 minutes.

These Process Definitions run according to the built-in Submit Frames System_ProcessKeepClausesSubmitFrame and System_DeleteJobFilesSubmitFrame. You can modify these Submit Frames to change the interval at which the Process Definitions are executed.

Tip: Because these are maintenance processes, they are not displayed in the default All (non-maintenance) processes filter. If you need to monitor these processes, use the Maintenance (background) processes filter.

Partition-wide and System-wide Keep Clause

You can set Partition-wide and system-wide keep clause defaults by creating or editing <Partition>.System_Defaults_Partition and/or GLOBAL.System_Defaults_System Process Definitions. These Process Definitions are also used to configure default restart behavior. For more information, see Controlling Global and Partition Restart Behavior.

Tip: You can set a default keep clause for all processes to which no other keep clause applies. To do so, use the defaultjobretention group of registry keys.

DefaultKeepStatus Registry Key

Processes that reach one of statuses defined by the /configuration/DefaultKeepStatus registry key are excluded from the keep clause of Process and Chain Definitions. These processes are kept indefinitely.

Set this registry entry to a series of the following codes, in any order, without any separators (for example: KAUE). Processes that have been excluded from the keep clause this way are kept forever.

  • A - Canceled
  • C - Completed
  • E - Error
  • K - Killed
  • U - Unknown

KeepExpiration Field on the Job Table (Model)

You can set the Job.KeepExpiration field of a process in the database to override the keep clause specified on the Process Definition for that process.

Retention Warnings

A warning displays if (a) 80% of non-system Process Definitions do not have a keep clause set, and (b) the system reaches either the maximum number of processes (100,000 by default) or the maximum number of new processes within 24 hours(10,000 by default). You can specify your own criteria using the retention registry entries.