com.aspc.job

Class JobCommand

java.lang.Object

com.aspc.commander.Command

com.aspc.commander.sql.SQLCommand

com.aspc.job.JobCommand

All Implemented Interfaces:

CommandBuildResultsPlugin

public class JobCommand
extends SQLCommand

The command JOB allows a series of commands to be scheduled as a job. The Schedule command will return a unique job id. With this job id repeated calls to JOB WAIT will return the result sets for the scheduled job in a horizontal result set format.

A job maybe SCHEDULE or EXECUTE either way a unique job_id with be created and the commands will be processed as a background task. The difference is that a SCHEDULED job is written to the database and are fault tolerant as it'll be rerun if the server falls over. A EXECUTEd job is only stored in memory ( unless notified by the memory manager of a low memory situation). The results of a EXECUTE job maybe cleared after being returned to the client or after a period of time ( client may have been closed). EXECUTEd jobs will be faster as they don't have the overhead of writing to the database. SCHEDULEd jobs are guaranteed to return a result ( not forgotten about even if the server or worker falls over)

The commander will be changed to be "memory handler". In low memory situations the commander will write the queued jobs that are not currently in a RUN state to the database and the held memory can now be release through the normal DBObject caching routines. These jobs will now be treated as a SCHEDULED job.

The WAIT command will return the current state of the job. If the job hasn't completed the command will wait for a specified duration. If the duration isn't specified we will WAIT for 1 second.

An ACL will prevent a USER from seeing the results of a job not created by them.

A Job Manager will only run a limited number of jobs concurrently. The Job Manager can be run from within the servlet engine ( JRun or Tomcat) or in a stand alone application on other machines. These independent Job Manager applications will have a per application configuration of the number of concurrent jobs to run and from which queues. A Job Manager will start each job in a background thread that runs the job will be run at a lower priority than the normal requests. The Job Manager will monitor each job that it starts and ( SCHEDULED jobs only) will update the Job record in the database every five minutes will a heart beat. If a job has a "Running" status and the time stamp of the last heart beat is more than ten minutes old the job will be declared DEAD as the monitoring Job Manager must of died. A Job Manager started on a separate machine could be configured to run jobs from a specific queue such as "REPORT", how all jobs SCHEDULED for the queue "REPORT" will be run on this separate machine leaving the real time user requests unaffected. When the Job Manager moves a job from a WAIT state to RUN state a race condition is prevented between Job Managers by the dirty cache check in the normal DBObject save routines.

The JOB command is runs the commands within a new mutable data source transaction. Any modified records in the calling transactions will not be viewable to the job command. This is different to normal commands which are normally run in an "auto commit off" mode and all changes are only viewable from within the session/transaction until the changes are committed.

Excel will be changed to schedule/queue long running commands such as save sheet and large batch retrievals and then repeatedly ask the status of the job. A progress bar will be shown with the current status of the job as returned by the JOB WAIT command. This prevents the problem of the soap call timing out after 15 minutes. Also if the process is taking longer than the user wishes they can press "cancel" which sends a JOB KILL command. Currently when a process takes a long time the user tends to close Excel and then retries the calculation, this actually worsens the problem. If the system is taken a long time to calculate one spread sheet submitting it two or three more times just makes it worst.

The command JOB KILL will kill a currently running Job via commander's normal kill routines. The command JOB PAUSE will pause the job at it's next command.

A job that is scheduled may write the raw commands and results to files not to the database as text fields. This will dramatically reduce the database over head in very large commands. The Job record has two derived fields "commands" and "results" that read the information from either the file or the raw text field.
Syntax

  JOB [ [ BEAT max_duration] [MAX_RUN_TIME max_run_time_duration] SCHEDULE
          [ CODE job_code]
          [ WAIT [SUCCESS] [wait_job_code[,...]] [ID wait_job_id[,...]] ]
          [ DISABLE_CANCELLATION disable_cancel]
          [ STATISTIC statistic_code[,...] ]
          [ PRIORITY priority_number ]
          [ RETRY retry_count ]
          [ AT job_manager ]
          [ QUEUE queue_name ]
          [ FILE file_name ] | [ command ]
          [
              ON [ERROR error_command]
                 [COMPLETE complete_command]
                 [RETRY retry_command]
                 [START start_command]
                 [WARNING warn_command]
          ]
      ] |
      [ [ BEAT max_duration] [MAX_RUN_TIME max_run_time_duration] EXECUTE
          [ CODE job_code]
          [ WAIT [SUCCESS] [wait_job_code[,...]] [ID wait_job_id[,...]] ]
          [ FILE file_name ] | [ command ]
          [
              ON [ERROR error_command]
                 [COMPLETE complete_command]
                 [START start_command]
          ]
      ] |
      [ STATUS job_id [BLOCK msecs]] |
      [ RESULT job_id ] |
      [ DATA job_id [COMPRESSION encoding]]   |
      [ KILL job_id ]   |
      [ CANCEL job_id ] |
      [ PAUSE job_id ]  |
      [ RESUME job_id ] |
      [ WAIT [SUCCESS] [[CODE] wait_job_code[,...]] [ID wait_job_id[,...]] [BLOCK msecs]] |
      [ MANAGER [[ENABLE]|[DISABLE]]|[QUEUE queue_name[,...]]] |
      [ QUEUE [START]|[SUSPEND] queue_name[,...]]
  

Usage

  /* Start a simple background job */
  JOB SCHEDULE {
     SELECT * FROM person;
     SLEEP 5;
     LOG WRITE 'done...';
  }

  /* Gets the status of JOB 7 in layer 5 */
  JOB STATUS 7@5;

  /* A JOB that the client must call JOB STATUS least every minute */
  JOB BEAT 1 minute EXECUE {sleep 2000};

  /* Get the status of a JOB and wait 5 seconds for it to complete */
  JOB STATUS 1@1 BLOCK 5000;

  /*
   * Return the XML results from a JOB in a non compressed format
   * The encoding values maybe NONE|DEFLATE|GZIP
   */
  JOB DATA 1@1 COMPRESSION none

  /**
   * Sechedule a JOB and retry it a number of times.
   */
  JOB SCHEDULE RETRY 10 {
     INSERT INTO opportunity(name) values('big bank');

  /* Wait 1 minute for a Job to finish before returning result */
  JOB WAIT ID 1@1 BLOCK 60000;
  }
  

THREAD MODE: SINGLE THREADED command

Since:

3 December 2003

Author:

Nigel Leck

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	JOB_EXECUTE the key to use when putting into the connection attributes
static java.lang.String	KEYWORD_ENABLE_QUEUE Start the job queue - for MANAGE_QUEUE
static java.lang.String	KEYWORD_QUEUE The QUEUE keyword
static java.lang.String	KEYWORD_SUCCESS JOB [ WAIT [SUCCESS] [ID wait_job_id] | [CODE wait_job_code]] throw an error if not successful.
static java.lang.String	KEYWORD_SUSPEND_QUEUE Suspend the job queue - for MANAGE_QUEUE
static java.lang.String	TYPE_CANCEL cancel a job
static java.lang.String	TYPE_DATA return the results for a job in a optimized format
static java.lang.String	TYPE_DISABLED Disable the job Manager to halt the process
static java.lang.String	TYPE_ENABLED Enable the job Manager to continue on process
static java.lang.String	TYPE_EXECUTE EXECUTE a Job
static java.lang.String	TYPE_KILL kill a job
static java.lang.String	TYPE_PAUSE pause a job
static java.lang.String	TYPE_QUEUE The MANAGE_QUEUE keyword QUEUE [ENABLE]|[SUSPEND] queue_name[,...]
static java.lang.String	TYPE_RESULT return the results for this job
static java.lang.String	TYPE_RESUME resume a job
static java.lang.String	TYPE_SCHEDULE SCHEDULE a JOB
static java.lang.String	TYPE_STATUS return the job status
static java.lang.String	TYPE_WAIT wait for a job
static java.lang.String	VARIABLE_DISABLE_CANCEL The job manager variable
static java.lang.String	VARIABLE_JOB_CODE The JOB code
static java.lang.String	VARIABLE_JOB_ID The ASCII version of the JOB id
static java.lang.String	VARIABLE_JOB_MANAGER The job manager variable
static java.lang.String	VARIABLE_MAX_RUN_TIME The max run time count variable
static java.lang.String	VARIABLE_PRIORITY_NUMBER The job manager variable
static java.lang.String	VARIABLE_QUEUE_NAME the queue code
static java.lang.String	VARIABLE_RETRY_COUNT The retry count variable
static java.lang.String	VARIABLE_WAIT_JOB_CODE The job code to wait for
static java.lang.String	VARIABLE_WAIT_JOB_ID The job id to wait for

Fields inherited from class com.aspc.commander.sql.SQLCommand

CURSOR_SIZE

Constructor Summary

Constructors

Constructor and Description
JobCommand(Connection connection, Pattern pattern) constructor for the job command

Method Summary

Modifier and Type	Method and Description
void	appendResult(org.w3c.dom.Element group, int seq) send back the job's results
protected void	execute(Parser p) Execute the JOB command.
java.lang.String	getColumnName(int col) The column name
java.lang.String	getColumnType(int col) The column type
java.lang.String	getRecordID(int row) The record ID for this row
java.lang.Object	getValue(int row, int col) The value for this row/col
static java.lang.String	makeAttributeKey(GlobalKey gk) make the key for EXECUTEd Jobs

Methods inherited from class com.aspc.commander.sql.SQLCommand

appendResult, getCursorName, getRecordKey, isExecuteOnly

Methods inherited from class com.aspc.commander.Command

execute, extMakeThreadName, getConnection, getPattern, getRawTitle, getTitle, makeThreadName, recordCall, setTitle

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.aspc.commander.sql.results.CommandBuildResultsPlugin

getRawTitle

Field Detail

TYPE_SCHEDULE

public static final java.lang.String TYPE_SCHEDULE

SCHEDULE a JOB

See Also:

Constant Field Values

TYPE_EXECUTE

public static final java.lang.String TYPE_EXECUTE

EXECUTE a Job

See Also:

Constant Field Values

TYPE_STATUS

public static final java.lang.String TYPE_STATUS

return the job status

See Also:

Constant Field Values

TYPE_RESULT

public static final java.lang.String TYPE_RESULT

return the results for this job

See Also:

Constant Field Values

TYPE_DATA

public static final java.lang.String TYPE_DATA

return the results for a job in a optimized format

See Also:

Constant Field Values

TYPE_KILL

public static final java.lang.String TYPE_KILL

kill a job

See Also:

Constant Field Values

TYPE_CANCEL

public static final java.lang.String TYPE_CANCEL

cancel a job

See Also:

Constant Field Values

TYPE_PAUSE

public static final java.lang.String TYPE_PAUSE

pause a job

See Also:

Constant Field Values

TYPE_RESUME

public static final java.lang.String TYPE_RESUME

resume a job

See Also:

Constant Field Values

TYPE_WAIT

public static final java.lang.String TYPE_WAIT

wait for a job

See Also:

Constant Field Values

TYPE_ENABLED

public static final java.lang.String TYPE_ENABLED

Enable the job Manager to continue on process

See Also:

Constant Field Values

TYPE_DISABLED

public static final java.lang.String TYPE_DISABLED

Disable the job Manager to halt the process

See Also:

Constant Field Values

VARIABLE_JOB_ID

public static final java.lang.String VARIABLE_JOB_ID

The ASCII version of the JOB id

See Also:

Constant Field Values

VARIABLE_JOB_CODE

public static final java.lang.String VARIABLE_JOB_CODE

The JOB code

See Also:

Constant Field Values

VARIABLE_QUEUE_NAME

public static final java.lang.String VARIABLE_QUEUE_NAME

the queue code

See Also:

Constant Field Values

VARIABLE_JOB_MANAGER

public static final java.lang.String VARIABLE_JOB_MANAGER

The job manager variable

See Also:

Constant Field Values

VARIABLE_PRIORITY_NUMBER

public static final java.lang.String VARIABLE_PRIORITY_NUMBER

The job manager variable

See Also:

Constant Field Values

VARIABLE_DISABLE_CANCEL

public static final java.lang.String VARIABLE_DISABLE_CANCEL

The job manager variable

See Also:

Constant Field Values

VARIABLE_RETRY_COUNT

public static final java.lang.String VARIABLE_RETRY_COUNT

The retry count variable

See Also:

Constant Field Values

VARIABLE_MAX_RUN_TIME

public static final java.lang.String VARIABLE_MAX_RUN_TIME

The max run time count variable

See Also:

Constant Field Values

VARIABLE_WAIT_JOB_CODE

public static final java.lang.String VARIABLE_WAIT_JOB_CODE

The job code to wait for

See Also:

Constant Field Values

VARIABLE_WAIT_JOB_ID

public static final java.lang.String VARIABLE_WAIT_JOB_ID

The job id to wait for

See Also:

Constant Field Values

KEYWORD_QUEUE

public static final java.lang.String KEYWORD_QUEUE

The QUEUE keyword

See Also:

Constant Field Values

KEYWORD_SUCCESS

public static final java.lang.String KEYWORD_SUCCESS

JOB [ WAIT [SUCCESS] [ID wait_job_id] | [CODE wait_job_code]] throw an error if not successful.

See Also:

Constant Field Values

JOB_EXECUTE

public static final java.lang.String JOB_EXECUTE

the key to use when putting into the connection attributes

See Also:

Constant Field Values

TYPE_QUEUE

public static final java.lang.String TYPE_QUEUE

The MANAGE_QUEUE keyword QUEUE [ENABLE]|[SUSPEND] queue_name[,...]

See Also:

Constant Field Values

KEYWORD_SUSPEND_QUEUE

public static final java.lang.String KEYWORD_SUSPEND_QUEUE

Suspend the job queue - for MANAGE_QUEUE

See Also:

Constant Field Values

KEYWORD_ENABLE_QUEUE

public static final java.lang.String KEYWORD_ENABLE_QUEUE

Start the job queue - for MANAGE_QUEUE

See Also:

Constant Field Values

Constructor Detail

JobCommand

public JobCommand(Connection connection,
                  Pattern pattern)

constructor for the job command

Parameters:

connection - The current connection

pattern - the command pattern

Method Detail

execute

protected void execute(Parser p)
                throws java.lang.Exception

Execute the JOB command.

Specified by:

execute in class Command

Parameters:

p - the parameters

Throws:

java.lang.Exception - a serious problem

appendResult

public final void appendResult(org.w3c.dom.Element group,
                               int seq)
                        throws java.lang.Exception

send back the job's results

Overrides:

appendResult in class SQLCommand

Parameters:

group - the result group

seq - the sequence

Throws:

java.lang.Exception - a serious problem

getColumnName

public java.lang.String getColumnName(int col)

The column name

Parameters:

col - offset from 1

Returns:

The name

getColumnType

public java.lang.String getColumnType(int col)
                               throws java.lang.Exception

The column type

Parameters:

col - offset from 1

Returns:

The column type

Throws:

java.lang.Exception - a serious problem

getRecordID

public java.lang.String getRecordID(int row)

The record ID for this row

Parameters:

row - offset from 1

Returns:

The record ID

getValue

public java.lang.Object getValue(int row,
                                 int col)
                          throws java.lang.Exception

The value for this row/col

Parameters:

row - offset from 1

col - offset from 1

Returns:

The value

Throws:

java.lang.Exception - a serious problem

makeAttributeKey

public static java.lang.String makeAttributeKey(GlobalKey gk)

make the key for EXECUTEd Jobs

Parameters:

gk - The job key

Returns:

The session key