Versions Compared

Version Old Version 10 New Version 11
Changes made by

Alex

Alex

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. create "ITaskHandler" interface with "handleTaskRun(kDBItem $task_run)" method - 0.5h
  2. create "TaskQueue" database table (unit name "task-queue") with following columns: - 1h
    • Id
    • TaskHandlerClass - FQCN of PHP class, which is responsible for processing this task queue record (must implement "ITaskHandler" interface)
    • TaskData - JSON-encoded data, that is needed for task execution (e.g. e-mail recipient, IDs of records to be processed)
    • ScheduledOn - when task needs to be processed (set at queuing time)
    • QueuedOn - when task was queued
    • QueuedById - who queued the task
    • LastStatus - status from last attempt of this queue record processing (same options as for "TaskRuns.Status" field)
    • MaxRetries - if task fails specified number of times (5 by default), then don't retry it
    • TaskRunsFailed - number failed task runs (number is reset, when task execution was successful)
  3. create "TaskRuns" database table (unit name: "task-run") with following columns: - 1h
    • Id
    • TaskQueueId - ID of record from "TaskQueue" table, that is responsible for creation this run
    • StartedOn - when task run was started; NULL set to moment, when status changes from "scheduled" to "processing"
    • PercentsCompleted - "0" by default, but will be updated as task is being processed
    • FinishedOn - when task was finished executing (regardless of status)
    • Results - JSON-encoded results in any format, that be later displayed in human-readable form
    • Status - same statuses as for "TaskQueue.LastStatus" column
      • "scheduled" - initially, when record is created;
      • "running" - when somebody is processing the record;
      • "success" - when execution finished without errors;
      • "error" - when known error happened during processing;
      • "timeout" - when associated task runner died unexpectedly
    • StandardOutput - what was written to "stdout" stream during this task run
    • ErrorOutput - what was written to "stderr" stream during this task run
    • ErrorCode - non-empty when known error happened
    • ErrorMessage - non-empty when known error happened
    • TaskRunnerId - NULL by default; ID of task runner that processing/processed given task run
  4. create "TaskRunners" database table (unit name: "task-runner") with following columns: - 1h
    1. Id
    2. ProcessId - the PID of process, that started/created task runner
    3. StartedOn - when process was started
    4. FinishedOn - when process was finished; NULL initially
    5. Status - status of task runner:
      1. "running" - default; means task runner is running
      2. "success" - set, when task runner decides to kill itself
      3. "timeout" - set by overseer when task runner in "running" status and associated process isn't running
  5. in "task-run:OnAfterItemUpdate" event will: - 0.5h
    1. load "task-queue" object associated with updated task run
    2. get all "task-run" records for that "task-queue" (via sql); then sort them from recent to old (via php)
    3. set following fields on "task-queue" object:
      1. "LastStatus" to "Status" of most recent "task-run"
      2. "TaskRunsFailed" to count of "task-run" records in "error" and "timeout" statuses (if last run is failed)
      3. "TaskRunsFailed" to "0" (if last run was successful)
  6. in "task-runner::OnBeforeItemCreate" event set "ProcessId" to PID of current process - 0.1h
  7. in "task-runner:OnAfterItemUpdate" event, when "Status" changes from "running" to "timeout" set all "task-run" status, that are processed by this task runner from "running" to "timeout" as well - 0.3h
  8. the "task-run" would be sub-item of "task-queue" unit - 0.1h

...

  1. add "declare(ticks = 1);" on top of "/tools/run_event.php" file - 0.1h
  2. create new "TaskRunnerLimit" system setting set to "8" by default - 0.2h
  3. create "TaskRunner" class with: - 0.2h
    1. add "TaskRunner::taskRunnerId" property
    2. add "TaskRunner::lastSignal" property
  4. add protected "TaskRunner::signalHandler" method, that will store received signal in the "TaskRunner::lastSignal" property - 0.5h
  5. add "TaskRunner::__construct($task_runner_id)" method, that will: - 0.5h
    1. store given "$task_runner_id" into "TaskRunner:taskRunnerId" property
    2. if executed from CLI (PHP_SAPI constant check), then use "pcntl_signal" function to register "TaskRunner::signalHandler" method as signal listeners for following signals:
      1. SIGINT
      2. SIGTERM
      3. SIGKILL
      4. SIGHUP
  6. add public "TaskQueue::getTaskRunner()" method, that will: - 0.5h
    1. call "TaskQueue::getMissingTaskRunnerCount" method
    2. if method returned "0", then return "null"
    3. create new "task-runner" object
    4. return instance of "TaskRunner" class initialized with ID of just created task runner
  7. add protected "TaskRunner::getNextTaskRunId()" method, that will: - 0.5h
    1. acquire WRITE lock "TaskRuns" database table (solves racing condition in parallel environment)
    2. pick 1st available "task-run" in "scheduled" status (FIFO logic)
    3. release above acquired lock
    4. return found task run id or "null" when nothing was found
  8. add protected "TaskRunner::processTaskRun($task_run_id)", that will: - 1h
    1. load "task-run" by given ID from the database or throw an exception if wasn't found
    2. if given "task-run" isn't in "scheduled" status, then throw an exception
    3. set following fields and save changes to db immediately:
      1. "TaskRunnerId"  to value of "TaskRunner::taskRunnerId" property
      2. "Status" to "processing"
    4. create task handler by calling "TaskQueue::createTaskHandler" method
    5. enable redirection of "stdout" and "stderr" into temp files
    6. call the "handleTaskRun" (wrapped within try/catch block) on that object providing task run object (was loaded above) as an argument
    7. store contents of above temp files into "StandardOutput" and "ErrorOutput" fields of "task-run" object
    8. the above method can update given object fields at will and save to db (e.g. "PercentsCompleted" and "Results")
    9. when exception was caught, then:
      1. set "Status" to "error"
      2. set "ErrorCode" to exception code
      3. set "ErrorMessage" to exception message
    10. when no exception was caught, then:
      1. set "Status" to "success"
      2. set "ErrorCode" and "ErrorMessage" to empty value
    11. set "FinishedOn" to time, when task was finished (with error or not)
    1. save changes to db
  9. add public "TaskRunner::process()" method, that will consist of while loop, where each iteration will: - 0.5h
    1. call "TaskRunner::getNextTaskRunId" method
    2. call "TaskRunner::processTaskRun" with ID found above (if ID was found)
    3. in either of following cases set "FinishedOn" to NOW() on associated "task-runner" record and exit
      1. "TaskRunner::lastSignal" is set
      2. overall memory consumption is more than 100MB
      3. it's not CLI mode
    4. sleep for X of seconds
  10. add "task-runner:OnProcess" event, that will: - 0.5h
    1. call the "TaskQueue::getTaskRunner" method
    2. if an object is returned, then call "->process()" method on it

...