Difference between revisions of "MCPServer"

From Archivematica
Jump to navigation Jump to search
(Creating MCP Page.)
 
(Start of MCP documentation.)
Line 1: Line 1:
 
[[Main Page]] > [[Development]] > [[:Category:Development documentation|Development documentation]] > MCP
 
[[Main Page]] > [[Development]] > [[:Category:Development documentation|Development documentation]] > MCP
 +
 +
 +
<div class="status">
 +
<div>
 +
Design
 +
<div class="description">
 +
This page proposes a new feature and reviews design options
 +
</div>
 +
</div><div class="active">
 +
Development
 +
<div class="description">
 +
This page describes a feature that's in development
 +
</div>
 +
</div><div>
 +
Documentation
 +
<div class="description">
 +
This page documents an implemented feature
 +
</div>
 +
</div>
 +
</div>
 +
 +
==Overview==
 +
The MCP is the Archivematica [[micro-services]] tool to control flow in the Archivematica system. It "knows" what things need to be done, who can do them, what is currently being processed across the distributed system, and is responsible for distributing the work. The user controls and monitors the MCP via the [[ dashboard ]]. The MCP maintains a log of all completed work.
 +
 +
The MCP is a client server based architecture. The clients are relatively "dumb". They inform the server what tasks they can perform, and wait for the server to assign them a task.
 +
 +
The system relies on client and server having access to the same directory, to process the commands.
 +
 +
==Server==
 +
The server is the core of the MCP. It uses a set of modules, created at run time based on configurations (mcpModulesConfigs). These configs specify a directory to watch, and a series of commands to execute on anything placed in the directory.
 +
 +
===Server Implementation===
 +
As stated above, the MCP watches directories specified in the modules, and has a set of commands to issue on items placed in those folders. Those commands will not be performed by the MCP server itself, but rather delegated to a client.
 +
 +
===mcp Modules===
 +
The mcp Modules are XML based, and contain a number of fields:
 +
<module>
 +
Wait for user approval, before creating and assigning tasks.
 +
<requiresUserApproval>Yes/No</requiresUserApproval>
 +
Description to give user to approve Job.(not implemented yet)
 +
<descriptionForApproval></descriptionForApproval>
 +
<notificationStarted></notificationStarted>
 +
<notificationCompletedWithoutErrors></notificationCompletedWithoutErrors>
 +
<notificationCompletedWithErrors></notificationCompletedWithErrors>
 +
 +
 +
<directories>
 +
Directory to watch for folders moved to.
 +
  <watchDirectory>%watchDirectoryPath%appraiseSIP</watchDirectory>
 +
Standard directory to move to while the clients are performing tasks. Defined in [[ archivematica.conf ]].
 +
  <processingDirectory>%processingDirectory%</processingDirectory>
 +
The output directory is determined by the return value of the commands, and the corresponding config folder.
 +
Chaining output folders and watch directories allows for flow through the system.
 +
  <successDirectory>%watchDirectoryPath%...</successDirectory>
 +
  <failureDirectory>%watchDirectoryPath%failed/</failureDirectory>
 +
</directories>
 +
 +
 +
<commands>
 +
A command to execute on each file or folder, the details of which are described below in 'mcp Modules Command'.
 +
The exe command is the command to run on the Event.
 +
<exeCommand>
 +
The verification command is used if the return of the exeCommand is not reliable to determin the result of the command. (IE virus scan exits zero and logs there is a virus)
 +
<verificationCommand>
 +
Cleanup allows for some post processing before the Event folder is moved to the success or fail directory.
 +
<cleanupSuccessfulCommand>
 +
<cleanupUnsuccessfulCommand>
 +
</commands>
 +
</module>
 +
 +
 +
====mcp Modules Command====
 +
Each command consists of a number of parts.
 +
<command>
 +
 +
<descriptionWhileExecuting> </descriptionWhileExecuting>
 +
To skip this command, and not execute it at all.
 +
<skip>Yes/No</skip>
 +
Filter what files/folder the command will operate on.
 +
<filterFileEnd></filterFileEnd>
 +
<filterFileStart></filterFileStart>
 +
<filterSubDir></filterSubDir>
 +
If the output of the command is all going to the same file, multiple threads may try to write to the same file simultaneously and cause a collision. Set this option to stop that occurrence.
 +
<requiresOutputLock>No</requiresOutputLock>
 +
Not used - placeholder
 +
<standardIn></standardIn>
 +
File to write standard out to.
 +
<standardOut></standardOut>
 +
File to write standard error to.
 +
<standardError></standardError>
 +
 +
<failureNotification> </failureNotification>
 +
Define the command the client is to execute. This will need to match an entry in the client's supported modules defined int [[ archivematicaClientConfig ]]
 +
<execute> </execute>
 +
The arguments to give the command.
 +
<arguments> </arguments>
 +
Does this command execute once on the SIP, or once for each file?
 +
<executeOnEachFile>yes/no</executeOnEachFile>
 +
</command>
 +
 +
==Client==
 +
===Client Requirements===
 +
===Client Implementation===
 +
[[Category:Development documentation]]

Revision as of 15:25, 4 October 2010

Main Page > Development > Development documentation > MCP


Design

This page proposes a new feature and reviews design options

Development

This page describes a feature that's in development

Documentation

This page documents an implemented feature

Overview

The MCP is the Archivematica micro-services tool to control flow in the Archivematica system. It "knows" what things need to be done, who can do them, what is currently being processed across the distributed system, and is responsible for distributing the work. The user controls and monitors the MCP via the dashboard . The MCP maintains a log of all completed work.

The MCP is a client server based architecture. The clients are relatively "dumb". They inform the server what tasks they can perform, and wait for the server to assign them a task.

The system relies on client and server having access to the same directory, to process the commands.

Server

The server is the core of the MCP. It uses a set of modules, created at run time based on configurations (mcpModulesConfigs). These configs specify a directory to watch, and a series of commands to execute on anything placed in the directory.

Server Implementation

As stated above, the MCP watches directories specified in the modules, and has a set of commands to issue on items placed in those folders. Those commands will not be performed by the MCP server itself, but rather delegated to a client.

mcp Modules

The mcp Modules are XML based, and contain a number of fields: <module> Wait for user approval, before creating and assigning tasks.

<requiresUserApproval>Yes/No</requiresUserApproval>

Description to give user to approve Job.(not implemented yet)

<descriptionForApproval></descriptionForApproval>
<notificationStarted></notificationStarted>
<notificationCompletedWithoutErrors></notificationCompletedWithoutErrors>
<notificationCompletedWithErrors></notificationCompletedWithErrors>


<directories>

Directory to watch for folders moved to.

 <watchDirectory>%watchDirectoryPath%appraiseSIP</watchDirectory>

Standard directory to move to while the clients are performing tasks. Defined in archivematica.conf .

 <processingDirectory>%processingDirectory%</processingDirectory>

The output directory is determined by the return value of the commands, and the corresponding config folder. Chaining output folders and watch directories allows for flow through the system.

 <successDirectory>%watchDirectoryPath%...</successDirectory>
 <failureDirectory>%watchDirectoryPath%failed/</failureDirectory>
</directories>


<commands> 

A command to execute on each file or folder, the details of which are described below in 'mcp Modules Command'. The exe command is the command to run on the Event.

<exeCommand> 

The verification command is used if the return of the exeCommand is not reliable to determin the result of the command. (IE virus scan exits zero and logs there is a virus)

<verificationCommand> 

Cleanup allows for some post processing before the Event folder is moved to the success or fail directory.

<cleanupSuccessfulCommand> 
<cleanupUnsuccessfulCommand>
</commands> 

</module>


mcp Modules Command

Each command consists of a number of parts.

<command>
<descriptionWhileExecuting> </descriptionWhileExecuting>

To skip this command, and not execute it at all.

<skip>Yes/No</skip>

Filter what files/folder the command will operate on.

<filterFileEnd></filterFileEnd>
<filterFileStart></filterFileStart>
<filterSubDir></filterSubDir>

If the output of the command is all going to the same file, multiple threads may try to write to the same file simultaneously and cause a collision. Set this option to stop that occurrence.

<requiresOutputLock>No</requiresOutputLock>

Not used - placeholder

<standardIn></standardIn>

File to write standard out to.

<standardOut></standardOut>

File to write standard error to.

<standardError></standardError>
<failureNotification> </failureNotification>

Define the command the client is to execute. This will need to match an entry in the client's supported modules defined int archivematicaClientConfig

<execute> </execute>

The arguments to give the command.

<arguments> </arguments>

Does this command execute once on the SIP, or once for each file?

<executeOnEachFile>yes/no</executeOnEachFile>
</command>

Client

Client Requirements

Client Implementation