Adding Format Identification Tools - Revision history

Sallain at 19:02, 11 February 2020

2020-02-11T19:02:45Z

Hbecker: Move to feature requirements category

2017-03-23T19:44:49Z

Move to feature requirements category

Hbecker: Deprecation warning

2016-06-27T17:08:52Z

Deprecation warning

Hbecker: Formatting, clarification, bit of reordering

2013-07-17T17:06:43Z

Formatting, clarification, bit of reordering

Hbecker: Reordered, so that dependencies came first, and things that depend on them later.

2013-06-01T00:10:12Z

Reordered, so that dependencies came first, and things that depend on them later.

Show changes

Hbecker: /* Run tool */ Clarifications

2013-05-31T23:09:04Z

Run tool: Clarifications

Hbecker: Clarifications and expansions

2013-05-31T22:59:19Z

Clarifications and expansions

Hbecker: /* Set selection */ Spelling

2013-05-31T22:03:12Z

Set selection: Spelling

Joseph at 22:28, 29 May 2013

2013-05-29T22:28:27Z

Joseph: /* Set selection */

2013-05-24T23:14:52Z

Set selection

@@ Line 1: / Line 1: @@
 {| class="wikitable" style="background-color:#ffcccc; font-size: 120%; font-weight: bold; " cellpadding="10"
-| Warning: These instructions are out of date as of Archivematica 1.0. Please see documentation on the [[Format_policy_registry_requirements | FPR]] and [[MCP | MCPServer]]
+| Warning: These instructions are out of date as of Archivematica 1.0. Please see the [https://www.archivematica.org/docs/latest official Archivematica documentation].
 |}

← Older revision		Revision as of 19:44, 23 March 2017
Line 194:		Line 194:


−	[[Category:~~Development documentation~~]]	+	[[Category:Deprecated]]

← Older revision		Revision as of 17:08, 27 June 2016
Line 1:		Line 1:
		+
		+	{\| class="wikitable" style="background-color:#ffcccc; font-size: 120%; font-weight: bold; " cellpadding="10"
		+	\| Warning: These instructions are out of date as of Archivematica 1.0. Please see documentation on the [[Format_policy_registry_requirements \| FPR]] and [[MCP \| MCPServer]]
		+	\|}

	This will show a developer how to add a new tool new tool to identify file formats (file ids), and allow normalization based on the newly identified file formats.		This will show a developer how to add a new tool new tool to identify file formats (file ids), and allow normalization based on the newly identified file formats.

@@ Line 16: / Line 16: @@
 ==FileIDTypes==
-First, an entry for FileIDTypes is required.
+First, an entry for FileIDTypes is required for the new tool.
 <pre>
-INSERT INTO `FileIDTypes` (`pk`, `description`, `replaces`, `lastModified`, `enabled`) VALUES ('afdbee13-eec5-4182-8c6c-f5638ee290f3','FileIDByFIDO',NULL,'2013-03-12 01:54:24',1);
+INSERT INTO
 </pre>
@@ Line 48: / Line 51: @@
 </pre>
 <pre>
-INSERT INTO StandardTasksConfigs (pk, filterFileEnd, filterFileStart, filterSubDir, requiresOutputLock, standardOutputFile, standardErrorFile, execute, arguments)
+INSERT INTO
-     VALUES (@TasksConfigPKReference, NULL, NULL, 'objects/', FALSE, NULL, NULL, 'archivematicaFido_v0.0', '--fileUUID "%fileUUID%" --SIPUUID "%SIPUUID%" --filePath "%relativeLocation%" --eventIdentifierUUID "%taskUUID%" --date "%date%" --fileGrpUse "%fileGrpUse%"');
+    StandardTasksConfigs (pk, filterFileEnd, filterFileStart, filterSubDir, requiresOutputLock, standardOutputFile, standardErrorFile, execute, arguments)
 </pre>
 * This creates the configuration for running the script
@@ Line 57: / Line 62: @@
 <pre>
-INSERT INTO TasksConfigs (pk, taskType, taskTypePKReference, description)
+INSERT INTO
-    VALUES
+    TasksConfigs (pk, taskType, taskTypePKReference, description)
      (@TasksConfig, 'a6b1c323-7d36-428e-846a-e7e819423577', @TasksConfigPKReference, 'Identify file formats with FIDO');
 </pre>
@@ Line 66: / Line 72: @@
 <pre>
-INSERT INTO MicroServiceChainLinks (pk, microserviceGroup, currentTask, defaultNextChainLink)
+INSERT INTO
-     VALUES (@MicroServiceChainLink, @microserviceGroup, @TasksConfig, @defaultNextChainLink);
+    MicroServiceChainLinks (pk, microserviceGroup, currentTask, defaultNextChainLink)
 </pre>
 * This creates the actual link to run
@@ Line 75: / Line 83: @@
 <pre>
-INSERT INTO MicroServiceChainLinksExitCodes (pk, microServiceChainLink, exitCode, nextMicroServiceChainLink)
+INSERT INTO
-     VALUES (@MicroServiceChainLinksExitCodes, @MicroServiceChainLink, 0, @NextMicroServiceChainLink);
+    MicroServiceChainLinksExitCodes (pk, microServiceChainLink, exitCode, nextMicroServiceChainLink)
-SET @NextMicroServiceChainLink = @MicroServiceChainLink;
+VALUES
 </pre>
 * This configures where to go once the MicroServiceChainLink is complete, based on exit code
@@ Line 106: / Line 115: @@
 For FIDO:
 <pre>
-INSERT INTO `MicroServiceChainLinks` (`pk`, `currentTask`, `defaultNextChainLink`, `defaultPlaySound`, `microserviceGroup`, `reloadFileList`, `defaultExitMessage`, `replaces`, `lastModified`)
+INSERT INTO
-     VALUES ('982229bd-73b8-432e-a1d9-2d9d15d7287d','1e516ea6-6814-4292-9ea9-552ebfaa0d23','4c4281a1-43cd-4c6e-b1dc-573bd1a23c43',NULL,'Normalize',1,'Failed',NULL,'2012-10-23 19:41:23');
+    `MicroServiceChainLinks` (`pk`, `currentTask`, `defaultNextChainLink`, `defaultPlaySound`, `microserviceGroup`, `reloadFileList`, `defaultExitMessage`)
-INSERT INTO `TasksConfigs` (`pk`, `taskType`, `taskTypePKReference`, `description`, `replaces`, `lastModified`)
+VALUES
-    VALUES ('1e516ea6-6814-4292-9ea9-552ebfaa0d23','6f0b612c-867f-4dfd-8e43-5b35b7f882d7','f130c16d-d419-4063-8c8b-2e4c3ad138bb','Set SIP to normalize with FIDO file identification.',NULL,'2012-10-23 19:41:23');
+     ('982229bd-73b8-432e-a1d9-2d9d15d7287d','1e516ea6-6814-4292-9ea9-552ebfaa0d23','4c4281a1-43cd-4c6e-b1dc-573bd1a23c43',NULL,'Normalize',1,'Failed');
 </pre>
 ==Chain Choice==
@@ Line 140: / Line 173: @@
 +--------------------------------------+--------------------------------------+---------------------+
 </pre>
 Finally, we add the actual choice:
 <pre>
-INSERT INTO MicroServiceChainChoice (pk, choiceAvailableAtLink, chainAvailable)
+INSERT INTO
-     VALUES ('e95b8f27-ea52-4247-bdf0-615273bc5fca', 'f4dea20e-f3fe-4a37-b20f-0e70a7bc960e', 'c76624a8-6f85-43cf-8ea7-0663502c712f');
+    MicroServiceChainChoice (pk, choiceAvailableAtLink, chainAvailable)
 </pre>
 * MicroServiceChainChoice.choiceAvailableAtLink: MicroServiceChainLink where the choice is made available.  We know this is f4dea20e-f3fe-4a37-b20f-0e70a7bc960e

@@ Line 71: / Line 71: @@
 ==Run tool==
-The next step is to run the tool on the objects. This requires both definition of the workflow, and a task to support it.
+The next step is to run the new tool on the objects. This requires both definition of the workflow, a task to support it, and the script to do all the work.
-===workflow===
+===Workflow===
 FIDO workflow link:
 <pre>
@@ Line 97: / Line 97: @@
 </pre>
-===Identify files script===
+===Identify Files Script===
-This script must be added to the client as a supported module.
+The script to identify the file must be created, and an entry for it added to the FilesIdentifiedIDs table.  For Fido this script is [https://github.com/artefactual/archivematica/blob/642a3df29707e014f6c9ee5b7ea64785454c76cc/src/MCPClient/lib/clientScripts/archivematicaFido.py archivematicaFido.py]  Obviously, all the arguments the script accepts must be the same as the arguments listed in StandardTasksConfigs, above.
 <pre>
 archivematicaFido_v0.0 = %clientScriptsDirectory%archivematicaFido.py
 </pre>
 ==Return==

@@ Line 1: / Line 1: @@
-These are developer facing instructions for adding a new tool to base normalization on.
+This will show a developer how to add a new tool new tool to identify file formats (file ids), and allow normalization based on the newly identified file formats.
-The basic steps this section is looking to implement are:
+The workflow this section is looking to implement are:
 * Make the fileID tool a selectable choice
-* Specify that normalization will be using that fileIDType during normalization
+* Specify that normalization will be using that fileIDType
 * Identify files using that tool, with valid archivematica fileIDs (format IDs)
 * Use those format IDs and associated commands to normalize files.
 =Add tool=
 To include a new tool in the archivematica packages, it should be a dependancy package itself.
@@ Line 39: / Line 42: @@
 INSERT INTO MicroServiceChains (pk, startingLink, description)
      VALUES ('c76624a8-6f85-43cf-8ea7-0663502c712f', '982229bd-73b8-432e-a1d9-2d9d15d7287d', 'FIDO');
 INSERT INTO MicroServiceChainChoice (pk, choiceAvailableAtLink, chainAvailable)
      VALUES ('e95b8f27-ea52-4247-bdf0-615273bc5fca', 'f4dea20e-f3fe-4a37-b20f-0e70a7bc960e', 'c76624a8-6f85-43cf-8ea7-0663502c712f');
 </pre>
-==Set selection==
+==Set Selection==
 The first step in the workflow is to set the selection as the tool to use during normalization. This is done by making an insert into the unit's variables table for the variable normalizationFileIdentificationToolIdentifierTypes. The value set is a piece of a SQL query used in [https://github.com/artefactual/archivematica/blob/master/src/MCPServer/lib/linkTaskManagerSplitOnFileIdAndruleset.py linkTaskManagerSplitOnFileIdAndruleset.py] to restrict the fileIDs used to the desired type.