Requirements/MediaConch integration
Page for requirements gathering for integration of Media Conch in Archivematica. See also, MediaConch workflow.
Use cases[edit]
MKV conformance checking[edit]
Pre-normalization[edit]
User Story: As a digital preservationist, I need to know whether or not mkv files which I am preserving are valid before I process them, because if they are invalid it might impact further preservation decisions.
Specification
Feature: Pre-ingest conformance check
Archivists need to know whether or not mkv files which they are preserving are valid before they process them, because if they are invalid it might impact further preservation decisions.
Background: the file has been identified as an .mkv
Scenario outline: Jamie wants to confirm that their file is a valid .mkv
When: file validation is run
And: the file is <file_validity> .mkv
Then: validation micro-service output is <microservice_output>
And: <am_action>
And: a PREMIS validation event is generated with eventOutcome = <eventOutcome>
Examples:
| file_validity | microservice_output | am_action | eventOutcome |
| valid | "Completed successfully" | Archivematica continues processing | pass |
| not valid | "Failed" | Archivematica continues processing | fail |
Process:
1. In the Archivematica FPR, a new command will be written under Validation to run Mediaconch.
Possible command:
mediaconch -mc -iv 4 -fx nameOfFile.mkv
Where iv defines the verbosity of the output and -fx prints the output in xml. Note that presently MediaConch has the same verbosity for anything under 5. More granularity of verbosity will be available later in the development project.
Outcome of above:
<?xml version="1.0" encoding="UTF-8"?>
<MediaConch xmlns="https://mediaarea.net/mediaconch" version="0.1">
<media ref="testFiles/MakeUp-029f3287-4f5d-4399-8330-8b31f2b749c8.mkv">
<implementationChecks>
<name>MediaConch EBML Implementation Checker</name>
</implementationChecks>
<implementationChecks>
<name>MediaConch FFV1 Implementation Checker</name>
<check icid="FFV1-VALID-VERSION-VALUE" version="1">
<context field="Valid Values" value="1 3"/>
<test outcome="pass">
<value offset="@offset" name="FFV1 version"/>
</test>
</check>
<check icid="FFV1-VALID-CODERTYPE-VALUE" version="1">
<context field="Valid Values" value="0 1 2"/>
<test outcome="pass">
<value offset="@offset" name="FFV1 coder type"/>
</test>
</check>
<check icid="FFV1-VALID-COLORSPACETYPE-VALUE" version="1">
<context field="Valid Values" value="0 1"/>
<test outcome="pass">
<value offset="@offset" name="FFV1 colorspace type"/>
</test>
</check>
</implementationChecks>
</media>
</MediaConch>
2. In the Archivematica FPR, a new validation rule is written to use the above command to validate any original file in mkv format.
3. The command runs in the Validation micro-service. If the outcome of any of the checks is fail, the job fails and indicates failure by colour. In the case of either success or failure, the process should proceed to the next micro-service (Examine contents). The user has the opportunity to reject the Transfer in the Create SIP from Transfer micro-service, a decision they can make based on the outcome of Validation.
4. Archivematica writes a PREMIS event for validation to the AIP METS file.
Diagram
Mockups
PREMIS:
<mets:digiprovMD ID="digiprovMD_15">
<mets:mdWrap MDTYPE="PREMIS:EVENT">
<mets:xmlData>
<premis:event xmlns:premis="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
<premis:eventIdentifier>
<premis:eventIdentifierType>UUID</premis:eventIdentifierType>
<premis:eventIdentifierValue>ceb56635-f114-46d7-af3d-5766c8f6c75f</premis:eventIdentifierValue>
</premis:eventIdentifier>
<premis:eventType>validation</premis:eventType>
<premis:eventDateTime>2016-05-25T20:55:38</premis:eventDateTime>
<premis:eventDetail>program="MediaConch"; version="16.03"</premis:eventDetail>
<premis:eventOutcomeInformation>
<premis:eventOutcome>pass</premis:eventOutcome>
<premis:eventOutcomeDetail>
<premis:eventOutcomeDetailNote>version="1" outcome="pass"; coder type="1" outcome="pass"; colorspace="1" outcome="pass"</premis:eventOutcomeDetailNote>
</premis:eventOutcomeDetail>
</premis:eventOutcomeInformation>
<premis:linkingAgentIdentifier>
<premis:linkingAgentIdentifierType>Archivematica user pk</premis:linkingAgentIdentifierType>
<premis:linkingAgentIdentifierValue>1</premis:linkingAgentIdentifierValue>
</premis:linkingAgentIdentifier>
<premis:linkingAgentIdentifier>
<premis:linkingAgentIdentifierType>preservation system</premis:linkingAgentIdentifierType>
<premis:linkingAgentIdentifierValue>Archivematica-1.4.1</premis:linkingAgentIdentifierValue>
</premis:linkingAgentIdentifier>
<premis:linkingAgentIdentifier>
<premis:linkingAgentIdentifierType>repository code</premis:linkingAgentIdentifierType>
<premis:linkingAgentIdentifierValue>ORG</premis:linkingAgentIdentifierValue>
</premis:linkingAgentIdentifier>
</premis:event>
</mets:xmlData>
</mets:mdWrap>
</mets:digiprovMD>
Post-normalization[edit]
User Story: As a digital preservationist, I need to verify that the preservation copies that Archivematica has created through normalization are in conformance with the FFV1 specification so that I know I am placing preservation copies in storage which are reliable.
Specification
Feature:Post-normalization conformance check
Archivists need to verify that the preservation and access copies that Archivematica has created through normalization are in conformance with the .mkv specification so that they know they are placing derivatives in storage/repositories which are reliable.
Background: file has been normalized to .mkv
Scenario outline: Jamie wants to confirm that normalization to .mkv is successful
When: file validation is run
And: the file is <file_validity> .mkv
Then: validation microservice output is <microservice_output>
And: Archivematica outputs a <validation_result> validation in the normalization report
Examples:
| file_validity | microservice_output | validation_result |
| valid | "Completed successfully" | successful |
| not valid | "Failed" | failed |
Scenario outline: Jamie wants to approve normalization of a .mkv for preservation
Given: normalized file is <file_validity> .mkv
When: Jamie approved normalization
Then: A PREMIS validation event is generated with eventOutcome = <eventOutcome>
And: Archivematica continues processing
Examples:
| file_validity | eventOutcome |
| valid | pass |
| not valid | fail |
Process:
1. In the Archivematica FPR, a new command will be written under Verfication to run MediaConch.
Possible command:
mediaconch -mc -iv 4 -fx nameOfFile.mkv
Where iv defines the verbosity of the output and -fx prints the output in xml. Note that presently MediaConch has the same verbosity for anything under 5. More granularity of verbosity will be available later in the development project.
Outcome of above:
<?xml version="1.0" encoding="UTF-8"?>
<MediaConch xmlns="https://mediaarea.net/mediaconch" version="0.1">
<media ref="testFiles/MakeUp-029f3287-4f5d-4399-8330-8b31f2b749c8.mkv">
<implementationChecks>
<name>MediaConch EBML Implementation Checker</name>
</implementationChecks>
<implementationChecks>
<name>MediaConch FFV1 Implementation Checker</name>
<check icid="FFV1-VALID-VERSION-VALUE" version="1">
<context field="Valid Values" value="1 3"/>
<test outcome="pass">
<value offset="@offset" name="FFV1 version"/>
</test>
</check>
<check icid="FFV1-VALID-CODERTYPE-VALUE" version="1">
<context field="Valid Values" value="0 1 2"/>
<test outcome="pass">
<value offset="@offset" name="FFV1 coder type"/>
</test>
</check>
<check icid="FFV1-VALID-COLORSPACETYPE-VALUE" version="1">
<context field="Valid Values" value="0 1"/>
<test outcome="pass">
<value offset="@offset" name="FFV1 colorspace type"/>
</test>
</check>
</implementationChecks>
</media>
</MediaConch>
2. In the Archivematica FPR, a rule is written for validation to run the above command when an mkv file has been created for preservation or access.
3. A new job in the Normalization micro-service runs the command (must be before Job: Move to approve normalization directory). If the outcome of any of the checks is fail, the job fails and indicates failure by colour. The normalization micro-service should proceed to the Approval step. Failures could also be indicated in the Approve normalization report.
4. Archivematica writes a PREMIS event in the METS to record the outcomes of validation.
Diagram
Mockups
Normalization report:
Columns for checking validity of both preservation and access derivatives have been added to the current Archivematica normalization report.
PREMIS:
<mets:digiprovMD ID="digiprovMD_428512" CREATED="2016-04-05T18:55:56">
<mets:mdWrap MDTYPE="PREMIS:EVENT">
<mets:xmlData>
<premis:event xmlns:premis="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
<premis:eventIdentifier>
<premis:eventIdentifierType>UUID</premis:eventIdentifierType>
<premis:eventIdentifierValue>e2a85831-8dcd-43be-ba3d-9b7cae0f42b5</premis:eventIdentifierValue>
</premis:eventIdentifier>
<premis:eventType>verification</premis:eventType>
<premis:eventDateTime>2016-05-19T18:03:03</premis:eventDateTime>
<premis:eventDetail>program="MediaConch"; version="16.03"</premis:eventDetail>
<premis:eventOutcomeInformation>
<premis:eventOutcome>pass</premis:eventOutcome>
<premis:eventOutcomeDetail>
<premis:eventOutcomeDetailNote>version="1" outcome="pass"; coder type="1" outcome="pass"; colorspace="1" outcome="pass"</premis:eventOutcomeDetailNote>
</premis:eventOutcomeDetail>
</premis:eventOutcomeInformation>
<premis:linkingAgentIdentifier>
<premis:linkingAgentIdentifierType>Archivematica user pk</premis:linkingAgentIdentifierType>
<premis:linkingAgentIdentifierValue>1</premis:linkingAgentIdentifierValue>
</premis:linkingAgentIdentifier>
<premis:linkingAgentIdentifier>
<premis:linkingAgentIdentifierType>preservation system</premis:linkingAgentIdentifierType>
<premis:linkingAgentIdentifierValue>Archivematica-1.7</premis:linkingAgentIdentifierValue>
</premis:linkingAgentIdentifier>
<premis:linkingAgentIdentifier>
<premis:linkingAgentIdentifierType>repository code</premis:linkingAgentIdentifierType>
<premis:linkingAgentIdentifierValue>Example Repository</premis:linkingAgentIdentifierValue>
</premis:linkingAgentIdentifier>
</premis:event>
</mets:xmlData>
</mets:mdWrap>
</mets:digiprovMD>
Policy checking[edit]
User stories
- As a digital preservationist, I want to check vendor-digitized videos against a policy to ensure that the agreed-upon specifications were followed during digitization.
- As a digital preservationist, I want to ensure that Archivematica is normalizing for access according to my policy so that the access files are optimized for my institution's access system.
- As a digital preservationist, I want to ensure that Archivematica is normalizing for preservation according to my policy so that preservation files meet my institution's specifications.
Specification
Feature: Ingest policy check
Archivists want to check that the files that are defined for access or preservation conform to a pre-defined policy/policies. The access and preservation derivatives may have been created by Archivematica through normalization or may have been predefined by the user prior to transfer.
Scenario Outline: Jamie has preservation derivatives and they need to know whether they conform to their preservation policy
Given: a policy has been provided for preservation
And: <does_file_conform> to the policy
When: file verification is run
Then: verification microservice output is <microservice_output>
And: a PREMIS verification event is generated with eventOutcome = <eventOutcome>
And: Archivematica outputs a <verification_result> verification in the normalization report
Examples:
| does_file_conform | microservice_output | eventOutcome | validation_result |
| conforms | pass | pass | successful |
| does not conform | fail | fail | failed |