Requirements/MediaConch integration

From Archivematica
Jump to navigation Jump to search

Page for requirements gathering for integration of Media Conch in Archivematica.

Use cases

MKV conformance checking

Pre-normalization

User Story: As a digital preservationist, I need to know whether or not mkv files which I am preserving are valid before I process them, because if they are invalid it might impact further preservation decisions.

Specification

Feature: Pre-ingest conformance check

Archivists need to know whether or not mkv files which they are preserving are valid before they process them, because if they are invalid it might impact further preservation decisions.

Background: the file has been identified as an .mkv

Scenario outline: Jamie wants to confirm that their file is a valid .mkv

When: file validation is run

And: the file is <file_validity> .mkv

Then: validation micro-service output is <microservice_output>

And: <am_action>

And: a PREMIS validation event is generated with eventOutcome = <eventOutcome>

Examples:

file_validity microservice_output am_action eventOutcome
valid "Completed successfully" Archivematica continues processing pass
not valid "Failed" Archivematica continues processing fail

Process:

1. In the Archivematica FPR, a new command will be written under Validation to run Mediaconch.

Possible command: 

 mediaconch -mc -iv 4 -fx nameOfFile.mkv

Where iv defines the verbosity of the output and -fx prints the output in xml. Note that presently MediaConch has the same verbosity for anything under 5. More granularity of verbosity will be available later in the development project.

Outcome of above: 

<?xml version="1.0" encoding="UTF-8"?>
<MediaConch xmlns="https://mediaarea.net/mediaconch" version="0.1">
  <media ref="testFiles/MakeUp-029f3287-4f5d-4399-8330-8b31f2b749c8.mkv">
    <implementationChecks>
      <name>MediaConch EBML Implementation Checker</name>
    </implementationChecks>
    <implementationChecks>
      <name>MediaConch FFV1 Implementation Checker</name>
      <check icid="FFV1-VALID-VERSION-VALUE" version="1">
        <context field="Valid Values" value="1 3"/>
        <test outcome="pass">
          <value offset="@offset" name="FFV1 version"/>
        </test>
      </check>
      <check icid="FFV1-VALID-CODERTYPE-VALUE" version="1">
        <context field="Valid Values" value="0 1 2"/>
        <test outcome="pass">
          <value offset="@offset" name="FFV1 coder type"/>
        </test>
      </check>
      <check icid="FFV1-VALID-COLORSPACETYPE-VALUE" version="1">
        <context field="Valid Values" value="0 1"/>
        <test outcome="pass">
          <value offset="@offset" name="FFV1 colorspace type"/>
        </test>
      </check>
    </implementationChecks>
  </media>
</MediaConch>

2. In the Archivematica FPR, a new validation rule is written to use the above command to validate any original file in mkv format.

3. The command runs in the Validation micro-service. If the outcome of any of the checks is fail, the job fails and indicates failure by colour. In the case of either success or failure, the process should proceed to the next micro-service (Examine contents). The user has the opportunity to reject the Transfer in the Create SIP from Transfer micro-service, a decision they can make based on the outcome of Validation.

4. Archivematica writes a PREMIS event for validation to the AIP METS file.

Diagram

Pre norm checker.png

Mockups

PREMIS: 

 <mets:digiprovMD ID="digiprovMD_15">
      <mets:mdWrap MDTYPE="PREMIS:EVENT">
        <mets:xmlData>
          <premis:event xmlns:premis="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
            <premis:eventIdentifier>
              <premis:eventIdentifierType>UUID</premis:eventIdentifierType>
              <premis:eventIdentifierValue>ceb56635-f114-46d7-af3d-5766c8f6c75f</premis:eventIdentifierValue>
            </premis:eventIdentifier>
            <premis:eventType>validation</premis:eventType>
            <premis:eventDateTime>2016-05-25T20:55:38</premis:eventDateTime>
            <premis:eventDetail>program="MediaConch"; version="16.03"</premis:eventDetail>
               <premis:eventOutcomeInformation>
              <premis:eventOutcome>pass</premis:eventOutcome>
              <premis:eventOutcomeDetail>
                <premis:eventOutcomeDetailNote>version="1" outcome="pass"; coder type="1" outcome="pass"; colorspace="1" outcome="pass"</premis:eventOutcomeDetailNote>
              </premis:eventOutcomeDetail>
            </premis:eventOutcomeInformation>
            <premis:linkingAgentIdentifier>
              <premis:linkingAgentIdentifierType>Archivematica user pk</premis:linkingAgentIdentifierType>
              <premis:linkingAgentIdentifierValue>1</premis:linkingAgentIdentifierValue>
            </premis:linkingAgentIdentifier>
            <premis:linkingAgentIdentifier>
              <premis:linkingAgentIdentifierType>preservation system</premis:linkingAgentIdentifierType>
              <premis:linkingAgentIdentifierValue>Archivematica-1.4.1</premis:linkingAgentIdentifierValue>
            </premis:linkingAgentIdentifier>
            <premis:linkingAgentIdentifier>
              <premis:linkingAgentIdentifierType>repository code</premis:linkingAgentIdentifierType>
              <premis:linkingAgentIdentifierValue>ORG</premis:linkingAgentIdentifierValue>
            </premis:linkingAgentIdentifier>
          </premis:event>
        </mets:xmlData>
      </mets:mdWrap>
    </mets:digiprovMD>

Post-normalization

User Story: As a digital preservationist, I need to verify that the preservation copies that Archivematica has created through normalization are in conformance with the FFV1 specification so that I know I am placing preservation copies in storage which are reliable.

Specification

Feature:Post-normalization conformance check

Archivists need to verify that the preservation and access copies that Archivematica has created through normalization are in conformance with the .mkv specification so that they know they are placing derivatives in storage/repositories which are reliable.

Background: file has been normalized to .mkv

Scenario outline: Jamie wants to confirm that normalization to .mkv is successful

When: file validation is run

And: the file is <file_validity> .mkv

Then: validation microservice output is <microservice_output>

And: Archivematica outputs a <validation_result> validation in the normalization report

Examples:

file_validity microservice_output validation_result
valid "Completed successfully" successful
not valid "Failed" failed

Scenario outline: Jamie wants to approve normalization of a .mkv for preservation

Given: normalized file is <file_validity> .mkv

When: Jamie approved normalization

Then: A PREMIS validation event is generated with eventOutcome = <eventOutcome>

And: Archivematica continues processing

Examples:

file_validity eventOutcome
valid pass
not valid fail

Process:

1. In the Archivematica FPR, a new command will be written under Verfication to run MediaConch.

Possible command: 

 mediaconch -mc -iv 4 -fx nameOfFile.mkv

Where iv defines the verbosity of the output and -fx prints the output in xml. Note that presently MediaConch has the same verbosity for anything under 5. More granularity of verbosity will be available later in the development project.

Outcome of above: 

<?xml version="1.0" encoding="UTF-8"?>
<MediaConch xmlns="https://mediaarea.net/mediaconch" version="0.1">
  <media ref="testFiles/MakeUp-029f3287-4f5d-4399-8330-8b31f2b749c8.mkv">
    <implementationChecks>
      <name>MediaConch EBML Implementation Checker</name>
    </implementationChecks>
    <implementationChecks>
      <name>MediaConch FFV1 Implementation Checker</name>
      <check icid="FFV1-VALID-VERSION-VALUE" version="1">
        <context field="Valid Values" value="1 3"/>
        <test outcome="pass">
          <value offset="@offset" name="FFV1 version"/>
        </test>
      </check>
      <check icid="FFV1-VALID-CODERTYPE-VALUE" version="1">
        <context field="Valid Values" value="0 1 2"/>
        <test outcome="pass">
          <value offset="@offset" name="FFV1 coder type"/>
        </test>
      </check>
      <check icid="FFV1-VALID-COLORSPACETYPE-VALUE" version="1">
        <context field="Valid Values" value="0 1"/>
        <test outcome="pass">
          <value offset="@offset" name="FFV1 colorspace type"/>
        </test>
      </check>
    </implementationChecks>
  </media>
</MediaConch>

2. In the Archivematica FPR, a rule is written for validation to run the above command when an mkv file has been created for preservation or access.

3. A new job in the Normalization micro-service runs the command (must be before Job: Move to approve normalization directory). If the outcome of any of the checks is fail, the job fails and indicates failure by colour. The normalization micro-service should proceed to the Approval step. Failures could also be indicated in the Approve normalization report.

4. Archivematica writes a PREMIS event in the METS to record the outcomes of validation.

Diagram

Post norm checker.png

Mockups

Normalization report:

Conformance norm report.png

Columns for checking validity of both preservation and access derivatives have been added to the current Archivematica normalization report.

PREMIS: 

<mets:digiprovMD ID="digiprovMD_428512" CREATED="2016-04-05T18:55:56">
      <mets:mdWrap MDTYPE="PREMIS:EVENT">
        <mets:xmlData>
          <premis:event xmlns:premis="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
            <premis:eventIdentifier>
              <premis:eventIdentifierType>UUID</premis:eventIdentifierType>
              <premis:eventIdentifierValue>e2a85831-8dcd-43be-ba3d-9b7cae0f42b5</premis:eventIdentifierValue>
            </premis:eventIdentifier>
            <premis:eventType>verification</premis:eventType>
            <premis:eventDateTime>2016-05-19T18:03:03</premis:eventDateTime>
            <premis:eventDetail>program="MediaConch"; version="16.03"</premis:eventDetail>
            <premis:eventOutcomeInformation>
              <premis:eventOutcome>pass</premis:eventOutcome>
              <premis:eventOutcomeDetail>
                <premis:eventOutcomeDetailNote>version="1" outcome="pass"; coder type="1" outcome="pass"; colorspace="1" outcome="pass"</premis:eventOutcomeDetailNote>
              </premis:eventOutcomeDetail>
            </premis:eventOutcomeInformation>
            <premis:linkingAgentIdentifier>
              <premis:linkingAgentIdentifierType>Archivematica user pk</premis:linkingAgentIdentifierType>
              <premis:linkingAgentIdentifierValue>1</premis:linkingAgentIdentifierValue>
            </premis:linkingAgentIdentifier>
            <premis:linkingAgentIdentifier>
              <premis:linkingAgentIdentifierType>preservation system</premis:linkingAgentIdentifierType>
              <premis:linkingAgentIdentifierValue>Archivematica-1.7</premis:linkingAgentIdentifierValue>
            </premis:linkingAgentIdentifier>
            <premis:linkingAgentIdentifier>
              <premis:linkingAgentIdentifierType>repository code</premis:linkingAgentIdentifierType>
              <premis:linkingAgentIdentifierValue>Example Repository</premis:linkingAgentIdentifierValue>
            </premis:linkingAgentIdentifier>
          </premis:event>
        </mets:xmlData>
      </mets:mdWrap>
    </mets:digiprovMD>

Policy checking

User stories

  • As a digital preservationist, I want to check vendor-digitized videos against a policy to ensure that the agreed-upon specifications were followed during digitization.
  • As a digital preservationist, I want to ensure that Archivematica is normalizing for access according to my policy so that the access files are optimized for my institution's access system.
  • As a digital preservationist, I want to ensure that Archivematica is normalizing for preservation according to my policy so that preservation files meet my institution's specifications.
Retrieved from "https://wiki.archivematica.org/index.php?title=Requirements/MediaConch_integration&oldid=11475"