Difference between revisions of "LOCKSS Integration"

From Archivematica
Jump to navigation Jump to search
(Created page with "Main Page > Development > Development documentation > LOCKSS Integration Development Status: In Progress Public Release: estim...")
 
 
(30 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 
[[Main Page]] > [[Development]] > [[:Category:Development documentation|Development documentation]] > LOCKSS Integration
 
[[Main Page]] > [[Development]] > [[:Category:Development documentation|Development documentation]] > LOCKSS Integration
  
−
Development Status: In Progress
+
<div style="padding: 10px 10px; border: 1px solid black; background-color: #F79086;">This page is no longer being maintained and may contain inaccurate information. Please see the [https://www.archivematica.org/docs/latest/ Archivematica documentation] for up-to-date information.</div><p>
−
Public Release: estimated April 2014
+
 
 +
[[Category:Feature requirements]]
 +
 
 +
==Status==
 +
 
 +
'''Sponsored''': SFU Library<br>
 +
'''Development Status''': Complete<br>
 +
'''Public Release''': September 2014
 +
 
 +
 
 +
==Overview==
 +
 
 +
LOCKSS (Lots of Copies Keeps Stuff Safe) http://www.lockss.org/ is treated by Archivematica as a storage sub-system.  The Archivematica Storage Service can be configured to store AIPs in a LOCKSS network.  The procedure for storing AIPs follows the SWORD v2 protocol: http://swordapp.org/sword-v2/ .
 +
 
 +
The Archivematica Storage Service contains an implementation of a SWORD client, which communicates with a SWORD server.  The implementation of the SWORD Server is provided by LOCKSS-O-Matic https://github.com/mjordan/lockss-o-matic.  LOCKSS-O-Matic is responsible for communicating with the Private LOCKSS Network (PLN)
 +
 
 +
===Basic Workflow===
 +
 
 +
* Digital Objects are processed and packaged into AIPs in an Archivematica pipeline
 +
* Archivematica pipeline sends the final AIP to the Storage Service using the 'Store AIP' micro-service
 +
* Storage Service keeps a local copy of the AIP
 +
* Storage Service POSTs the list of files (e.g., a .7z file or parts of one created with split) that make up the AIP to LOCKSS-O-Matic. This list is in the form of an Atom document.
 +
* LOCKSS-O-Matic parses the list of files in the Atom document and registers them in its database.
 +
* LOCKSS-O-Matic edits the configuration files of all the LOCKSS boxes in the Private LOCKSS Network
 +
* LOCKSS boxes harvest the AIP files from the Storage Service
 +
* Storage Service polls LOCKSS-O-Matic (GETs the State document) to determine when the AIP has been stored in LOCKSS
 +
* If/When the Storage Service wants to remove its local copy of the AIP, it first POSTs a metadata update to LOCKSS-O-Matic
 +
* LOCKSS-O-Matic then updates the configuration files in LOCKSS boxes so they will stop harvesting the AIP
 +
 
 +
==Technical Details==
  
−
Overview
 
 
This document outlines a minimalist SWORD API where LOCKSS-O-Matic is the server, and the Archivematica Storage Service is the client. @todos and questions appear in <!-- comments --> throughout.
 
This document outlines a minimalist SWORD API where LOCKSS-O-Matic is the server, and the Archivematica Storage Service is the client. @todos and questions appear in <!-- comments --> throughout.
  
 
In the examples below, Archivematica is at http://archivematica.example.org and LOCKSS-O-Matic is at http://lockssomatic.example.org. The content file being managed by the SWORD deposit is an Archivematica AIP with the UUID dd3e3247-8466-4f2a-bb32-22a210cfce60. LOCKSS-O-Matic is managing a small Private LOCKSS Network containing two boxes, http://lockss1.example.org and http://lockss2.example.org.
 
In the examples below, Archivematica is at http://archivematica.example.org and LOCKSS-O-Matic is at http://lockssomatic.example.org. The content file being managed by the SWORD deposit is an Archivematica AIP with the UUID dd3e3247-8466-4f2a-bb32-22a210cfce60. LOCKSS-O-Matic is managing a small Private LOCKSS Network containing two boxes, http://lockss1.example.org and http://lockss2.example.org.
  
−
Service Document
+
===Service Document===
−
Archivematica issues a GET (with an HTTP header of ‘LOM-Content-Provider’ with the value of the Archivematica instance’s content provider ID in the target LOM instance) to the SD-IRI (http://lockssomatic.example.org/api/sword/2.0/sd-iri). LOCKSS-O-Matic responds with a Service Document like:
+
Archivematica issues a GET (with an HTTP header of ‘On-Behalf-Of’ with the value of the Archivematica instance’s content provider ID in the target LOM instance) to the SD-IRI (http://lockssomatic.example.org/api/sword/2.0/sd-iri). LOCKSS-O-Matic responds with a Service Document like:
  
−
LOM-Content-Provider: 12
+
''On-Behalf-Of: 12''
  
 +
<pre>
 
<service xmlns:dcterms="http://purl.org/dc/terms/"
 
<service xmlns:dcterms="http://purl.org/dc/terms/"
 
     xmlns:sword="http://purl.org/net/sword/terms/"
 
     xmlns:sword="http://purl.org/net/sword/terms/"
Line 22: Line 51:
 
     <sword:version>2.0</sword:version>
 
     <sword:version>2.0</sword:version>
 
     <!-- maxUploadSize is configurable in LOCKSS-O-Matic. →
 
     <!-- maxUploadSize is configurable in LOCKSS-O-Matic. →
−
     <!-- measured in kB -->
+
     <!-- measured in kB, where kB = 1000*Byte -->
 
     <sword:maxUploadSize>102400</sword:maxUploadSize>
 
     <sword:maxUploadSize>102400</sword:maxUploadSize>
 
    
 
    
Line 39: Line 68:
 
     </workspace>
 
     </workspace>
 
</service>
 
</service>
 +
</pre>
  
  
 +
===Creating a Resource with an Atom Entry===
 +
Archivematica issues a POST to the Col-IRI, ensuring that the <id> element in the Atom entry contains the UUID of the AIP, and that the URLs to the files to be preserved are listed in lom:content elements:
  
−
Creating a Resource with an Atom Entry
+
<pre>
−
Archivematica issues a POST to the Col-IRI, ensuring that the <id> element in the Atom entry contains the UUID of the AIP:
 
−
 
 
 
<entry xmlns="http://www.w3.org/2005/Atom"     
 
<entry xmlns="http://www.w3.org/2005/Atom"     
 
         xmlns:dcterms="http://purl.org/dc/terms/"
 
         xmlns:dcterms="http://purl.org/dc/terms/"
Line 53: Line 83:
 
     <author><name>Name of PREMIS agent owning AIP</name></author>
 
     <author><name>Name of PREMIS agent owning AIP</name></author>
 
     <summary type="text">The AIP’s dc:description if it has one. If not, use a generic summary.</summary>
 
     <summary type="text">The AIP’s dc:description if it has one. If not, use a generic summary.</summary>
−
     <lom:content size="102400" checksumType="md5" checksumValue="bd4a9b642562547754086de2dab26b7d">http://archivematica.example.org/archival-storage/download/aip/dd3e3247-8466-4f2a-bb32-22a210cfce60.001</lom:content>
+
    <!-- size is measured in kB, where kB = 1000*Byte -->
−
     <lom:content size="46899" checksumType="md5" checksumValue="226190d94b21d1b0c7b1a42d855e419d">http://archivematica.example.org/archival-storage/download/aip/dd3e3247-8466-4f2a-bb32-22a210cfce60.002</lom:content>
+
     <lom:content size="102400" checksumType="md5" checksumValue="bd4a9b642562547754086de2dab26b7d">
 +
        http://archivematica.example.org/archival-storage/download/aip/dd3e3247-8466-4f2a-bb32-22a210cfce60.001
 +
    </lom:content>
 +
     <lom:content size="46899" checksumType="md5" checksumValue="226190d94b21d1b0c7b1a42d855e419d">
 +
        http://archivematica.example.org/archival-storage/download/aip/dd3e3247-8466-4f2a-bb32-22a210cfce60.002
 +
    </lom:content>
 
</entry>
 
</entry>
 +
</pre>
  
 +
LOCKSS-O-Matic responds with a 201 Created and the Location header with the entry’s Edit-IRI, and the deposit receipt.
  
−
LOCKSS-O-Matic responds with a 201 Created and the Location header with the entry’s Edit-IRI, and the deposit receipt.
+
===Deposit Receipt===
−
 
 
−
Deposit Receipt
 
 
Contains the Cont-IRI, the EM-IRI, and the State-IRI.
 
Contains the Cont-IRI, the EM-IRI, and the State-IRI.
  
 +
<pre>
 
<entry xmlns="http://www.w3.org/2005/Atom" xmlns:sword="http://purl.org/net/sword/">
 
<entry xmlns="http://www.w3.org/2005/Atom" xmlns:sword="http://purl.org/net/sword/">
 
     <sword:treatment>Stored in LOCKSS via LOCKSS-o-matic</sword:treatment>
 
     <sword:treatment>Stored in LOCKSS via LOCKSS-o-matic</sword:treatment>
Line 76: Line 112:
 
     <link rel="edit" href="http://lockssomatic.example.org/api/sword/2.0/cont-iri/12/dd3e3247-8466-4f2a-bb32-22a210cfce60/edit" />
 
     <link rel="edit" href="http://lockssomatic.example.org/api/sword/2.0/cont-iri/12/dd3e3247-8466-4f2a-bb32-22a210cfce60/edit" />
 
     <!-- In LOCKSS-O-Matic, the State-IRI will be the EM-IRI/Cont-IRI with the string ‘/state’ appended. -->
 
     <!-- In LOCKSS-O-Matic, the State-IRI will be the EM-IRI/Cont-IRI with the string ‘/state’ appended. -->
−
     <link rel="http://purl.org/net/sword/terms/statement" type="application/atom+xml;type=feed" href="http://lockssomatic.example.org/api/sword/2.0/cont-IRI/12/dd3e3247-8466-4f2a-bb32-22a210cfce60/state" />
+
     <link rel="http://purl.org/net/sword/terms/statement" type="application/atom+xml;type=feed"  
 +
      href="http://lockssomatic.example.org/api/sword/2.0/cont-IRI/12/dd3e3247-8466-4f2a-bb32-22a210cfce60/state" />
 
</entry>
 
</entry>
 +
</pre>
  
−
 
+
===SWORD Statement===
−
SWORD Statement
 
 
State-IRI is defined in the deposit receipt. GET requests to the State-IRI for a resource will return an Atom feed as described below.
 
State-IRI is defined in the deposit receipt. GET requests to the State-IRI for a resource will return an Atom feed as described below.
  
 
These state terms apply to the Allowed state term and message values, using the namespace http://lockssomatic.info/SWORD2,  are:
 
These state terms apply to the Allowed state term and message values, using the namespace http://lockssomatic.info/SWORD2,  are:
  
−
Term
+
{| class="wikitable"
−
Message (one of)
+
|-
−
failed
+
! Term
−
Content cannot be harvested by LOCKSS.
+
! Message (one of)
−
 
+
|-
−
 
+
| failed
−
disagreement
+
| Content cannot be harvested by LOCKSS.
−
LOCKSS network is not in agreement on content checksums.
+
|-
−
agreement
+
| disagreement
−
LOCKSS network agrees internally on content checksums.
+
| LOCKSS network is not in agreement on content checksums.
 +
|-
 +
| agreement
 +
| LOCKSS network agrees internally on content checksums.
 +
|}
  
 
If LOCKSS-O-Matic reports ‘agreement’, Archivematica may delete the AIP from local storage.
 
If LOCKSS-O-Matic reports ‘agreement’, Archivematica may delete the AIP from local storage.
  
 
Sample SWORD statement serialized as an Atom feed:
 
Sample SWORD statement serialized as an Atom feed:
−
 
+
<pre>
 
<atom:feed xmlns:sword="http://purl.org/net/sword/terms/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:lom="http://lockssomatic.info/SWORD2">
 
<atom:feed xmlns:sword="http://purl.org/net/sword/terms/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:lom="http://lockssomatic.info/SWORD2">
 
   <atom:entry>
 
   <atom:entry>
Line 121: Line 162:
 
   </atom:entry>
 
   </atom:entry>
 
</atom:feed>
 
</atom:feed>
 +
</pre>
  
 +
===Updating Metadata===
 +
Archivematica informs LOCKSS-O-Matic that it wants to delete the local copy of a file by PUTing to the SE-IRI an Atom document containing lom:content elements with a recrawl="false" attribute for each file. LOCKSS-O-Matic responds with a 202 Accepted and updates the LOCKSS configuration files to indicate that the AU the file is in should not be reharvested. After the configuration files have been updated, LOCKSS-O-Matic will respond with a 200 OK to the same request.
  
−
Updating Metadata
+
PUT SE-IRI
−
Informing LOCKSS-o-matic that Archivematica wants to delete the local copy , and that LOM should not harvest it anymore.
+
<pre>
−
 
 
−
POST SE-IRI  
 
 
<?xml version="1.0"?>
 
<?xml version="1.0"?>
 
<entry xmlns="http://www.w3.org/2005/Atom" xmlns:dcterms="http://purl.org/dc/terms/" xmlns:lom="http://lockssomatic.info/SWORD2">
 
<entry xmlns="http://www.w3.org/2005/Atom" xmlns:dcterms="http://purl.org/dc/terms/" xmlns:lom="http://lockssomatic.info/SWORD2">
Line 134: Line 176:
 
     <lom:content recrawl="false">http://archivematica.example.org/archival-storage/download/aip/dd3e3247-8466-4f2a-bb32-22a210cfce60.002</lom:content>
 
     <lom:content recrawl="false">http://archivematica.example.org/archival-storage/download/aip/dd3e3247-8466-4f2a-bb32-22a210cfce60.002</lom:content>
 
</entry>
 
</entry>
−
 
+
</pre>
 
LOM can return:
 
LOM can return:
−
- HTTP 202 (Accepted) meaning LOM is updating the LOCKSS config files saying not to harvest this, but it is not done yet.
+
*- HTTP 202 (Accepted) meaning LOM is updating the LOCKSS config files saying not to harvest this, but it is not done yet.
−
- HTTP 200 (OK) meaning all config updates are complete
+
*- HTTP 200 (OK) meaning all config updates are complete
−
- HTTP 204 (No Content) if there is no matching aip
+
*- HTTP 204 (No Content) if there is no matching AIP
−
- HTTP 409 (Conflict) There are files in the LOCKSS AU that do not have ‘recrawl=false’.
+
*- HTTP 409 (Conflict) There are files in the LOCKSS Archival Unit (AU) that do not have ‘recrawl=false’.
  
 
“Once the server has processed the request it MUST return an HTTP status code 200 (OK) or 204 (No Content), or an appropriate error code. “
 
“Once the server has processed the request it MUST return an HTTP status code 200 (OK) or 204 (No Content), or an appropriate error code. “
−
Comments/questions
 
  
−
Can something be deleted from LOCKSS-o-matic? How?  
+
===Comments/questions===
−
Not really, since LOCKSS doesn’t let you delete content, not directly anyway (it can be done on the boxes’ command lines). In LOM we could have a flag that specifies that content should not be harvested into the LOCKSS network.  
+
 
 +
;'''Can an AIP be deleted from LOCKSS-O-Matic? How?  
 +
:Not really, since LOCKSS doesn’t let you delete content, not directly anyway (it can be done on the boxes’ command lines). In LOM we could have a flag that specifies that content should not be harvested into the LOCKSS network. In the Archivematica dashboard, an AIP deletion request can be made.  This deletion request goes to the Storage Service, which would tell LOM to stop harvesting the AIP, and then delete the local copy.  The copies in the PLN would not be deleted, and would have to be manually deleted by a LOCKSS administrator.
  
−
Could be useful.
+
;'''Can an AIP be retrieved from LOCKSS via the Archivematica Dashboard?
−
Is there any sort of authentication/authorization on the API?
+
:Yes - with the following caveat.  All AIP's stored in the Storage Service are searchable via the Archival Storage tab in the dashboard.  The Storage Service keeps a local copy of AIP's that are stored in LOCKSS, so when a user clicks on an AIP in the Archival Storage tab, that local copy will be delivered.  If the local copy has been deleted (which will eventually happen, exactly when is based on Storage Service configuration settings) then the dashboard user will get a 'local copy not available, contact your storage administrator' message.  From the Storage Service, it is possible for a storage administrator to click a button to download a copy of the AIP directly from one of the LOCKSS boxes, to the storage service.  Once that is done, the AIP will then be retrievable from a dashboard.
−
At a minimum IP whitelisting. We can also do HTTP Basic (which is a “SHOULD” in SWORD). How are you handling it with Islandora?
+
−
I’m not working on Islandora - I’ll have to ask MikeC
+
;'''Is there any sort of authentication/authorization on the API?
 +
: This should support HTTP Basic Auth (which is a “SHOULD” in SWORD). At a minimum IP whitelisting.
 
LOCKSS itself relies on both whitelisting and HTTP Basic
 
LOCKSS itself relies on both whitelisting and HTTP Basic
  
−
Voting
+
;'''Voting
−
Each LOCKSS box initiates a ‘poll’ at random intervals bounded by two box-specific configuration settings:
+
:Each LOCKSS box initiates a ‘poll’ at random intervals bounded by two box-specific configuration settings:
  
 +
<pre>
 
  <property name="contentpoll.min" value="30m" />
 
  <property name="contentpoll.min" value="30m" />
−
      <property name="contentpoll.max" value="2d" />
+
<property name="contentpoll.max" value="2d" />
 +
</pre>
 +
 
 +
;'''How long does it take for 100% agreement on an Archival Unit (AU)?
 +
:Because each box recrawls AUs (looking for changes/additions) on its own schedule, AUs that are constantly changing likely won't achieve 100% agreement until they stop changing. If all the copies of the AU are the same when it settles down, polling will reach 100% agreement once all boxes have called and completed a poll. If the copies aren't identical (usually because of random crawl errors) it may take a couple cycles of polling to resolve the differences.
 +
 
 +
== Sample pointer files ==
 +
 
 +
The pointer file will be updated to track the current state of files in LOCKSS (more details at #6212).  Here are some sample pointer files representing different states of the package stored in LOCKSS.
 +
 
 +
=== Base pointer file ===
 +
 
 +
The pointer file as it arrives in the storage service, unmodified for LOCKSS.
 +
 
 +
<pre>
 +
<mets xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.loc.gov/METS/" xsi:schemaLocation="http://www.loc.gov/METS/ http://www.loc.gov/standards/mets/version18/mets.xsd">
 +
  <metsHdr CREATEDATE="2014-01-14T14:35:15"/>
 +
  <amdSec ID="amdSec_1">
 +
    <techMD ID="techMD_1">
 +
      <mdWrap MDTYPE="PREMIS:OBJECT">
 +
        <xmlData>
 +
          <object version="2.2" xmlns="info:lc/xmlns/premis-v2" xsi:type="file" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd">
 +
            <objectIdentifier>
 +
              <objectIdentifierType>UUID</objectIdentifierType>
 +
              <objectIdentifierValue>e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6</objectIdentifierValue>
 +
            </objectIdentifier>
 +
            <objectCharacteristics>
 +
              <compositionLevel>0</compositionLevel>
 +
              <fixity>
 +
                <messageDigestAlgorithm>sha256</messageDigestAlgorithm>
 +
                <messageDigest>7d4ede07b9288960d82c2ee146f6b4368d7dd065a99fae3523e3aa0d932bdb66</messageDigest>
 +
              </fixity>
 +
              <size>190989</size>
 +
              <format>
 +
                <formatDesignation>
 +
                  <formatName>7Zip format</formatName>
 +
                  <formatVersion/>
 +
                </formatDesignation>
 +
                <formatRegistry>
 +
                  <formatRegistryName>PRONOM</formatRegistryName>
 +
                  <formatRegistryKey>fmt/484</formatRegistryKey>
 +
                </formatRegistry>
 +
              </format>
 +
              <creatingApplication>
 +
                <creatingApplicationName>7-Zip</creatingApplicationName>
 +
                <creatingApplicationVersion>9.20</creatingApplicationVersion>
 +
                <dateCreatedByApplication>2014-01-14T14:35:15</dateCreatedByApplication>
 +
              </creatingApplication>
 +
            </objectCharacteristics>
 +
          </object>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </techMD>
 +
    <digiprovMD ID="digiprovMD_1">
 +
      <mdWrap MDTYPE="PREMIS:EVENT">
 +
        <xmlData>
 +
          <event xmlns="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
 +
            <eventIdentifier>
 +
              <eventIdentifierType>UUID</eventIdentifierType>
 +
              <eventIdentifierValue>7b16bct8-389x-4c0e-815c-98df287b234k</eventIdentifierValue>
 +
            </eventIdentifier>
 +
            <eventType>compression</eventType>
 +
            <eventDateTime>2014-01-14T22:31:22</eventDateTime>
 +
            <eventDetail><!-- information about tool--></eventDetail>
 +
            <eventOutcomeInformation>
 +
              <eventOutcome></eventOutcome>
 +
              <eventOutcomeDetail>
 +
                <eventOutcomeDetailNote><!--tool output --></eventOutcomeDetailNote>
 +
              </eventOutcomeDetail>
 +
            </eventOutcomeInformation>
 +
            <linkingAgentIdentifier>
 +
              <linkingAgentIdentifierType>preservation system</linkingAgentIdentifierType>
 +
              <linkingAgentIdentifierValue>Archivematica-1.0</linkingAgentIdentifierValue>
 +
            </linkingAgentIdentifier>
 +
          </event>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </digiprovMD>
 +
    <digiprovMD ID="digiprovMD_2">
 +
      <mdWrap MDTYPE="PREMIS:AGENT">
 +
        <xmlData>
 +
          <agent>
 +
            <agentIdentifier>
 +
              <agentIdentifierType>preservation system</agentIdentifierType>
 +
              <agentIdentifierValue>Archivematica-1.0</agentIdentifierValue>
 +
            </agentIdentifier>
 +
            <agentName>Archivematica</agentName>
 +
            <agentType>software</agentType>
 +
          </agent>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </digiprovMD>
 +
  </amdSec>
 +
  <fileSec>
 +
    <fileGrp USE="Archival Information Package">
 +
      <file ID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6">
 +
        <FLocat OTHERLOCTYPE="SYSTEM" LOCTYPE="OTHER" xlink:href="/filepath/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6.7z"/>
 +
        <transformFile TRANSFORMALGORITHM="bzip2" TRANSFORMORDER="1" TRANSFORMTYPE="decompression"/>
 +
      </file>
 +
    </fileGrp>
 +
  </fileSec>
 +
  <structMap TYPE="physical">
 +
    <div TYPE="Archival Information Package" ADMID="amdSec_1"> 
 +
      <fptr FILEID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6"/>
 +
    </div>
 +
  </structMap>
 +
</mets>
 +
</pre>
 +
 
 +
=== Package unsplit, local copy kept ===
 +
 
 +
The package was smaller than the max file size and therefore unsplit.  The package is fully stored, and the local copy was retained.
 +
 
 +
<pre>
 +
<mets xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.loc.gov/METS/" xsi:schemaLocation="http://www.loc.gov/METS/ http://www.loc.gov/standards/mets/version18/mets.xsd">
 +
  <metsHdr CREATEDATE="2014-01-14T14:35:15"/>
 +
  <amdSec ID="amdSec_1">
 +
    <techMD ID="techMD_1">
 +
      <mdWrap MDTYPE="PREMIS:OBJECT">
 +
        <xmlData>
 +
          <object version="2.2" xmlns="info:lc/xmlns/premis-v2" xsi:type="file" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd">
 +
            <objectIdentifier>
 +
              <objectIdentifierType>UUID</objectIdentifierType>
 +
              <objectIdentifierValue>e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6</objectIdentifierValue>
 +
            </objectIdentifier>
 +
            <objectCharacteristics>
 +
              <compositionLevel>1</compositionLevel>
 +
              <fixity>
 +
                <messageDigestAlgorithm>sha256</messageDigestAlgorithm>
 +
                <messageDigest>7d4ede07b9288960d82c2ee146f6b4368d7dd065a99fae3523e3aa0d932bdb66</messageDigest>
 +
              </fixity>
 +
              <size>190989</size>
 +
              <format>
 +
                <formatDesignation>
 +
                  <formatName>7Zip format</formatName>
 +
                  <formatVersion/>
 +
                </formatDesignation>
 +
                <formatRegistry>
 +
                  <formatRegistryName>PRONOM</formatRegistryName>
 +
                  <formatRegistryKey>fmt/484</formatRegistryKey>
 +
                </formatRegistry>
 +
              </format>
 +
              <creatingApplication>
 +
                <creatingApplicationName>7-Zip</creatingApplicationName>
 +
                <creatingApplicationVersion>9.20</creatingApplicationVersion>
 +
                <dateCreatedByApplication>2014-01-14T14:35:15</dateCreatedByApplication>
 +
              </creatingApplication>
 +
            </objectCharacteristics>
 +
          </object>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </techMD>
 +
    <digiprovMD ID="digiprovMD_1">
 +
      <mdWrap MDTYPE="PREMIS:EVENT">
 +
        <xmlData>
 +
          <event xmlns="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
 +
            <eventIdentifier>
 +
              <eventIdentifierType>UUID</eventIdentifierType>
 +
              <eventIdentifierValue>7b16bct8-389x-4c0e-815c-98df287b234k</eventIdentifierValue>
 +
            </eventIdentifier>
 +
            <eventType>compression</eventType>
 +
            <eventDateTime>2014-01-14T22:31:22</eventDateTime>
 +
            <eventDetail><!-- information about tool--></eventDetail>
 +
            <eventOutcomeInformation>
 +
              <eventOutcome></eventOutcome>
 +
              <eventOutcomeDetail>
 +
                <eventOutcomeDetailNote><!--tool output --></eventOutcomeDetailNote>
 +
              </eventOutcomeDetail>
 +
            </eventOutcomeInformation>
 +
            <linkingAgentIdentifier>
 +
              <linkingAgentIdentifierType>preservation system</linkingAgentIdentifierType>
 +
              <linkingAgentIdentifierValue>Archivematica-1.0</linkingAgentIdentifierValue>
 +
            </linkingAgentIdentifier>
 +
          </event>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </digiprovMD>
 +
    <digiprovMD ID="digiprovMD_2">
 +
      <mdWrap MDTYPE="PREMIS:AGENT">
 +
        <xmlData>
 +
          <agent>
 +
            <agentIdentifier>
 +
              <agentIdentifierType>preservation system</agentIdentifierType>
 +
              <agentIdentifierValue>Archivematica-1.0</agentIdentifierValue>
 +
            </agentIdentifier>
 +
            <agentName>Archivematica</agentName>
 +
            <agentType>software</agentType>
 +
          </agent>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </digiprovMD>
 +
  </amdSec>
 +
  <fileSec>
 +
    <fileGrp USE="Archival Information Package">
 +
      <file ID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6">
 +
        <FLocat OTHERLOCTYPE="SYSTEM" LOCTYPE="OTHER" xlink:href="/filepath/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6.7z"/>
 +
        <FLocat LOCTYPE="URL" xlink:href="http://lockss1.example.org:8083/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6.7z"/>
 +
        <transformFile TRANSFORMALGORITHM="bzip2" TRANSFORMORDER="1" TRANSFORMTYPE="decompression"/>
 +
      </file>
 +
    </fileGrp>
 +
  </fileSec>
 +
  <structMap TYPE="physical">
 +
    <div TYPE="Archival Information Package" ADMID="amdSec_1"> 
 +
      <fptr FILEID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6"/>
 +
    </div>
 +
  </structMap>
 +
</mets>
 +
</pre>
 +
 
 +
=== Package unsplit, local copy deleted ===
 +
 
 +
The package was smaller than the max file size and therefore unsplit.  The package is fully stored, and the local copy was deleted.
 +
 
 +
<pre>
 +
<mets xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.loc.gov/METS/" xsi:schemaLocation="http://www.loc.gov/METS/ http://www.loc.gov/standards/mets/version18/mets.xsd">
 +
  <metsHdr CREATEDATE="2014-01-14T14:35:15"/>
 +
  <amdSec ID="amdSec_1">
 +
    <techMD ID="techMD_1">
 +
      <mdWrap MDTYPE="PREMIS:OBJECT">
 +
        <xmlData>
 +
          <object version="2.2" xmlns="info:lc/xmlns/premis-v2" xsi:type="file" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd">
 +
            <objectIdentifier>
 +
              <objectIdentifierType>UUID</objectIdentifierType>
 +
              <objectIdentifierValue>e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6</objectIdentifierValue>
 +
            </objectIdentifier>
 +
            <objectCharacteristics>
 +
              <compositionLevel>1</compositionLevel>
 +
              <fixity>
 +
                <messageDigestAlgorithm>sha256</messageDigestAlgorithm>
 +
                <messageDigest>7d4ede07b9288960d82c2ee146f6b4368d7dd065a99fae3523e3aa0d932bdb66</messageDigest>
 +
              </fixity>
 +
              <size>190989</size>
 +
              <format>
 +
                <formatDesignation>
 +
                  <formatName>7Zip format</formatName>
 +
                  <formatVersion/>
 +
                </formatDesignation>
 +
                <formatRegistry>
 +
                  <formatRegistryName>PRONOM</formatRegistryName>
 +
                  <formatRegistryKey>fmt/484</formatRegistryKey>
 +
                </formatRegistry>
 +
              </format>
 +
              <creatingApplication>
 +
                <creatingApplicationName>7-Zip</creatingApplicationName>
 +
                <creatingApplicationVersion>9.20</creatingApplicationVersion>
 +
                <dateCreatedByApplication>2014-01-14T14:35:15</dateCreatedByApplication>
 +
              </creatingApplication>
 +
            </objectCharacteristics>
 +
          </object>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </techMD>
 +
    <digiprovMD ID="digiprovMD_1">
 +
      <mdWrap MDTYPE="PREMIS:EVENT">
 +
        <xmlData>
 +
          <event xmlns="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
 +
            <eventIdentifier>
 +
              <eventIdentifierType>UUID</eventIdentifierType>
 +
              <eventIdentifierValue>7b16bct8-389x-4c0e-815c-98df287b234k</eventIdentifierValue>
 +
            </eventIdentifier>
 +
            <eventType>compression</eventType>
 +
            <eventDateTime>2014-01-14T22:31:22</eventDateTime>
 +
            <eventDetail><!-- information about tool--></eventDetail>
 +
            <eventOutcomeInformation>
 +
              <eventOutcome></eventOutcome>
 +
              <eventOutcomeDetail>
 +
                <eventOutcomeDetailNote><!--tool output --></eventOutcomeDetailNote>
 +
              </eventOutcomeDetail>
 +
            </eventOutcomeInformation>
 +
            <linkingAgentIdentifier>
 +
              <linkingAgentIdentifierType>preservation system</linkingAgentIdentifierType>
 +
              <linkingAgentIdentifierValue>Archivematica-1.0</linkingAgentIdentifierValue>
 +
            </linkingAgentIdentifier>
 +
          </event>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </digiprovMD>
 +
    <digiprovMD ID="digiprovMD_2">
 +
      <mdWrap MDTYPE="PREMIS:EVENT">
 +
        <xmlData>
 +
          <event xmlns="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
 +
            <eventIdentifier>
 +
              <eventIdentifierType>UUID</eventIdentifierType>
 +
              <eventIdentifierValue>5e16bccb-384a-4c0e-815c-01df286b235b</eventIdentifierValue>
 +
            </eventIdentifier>
 +
            <eventType>deletion</eventType>
 +
            <eventDateTime>2014-01-14T22:33:48</eventDateTime>
 +
            <eventDetail></eventDetail>
 +
            <eventOutcomeInformation>
 +
              <eventOutcome></eventOutcome>
 +
              <eventOutcomeDetail>
 +
                <eventOutcomeDetailNote>AIP deleted from local storage</eventOutcomeDetailNote>
 +
              </eventOutcomeDetail>
 +
            </eventOutcomeInformation>
 +
            <linkingAgentIdentifier>
 +
              <linkingAgentIdentifierType>storage service</linkingAgentIdentifierType>
 +
              <linkingAgentIdentifierValue>Archivematica Storage Service-1.0</linkingAgentIdentifierValue>
 +
            </linkingAgentIdentifier>
 +
          </event>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </digiprovMD>
 +
    <digiprovMD ID="digiprovMD_3">
 +
      <mdWrap MDTYPE="PREMIS:AGENT">
 +
        <xmlData>
 +
          <agent>
 +
            <agentIdentifier>
 +
              <agentIdentifierType>preservation system</agentIdentifierType>
 +
              <agentIdentifierValue>Archivematica-1.0</agentIdentifierValue>
 +
            </agentIdentifier>
 +
            <agentName>Archivematica</agentName>
 +
            <agentType>software</agentType>
 +
          </agent>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </digiprovMD>
 +
    <digiprovMD ID="digiprovMD_4">
 +
      <mdWrap MDTYPE="PREMIS:AGENT">
 +
        <xmlData>
 +
          <agent>
 +
            <agentIdentifier>
 +
              <agentIdentifierType>storage service</agentIdentifierType>
 +
              <agentIdentifierValue>Archivematica Storage Service-1.0</agentIdentifierValue>
 +
            </agentIdentifier>
 +
            <agentName>Archivematica Storage Service</agentName>
 +
            <agentType>software</agentType>
 +
          </agent>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </digiprovMD>
 +
  </amdSec>
 +
  <fileSec>
 +
    <fileGrp USE="Archival Information Package">
 +
      <file ID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6">
 +
        <FLocat LOCTYPE="URL" xlink:href="http://lockss1.example.org:8083/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6.7z"/>
 +
        <transformFile TRANSFORMALGORITHM="bzip2" TRANSFORMORDER="1" TRANSFORMTYPE="decompression"/>
 +
      </file>
 +
    </fileGrp>
 +
  </fileSec>
 +
  <structMap TYPE="physical">
 +
    <div TYPE="Archival Information Package" ADMID="amdSec_1"> 
 +
      <fptr FILEID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6"/>
 +
    </div>
 +
  </structMap>
 +
</mets>
 +
</pre>
  
−
How long does it take for 100% agreement on an Archival Unit (AU)?
+
=== Package split, local copy kept ===
−
Tom Lipkis, LOCKSS lead developer, says:
 
  
−
"Because each box recrawls AUs (looking for changes/additions) on its own schedule, AUs that are constantly changing likely won't achieve 100% agreement until they stop changing.  We have some solutions for this in mind (the simplest of which is to exclude from polling files that appeared more recently than the recrawl interval), but at present monitoring systems shouldn't be alarmed at less than 100% agreement in this situation. If all the copies of the AU are the same when it settles down it will reach 100% agreement once all boxes have called and completed a poll [....] If the copies aren't identical (usually because of transient crawl errors) it may take a couple cycles of polling to resolve the differences."
+
The package was larger than the max file size and therefore split into several smaller chunks.  The package is fully stored, and the local copy was retained.
  
 +
<pre>
 +
<mets xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.loc.gov/METS/" xsi:schemaLocation="http://www.loc.gov/METS/ http://www.loc.gov/standards/mets/version18/mets.xsd">
 +
  <metsHdr CREATEDATE="2014-01-14T14:35:15"/>
 +
  <amdSec ID="amdSec_1">
 +
    <techMD ID="techMD_1">
 +
      <mdWrap MDTYPE="PREMIS:OBJECT">
 +
        <xmlData>
 +
          <object version="2.2" xmlns="info:lc/xmlns/premis-v2" xsi:type="file" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd">
 +
            <objectIdentifier>
 +
              <objectIdentifierType>UUID</objectIdentifierType>
 +
              <objectIdentifierValue>e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6</objectIdentifierValue>
 +
            </objectIdentifier>
 +
            <objectCharacteristics>
 +
              <compositionLevel>1</compositionLevel>
 +
              <fixity>
 +
                <messageDigestAlgorithm>sha256</messageDigestAlgorithm>
 +
                <messageDigest>7d4ede07b9288960d82c2ee146f6b4368d7dd065a99fae3523e3aa0d932bdb66</messageDigest>
 +
              </fixity>
 +
              <size>190989</size>
 +
              <format>
 +
                <formatDesignation>
 +
                  <formatName>7Zip format</formatName>
 +
                  <formatVersion/>
 +
                </formatDesignation>
 +
                <formatRegistry>
 +
                  <formatRegistryName>PRONOM</formatRegistryName>
 +
                  <formatRegistryKey>fmt/484</formatRegistryKey>
 +
                </formatRegistry>
 +
              </format>
 +
              <creatingApplication>
 +
                <creatingApplicationName>7-Zip</creatingApplicationName>
 +
                <creatingApplicationVersion>9.20</creatingApplicationVersion>
 +
                <dateCreatedByApplication>2014-01-14T14:35:15</dateCreatedByApplication>
 +
              </creatingApplication>
 +
            </objectCharacteristics>
 +
          </object>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </techMD>
 +
    <digiprovMD ID="digiprovMD_1">
 +
      <mdWrap MDTYPE="PREMIS:EVENT">
 +
        <xmlData>
 +
          <event xmlns="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
 +
            <eventIdentifier>
 +
              <eventIdentifierType>UUID</eventIdentifierType>
 +
              <eventIdentifierValue>7b16bct8-389x-4c0e-815c-98df287b234k</eventIdentifierValue>
 +
            </eventIdentifier>
 +
            <eventType>compression</eventType>
 +
            <eventDateTime>2014-01-14T22:31:22</eventDateTime>
 +
            <eventDetail><!-- information about tool--></eventDetail>
 +
            <eventOutcomeInformation>
 +
              <eventOutcome></eventOutcome>
 +
              <eventOutcomeDetail>
 +
                <eventOutcomeDetailNote><!--tool output --></eventOutcomeDetailNote>
 +
              </eventOutcomeDetail>
 +
            </eventOutcomeInformation>
 +
            <linkingAgentIdentifier>
 +
              <linkingAgentIdentifierType>preservation system</linkingAgentIdentifierType>
 +
              <linkingAgentIdentifierValue>Archivematica-1.0</linkingAgentIdentifierValue>
 +
            </linkingAgentIdentifier>
 +
          </event>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </digiprovMD>
 +
    <digiprovMD ID="digiprovMD_2">
 +
      <mdWrap MDTYPE="PREMIS:EVENT">
 +
        <xmlData>
 +
          <event xmlns="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
 +
            <eventIdentifier>
 +
              <eventIdentifierType>UUID</eventIdentifierType>
 +
              <eventIdentifierValue>5e16bccb-384a-4c0e-815c-01df286b235b</eventIdentifierValue>
 +
            </eventIdentifier>
 +
            <eventType>division</eventType>
 +
            <eventDateTime>2014-01-14T22:33:48</eventDateTime>
 +
            <eventDetail><!-- information about tool--></eventDetail>
 +
            <eventOutcomeInformation>
 +
              <eventOutcome></eventOutcome>
 +
              <eventOutcomeDetail>
 +
                <eventOutcomeDetailNote>3 LOCKSS chunks created</eventOutcomeDetailNote>
 +
              </eventOutcomeDetail>
 +
            </eventOutcomeInformation>
 +
            <linkingAgentIdentifier>
 +
              <linkingAgentIdentifierType>storage service</linkingAgentIdentifierType>
 +
              <linkingAgentIdentifierValue>Archivematica Storage Service-1.0</linkingAgentIdentifierValue>
 +
            </linkingAgentIdentifier>
 +
          </event>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </digiprovMD>
 +
    <digiprovMD ID="digiprovMD_3">
 +
      <mdWrap MDTYPE="PREMIS:AGENT">
 +
        <xmlData>
 +
          <agent>
 +
            <agentIdentifier>
 +
              <agentIdentifierType>preservation system</agentIdentifierType>
 +
              <agentIdentifierValue>Archivematica-1.0</agentIdentifierValue>
 +
            </agentIdentifier>
 +
            <agentName>Archivematica</agentName>
 +
            <agentType>software</agentType>
 +
          </agent>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </digiprovMD>
 +
    <digiprovMD ID="digiprovMD_4">
 +
      <mdWrap MDTYPE="PREMIS:AGENT">
 +
        <xmlData>
 +
          <agent>
 +
            <agentIdentifier>
 +
              <agentIdentifierType>storage service</agentIdentifierType>
 +
              <agentIdentifierValue>Archivematica Storage Service-1.0</agentIdentifierValue>
 +
            </agentIdentifier>
 +
            <agentName>Archivematica Storage Service</agentName>
 +
            <agentType>software</agentType>
 +
          </agent>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </digiprovMD>
 +
  </amdSec>
 +
  <fileSec>
 +
    <fileGrp USE="Archival Information Package">
 +
      <file ID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6">
 +
        <FLocat OTHERLOCTYPE="SYSTEM" LOCTYPE="OTHER" xlink:href="/filepath/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6.7z"/>
 +
        <transformFile TRANSFORMALGORITHM="bzip2" TRANSFORMORDER="1" TRANSFORMTYPE="decompression"/>
 +
      </file>
 +
    </fileGrp>
 +
    <fileGrp USE="LOCKSS chunk">
 +
      <file ID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_001" SIZE="19383838" CHECKSUM="8g4ede07b9288960d82c2ee146f6b4368dqad065a99fae3523e3aa0d932bda87" CHECKSUMTYPE="SHA-256">
 +
        <FLocat OTHERLOCTYPE="SYSTEM" LOCTYPE="OTHER" xlink:href="/filepath/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_001"/>       
 +
        <FLocat LOCTYPE="URL" xlink:href="http://lockss1.example.org:8083/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.001"/>
 +
        <FLocat LOCTYPE="URL" xlink:href="http://lockss2.example.org:8085/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.001"/>
 +
        <FLocat LOCTYPE="URL" xlink:href="http://lockss3.example.org:8080/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.001"/>
 +
    </file>
 +
    <file ID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_002" SIZE="19273844" CHECKSUM="9h4ede07b9288960d82c2ee146f6b4368hmad065a99fae3523e3aa0d932bda56" CHECKSUMTYPE="SHA-256">
 +
        <FLocat OTHERLOCTYPE="SYSTEM" LOCTYPE="OTHER" xlink:href="/filepath/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_002"/>       
 +
        <FLocat LOCTYPE="URL" xlink:href="http://lockss1.example.org:8083/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.002"/>
 +
        <FLocat LOCTYPE="URL" xlink:href="http://lockss2.example.org:8085/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.002"/>
 +
        <FLocat LOCTYPE="URL" xlink:href="http://lockss3.example.org:8080/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.002"/>
 +
    </file>
 +
    </fileGrp>
 +
  </fileSec>
 +
  <structMap TYPE="logical">
 +
    <div TYPE="Archival Information Package" ADMID="amdSec_1">
 +
      <div TYPE="Local copy"> 
 +
        <fptr FILEID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6"/>
 +
      </div>
 +
      <div TYPE="LOCKSS chunk" ORDER="1"> 
 +
        <fptr FILEID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_001"/>
 +
      </div>
 +
      <div TYPE="LOCKSS chunk" ORDER="2"> 
 +
        <fptr FILEID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_002"/>
 +
      </div>
 +
    </div>
 +
  </structMap>
 +
</mets>
 +
</pre>
  
 +
=== Package split, local copy deleted ===
  
 +
The package was larger than the max file size and therefore split into several smaller chunks.  The package is fully stored, and the local copy was deleted.
  
−
[[Category:Development documentation]]
+
<pre>
 +
<mets xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.loc.gov/METS/" xsi:schemaLocation="http://www.loc.gov/METS/ http://www.loc.gov/standards/mets/version18/mets.xsd">
 +
  <metsHdr CREATEDATE="2014-01-14T14:35:15"/>
 +
  <amdSec ID="amdSec_1">
 +
    <techMD ID="techMD_1">
 +
      <mdWrap MDTYPE="PREMIS:OBJECT">
 +
        <xmlData>
 +
          <object version="2.2" xmlns="info:lc/xmlns/premis-v2" xsi:type="file" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd">
 +
            <objectIdentifier>
 +
              <objectIdentifierType>UUID</objectIdentifierType>
 +
              <objectIdentifierValue>e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6</objectIdentifierValue>
 +
            </objectIdentifier>
 +
            <objectCharacteristics>
 +
              <compositionLevel>1</compositionLevel>
 +
              <fixity>
 +
                <messageDigestAlgorithm>sha256</messageDigestAlgorithm>
 +
                <messageDigest>7d4ede07b9288960d82c2ee146f6b4368d7dd065a99fae3523e3aa0d932bdb66</messageDigest>
 +
              </fixity>
 +
              <size>190989</size>
 +
              <format>
 +
                <formatDesignation>
 +
                  <formatName>7Zip format</formatName>
 +
                  <formatVersion/>
 +
                </formatDesignation>
 +
                <formatRegistry>
 +
                  <formatRegistryName>PRONOM</formatRegistryName>
 +
                  <formatRegistryKey>fmt/484</formatRegistryKey>
 +
                </formatRegistry>
 +
              </format>
 +
              <creatingApplication>
 +
                <creatingApplicationName>7-Zip</creatingApplicationName>
 +
                <creatingApplicationVersion>9.20</creatingApplicationVersion>
 +
                <dateCreatedByApplication>2014-01-14T14:35:15</dateCreatedByApplication>
 +
              </creatingApplication>
 +
            </objectCharacteristics>
 +
          </object>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </techMD>
 +
    <digiprovMD ID="digiprovMD_1">
 +
      <mdWrap MDTYPE="PREMIS:EVENT">
 +
        <xmlData>
 +
          <event xmlns="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
 +
            <eventIdentifier>
 +
              <eventIdentifierType>UUID</eventIdentifierType>
 +
              <eventIdentifierValue>7b16bct8-389x-4c0e-815c-98df287b234k</eventIdentifierValue>
 +
            </eventIdentifier>
 +
            <eventType>compression</eventType>
 +
            <eventDateTime>2014-01-14T22:31:22</eventDateTime>
 +
            <eventDetail><!-- information about tool--></eventDetail>
 +
            <eventOutcomeInformation>
 +
              <eventOutcome></eventOutcome>
 +
              <eventOutcomeDetail>
 +
                <eventOutcomeDetailNote><!--tool output --></eventOutcomeDetailNote>
 +
              </eventOutcomeDetail>
 +
            </eventOutcomeInformation>
 +
            <linkingAgentIdentifier>
 +
              <linkingAgentIdentifierType>preservation system</linkingAgentIdentifierType>
 +
              <linkingAgentIdentifierValue>Archivematica-1.0</linkingAgentIdentifierValue>
 +
            </linkingAgentIdentifier>
 +
          </event>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </digiprovMD>
 +
    <digiprovMD ID="digiprovMD_2">
 +
      <mdWrap MDTYPE="PREMIS:EVENT">
 +
        <xmlData>
 +
          <event xmlns="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
 +
            <eventIdentifier>
 +
              <eventIdentifierType>UUID</eventIdentifierType>
 +
              <eventIdentifierValue>5e16bccb-384a-4c0e-815c-01df286b235b</eventIdentifierValue>
 +
            </eventIdentifier>
 +
            <eventType>division</eventType>
 +
            <eventDateTime>2014-01-14T22:33:48</eventDateTime>
 +
            <eventDetail><!-- information about tool--></eventDetail>
 +
            <eventOutcomeInformation>
 +
              <eventOutcome>3 LOCKSS chunks created</eventOutcome>
 +
              <eventOutcomeDetail>
 +
                <eventOutcomeDetailNote><!--tool output --></eventOutcomeDetailNote>
 +
              </eventOutcomeDetail>
 +
            </eventOutcomeInformation>
 +
            <linkingAgentIdentifier>
 +
              <linkingAgentIdentifierType>storage service</linkingAgentIdentifierType>
 +
              <linkingAgentIdentifierValue>Archivematica Storage Service-1.0</linkingAgentIdentifierValue>
 +
            </linkingAgentIdentifier>
 +
          </event>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </digiprovMD>
 +
    <digiprovMD ID="digiprovMD_3">
 +
      <mdWrap MDTYPE="PREMIS:EVENT">
 +
        <xmlData>
 +
          <event xmlns="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
 +
            <eventIdentifier>
 +
              <eventIdentifierType>UUID</eventIdentifierType>
 +
              <eventIdentifierValue>5e16bccb-384a-4c0e-815c-01df286b235b</eventIdentifierValue>
 +
            </eventIdentifier>
 +
            <eventType>deletion</eventType>
 +
            <eventDateTime>2014-01-14T22:33:48</eventDateTime>
 +
            <eventDetail></eventDetail>
 +
            <eventOutcomeInformation>
 +
              <eventOutcome></eventOutcome>
 +
              <eventOutcomeDetail>
 +
                <eventOutcomeDetailNote>AIP deleted from local storage</eventOutcomeDetailNote>
 +
              </eventOutcomeDetail>
 +
            </eventOutcomeInformation>
 +
            <linkingAgentIdentifier>
 +
              <linkingAgentIdentifierType>storage service</linkingAgentIdentifierType>
 +
              <linkingAgentIdentifierValue>Archivematica Storage Service-1.0</linkingAgentIdentifierValue>
 +
            </linkingAgentIdentifier>
 +
          </event>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </digiprovMD>
 +
    <digiprovMD ID="digiprovMD_4">
 +
      <mdWrap MDTYPE="PREMIS:AGENT">
 +
        <xmlData>
 +
          <agent>
 +
            <agentIdentifier>
 +
              <agentIdentifierType>preservation system</agentIdentifierType>
 +
              <agentIdentifierValue>Archivematica-1.0</agentIdentifierValue>
 +
            </agentIdentifier>
 +
            <agentName>Archivematica</agentName>
 +
            <agentType>software</agentType>
 +
          </agent>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </digiprovMD>
 +
    <digiprovMD ID="digiprovMD_5">
 +
      <mdWrap MDTYPE="PREMIS:AGENT">
 +
        <xmlData>
 +
          <agent>
 +
            <agentIdentifier>
 +
              <agentIdentifierType>storage service</agentIdentifierType>
 +
              <agentIdentifierValue>Archivematica Storage Service-1.0</agentIdentifierValue>
 +
            </agentIdentifier>
 +
            <agentName>Archivematica Storage Service</agentName>
 +
            <agentType>software</agentType>
 +
          </agent>
 +
        </xmlData>
 +
      </mdWrap>
 +
    </digiprovMD>
 +
  </amdSec>
 +
  <fileSec>
 +
    <fileGrp USE="LOCKSS chunk">
 +
      <file ID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_001" SIZE="19383838" CHECKSUM="8g4ede07b9288960d82c2ee146f6b4368dqad065a99fae3523e3aa0d932bda87" CHECKSUMTYPE="SHA-256">
 +
        <FLocat LOCTYPE="URL" xlink:href="http://lockss1.example.org:8083/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.001"/>
 +
        <FLocat LOCTYPE="URL" xlink:href="http://lockss2.example.org:8085/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.001"/>
 +
        <FLocat LOCTYPE="URL" xlink:href="http://lockss3.example.org:8080/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.001"/>
 +
    </file>
 +
    <file ID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_002" SIZE="19273844" CHECKSUM="9h4ede07b9288960d82c2ee146f6b4368hmad065a99fae3523e3aa0d932bda56" CHECKSUMTYPE="SHA-256">
 +
        <FLocat LOCTYPE="URL" xlink:href="http://lockss1.example.org:8083/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.002"/>
 +
        <FLocat LOCTYPE="URL" xlink:href="http://lockss2.example.org:8085/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.002"/>
 +
        <FLocat LOCTYPE="URL" xlink:href="http://lockss3.example.org:8080/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.002"/>
 +
    </file>
 +
    </fileGrp>
 +
  </fileSec>
 +
  <structMap TYPE="logical">
 +
    <div TYPE="Archival Information Package" ADMID="amdSec_1">
 +
      <div TYPE="LOCKSS chunk" ORDER="1"> 
 +
        <fptr FILEID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_001"/>
 +
      </div>
 +
      <div TYPE="LOCKSS chunk" ORDER="2"> 
 +
        <fptr FILEID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_002"/>
 +
      </div>
 +
    </div>
 +
  </structMap>
 +
</mets>
 +
</pre>

Latest revision as of 16:25, 11 February 2020

Main Page > Development > Development documentation > LOCKSS Integration

This page is no longer being maintained and may contain inaccurate information. Please see the Archivematica documentation for up-to-date information.

Status[edit]

Sponsored: SFU Library
Development Status: Complete
Public Release: September 2014


Overview[edit]

LOCKSS (Lots of Copies Keeps Stuff Safe) http://www.lockss.org/ is treated by Archivematica as a storage sub-system. The Archivematica Storage Service can be configured to store AIPs in a LOCKSS network. The procedure for storing AIPs follows the SWORD v2 protocol: http://swordapp.org/sword-v2/ .

The Archivematica Storage Service contains an implementation of a SWORD client, which communicates with a SWORD server. The implementation of the SWORD Server is provided by LOCKSS-O-Matic https://github.com/mjordan/lockss-o-matic. LOCKSS-O-Matic is responsible for communicating with the Private LOCKSS Network (PLN)

Basic Workflow[edit]

  • Digital Objects are processed and packaged into AIPs in an Archivematica pipeline
  • Archivematica pipeline sends the final AIP to the Storage Service using the 'Store AIP' micro-service
  • Storage Service keeps a local copy of the AIP
  • Storage Service POSTs the list of files (e.g., a .7z file or parts of one created with split) that make up the AIP to LOCKSS-O-Matic. This list is in the form of an Atom document.
  • LOCKSS-O-Matic parses the list of files in the Atom document and registers them in its database.
  • LOCKSS-O-Matic edits the configuration files of all the LOCKSS boxes in the Private LOCKSS Network
  • LOCKSS boxes harvest the AIP files from the Storage Service
  • Storage Service polls LOCKSS-O-Matic (GETs the State document) to determine when the AIP has been stored in LOCKSS
  • If/When the Storage Service wants to remove its local copy of the AIP, it first POSTs a metadata update to LOCKSS-O-Matic
  • LOCKSS-O-Matic then updates the configuration files in LOCKSS boxes so they will stop harvesting the AIP

Technical Details[edit]

This document outlines a minimalist SWORD API where LOCKSS-O-Matic is the server, and the Archivematica Storage Service is the client. @todos and questions appear in throughout.

In the examples below, Archivematica is at http://archivematica.example.org and LOCKSS-O-Matic is at http://lockssomatic.example.org. The content file being managed by the SWORD deposit is an Archivematica AIP with the UUID dd3e3247-8466-4f2a-bb32-22a210cfce60. LOCKSS-O-Matic is managing a small Private LOCKSS Network containing two boxes, http://lockss1.example.org and http://lockss2.example.org.

Service Document[edit]

Archivematica issues a GET (with an HTTP header of ‘On-Behalf-Of’ with the value of the Archivematica instance’s content provider ID in the target LOM instance) to the SD-IRI (http://lockssomatic.example.org/api/sword/2.0/sd-iri). LOCKSS-O-Matic responds with a Service Document like:

On-Behalf-Of: 12 

<service xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:sword="http://purl.org/net/sword/terms/"
    xmlns:atom="http://www.w3.org/2005/Atom"
    xmlns:lom="http://lockssomatic.info/SWORD2"
    xmlns="http://www.w3.org/2007/app">

    <sword:version>2.0</sword:version>
    <!-- maxUploadSize is configurable in LOCKSS-O-Matic. →
    <!-- measured in kB, where kB = 1000*Byte -->
    <sword:maxUploadSize>102400</sword:maxUploadSize>
   
    <!-- uploadChecksumType chosen from a list - suggested: md5, sha1, sha256 -->
    <lom:uploadChecksumType>md5</lom:uploadChecksumType>

    <workspace>
        <atom:title>LOCKSS-O-Matic at Simon Fraser University</atom:title>     
        <!-- Each LOCKSS-O-Matic content provider will have its own SWORD collection; 
            in this case, http://archivematica.example.org  is collection 12. -->
        <collection href="http://lockssomatic.example.org/api/sword/2.0/col-iri/12">
            <atom:title>SFU Archivematica content provider</atom:title>
            <accept>application/atom_xml;type=entry</accept>
            <sword:mediation>true</sword:mediation>
         </collection>
    </workspace>
</service>


Creating a Resource with an Atom Entry[edit]

Archivematica issues a POST to the Col-IRI, ensuring that the <id> element in the Atom entry contains the UUID of the AIP, and that the URLs to the files to be preserved are listed in lom:content elements: 

<entry xmlns="http://www.w3.org/2005/Atom"    
        xmlns:dcterms="http://purl.org/dc/terms/"
        xmlns:lom="http://lockssomatic.info/SWORD2">
    <title>Some AIP</title>
    <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
    <updated>2013-10-07T17:17:08Z</updated>
    <author><name>Name of PREMIS agent owning AIP</name></author>
    <summary type="text">The AIP’s dc:description if it has one. If not, use a generic summary.</summary>
    <!-- size is measured in kB, where kB = 1000*Byte -->
    <lom:content size="102400" checksumType="md5" checksumValue="bd4a9b642562547754086de2dab26b7d">
        http://archivematica.example.org/archival-storage/download/aip/dd3e3247-8466-4f2a-bb32-22a210cfce60.001
    </lom:content>
    <lom:content size="46899" checksumType="md5" checksumValue="226190d94b21d1b0c7b1a42d855e419d">
        http://archivematica.example.org/archival-storage/download/aip/dd3e3247-8466-4f2a-bb32-22a210cfce60.002
    </lom:content>
</entry>

LOCKSS-O-Matic responds with a 201 Created and the Location header with the entry’s Edit-IRI, and the deposit receipt.

Deposit Receipt[edit]

Contains the Cont-IRI, the EM-IRI, and the State-IRI. 

<entry xmlns="http://www.w3.org/2005/Atom" xmlns:sword="http://purl.org/net/sword/">
    <sword:treatment>Stored in LOCKSS via LOCKSS-o-matic</sword:treatment>
    <content type="application/x-7z-compressed" src="http://lockssomatic.example.org/api/sword/2.0/cont-iri/12/dd3e3247-8466-4f2a-bb32-22a210cfce60" />

    <!-- EM-IRI. The EM-IRI and Cont-IRI can (and in LOCKSS-O-Matic, will) have the same value.-->
    <link rel="edit-media" href="http://lockssomatic.example.org/api/sword/2.0/cont-iri/12/dd3e3247-8466-4f2a-bb32-22a210cfce60" />

    <!-- SE-IRI (can be same as Edit-IRI) -->
    <!-- Archivematica will POST to this iri when deleting local content -->
    <link rel="http://purl.org/net/sword/terms/add" href="http://lockssomatic.example.org/api/sword/2.0/cont-iri/12/dd3e3247-8466-4f2a-bb32-22a210cfce60/edit" />
    <!-- Edit-IRI -->
    <link rel="edit" href="http://lockssomatic.example.org/api/sword/2.0/cont-iri/12/dd3e3247-8466-4f2a-bb32-22a210cfce60/edit" />
    <!-- In LOCKSS-O-Matic, the State-IRI will be the EM-IRI/Cont-IRI with the string ‘/state’ appended. -->
    <link rel="http://purl.org/net/sword/terms/statement" type="application/atom+xml;type=feed" 
      href="http://lockssomatic.example.org/api/sword/2.0/cont-IRI/12/dd3e3247-8466-4f2a-bb32-22a210cfce60/state" />
</entry>

SWORD Statement[edit]

State-IRI is defined in the deposit receipt. GET requests to the State-IRI for a resource will return an Atom feed as described below.

These state terms apply to the Allowed state term and message values, using the namespace http://lockssomatic.info/SWORD2, are:

Term Message (one of)
failed Content cannot be harvested by LOCKSS.
disagreement LOCKSS network is not in agreement on content checksums.
agreement LOCKSS network agrees internally on content checksums.

If LOCKSS-O-Matic reports ‘agreement’, Archivematica may delete the AIP from local storage.

Sample SWORD statement serialized as an Atom feed: 

<atom:feed xmlns:sword="http://purl.org/net/sword/terms/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:lom="http://lockssomatic.info/SWORD2">
  <atom:entry>
    <atom:category scheme="http://purl.org/net/sword/terms/" term="http://purl.org/net/sword/terms/originalDeposit" label="Orignal Deposit"/>

    <!-- This content node is for a chunk (i.e. a LOM content entry). -->
    <lom:content id="http://archivematica.example.org/archival-storage/download/aip/dd3e3247-8466-4f2a-bb32-22a210cfce60.001">
      <lom:serverlist>
        <lom:server id="1" state="agreement" src="http://lockss1.example.org:8083/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.001" checksumType="md5" checksumValue="bd4a9b642562547754086de2dab26b7d" />
        <lom:server id="2" state="failed" src="http://lockss2.example.org:8083/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.001" checksumType="md5" checksumValue="bd4a9b642562547754086de2dab26b7d" />
      </lom:serverlist>
    </lom:content>

    <lom:content id="http://archivematica.example.org/archival-storage/download/aip/dd3e3247-8466-4f2a-bb32-22a210cfce60.002">
      <lom:serverlist>
        <lom:server id="1" state="agreement" src="http://lockss1.example.org:8083/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce602.002" checksumType="md5" checksumValue="226190d94b21d1b0c7b1a42d855e419d" />
        <lom:server id="2" state="disagreement" src="http://lockss2.example.org:8083/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce602.002" checksumType="md5" checksumValue="226190d94b21d1b0c7b1a42d855e419d" />
       </lom:serverlist>
    </lom:content>

  </atom:entry>
</atom:feed>

Updating Metadata[edit]

Archivematica informs LOCKSS-O-Matic that it wants to delete the local copy of a file by PUTing to the SE-IRI an Atom document containing lom:content elements with a recrawl="false" attribute for each file. LOCKSS-O-Matic responds with a 202 Accepted and updates the LOCKSS configuration files to indicate that the AU the file is in should not be reharvested. After the configuration files have been updated, LOCKSS-O-Matic will respond with a 200 OK to the same request.

PUT SE-IRI 

 
<?xml version="1.0"?>
<entry xmlns="http://www.w3.org/2005/Atom" xmlns:dcterms="http://purl.org/dc/terms/" xmlns:lom="http://lockssomatic.info/SWORD2">
    <id>urn:uuid:dd3e3247-8466-4f2a-bb32-22a210cfce60</id>
    <lom:content recrawl="false">http://archivematica.example.org/archival-storage/download/aip/dd3e3247-8466-4f2a-bb32-
+22a210cfce60.001</lom:content>
    <lom:content recrawl="false">http://archivematica.example.org/archival-storage/download/aip/dd3e3247-8466-4f2a-bb32-22a210cfce60.002</lom:content>
</entry>

LOM can return:

  • - HTTP 202 (Accepted) meaning LOM is updating the LOCKSS config files saying not to harvest this, but it is not done yet.
  • - HTTP 200 (OK) meaning all config updates are complete
  • - HTTP 204 (No Content) if there is no matching AIP
  • - HTTP 409 (Conflict) There are files in the LOCKSS Archival Unit (AU) that do not have ‘recrawl=false’.

“Once the server has processed the request it MUST return an HTTP status code 200 (OK) or 204 (No Content), or an appropriate error code. “

Comments/questions[edit]

Can an AIP be deleted from LOCKSS-O-Matic? How?
Not really, since LOCKSS doesn’t let you delete content, not directly anyway (it can be done on the boxes’ command lines). In LOM we could have a flag that specifies that content should not be harvested into the LOCKSS network. In the Archivematica dashboard, an AIP deletion request can be made. This deletion request goes to the Storage Service, which would tell LOM to stop harvesting the AIP, and then delete the local copy. The copies in the PLN would not be deleted, and would have to be manually deleted by a LOCKSS administrator.
Can an AIP be retrieved from LOCKSS via the Archivematica Dashboard?
Yes - with the following caveat. All AIP's stored in the Storage Service are searchable via the Archival Storage tab in the dashboard. The Storage Service keeps a local copy of AIP's that are stored in LOCKSS, so when a user clicks on an AIP in the Archival Storage tab, that local copy will be delivered. If the local copy has been deleted (which will eventually happen, exactly when is based on Storage Service configuration settings) then the dashboard user will get a 'local copy not available, contact your storage administrator' message. From the Storage Service, it is possible for a storage administrator to click a button to download a copy of the AIP directly from one of the LOCKSS boxes, to the storage service. Once that is done, the AIP will then be retrievable from a dashboard.
Is there any sort of authentication/authorization on the API?
This should support HTTP Basic Auth (which is a “SHOULD” in SWORD). At a minimum IP whitelisting.

LOCKSS itself relies on both whitelisting and HTTP Basic

Voting
Each LOCKSS box initiates a ‘poll’ at random intervals bounded by two box-specific configuration settings:
 <property name="contentpoll.min" value="30m" />
 <property name="contentpoll.max" value="2d" />
How long does it take for 100% agreement on an Archival Unit (AU)?
Because each box recrawls AUs (looking for changes/additions) on its own schedule, AUs that are constantly changing likely won't achieve 100% agreement until they stop changing. If all the copies of the AU are the same when it settles down, polling will reach 100% agreement once all boxes have called and completed a poll. If the copies aren't identical (usually because of random crawl errors) it may take a couple cycles of polling to resolve the differences.

Sample pointer files[edit]

The pointer file will be updated to track the current state of files in LOCKSS (more details at #6212). Here are some sample pointer files representing different states of the package stored in LOCKSS.

Base pointer file[edit]

The pointer file as it arrives in the storage service, unmodified for LOCKSS. 

<mets xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.loc.gov/METS/" xsi:schemaLocation="http://www.loc.gov/METS/ http://www.loc.gov/standards/mets/version18/mets.xsd">
  <metsHdr CREATEDATE="2014-01-14T14:35:15"/>
  <amdSec ID="amdSec_1">
    <techMD ID="techMD_1">
      <mdWrap MDTYPE="PREMIS:OBJECT">
        <xmlData>
          <object version="2.2" xmlns="info:lc/xmlns/premis-v2" xsi:type="file" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd">
            <objectIdentifier>
              <objectIdentifierType>UUID</objectIdentifierType>
              <objectIdentifierValue>e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6</objectIdentifierValue>
            </objectIdentifier>
            <objectCharacteristics>
              <compositionLevel>0</compositionLevel>
              <fixity>
                <messageDigestAlgorithm>sha256</messageDigestAlgorithm>
                <messageDigest>7d4ede07b9288960d82c2ee146f6b4368d7dd065a99fae3523e3aa0d932bdb66</messageDigest>
              </fixity>
              <size>190989</size>
              <format>
                <formatDesignation>
                  <formatName>7Zip format</formatName>
                  <formatVersion/>
                </formatDesignation>
                <formatRegistry>
                  <formatRegistryName>PRONOM</formatRegistryName>
                  <formatRegistryKey>fmt/484</formatRegistryKey>
                </formatRegistry>
              </format>
              <creatingApplication>
                <creatingApplicationName>7-Zip</creatingApplicationName>
                <creatingApplicationVersion>9.20</creatingApplicationVersion>
                <dateCreatedByApplication>2014-01-14T14:35:15</dateCreatedByApplication>
              </creatingApplication>
            </objectCharacteristics>
          </object>
        </xmlData>
      </mdWrap>
    </techMD>
    <digiprovMD ID="digiprovMD_1">
      <mdWrap MDTYPE="PREMIS:EVENT">
        <xmlData>
          <event xmlns="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
            <eventIdentifier>
              <eventIdentifierType>UUID</eventIdentifierType>
              <eventIdentifierValue>7b16bct8-389x-4c0e-815c-98df287b234k</eventIdentifierValue>
            </eventIdentifier>
            <eventType>compression</eventType>
            <eventDateTime>2014-01-14T22:31:22</eventDateTime>
            <eventDetail><!-- information about tool--></eventDetail>
            <eventOutcomeInformation>
              <eventOutcome></eventOutcome>
              <eventOutcomeDetail>
                <eventOutcomeDetailNote><!--tool output --></eventOutcomeDetailNote>
              </eventOutcomeDetail>
            </eventOutcomeInformation>
            <linkingAgentIdentifier>
              <linkingAgentIdentifierType>preservation system</linkingAgentIdentifierType>
              <linkingAgentIdentifierValue>Archivematica-1.0</linkingAgentIdentifierValue>
            </linkingAgentIdentifier>
          </event>
        </xmlData>
      </mdWrap>
    </digiprovMD>
    <digiprovMD ID="digiprovMD_2">
      <mdWrap MDTYPE="PREMIS:AGENT">
        <xmlData>
          <agent>
            <agentIdentifier>
              <agentIdentifierType>preservation system</agentIdentifierType>
              <agentIdentifierValue>Archivematica-1.0</agentIdentifierValue>
            </agentIdentifier>
            <agentName>Archivematica</agentName>
            <agentType>software</agentType>
          </agent>
        </xmlData>
      </mdWrap>
    </digiprovMD>
  </amdSec>
  <fileSec>
    <fileGrp USE="Archival Information Package">
      <file ID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6">
        <FLocat OTHERLOCTYPE="SYSTEM" LOCTYPE="OTHER" xlink:href="/filepath/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6.7z"/>
        <transformFile TRANSFORMALGORITHM="bzip2" TRANSFORMORDER="1" TRANSFORMTYPE="decompression"/>
      </file>
    </fileGrp>
  </fileSec>
  <structMap TYPE="physical">
    <div TYPE="Archival Information Package" ADMID="amdSec_1">  
      <fptr FILEID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6"/>
    </div>
  </structMap>
</mets>

Package unsplit, local copy kept[edit]

The package was smaller than the max file size and therefore unsplit. The package is fully stored, and the local copy was retained. 

<mets xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.loc.gov/METS/" xsi:schemaLocation="http://www.loc.gov/METS/ http://www.loc.gov/standards/mets/version18/mets.xsd">
  <metsHdr CREATEDATE="2014-01-14T14:35:15"/>
  <amdSec ID="amdSec_1">
    <techMD ID="techMD_1">
      <mdWrap MDTYPE="PREMIS:OBJECT">
        <xmlData>
          <object version="2.2" xmlns="info:lc/xmlns/premis-v2" xsi:type="file" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd">
            <objectIdentifier>
              <objectIdentifierType>UUID</objectIdentifierType>
              <objectIdentifierValue>e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6</objectIdentifierValue>
            </objectIdentifier>
            <objectCharacteristics>
              <compositionLevel>1</compositionLevel>
              <fixity>
                <messageDigestAlgorithm>sha256</messageDigestAlgorithm>
                <messageDigest>7d4ede07b9288960d82c2ee146f6b4368d7dd065a99fae3523e3aa0d932bdb66</messageDigest>
              </fixity>
              <size>190989</size>
              <format>
                <formatDesignation>
                  <formatName>7Zip format</formatName>
                  <formatVersion/>
                </formatDesignation>
                <formatRegistry>
                  <formatRegistryName>PRONOM</formatRegistryName>
                  <formatRegistryKey>fmt/484</formatRegistryKey>
                </formatRegistry>
              </format>
              <creatingApplication>
                <creatingApplicationName>7-Zip</creatingApplicationName>
                <creatingApplicationVersion>9.20</creatingApplicationVersion>
                <dateCreatedByApplication>2014-01-14T14:35:15</dateCreatedByApplication>
              </creatingApplication>
            </objectCharacteristics>
          </object>
        </xmlData>
      </mdWrap>
    </techMD>
    <digiprovMD ID="digiprovMD_1">
      <mdWrap MDTYPE="PREMIS:EVENT">
        <xmlData>
          <event xmlns="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
            <eventIdentifier>
              <eventIdentifierType>UUID</eventIdentifierType>
              <eventIdentifierValue>7b16bct8-389x-4c0e-815c-98df287b234k</eventIdentifierValue>
            </eventIdentifier>
            <eventType>compression</eventType>
            <eventDateTime>2014-01-14T22:31:22</eventDateTime>
            <eventDetail><!-- information about tool--></eventDetail>
            <eventOutcomeInformation>
              <eventOutcome></eventOutcome>
              <eventOutcomeDetail>
                <eventOutcomeDetailNote><!--tool output --></eventOutcomeDetailNote>
              </eventOutcomeDetail>
            </eventOutcomeInformation>
            <linkingAgentIdentifier>
              <linkingAgentIdentifierType>preservation system</linkingAgentIdentifierType>
              <linkingAgentIdentifierValue>Archivematica-1.0</linkingAgentIdentifierValue>
            </linkingAgentIdentifier>
          </event>
        </xmlData>
      </mdWrap>
    </digiprovMD>
    <digiprovMD ID="digiprovMD_2">
      <mdWrap MDTYPE="PREMIS:AGENT">
        <xmlData>
          <agent>
            <agentIdentifier>
              <agentIdentifierType>preservation system</agentIdentifierType>
              <agentIdentifierValue>Archivematica-1.0</agentIdentifierValue>
            </agentIdentifier>
            <agentName>Archivematica</agentName>
            <agentType>software</agentType>
          </agent>
        </xmlData>
      </mdWrap>
    </digiprovMD>
  </amdSec>
  <fileSec>
    <fileGrp USE="Archival Information Package">
      <file ID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6">
        <FLocat OTHERLOCTYPE="SYSTEM" LOCTYPE="OTHER" xlink:href="/filepath/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6.7z"/>
        <FLocat LOCTYPE="URL" xlink:href="http://lockss1.example.org:8083/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6.7z"/>
        <transformFile TRANSFORMALGORITHM="bzip2" TRANSFORMORDER="1" TRANSFORMTYPE="decompression"/>
      </file>
    </fileGrp>
  </fileSec>
  <structMap TYPE="physical">
    <div TYPE="Archival Information Package" ADMID="amdSec_1">  
      <fptr FILEID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6"/>
    </div>
  </structMap>
</mets>

Package unsplit, local copy deleted[edit]

The package was smaller than the max file size and therefore unsplit. The package is fully stored, and the local copy was deleted. 

<mets xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.loc.gov/METS/" xsi:schemaLocation="http://www.loc.gov/METS/ http://www.loc.gov/standards/mets/version18/mets.xsd">
  <metsHdr CREATEDATE="2014-01-14T14:35:15"/>
  <amdSec ID="amdSec_1">
    <techMD ID="techMD_1">
      <mdWrap MDTYPE="PREMIS:OBJECT">
        <xmlData>
          <object version="2.2" xmlns="info:lc/xmlns/premis-v2" xsi:type="file" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd">
            <objectIdentifier>
              <objectIdentifierType>UUID</objectIdentifierType>
              <objectIdentifierValue>e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6</objectIdentifierValue>
            </objectIdentifier>
            <objectCharacteristics>
              <compositionLevel>1</compositionLevel>
              <fixity>
                <messageDigestAlgorithm>sha256</messageDigestAlgorithm>
                <messageDigest>7d4ede07b9288960d82c2ee146f6b4368d7dd065a99fae3523e3aa0d932bdb66</messageDigest>
              </fixity>
              <size>190989</size>
              <format>
                <formatDesignation>
                  <formatName>7Zip format</formatName>
                  <formatVersion/>
                </formatDesignation>
                <formatRegistry>
                  <formatRegistryName>PRONOM</formatRegistryName>
                  <formatRegistryKey>fmt/484</formatRegistryKey>
                </formatRegistry>
              </format>
              <creatingApplication>
                <creatingApplicationName>7-Zip</creatingApplicationName>
                <creatingApplicationVersion>9.20</creatingApplicationVersion>
                <dateCreatedByApplication>2014-01-14T14:35:15</dateCreatedByApplication>
              </creatingApplication>
            </objectCharacteristics>
          </object>
        </xmlData>
      </mdWrap>
    </techMD>
    <digiprovMD ID="digiprovMD_1">
      <mdWrap MDTYPE="PREMIS:EVENT">
        <xmlData>
          <event xmlns="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
            <eventIdentifier>
              <eventIdentifierType>UUID</eventIdentifierType>
              <eventIdentifierValue>7b16bct8-389x-4c0e-815c-98df287b234k</eventIdentifierValue>
            </eventIdentifier>
            <eventType>compression</eventType>
            <eventDateTime>2014-01-14T22:31:22</eventDateTime>
            <eventDetail><!-- information about tool--></eventDetail>
            <eventOutcomeInformation>
              <eventOutcome></eventOutcome>
              <eventOutcomeDetail>
                <eventOutcomeDetailNote><!--tool output --></eventOutcomeDetailNote>
              </eventOutcomeDetail>
            </eventOutcomeInformation>
            <linkingAgentIdentifier>
              <linkingAgentIdentifierType>preservation system</linkingAgentIdentifierType>
              <linkingAgentIdentifierValue>Archivematica-1.0</linkingAgentIdentifierValue>
            </linkingAgentIdentifier>
          </event>
        </xmlData>
      </mdWrap>
    </digiprovMD>
    <digiprovMD ID="digiprovMD_2">
      <mdWrap MDTYPE="PREMIS:EVENT">
        <xmlData>
          <event xmlns="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
            <eventIdentifier>
              <eventIdentifierType>UUID</eventIdentifierType>
              <eventIdentifierValue>5e16bccb-384a-4c0e-815c-01df286b235b</eventIdentifierValue>
            </eventIdentifier>
            <eventType>deletion</eventType>
            <eventDateTime>2014-01-14T22:33:48</eventDateTime>
            <eventDetail></eventDetail>
            <eventOutcomeInformation>
              <eventOutcome></eventOutcome>
              <eventOutcomeDetail>
                <eventOutcomeDetailNote>AIP deleted from local storage</eventOutcomeDetailNote>
              </eventOutcomeDetail>
            </eventOutcomeInformation>
            <linkingAgentIdentifier>
              <linkingAgentIdentifierType>storage service</linkingAgentIdentifierType>
              <linkingAgentIdentifierValue>Archivematica Storage Service-1.0</linkingAgentIdentifierValue>
            </linkingAgentIdentifier>
          </event>
        </xmlData>
      </mdWrap>
    </digiprovMD>
    <digiprovMD ID="digiprovMD_3">
      <mdWrap MDTYPE="PREMIS:AGENT">
        <xmlData>
          <agent>
            <agentIdentifier>
              <agentIdentifierType>preservation system</agentIdentifierType>
              <agentIdentifierValue>Archivematica-1.0</agentIdentifierValue>
            </agentIdentifier>
            <agentName>Archivematica</agentName>
            <agentType>software</agentType>
          </agent>
        </xmlData>
      </mdWrap>
    </digiprovMD>
    <digiprovMD ID="digiprovMD_4">
      <mdWrap MDTYPE="PREMIS:AGENT">
        <xmlData>
          <agent>
            <agentIdentifier>
              <agentIdentifierType>storage service</agentIdentifierType>
              <agentIdentifierValue>Archivematica Storage Service-1.0</agentIdentifierValue>
            </agentIdentifier>
            <agentName>Archivematica Storage Service</agentName>
            <agentType>software</agentType>
          </agent>
        </xmlData>
      </mdWrap>
    </digiprovMD>
  </amdSec>
  <fileSec>
    <fileGrp USE="Archival Information Package">
      <file ID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6">
        <FLocat LOCTYPE="URL" xlink:href="http://lockss1.example.org:8083/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6.7z"/>
        <transformFile TRANSFORMALGORITHM="bzip2" TRANSFORMORDER="1" TRANSFORMTYPE="decompression"/>
      </file>
    </fileGrp>
  </fileSec>
  <structMap TYPE="physical">
    <div TYPE="Archival Information Package" ADMID="amdSec_1">  
      <fptr FILEID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6"/>
    </div>
  </structMap>
</mets>

Package split, local copy kept[edit]

The package was larger than the max file size and therefore split into several smaller chunks. The package is fully stored, and the local copy was retained. 

<mets xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.loc.gov/METS/" xsi:schemaLocation="http://www.loc.gov/METS/ http://www.loc.gov/standards/mets/version18/mets.xsd">
  <metsHdr CREATEDATE="2014-01-14T14:35:15"/>
  <amdSec ID="amdSec_1">
    <techMD ID="techMD_1">
      <mdWrap MDTYPE="PREMIS:OBJECT">
        <xmlData>
          <object version="2.2" xmlns="info:lc/xmlns/premis-v2" xsi:type="file" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd">
            <objectIdentifier>
              <objectIdentifierType>UUID</objectIdentifierType>
              <objectIdentifierValue>e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6</objectIdentifierValue>
            </objectIdentifier>
            <objectCharacteristics>
              <compositionLevel>1</compositionLevel>
              <fixity>
                <messageDigestAlgorithm>sha256</messageDigestAlgorithm>
                <messageDigest>7d4ede07b9288960d82c2ee146f6b4368d7dd065a99fae3523e3aa0d932bdb66</messageDigest>
              </fixity>
              <size>190989</size>
              <format>
                <formatDesignation>
                  <formatName>7Zip format</formatName>
                  <formatVersion/>
                </formatDesignation>
                <formatRegistry>
                  <formatRegistryName>PRONOM</formatRegistryName>
                  <formatRegistryKey>fmt/484</formatRegistryKey>
                </formatRegistry>
              </format>
              <creatingApplication>
                <creatingApplicationName>7-Zip</creatingApplicationName>
                <creatingApplicationVersion>9.20</creatingApplicationVersion>
                <dateCreatedByApplication>2014-01-14T14:35:15</dateCreatedByApplication>
              </creatingApplication>
            </objectCharacteristics>
          </object>
        </xmlData>
      </mdWrap>
    </techMD>
    <digiprovMD ID="digiprovMD_1">
      <mdWrap MDTYPE="PREMIS:EVENT">
        <xmlData>
          <event xmlns="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
            <eventIdentifier>
              <eventIdentifierType>UUID</eventIdentifierType>
              <eventIdentifierValue>7b16bct8-389x-4c0e-815c-98df287b234k</eventIdentifierValue>
            </eventIdentifier>
            <eventType>compression</eventType>
            <eventDateTime>2014-01-14T22:31:22</eventDateTime>
            <eventDetail><!-- information about tool--></eventDetail>
            <eventOutcomeInformation>
              <eventOutcome></eventOutcome>
              <eventOutcomeDetail>
                <eventOutcomeDetailNote><!--tool output --></eventOutcomeDetailNote>
              </eventOutcomeDetail>
            </eventOutcomeInformation>
            <linkingAgentIdentifier>
              <linkingAgentIdentifierType>preservation system</linkingAgentIdentifierType>
              <linkingAgentIdentifierValue>Archivematica-1.0</linkingAgentIdentifierValue>
            </linkingAgentIdentifier>
          </event>
        </xmlData>
      </mdWrap>
    </digiprovMD>
    <digiprovMD ID="digiprovMD_2">
      <mdWrap MDTYPE="PREMIS:EVENT">
        <xmlData>
          <event xmlns="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
            <eventIdentifier>
              <eventIdentifierType>UUID</eventIdentifierType>
              <eventIdentifierValue>5e16bccb-384a-4c0e-815c-01df286b235b</eventIdentifierValue>
            </eventIdentifier>
            <eventType>division</eventType>
            <eventDateTime>2014-01-14T22:33:48</eventDateTime>
            <eventDetail><!-- information about tool--></eventDetail>
            <eventOutcomeInformation>
              <eventOutcome></eventOutcome>
              <eventOutcomeDetail>
                <eventOutcomeDetailNote>3 LOCKSS chunks created</eventOutcomeDetailNote>
              </eventOutcomeDetail>
            </eventOutcomeInformation>
            <linkingAgentIdentifier>
              <linkingAgentIdentifierType>storage service</linkingAgentIdentifierType>
              <linkingAgentIdentifierValue>Archivematica Storage Service-1.0</linkingAgentIdentifierValue>
            </linkingAgentIdentifier>
          </event>
        </xmlData>
      </mdWrap>
    </digiprovMD>
    <digiprovMD ID="digiprovMD_3">
      <mdWrap MDTYPE="PREMIS:AGENT">
        <xmlData>
          <agent>
            <agentIdentifier>
              <agentIdentifierType>preservation system</agentIdentifierType>
              <agentIdentifierValue>Archivematica-1.0</agentIdentifierValue>
            </agentIdentifier>
            <agentName>Archivematica</agentName>
            <agentType>software</agentType>
          </agent>
        </xmlData>
      </mdWrap>
    </digiprovMD>
    <digiprovMD ID="digiprovMD_4">
      <mdWrap MDTYPE="PREMIS:AGENT">
        <xmlData>
          <agent>
            <agentIdentifier>
              <agentIdentifierType>storage service</agentIdentifierType>
              <agentIdentifierValue>Archivematica Storage Service-1.0</agentIdentifierValue>
            </agentIdentifier>
            <agentName>Archivematica Storage Service</agentName>
            <agentType>software</agentType>
          </agent>
        </xmlData>
      </mdWrap>
    </digiprovMD>
  </amdSec>
  <fileSec>
    <fileGrp USE="Archival Information Package">
      <file ID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6">
        <FLocat OTHERLOCTYPE="SYSTEM" LOCTYPE="OTHER" xlink:href="/filepath/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6.7z"/>
        <transformFile TRANSFORMALGORITHM="bzip2" TRANSFORMORDER="1" TRANSFORMTYPE="decompression"/>
      </file>
    </fileGrp>
    <fileGrp USE="LOCKSS chunk">
      <file ID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_001" SIZE="19383838" CHECKSUM="8g4ede07b9288960d82c2ee146f6b4368dqad065a99fae3523e3aa0d932bda87" CHECKSUMTYPE="SHA-256">
        <FLocat OTHERLOCTYPE="SYSTEM" LOCTYPE="OTHER" xlink:href="/filepath/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_001"/>        
        <FLocat LOCTYPE="URL" xlink:href="http://lockss1.example.org:8083/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.001"/>
        <FLocat LOCTYPE="URL" xlink:href="http://lockss2.example.org:8085/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.001"/>
        <FLocat LOCTYPE="URL" xlink:href="http://lockss3.example.org:8080/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.001"/>
     </file>
     <file ID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_002" SIZE="19273844" CHECKSUM="9h4ede07b9288960d82c2ee146f6b4368hmad065a99fae3523e3aa0d932bda56" CHECKSUMTYPE="SHA-256">
        <FLocat OTHERLOCTYPE="SYSTEM" LOCTYPE="OTHER" xlink:href="/filepath/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_002"/>        
        <FLocat LOCTYPE="URL" xlink:href="http://lockss1.example.org:8083/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.002"/>
        <FLocat LOCTYPE="URL" xlink:href="http://lockss2.example.org:8085/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.002"/>
        <FLocat LOCTYPE="URL" xlink:href="http://lockss3.example.org:8080/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.002"/>
     </file>
    </fileGrp>
  </fileSec>
  <structMap TYPE="logical">
    <div TYPE="Archival Information Package" ADMID="amdSec_1">
      <div TYPE="Local copy">   
        <fptr FILEID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6"/>
      </div>
      <div TYPE="LOCKSS chunk" ORDER="1">   
        <fptr FILEID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_001"/>
      </div>
      <div TYPE="LOCKSS chunk" ORDER="2">   
        <fptr FILEID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_002"/>
      </div>
    </div>
  </structMap>
</mets>

Package split, local copy deleted[edit]

The package was larger than the max file size and therefore split into several smaller chunks. The package is fully stored, and the local copy was deleted. 

<mets xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.loc.gov/METS/" xsi:schemaLocation="http://www.loc.gov/METS/ http://www.loc.gov/standards/mets/version18/mets.xsd">
  <metsHdr CREATEDATE="2014-01-14T14:35:15"/>
  <amdSec ID="amdSec_1">
    <techMD ID="techMD_1">
      <mdWrap MDTYPE="PREMIS:OBJECT">
        <xmlData>
          <object version="2.2" xmlns="info:lc/xmlns/premis-v2" xsi:type="file" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd">
            <objectIdentifier>
              <objectIdentifierType>UUID</objectIdentifierType>
              <objectIdentifierValue>e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6</objectIdentifierValue>
            </objectIdentifier>
            <objectCharacteristics>
              <compositionLevel>1</compositionLevel>
              <fixity>
                <messageDigestAlgorithm>sha256</messageDigestAlgorithm>
                <messageDigest>7d4ede07b9288960d82c2ee146f6b4368d7dd065a99fae3523e3aa0d932bdb66</messageDigest>
              </fixity>
              <size>190989</size>
              <format>
                <formatDesignation>
                  <formatName>7Zip format</formatName>
                  <formatVersion/>
                </formatDesignation>
                <formatRegistry>
                  <formatRegistryName>PRONOM</formatRegistryName>
                  <formatRegistryKey>fmt/484</formatRegistryKey>
                </formatRegistry>
              </format>
              <creatingApplication>
                <creatingApplicationName>7-Zip</creatingApplicationName>
                <creatingApplicationVersion>9.20</creatingApplicationVersion>
                <dateCreatedByApplication>2014-01-14T14:35:15</dateCreatedByApplication>
              </creatingApplication>
            </objectCharacteristics>
          </object>
        </xmlData>
      </mdWrap>
    </techMD>
    <digiprovMD ID="digiprovMD_1">
      <mdWrap MDTYPE="PREMIS:EVENT">
        <xmlData>
          <event xmlns="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
            <eventIdentifier>
              <eventIdentifierType>UUID</eventIdentifierType>
              <eventIdentifierValue>7b16bct8-389x-4c0e-815c-98df287b234k</eventIdentifierValue>
            </eventIdentifier>
            <eventType>compression</eventType>
            <eventDateTime>2014-01-14T22:31:22</eventDateTime>
            <eventDetail><!-- information about tool--></eventDetail>
            <eventOutcomeInformation>
              <eventOutcome></eventOutcome>
              <eventOutcomeDetail>
                <eventOutcomeDetailNote><!--tool output --></eventOutcomeDetailNote>
              </eventOutcomeDetail>
            </eventOutcomeInformation>
            <linkingAgentIdentifier>
              <linkingAgentIdentifierType>preservation system</linkingAgentIdentifierType>
              <linkingAgentIdentifierValue>Archivematica-1.0</linkingAgentIdentifierValue>
            </linkingAgentIdentifier>
          </event>
        </xmlData>
      </mdWrap>
    </digiprovMD>
    <digiprovMD ID="digiprovMD_2">
      <mdWrap MDTYPE="PREMIS:EVENT">
        <xmlData>
          <event xmlns="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
            <eventIdentifier>
              <eventIdentifierType>UUID</eventIdentifierType>
              <eventIdentifierValue>5e16bccb-384a-4c0e-815c-01df286b235b</eventIdentifierValue>
            </eventIdentifier>
            <eventType>division</eventType>
            <eventDateTime>2014-01-14T22:33:48</eventDateTime>
            <eventDetail><!-- information about tool--></eventDetail>
            <eventOutcomeInformation>
              <eventOutcome>3 LOCKSS chunks created</eventOutcome>
              <eventOutcomeDetail>
                <eventOutcomeDetailNote><!--tool output --></eventOutcomeDetailNote>
              </eventOutcomeDetail>
            </eventOutcomeInformation>
            <linkingAgentIdentifier>
              <linkingAgentIdentifierType>storage service</linkingAgentIdentifierType>
              <linkingAgentIdentifierValue>Archivematica Storage Service-1.0</linkingAgentIdentifierValue>
            </linkingAgentIdentifier>
          </event>
        </xmlData>
      </mdWrap>
    </digiprovMD>
    <digiprovMD ID="digiprovMD_3">
      <mdWrap MDTYPE="PREMIS:EVENT">
        <xmlData>
          <event xmlns="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
            <eventIdentifier>
              <eventIdentifierType>UUID</eventIdentifierType>
              <eventIdentifierValue>5e16bccb-384a-4c0e-815c-01df286b235b</eventIdentifierValue>
            </eventIdentifier>
            <eventType>deletion</eventType>
            <eventDateTime>2014-01-14T22:33:48</eventDateTime>
            <eventDetail></eventDetail>
            <eventOutcomeInformation>
              <eventOutcome></eventOutcome>
              <eventOutcomeDetail>
                <eventOutcomeDetailNote>AIP deleted from local storage</eventOutcomeDetailNote>
              </eventOutcomeDetail>
            </eventOutcomeInformation>
            <linkingAgentIdentifier>
              <linkingAgentIdentifierType>storage service</linkingAgentIdentifierType>
              <linkingAgentIdentifierValue>Archivematica Storage Service-1.0</linkingAgentIdentifierValue>
            </linkingAgentIdentifier>
          </event>
        </xmlData>
      </mdWrap>
    </digiprovMD>
    <digiprovMD ID="digiprovMD_4">
      <mdWrap MDTYPE="PREMIS:AGENT">
        <xmlData>
          <agent>
            <agentIdentifier>
              <agentIdentifierType>preservation system</agentIdentifierType>
              <agentIdentifierValue>Archivematica-1.0</agentIdentifierValue>
            </agentIdentifier>
            <agentName>Archivematica</agentName>
            <agentType>software</agentType>
          </agent>
        </xmlData>
      </mdWrap>
    </digiprovMD>
    <digiprovMD ID="digiprovMD_5">
      <mdWrap MDTYPE="PREMIS:AGENT">
        <xmlData>
          <agent>
            <agentIdentifier>
              <agentIdentifierType>storage service</agentIdentifierType>
              <agentIdentifierValue>Archivematica Storage Service-1.0</agentIdentifierValue>
            </agentIdentifier>
            <agentName>Archivematica Storage Service</agentName>
            <agentType>software</agentType>
          </agent>
        </xmlData>
      </mdWrap>
    </digiprovMD>
  </amdSec>
  <fileSec>
    <fileGrp USE="LOCKSS chunk">
      <file ID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_001" SIZE="19383838" CHECKSUM="8g4ede07b9288960d82c2ee146f6b4368dqad065a99fae3523e3aa0d932bda87" CHECKSUMTYPE="SHA-256">
        <FLocat LOCTYPE="URL" xlink:href="http://lockss1.example.org:8083/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.001"/>
        <FLocat LOCTYPE="URL" xlink:href="http://lockss2.example.org:8085/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.001"/>
        <FLocat LOCTYPE="URL" xlink:href="http://lockss3.example.org:8080/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.001"/>
     </file>
     <file ID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_002" SIZE="19273844" CHECKSUM="9h4ede07b9288960d82c2ee146f6b4368hmad065a99fae3523e3aa0d932bda56" CHECKSUMTYPE="SHA-256">
        <FLocat LOCTYPE="URL" xlink:href="http://lockss1.example.org:8083/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.002"/>
        <FLocat LOCTYPE="URL" xlink:href="http://lockss2.example.org:8085/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.002"/>
        <FLocat LOCTYPE="URL" xlink:href="http://lockss3.example.org:8080/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/dd3e3247-8466-4f2a-bb32-22a210cfce60.002"/>
     </file>
    </fileGrp>
  </fileSec>
  <structMap TYPE="logical">
    <div TYPE="Archival Information Package" ADMID="amdSec_1">
      <div TYPE="LOCKSS chunk" ORDER="1">   
        <fptr FILEID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_001"/>
      </div>
      <div TYPE="LOCKSS chunk" ORDER="2">   
        <fptr FILEID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6_002"/>
      </div>
    </div>
  </structMap>
</mets>
Retrieved from "https://wiki.archivematica.org/index.php?title=LOCKSS_Integration&oldid=13381"