Difference between revisions of "AIP pointer file"

From Archivematica
Jump to navigation Jump to search
m
(Added information about purpose)
 
(4 intermediate revisions by the same user not shown)
Line 1: Line 1:
Documentation in progress
+
Pointer files have been used to help provide sustainable access to compressed AIPs since Archivematica 1.0. [https://groups.google.com/d/msg/archivematica/VPQGd4s7hI8/x3WFdqO9AQAJ [1]]. The documentation about them, below, is currently in progress.
 +
 
 +
== Purpose ==
 +
 
 +
Pointer files can help users ascertain information about their AIPs without needing to consult the Storage Service API or database.
 +
 
 +
As an example, pointer files include:
 +
 
 +
* Information about compression, currently used internally by Archivematica itself for Re-ingest or unpacking for delivery via the API
 +
* Information about encryption so that AIPs can be recovered if encrypted.
 +
* Information about where the AIP is stored relative to the storage service. NB. there are some variations, see below.
 +
 
 +
=== Describing the location of the AIP ===
 +
 
 +
Information about the location of an AIP is stored in a <pre>mets:filesec</pre> inside the pointer file:
 +
 
 +
<pre>
 +
  <mets:fileSec>
 +
    <mets:fileGrp USE="Archival Information Package">
 +
      <mets:file ID="rs_s3-cc53bf2e-0eb9-4d00-a752-8f07c2886fe9"
 +
                GROUPID="Group-cc53bf2e-0eb9-4d00-a752-8f07c2886fe9"
 +
                ADMID="amdSec_698926">
 +
        <mets:FLocat xlink:href="/var/archivematica/storage_service/aipstore/cc53/.../ex1-cc53bf2e-0eb9-4d00-a752-8f07c2886fe9.7z"
 +
                    LOCTYPE="OTHER" OTHERLOCTYPE="SYSTEM"/>
 +
        <mets:transformFile TRANSFORMTYPE="decompression" TRANSFORMORDER="1" TRANSFORMALGORITHM="bzip2"/>
 +
      </mets:file>
 +
    </mets:fileGrp>
 +
  </mets:fileSec>
 +
</pre>
 +
 
 +
In some storage options that can be configured via the storage service, such as LOCKSS you will see information about the remote location of the AIP:
 +
 
 +
<pre>
 +
<mets:fileSec>
 +
    <mets:fileGrp USE="Archival Information Package">
 +
      <mets:file ID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6">
 +
        <mets:FLocat LOCTYPE="URL" xlink:href="http://lockss1.example.org:8083/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6.7z"/>
 +
        <mets:transformFile TRANSFORMALGORITHM="bzip2" TRANSFORMORDER="1" TRANSFORMTYPE="decompression"/>
 +
      </mets:file>
 +
    </mets:fileGrp>
 +
  </mets:fileSec>
 +
</pre>
 +
 
 +
'''Note:''' Not all storage options will show the information about the remote location. Either this information isn't as easy to replay to the user, or it is functionality that was beyond the scope of development of a storage adapter at the time. S3 is a current example where the user would see the path as it is shown above: <pre>/var/archivematica/storage_service/aipstore/cc53/.../ex1-cc53bf2e-0eb9-4d00-a752-8f07c2886fe9.7z</pre> and they would need to ascertain for themselves that the location was relative to the S3 location created for use as an package store online.
 +
 
 +
== Accessing a pointer file ==
 +
 
 +
Accessing pointer files inside Archivematica can be done by navigating the Dashboard to '''Archival storage''' finding a suitable AIP, e.g. at time of writing a compressed AIP, and under '''Pointer file''' clicking '''View'''. Archivematica will make a call to the Storage Service API to return the file for that AIP. More information can be found about this in the Archivematica documentation under [https://www.archivematica.org/en/docs/archivematica-1.9/user-manual/archival-storage/archival-storage/#storing-an-aip Storing an AIP].
 +
 
 +
* [https://wiki.archivematica.org/Storage_Service_API#Get_pointer_file Storage Service API reference to return a Pointer File]
 +
 
 +
== Developer notes ==
 +
 
 +
=== Location of the pointer file inside Storage Service ===
 +
 
 +
Pointer files are stored in the Storage Service directories, for example:
 +
 
 +
  /var/archivematica/storage_service/d5c3/3360/7b92/4a2f/9002/32f5/a143/daaf/pointer.d5c33360-7b92-4a2f-9002-32f5a143daaf.xml
 +
 
 +
You can see where they are stored looking up the Storage Service database:
 +
 
 +
<source lang="sql">
 +
select locations_location.uuid,
 +
      locations_location.relative_path,
 +
  locations_location.description,
 +
      locations_location.purpose
 +
from locations_location
 +
inner join locations_package
 +
on locations_location.uuid =
 +
  locations_package.pointer_file_location_id
 +
</source>
 +
 
 +
== Additional references ==
 +
 
 +
* [https://wiki.archivematica.org/LOCKSS_Integration LOCKSS integration]
 +
 
 +
== Sample Pointer Files ==
 +
 
 +
* [https://wiki.archivematica.org/Dataset_preservation#Sample_pointer.xml_file Sample XML Pointer File]
 +
* [https://wiki.archivematica.org/LOCKSS_Integration#Sample_pointer_files LOCKSS Integration Pointer File]
  
 
== External Links ==
 
== External Links ==
  
 
* [https://groups.google.com/d/msg/archivematica/VPQGd4s7hI8/x3WFdqO9AQAJ Justin Simpson on the structure and purpose of pointer files at Google Groups]
 
* [https://groups.google.com/d/msg/archivematica/VPQGd4s7hI8/x3WFdqO9AQAJ Justin Simpson on the structure and purpose of pointer files at Google Groups]
 +
* [https://projects.artefactual.com/issues/5159 Feature #5159 The original tranche of work resulting in the implementation of pointer files]

Latest revision as of 13:40, 4 April 2019

Pointer files have been used to help provide sustainable access to compressed AIPs since Archivematica 1.0. [1]. The documentation about them, below, is currently in progress.

Purpose[edit]

Pointer files can help users ascertain information about their AIPs without needing to consult the Storage Service API or database.

As an example, pointer files include:

  • Information about compression, currently used internally by Archivematica itself for Re-ingest or unpacking for delivery via the API
  • Information about encryption so that AIPs can be recovered if encrypted.
  • Information about where the AIP is stored relative to the storage service. NB. there are some variations, see below.

Describing the location of the AIP[edit]

Information about the location of an AIP is stored in a

mets:filesec

inside the pointer file:

  <mets:fileSec>
    <mets:fileGrp USE="Archival Information Package">
      <mets:file ID="rs_s3-cc53bf2e-0eb9-4d00-a752-8f07c2886fe9" 
                 GROUPID="Group-cc53bf2e-0eb9-4d00-a752-8f07c2886fe9" 
                 ADMID="amdSec_698926">
        <mets:FLocat xlink:href="/var/archivematica/storage_service/aipstore/cc53/.../ex1-cc53bf2e-0eb9-4d00-a752-8f07c2886fe9.7z" 
                     LOCTYPE="OTHER" OTHERLOCTYPE="SYSTEM"/>
        <mets:transformFile TRANSFORMTYPE="decompression" TRANSFORMORDER="1" TRANSFORMALGORITHM="bzip2"/>
      </mets:file>
    </mets:fileGrp>
  </mets:fileSec>

In some storage options that can be configured via the storage service, such as LOCKSS you will see information about the remote location of the AIP:

 <mets:fileSec>
    <mets:fileGrp USE="Archival Information Package">
      <mets:file ID="FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6">
        <mets:FLocat LOCTYPE="URL" xlink:href="http://lockss1.example.org:8083/ServeContent?url=http://archivematicastorage.example.com/lockssomatic/FLocatTest-e0ff9a0e-a7f5-4bee-b9c2-7e2321dde7a6.7z"/>
        <mets:transformFile TRANSFORMALGORITHM="bzip2" TRANSFORMORDER="1" TRANSFORMTYPE="decompression"/>
      </mets:file>
    </mets:fileGrp>
  </mets:fileSec>

Note: Not all storage options will show the information about the remote location. Either this information isn't as easy to replay to the user, or it is functionality that was beyond the scope of development of a storage adapter at the time. S3 is a current example where the user would see the path as it is shown above:

/var/archivematica/storage_service/aipstore/cc53/.../ex1-cc53bf2e-0eb9-4d00-a752-8f07c2886fe9.7z

and they would need to ascertain for themselves that the location was relative to the S3 location created for use as an package store online.

Accessing a pointer file[edit]

Accessing pointer files inside Archivematica can be done by navigating the Dashboard to Archival storage finding a suitable AIP, e.g. at time of writing a compressed AIP, and under Pointer file clicking View. Archivematica will make a call to the Storage Service API to return the file for that AIP. More information can be found about this in the Archivematica documentation under Storing an AIP.

Developer notes[edit]

Location of the pointer file inside Storage Service[edit]

Pointer files are stored in the Storage Service directories, for example:

 /var/archivematica/storage_service/d5c3/3360/7b92/4a2f/9002/32f5/a143/daaf/pointer.d5c33360-7b92-4a2f-9002-32f5a143daaf.xml

You can see where they are stored looking up the Storage Service database:

select locations_location.uuid, 
       locations_location.relative_path, 
	   locations_location.description, 
       locations_location.purpose
from locations_location
inner join locations_package 
on locations_location.uuid = 
   locations_package.pointer_file_location_id

Additional references[edit]

Sample Pointer Files[edit]

External Links[edit]