Improvements/AIP Packaging - Revision history

Sallain at 21:54, 11 February 2020

2020-02-11T21:54:22Z

Peter: /* Use case: Oxford Common File Layout */

2018-08-14T19:37:34Z

Use case: Oxford Common File Layout

Peter: /* Analysis: */

2018-08-14T19:13:46Z

Analysis:

Jhs: /* AIP Splitting scenarios */

2018-08-14T18:36:14Z

AIP Splitting scenarios

Jhs at 18:32, 14 August 2018

2018-08-14T18:32:50Z

Hbecker: Add encryption use case

2017-03-10T20:01:21Z

Add encryption use case

Hbecker: Add development documentation category

2017-03-10T19:58:12Z

Add development documentation category

Evelyn at 23:43, 25 November 2015

2015-11-25T23:43:31Z

Sromkey: Sromkey moved page Improvements/aippackage to Improvements/AIP Packaging

2015-10-16T19:02:55Z

Sromkey moved page Improvements/aippackage to Improvements/AIP Packaging

Sromkey: /* Analysis: */

2015-10-16T18:55:24Z

Analysis:

← Older revision		Revision as of 21:54, 11 February 2020
Line 1:		Line 1:
		+	<div style="padding: 10px 10px; border: 1px solid black; background-color: #F79086;">This page is no longer being maintained and may contain inaccurate information. Please see the [https://www.archivematica.org/docs/latest/ Archivematica documentation] for up-to-date information. </div> <p>
		+
	== User story ==		== User story ==

← Older revision		Revision as of 19:37, 14 August 2018
Line 99:		Line 99:

	=== Use case: Oxford Common File Layout ===		=== Use case: Oxford Common File Layout ===
		+
		+	[https://ocfl.io/ https://ocfl.io/]

	[[Category:Development documentation]]		[[Category:Development documentation]]

← Older revision		Revision as of 19:13, 14 August 2018
Line 98:		Line 98:
	An AIP should be encrypted before storing, independent of where it is stored. The AIP pointer file needs to track information required to unencrypt the AIP on retrieval.		An AIP should be encrypted before storing, independent of where it is stored. The AIP pointer file needs to track information required to unencrypt the AIP on retrieval.

		+	=== Use case: Oxford Common File Layout ===

	[[Category:Development documentation]]		[[Category:Development documentation]]

@@ Line 29: / Line 29: @@
 An AIP is stored as a bag, and the bag is then turned into a .7z file.  The .7z file is then split into multiple parts.  This can be done with the unix split command ([http://man7.org/linux/man-pages/man1/split.1.html split man page]), with the -v argument to 7z ([https://sevenzip.osdn.jp/chm/cmdline/switches/volume.htm 7z volumes]) or by some other method.  The result would look like:
 └── AIP1 (folder)
-    ├── AIP1.7z.001 (binary chunk)
+ └── AIP1 (folder)
-    ├── AIP1.7z.002 (binary chunk)
+     ├── AIP1.7z.001 (binary chunk)
-    ├── AIP1.7z.003 (binary chunk)
+     ├── AIP1.7z.002 (binary chunk)
-    └── pointer.xml (xml file)
+     ├── AIP1.7z.003 (binary chunk)
 The pointer file would contain metadata outlining how to pt the 3 parts back together into a single .7z file and unpack it.  The result of this would be the original bag containing the AIP.
@@ Line 41: / Line 42: @@
 One problem with the first scenario is that the AIP1 folder is not structured according to any standard.  Some storage systems may have a requirement to store content in bags.  To satisfy this, this 2nd scenario adds an additional step - create a bag to hold the chunks:
-−
+ .
-└── AIP1 (folder)
+ └── AIP1 (folder)
-    ├── bag-info.txt
+     ├── bag-info.txt
-    ├── bagit.txt
+     ├── bagit.txt
-    ├── data
+     ├── data
-    │   ├── AIP1.7z.001 (binary chunk)
+     │   ├── AIP1.7z.001 (binary chunk)
-    │   ├── AIP1.7z.002 (binary chunk)
+     │   ├── AIP1.7z.002 (binary chunk)
-    │   ├── AIP1.7z.003 (binary chunk)
+     │   ├── AIP1.7z.003 (binary chunk)
-    │   └── pointer.xml
+     │   └── pointer.xml
-    │
+     │
-    ├── manifest-md5.txt
+     ├── manifest-md5.txt
-    ├── manifest-sha256.txt
+     ├── manifest-sha256.txt
-    ├── tagmanifest-md5.txt
+     ├── tagmanifest-md5.txt
-    └── tagmanifest-sha256.txt
+     └── tagmanifest-sha256.txt
 In this scenario, there are actually 2 bags being created - one is holding the chunks (parts 1 to 3) and the pointer file.  Once the chunks are stitched back together and unpacked, the result would be the original bag containing the AIP.
@@ Line 62: / Line 63: @@
 This scenario is a bit more complicated than Scenario 2.  The only advantage it brings is the ability to further transform each bag (e.g. compress, encrypt).  This might be a requirement if using an object storage system, where it is desirable to store each bag as a single file. This is not possible in scenario 2 without exceeding the maximum file size of the storage system.
-−
+ .
-└── AIP1 (folder)
+ └── AIP1 (folder)
-    ├── AIP1.001 (folder)
+     ├── AIP1.001 (folder)
-    │   ├──bag-info.txt
+     │   ├──bag-info.txt
-    │   ├── bagit.txt
+     │   ├── bagit.txt
-    │   ├── data
+     │   ├── data
-    │   │   └── AIP1.7z.001
+     │   │   └── AIP1.7z.001
-    │   ├── manifest-md5.txt
+     │   ├── manifest-md5.txt
-    │   ├── manifest-sha256.txt
+     │   ├── manifest-sha256.txt
-    │   ├── tagmanifest-md5.txt
+     │   ├── tagmanifest-md5.txt
-    │   └── tagmanifest-sha256.txt
+     │   └── tagmanifest-sha256.txt
-    ├── AIP1.002 (folder)
+     ├── AIP1.002 (folder)
-    │   ├──bag-info.txt
+     │   ├──bag-info.txt
-    │   ├── bagit.txt
+     │   ├── bagit.txt
-    │   ├── data
+     │   ├── data
-    │   │   └── AIP1.7z.002
+     │   │   └── AIP1.7z.002
-    │   ├── manifest-md5.txt
+     │   ├── manifest-md5.txt
-    │   ├── manifest-sha256.txt
+     │   ├── manifest-sha256.txt
-    │   ├── tagmanifest-md5.txt
+     │   ├── tagmanifest-md5.txt
-    │   └── tagmanifest-sha256.txt
+     │   └── tagmanifest-sha256.txt
-    ├── AIP1.003 (folder)
+     ├── AIP1.003 (folder)
-    │   ├──bag-info.txt
+     │   ├──bag-info.txt
-    │   ├── bagit.txt
+     │   ├── bagit.txt
-    │   ├── data
+     │   ├── data
-    │   │   └── AIP1.7z.003
+     │   │   └── AIP1.7z.003
-    │   ├── manifest-md5.txt
+     │   ├── manifest-md5.txt
-    │   ├── manifest-sha256.txt
+     │   ├── manifest-sha256.txt
-    │   ├── tagmanifest-md5.txt
+     │   ├── tagmanifest-md5.txt
-    │   └── tagmanifest-sha256.txt
+     │   └── tagmanifest-sha256.txt
-    └─ pointer.xml
+     └─ pointer.xml
 === Use case: Encryption ===

@@ Line 13: / Line 13: @@
 == Analysis: ==
-Currently, Archivematica can only package a single AIP as a single bag. This has limitations in some repository environments and does not allow archivists/repository managers flexibility in how AIPs are stored and accessed.
+Currently, Archivematica can only package a single AIP as a single bag. This bag can be stored as a folder (referred to as 'uncompressed' in the Archivematica UI) or as a single file (by default a 7zip file).
 === Use case: AIP split into multiple parts ===
-An AIP is split into multiple zipped packages and loose files for storage and retrieval purposes. The AIP pointer file needs to indicate the existence and locations all of the zipped packages and loose parts, provide compression information about the zipped packages, indicate that the AIP has been unpackaged from its bag, and indicate the location and purpose of the AIP METS file.
+An AIP is split into multiple pieces (zipped packages, loose files, binary chunks) for storage and retrieval purposes. There needs to be a way to record metadata that indicates the existence and locations all of the parts, record PREMIS events for each transformation that was applied to the AIP, and details about how to reverse each transformation.
 === Use case: Encryption ===

@@ Line 5: / Line 5: @@
 == Status ==
-Analysis is ongoing and development is sponsored.
+Analysis is ongoing.
 == Interest ==
@@ Line 18: / Line 18: @@
 An AIP is split into multiple zipped packages and loose files for storage and retrieval purposes. The AIP pointer file needs to indicate the existence and locations all of the zipped packages and loose parts, provide compression information about the zipped packages, indicate that the AIP has been unpackaged from its bag, and indicate the location and purpose of the AIP METS file.
 [[Category:Development documentation]]

← Older revision		Revision as of 19:58, 10 March 2017
Line 18:		Line 18:

	An AIP is split into multiple zipped packages and loose files for storage and retrieval purposes. The AIP pointer file needs to indicate the existence and locations all of the zipped packages and loose parts, provide compression information about the zipped packages, indicate that the AIP has been unpackaged from its bag, and indicate the location and purpose of the AIP METS file.		An AIP is split into multiple zipped packages and loose files for storage and retrieval purposes. The AIP pointer file needs to indicate the existence and locations all of the zipped packages and loose parts, provide compression information about the zipped packages, indicate that the AIP has been unpackaged from its bag, and indicate the location and purpose of the AIP METS file.
		+
		+
		+	[[Category:Development documentation]]

← Older revision		Revision as of 23:43, 25 November 2015
Line 14:		Line 14:

	Currently, Archivematica can only package a single AIP as a single bag. This has limitations in some repository environments and does not allow archivists/repository managers flexibility in how AIPs are stored and accessed.		Currently, Archivematica can only package a single AIP as a single bag. This has limitations in some repository environments and does not allow archivists/repository managers flexibility in how AIPs are stored and accessed.
		+
		+	=== Use case: AIP split into multiple parts ===
		+
		+	An AIP is split into multiple zipped packages and loose files for storage and retrieval purposes. The AIP pointer file needs to indicate the existence and locations all of the zipped packages and loose parts, provide compression information about the zipped packages, indicate that the AIP has been unpackaged from its bag, and indicate the location and purpose of the AIP METS file.

← Older revision	Revision as of 19:02, 16 October 2015
(No difference)

← Older revision		Revision as of 18:55, 16 October 2015
Line 12:		Line 12:

	== Analysis: ==		== Analysis: ==
		+
		+	Currently, Archivematica can only package a single AIP as a single bag. This has limitations in some repository environments and does not allow archivists/repository managers flexibility in how AIPs are stored and accessed.