Difference between revisions of "Large datasets"
Jump to navigation
Jump to search
| (One intermediate revision by one other user not shown) | |||
| Line 1: | Line 1: | ||
[[Main Page]] > [[Development]] > [[:Category:Development documentation|Development documentation]] > Large datasets | [[Main Page]] > [[Development]] > [[:Category:Development documentation|Development documentation]] > Large datasets | ||
| + | |||
| + | <div style="padding: 10px 10px; border: 1px solid black; background-color: #F79086;">This page is no longer being maintained and may contain inaccurate information. Please see the [https://www.archivematica.org/docs/latest/ Archivematica documentation] for up-to-date information. </div> <p> | ||
What happens when a body of materials to be ingested consists of thousands of files (eg a large social science research dataset), or when one file is extremely large (eg an HD video file)? | What happens when a body of materials to be ingested consists of thousands of files (eg a large social science research dataset), or when one file is extremely large (eg an HD video file)? | ||
| Line 5: | Line 7: | ||
**The dataset could be broken into a parent AIP which acts as an Archival Information Collection, consisting entirely of a METS structMap listing all its child AIPs; each child AIP would have a link back to the parent AIP in its own structMap. | **The dataset could be broken into a parent AIP which acts as an Archival Information Collection, consisting entirely of a METS structMap listing all its child AIPs; each child AIP would have a link back to the parent AIP in its own structMap. | ||
*The large single file could be broken into multiple segments, each in its own AIP. Video files could be delivered to end users in these segments, the way large video files are delivered on Youtube, for example. | *The large single file could be broken into multiple segments, each in its own AIP. Video files could be delivered to end users in these segments, the way large video files are delivered on Youtube, for example. | ||
| − | *Other types of large files might have to be merged back into one for delivery to a user. | + | **Other types of large files might have to be merged back into one for delivery to a user. |
[[Category:Development documentation]] | [[Category:Development documentation]] | ||
Latest revision as of 15:55, 11 February 2020
Main Page > Development > Development documentation > Large datasets
This page is no longer being maintained and may contain inaccurate information. Please see the Archivematica documentation for up-to-date information.
What happens when a body of materials to be ingested consists of thousands of files (eg a large social science research dataset), or when one file is extremely large (eg an HD video file)?
- The large number of files could be broken up and distributed across multiple AIPs, with relationships between them expressed in the METS structMaps.
- The dataset could be broken into a parent AIP which acts as an Archival Information Collection, consisting entirely of a METS structMap listing all its child AIPs; each child AIP would have a link back to the parent AIP in its own structMap.
- The large single file could be broken into multiple segments, each in its own AIP. Video files could be delivered to end users in these segments, the way large video files are delivered on Youtube, for example.
- Other types of large files might have to be merged back into one for delivery to a user.