Difference between revisions of "UM transfer"

From Archivematica
Jump to navigation Jump to search
Line 1: Line 1:
[[Main Page]] > [[Documentation]] > [[User Manual|User manual]] > [[User manual 0.8]] > Transfer
+
[[Main Page]] > [[Documentation]] > [[User Manual|User manual]] > [[User manual 0.9]] > Transfer
  
  
Line 6: Line 6:
 
== General description ==
 
== General description ==
  
During transfer, digital objects are moved into Archivematica and run through several micro-services: UUID assignment; checksum verification (if checksums are present); package extraction (i.e. unzipping of zipped or otherwise packaged files); virus checking; format identification and validation; and metadata extraction. For a definition of "transfer", please see the [[UM glossary|Glossary]].  </br>
+
During transfer, digital objects are moved from source directories into Archivematica and run through several micro-services: UUID assignment; checksum verification (if checksums are present); package extraction (i.e. unzipping of zipped or otherwise packaged files); virus checking; indexing; format identification and validation; and metadata extraction. For a definition of "transfer", please see the [[UM glossary|Glossary]].  </br>
  
At the end of transfer, the user can select a number of ways to create one or more SIPs from the transfer. Once this is done, the SIP(s) are moved into ingest.
+
At the end of transfer, the user creates a SIP from one or more standard transfer(s). Once this is done, the SIP is moved into ingest.
  
If your transfer is a DSpace export, please see [[UM DSpace exports|DSpace exports]].
+
If your transfer is a DSpace export, please see [[UM DSpace exports|DSpace export]].
 +
 
 +
If your transfer is a CONTENTdm export, please see [[UM CONTENTdm export|CONTENTdm export]]
 +
 
 +
If your transfer is a bag or a zipped bag, please see [[UM Bags|Bags]]
 +
 
 +
If your transfer is an email transfer, this feature is still under development, please see [[]]
  
 
If your transfer is composed of objects that are the result of digitization, please see [[UM digitization output|Digitization output]].
 
If your transfer is composed of objects that are the result of digitization, please see [[UM digitization output|Digitization output]].
Line 16: Line 22:
 
== Create a transfer ==
 
== Create a transfer ==
  
#Open the file browser by clicking on the Home folder on the Archivematica desktop ('''figure 1'''). [[Image:desktopg.png|600px|right|thumb|'''Figure 1'''  Click on the Home folder on the Archivematica desktop]]
+
#See [[Administrator Manual|Administrator manual]] for instructions on how to set up shared source directories.
 +
#Open the web browser and the Archivematica dashboard ('''figure 1'''). [[Image:desktopg.png|600px|right|thumb|'''Figure 1'''  Click on the Home folder on the Archivematica desktop]]
  
 
</div>
 
</div>
Line 23: Line 30:
  
 
</div>
 
</div>
 +
=== Option 1 ===
  
=== Option 1 ===
+
Use this option if you do not wish to structure your transfer using the file browser or include submission documentation.
Use this option if you have not yet compiled your objects and/or directories into a folder ready for processing.
+
 
#Navigate to the standardTransfer directory by using the shortcut on the left panel of the file manager.  
+
#In transfer tab, name your transfer.
#Hover your cursor over the empty right-hand panel, right-click and select "Create a structured directory" ('''figure 2'''). For a definition of "structured directory", please see [[UM glossary|Glossary]]
+
 
#The structured directory will contain three subdirectories: ''logs'', ''metadata'', ''objects''. Copy the digital files to be preserved into the ''objects'' directory. Note that you can create subdirectories within ''objects'' if desired.[[Image:structuredDirectoryg.png|600px|right|thumb|'''Figure 2'''  Option 1: right-click and create a structured directory]]
+
#Browse to a source directory and select your object or set of objects for upload. Your transfer can be composed of multiple directories and files from different source directories. Repeat this step if your transfer is composed from multiple sources. [[Image:desktopg.png|600px|right|thumb|'''Figure 2''' Upload transfer contents]]
 +
 
 +
#Once all of your sources have been uploaded, hit the Start Transfer button for the transfer processing to begin. [[Image:desktopg.png|600px|right|thumb|'''Figure 3'''  Start transfer]]
  
 
</div>
 
</div>
Line 37: Line 47:
  
 
=== Option 2 ===
 
=== Option 2 ===
Use this option if you have already compiled your objects and/or directories into a folder ready for processing.
+
 
#Navigate to a directory of files to be preserved that has not been created using the right-click script in option 1, above.  
+
Use this option if you wish to compile your objects and/or directories into a folder ready for processing using the file browser interface.  
#Right-click the directory and select "Restructure for compliance" ('''figure 3'''). This will create a structured directory containing the subdirectories required to transfer the files into Archivematica: ''logs'', ''metadata'', ''objects''.
+
 
#Copy and paste the directory to ''standardTransfer'' using the shortcut on the left panel of the file manager.[[Image:restructureg.png|600px|right|thumb|'''Figure 3'''  Option 2: right-click on a directory and restructure it for compliance]]
+
#Navigate to a directory of files to be preserved.  
 +
 
 +
#Right-click the directory and select "Restructure for compliance" ('''figure 4'''). This will create a structured directory containing the subdirectories required to transfer the files into Archivematica: ''logs'', ''metadata'', ''objects''. You can create subdirectories within ''objects'' if desired.(Note that in Option 1, these subdirectories were automatically generated when you hit Start Transfer)
 +
 
 +
#If you would like to add Submission Documentation, in the transfer you have just created, navigate to the ''/metadata/submissionDocumentation'' folder and add files if desired. These files could be donor agreements, transfer forms, copyright agreements and any correspondence or other documentation relating to the transfer. Any SIPs subsequently made from this transfer will automatically contain copies of this documentation.
 +
 
 +
#To upload your transfer into Archivematica and begin processing, follow the steps in Option 1.  
  
 
</div>
 
</div>
Line 48: Line 64:
 
</div>
 
</div>
  
== Add submission documentation ==
 
  
In the transfer you have just created, navigate to the ''/metadata/submissionDocumentation'' folder and add files if desired. These files could be donor agreements, transfer forms, copyright agreements and any correspondence or other documentation relating to the transfer. Any SIPs subsequently made from this transfer will automatically contain copies of this documentation.
+
 
  
 
== Process the transfer ==
 
== Process the transfer ==
#Open the Archivematica dashboard by clicking on the Dashboard icon in the Archivematica desktop ('''figure 1''').[[Image:transfer1g.png|600px|right|thumb|'''Figure 4'''  Hover cursor over the transfer. Click on the spyglass icon on the right-hand side]]
+
 
#Click on the transfer tab.
+
#In the dashboard transfer tab, the transfer will appear in the dashboard with a bell icon next to it. This means that it is awaiting a decision by the user. [[Image:transfer2g.png|600px|right|thumb|'''Figure 5''' In the Actions drop-down menu, select "Transfer complete"]]
#The transfer will appear in the dashboard with a bell icon next to it. This means that it is awaiting a decision by the user.
+
 
#Hover your cursor over the transfer. This will bring up three icons on the right-hand side ('''figure 4''').
+
#Click on the micro-service to display jobs that have completed, including the one requiring action.
#Click on the spyglass icon ('''figure 4''').
+
 
#In the Actions drop-down menu, select "Transfer complete" to begin processing the transfer ('''figure 5'''). [[Image:transfer2g.png|600px|right|thumb|'''Figure 5'''  In the Actions drop-down menu, select "Transfer complete"]]
+
#In the Actions drop-down menu, select "Approve transfer" to begin processing the transfer ('''figure 5'''). You may also "Reject transfer" and quit processing. [[Image:transfer2g.png|600px|right|thumb|'''Figure 5'''  In the Actions drop-down menu, select "Approve transfer"]]
 +
 
 
#The transfer will now run through a series of micro-services. These include:
 
#The transfer will now run through a series of micro-services. These include:
 
#*Verify transfer compliance (verifies that the transfer is properly structured - i.e. with the logs, metadata and objects folders)
 
#*Verify transfer compliance (verifies that the transfer is properly structured - i.e. with the logs, metadata and objects folders)
Line 70: Line 86:
 
#*Scan for viruses (scans for viruses and malware)
 
#*Scan for viruses (scans for viruses and malware)
 
#*Characterize and extract metadata (identifies and validates file formats; extracts technical metadata embedded in the files)
 
#*Characterize and extract metadata (identifies and validates file formats; extracts technical metadata embedded in the files)
 +
#* Index transfer (Indexes transfer)
 +
 
#A transfer that is in the middle of processing will show which micro-services have been completed (green) and which are in progress (orange) ('''figure 6'''). [[Image:transfer3g.png|600px|right|thumb|'''Figure 6'''  A transfer in the middle of the "Characterize and extract metadata" micro-service]]
 
#A transfer that is in the middle of processing will show which micro-services have been completed (green) and which are in progress (orange) ('''figure 6'''). [[Image:transfer3g.png|600px|right|thumb|'''Figure 6'''  A transfer in the middle of the "Characterize and extract metadata" micro-service]]
 +
 
#When a micro-service fails or encounters an error, the micro-service background turns from green to pink and a "failed" icon appears next to the transfer or SIP name. See [[UM error handling|Error handling]] for more information about how to handle an error.
 
#When a micro-service fails or encounters an error, the micro-service background turns from green to pink and a "failed" icon appears next to the transfer or SIP name. See [[UM error handling|Error handling]] for more information about how to handle an error.
#Once the transfer micro-services are completed, a bell icon will appear next to the transfer. This means that the transfer is ready to be packaged into one or more SIPs ('''figure 7'''). See [[UM ingest|Ingest]] for next steps.[[Image:transfer4g.png|600px|right|thumb|'''Figure 7'''  A transfer that is ready to be packaged into one or more SIPs and moved into ingest]]
+
 
 +
#Once the transfer micro-services are completed, a bell icon will appear next to the transfer. This means that the transfer is ready to be packaged into a SIP for ingest ('''figure 7'''). See [[UM ingest|Ingest]] for next steps.[[Image:transfer4g.png|600px|right|thumb|'''Figure 7'''  A transfer that is ready to be packaged into a SIP and moved into ingest]]

Revision as of 12:46, 23 August 2012

Main Page > Documentation > User manual > User manual 0.9 > Transfer


General description

During transfer, digital objects are moved from source directories into Archivematica and run through several micro-services: UUID assignment; checksum verification (if checksums are present); package extraction (i.e. unzipping of zipped or otherwise packaged files); virus checking; indexing; format identification and validation; and metadata extraction. For a definition of "transfer", please see the Glossary.

At the end of transfer, the user creates a SIP from one or more standard transfer(s). Once this is done, the SIP is moved into ingest.

If your transfer is a DSpace export, please see DSpace export.

If your transfer is a CONTENTdm export, please see CONTENTdm export

If your transfer is a bag or a zipped bag, please see Bags

If your transfer is an email transfer, this feature is still under development, please see [[]]

If your transfer is composed of objects that are the result of digitization, please see Digitization output.

Create a transfer

  1. See Administrator manual for instructions on how to set up shared source directories.
  2. Open the web browser and the Archivematica dashboard (figure 1).
    Figure 1 Click on the Home folder on the Archivematica desktop

Option 1

Use this option if you do not wish to structure your transfer using the file browser or include submission documentation.

  1. In transfer tab, name your transfer.
  1. Browse to a source directory and select your object or set of objects for upload. Your transfer can be composed of multiple directories and files from different source directories. Repeat this step if your transfer is composed from multiple sources.
    Figure 2 Upload transfer contents
  1. Once all of your sources have been uploaded, hit the Start Transfer button for the transfer processing to begin.
    Figure 3 Start transfer

Option 2

Use this option if you wish to compile your objects and/or directories into a folder ready for processing using the file browser interface.

  1. Navigate to a directory of files to be preserved.
  1. Right-click the directory and select "Restructure for compliance" (figure 4). This will create a structured directory containing the subdirectories required to transfer the files into Archivematica: logs, metadata, objects. You can create subdirectories within objects if desired.(Note that in Option 1, these subdirectories were automatically generated when you hit Start Transfer)
  1. If you would like to add Submission Documentation, in the transfer you have just created, navigate to the /metadata/submissionDocumentation folder and add files if desired. These files could be donor agreements, transfer forms, copyright agreements and any correspondence or other documentation relating to the transfer. Any SIPs subsequently made from this transfer will automatically contain copies of this documentation.
  1. To upload your transfer into Archivematica and begin processing, follow the steps in Option 1.



Process the transfer

  1. In the dashboard transfer tab, the transfer will appear in the dashboard with a bell icon next to it. This means that it is awaiting a decision by the user.
    Figure 5 In the Actions drop-down menu, select "Transfer complete"
  1. Click on the micro-service to display jobs that have completed, including the one requiring action.
  1. In the Actions drop-down menu, select "Approve transfer" to begin processing the transfer (figure 5). You may also "Reject transfer" and quit processing.
    Figure 5 In the Actions drop-down menu, select "Approve transfer"
  1. The transfer will now run through a series of micro-services. These include:
    • Verify transfer compliance (verifies that the transfer is properly structured - i.e. with the logs, metadata and objects folders)
    • Create transfer UUID (assigns a unique universal identifier for the transfer as a whole)
    • Assign file UUIDs to objects (assigns a unique universal identifier to each file in the /objects directory)
    • Verify metadata directory checksums (verifies any checksums included with the transfer)
    • Assign checksums to objects (assigns a sha-256 checksum to each file in the /objects directory)
    • Generate METS.xml document (creates a METS file capturing the original order of the transfer. This METS file is added to any SIPs generated from this transfer)
    • Extract packages (extracts contents from zipped or otherwise packaged files)
    • Sanitize object's file and directory names (removes prohibited characters from folder and filenames, such as ampersands)
    • Scan for viruses (scans for viruses and malware)
    • Characterize and extract metadata (identifies and validates file formats; extracts technical metadata embedded in the files)
    • Index transfer (Indexes transfer)
  1. A transfer that is in the middle of processing will show which micro-services have been completed (green) and which are in progress (orange) (figure 6).
    Figure 6 A transfer in the middle of the "Characterize and extract metadata" micro-service
  1. When a micro-service fails or encounters an error, the micro-service background turns from green to pink and a "failed" icon appears next to the transfer or SIP name. See Error handling for more information about how to handle an error.
  1. Once the transfer micro-services are completed, a bell icon will appear next to the transfer. This means that the transfer is ready to be packaged into a SIP for ingest (figure 7). See Ingest for next steps.
    Figure 7 A transfer that is ready to be packaged into a SIP and moved into ingest