Difference between revisions of "UM transfer"

From Archivematica
Jump to navigation Jump to search
(Undo revision 8787 by Courtney (talk))
 
(35 intermediate revisions by 2 users not shown)
Line 1: Line 1:
[[Main Page]] > [[Documentation]] > [[User Manual|User manual]] > [[User manual 0.9]] > Transfer
+
[[Main Page]] > [[Documentation]] > [[User Manual|User manual]] > [[User manual 0.10]] > Transfer
  
  
Line 6: Line 6:
 
== General description ==
 
== General description ==
  
During transfer, the user moves digital objects from source directories accessible via the MCP server into Archivematica. See [[Administrator_manual|Administrator manual]] for instructions on how to set up shared source directories. Once uploaded to the dashboard, transfers run through several micro-services: UUID assignment; checksum verification (if checksums are present); package extraction (i.e. unzipping of zipped or otherwise packaged files); virus checking; indexing; format identification and validation; and metadata extraction. For a definition of "transfer", please see the [[UM glossary|Glossary]].  </br>
+
In Archivematica, Transfer is the process of transforming any set of digital objects and/or directories into a SIP. Transformation may include appraisal, arrangement, description and identification of donor restricted, private or confidential contents.
 +
 
 +
In the Transfer tab of the Dashboard, the user moves digital objects from source directories accessible via the MCP server into Archivematica. See [[Administrator_manual_0.9#Transfer_source_directories|Administrator manual]] for instructions on how to set up shared source directories. Once uploaded to the dashboard, transfers run through several micro-services: UUID assignment; checksum verification (if checksums are present); package extraction (i.e. unzipping of zipped or otherwise packaged files); virus checking; indexing; format identification and validation; and metadata extraction. </br>
  
 
At the end of transfer, the user creates a SIP from one or more standard transfer(s). Once this is done, the SIP is moved into ingest.
 
At the end of transfer, the user creates a SIP from one or more standard transfer(s). Once this is done, the SIP is moved into ingest.
  
If your transfer is a DSpace export, please see [[UM DSpace exports|DSpace export]].
+
If you would like to import lower-level metadata with your transfer (i.e. metadata to be attached to subdirectories and files within a SIP), see [[UM Transfer metadata import|Metadata import]]
 +
 
 +
If you transfer is from CONTENTdm, please see [[CONTENTdm]]
 +
 
 +
If your transfer is a DSpace export, please see [[UM DSpace exports .10|DSpace export]].
  
If your transfer is a CONTENTdm export, please see [[UM CONTENTdm export|CONTENTdm export]]
+
If your transfer is a bag or a zipped bag, please see [[UM Bags.10|Bags]]
  
If your transfer is a bag or a zipped bag, please see [[UM Bags|Bags]]
+
If your transfer is an email transfer, this feature is still under development, please see [[Email]]
  
If your transfer is an email transfer, this feature is still under development, please see [[]]
+
If your transfer is composed of objects that are the result of digitization, please see [[UM digitization output .10|Digitization output]].
  
If your transfer is composed of objects that are the result of digitization, please see [[UM digitization output|Digitization output]].
+
If you would like to skip some of the default decision points or make preconfigured choices for your desired workflow, see [[Administrator_manual_0.10#Processing_configuration|Administrator manual - Processing configuration]]
  
 
== Create a transfer ==
 
== Create a transfer ==
  
Open the web browser and the Archivematica dashboard to sign in ('''figure 1'''). Default user name is demo, with password demo.  
+
Open the web browser and the Archivematica dashboard to sign in with your username and password. Please note that if this is your first time logging in to a newly installed instance of Archivematica 0.10-beta, you will see a log-in that allows you to [[Register-0.10-beta|register]] your repository and get updates to the [[Administrator_manual_0.10#Format_Policy_Registry_.28FPR.29|Format Policy Registry (FPR)]]
  
[[Image:1_login.png|600px|right|thumb|'''Figure 1'''  Log in to the Archivematica dashboard]]
+
</div>
 +
 
 +
<div class="clearfix">
 +
 
 +
</div>
 +
 
 +
#Set up your transfer source directories in the Administration tab. See [[Administrator_manual_0.10#Transfer_source_directories|Administrator manual]] for instructions on how to set up source directories.
 +
#In transfer tab, name your transfer and browse to a source directory to select your object or set of objects for upload. Your transfer can be composed of multiple directories from different sources. Repeat this step if your transfer is composed from multiple sources. ('''figure 1''') [[Image:Browse-10.png|600px|right|thumb|'''Figure 1'''  Select transfer(s) from source directory(ies)]]
 +
#If applicable, enter the accession number for the transfer.
 +
#Once all of your digital object sources have been uploaded, hit the Start Transfer button for the transfer processing to begin. ('''figure 2''')[[Image:StartTransfer-10.png|600px|right|thumb|'''Figure 2'''  Start transfer]]
  
 
</div>
 
</div>
Line 32: Line 47:
 
</div>
 
</div>
  
#In transfer tab, name your transfer and browse to a source directory to select your object or set of objects for upload. Your transfer can be composed of multiple directories and files from different source directories. Repeat this step if your transfer is composed from multiple sources. ('''figure 2''') [[Image:3_SelectDirForTrans.png|600px|right|thumb|'''Figure 2''' Select transfer(s) from source directory(ies)]]
+
===Create a structured transfer outside the system to include submission documentation===
#*Note that in order for source directories to appear in the transfer panel, you must first have set up your source directories in the Administration tab. See [[Administrator_manual|Administrator manual]] for instructions on how to set up source directories.
+
 
#Once all of your digital object sources have been uploaded, hit the Start Transfer button for the transfer processing to begin. ('''figure 3''')[[Image:4_StartTransfer.png|600px|right|thumb|'''Figure 3'''  Start transfer]]
+
Another option is to create a transfer in a structured directory prior to beginning processing in Archivematica. The structured directory in Archivematica is the basic configuration of the transfer. If you just add a directory to the dashboard and start transfer processing, Archivematica will restructure it so it complies with this structure. There should be three subdirectories: logs, metadata, objects. The objects directory contains the digital objects that are to be preserved. The metadata directory contains the checksum, the METS file, and a submissionDocumentation subfolder, which can be used for transfer forms, donation agreements or any other documents that relate to the acquisition of the records. The logs folder will eventually contain logs generated when processing the transfer in Archivematica. You can create subdirectories within objects if desired.  
 +
 
 +
#Open the file browser by clicking on the Home folder on the Archivematica desktop 
 +
#The structured directory should contain three subdirectories: ''logs'', ''metadata'', ''objects''. Copy the digital files to be preserved into the ''objects'' directory. Note that you can create subdirectories within ''objects''.
 +
#Add submission documentation. In the transfer you have just created, navigate to the ''/metadata/'' folder and add a ''/submissionDocumentation'' directory. Add files to that folder like donor agreements, transfer forms, copyright agreements and any correspondence or other documentation relating to the transfer. Any SIPs subsequently made from this transfer will automatically contain copies of this documentation.
  
 
</div>
 
</div>
Line 44: Line 63:
 
== Process the transfer ==
 
== Process the transfer ==
  
#In the dashboard transfer tab, the transfer will appear in the dashboard with a bell icon next to it. This means that it is awaiting a decision by the user. ('''figure 4''') [[Image:5-ApproveTransfer.png|600px|right|thumb|'''Figure 4'''  In the Actions drop-down menu, select "Approve transfer"]]
+
#In the dashboard transfer tab, the transfer will appear in the dashboard with a bell icon next to it. This means that it is awaiting a decision by the user. ('''figure 3''') [[Image:ApproveTransfer-10.png|600px|right|thumb|'''Figure 3'''  In the Actions drop-down menu, select "Approve transfer"]]
 
#Click on the micro-service to display jobs that have completed, including the one requiring action.
 
#Click on the micro-service to display jobs that have completed, including the one requiring action.
#In the Actions drop-down menu, select "Approve transfer" to begin processing the transfer ('''figure 4'''). You may also "Reject transfer" and quit processing.  
+
#In the Actions drop-down menu, select "Approve transfer" to begin processing the transfer ('''figure 3'''). You may also "Reject transfer" and quit processing.  
 
#The transfer will now run through a series of micro-services. These include:
 
#The transfer will now run through a series of micro-services. These include:
 
#*Verify transfer compliance (verifies that the transfer is properly structured - i.e. with the logs, metadata and objects folders)
 
#*Verify transfer compliance (verifies that the transfer is properly structured - i.e. with the logs, metadata and objects folders)
Line 61: Line 80:
 
#A transfer that is in the middle of processing will show which micro-services have been completed (green) and which are in progress (orange).
 
#A transfer that is in the middle of processing will show which micro-services have been completed (green) and which are in progress (orange).
 
#When a micro-service fails or encounters an error, the micro-service background turns from green to pink and a "failed" icon appears next to the transfer or SIP name. See [[UM error handling|Error handling]] for more information about how to handle an error.
 
#When a micro-service fails or encounters an error, the micro-service background turns from green to pink and a "failed" icon appears next to the transfer or SIP name. See [[UM error handling|Error handling]] for more information about how to handle an error.
#Once the transfer micro-services are completed, a bell icon will appear next to the transfer. This means that the transfer is ready to be packaged into a SIP for ingest ('''figure 5'''). Select "Create single SIP and continue processing". (Note that Create SIP(s) manually is not currently functional. However, the ability to create one or more SIP(s) from one or more transfer(s) will return in Archivematica 1.0 with improved functionality in the web browser. See [File_Browser_Requirements|File browser requirements - Create SIP] and [Transfer_and_SIP_creation|Transfer and SIP creation workflows)
+
#Once the transfer micro-services are completed, a bell icon will appear next to the transfer. This means that the transfer is ready to be packaged into a SIP for ingest or sent to a backlog, indexed and stored to be retrieved for processing at a later date('''figure 4''').  
#See [[UM ingest|Ingest]] for next steps.[[Image:CreateSingleSip.png|600px|right|thumb|'''Figure 5'''  A transfer that is ready to be packaged into a SIP and moved into ingest]]
+
## Option 1: Select "Create single SIP and continue processing". (Note that Create SIP(s) manually is not currently functional. However, the ability to create one or more SIP(s) from one or more transfer(s) will return in Archivematica 1.0 with improved functionality in the web browser. See [File_Browser_Requirements|File browser requirements - Create SIP] and [Transfer_and_SIP_creation|Transfer and SIP creation workflows)
 +
## Option 2: Select "Send transfer to backlog". In this case, your transfer will be stored in a backlog in the same location as your AIP store so that you can retrieve one or more transfers from the Ingest tab for processing at a later date. See [[UM_Transfer_backlog|Managing a backlog]]
 +
#See [[UM ingest|Ingest]] for next steps.
 +
[[Image:CreateSIPs-10.png|600px|right|thumb|'''Figure 4'''  A transfer that is ready to be packaged into a SIP or stored in backlog]]

Latest revision as of 17:56, 30 September 2013

Main Page > Documentation > User manual > User manual 0.10 > Transfer


General description[edit]

In Archivematica, Transfer is the process of transforming any set of digital objects and/or directories into a SIP. Transformation may include appraisal, arrangement, description and identification of donor restricted, private or confidential contents.

In the Transfer tab of the Dashboard, the user moves digital objects from source directories accessible via the MCP server into Archivematica. See Administrator manual for instructions on how to set up shared source directories. Once uploaded to the dashboard, transfers run through several micro-services: UUID assignment; checksum verification (if checksums are present); package extraction (i.e. unzipping of zipped or otherwise packaged files); virus checking; indexing; format identification and validation; and metadata extraction.

At the end of transfer, the user creates a SIP from one or more standard transfer(s). Once this is done, the SIP is moved into ingest.

If you would like to import lower-level metadata with your transfer (i.e. metadata to be attached to subdirectories and files within a SIP), see Metadata import

If you transfer is from CONTENTdm, please see CONTENTdm

If your transfer is a DSpace export, please see DSpace export.

If your transfer is a bag or a zipped bag, please see Bags

If your transfer is an email transfer, this feature is still under development, please see Email

If your transfer is composed of objects that are the result of digitization, please see Digitization output.

If you would like to skip some of the default decision points or make preconfigured choices for your desired workflow, see Administrator manual - Processing configuration

Create a transfer[edit]

Open the web browser and the Archivematica dashboard to sign in with your username and password. Please note that if this is your first time logging in to a newly installed instance of Archivematica 0.10-beta, you will see a log-in that allows you to register your repository and get updates to the Format Policy Registry (FPR)

  1. Set up your transfer source directories in the Administration tab. See Administrator manual for instructions on how to set up source directories.
  2. In transfer tab, name your transfer and browse to a source directory to select your object or set of objects for upload. Your transfer can be composed of multiple directories from different sources. Repeat this step if your transfer is composed from multiple sources. (figure 1)
    Figure 1 Select transfer(s) from source directory(ies)
  3. If applicable, enter the accession number for the transfer.
  4. Once all of your digital object sources have been uploaded, hit the Start Transfer button for the transfer processing to begin. (figure 2)
    Figure 2 Start transfer

Create a structured transfer outside the system to include submission documentation[edit]

Another option is to create a transfer in a structured directory prior to beginning processing in Archivematica. The structured directory in Archivematica is the basic configuration of the transfer. If you just add a directory to the dashboard and start transfer processing, Archivematica will restructure it so it complies with this structure. There should be three subdirectories: logs, metadata, objects. The objects directory contains the digital objects that are to be preserved. The metadata directory contains the checksum, the METS file, and a submissionDocumentation subfolder, which can be used for transfer forms, donation agreements or any other documents that relate to the acquisition of the records. The logs folder will eventually contain logs generated when processing the transfer in Archivematica. You can create subdirectories within objects if desired.

  1. Open the file browser by clicking on the Home folder on the Archivematica desktop
  2. The structured directory should contain three subdirectories: logs, metadata, objects. Copy the digital files to be preserved into the objects directory. Note that you can create subdirectories within objects.
  3. Add submission documentation. In the transfer you have just created, navigate to the /metadata/ folder and add a /submissionDocumentation directory. Add files to that folder like donor agreements, transfer forms, copyright agreements and any correspondence or other documentation relating to the transfer. Any SIPs subsequently made from this transfer will automatically contain copies of this documentation.

Process the transfer[edit]

  1. In the dashboard transfer tab, the transfer will appear in the dashboard with a bell icon next to it. This means that it is awaiting a decision by the user. (figure 3)
    Figure 3 In the Actions drop-down menu, select "Approve transfer"
  2. Click on the micro-service to display jobs that have completed, including the one requiring action.
  3. In the Actions drop-down menu, select "Approve transfer" to begin processing the transfer (figure 3). You may also "Reject transfer" and quit processing.
  4. The transfer will now run through a series of micro-services. These include:
    • Verify transfer compliance (verifies that the transfer is properly structured - i.e. with the logs, metadata and objects folders)
    • Create transfer UUID (assigns a unique universal identifier for the transfer as a whole)
    • Assign file UUIDs to objects (assigns a unique universal identifier to each file in the /objects directory)
    • Verify metadata directory checksums (verifies any checksums included with the transfer)
    • Assign checksums to objects (assigns a sha-256 checksum to each file in the /objects directory)
    • Generate METS.xml document (creates a METS file capturing the original order of the transfer. This METS file is added to any SIPs generated from this transfer)
    • Extract packages (extracts contents from zipped or otherwise packaged files)
    • Sanitize object's file and directory names (removes prohibited characters from folder and filenames, such as ampersands)
    • Scan for viruses (scans for viruses and malware)
    • Characterize and extract metadata (identifies and validates file formats; extracts technical metadata embedded in the files)
    • Index transfer (Indexes transfer)
  5. A transfer that is in the middle of processing will show which micro-services have been completed (green) and which are in progress (orange).
  6. When a micro-service fails or encounters an error, the micro-service background turns from green to pink and a "failed" icon appears next to the transfer or SIP name. See Error handling for more information about how to handle an error.
  7. Once the transfer micro-services are completed, a bell icon will appear next to the transfer. This means that the transfer is ready to be packaged into a SIP for ingest or sent to a backlog, indexed and stored to be retrieved for processing at a later date(figure 4).
    1. Option 1: Select "Create single SIP and continue processing". (Note that Create SIP(s) manually is not currently functional. However, the ability to create one or more SIP(s) from one or more transfer(s) will return in Archivematica 1.0 with improved functionality in the web browser. See [File_Browser_Requirements|File browser requirements - Create SIP] and [Transfer_and_SIP_creation|Transfer and SIP creation workflows)
    2. Option 2: Select "Send transfer to backlog". In this case, your transfer will be stored in a backlog in the same location as your AIP store so that you can retrieve one or more transfers from the Ingest tab for processing at a later date. See Managing a backlog
  8. See Ingest for next steps.
Figure 4 A transfer that is ready to be packaged into a SIP or stored in backlog