Difference between revisions of "Archivematica 0.9 Micro-services"

From Archivematica
Jump to navigation Jump to search
(Created page with "Main Page > Documentation > Technical Architecture > Micro-services > Archivematica 0.9 Micro-services This page describes the key micro-services that are und...")
 
Line 159: Line 159:
  
 
=0.9=
 
=0.9=
−
 
  
 
{| border="1" cellpadding="10" cellspacing="0" width=90%
 
{| border="1" cellpadding="10" cellspacing="0" width=90%
Line 168: Line 167:
 
|-
 
|-
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Rename with transfer UUID'''<div class="mw-collapsible-content">
−
thumbnail normalization failed - copying
+
<br/>Set file permissions<br/>
−
|                                          
+
Move to processing directory
 +
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Rename with transfer UUID'''
−
Access normalization failed - copying
+
</div>
−
|                                              
+
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Rename with transfer UUID'''<div class="mw-collapsible-content">
−
Normalize thumbnail
+
<br/>Set transfer type: Maildir
−
|                                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Rename with transfer UUID'''<div class="mw-collapsible-content">
−
Normalize access - exclude
+
<br/>Attempt restructure for compliance
−
|                                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Rename with transfer UUID'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Extract packages'''<div class="mw-collapsible-content">
−
Set file permissions
+
<br/>Extract attachments
−
|                                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Rename with transfer UUID'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Approve transfer'''<div class="mw-collapsible-content">
−
Move to processing directory
+
<br/>Approve bagit transfer
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Rename with transfer UUID'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Approve transfer'''<div class="mw-collapsible-content">
−
Set transfer type: Maildir
+
<br/>Extract zipped bag transfer
−
|                                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Rename with transfer UUID'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Approve transfer'''<div class="mw-collapsible-content">
−
Attempt restructure for compliance
+
<br/>Set file permissions
−
|                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Extract packages'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Approve transfer'''<div class="mw-collapsible-content">
−
Extract attachments
+
<br/>Move to processing directory
−
|                                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Approve transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Approve transfer'''<div class="mw-collapsible-content">
−
Approve bagit transfer
+
<br/>Rename with transfer UUID
−
|                                                            
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Approve transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Approve transfer'''<div class="mw-collapsible-content">
−
Extract zipped bag transfer
+
<br/>Assign file UUIDs to objects
−
|                                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Approve transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Approve transfer'''<div class="mw-collapsible-content">
−
Set file permissions
+
<br/>Assign checksums and file sizes to objects
−
|                                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Approve transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Approve transfer'''<div class="mw-collapsible-content">
−
Move to processing directory
+
<br/>Verify bag, and restructure for compliance
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Approve transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Approve transfer'''<div class="mw-collapsible-content">
−
Rename with transfer UUID
+
<br/>Include default Transfer processingMCP.xml
−
|                                                          
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Approve transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Approve transfer'''<div class="mw-collapsible-content">
−
Assign file UUIDs to objects
+
<br/>Approve DSpace transfer
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Approve transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Verify transfer compliance'''<div class="mw-collapsible-content">
−
Assign checksums and file sizes to objects
+
<br/>Set transfer type: DSpace
−
|                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Approve transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Include default Transfer processingMCP.xml'''<div class="mw-collapsible-content">
−
Verify bag, and restructure for compliance
+
<br/>Failed compliance. See output in dashboard. Transfer moved back to activeTransfers.
−
|                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Approve transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Include default Transfer processingMCP.xml'''<div class="mw-collapsible-content">
−
Include default Transfer processingMCP.xml
+
<br/>Rename with transfer UUID
−
|                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Approve transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Include default Transfer processingMCP.xml'''<div class="mw-collapsible-content">
−
Approve DSpace transfer
+
<br/>Create transfer backup (sharedDirectory/transferBackups)
−
|                                                            
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Verify transfer compliance'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Verify transfer checksum'''<div class="mw-collapsible-content">
−
Set transfer type: DSpace
+
<br/>Verify metadata directory checksums
−
|                                                          
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Include default Transfer processingMCP.xml'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Generate METS.xml'''<div class="mw-collapsible-content">
−
Failed compliance. See output in dashboard. Transfer moved back to activeTransfers.
+
<br/>Generate METS.xml document
−
|  
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Include default Transfer processingMCP.xml'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Quarantine'''<div class="mw-collapsible-content">
−
Rename with transfer UUID
+
<br/>Designate to process as a DSpace transfer
−
|                                                          
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Include default Transfer processingMCP.xml'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Quarantine'''<div class="mw-collapsible-content">
−
Create transfer backup (sharedDirectory/transferBackups)
+
<br/>Designate to process as a DSpace transfer when unquarantined
−
|                          
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Verify transfer checksum'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Quarantine'''<div class="mw-collapsible-content">
−
Verify metadata directory checksums
+
<br/>Move to quarantined
−
|                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Generate METS.xml'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Verify transfer checksums'''<div class="mw-collapsible-content">
−
Generate METS.xml document
+
<br/>Verify checksums in fileSec of DSpace METS files
−
|                                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Quarantine'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Clean up names'''<div class="mw-collapsible-content">
−
Designate to process as a DSpace transfer
+
<br/>Identify DSpace text files
−
|                                          
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Quarantine'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Clean up names'''<div class="mw-collapsible-content">
−
Designate to process as a DSpace transfer when unquarantined
+
<br/>Identify DSpace license files
−
|                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Quarantine'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Clean up names'''<div class="mw-collapsible-content">
−
Move to quarantined
+
<br/>Identify DSpace mets files
−
|                                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Verify transfer checksums'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Characterize and extract metadata'''<div class="mw-collapsible-content">
−
Verify checksums in fileSec of DSpace METS files
+
<br/>Characterize and extract metadata on objects
−
|                                  
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Clean up names'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Approve transfer'''<div class="mw-collapsible-content">
−
Identify DSpace text files
+
<br/>Approve transfer
−
|                                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Clean up names'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Verify transfer compliance'''<div class="mw-collapsible-content">
−
Identify DSpace license files
+
<br/>Set file permissions
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Clean up names'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Verify transfer compliance'''<div class="mw-collapsible-content">
−
Identify DSpace mets files
+
<br/>Move to processing directory
−
|                                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Characterize and extract metadata'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Verify transfer compliance'''<div class="mw-collapsible-content">
−
Characterize and extract metadata on objects
+
<br/>Set transfer type: Standard
−
|                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Approve transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Verify transfer compliance'''<div class="mw-collapsible-content">
−
Approve transfer
+
<br/>Remove hidden files and directories
−
|                                                                  
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Verify transfer compliance'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Verify transfer compliance'''<div class="mw-collapsible-content">
−
Set file permissions
+
<br/>Attempt restructure for compliance
−
|                                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Verify transfer compliance'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Verify transfer compliance'''<div class="mw-collapsible-content">
−
Move to processing directory
+
<br/>Verify transfer compliance
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Verify transfer compliance'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Failed transfer compliance'''<div class="mw-collapsible-content">
−
Set transfer type: Standard
+
<br/>Designate to process as a standard transfer
−
|                                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Verify transfer compliance'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Failed transfer compliance'''<div class="mw-collapsible-content">
−
Remove hidden files and directories
+
<br/>Failed compliance.
−
|                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Verify transfer compliance'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Failed transfer compliance'''<div class="mw-collapsible-content">
−
Attempt restructure for compliance
+
<br/>Find branch to continue processing
−
|                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Verify transfer compliance'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Failed transfer compliance'''<div class="mw-collapsible-content">
−
Verify transfer compliance
+
<br/>Attempt restructure for compliance?
−
|                                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Failed transfer compliance'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Failed transfer compliance'''<div class="mw-collapsible-content">
−
Designate to process as a standard transfer
+
<br/>Move to processing directory
−
|                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Failed transfer compliance'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Failed transfer compliance'''<div class="mw-collapsible-content">
−
Failed compliance.
+
<br/>Attempt restructure for compliance
−
|                                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Failed transfer compliance'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Failed transfer compliance'''<div class="mw-collapsible-content">
−
Find branch to continue processing
+
<br/>Move transfer back to activeTransfers directory.
−
|                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Failed transfer compliance'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Rename with transfer UUID'''<div class="mw-collapsible-content">
−
Attempt restructure for compliance?
+
<br/>Rename with transfer UUID
−
|                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Failed transfer compliance'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Include default Transfer processingMCP.xml'''<div class="mw-collapsible-content">
−
Move to processing directory
+
<br/>Include default Transfer processingMCP.xml
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Failed transfer compliance'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Assign file UUIDs and checksums'''<div class="mw-collapsible-content">
−
Attempt restructure for compliance
+
<br/>Designate to process as a standard transfer
−
|                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Failed transfer compliance'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Assign file UUIDs and checksums'''<div class="mw-collapsible-content">
−
Move transfer back to activeTransfers directory.
+
<br/>Move to workFlowDecisions-createTransferBackup directory
−
|                                  
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Rename with transfer UUID'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Assign file UUIDs and checksums'''<div class="mw-collapsible-content">
−
Rename with transfer UUID
+
<br/>Find type to process as
−
|                                                          
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Include default Transfer processingMCP.xml'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Assign file UUIDs and checksums'''<div class="mw-collapsible-content">
−
Include default Transfer processingMCP.xml
+
<br/>Workflow decision - create transfer backup
−
|                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Assign file UUIDs and checksums'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Assign file UUIDs and checksums'''<div class="mw-collapsible-content">
−
Designate to process as a standard transfer
+
<br/>Create transfer backup (sharedDirectory/transferBackups)
−
|                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Assign file UUIDs and checksums'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Assign file UUIDs and checksums'''<div class="mw-collapsible-content">
−
Move to workFlowDecisions-createTransferBackup directory
+
<br/>Move to processing directory
−
|                          
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Assign file UUIDs and checksums'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Assign file UUIDs and checksums'''<div class="mw-collapsible-content">
−
Find type to process as
+
<br/>Set file permissions
−
|                                                            
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Assign file UUIDs and checksums'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Assign file UUIDs and checksums'''<div class="mw-collapsible-content">
−
Workflow decision - create transfer backup
+
<br/>Assign file UUIDs to objects
−
|                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Assign file UUIDs and checksums'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Assign file UUIDs and checksums'''<div class="mw-collapsible-content">
−
Create transfer backup (sharedDirectory/transferBackups)
+
<br/>Assign checksums and file sizes to objects
−
|                          
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Assign file UUIDs and checksums'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Verify transfer checksums'''<div class="mw-collapsible-content">
−
Move to processing directory
+
<br/>Verify metadata directory checksums
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Assign file UUIDs and checksums'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Generate METS.xml document'''<div class="mw-collapsible-content">
−
Set file permissions
+
<br/>Generate METS.xml document
−
|                                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Assign file UUIDs and checksums'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Quarantine'''<div class="mw-collapsible-content">
−
Assign file UUIDs to objects
+
<br/>Designate to process as a standard transfer
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Assign file UUIDs and checksums'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Quarantine'''<div class="mw-collapsible-content">
−
Assign checksums and file sizes to objects
+
<br/>Move to workFlowDecisions-quarantineSIP directory
−
|                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Verify transfer checksums'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Quarantine'''<div class="mw-collapsible-content">
−
Verify metadata directory checksums
+
<br/>Find type to process as
−
|                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Generate METS.xml document'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Quarantine'''<div class="mw-collapsible-content">
−
Generate METS.xml document
+
<br/>Workflow decision - send transfer to quarantine
−
|                                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Quarantine'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Quarantine'''<div class="mw-collapsible-content">
−
Designate to process as a standard transfer
+
<br/>Set quarantine permissions on transfer
−
|                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Quarantine'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Quarantine'''<div class="mw-collapsible-content">
−
Move to workFlowDecisions-quarantineSIP directory
+
<br/>Create quarantine PREMIS events
−
|                                  
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Quarantine'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Quarantine'''<div class="mw-collapsible-content">
−
Find type to process as
+
<br/>Designate to process as a standard transfer when unquarantined
−
|                                                            
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Quarantine'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Quarantine'''<div class="mw-collapsible-content">
−
Workflow decision - send transfer to quarantine
+
<br/>Move to quarantine
−
|                                    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Quarantine'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Quarantine'''<div class="mw-collapsible-content">
−
Set quarantine permissions on transfer
+
<br/>Find type to remove from quarantine as
−
|                                            
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Quarantine'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Quarantine'''<div class="mw-collapsible-content">
−
Create quarantine PREMIS events
+
<br/>Remove from quarantine
−
|                                                    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Quarantine'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Quarantine'''<div class="mw-collapsible-content">
−
Designate to process as a standard transfer when unquarantined
+
<br/>Set unquarantined file permissions on Transfer
−
|                    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Quarantine'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Quarantine'''<div class="mw-collapsible-content">
−
Move to quarantine
+
<br/>Move to processing directory
−
|                                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Quarantine'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Quarantine'''<div class="mw-collapsible-content">
−
Find type to remove from quarantine as
+
<br/>Create unquarantine PREMIS events
−
|                                            
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Quarantine'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Extract packages'''<div class="mw-collapsible-content">
−
Remove from quarantine
+
<br/>Move to processing directory
−
|                                                            
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Quarantine'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Extract packages'''<div class="mw-collapsible-content">
−
Set unquarantined file permissions on Transfer
+
<br/>Extract packages
−
|                                    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Quarantine'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Scan for viruses'''<div class="mw-collapsible-content">
−
Move to processing directory
+
<br/>Scan for viruses
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Quarantine'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Clean up names'''<div class="mw-collapsible-content">
−
Create unquarantine PREMIS events
+
<br/>Sanitize object's file and directory names
−
|                                                  
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Extract packages'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Clean up names'''<div class="mw-collapsible-content">
−
Move to processing directory
+
<br/>Sanitize Transfer name
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Extract packages'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Characterize and extract metadata'''<div class="mw-collapsible-content">
−
Extract packages
+
<br/>Characterize and extract metadata
−
|                                                                  
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Scan for viruses'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Characterize and extract metadata'''<div class="mw-collapsible-content">
−
Scan for viruses
+
<br/>Identify Files ByExtension
−
|                                                                  
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Clean up names'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Characterize and extract metadata'''<div class="mw-collapsible-content">
−
Sanitize object's file and directory names
+
<br/>Load labels from metadata/file_labels.csv
−
|                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Clean up names'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Complete transfer'''<div class="mw-collapsible-content">
−
Sanitize Transfer name
+
<br/>Set file permissions
−
|                                                            
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Characterize and extract metadata'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Complete transfer'''<div class="mw-collapsible-content">
−
Characterize and extract metadata
+
<br/>Move to completedTransfers directory
−
|                                                  
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Characterize and extract metadata'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Complete transfer'''<div class="mw-collapsible-content">
−
Identify Files ByExtension
+
<br/>Index transfer contents
−
|                                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Characterize and extract metadata'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Create SIP from Transfer'''<div class="mw-collapsible-content">
−
Load labels from metadata/file_labels.csv
+
<br/>Check transfer directory for objects
−
|                                          
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Complete transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Create SIP from Transfer'''<div class="mw-collapsible-content">
−
Set file permissions
+
<br/>Create SIP(s)
−
|                                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Complete transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Create SIP from Transfer'''<div class="mw-collapsible-content">
−
Move to completedTransfers directory
+
<br/>Move to processing directory
−
|                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Complete transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Create SIP from Transfer'''<div class="mw-collapsible-content">
−
Index transfer contents
+
<br/>Create SIP from transfer objects
−
|                                                            
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Create SIP from Transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Create SIP from Transfer'''<div class="mw-collapsible-content">
−
Check transfer directory for objects
+
<br/>Move to completedTransfers directory
−
|                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Create SIP from Transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Reject transfer'''<div class="mw-collapsible-content">
−
Create SIP(s)
+
<br/>Move to the rejected directory
−
|                                                                    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Create SIP from Transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Failed transfer'''<div class="mw-collapsible-content">
−
Move to processing directory
+
<br/>Move to the failed directory
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Create SIP from Transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Approve SIP creation'''<div class="mw-collapsible-content">
−
Create SIP from transfer objects
+
<br/>Approve SIP Creation
−
|                                                  
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Create SIP from Transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Verify SIP compliance'''<div class="mw-collapsible-content">
−
Move to completedTransfers directory
+
<br/>Set file permissions
−
|                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Reject transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Verify SIP compliance'''<div class="mw-collapsible-content">
−
Move to the rejected directory
+
<br/>Move to processing directory
−
|                                                    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Failed transfer'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Verify SIP compliance'''<div class="mw-collapsible-content">
−
Move to the failed directory
+
<br/>Verify SIP compliance
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Approve SIP creation'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Verify transfer compliance'''<div class="mw-collapsible-content">
−
Approve SIP Creation
+
<br/>Verify mets_structmap.xml compliance
−
|                                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Verify SIP compliance'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Failed compliance'''<div class="mw-collapsible-content">
−
Set file permissions
+
<br/>Failed compliance. See output in dashboard. SIP moved back to SIPsUnderConstruction
−
|                                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Verify SIP compliance'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Rename SIP directory with SIP UUID'''<div class="mw-collapsible-content">
−
Move to processing directory
+
<br/>Rename SIP directory with SIP UUID
−
|                                                       
+
</div>
−
|-
+
</div>
−
|  '''Verify SIP compliance'''<br/>
+
|
−
Verify SIP compliance
 
−
|                                                             
 
−
|-  
 
−
|  '''Verify transfer compliance'''<br/>
 
−
Verify mets_structmap.xml compliance
 
−
|                                               
 
−
|-
 
−
|  '''Failed compliance'''<br/>
 
−
Failed compliance. See output in dashboard. SIP moved back to SIPsUnderConstruction
 
−
|
 
−
|-
 
−
|  '''Rename SIP directory with SIP UUID'''<br/>
 
−
Rename SIP directory with SIP UUID
 
−
|                                                
 
 
|-  
 
|-  
−
|  '''Include default SIP processingMCP.xml'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Include default SIP processingMCP.xml'''<div class="mw-collapsible-content">
−
Include default SIP processingMCP.xml
+
<br/>Include default SIP processingMCP.xml
−
|                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Remove cache files'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Remove cache files'''<div class="mw-collapsible-content">
−
Remove cache files
+
<br/>Remove cache files
−
|                                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Clean up names'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Clean up names'''<div class="mw-collapsible-content">
−
Set file permissions
+
<br/>Set file permissions
−
|                                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Clean up names'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Clean up names'''<div class="mw-collapsible-content">
−
Sanitize SIP name
+
<br/>Sanitize SIP name
−
|                                                                  
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Check for Service directory
+
<br/>Check for Service directory
−
|                                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Check for Access directory
+
<br/>Check for Access directory
−
|                                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Grant normalization options for pre-existing DIP
+
<br/>Grant normalization options for pre-existing DIP
−
|                                  
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Grant normalization options for no pre-existing DIP
+
<br/>Grant normalization options for no pre-existing DIP
−
|                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Move to workFlowDecisions-createDip directory
+
<br/>Move to workFlowDecisions-createDip directory
−
|                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Find options to normalize as
+
<br/>Find options to normalize as
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Normalize     
+
<br/>Normalize     
−
|                                                                    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Process submission documentation'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Process submission documentation'''<div class="mw-collapsible-content">
−
Remove files without linking information (failed normalization artifacts etc.)
+
<br/>Remove files without linking information (failed normalization artifacts etc.)
−
|    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Process submission documentation'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Process submission documentation'''<div class="mw-collapsible-content">
−
Normalize submission documentation to thumbnail format
+
<br/>Normalize submission documentation to thumbnail format
−
|                            
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Verify checksums generated on ingest
+
<br/>Verify checksums generated on ingest
−
|                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Copy transfers metadata and logs
+
<br/>Copy transfers metadata and logs
−
|                                                  
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Generate METS.xml document
+
<br/>Generate METS.xml document
−
|                                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Normalize service files for access
+
<br/>Normalize service files for access
−
|                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Normalize access
+
<br/>Normalize access
−
|                                                                  
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Create DIP directory
+
<br/>Create DIP directory
−
|                                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Normalize for preservation and access
+
<br/>Normalize for preservation and access
−
|                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Normalize preservation
+
<br/>Normalize preservation
−
|                                                            
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Remove files without linking information (failed normalization artifacts etc.)
+
<br/>Remove files without linking information (failed normalization artifacts etc.)
−
|    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Move to approve normalization directory
+
<br/>Move to approve normalization directory
−
|                                            
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Approve normalization
+
<br/>Approve normalization
−
|                                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Move to processing directory
+
<br/>Move to processing directory
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Set file permissions
+
<br/>Set file permissions
−
|                                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Create thumbnails directory
+
<br/>Create thumbnails directory
−
|                                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Normalize thumbnails
+
<br/>Normalize thumbnails
−
|                                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Process submission documentation'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Process submission documentation'''<div class="mw-collapsible-content">
−
Move to processing directory
+
<br/>Move to processing directory
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Process submission documentation'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Process submission documentation'''<div class="mw-collapsible-content">
−
Set file permissions
+
<br/>Set file permissions
−
|                                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Process submission documentation'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Process submission documentation'''<div class="mw-collapsible-content">
−
Copy transfer submission documentation
+
<br/>Copy transfer submission documentation
−
|                                            
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Process submission documentation'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Process submission documentation'''<div class="mw-collapsible-content">
−
Check for submission documentation
+
<br/>Check for submission documentation
−
|                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Process submission documentation'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Process submission documentation'''<div class="mw-collapsible-content">
−
Move submission documentation into objects directory
+
<br/>Move submission documentation into objects directory
−
|                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Process submission documentation'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Process submission documentation'''<div class="mw-collapsible-content">
−
Assign file UUIDs to submission documentation
+
<br/>Assign file UUIDs to submission documentation
−
|                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Process submission documentation'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Process submission documentation'''<div class="mw-collapsible-content">
−
Assign checksums and file sizes to submissionDocumentation
+
<br/>Assign checksums and file sizes to submissionDocumentation
−
|                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Process submission documentation'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Process submission documentation'''<div class="mw-collapsible-content">
−
Extract packages in submission documentation
+
<br/>Extract packages in submission documentation
−
|                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Process submission documentation'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Process submission documentation'''<div class="mw-collapsible-content">
−
Sanitize file and directory names in submission documentation
+
<br/>Sanitize file and directory names in submission documentation
−
|                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Process submission documentation'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Process submission documentation'''<div class="mw-collapsible-content">
−
Scan for viruses in submission documentation
+
<br/>Scan for viruses in submission documentation
−
|                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Process submission documentation'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Process submission documentation'''<div class="mw-collapsible-content">
−
Characterize and extract metadata on submission documentation
+
<br/>Characterize and extract metadata on submission documentation
−
|                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Process submission documentation'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Process submission documentation'''<div class="mw-collapsible-content">
−
Identify Files ByExtension
+
<br/>Identify Files ByExtension
−
|                                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Process submission documentation'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Process submission documentation'''<div class="mw-collapsible-content">
−
Normalize submission documentation to preservation format
+
<br/>Normalize submission documentation to preservation format
−
|                          
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Prepare AIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Prepare AIP'''<div class="mw-collapsible-content">
−
Remove files without linking information (failed normalization artifacts etc.)
+
<br/>Remove files without linking information (failed normalization artifacts etc.)
−
|    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Prepare AIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Prepare AIP'''<div class="mw-collapsible-content">
−
Verify checksums generated on ingest
+
<br/>Verify checksums generated on ingest
−
|                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Prepare AIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Prepare AIP'''<div class="mw-collapsible-content">
−
Copy transfers metadata and logs
+
<br/>Copy transfers metadata and logs
−
|                                                  
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Prepare AIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Prepare AIP'''<div class="mw-collapsible-content">
−
Generate METS.xml document
+
<br/>Generate METS.xml document
−
|                                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Prepare DIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Prepare DIP'''<div class="mw-collapsible-content">
−
Copy thumbnails to DIP directory
+
<br/>Copy thumbnails to DIP directory
−
|                                                  
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Prepare DIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Prepare DIP'''<div class="mw-collapsible-content">
−
Copy METS to DIP directory
+
<br/>Copy METS to DIP directory
−
|                                                        
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Prepare DIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Prepare DIP'''<div class="mw-collapsible-content">
−
Set file permissions
+
<br/>Set file permissions
−
|                                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Prepare DIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Prepare DIP'''<div class="mw-collapsible-content">
−
Generate DIP  
+
<br/>Generate DIP  
−
|                                                                    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Prepare AIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Prepare AIP'''<div class="mw-collapsible-content">
−
Index AIP contents
+
<br/>Index AIP contents
−
|                                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Prepare AIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Prepare AIP'''<div class="mw-collapsible-content">
−
Prepare AIP   
+
<br/>Prepare AIP   
−
|                                                                    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Prepare AIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Prepare AIP'''<div class="mw-collapsible-content">
−
Select compression algorithm
+
<br/>Select compression algorithm
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Prepare AIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Prepare AIP'''<div class="mw-collapsible-content">
−
Select compression level
+
<br/>Select compression level
−
|                                                          
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Prepare AIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Prepare AIP'''<div class="mw-collapsible-content">
−
Compress AIP  
+
<br/>Compress AIP  
−
|                                                                    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Prepare AIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Prepare AIP'''<div class="mw-collapsible-content">
−
Set bag file permissions
+
<br/>Set bag file permissions
−
|                                                          
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Prepare AIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Prepare AIP'''<div class="mw-collapsible-content">
−
Removed bagged files
+
<br/>Removed bagged files
−
|                                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Store AIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Store AIP'''<div class="mw-collapsible-content">
−
Move to the store AIP approval directory
+
<br/>Move to the store AIP approval directory
−
|                                          
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Store AIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Store AIP'''<div class="mw-collapsible-content">
−
Store AIP     
+
<br/>Store AIP     
−
|                                                                    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Store AIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Store AIP'''<div class="mw-collapsible-content">
−
Store AIP location
+
<br/>Store AIP location
−
|                                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Store AIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Store AIP'''<div class="mw-collapsible-content">
−
Move to processing directory
+
<br/>Move to processing directory
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Store AIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Store AIP'''<div class="mw-collapsible-content">
−
Store the AIP
+
<br/>Store the AIP
−
|                                                                    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Store AIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Store AIP'''<div class="mw-collapsible-content">
−
Remove the processing directory
+
<br/>Remove the processing directory
−
|                                                    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Upload DIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Upload DIP'''<div class="mw-collapsible-content">
−
Select target CONTENTdm server
+
<br/>Select target CONTENTdm server
−
|                                                    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Upload DIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Upload DIP'''<div class="mw-collapsible-content">
−
Get list of collections on server
+
<br/>Get list of collections on server
−
|                                                  
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Upload DIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Upload DIP'''<div class="mw-collapsible-content">
−
Select destination collection
+
<br/>Select destination collection
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Upload DIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Upload DIP'''<div class="mw-collapsible-content">
−
Select upload type (Project Client or direct upload)
+
<br/>Select upload type (Project Client or direct upload)
−
|                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Upload DIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Upload DIP'''<div class="mw-collapsible-content">
−
Restructure DIP for CONTENTdm upload
+
<br/>Restructure DIP for CONTENTdm upload
−
|                                              
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Upload DIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Upload DIP'''<div class="mw-collapsible-content">
−
Upload DIP to contentDM
+
<br/>Upload DIP to contentDM
−
|                                                            
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Upload DIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Upload DIP'''<div class="mw-collapsible-content">
−
Upload DIP   
+
<br/>Upload DIP   
−
|                                                                    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Upload DIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Upload DIP'''<div class="mw-collapsible-content">
−
Move to the uploadedDIPs directory
+
<br/>Move to the uploadedDIPs directory
−
|                                                
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Reject DIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Reject DIP'''<div class="mw-collapsible-content">
−
Move to the rejected directory
+
<br/>Move to the rejected directory
−
|                                                    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Reject SIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Reject SIP'''<div class="mw-collapsible-content">
−
Move to the rejected directory
+
<br/>Move to the rejected directory
−
|                                                    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Failed SIP'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Failed SIP'''<div class="mw-collapsible-content">
−
Move to the failed directory
+
<br/>Move to the failed directory
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Find preservation links to run.
+
<br/>Find preservation links to run.
−
|                                                    
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Find access links to run.
+
<br/>Find access links to run.
−
|                                                          
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Normalize'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Normalize'''<div class="mw-collapsible-content">
−
Find thumbnail links to run.
+
<br/>Find thumbnail links to run.
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Process submission documentation'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Process submission documentation'''<div class="mw-collapsible-content">
−
Find thumbnail links to run.
+
<br/>Find thumbnail links to run.
−
|                                                      
+
</div>
 +
</div>
 +
|
 
|-  
 
|-  
−
|  '''Process submission documentation'''<br/>
+
|  <div class="toccolours mw-collapsible mw-collapsed">'''Process submission documentation'''<div class="mw-collapsible-content">
−
Find preservation links to run.                                                    |
+
<br/>Find preservation links to run.                                                    |
 +
</div>
 +
</div>
 
|
 
|
 
|-
 
|-
 
|}
 
|}

Revision as of 07:11, 24 August 2012

Main Page > Documentation > Technical Architecture > Micro-services > Archivematica 0.9 Micro-services

This page describes the key micro-services that are undertaken during transfer and ingest.

Transfer

Micro-service Description
Approve transfer Once the transfer has all its digital objects and has been formatted for processing, the user selects "Transfer complete" from the Actions drop-down menu.
Verify transfer compliance Verifies that the transfer conforms to the folder structure required for processing in Archivematica. The structure is as follows: /logs/, /metadata/, /metadata/submissionDocumentation/, /objects/.
Rename with transfer UUID Adds a unique universal identifier to the transfer folder name.
Include default Transfer processingMCP.xml Adds defaultTransferProcessing.xml file from /sharedDirectoryStructure/sharedMicroServiceTasksConfigs/ to the transfer directory. This xml file can configure workflow options such as creating transfer backups, quarantining the transfer and selecting a SIP creation option.
Workflow decision - create transfer backup The user can choose to create a complete backup of the transfer in case transfer or ingest are interrupted or fail. The transfer backup is placed in /sharedDirectoryStructure/transferBackups/ and will automatically be deleted once the AIP has been moved into storage.
Assign file UUIDs to objects Assigns a unique universal identifier to each file in the /objects/ directory.
Assign checksums and file sizes to objects Assigns a sha-256 checksum to each file in the /objects/ directory and calculates file sizes.
Verify metadata directory checksums Checks any checksum files that were placed in the /metadata/ folder of the SIP prior to ingest. Note that the filenames need to be named based on their algorithm: checksum.sha1, checksum.sha256, checksum.md5.
Generate METS.xml document Generates a basic mets file with a fileSec and structMap to record the presence of all objects in the /objects/ directory and their location in any subdirectories. Designed to capture the original order of the transfer in the event the user chooses subsequently to delete, rename or move files or break the transfer into multiple SIPs. The mets file is automatically added to any SIP generated from the transfer.
Extract packages Extracts objects from any zipped files or other packages.
Scan for viruses Uses ClamAV, parses the output and creates a PREMIS event. If a virus is found, the SIP is automatically placed in /sharedDirectoryStructure/failed/.
Sanitize object's file and directory names Some file systems do not support unicode or other special characters in filenames. This micro-service removes prohibited characters and replaces them with dashes. Original filenames are preserved in the PREMIS metadata.
Sanitize transfer name Same as above except does it for the transfer folder name.
Characterize and extract metadata Identifies and validates formats and extracts object metadata using the File Information Tool Set (FITS). Adds output to the PREMIS metadata.
Create SIP(s) The user chooses among a number of SIP creation options. See the user manual for details.


Ingest

Micro-service Description
Verify SIP compliance Verifies that the SIP conforms to the folder structure required for processing in Archivematica. The structure is as follows: /logs/, /metadata/, /metadata/submissionDocumentation/, /objects/.
Rename SIP directory with SIP UUID Adds a unique universal identifier to the SIP folder name.
Include default SIP processingMCP.xml Adds defaultSIPProcessing.xml file from /sharedDirectoryStructure/sharedMicroServiceTasksConfigs/ to the transfer directory. This xml file canbe used to configure SIP workflow options.
Remove thumbs.db files Removes any Thumbs.db files. May be expanded to others in future releases.
Sanitize object's file and directory names If user created new folder and/or file names during SIP creation, any prhobited characters are removed from these names and replaced with dashes.
Sanitize SIP name Same as above except that it's done for the SIP folder name.
Check for Service directory For digitization output workflows, checks to see if the SIP contains any service copies of master files. See the user manual for details.
Check for Access directory For digitization output workflows, checks to see if the SIP contains any access copies of master files. See the user manual for details.
Normalize The user can choose to create preservation and/or access copies of the ingested files based on rules in the transcoder database. These rules can be seen under the Preservation planning tab in the Archivematica dashboard.
Normalize access Creates access copies to be added to the DIP.
Normalize preservation Creates preservation copies to be included in the AIP along with the original files.
Approve normalization If desired, the user can verify the quality of normalized files in /sharedDirectoryStructure/watchedDirectories/approveNormalization/.
Check for submission documentation Checks for files in /metadata/submissionDocumentation/.
Move submission documentation to objects directory Moves the /submissionDocumentation/ directory from /metadata/ to /objects/ for ingest processing.
Assign file UUIDs to submission documentation Assigns a unique universal identifier to each file in the /objects/submissionDocumentation/ directory.
Assign checksums and filesizes to submission documentation Assigns a sha-256 checksum to each file in the /objects/submissionDocumentation directory and calculates file sizes.
Extract packages in submission documentation Extracts objects from any zipped files or other packages in the /objects/ directory.
Characterize and extract metadata on submission documentation Identifies and validates formats and extracts object metadata for files in the /objects/submissionDocumentation directory using the File Information Tool Set (FITS). Adds output to the PREMIS metadata.
Normalize submission documentation to preservation format Creates preservation copies of all files in the /objects/submissionDocumentation directory to be included in the AIP along with the original files.
Verify checksums generated on ingest Verifies checksums that were generated during transfer processing to ensure that the files have not been corrupted during transfer or ingest.
Remove empty directories Removes any empty directories from the SIP.
Generate METS.xml document Generates a METS file with PREMIS metadata. For more information on the METS file, see METS.
Copy transfers metadata and logs Copies all submission documentation included in the original transfer to the SIP. Copies all logs generated during transfer processing to the SIP.
Copy METS to DIP directory Creates a copy of the METS file in the DIP directory.
Generate DIP Moves the DIP to the DIP upload directory.
Upload DIP The user uploads the DIP to a selected description in the access system. See the user manual for details.
Prepare AIP Packages the SIP into an AIP using BagIt
Compress AIP Losslessly compresses the AIP for storage using p7zip.
Store AIP Moves the AIP to a specified directory. In the demo version of Archivematica the directory is /sharedDirectoryStructure/www/AIPsStore/. In other environments it can be a remote network mounted directory. The directory structure of the AIP store contains UUID quad directories and an index.html file listing the AIPs in storage. The index.html file is displayed in the dashboard in the Archival storage tab.

Once the AIP has been stored, a copy of the AIP is extracted from storage to a local temp directory, and is validated with the various BagIt checks: verifyvalid, checkpayloadoxum, verifycomplete, verifypayloadmanifests, verifytagmanifests.


0.9

Micro-service Description
Rename with transfer UUID
Rename with transfer UUID
Rename with transfer UUID
Extract packages
Approve transfer
Approve transfer
Approve transfer
Approve transfer
Approve transfer
Approve transfer
Approve transfer
Approve transfer
Approve transfer
Approve transfer
Verify transfer compliance
Include default Transfer processingMCP.xml
Include default Transfer processingMCP.xml
Include default Transfer processingMCP.xml
Verify transfer checksum
Generate METS.xml
Quarantine
Quarantine
Quarantine
Verify transfer checksums
Clean up names
Clean up names
Clean up names
Characterize and extract metadata
Approve transfer
Verify transfer compliance
Verify transfer compliance
Verify transfer compliance
Verify transfer compliance
Verify transfer compliance
Verify transfer compliance
Failed transfer compliance
Failed transfer compliance
Failed transfer compliance
Failed transfer compliance
Failed transfer compliance
Failed transfer compliance
Failed transfer compliance
Rename with transfer UUID
Include default Transfer processingMCP.xml
Assign file UUIDs and checksums
Assign file UUIDs and checksums
Assign file UUIDs and checksums
Assign file UUIDs and checksums
Assign file UUIDs and checksums
Assign file UUIDs and checksums
Assign file UUIDs and checksums
Assign file UUIDs and checksums
Assign file UUIDs and checksums
Verify transfer checksums
Generate METS.xml document
Quarantine
Quarantine
Quarantine
Quarantine
Quarantine
Quarantine
Quarantine
Quarantine
Quarantine
Quarantine
Quarantine
Quarantine
Quarantine
Extract packages
Extract packages
Scan for viruses
Clean up names
Clean up names
Characterize and extract metadata
Characterize and extract metadata
Characterize and extract metadata
Complete transfer
Complete transfer
Complete transfer
Create SIP from Transfer
Create SIP from Transfer
Create SIP from Transfer
Create SIP from Transfer
Create SIP from Transfer
Reject transfer
Failed transfer
Approve SIP creation
Verify SIP compliance
Verify SIP compliance
Verify SIP compliance
Verify transfer compliance
Failed compliance
Rename SIP directory with SIP UUID
Include default SIP processingMCP.xml
Remove cache files
Clean up names
Clean up names
Normalize
Normalize
Normalize
Normalize
Normalize
Normalize
Normalize
Process submission documentation
Process submission documentation
Normalize
Normalize
Normalize
Normalize
Normalize
Normalize
Normalize
Normalize
Normalize
Normalize
Normalize
Normalize
Normalize
Normalize
Normalize
Process submission documentation
Process submission documentation
Process submission documentation
Process submission documentation
Process submission documentation
Process submission documentation
Process submission documentation
Process submission documentation
Process submission documentation
Process submission documentation
Process submission documentation
Process submission documentation
Process submission documentation
Prepare AIP
Prepare AIP
Prepare AIP
Prepare AIP
Prepare DIP
Prepare DIP
Prepare DIP
Prepare DIP
Prepare AIP
Prepare AIP
Prepare AIP
Prepare AIP
Prepare AIP
Prepare AIP
Prepare AIP
Store AIP
Store AIP
Store AIP
Store AIP
Store AIP
Store AIP
Upload DIP
Upload DIP
Upload DIP
Upload DIP
Upload DIP
Upload DIP
Upload DIP
Upload DIP
Reject DIP
Reject SIP
Failed SIP
Normalize
Normalize
Normalize
Process submission documentation
Process submission documentation
Retrieved from "https://wiki.archivematica.org/index.php?title=Archivematica_0.9_Micro-services&oldid=6395"