Archivematica 1.0 Micro-services

Main Page > Documentation > Technical Architecture > Micro-services > Archivematica 1.0 Micro-services

A micro-service may consist of a number of discrete tasks, or jobs. In the Archivematica 1.o dashboard, micro-services are always shown, while jobs may be viewed by expanding the micro-service (i.e. by clicking on the grey background behind the micro-service name).

Arhivematica dashboard showing a micro-service and its jobs

The table below shows micro-services and jobs in Archivematica 1.0. Note that this is is only a list of micro-services; detailed user instructions are available in the user manual.

Transfer

Micro-service	Description
Approve Transfer Set file permissions	This is the approval step that moves the transfer into the Archivematica processing pipeline.
Verify transfer compliance Set file permissions Move to processing directory Set transfer type: (Standard, Zipped bag, Unzipped bag, DSpace, Maildir) Remove hidden files and directories Remove unneeded files Attempt restructure for compliance Verify transfer compliance Verify mets_structmap.xml compliance	Moves the transfer to a processing directory based on selected transfer type (standard, zipped bag, unzipped bag, DSPace export or maildir). Verifies that the transfer conforms to the folder structure required for processing in Archivematica and restructures if required. The structure is as follows: /logs/, /metadata/, /metadata/submissionDocumentation/, /objects/.
Rename with transfer UUID Rename with transfer UUID	Directly associates the transfer with its metadata by appending the transfer UUID to the transfer directory name.
Include default Transfer processingMCP.xml Include default Transfer processingMCP.xml	Adds a file named processingMCP.xml to the root of the transfer. This is a configurable xml file to pre-configure processing decisions. It can configure workflow options such as creating transfer backups, quarantining the transfer and selecting a SIP creation option.
Assign file UUIDs and checksums Set file permissions Assign file UUIDs to objects Assign checksums and file sizes to objects	Assigns a unique universal identifier and sha-256 checksum to each file in the /objects/ directory and sets file permission to allow for continued processing.
Verify transfer checksums Verify metadata directory checksums	Checks any checksum files that were placed in the /metadata/ folder of the transfer prior to moving the transfer into Archivematica.
Generate METS.xml document Generate METS.xml document	Generates a basic METS file with a fileSec and structMap to record the presence of all objects in the /objects/ directory and their locations in any subdirectories. Designed to capture the original order of the transfer in the event the user chooses subsequently to delete, rename or move files or break the transfer into multiple SIPs. A copy of the METS file is automatically added to any SIP generated from the transfer.
Quarantine Workflow decision - send transfer to quarantine Move to quarantine Remove from quarantine	Quarantine's the transfer for a set duration, to allow virus definitions to update, before virus scan.
Scan for viruses Scan for viruses	Uses ClamAV to scan for viruses and other malware. If a virus is found, the transfer is automatically placed in /sharedDirectoryStructure/failed/ and all processing on the transfer is stopped.
Clean up names Sanitize object's file and directory names Sanitize Transfer name	Some file systems do not support unicode or other special characters in filenames. This micro-service removes prohibited characters and replaces them with dashes. Original filenames are preserved in the PREMIS metadata.
Identify file format Move to select file ID tool Select file format identification command Determine which files to identify Identify file format	Identifies formats of the objects in the transfer using either FIDO or file extension based on user choice. Format types are managed in the Format Policy Registry. This micro-service can be skipped and done in Ingest instead.
Extract packages Extract contents from compressed archives	Extracts objects from any zipped files or other packages. Extracts attachments from maildir transfers.
Characterize and extract metadata Characterize and extract metadata Load labels from metadata/file_labels.csv Set file permissions Check for specialized processing	Characterizes and validates formats and extracts object metadata using the File Information Tool Set (FITS).
Complete transfer Index transfer contents Move to SIP creation directory for completed transfers	Indexes transfer contents, then marks the transfer as complete.
Create SIP from Transfer Check transfer directory for objects Load options to create SIPs Create SIP(s) Move to processing directory Create SIP from transfer objects Send transfer to backlog Move to SIP creation directory for completed transfers	This is the approval step that moves the transfer to the SIP packaging micro-services (Ingest) if user chooses to Create single SIP and continue processing. User can also choose to Send transfer to backlog at this time.

Ingest

Micro-service	Description
Verify SIP compliance Set file permissions Move to processing directory Verify SIP compliance	Verifies that the SIP conforms to the folder structure required for processing in Archivematica. The structure is as follows: /logs/, /metadata/, /metadata/submissionDocumentation/, /objects/.
Rename SIP directory with SIP UUID Rename SIP directory with SIP UUID	Directly associates the SIP with its metadata by appending the SIP UUID to the SIP directory name.
Include default SIP processingMCP.xml Include default SIP processingMCP.xml	Copies the processing config file added to the transfer in Include default Transfer processingMCP.xml, above, to the SIP.
Remove cache files Remove cache files	Removes any Thumbs.db files.
Normalize Identify manually normalized files Check for Service directory Check for Access directory Find options to normalize as Create DIP directory Normalize Normalize access Normalize preservation Remove files without linking information (failed normalization artifacts etc.) Create thumbnails directory Normalize thumbnails Verify checksums generated on ingest Approve normalization Check for manual normalized files Move access files to DIP Assign UUIDs to manually normalized preservation files Assign checksums to manually normalized preservation files Run FITS on manually normalized preservation files Relate manually normalized preservation files to the original files	Determines which normalization options are available for the SIP and presents them to the user as choices. Normalizes (i.e. generates preservation and/or access copies) based on selection. Thumbnail files are also generated during this micro-service.
Process submission documentation Check for submission documentation Copy transfer submission documentation Move to processing directory Set file permissions Assign file UUIDs to submission documentation Assign checksums and file sizes to submissionDocumentation Extract packages in submission documentation Sanitize file and directory names in submission documentation Scan for viruses in submission documentation Characterize and extract metadata on submission documentation Identify Files ByExtension Normalize submission documentation to preservation format	Processes any submission documentation included in the SIP and adds it to the /objects/ directory.
Prepare DIP Copy thumbnails to DIP directory Copy METS to DIP directory Set file permissions Generate DIP	Creates a DIP containing access copies of the objects, thumbnails and a copy of the METS file.
Upload DIP Upload DIP	Allows the user to choose to upload the DIP to either ICA-AtoM or CONTENTdm.
Upload DIP to ICA-AtoM Upload DIP Move to the uploadedDIPs directory	The user uploads the DIP to a selected description in ICA-AtoM.
Upload DIP to CONTENTdm Restructure DIP for CONTENTdm upload Select upload type (Project Client or direct upload) Select target CONTENTdm server Get list of collections on server Select destination collection Upload DIP to contentDM Move to the uploadedDIPs directory	The user uploads the DIP to a selected description in CONTENTdm.
Prepare AIP Copy transfers metadata and logs Remove files without linking information (failed normalization artifacts etc.) Verify checksums generated on ingest Generate METS.xml document Index AIP contents Prepare AIP Select compression algorithm Select compression level Compress AIP Set bag file permissions Removed bagged files	Creates an AIP in Bagit format. Indexes the AIP, then losslessly compresses it.
Store AIP Move to the store AIP approval directory Store AIP Store AIP location Store the AIP	Moves the AIP to /sharedDirectoryStructure/www/AIPsStore/ or another specified directory. Once the AIP has been stored, a copy of it is extracted from storage to a local temp directory, where it is subjected to standard BagIt checks: verifyvalid, checkpayloadoxum, verifycomplete, verifypayloadmanifests, verifytagmanifests.