Micro-services

Main Page > Documentation > Technical Architecture > Micro-services

The Archivematica micro-services are granular system tasks which operate on a conceptual entity that is equivalent to an OAIS information package: Submission Information Package (SIP), Archival Information Package (AIP), Dissemination Information Package (DIP). The physical structure of an information package will include files, checksums, logs, XML metadata, etc..

These information packages are moved from one service to the next using the Unix pipeline design pattern. Each micro-service is defined in a simple XML configuration file and associated with a watched directory. When an information package is moved to that directory it triggers the micro-service.

Each service is provided by a combination of Archivematica Python scrips and one or more open-source software tools bundled in the Archivematica system. Each micro-service results in a success or error state and the information package is moved accordingly to a success or error directory. Each success or error directory is the watched directory for a subsequent micro-service. This allows for the chaining of directories into complex, custom workflows. Archivematica implements a default ingest to access workflow that is compliant with the ISO-OAIS functional model.

Archivematica Micro-services

Micro-service	Description
Create SIP backup	Archivematica automatically creates a backup of the entire SIP as soon as it is ingested.
Verify SIP compliance	This micro-service verifies that the SIP conforms to the folder structure required for processing in Archivematica.
Assign file UUID and checksums	Each file in the SIP is assigned a universal unique identifer and a sha-1 checksum.
Verify metadata directory checksums	If the SIP contained a checksum.md5 file on ingest, Archivematica will check it to confirm that none of the files were deleted or altered on ingest.
Create DC	A Dublin Core xml template is added to the metadata folder in the SIP. The user can fill in fields as desired. The elements map to fields in ICA-AtoM when the DIP is uploaded.
Appraise SIP for submission	The archivist reviews the SIP, if desired, to confirm that it complies with any submission agreements. The archivist can delete unwanted files at this point; Archivematica will keep a log of the deleted files.
Quarantine	The SIP is placed in quarantine for a pre-set period of time. The archivist can move the SIP out of quarantine before the pre-set time has expired, if desired.
Extract packages	Files are extracted from any .zip files or other packages; each extracted file is assigned a universal unique identifier and a sha-1 checksum.
Sanitize file and directory names	Prohibited characters, such as spaces or ampersands, are removed from file and folder names and replaced with underscores.
Scan for viruses	ClamAV scans all files. In the event that a virus or other malware is found, the SIP is placed in a folder called SIPerrors and all processing on the SIP is stopped.
Characterize and extract metadata	File formats are identified and the files validated against external format specifications. Technical metadata is extracted from the file.
Appraise SIP for preservation	The archivist appraises the contents of the SIP, if desired, and deletes unwanted files. Archivematica will keep a log of the deleted files. In future releases of Archivematica, appraisal will be assisted by summary technical information on file formats, validation status and the presence of characteristics that might affect preservation.
Normalize	Archivematica creates a preservation copy and an access copy of each file. For more on normalization, see Media type preservation plans.
Compile METS file	Archivematica compiles a METS file with a complete set of PREMIS metadata for each ingested file. The technical metadata that were extracted during the "Characterize and extract metadata" micro-service are placed in the PREMIS objectCharacteristicsExtension element.
Create AIP checksum	A checksum for all the contents of the AIP is generated.
Prepare AIP	The AIP is packaged using the Library of Congress Bagit specification.
Store AIP	The archivist reviews the AIP if desired, and approves it for archival storage. The AIP is moved into the AIPsStore folder, which is linked to the institution's storage system.
Generate DIP	The access copies that were created during the "Normalize" micro-service are placed in a DIP folder and the METS file is added to the DIP.
Upload DIP	The archivist reviews the DIP, if desired, and removes any access copies that cannot be sent to the public access system due to copyright, security or other issues. The archivist then approves the DIP for upload and the DIP is uploaded into the public access system (in Archivematica, the default access system is the open-source archival description tool ICA-AtoM). A backup copy of the DIP, including files that were deleted, is sent to the DIPbackups folder.