Archivematica 1.10 and Storage Service 0.15 release notes

Home > Release Notes > Archivematica 1.10 and Storage Service 0.15

Release date September 5, 2019

Supported environments

Please see the installation instructions.

Archivematica 1.10 and Storage Service 0.15 are supported for production use in the following environments:

• Ubuntu 16.04 64-bit Server Edition
• Ubuntu 18.04 64-bit Server Edition
• CentOS 7 64-bit

For development purposes, most of our developers prefer to use Docker containers. These and all above supported environments are linked from the installation instructions above.

Added

Information Packaging (Backlog) Workflow Enhancements

Simon Fraser University has sponsored a development project to improve the ability to store Transfers for a long period of time. The goal is to improve the metadata quality of Transfers to reduce the risk of storing them for a long period of time in a backlog. As a result of this project:

• Transfers placed in backlog are now packaged as bags
• The Transfer METS file now records all preservation actions that took place during Transfer.
• The Appraisal Tab now handles packages with more than 1,000 files.
• A backlog can be populated from a new pipeline from transfers created by another pipeline.

• Documentation: https://www.archivematica.org/en/docs/archivematica-1.10/user-manual/appraisal/appraisal/#appraisal
• Issues (linked from epic): https://github.com/archivematica/Issues/issues/391

"Vintage" AIP handling

AIPS created in Archivematica prior to version 1.0 can’t currently be re-ingested in more recent versions (due to namespace collisions with the use of PREMIS 2.1).

This change will make it possible to reingest these older AIPS into Archivematica. This work is sponsored by the City of Vancouver Archives.

• Issue: https://github.com/archivematica/Issues/issues/24

LDAP support for Storage Service

The Archivematica Storage Service now supports LDAP integration. This was sponsored by Piql for the Norwegian Health Archives.

• Issue: https://github.com/archivematica/Issues/issues/680

External PID declaration

This feature, sponsored by the International Institute of Social History, allows users to include identifiers minted outside of Archivematica in their transfer and written to the premis:object metadata in the AIP METS file. This is done via an identifiers.json file included in the transfer.

• Issue: https://github.com/archivematica/Issues/issues/133
• Documentation: https://www.archivematica.org/en/docs/archivematica-1.10/user-manual/transfer/transfer/#transfer-pids

Avalon Integration

Archivematica 1.10 has added the ability to integrate with Avalon Media System through the automation tools. This was sponsored by Northwestern University and Indiana University through IMLS funding. The feature allows users to prepare and store a DIP appropriate for upload the Avalon, and then have the automation tools send it to Avalon for ingest into that system.

• Issue: https://github.com/archivematica/Issues/issues/643
• Documentation: https://www.archivematica.org/en/docs/archivematica-1.10/admin-manual/installation-setup/integrations/integrations/#avalon-media-system-integration

Include AIP UUID in bag-info.txt

This change adds the UUID of the AIP to the External-Identifier field of the bag-info.txt file. This change is a community contribution by Helen Sherwood-Taylor (Wellcome Collection)- thank you!

• Issue: https://github.com/archivematica/Issues/issues/492
• Documentation: https://www.archivematica.org/en/docs/archivematica-1.10/user-manual/archival-storage/aip-structure/#bagit-doc

Allow designation of ArchivesSpace repository using DSpace REST location

This feature allows designation of more than one ArchivesSpace repository by enabling it via the DSpace REST location rather than through the configuration in the Storage Service. This was a community contribution by Hrafn Malmquist (University of Edinburgh)- thank you!

• Issue: https://github.com/archivematica/Issues/issues/435
• Documentation: https://www.archivematica.org/en/docs/storage-service-0.15/administrators/#dspace-via-rest-api

Changed

AIP METS in PREMIS 3

As a result of the vintage AIP work described above, we have upgraded the AIP METS file to use PREMIS 3 exclusively. In previous versions it had a mix of versions 2 and 3.

• Issue: https://github.com/archivematica/Issues/issues/370

API endpoint for manifest validation (beta)

As part of the Avalon integration described above, we added an endpoint that can be called to validate a manifest. While this is currently only implemented for the Avalon manifest, it could for example be extended in the future for metadata.csv validation and similar.

• Issue: https://github.com/archivematica/Issues/issues/618
• Documentation: https://wiki.archivematica.org/Archivematica_API#Validate

Extended service callbacks

This change allows the creation of callbacks for AIP, AIC, and DIP storage events. This was done in service to integration with SCOPE, a DIP access platform in use by the Canadian Centre for Architecture, but is widely applicable to other use cases.

• Issue: https://github.com/archivematica/Issues/issues/147

Storage Service packages tab tidy-up

This change was the result of a reported bug for the Storage Service packages tab timing out- we addressed that issue but also tidied up the tab to make it more generally usable. See the issue for discussion of changes.

• Issue: https://github.com/archivematica/Issues/issues/676
• Documentation: https://www.archivematica.org/en/docs/storage-service-0.15/administrators/#packages-tab

Code formatting to be handled by black

The developer team has decided to use a code formatting tool (black) to make formatting more consistent. Pull requests to Archivematica are now checked with a linter to ensure consistency with black.

• Issue: https://github.com/archivematica/Issues/issues/393

Additions and improvements to the AMAUATs

The AMAUATs are the Archivematica Automated User Acceptance Tests. Thanks to sponsorship by Wellcome Collection we have greatly expanded the number of automated tests that run and made most of them "black box" tests that run via the API- this means they are more robust than tests that rely on the user interface, which can break anytime the user interface changes in some way. In addition to these improvements, to aid in the release process we have made the AMAUATs have versions that go along with the Archivematica version being released. Please see the AMAUAT repo for more information: https://github.com/artefactual-labs/archivematica-acceptance-tests

Fixed

• A number of fixes related to custom structMap import have been sponsored in this release by International Institute of Social History. Thank you!
  • Custom structMaps do not support nested directories: https://github.com/archivematica/Issues/issues/283
  • Purpose of Verify structMap in ingest is unclear: https://github.com/archivematica/Issues/issues/286
  • Custom structMaps labelled as structMap_2: https://github.com/archivematica/Issues/issues/633
  • We also added documentation for custom structMap import: https://www.archivematica.org/en/docs/archivematica-1.10/user-manual/transfer/import-metadata/#import-metadata
• Verify transfer checksums succeeding without verifying the checksums: https://github.com/artefactual/archivematica/issues/1061
• Parallel bzip2 algorithm isn't written to the pointer file when used: https://github.com/archivematica/Issues/issues/714
• metsrw cannot process unicode characters: https://github.com/archivematica/Issues/issues/295
• Deleted AIPs are not removed from S3: https://github.com/archivematica/Issues/issues/696
• Cannot store AIP in S3 us-west-2-region: https://github.com/archivematica/Issues/issues/639
• S3 bucket does not use the Region set in S3 space: https://github.com/archivematica/Issues/issues/638
• File list pane always removes last tag of a file (Sponsored by Simon Fraser University- thank you!): https://github.com/archivematica/Issues/issues/472
• Storage Service import_aip leaves uncompressed AIP data in /tmp: https://github.com/archivematica/Issues/issues/706
• AgentArchives does not log out of ArchivesSpace (Community contribution by Hrafn Malmquist (University of Edinburgh)- thank you!): https://github.com/artefactual-labs/agentarchives/issues/47
• Storage Service sort by size does not increment correctly: https://github.com/archivematica/Issues/issues/678
• verify_checksum PREMIS events have no agent information: https://github.com/archivematica/Issues/issues/774
• Versions shown for FPR tools are outdated: https://github.com/archivematica/Issues/issues/191
• Jhove failure event has result "well-formed and valid": https://github.com/archivematica/Issues/issues/164
• Create METS script loops after exception (Community contribution by Helen Sherwood-Taylor (Wellcome Collection)- thank you!): https://github.com/archivematica/Issues/issues/620
• Archivematica should pin exact package versions in requirements.txt for predictable deployments (Community contribution by Helen Sherwood-Taylor (Wellcome Collection)- thank you!): https://github.com/archivematica/Issues/issues/634
• Delete links are not consistent in the Packages tab of the Storage Service: https://github.com/archivematica/Issues/issues/711
• MCP server doesn't report when the Gearman server is unavailable (Community contribution by Alex Chan (Wellcome Collection)- thank you!): https://github.com/archivematica/Issues/issues/553
• Archivematica uses both bagit-python and bagit-java: https://github.com/archivematica/Issues/issues/246
• DSpace REST location does not fall back to default values (Community contribution by Hrafn Malmquist (University of Edinburgh)- thank you!): https://github.com/archivematica/Issues/issues/458
• Validate function of bind_pids occurs too early in the microservice script: https://github.com/archivematica/Issues/issues/776
• Storage Service object-counting disabling option fails: https://github.com/archivematica/Issues/issues/657
• chunkids in manifest file do not match id in DuraCloud: https://github.com/archivematica/Issues/issues/574
• Archivematica attempts to delete files from transfer source: https://github.com/archivematica/Issues/issues/646
• Double replications being creation from first replication, not original (Sponsored by Piql for the Norwegian Health Archive- thank you!): https://github.com/artefactual/archivematica-storage-service/issues/270
• AtoM DIP upload link in Access tab is wrong: https://github.com/archivematica/Issues/issues/411
• Metadata-only DIP fails without format version: https://github.com/archivematica/Issues/issues/857

And more! For a complete list of fixes and changes please see: https://github.com/archivematica/Issues/milestone/6

Upgraded tools and dependencies

• Ghostscript upgraded to 9.2.x across all platforms: https://github.com/archivematica/Issues/issues/714
• JHOVE upgraded to 1.20: https://github.com/archivematica/Issues/issues/521

Deprecated

Issue 174 As of Archivematica 1.10, Archivist's Toolkit integration has been removed. Please see this announcement for more details.