Archivematica 1.8 and Storage Service 0.13 release notes

From Archivematica
Jump to navigation Jump to search

Home > Release Notes > Archivematica 1.8 and Storage Service 0.13

Release date: Nov 20th 2018

Supported environments[edit]

Please see installation instructions link here.

Archivematica 1.8 and Storage Service 0.13 are supported for production use in the following environments:

  • Ubuntu 16.04 64-bit Server Edition
  • Ubuntu 18.04 64-bit Server Edition
  • CentOS 7 64-bit

For development purposes, most of our developers prefer to use Docker containers. These and all above supported environments are linked from the installation instructions above.

Added[edit]

Dataverse integration[edit]

Archivematica can now be configured to use a Dataverse research data repository as a transfer source location. Dataverse transfer source locations can be configured to display all available datasets or a subset of them. Datasets are retrieved directly using the Dataverse API and processed using a new “Dataverse” transfer type. New Dataverse specific processing includes:

  • fixity checking using checksums generated by dataverse
  • retrieval of derivative and metadata files associated with tabular data files
  • creation of a Dataverse METS file describing the dataset as retrieved from Dataverse
  • Dataverse metadata included in the AIP METS

Some advanced or more complex use cases are not fully supported, such as handling of datasets with restricted files, versioning of datasets and reingest of datasets. For a full list of known issues and enhancement ideas, refer to the Archivematica issues repository using Dataverse label and the Archivematica-Dataverse Integration Project Wiki.

This work was sponsored by Scholars Portal, a service of the Ontario Council of University Libraries (OCUL). Thank you!

Public URL[edit]

Archivematica now has a concept of a public URL in the general configuration. Archivematica usually registers itself with the Storage Service, but if you have deployed Archivematica in an environment where the URL or IP address changes frequently (i.e. in some Docker environments) this can cause issues. In these types of environments, users can now declare a stable public URL that Archivematica can use to register with the Storage Service.

This work was sponsored by Jisc. Thank you!

Package storage in DSpace via its REST API[edit]

Since Archivematica 1.6 it's been possible to store AIPs in DSpace, but they have been stored via SWORD. Some users have the need/desire to store packages in DSpace via the REST API. This is now possible in Storage Service 0.13.

This work was undertaken by Hrafn Malmquist at University of Edinburgh, with assistance from developers at Artefactual Systems. Thank you!

Changed[edit]

Enhancements or major fixes.

Automatic approval of transfers[edit]

It is no longer necessary to approve transfers started in the user interface. By default, a checkbox is ticked for the transfer to automatically be approved- users may uncheck the box and approve their transfers manually if desired.

This enhancement was sponsored by Jisc. Thank you!

Streamline checksum verification[edit]

This enhancement de-duplicates checksum verification in Archivematica, which helps to improve the performance of Archivematica in processing large transfers (many files and/or large files). This enhancement includes three changes:

  • Remove the "Verify checksums generated on ingest" micro-service
  • Enhance the "Verify AIP" micro-service to bulk query the database for transfer-generated checksums and then verify that they match what is documented in the bag-generated manifest-<ALGORITHM>.txt.
  • Have "Verify AIP" create an AIP-level "fixity check" PREMIS:EVENT that it can pass to the Storage Service, which will document this verification in the pointer file.

This should not impact regular workflows, but it is worth noting that there is no AIP-level fixity check PREMIS event for uncompressed AIPs, which don't have pointer files. For uncompressed AIPs, there are still object-level fixity events in place. Note that there is an issue in the Archivematica Issues repository regarding this note - Problem: uncompressed AIPs need pointer files #32

This work was sponsored by Columbia University Library. Thank you!

Indexing can be enabled/disabled for Transfers and/or Archival Storage[edit]

Previously, the ElasticSearch index feature could be disabled globally as a scalability measure since indexing consumes a lot of resources. However, this also disabled Backlog and Appraisal features (which also uses indexing) and which some users still wanted to access. As of release 1.8, Archivematica can be deployed to run with indexing enabled just for Transfers (Backlog and Appraisal enabled), just for Archival Storage (Backlog and Appraisal disabled), for both indexes, or for none.

Configure email settings[edit]

This change improves the ways that the email client in Archivematica can be configured, including allowing an administrator to set the sender email address for emails sent by Archivematica (i.e. normalization reports, failure reports) to comply with local IT requirements.

This work was sponsored by Jisc. Thank you!

Download processing configuration and reset to default[edit]

Previous versions of Archivematica introduced the ability to add custom processing configurations, but users had to retrieve the custom configuration file via the command line to use it. There is now a download button on Administration > Processing configuration so that you can download the processing config from the user interface.

You can also reset a processing configuration to the installation pre-set by clicking on the new reset button on Administration > Processing configuration.

The documentation for using a custom processing configuration has also been updated.

This work was sponsored by Jisc. Thank you!

MCP batching for scalability & performance[edit]

This feature refactors how tasks are scheduled, executed & managed within Archivematica, by grouping tasks into batches. It introduces processing efficiencies that significantly decrease the processing power and time required to complete Transfer and Ingest. It includes new configuration options to further optimize processing efficiency for particular types of Transfers (e.g. few large files vs. many small files) and for different deployment patterns (e.g. installing components across multiple machines).

This work was sponsored by Jisc. Thank you!

Binder integration improvements[edit]

Archivematica has had an integration with Binder for several years. Binder is an open-source web application for managing time-based media and born-digital artworks. Binder depends on integration with both Archivematica and TMS (The Museum System).

Since Binder is built off of AtoM, much of the integration configuration was repurposed from the AtoM integration. Archivematica 1.8 makes it explicit, for example, that the job "DIP Upload to AtoM" is actually "DIP Upload to AtoM/Binder". In the Administration tab, the configuration section for AtoM has also been renamed to include Binder.

Enhancing the Binder integration itself, Archivematica's transfer tab now includes an "Access system ID" box. This allows users to pre-populate an access system ID for AtoM or Binder, so that DIPs can be automatically uploaded without having to stop at the Upload DIP microservice. Users can still use the Upload DIP popup if desired.

Finally, we've added documentation on using Binder with Archivematica.

This work was sponsored by Tate. Thank you!

Translations[edit]

First added in 1.7, translations in Archivematica are growing! In this release, we've pulled in translations for Archivematica.org, the documentation, the FPR, the Storage Service, and the Archivematica interface. The biggest part that is still missing is the Archivematica's workflow engine - that is, all of the microservice and job names in the interface. We are planning to include workflow translations in 1.9.

Thanks to our wonderful community of translators on Transifex, Archivematica resources can now be translated from English into French, Spanish, Japanese, Portuguese, Brazilian Portuguese, and Swedish. Note that the completeness of each language for each resource depends on volunteer contributions in Transifex.

This work was originally sponsored by the Canadian Council on Archives through a DHCP (Documentary Heritage Community Programs) grant. Thank you!

File format identification updates[edit]

Archivematica 1.8 is now up to date with PRONOM v.94! For more information on new data added to PRONOM, check the PRONOM release notes.

This work was sponsored by the Denver Art Museum. Thank you!

Thumbnail normalization changes[edit]

It is now easier to configure whether or not, and how, thumbnails are created. In the processing configuration, users can choose between yes, normalize for thumbnails, no, do not normalize for thumbnails, or yes, do so but only if there is a default rule in place. For users who do not need thumbnails this could make their processing faster.

This work was sponsored by Columbia University Library. Thank you!

Fixed[edit]

Upgraded tools and dependencies[edit]

  • Fido has been upgraded to version 1.3.12
  • Siegfried has been upgraded to version 1.7.10
  • FITS has been upgraded to version 1.1.0
  • gunicorn has been upgraded to version 19.9.0

End of life dependencies[edit]

Archivists' Toolkit integration[edit]

Archivists' Toolkit has been deprecated since 2013. The Archivists' Toolkit DIP upload feature has not had active development or testing since then. There are no plans to start testing or to fix any problems with the feature. As a result, there is a proposal deprecate this feature in Archivematica 1.9. Community response is welcome via a comment on the issue in GitHub.

Known issues[edit]

  • There is a bug preventing the CONTENTdm workflow in this release. There will be a patch available with a fix.
Retrieved from "https://wiki.archivematica.org/index.php?title=Archivematica_1.8_and_Storage_Service_0.13_release_notes&oldid=12858"