Difference between revisions of "Archivematica 1.8 and Storage Service 0.13 release notes"

From Archivematica
Jump to navigation Jump to search
Line 33: Line 33:
 
* Dataverse metadata included in the AIP METS
 
* Dataverse metadata included in the AIP METS
  
−
Dataverse integration is currently a “Beta” feature. Some advanced or more complex use cases are not fully supported, such as handling of datasets with restricted files, versioning of datasets and reingest of datasets. For a full list of known issues and enhancement ideas, refer to the [https://github.com/archivematica/Issues/labels/OCUL%3A%20AM-Dataverse Archivematica issues repository using Dataverse label] and the [https://wiki.archivematica.org/Dataverse Archivematica Wiki].   
+
Some advanced or more complex use cases are not fully supported, such as handling of datasets with restricted files, versioning of datasets and reingest of datasets. For a full list of known issues and enhancement ideas, refer to the [https://github.com/archivematica/Issues/labels/OCUL%3A%20AM-Dataverse Archivematica issues repository using Dataverse label] and the [https://wiki.archivematica.org/Dataverse Archivematica Wiki].   
  
 
This work was sponsored by [https://scholarsportal.info/ Scholars Portal], a service of the Ontario Council of University Libraries (OCUL). Thank you!
 
This work was sponsored by [https://scholarsportal.info/ Scholars Portal], a service of the Ontario Council of University Libraries (OCUL). Thank you!

Revision as of 14:09, 24 October 2018

Home > Release Notes > Major release notes template

Work in progress

Supported environments

Link to installation instructions.

Specify supported environments.

Make special note of any changes to supported environment.

Added

Describe new features.

New feature template

This is a description of this amazing feature! Here's why it's a net benefit to the project and the community. Also included are any special notes, like if it's a beta feature.

This work was sponsored by some amazing institution. Thank you!

  • Documentation: link
  • Pull requests: link

Dataverse integration

Archivematica can now be configured to use a Dataverse research data repository as a transfer source location. Dataverse transfer source locations can be configured to display all available datasets or a subset of them. Datasets are retrieved directly using the Dataverse API and processed using a new “Dataverse” transfer type. New dataverse specific processing includes:

  • fixity checking using checksums generated by dataverse
  • retrieval of derivative and metadata files associated with tabular data files
  • creation of a Dataverse METS file describing the dataset as retrieved from Dataverse
  • Dataverse metadata included in the AIP METS

Some advanced or more complex use cases are not fully supported, such as handling of datasets with restricted files, versioning of datasets and reingest of datasets. For a full list of known issues and enhancement ideas, refer to the Archivematica issues repository using Dataverse label and the Archivematica Wiki.

This work was sponsored by Scholars Portal, a service of the Ontario Council of University Libraries (OCUL). Thank you!

Processing configuration reset and download buttons

A new installation of Archivematica comes with a pre-set processing configuration called "default", and a second one (used only in Jisc workflows) called "automated". In testing, users are encouraged to change the configurations to suit their workflows, but may need to reset the configuration to the installation pre-sets. A reset button has been added so that users can easily change the default and automated processing configurations back to their installation pre-sets.

The second part of this feature is the addition of a download button for the processing configuration files. If you create a custom processing configuration, you can download the resulting processingMCP.xml file using the button and then include it at the top level of your transfer. Archivematica will then use this to automate your transfer selections, rather than the default configuration.

This work was sponsored by Jisc. Thank you!

Changed

Describe enhancements or major fixes.

Streamline checksum verification

This enhancement de-duplicates checksum verification in Archivematica, which helps to improve the performance of Archivematica in processing large transfers (many files and/or large files). This enhancement includes three changes:

  • Remove the "Verify checksums generated on ingest" micro-service
  • Enhance the "Verify AIP" micro-service to bulk query the database for transfer-generated checksums and then verify that they match what is documented in the bag-generated manifest-<ALGORITHM>.txt.
  • Have "Verify AIP" create an AIP-level "fixity check" PREMIS:EVENT that it can pass to the Storage Service, which will document this verification in the pointer file.

This should not impact regular workflows, but it is worth noting that there is no AIP-level fixity check PREMIS event for uncompressed AIPs, which don't have pointer files. For uncompressed AIPs, there are still object-level fixity events in place. Note that there is an issue in the Archivematica Issues repository regarding this note - Problem: uncompressed AIPs need pointer files #32

This work was sponsored by Columbia University Library. Thank you!

File format identification updates

Archivematica 1.8 is now up to date with PRONOM v.94! For more information on new data added to PRONOM, check the PRONOM release notes.

This work was sponsored by the Denver Art Museum. Thank you!

Indexing can be enabled/disabled for Transfers and/or Archival Storage

Previously, the ElasticSearch index feature could be disabled globally as a scalability measure since indexing consumes a lot of resources. However, this also disabled Backlog and Appraisal features (which also uses indexing) and which some users still wanted to access. As of release 1.8, Archivematica can be deployed to run with indexing enabled just for Transfers (Backlog and Appraisal enabled), just for Archival Storage (Backlog and Appraisal disabled), for both indexes, or for none.

Configure email settings

This change improves the ways that the email client in Archivematica can be configured, including allowing an administrator to set the sender email address for emails sent by Archivematica (i.e. normalization reports, failure reports) to comply with local IT requirements.

This work was sponsored by Jisc. Thank you!

Download processing configuration and reset to default

Previous versions of Archivematica introduced the ability to add custom processing configurations, but users had to retrieve the custom configuration file via the command line to use it. There is now a download button on Administration > Processing configuration so that you can download the processing config from the user interface.

You can also reset a processing configuration to the installation pre-set by clicking on the new reset button on Administration > Processing configuration.

The documentation for using a custom processing configuration has also been updated.


MCP Batching for scalability & performance

This feature refactors how tasks are scheduled, executed & managed within Archivematica, by grouping tasks into batches. It introduces processing efficiencies that significantly decrease the processing power and time required to complete Transfer and Ingest. It includes new configuration options to further optimize processing efficiency for particular types of Transfers (e.g. few large files vs. many small files) and for different deployment patterns (e.g. installing components across multiple machines).

This feature does not impact the functionality or appearance of Archivematica.

This work was sponsored by Jisc. Thank you!

Fixed

Upgraded tools and dependencies

  • Fido has been upgraded to version 1.3.12
  • Siegfried has been upgraded to version 1.7.10
  • FITS has been upgraded to version 1.1.0

End of life dependencies

Archivists' Toolkit integration

Archivists' Toolkit has been deprecated since 2013. The Archivists' Toolkit DIP upload feature has not had active development or testing since then. There are no plans to start testing or to fix any problems with the feature. As a result, there is a proposal deprecate this feature in Archivematica 1.9. Community response is welcome via a comment on the issue in GitHub.

Retrieved from "https://wiki.archivematica.org/index.php?title=Archivematica_1.8_and_Storage_Service_0.13_release_notes&oldid=12725"