Difference between revisions of "Archivematica 1.10 and Storage Service 0.15 release notes"

From Archivematica
Jump to navigation Jump to search
 
(56 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
[[Main_Page|Home]] > [[Release_Notes|Release Notes]] > Archivematica 1.10 and Storage Service 0.15
 
[[Main_Page|Home]] > [[Release_Notes|Release Notes]] > Archivematica 1.10 and Storage Service 0.15
  
−
'''Release date'''
+
'''Release date''' September 5, 2019
  
  
 
==Supported environments==
 
==Supported environments==
  
−
Please see the [https://www.archivematica.org/en/docs/archivematica-1.9/admin-manual/installation-setup/installation/installation/#installation installation instructions].
+
Please see the [https://www.archivematica.org/en/docs/archivematica-1.10/admin-manual/installation-setup/installation/installation/#installation installation instructions].
  
−
Archivematica 1.9 and Storage Service 0.14 are supported for production use in the following environments:
+
Archivematica 1.10 and Storage Service 0.15 are supported for production use in the following environments:
  
 
* Ubuntu 16.04 64-bit Server Edition
 
* Ubuntu 16.04 64-bit Server Edition
Line 27: Line 27:
  
  
−
* Documentation: link
+
* Documentation: https://www.archivematica.org/en/docs/archivematica-1.10/user-manual/appraisal/appraisal/#appraisal
 
* Issues (linked from epic): https://github.com/archivematica/Issues/issues/391
 
* Issues (linked from epic): https://github.com/archivematica/Issues/issues/391
  
−
===New feature 2===
+
==="Vintage" AIP handling===
  
−
Here is a description of this amazing feature! Here's why it's a net benefit to the project and the community, and here is how it will impact your workflow. Also included are any special notes, like if it's a beta feature.
+
AIPS created in Archivematica prior to version 1.0 can’t currently be re-ingested in more recent versions (due to namespace collisions with the use of PREMIS 2.1).
  
−
This work was sponsored by some amazing institution. Thank you!
+
This change will make it possible to reingest these older AIPS into Archivematica. This work is sponsored by the City of Vancouver Archives.
  
−
* Documentation: link
+
* Issue: https://github.com/archivematica/Issues/issues/24
−
* Pull requests: link
+
 
−
* Feature files: link
+
===LDAP support for Storage Service===
 +
 
 +
The Archivematica Storage Service now supports LDAP integration. This was sponsored by Piql for the Norwegian Health Archives.
 +
 
 +
* Issue: https://github.com/archivematica/Issues/issues/680
 +
 
 +
===External PID declaration===
 +
 
 +
This feature, sponsored by the International Institute of Social History, allows users to include identifiers minted outside of Archivematica in their transfer and written to the premis:object metadata in the AIP METS file. This is done via an identifiers.json file included in the transfer.
 +
 
 +
* Issue: https://github.com/archivematica/Issues/issues/133
 +
* Documentation: https://www.archivematica.org/en/docs/archivematica-1.10/user-manual/transfer/transfer/#transfer-pids
 +
 
 +
===Avalon Integration===
 +
 
 +
Archivematica 1.10 has added the ability to integrate with [https://www.avalonmediasystem.org/ Avalon Media System] through the automation tools. This was sponsored by Northwestern University and Indiana University through IMLS funding. The feature allows users to prepare and store a DIP appropriate for upload the Avalon, and then have the automation tools send it to Avalon for ingest into that system.
 +
 
 +
* Issue: https://github.com/archivematica/Issues/issues/643
 +
* Documentation: https://www.archivematica.org/en/docs/archivematica-1.10/admin-manual/installation-setup/integrations/integrations/#avalon-media-system-integration
 +
 
 +
===Include AIP UUID in bag-info.txt===
 +
 
 +
This change adds the UUID of the AIP to the External-Identifier field of the bag-info.txt file. This change is a '''community contribution''' by Helen Sherwood-Taylor (Wellcome Collection)- thank you!
 +
 
 +
* Issue: https://github.com/archivematica/Issues/issues/492
 +
* Documentation: https://www.archivematica.org/en/docs/archivematica-1.10/user-manual/archival-storage/aip-structure/#bagit-doc
 +
 
 +
===Allow designation of ArchivesSpace repository using DSpace REST location===
 +
 
 +
This feature allows designation of more than one ArchivesSpace repository by enabling it via the DSpace REST location rather than through the configuration in the Storage Service. This was a '''community contribution''' by Hrafn Malmquist (University of Edinburgh)- thank you!
 +
 
 +
* Issue: https://github.com/archivematica/Issues/issues/435
 +
* Documentation: https://www.archivematica.org/en/docs/storage-service-0.15/administrators/#dspace-via-rest-api
  
 
==Changed==
 
==Changed==
  
−
Describe enhancements or major fixes.
+
===AIP METS in PREMIS 3===
 +
 
 +
As a result of the vintage AIP work described above, we have upgraded the AIP METS file to use PREMIS 3 exclusively. In previous versions it had a mix of versions 2 and 3.
 +
 
 +
* Issue: https://github.com/archivematica/Issues/issues/370
 +
 
 +
===API endpoint for manifest validation (beta)===
 +
 
 +
As part of the Avalon integration described above, we added an endpoint that can be called to validate a manifest. While this is currently only implemented for the Avalon manifest, it could for example be extended in the future for metadata.csv validation and similar.
 +
 
 +
* Issue: https://github.com/archivematica/Issues/issues/618
 +
* Documentation: https://wiki.archivematica.org/Archivematica_API#Validate
 +
 
 +
 
 +
===Extended service callbacks===
 +
 
 +
This change allows the creation of callbacks for AIP, AIC, and DIP storage events. This was done in service to integration with SCOPE, a DIP access platform in use by the Canadian Centre for Architecture, but is widely applicable to other use cases.
 +
 
 +
* Issue: https://github.com/archivematica/Issues/issues/147
 +
 
 +
===Storage Service packages tab tidy-up===
 +
 
 +
This change was the result of a reported bug for the Storage Service packages tab timing out- we addressed that issue but also tidied up the tab to make it more generally usable. See the issue for discussion of changes.
 +
 
 +
* Issue: https://github.com/archivematica/Issues/issues/676
 +
* Documentation: https://www.archivematica.org/en/docs/storage-service-0.15/administrators/#packages-tab
 +
 
 +
===Code formatting to be handled by black===
  
−
===Enhancement 1===
+
The developer team has decided to use a code formatting tool (black) to make formatting more consistent. Pull requests to Archivematica are now checked with a linter to ensure consistency with black.
  
−
We fixed this issue. Here's why it's a net benefit to the project and the community, and here is how it will impact your workflow. Also included are any special notes, like if it's a beta feature.
+
* Issue: https://github.com/archivematica/Issues/issues/393
  
−
This work was sponsored by some amazing institution. Thank you!
+
===Additions and improvements to the AMAUATs===
  
−
* Documentation: link
+
The AMAUATs are the Archivematica Automated User Acceptance Tests. Thanks to '''sponsorship by Wellcome Collection''' we have greatly expanded the number of automated tests that run and made most of them "black box" tests that run via the API- this means they are more robust than tests that rely on the user interface, which can break anytime the user interface changes in some way. In addition to these improvements, to aid in the release process we have made the AMAUATs have versions that go along with the Archivematica version being released. Please see the AMAUAT repo for more information: https://github.com/artefactual-labs/archivematica-acceptance-tests
−
* Pull requests: link
 
−
* Feature files: link
 
  
 
==Fixed==
 
==Fixed==
  
−
List bugfixes with a link to the Github issue.
 
  
−
* Bugfix 1: link
+
* A number of fixes related to custom structMap import have been '''sponsored''' in this release by International Institute of Social History. Thank you!
−
* Bugfix 2: link. Sponsored by someone!
+
** Custom structMaps do not support nested directories: https://github.com/archivematica/Issues/issues/283
−
* Bugfix 3: link
+
** Purpose of Verify structMap in ingest is unclear: https://github.com/archivematica/Issues/issues/286
 +
** Custom structMaps labelled as structMap_2: https://github.com/archivematica/Issues/issues/633
 +
** We also added documentation for custom structMap import: https://www.archivematica.org/en/docs/archivematica-1.10/user-manual/transfer/import-metadata/#import-metadata
 +
* Verify transfer checksums succeeding without verifying the checksums: https://github.com/artefactual/archivematica/issues/1061
 +
* Parallel bzip2 algorithm isn't written to the pointer file when used: https://github.com/archivematica/Issues/issues/714
 +
* metsrw cannot process unicode characters: https://github.com/archivematica/Issues/issues/295
 +
* Deleted AIPs are not removed from S3: https://github.com/archivematica/Issues/issues/696
 +
* Cannot store AIP in S3 us-west-2-region: https://github.com/archivematica/Issues/issues/639
 +
* S3 bucket does not use the Region set in S3 space: https://github.com/archivematica/Issues/issues/638
 +
* File list pane always removes last tag of a file ('''Sponsored''' by Simon Fraser University- thank you!): https://github.com/archivematica/Issues/issues/472
 +
* Storage Service import_aip leaves uncompressed AIP data in /tmp: https://github.com/archivematica/Issues/issues/706
 +
* AgentArchives does not log out of ArchivesSpace ('''Community contribution''' by Hrafn Malmquist (University of Edinburgh)- thank you!): https://github.com/artefactual-labs/agentarchives/issues/47
 +
* Storage Service sort by size does not increment correctly: https://github.com/archivematica/Issues/issues/678
 +
* verify_checksum PREMIS events have no agent information: https://github.com/archivematica/Issues/issues/774
 +
* Versions shown for FPR tools are outdated: https://github.com/archivematica/Issues/issues/191
 +
* Jhove failure event has result "well-formed and valid": https://github.com/archivematica/Issues/issues/164
 +
* Create METS script loops after exception ('''Community contribution''' by Helen Sherwood-Taylor (Wellcome Collection)- thank you!): https://github.com/archivematica/Issues/issues/620
 +
* Archivematica should pin exact package versions in requirements.txt for predictable deployments ('''Community contribution''' by Helen Sherwood-Taylor (Wellcome Collection)- thank you!): https://github.com/archivematica/Issues/issues/634
 +
* Delete links are not consistent in the Packages tab of the Storage Service: https://github.com/archivematica/Issues/issues/711
 +
* MCP server doesn't report when the Gearman server is unavailable ('''Community contribution''' by Alex Chan (Wellcome Collection)- thank you!): https://github.com/archivematica/Issues/issues/553
 +
* Archivematica uses both bagit-python and bagit-java: https://github.com/archivematica/Issues/issues/246
 +
* DSpace REST location does not fall back to default values ('''Community contribution''' by Hrafn Malmquist (University of Edinburgh)- thank you!): https://github.com/archivematica/Issues/issues/458
 +
* Validate function of bind_pids occurs too early in the microservice script: https://github.com/archivematica/Issues/issues/776
 +
* Storage Service object-counting disabling option fails: https://github.com/archivematica/Issues/issues/657
 +
* chunkids in manifest file do not match id in DuraCloud: https://github.com/archivematica/Issues/issues/574
 +
* Archivematica attempts to delete files from transfer source: https://github.com/archivematica/Issues/issues/646
 +
* Double replications being creation from first replication, not original ('''Sponsored''' by Piql for the Norwegian Health Archive- thank you!): https://github.com/artefactual/archivematica-storage-service/issues/270
 +
* AtoM DIP upload link in Access tab is wrong: https://github.com/archivematica/Issues/issues/411
 +
* Metadata-only DIP fails without format version: https://github.com/archivematica/Issues/issues/857
 +
 
 +
And more! For a complete list of fixes and changes please see: https://github.com/archivematica/Issues/milestone/6
  
 
==Upgraded tools and dependencies==
 
==Upgraded tools and dependencies==
  
−
List any tools and dependencies that have been upgraded.
 
  
−
* Tool has been updated to version X.
+
* Ghostscript upgraded to 9.2.x across all platforms: https://github.com/archivematica/Issues/issues/714
 +
* JHOVE upgraded to 1.20: https://github.com/archivematica/Issues/issues/521
  
 
==Deprecated==
 
==Deprecated==
  
 
[https://github.com/archivematica/Issues/issues/174 Issue 174] As of Archivematica 1.10, Archivist's Toolkit integration has been removed. Please see [https://groups.google.com/d/msg/archivematica/-LSHh3jGiQk/FlXJ-xnHCQAJ this announcement] for more details.
 
[https://github.com/archivematica/Issues/issues/174 Issue 174] As of Archivematica 1.10, Archivist's Toolkit integration has been removed. Please see [https://groups.google.com/d/msg/archivematica/-LSHh3jGiQk/FlXJ-xnHCQAJ this announcement] for more details.
−
 
−
==End of life dependencies==
 
−
 
−
List any dependencies that have reached end of life since the last release, as well as a note on the plan going forward.
 
−
 
−
Make note of any risks, and how users can mitigate them.
 

Latest revision as of 11:13, 11 November 2019

Home > Release Notes > Archivematica 1.10 and Storage Service 0.15

Release date September 5, 2019


Supported environments[edit]

Please see the installation instructions.

Archivematica 1.10 and Storage Service 0.15 are supported for production use in the following environments:

  • Ubuntu 16.04 64-bit Server Edition
  • Ubuntu 18.04 64-bit Server Edition
  • CentOS 7 64-bit

For development purposes, most of our developers prefer to use Docker containers. These and all above supported environments are linked from the installation instructions above.

Added[edit]

Information Packaging (Backlog) Workflow Enhancements[edit]

Simon Fraser University has sponsored a development project to improve the ability to store Transfers for a long period of time. The goal is to improve the metadata quality of Transfers to reduce the risk of storing them for a long period of time in a backlog. As a result of this project:

  • Transfers placed in backlog are now packaged as bags
  • The Transfer METS file now records all preservation actions that took place during Transfer.
  • The Appraisal Tab now handles packages with more than 1,000 files.
  • A backlog can be populated from a new pipeline from transfers created by another pipeline.


"Vintage" AIP handling[edit]

AIPS created in Archivematica prior to version 1.0 can’t currently be re-ingested in more recent versions (due to namespace collisions with the use of PREMIS 2.1).

This change will make it possible to reingest these older AIPS into Archivematica. This work is sponsored by the City of Vancouver Archives.

LDAP support for Storage Service[edit]

The Archivematica Storage Service now supports LDAP integration. This was sponsored by Piql for the Norwegian Health Archives.

External PID declaration[edit]

This feature, sponsored by the International Institute of Social History, allows users to include identifiers minted outside of Archivematica in their transfer and written to the premis:object metadata in the AIP METS file. This is done via an identifiers.json file included in the transfer.

Avalon Integration[edit]

Archivematica 1.10 has added the ability to integrate with Avalon Media System through the automation tools. This was sponsored by Northwestern University and Indiana University through IMLS funding. The feature allows users to prepare and store a DIP appropriate for upload the Avalon, and then have the automation tools send it to Avalon for ingest into that system.

Include AIP UUID in bag-info.txt[edit]

This change adds the UUID of the AIP to the External-Identifier field of the bag-info.txt file. This change is a community contribution by Helen Sherwood-Taylor (Wellcome Collection)- thank you!

Allow designation of ArchivesSpace repository using DSpace REST location[edit]

This feature allows designation of more than one ArchivesSpace repository by enabling it via the DSpace REST location rather than through the configuration in the Storage Service. This was a community contribution by Hrafn Malmquist (University of Edinburgh)- thank you!

Changed[edit]

AIP METS in PREMIS 3[edit]

As a result of the vintage AIP work described above, we have upgraded the AIP METS file to use PREMIS 3 exclusively. In previous versions it had a mix of versions 2 and 3.

API endpoint for manifest validation (beta)[edit]

As part of the Avalon integration described above, we added an endpoint that can be called to validate a manifest. While this is currently only implemented for the Avalon manifest, it could for example be extended in the future for metadata.csv validation and similar.


Extended service callbacks[edit]

This change allows the creation of callbacks for AIP, AIC, and DIP storage events. This was done in service to integration with SCOPE, a DIP access platform in use by the Canadian Centre for Architecture, but is widely applicable to other use cases.

Storage Service packages tab tidy-up[edit]

This change was the result of a reported bug for the Storage Service packages tab timing out- we addressed that issue but also tidied up the tab to make it more generally usable. See the issue for discussion of changes.

Code formatting to be handled by black[edit]

The developer team has decided to use a code formatting tool (black) to make formatting more consistent. Pull requests to Archivematica are now checked with a linter to ensure consistency with black.

Additions and improvements to the AMAUATs[edit]

The AMAUATs are the Archivematica Automated User Acceptance Tests. Thanks to sponsorship by Wellcome Collection we have greatly expanded the number of automated tests that run and made most of them "black box" tests that run via the API- this means they are more robust than tests that rely on the user interface, which can break anytime the user interface changes in some way. In addition to these improvements, to aid in the release process we have made the AMAUATs have versions that go along with the Archivematica version being released. Please see the AMAUAT repo for more information: https://github.com/artefactual-labs/archivematica-acceptance-tests

Fixed[edit]

And more! For a complete list of fixes and changes please see: https://github.com/archivematica/Issues/milestone/6

Upgraded tools and dependencies[edit]

Deprecated[edit]

Issue 174 As of Archivematica 1.10, Archivist's Toolkit integration has been removed. Please see this announcement for more details.

Retrieved from "https://wiki.archivematica.org/index.php?title=Archivematica_1.10_and_Storage_Service_0.15_release_notes&oldid=13261"