Asynchronous Processing in Storage Service

From Archivematica
Revision as of 18:17, 15 August 2017 by Sevein (talk | contribs)
Jump to navigation Jump to search


Synopsis

The Storage Service is implemented as a web application, written using the Django Framework. It exposes a REST API, which is consumed by other components in Archivematica (the dashboard, the automation tools) and can be used by third party applications as well. All of the AIP endpoints are currently synchronous - the http request made by a client blocks until the work is completed. Many of the tasks performed by the storage service involve significant disk i/o and can take a long time (minutes/hours) to complete.

Adding a way to perform work asynchronously is an important feature, probably a requirement before a 1.0.0 version of the Storage Service can be released.

A couple of experiments have been tried already:


API backwards compatibility

Possible approaches:

  • In /api/v2, add new endpoints that can do the job asynchronously. Maintain the existing endpoints.
  • In /api/v2, update existing parameters to accept a new query parameter ?async=true.
  • New /api/v3, similar API but everything is async.

List of endpoints affected

TODO.

Retrieved from "https://wiki.archivematica.org/index.php?title=Asynchronous_Processing_in_Storage_Service&oldid=12043"