Asynchronous Processing in Storage Service
Synopsis
The Storage Service is implemented as a web application, written using the Django Framework. It exposes a REST API, which is consumed by other components in Archivematica (the dashboard, the automation tools) and can be used by third party applications as well. All of the AIP endpoints are currently synchronous - the http request made by a client blocks until the work is completed. Many of the tasks performed by the storage service involve significant disk i/o and can take a long time (minutes/hours) to complete.
Adding a way to perform work asynchronously is an important feature, probably a requirement before a 1.0.0 version of the Storage Service can be released.
A couple of experiments have been tried already:
API backwards compatibility
Possible approaches:
- In
/api/v2, add new endpoints that can do the job asynchronously. Maintain the existing endpoints. - In
/api/v2, update existing parameters to accept a new query parameter?async=true.- It may not be a good idea when the properties of the response are going to be significantly different, e.g. a deferred call may expect a document with a single UUID that can be used later to query the status of the call in the future, instead of the document originally requested. This needs to be analyzed individually for each of the endpoints affected.
- New
/api/v3, similar API but everything is async.
List of endpoints affected
TODO.