Sword API

From Archivematica
Revision as of 17:08, 29 November 2013 by Mcantelon (talk | contribs)
Jump to navigation Jump to search

Overview

One of the 1.1 release features is a sponsored project to integrate Archivematica with Islandora. This integration will be accomplished by creating a Sword 2.0 API for the Archivematica storage service. Islandora development work will add functionality to Islandora to use this API to deposit digital objects into Archviematica.

As described elsewhere on this wiki, "the primary function of Archivematica is to process digital transfers (accessioned digital objects), turn them into SIPs, apply format policies and create high-quality, repository-independent Archival Information Packages (AIP)".

The Archivematica Sword API allows 3rd party applications to automate the process of creating Transfers.

  1. Create Transfer - set the name and other metadata about a Transfer
  2. Populate Transfer - add/edit/update digital objects in a Transfer, and associated metadata
  3. Finalize Transfer - indicates that the Transfer is ready to start processing.

After content has been deposited, if users have access to the deposit directory they can also manually manipulate the deposit directory aside from any manipulation using the API.

Once a Transfer has been created, populated and finalized, Archivematica will begin processing that Transfer.

Endpoints

Service document

The SWORD service document provides information about the SWORD provider's capacities and lists SWORD collections. Archivematica currently includes one SWORD collection to which SWORD deposits can be made: transfers.

The SWORD service document endpoint is located at /api/v1/sword.

List existing transfers

  • HTTP Get to the Collection IRI (defined by default as: /api/v1/space/[space UUID]/sword/collection/)
  • Transfers are listed as an Atom feed
  • Each Atom feed entry contains an "atom:link" element whose href attribute contains the URL needed to get details about the transfer
  • optional filters via get parameters (not yet implemented)

Create a new transfer

  • used to start a new transfer in Archivematica
  • HTTP POST of an Atom Entry Document to the Collection IRI (defined by default as: /api/v1/space/[space UUID]/sword/collection/)

Possible HTTP Response Codes

  • HTTP 200 OK - transfer already exists (not yet implemented)
  • HTTP 201 Created - transfer has been accepted
  • HTTP 412 Precondition Failed - required metadata missing or invalid

Valid requests will receive a Sword Deposit Receipt in the body of the response.

Required HTTP Headers The HTTP POST must include certain specific http headers.

Required by http 1.1 specification:

  • Host: Must be set to archivematica host name.
  • Content-Length: Must be set to the length of the atom entry document.
  • Content-Type: Must be set to "application/atom+xml;type=entry".

Required by Archivematica.

  • Authorization: Must include the api key and username assigned by Archivematica.

Required by SWORD 2.0 protocol:

  • In-Progress: Must be set to "true"

Optional in SWORD 2.0 protocol:

  • On-Behalf-Of: not implemented by Archivematica
  • Slug: not implemented by Archivematica

Required in Body The Body of the request must be a valid METS Document. The required fields in an Atom Entry Document are:

  • LABEL: used as the transfer name
  • id: used as an accession id (not yet implemented)
  • author: used as the source of acquisition (not yet implemented)
  • summary: not implemented yet. Could be used in the future as a description for the Transfer
  • updated: should be set to the current timestamp.

Example Example HTTP POST request: 

POST /api/v1/space/96606387-cc70-4b09-b422-a7220606488d/sword/collection/ HTTP/1.1
Host: localhost
Authorization: Archivematica-API api_key="XXXXXXXXXXXXXXXXXXXX", username="timh"
Content-Length: 213
Content-Type: application/atom+xml;type=entry
In-Progress: true

<?xml version="1.0" encoding="UTF-8"?>
<METS:mets EXT_VERSION="1.1" OBJID="islandora:72" LABEL="Cinderella"
  xmlns:METS="http://www.loc.gov/METS/"
  xmlns:xlink="http://www.w3.org/1999/xlink"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.loc.gov/METS/ http://www.fedora.info/definitions/1/0/mets-fedora-ext1-1.xsd">
    <METS:fileSec>
        <METS:fileGrp ID="DATASTREAMS">
            <METS:fileGrp ID="OBJ">
                <METS:file ID="OBJ.0">
                    <METS:FLocat xlink:title="Cinderella_(1865).pdf" LOCTYPE="URL"
                      xlink:href="http://localhost:8080/fedora/get/islandora:72/OBJ/2013-08-12T14:46:43.454Z" />
                </METS:file>
            </METS:fileGrp>
        </METS:fileGrp>
    </METS:fileSec>
</METS:mets>

Example Response: 

201 Created
Location: http://localhost/api/v1/location/91d4f258-0050-4914-9d0c-6a74815358d2/sword/

<entry xmlns="http://www.w3.org/2005/Atom"
        xmlns:sword="http://purl.org/net/sword/"
        xmlns:dcterms="http://purl.org/dc/terms/">

    <title>Cinderella</title>
    <id>91d4f258-0050-4914-9d0c-6a74815358d2</id>
    <updated></updated>
    <generator uri="https://www.archivematica.org" version="1.0"/>

    <summary>A deposit was started.</summary>
    <sword:treatment>A deposit directory was created and, optionally, asynchronous download of digital objects.</sword:treatment>

    <link rel="alternate" href="http://www.swordserver.ac.uk/col1/mydeposit.html"/>
    <link rel="edit-media" href="http://localhost/api/v1/location/91d4f258-0050-4914-9d0c-6a74815358d2/sword/media/"/>
    <link rel="edit" href="http://localhost/api/v1/location/91d4f258-0050-4914-9d0c-6a74815358d2/sword/" />
    <link rel="http://purl.org/net/sword/terms/add" href="http://192.168.1.76:8000/api/v1/location/91d4f258-0050-4914-9d0c-6a74815358d2/sword/" />
    <link rel="http://purl.org/net/sword/terms/statement" type="application/atom+xml;type=feed"
      href="http://localhost/api/v1/location/91d4f258-0050-4914-9d0c-6a74815358d2/sword/state/" />
</entry>

Add Files to a Transfer

Posting file metadata (not yet implemented):

  • POST a METS file to the SE-IRI, listed below.
  • HTTP headers are required similar to Create Transfer call (above).
  • Body must include a valid Atom Entry document, that has METS embedded inside it.

Posts file data:

  • It is possible to also POST the actual file to a deposit location's EM-IRI (/api/v1/location/[location UUID]/sword/media/)
  • The file should be posted as the request body and the filename specified using the Content-Disposition header (as per RFC 6266): for example "Content-Disposition: Attachment; filename=dog.jpg"
  • An MD5 checksum for an uploaded file can be provided via the Content-MD5 header (as per RFC 1544): for example "Content-MD5: Q2hlY2sgSW50ZWdyaXR5IQ=="
  • If only an Atom Entry Document is POST'ed, Archivematica will look for uri's in the file StructMap section of the METS file, and attempt to GET each file listed, to include in the Transfer. (not yet implemented)

Finalize a Transfer

  • POST with no body to the SE-IRI
  • set In-Progress HTTP header to : false
  • This will tell Archivematica that no further content will be added or removed from the Transfer.
  • Archivematica will finish fetching any files that were added previously, and once they have all been downloaded, the Transfer will start through the Archivematica pipeline.

Example 

POST http://localhost/api/v1/location/55ce8053-113a-4954-8aa3-fc9771da0bda/sword/ HTTP/1.1
Host: localhost
Authorization: Archivematica-API api_key="XXXXXXXXXXXXXXXXXXXX", username="timh"
Content-Length: 0
Content-Type: application/atom+xml;type=entry
In-Progress: false

The response will be HTTP 200/OK or 400/Error (400 if the Transfer was already finalized). If the transfer doesn't exist the response will be HTTP 404.

Get Status of Transfer

To check Status:

GET the State-IRI in this example:

GET http://localhost/api/v1/location/09224ddb-4c7d-404c-9218-2f2e1b5a599e/sword/state/ HTTP/1.1

response will include: 

<atom:feed xmlns:sword="http://purl.org/net/sword/terms/" 
            xmlns:atom="http://www.w3.org/2005/Atom">

    <atom:category scheme="http://purl.org/net/sword/terms/state"
        term="failed"
        label="State">
            Deposit initiation: Failed
    </atom:category>

</atom:feed>

the list of possible descriptions is not finalized.

The state term value will either be "processing" (asynchronous deposit still working), "complete" (ready to be finalized), or "failed" (asynchronous deposit encountered an error).

Additional Details

Transfer Details

EM-IRI

  • Edit-Media IRI
  • used to add new objects to an existing transfer
  • follows the form: [archivematica hostname]/api/v2/transfer/[uuid of transfer]/media
  • example: /api/v1/location/[deposit location UUID]/sword/media/

An HTTP POST to the em-iri for a transfer should include a file as the body

An HTTP GET should return a list of files in the transfer

An HTTP DELETE to the em-iri will remove all digital objects from the Transfer. This is a valid operation only while the Transfer is being assembled. Once the Transfer has been finalized, attempting to DELETE will return an error.

Edit-IRI

The client can replace the metadata of a resource by performing an HTTP PUT of a new Atom Entry on the Edit-IRI. (not yet implemented)

This would be used to update metadata about a transfer, such as the transfer name.

SE-IRI

State-IRI

  • used to retrieve status of transfer
  • implemented as Atom document
  • example: /api/v1/location/[deposit location UUID]/sword/state/
  • should be able to subscribe like RSS feed

Dashboard configuration

When a transfer deposit is finalized, an Archivematica instance can be notified in the background, via a REST API, that the transfer should be approved for processing. In this way the SWORD API can be used to start transfers.

Each SWORD space in a storage service is associated with an Archivematica instance. Both the Archivematica instance and the storage service, however, require configuration.

On the Archivematica dashboard side, the IP or hostname of the storage service must be added to the REST API whitelist (click "Administration" in the Archivematica navigation bar then "REST API" in the administration sidebar to get to the whitelist page).

Once you're whitelisted the storage service in the Archivematica dashboard, click "General" in the administration sidebar then note the dashboard UUID. In the storage service administration interface, click "Pipelines" in the navigation bar. Find the UUID matching the dashboard then click "Edit". In the "Remote name" field enter the IP of the Archivematica dashboard. In the "Api username" field enter an Archivematica dashboard username (click "Administration" in the Archivematica navigation bar then "Users" in the administration sidebar to access the user list). In the "Api key" field enter a valid API key (a user's API key can be found by clicking the "Edit" button corresponding to a specific user on the user list page).

Service document

Example session

Below is an example session using curl to manipulate the API. In the example a deposit is created, a file is added to it, and it is finalized.

NOTE: The SWORD API is in the process of being moved to the storage service. These URLs may not work. 

# create new deposit
curl -v -H "In-Progress: true" --data-binary @mets.xml --request POST http://localhost:8000/api/v1/location/52bbe770-3f96-4493-8d93-ded54c958c51/sword/collection/
# response XML includes endpoints for adding addition files, etc.
  
# add another file to the transfer
curl -v -H "Content-Disposition: attachment; filename=joke.jpg" --request POST \
  --data-binary "@joke.jpg" \
  http://localhost:8000/api/v1/deposit/8e1cba3c-f133-48b2-afef-e42505a7bdc8/sword/media/
  
# finalize transfer and approve processing
curl -v -H "In-Progress: false" --request POST http://localhost:8000/api/v1/deposit/8e1cba3c-f133-48b2-afef-e42505a7bdc8/sword/

You might, at some point, want to list the transfers that have been started. The following curl command will do so. 

curl -v http://localhost:8000/api/v1/location/52bbe770-3f96-4493-8d93-ded54c958c51/sword/collection/

If you're working on a transfer and want to list what files are included in it, the following curl command will list them. 

# list files in transfer
curl -v http://192.168.1.231:8000/api/v1/deposit/8e1cba3c-f133-48b2-bdc8/sword/media/

If you've started a deposit, but want to discard it, you can delete the deposit. The following curl command shows an example. 

curl -v -XDELETE http://localhost:8000/api/v1/deposit/8e1cba3c-f133-48b2-afef-e42505a7bdc8/sword/

Synchronous Deposit Examples

If you wanted to deposit a transfer then immediately finalize it you could use something like the following curl command. 

curl -v -H "In-Progress: false" --data-binary @mets.xml --request POST http://localhost:8000/api/v1/location/c0bee7c8-3e9b-41e3-8600-ee9b2c475da2/sword/collection/
Retrieved from "https://wiki.archivematica.org/index.php?title=Sword_API&oldid=9169"