File Browser Requirements

Main Page > Development > Development documentation > File Browser Requirements

This page describes requirements for moving all Archivematica interface to the web dashboard, including file browser functionality for transfer and SIP configuration, normalization output review, and AIP and DIP review prior to upload to storage and access. (mockups forthcoming)

TRANSFER

• Login Archivist - archivist's name will be attached to all processing actions. (PREMIS agent)

• Create structured directory (create three directories: logs, metadata, objects)

-OR-

• Restructure transfer for processing (put all contents into objects directory and create an empty log and metadata directory) (This should be automated)

• Run checksum on transfer and place checksum in folder called checksum.md5 in the metadata folder (checkbox) This is in case the transfer hasn't arrived with a checksum already. (This was discussed - currently, the checksum assigment and checksum check should occur upon beginning transfer, prior to backup. Process should be automated.) In fact, all transfers will go through all microservices for transfers before they are "backlogged"

• Assign transfer type (drop-down, user configurable: Generic, DSpace export, digitization output (Issue 713), VanDocs export, BagIt package (Issue 593))

Transfer backup requirements: See Transfer_Backup_Requirements

Create SIP

• Below the above contents, on the same page, would be the current 0.8 dashboard for Ingest, including popouts for tasks, etc, and decision points.
• original order and arrangement (issue 964) captured as logical <structMap> in SIP METS file.

File Viewer

• This will allow the user to see individual documents in the transfer to get a better idea of their contents and technical metadata before assigning them to SIPs.

Examine Contents

• This will allow the user to examine contents of a SIP for keywords, use visualization tools, and identify restricted records (by keyword or by type, cc number, SIN number, etc), tag records or groups of records and apply some basic metadata that will be carried over to the description

ADMINISTRATION TAB

• Assign transfer backup location (what is it backing up? at which stage?) This could be a staging area, in case many transfers will become one SIP or more time is needed to begin processing ADMIN

More to come on this...

• full-text indexing (Tika/Lucene/ElasticSearch) on recognizable text files as well as data visualization (ElasticSearch/Protovis) to assist with:

       identifying keywords for security classification and appraisal
       identify duplicates and close matches (using hex values?)
       high-level sorting and grouping of batches of objects into SIPs (incl. tracking difference between 'original order' and new physical/logical arrangement)
   

• Archivematica compliant SIP creation, including:

       log of original directory structure, diff to new structure (using METS <structMap> to represent both)
       use METS <structMap> (or physical directory arrangement of SIP?) to represent archival arrangement for rebuild in access system (see Issue 380)

• Change permissions of directory contents recursively (or should this be integrated in every step to avoid problems?)

indexing

• keyword & pattern matching for privacy/security sensitive information (e.g. social insurance numbers, credit card numbers, 'private', 'confidential')
• list of PDFs that have not been OCR'ed
• list of password protected / encrypted files