Manual normalization requirements and testing procedures

Main Page > Development > Development documentation > Manual normalization requirements and testing procedures

This page is no longer being maintained and may contain inaccurate information. Please see the Archivematica documentation for up-to-date information.

The following procedures describe functionality available in the 0.10-beta release of Archivematica.

Procedures for manual normalization

There are two manual normalization workflow options. First, the user can do some normalization before bringing any objects into Archivematica. Second, the user can process objects through to Ingest and then choose to do manual normalization on some of the objects. Below are the workflow steps to test each of these options.

These requirements and workflows were developed as a result of the following related issues: #4656, #4511. #4510

Option 1: Manual normalization prior to transfer into Archivematica

The manual normalization feature allows the user to prepare transfers that already contain manually normalized files. The user adds a /manualNormalization folder and its subdirectories to the transfer prior to processing. Archivematica recognizes that this folder already exists and will not add one during processing. This workflow requires that there must be a one-to-one relationship between the transfer and the SIP.

To test

1. Clone the newest sampledata to your local dev environment

You should find the sample to test in the following location: /archivematica-sampledata-2013-xx-xx-xx-xx/TestTransfers/manNormAtBeginning

Should you wish to create your own tests, you must create transfers with a structure that mimics the one in this sample. That is:

/topDirectory

/manualNormalization

/access

/example.bmp

/preservation

/example.prk

/file.tif

/file2.doc

/file3.docx

Where

• the /topDirectory is named whatever you want (you can still change this when you upload the transfer to the dashboard)
• /manualNormalization contains two directories, /access and /preservation which include in them the files you've already normalized for access and/or preservation prior to transfer
• other objects in the /topDirectory are to be treated as regular transfers

2. Upload the transfer to the dashboard from your source directories in the Transfer Tab.

For instructions detailing how to set up source directories, see the Administrator's manual here: Administrator_manual_0.9#Transfer_source_directories

3. Continue processing as you would a regular SIP. For instructions, see the online User manual here: User_manual_0.9 through to the Ingest tab.

4. In the Ingest tab, at normalization step, choose whether to normalize for access, preservation, or both. If you choose to review normalization results prior to accepting, you will find your /manualNormalization folder and its contents in the objects directory of your transfer, with it's /access and/or /preservation folders and their content intact. If you're happy with the results here, approve normalization.

Continue processing according to instructions in the User manual. Stop when you reach the Store AIP micro-service.

5. At Store AIP, you have the option to review your AIP. If you download the AIP from the review tab by simply clicking on the file with the 7z extension, you can download, unzip and view your AIP in your file browser. Your AIP will consist of a top level folder with the name you assigned plus a UUID. In that folder, you'll find a /data folder which contains your objects for preservation, logs, thumbnails and the METS xml. That data folder will contain your originals and normalized files, and the METS will include PREMIS events with "manual normalization" noted in the eventDetail of the appropriate normalization events. A discussion of the overall AIP structure can be found here: AIP_structure (Note that this was the 0.9-beta description and will change a bit for the next release: see related task at #4597)

Additionally, you might note that the manualNormalization folder and subfolders used during the manual normalization workflow have been deleted once normalization was approved rather than being packaged into the AIP.

6. If you are satisfied with everything here, continue to store your AIP and/or upload your DIP as per the User manual instructions.

• This process describes the basic test; however, we recommend putting the system through it's paces. Test with/without extra files in the transfer outside of the manualNormalization folder; test with only an access folder or preservation folder in the /manualNormalization folder; get creative!

Option 2: Manual normalization during Ingest

The manual normalization feature allows users to choose to do manual normalization at the normalization micro-service in the Ingest tab of the dashboard. The user does not have to manually normalize all of the objects for preservation and/or access; any files not manually normalized will be normalized according the regular Archivematica workflow.

To test

1. Start with any regular transfer in the Transfer tab of the dashboard. Follow the instructions in the user manual until you get to the normalization micro-service in the Ingest tab.

2. Select "Normalize manually" from the dropdown menu.

3. In your file browser, open /archivematica/src/MCPServer/sharedDirectoryStructure/watchedDirectories/approveNormalization/preservation/yourTransfer123abc/objects/

4. Choose which files you'd like to normalize for preservation and/or access

5. Do your normalization. Or, for testing purposees, you can simply copy and change the extension(s) of some of the objects in the transfer.

Once you've done all the normalization for preservation and access, place your preservation copies in the /manualNormalization/preservation folder and your access copies in the /manualNormalization/access folder. Note that any objects you haven't normalized manually will not be normalized. However, if you do not include anything in the /access folder, no DIP will be generated.

6. Return to the dashboard and approve normalization.

7. Continue processing according to instructions in the User manual. Stop when you reach the Store AIP micro-service.

8. At the Store AIP micro-service, you have the option to review your AIP. If you download the AIP from the review tab by simply clicking on the file with the 7z extension, you can download, unzip and view your AIP in your file browser. Your AIP will consist of a top level folder with the name you assigned plus a UUID. In that folder, you'll find a /data folder which contains your objects for preservation, logs, thumbnails and the METS xml. That data folder will contain your originals and normalized files, and the METS will include PREMIS events with "manual normalization" noted in the eventDetail of the appropriate normalization events. A discussion of the overall AIP structure can be found here: https://www.archivematica.org/wiki/AIP_structure (Note that this was the 0.9-beta description and will change a bit for the next release: see related task at #4597)

Additionally, you might note that the manualNormalization folder and subfolders used during the manual normalization workflow have been deleted once normalization was approved rather than being packaged into the AIP.

If you are satisfied with everything here, continue to store your AIP and/or upload your DIP as per the User manual instructions.

• This process describes the basic test; however, we recommend putting the system through it's paces. Test with/without access files in the manualNormalization folder; test with no objects outside of the /manualNormalization folder; get creative!