Difference between revisions of "Release Process"

From Archivematica
Jump to navigation Jump to search
(Create page)
 
(Add PRONOM instructions)
Line 1: Line 1:
 
This is an outline/checklist of the process to create Archivematica & Storage service releases.
 
This is an outline/checklist of the process to create Archivematica & Storage service releases.
  
 +
== Overview ==
 +
 +
# Merge new features
 +
# Test new features
 +
# [[#Update PRONOM]]
 +
# Write documentation
 +
# Create packages
 +
# Tag release
 +
# Announce release
 +
 +
 +
== Update PRONOM ==
 +
 +
PRONOM needs to be updated in our file identification tools, FIDO & Siegfried, as well as in the FPR.
 +
 +
=== Update FIDO ===
 +
 +
The FPR update currently use FIDO as a source for new PRONOM, since it is formatted nicer than PRONOM offers, so we depend on FIDO having updated their PRONOM.  If that has not happened, we can generate a new formats-v##.xml by updating signatures manually. Artefactual can also update PRONOM and submit a PR to FIDO.
 +
 +
# Checkout fido from https://github.com/openpreserve/fido
 +
# Update signatures
 +
#* Run <code>python -m fido.update_signatures</code> from the fido repository root
 +
# Add:
 +
#* New signature file <code>fido/conf/DROID_SignatureFile-v##.xml</code>
 +
#* New formats file <code>fido/conf/formats-v##.xml</code>
 +
#* New PRONOM zip file <code>fido/conf/pronom-xml-v##.zip</code>
 +
#* Updated <code>fido/conf/versions.xml</code>
 +
# Remove:
 +
#* Old signature file
 +
#* Old formats file
 +
#* Old PRONOM zip file
 +
# Update <code>fido/fido.py</code> to point to the new format_files
 +
# Create pull request.  This may trigger a release from FIDO
 +
 +
=== Package FIDO ===
 +
 +
Add packaging instructions here...
 +
 +
=== Package Siegfried ===
 +
 +
Add packaging instructions here...
 +
 +
=== Update FPR ===
 +
 +
This process is currently more convoluted than it needs to be, and should be simplified.
 +
 +
This depends on FIDO having updated their PRONOM files. See [[#Update FIDO]]
 +
 +
# Install the latest stable Archivematica release
 +
# Generate a JSON with the current version of the FPR (for use later)
 +
#* <code>./src/dashboard/src/manage.py dumpdata fpr > fpr-current.json</code>
 +
# Install the latest development version of Archivematica
 +
# Generate a blank data migration in the FPR-admin module
 +
#* E.g. <code>./src/dashboard/src/manage.py makemigrations --empty --name pronom_89 fpr</code>
 +
# Generate the FPR migration body.  This also updates the local database's FPR with the new PRONOM IDs
 +
#* E.g. <code>am import-pronom-ids fido/fido/conf/formats-v89.xml --output-filename archivematica/pronom89.txt</code>
 +
# Copy the output into the blank migration, add RunPython operation
 +
#* <code>migrations.RunPython(data_migration),</code>
 +
# Due to a [https://projects.artefactual.com/issues/10466#change-46673 bug], fix the imports
 +
#* Remove the <code>apps.get_model</code> lines
 +
#* Import the models directly with <code>from fpr.models import Format, FormatVersion, IDRule</code>
 +
# Commit
 +
# Deploy on testing pipeline
 +
# (Analysts) Update the new entries
 +
#* Move new formats to a more appropriate category
 +
#* Create rules & commands
 +
#* Test with data for new formats
 +
# Generate a JSON with the updated version of the FPR on the testing pipeline
 +
#* <code>./src/dashboard/src/manage.py dumpdata fpr > fpr-updated.json</code>
 +
# Get the updates as JSON
 +
#* E.g. <code>am get-fpr-changes fpr-current.json fpr-updated.json src/dashboard/src/fpr/migrations/pronom_89.json</code>
 +
# Update the migration to load the JSON updates
 +
#* Replace the contents of the data migration function with
 +
#** <code>fixture_file = os.path.join(os.path.dirname(__file__), 'pronom_89.json')</code>
 +
#** <code>call_command('loaddata', fixture_file, app_label='fpr')</code>
 +
#* Remove the direct imports from the bug
 +
#* Improvement Note: Because this is using loaddata, this will have problems if the FPR models are changed. A possible solution is to update get-fpr-changes to generate a migration instead of JSON
 +
# Commit
 +
# Merge
  
 
[[Category:Process documentation]]
 
[[Category:Process documentation]]

Revision as of 17:39, 28 March 2017

This is an outline/checklist of the process to create Archivematica & Storage service releases.

Overview

  1. Merge new features
  2. Test new features
  3. #Update PRONOM
  4. Write documentation
  5. Create packages
  6. Tag release
  7. Announce release


Update PRONOM

PRONOM needs to be updated in our file identification tools, FIDO & Siegfried, as well as in the FPR.

Update FIDO

The FPR update currently use FIDO as a source for new PRONOM, since it is formatted nicer than PRONOM offers, so we depend on FIDO having updated their PRONOM. If that has not happened, we can generate a new formats-v##.xml by updating signatures manually. Artefactual can also update PRONOM and submit a PR to FIDO.

  1. Checkout fido from https://github.com/openpreserve/fido
  2. Update signatures
    • Run python -m fido.update_signatures from the fido repository root
  3. Add:
    • New signature file fido/conf/DROID_SignatureFile-v##.xml
    • New formats file fido/conf/formats-v##.xml
    • New PRONOM zip file fido/conf/pronom-xml-v##.zip
    • Updated fido/conf/versions.xml
  4. Remove:
    • Old signature file
    • Old formats file
    • Old PRONOM zip file
  5. Update fido/fido.py to point to the new format_files
  6. Create pull request. This may trigger a release from FIDO

Package FIDO

Add packaging instructions here...

Package Siegfried

Add packaging instructions here...

Update FPR

This process is currently more convoluted than it needs to be, and should be simplified.

This depends on FIDO having updated their PRONOM files. See #Update FIDO

  1. Install the latest stable Archivematica release
  2. Generate a JSON with the current version of the FPR (for use later)
    • ./src/dashboard/src/manage.py dumpdata fpr > fpr-current.json
  3. Install the latest development version of Archivematica
  4. Generate a blank data migration in the FPR-admin module
    • E.g. ./src/dashboard/src/manage.py makemigrations --empty --name pronom_89 fpr
  5. Generate the FPR migration body. This also updates the local database's FPR with the new PRONOM IDs
    • E.g. am import-pronom-ids fido/fido/conf/formats-v89.xml --output-filename archivematica/pronom89.txt
  6. Copy the output into the blank migration, add RunPython operation
    • migrations.RunPython(data_migration),
  7. Due to a bug, fix the imports
    • Remove the apps.get_model lines
    • Import the models directly with from fpr.models import Format, FormatVersion, IDRule
  8. Commit
  9. Deploy on testing pipeline
  10. (Analysts) Update the new entries
    • Move new formats to a more appropriate category
    • Create rules & commands
    • Test with data for new formats
  11. Generate a JSON with the updated version of the FPR on the testing pipeline
    • ./src/dashboard/src/manage.py dumpdata fpr > fpr-updated.json
  12. Get the updates as JSON
    • E.g. am get-fpr-changes fpr-current.json fpr-updated.json src/dashboard/src/fpr/migrations/pronom_89.json
  13. Update the migration to load the JSON updates
    • Replace the contents of the data migration function with
      • fixture_file = os.path.join(os.path.dirname(__file__), 'pronom_89.json')
      • call_command('loaddata', fixture_file, app_label='fpr')
    • Remove the direct imports from the bug
    • Improvement Note: Because this is using loaddata, this will have problems if the FPR models are changed. A possible solution is to update get-fpr-changes to generate a migration instead of JSON
  14. Commit
  15. Merge
Retrieved from "https://wiki.archivematica.org/index.php?title=Release_Process&oldid=11762"