Difference between revisions of "Database Data Management"
(Created page with "Main Page > Development > Development documentation > Database Data Management Category:Development documentation =For the...") |
|||
| (8 intermediate revisions by one other user not shown) | |||
| Line 2: | Line 2: | ||
[[Category:Development documentation]] | [[Category:Development documentation]] | ||
| − | =For the | + | <div style="padding: 10px 10px; border: 1px solid black; background-color: #F79086;">This page is no longer being maintained and may contain inaccurate information. Please see the [https://www.archivematica.org/docs/latest/ Archivematica documentation] for up-to-date information. </div> <p> |
| + | |||
| + | =For the 0.10 Release= | ||
The data for the transcoder will need to be maintained by the FPR. The table structure will be maintained by the package. | The data for the transcoder will need to be maintained by the FPR. The table structure will be maintained by the package. | ||
| − | All pk INT's will be replaced will UUIDs. | + | All pk INT's will be replaced will UUIDs. Issue 1223. |
This will allow for local customizations to be distinguished from the original data provided in archivematica. | This will allow for local customizations to be distinguished from the original data provided in archivematica. | ||
| − | All tables will have a replaces UUID column added. | + | All tables will have a replaces UUID column added. Issue 1224 . |
This column will be populated when a provided rule is replaced with an updated/custom one. | This column will be populated when a provided rule is replaced with an updated/custom one. | ||
| − | As part of the update, the system will need to | + | As part of the update, the system will need to identify items that have been replaced, and update the foreign keys pointing at it to the updated rule. Warning: not all foreign keys are explicit in the database. Some are a "combined foriegn key", like Tasks Configs: Task type will inform the code what table to look at, and taskTypePKReference will specify the fk. |
| + | |||
| + | ==Development== | ||
| + | To separate data for the transcoder, we'll be using a series of mysql dumps. Issue 1225. Once the FPR data is seperate, the developer can work on a means of populating that into their database, with the columns they require, and getting them into the database through an API. | ||
| − | |||
| − | == | + | ==Deployed Data management== |
| − | + | Every change to the DB for workflow/FPR should be assigned a new UUID, so that it is not confused or overwritten on update. For tracking purposes the replaces column should be populated accordingly. | |
| + | ==Release== | ||
| + | A custom script for updating from the 0.9 to .10 will be developed to track existing SIPs/Tranfers/Files in the system, and not lose any data associated with them. This should be a msyql script to be run by the tool Austin has found for managing DB's across releases. Issue 1226. | ||
| − | |||
| + | =For the 1.0 Release= | ||
| + | Development changes for the db will occur through a sql script, applied against the previous release. Issue 1227. | ||
==Development== | ==Development== | ||
| + | The .10 schema will remain in the code. Rebuilding the database will consist of the following steps: | ||
| + | # Build the database from the .10 schema | ||
| + | # Apply update patch sql script file | ||
| + | # Run update script, that will replace the updated rows where specified. | ||
| + | |||
| + | The .10 schema will remain untouched from the .10 release. | ||
| + | <br/>The apply patch will be edited to make changes for the 1.0 release. | ||
| + | <br/>The update script will be developed for this release. | ||
| + | |||
| + | All dev changes to the db will occur in the patch or from the FPR. | ||
| + | ==Release== | ||
| + | Updating from .10 to 1.0 can run the patch. This requires no data loss. | ||
| − | + | New installs will run a script generated as part of the release process. It will be a dump of the .10 with the 1.0 patch applied. Future releases will use this as the base for development in the next release. | |
| + | ==Deployed Data management== | ||
| + | Same as .10 | ||
| + | ==How this works with local customizations== | ||
| + | The local customization made in .10 will not be overwritten by the ones made in the update, because all the fk's will already have been updated, so there would be nothing to replace, and point at the new rule. | ||
=For the future Releases= | =For the future Releases= | ||
| + | Each release keeps the base sql dump from the previous release. | ||
| + | A patch file is generated against this dump for the release. | ||
| + | We could look at generating the patch using http://dev.mysql.com/doc/refman/5.0/en/binary-log.html | ||
Latest revision as of 15:43, 11 February 2020
Main Page > Development > Development documentation > Database Data Management
For the 0.10 Release[edit]
The data for the transcoder will need to be maintained by the FPR. The table structure will be maintained by the package.
All pk INT's will be replaced will UUIDs. Issue 1223. This will allow for local customizations to be distinguished from the original data provided in archivematica.
All tables will have a replaces UUID column added. Issue 1224 . This column will be populated when a provided rule is replaced with an updated/custom one. As part of the update, the system will need to identify items that have been replaced, and update the foreign keys pointing at it to the updated rule. Warning: not all foreign keys are explicit in the database. Some are a "combined foriegn key", like Tasks Configs: Task type will inform the code what table to look at, and taskTypePKReference will specify the fk.
Development[edit]
To separate data for the transcoder, we'll be using a series of mysql dumps. Issue 1225. Once the FPR data is seperate, the developer can work on a means of populating that into their database, with the columns they require, and getting them into the database through an API.
Deployed Data management[edit]
Every change to the DB for workflow/FPR should be assigned a new UUID, so that it is not confused or overwritten on update. For tracking purposes the replaces column should be populated accordingly.
Release[edit]
A custom script for updating from the 0.9 to .10 will be developed to track existing SIPs/Tranfers/Files in the system, and not lose any data associated with them. This should be a msyql script to be run by the tool Austin has found for managing DB's across releases. Issue 1226.
For the 1.0 Release[edit]
Development changes for the db will occur through a sql script, applied against the previous release. Issue 1227.
Development[edit]
The .10 schema will remain in the code. Rebuilding the database will consist of the following steps:
- Build the database from the .10 schema
- Apply update patch sql script file
- Run update script, that will replace the updated rows where specified.
The .10 schema will remain untouched from the .10 release.
The apply patch will be edited to make changes for the 1.0 release.
The update script will be developed for this release.
All dev changes to the db will occur in the patch or from the FPR.
Release[edit]
Updating from .10 to 1.0 can run the patch. This requires no data loss.
New installs will run a script generated as part of the release process. It will be a dump of the .10 with the 1.0 patch applied. Future releases will use this as the base for development in the next release.
Deployed Data management[edit]
Same as .10
How this works with local customizations[edit]
The local customization made in .10 will not be overwritten by the ones made in the update, because all the fk's will already have been updated, so there would be nothing to replace, and point at the new rule.
For the future Releases[edit]
Each release keeps the base sql dump from the previous release. A patch file is generated against this dump for the release. We could look at generating the patch using http://dev.mysql.com/doc/refman/5.0/en/binary-log.html