Difference between revisions of "Database Data Management"
(updated release names 1.0 -> 0.10 and 1.1 -> 1.0) |
|||
| (2 intermediate revisions by the same user not shown) | |||
| Line 2: | Line 2: | ||
[[Category:Development documentation]] | [[Category:Development documentation]] | ||
| − | =For the | + | =For the 0.10 Release= |
The data for the transcoder will need to be maintained by the FPR. The table structure will be maintained by the package. | The data for the transcoder will need to be maintained by the FPR. The table structure will be maintained by the package. | ||
| − | All pk INT's will be replaced will UUIDs. | + | All pk INT's will be replaced will UUIDs. Issue 1223. |
This will allow for local customizations to be distinguished from the original data provided in archivematica. | This will allow for local customizations to be distinguished from the original data provided in archivematica. | ||
| − | All tables will have a replaces UUID column added. | + | All tables will have a replaces UUID column added. Issue 1224 . |
This column will be populated when a provided rule is replaced with an updated/custom one. | This column will be populated when a provided rule is replaced with an updated/custom one. | ||
As part of the update, the system will need to identify items that have been replaced, and update the foreign keys pointing at it to the updated rule. Warning: not all foreign keys are explicit in the database. Some are a "combined foriegn key", like Tasks Configs: Task type will inform the code what table to look at, and taskTypePKReference will specify the fk. | As part of the update, the system will need to identify items that have been replaced, and update the foreign keys pointing at it to the updated rule. Warning: not all foreign keys are explicit in the database. Some are a "combined foriegn key", like Tasks Configs: Task type will inform the code what table to look at, and taskTypePKReference will specify the fk. | ||
==Development== | ==Development== | ||
| − | To separate data for the transcoder, we'll be using a series of mysql dumps. Once the FPR data is seperate, the developer can work on a means of populating that into their database, with the columns they require, and getting them into the database through an API. | + | To separate data for the transcoder, we'll be using a series of mysql dumps. Issue 1225. Once the FPR data is seperate, the developer can work on a means of populating that into their database, with the columns they require, and getting them into the database through an API. |
| Line 20: | Line 20: | ||
==Release== | ==Release== | ||
| − | A custom script for updating from the 0.9 to | + | A custom script for updating from the 0.9 to .10 will be developed to track existing SIPs/Tranfers/Files in the system, and not lose any data associated with them. This should be a msyql script to be run by the tool Austin has found for managing DB's across releases. Issue 1226. |
| − | =For the 1. | + | =For the 1.0 Release= |
| − | Development changes for the db will occur through a sql script, applied against the previous release. | + | Development changes for the db will occur through a sql script, applied against the previous release. Issue 1227. |
==Development== | ==Development== | ||
| − | The | + | The .10 schema will remain in the code. Rebuilding the database will consist of the following steps: |
| − | # Build the database from the | + | # Build the database from the .10 schema |
# Apply update patch sql script file | # Apply update patch sql script file | ||
# Run update script, that will replace the updated rows where specified. | # Run update script, that will replace the updated rows where specified. | ||
| − | The | + | The .10 schema will remain untouched from the .10 release. |
| − | <br/>The apply patch will be edited to make changes for the 1. | + | <br/>The apply patch will be edited to make changes for the 1.0 release. |
<br/>The update script will be developed for this release. | <br/>The update script will be developed for this release. | ||
| Line 39: | Line 39: | ||
==Release== | ==Release== | ||
| − | Updating from | + | Updating from .10 to 1.0 can run the patch. This requires no data loss. |
| − | New installs will run a script generated as part of the release process. It will be a dump of the | + | New installs will run a script generated as part of the release process. It will be a dump of the .10 with the 1.0 patch applied. Future releases will use this as the base for development in the next release. |
==Deployed Data management== | ==Deployed Data management== | ||
| − | Same as | + | Same as .10 |
==How this works with local customizations== | ==How this works with local customizations== | ||
| − | The local customization made in | + | The local customization made in .10 will not be overwritten by the ones made in the update, because all the fk's will already have been updated, so there would be nothing to replace, and point at the new rule. |
=For the future Releases= | =For the future Releases= | ||
Each release keeps the base sql dump from the previous release. | Each release keeps the base sql dump from the previous release. | ||
A patch file is generated against this dump for the release. | A patch file is generated against this dump for the release. | ||
| + | We could look at generating the patch using http://dev.mysql.com/doc/refman/5.0/en/binary-log.html | ||
Revision as of 19:17, 13 March 2013
Main Page > Development > Development documentation > Database Data Management
For the 0.10 Release
The data for the transcoder will need to be maintained by the FPR. The table structure will be maintained by the package.
All pk INT's will be replaced will UUIDs. Issue 1223. This will allow for local customizations to be distinguished from the original data provided in archivematica.
All tables will have a replaces UUID column added. Issue 1224 . This column will be populated when a provided rule is replaced with an updated/custom one. As part of the update, the system will need to identify items that have been replaced, and update the foreign keys pointing at it to the updated rule. Warning: not all foreign keys are explicit in the database. Some are a "combined foriegn key", like Tasks Configs: Task type will inform the code what table to look at, and taskTypePKReference will specify the fk.
Development
To separate data for the transcoder, we'll be using a series of mysql dumps. Issue 1225. Once the FPR data is seperate, the developer can work on a means of populating that into their database, with the columns they require, and getting them into the database through an API.
Deployed Data management
Every change to the DB for workflow/FPR should be assigned a new UUID, so that it is not confused or overwritten on update. For tracking purposes the replaces column should be populated accordingly.
Release
A custom script for updating from the 0.9 to .10 will be developed to track existing SIPs/Tranfers/Files in the system, and not lose any data associated with them. This should be a msyql script to be run by the tool Austin has found for managing DB's across releases. Issue 1226.
For the 1.0 Release
Development changes for the db will occur through a sql script, applied against the previous release. Issue 1227.
Development
The .10 schema will remain in the code. Rebuilding the database will consist of the following steps:
- Build the database from the .10 schema
- Apply update patch sql script file
- Run update script, that will replace the updated rows where specified.
The .10 schema will remain untouched from the .10 release.
The apply patch will be edited to make changes for the 1.0 release.
The update script will be developed for this release.
All dev changes to the db will occur in the patch or from the FPR.
Release
Updating from .10 to 1.0 can run the patch. This requires no data loss.
New installs will run a script generated as part of the release process. It will be a dump of the .10 with the 1.0 patch applied. Future releases will use this as the base for development in the next release.
Deployed Data management
Same as .10
How this works with local customizations
The local customization made in .10 will not be overwritten by the ones made in the update, because all the fk's will already have been updated, so there would be nothing to replace, and point at the new rule.
For the future Releases
Each release keeps the base sql dump from the previous release. A patch file is generated against this dump for the release. We could look at generating the patch using http://dev.mysql.com/doc/refman/5.0/en/binary-log.html