https://wiki.archivematica.org/api.php?action=feedcontributions&feedformat=atom&user=Scollazo 2026-06-03T07:00:15Z User contributions MediaWiki 1.35.4 https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12912 2019-03-01T02:37:00Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> 1. Backup elasticsearch data directory <br /> <br /> sudo service elasticsearch stop<br /> sudo tar cfz var_lib_elasticsearch_$(date +%y%m%d).tgz /var/lib/elasticsearch<br /> sudo service elasticsearch start<br /> <br /> 2. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 24816 1 175.4mb 175.4mb <br /> yellow open transfers 5 1 0 0 720b 720b <br /> <br /> Check also if there are any configured snapshot:<br /> <br /> curl -XGET &quot;localhost:9200/_snapshot/_all?pretty&quot;<br /> <br /> 3. Remove Snapshots<br /> <br /> If there are any snapshot (step 1), please remove it:<br /> <br /> curl -XDELETE &quot;localhost:9200/_snapshot/backup-repo?pretty&quot;<br /> <br /> 4. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> 5. Check wich elasticsearch version is installed<br /> <br /> Redhat/Centos: rpm -q elasticsearch<br /> Ubuntu: dpkg -l elasticsearch<br /> <br /> 6. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.6.tar.gz<br /> <br /> <br /> 7. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.6.tar.gz<br /> cd elasticsearch-1.7.6<br /> <br /> <br /> 8. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> sudo cp /var/lib/elasticsearch data -rf<br /> <br /> 9. Adjust file permissions<br /> <br /> sudo chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 10. Launch the temporary elasticsearch instance<br /> <br /> ES_JAVA_OPTS=&quot;-Xms2g -Xmx2g&quot; ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500 -Des.discovery.zen.ping.multicast.enabled=false<br /> <br /> 11. Verify that the indexes were properly copied. Use the port specified in the previous command:<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 12. Remove elasticsearch 1.7.6 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove --purge elasticsearch<br /> sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.6<br /> sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.6<br /> <br /> 13. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 14. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 15. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster -t 60 -s 1 http://localhost:9500<br /> &quot;;<br /> <br /> The command should finish with<br /> <br /> All reindex requests ended successfully!<br /> <br /> Else, you might need to tweak the timeout (-t) and size (-s) parameters.<br /> <br /> <br /> 16. Verify that the new indexes were created and populated:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open transferfiles jlu2d2yZQpWwpKKT1ZzMAA 5 1 0 0 1.1kb 1.1kb<br /> yellow open aips VAvONAByRBuVcWBYhlhUmw 5 1 116 0 66.9mb 66.9mb<br /> yellow open transfers OhfLVBzuRqCNUYRCSSTidg 5 1 0 0 1.1kb 1.1kb<br /> yellow open aipfiles clX1gcqnT9CTdwTm8jeatw 5 1 24700 0 70.6mb 70.6mb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.6. The same happens with the transfers/transferfiles indexes.<br /> <br /> 17. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.6/elastic-tmp.pid)<br /> <br /> 18. Restart all archivematica services<br /> <br /> sudo service archivematica-mcp-server restart<br /> sudo service archivematica-mcp-client restart<br /> sudo service archivematica-dashboard restart<br /> sudo service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12911 2019-03-01T02:14:14Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> 1. Backup elasticsearch data directory <br /> <br /> sudo service elasticsearch stop<br /> sudo tar cfz var_lib_elasticsearch_$(date +%y%m%d).tgz /var/lib/elasticsearch<br /> sudo service elasticsearch start<br /> <br /> 2. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 24816 1 175.4mb 175.4mb <br /> yellow open transfers 5 1 0 0 720b 720b <br /> <br /> Check also if there are any configured snapshot:<br /> <br /> curl -XGET &quot;localhost:9200/_snapshot/_all?pretty&quot;<br /> <br /> 3. Remove Snapshots<br /> <br /> If there are any snapshot (step 1), please remove it:<br /> <br /> curl -XDELETE &quot;localhost:9200/_snapshot/backup-repo?pretty&quot;<br /> <br /> 4. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> 5. Check wich elasticsearch version is installed<br /> <br /> Redhat/Centos: rpm -q elasticsearch<br /> Ubuntu: dpkg -l elasticsearch<br /> <br /> 6. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 7. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 8. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> sudo cp /var/lib/elasticsearch data -rf<br /> <br /> 9. Adjust file permissions<br /> <br /> sudo chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 10. Launch the temporary elasticsearch instance<br /> <br /> ES_JAVA_OPTS=&quot;-Xms2g -Xmx2g&quot; ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500 -Des.discovery.zen.ping.multicast.enabled=false<br /> <br /> 11. Verify that the indexes were properly copied. Use the port specified in the previous command:<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 12. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove --purge elasticsearch<br /> sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.5<br /> sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.5<br /> <br /> 13. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 14. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 15. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster -t 60 -s 10 http://localhost:9500<br /> &quot;;<br /> <br /> The command should finish with<br /> <br /> All reindex requests ended successfully!<br /> <br /> Else, you might need to tweak the timeout (-t) and size (-s) parameters.<br /> <br /> <br /> 16. Verify that the new indexes were created and populated:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open transferfiles jlu2d2yZQpWwpKKT1ZzMAA 5 1 0 0 1.1kb 1.1kb<br /> yellow open aips VAvONAByRBuVcWBYhlhUmw 5 1 116 0 66.9mb 66.9mb<br /> yellow open transfers OhfLVBzuRqCNUYRCSSTidg 5 1 0 0 1.1kb 1.1kb<br /> yellow open aipfiles clX1gcqnT9CTdwTm8jeatw 5 1 24700 0 70.6mb 70.6mb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.<br /> <br /> 17. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)<br /> <br /> 18. Restart all archivematica services<br /> <br /> sudo service archivematica-mcp-server restart<br /> sudo service archivematica-mcp-client restart<br /> sudo service archivematica-dashboard restart<br /> sudo service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12910 2019-03-01T02:13:03Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> 1. Backup elasticsearch data directory <br /> <br /> sudo service elasticsearch stop<br /> sudo tar cfz var_lib_elasticsearch_$(date +%y%m%d).tgz /var/lib/elasticsearch<br /> sudo service elasticsearch start<br /> <br /> 2. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 24816 1 175.4mb 175.4mb <br /> yellow open transfers 5 1 0 0 720b 720b <br /> <br /> Check also if there are any configured snapshot:<br /> <br /> curl -XGET &quot;localhost:9200/_snapshot/_all?pretty&quot;<br /> <br /> 3. Remove Snapshots<br /> <br /> If there are any snapshot (step 1), please remove it:<br /> <br /> curl -XDELETE &quot;localhost:9200/_snapshot/backup-repo?pretty&quot;<br /> <br /> 4. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> 5. Check wich elasticsearch version is installed<br /> <br /> Redhat/Centos: rpm -q elasticsearch<br /> Ubuntu: dpkg -l elasticsearch<br /> <br /> 6. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 7. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 8. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> sudo cp /var/lib/elasticsearch data -rf<br /> <br /> 9. Adjust file permissions<br /> <br /> sudo chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 10. Launch the temporary elasticsearch instance<br /> <br /> ES_JAVA_OPTS=&quot;-Xms2g -Xmx2g&quot; ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500<br /> <br /> 11. Verify that the indexes were properly copied. Use the port specified in the previous command:<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 12. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove --purge elasticsearch<br /> sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.5<br /> sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.5<br /> <br /> 13. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 14. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 15. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster -t 60 -s 10 http://localhost:9500<br /> &quot;;<br /> <br /> The command should finish with<br /> <br /> All reindex requests ended successfully!<br /> <br /> Else, you might need to tweak the timeout (-t) and size (-s) parameters.<br /> <br /> <br /> 16. Verify that the new indexes were created and populated:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open transferfiles jlu2d2yZQpWwpKKT1ZzMAA 5 1 0 0 1.1kb 1.1kb<br /> yellow open aips VAvONAByRBuVcWBYhlhUmw 5 1 116 0 66.9mb 66.9mb<br /> yellow open transfers OhfLVBzuRqCNUYRCSSTidg 5 1 0 0 1.1kb 1.1kb<br /> yellow open aipfiles clX1gcqnT9CTdwTm8jeatw 5 1 24700 0 70.6mb 70.6mb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.<br /> <br /> 17. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)<br /> <br /> 18. Restart all archivematica services<br /> <br /> sudo service archivematica-mcp-server restart<br /> sudo service archivematica-mcp-client restart<br /> sudo service archivematica-dashboard restart<br /> sudo service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch_snapshots&diff=12906 2019-02-27T14:29:13Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 24816 1 175.4mb 175.4mb <br /> yellow open transfers 5 1 0 0 720b 720b <br /> <br /> 2. Create an ElasticSearch backup (they are called snapshots ) using the following commands:<br /> <br /> a. Remove and recreate the folder that stores the backup<br /> <br /> sudo rm -rf /var/lib/elasticsearch/backup-repo/<br /> sudo mkdir -p /var/lib/elasticsearch/backup-repo/<br /> sudo chown elasticsearch:elasticsearch /var/lib/elasticsearch/backup-repo/<br /> <br /> b. Allow elasticsearch to write files to the backup<br /> <br /> echo 'path.repo: [&quot;/var/lib/elasticsearch/backup-repo&quot;]' |sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> <br /> c. Restart ElasticSearch and wait for it to start<br /> <br /> sudo service elasticsearch restart<br /> sleep 60s<br /> <br /> d. Configure the ES backup<br /> curl -XPUT &quot;localhost:9200/_snapshot/backup-repo&quot; -H 'Content-Type: application/json' -d \<br /> '{<br /> &quot;type&quot;: &quot;fs&quot;,<br /> &quot;settings&quot;: {<br /> &quot;location&quot;: &quot;./&quot;,<br /> &quot;compress&quot;: true<br /> }<br /> }'<br /> <br /> e. Take the actual backup, and copy it to a safe place<br /> curl -X PUT &quot;localhost:9200/_snapshot/backup-repo/am_indexes_backup?wait_for_completion=true&quot;<br /> cp /var/lib/elasticsearch/backup-repo ~/elasticsearch-backup -rf<br /> <br /> <br /> <br /> 3. Check wich elasticsearch version is installed<br /> <br /> Redhat/Centos: rpm -qa | grep elasticsearch<br /> Ubuntu: dpkg -l | grep elasticsearch<br /> <br /> 4. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.6.tar.gz<br /> <br /> <br /> 5. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 6. Copy the backup you just created <br /> <br /> mkdir -p data<br /> cp /var/lib/elasticsearch/backup-repo data -rf # Use sudo if needed<br /> <br /> 7. Adjust file permissions<br /> <br /> sudo chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 8. Open other shell, and launch the temporary elasticsearch instance<br /> <br /> cd elasticsearch-1.7.5<br /> ES_JAVA_OPTS=&quot;-Xms2g -Xmx2g&quot; ./bin/elasticsearch -p elastic-tmp.pid -Des.http.port=9500 -Des.path.repo=data/backup-repo<br /> <br /> 9. Tell the temporary ElasticSearch instance about the backup files, and restore them:<br /> <br /> curl -XPUT &quot;localhost:9500/_snapshot/backup-repo&quot; -H 'Content-Type: application/json' -d \<br /> '{<br /> &quot;type&quot;: &quot;fs&quot;,<br /> &quot;settings&quot;: {<br /> &quot;location&quot;: &quot;./&quot;,<br /> &quot;compress&quot;: true<br /> }<br /> }'<br /> curl -X POST &quot;localhost:9500/_snapshot/backup-repo/am_indexes_backup/_restore?wait_for_completion=true&quot;<br /> <br /> 10. Verify that the temporary ElasticSearch service has the same content than the system one:<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 24816 1 175.4mb 175.4mb <br /> yellow open transfers 5 1 0 0 720b 720b <br /> <br /> <br /> 11. Remove system's ElasticSearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove --purge elasticsearch<br /> sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.5<br /> sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.5<br /> <br /> 12. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 13. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 14. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster -t 60 -s 1 http://localhost:9500<br /> &quot;;<br /> <br /> The command should finish with<br /> <br /> All reindex requests ended successfully!<br /> <br /> Else, you might need to tweak the timeout (-t) and size (-s) parameters.<br /> <br /> <br /> 15. Verify that the new indexes were created and populated:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open transferfiles jlu2d2yZQpWwpKKT1ZzMAA 5 1 0 0 1.1kb 1.1kb<br /> yellow open aips VAvONAByRBuVcWBYhlhUmw 5 1 116 0 66.9mb 66.9mb<br /> yellow open transfers OhfLVBzuRqCNUYRCSSTidg 5 1 0 0 1.1kb 1.1kb<br /> yellow open aipfiles clX1gcqnT9CTdwTm8jeatw 5 1 24700 0 70.6mb 70.6mb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.<br /> <br /> 16. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)<br /> <br /> 17. Restart all archivematica services<br /> <br /> service archivematica-dashboard restart<br /> service archivematica-mcp-server restart<br /> service archivematica-mcp-client restart<br /> service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch_snapshots&diff=12905 2019-02-27T14:18:22Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 24816 1 175.4mb 175.4mb <br /> yellow open transfers 5 1 0 0 720b 720b <br /> <br /> 2. Create an ElasticSearch backup (they are called snapshots ) using the following commands:<br /> <br /> a. Remove and recreate the folder that stores the backup<br /> <br /> sudo rm -rf /var/lib/elasticsearch/backup-repo/<br /> sudo mkdir -p /var/lib/elasticsearch/backup-repo/<br /> sudo chown elasticsearch:elasticsearch /var/lib/elasticsearch/backup-repo/<br /> <br /> b. Allow elasticsearch to write files to the backup<br /> <br /> echo 'path.repo: [&quot;/var/lib/elasticsearch/backup-repo&quot;]' |sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> <br /> c. Restart ElasticSearch and wait for it to start<br /> <br /> sudo service elasticsearch restart<br /> sleep 60s<br /> <br /> d. Configure the ES backup<br /> curl -XPUT &quot;localhost:9200/_snapshot/backup-repo&quot; -H 'Content-Type: application/json' -d \<br /> '{<br /> &quot;type&quot;: &quot;fs&quot;,<br /> &quot;settings&quot;: {<br /> &quot;location&quot;: &quot;./&quot;,<br /> &quot;compress&quot;: true<br /> }<br /> }'<br /> <br /> e. Take the actual backup, and copy it to a safe place<br /> curl -X PUT &quot;localhost:9200/_snapshot/backup-repo/am_indexes_backup?wait_for_completion=true&quot;<br /> cp /var/lib/elasticsearch/backup-repo ~/elasticsearch-backup -rf<br /> <br /> <br /> <br /> 3. Check wich elasticsearch version is installed<br /> <br /> Redhat/Centos: rpm -qa | grep elasticsearch<br /> Ubuntu: dpkg -l | grep elasticsearch<br /> <br /> 4. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.6.tar.gz<br /> <br /> <br /> 5. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 6. Copy the backup you just created <br /> <br /> mkdir -p data<br /> cp /var/lib/elasticsearch/backup-repo data -rf # Use sudo if needed<br /> <br /> 7. Adjust file permissions<br /> <br /> sudo chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 8. Open other shell, and launch the temporary elasticsearch instance<br /> <br /> cd elasticsearch-1.7.5<br /> ES_JAVA_OPTS=&quot;-Xms2g -Xmx2g&quot; ./bin/elasticsearch -p elastic-tmp.pid -Des.http.port=9500 -Des.path.repo=data/backup-repo<br /> <br /> 9. Tell the temporary ElasticSearch instance about the backup files, and restore them:<br /> <br /> curl -XPUT &quot;localhost:9500/_snapshot/backup-repo&quot; -H 'Content-Type: application/json' -d \<br /> '{<br /> &quot;type&quot;: &quot;fs&quot;,<br /> &quot;settings&quot;: {<br /> &quot;location&quot;: &quot;./&quot;,<br /> &quot;compress&quot;: true<br /> }<br /> }'<br /> curl -X POST &quot;localhost:9500/_snapshot/backup-repo/am_indexes_backup/_restore?wait_for_completion=true&quot;<br /> <br /> 10. Verify that the temporary ElasticSearch service has the same content than the system one:<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 24816 1 175.4mb 175.4mb <br /> yellow open transfers 5 1 0 0 720b 720b <br /> <br /> <br /> 11. Remove system's ElasticSearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove --purge elasticsearch<br /> sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.5<br /> sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.5<br /> <br /> 12. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 13. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 14. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster -t 60 -s 10 http://localhost:9500<br /> &quot;;<br /> <br /> The command should finish with<br /> <br /> All reindex requests ended successfully!<br /> <br /> Else, you might need to tweak the timeout (-t) and size (-s) parameters.<br /> <br /> <br /> 15. Verify that the new indexes were created and populated:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open transferfiles jlu2d2yZQpWwpKKT1ZzMAA 5 1 0 0 1.1kb 1.1kb<br /> yellow open aips VAvONAByRBuVcWBYhlhUmw 5 1 116 0 66.9mb 66.9mb<br /> yellow open transfers OhfLVBzuRqCNUYRCSSTidg 5 1 0 0 1.1kb 1.1kb<br /> yellow open aipfiles clX1gcqnT9CTdwTm8jeatw 5 1 24700 0 70.6mb 70.6mb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.<br /> <br /> 16. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)<br /> <br /> 17. Restart all archivematica services<br /> <br /> service archivematica-dashboard restart<br /> service archivematica-mcp-server restart<br /> service archivematica-mcp-client restart<br /> service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch_snapshots&diff=12904 2019-02-27T14:13:21Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 24816 1 175.4mb 175.4mb <br /> yellow open transfers 5 1 0 0 720b 720b <br /> <br /> 2. Create an ElasticSearch backup (they are called snapshots ) using the following commands:<br /> <br /> a. Remove and recreate the folder that stores the backup<br /> <br /> sudo rm -rf /var/lib/elasticsearch/backup-repo/<br /> sudo mkdir -p /var/lib/elasticsearch/backup-repo/<br /> sudo chown elasticsearch:elasticsearch /var/lib/elasticsearch/backup-repo/<br /> <br /> b. Allow elasticsearch to write files to the backup<br /> <br /> echo 'path.repo: [&quot;/var/lib/elasticsearch/backup-repo&quot;]' |sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> <br /> c. Restart ElasticSearch and wait for it to start<br /> <br /> sudo service elasticsearch restart<br /> sleep 60s<br /> <br /> d. Configure the ES backup<br /> curl -XPUT &quot;localhost:9200/_snapshot/backup-repo&quot; -H 'Content-Type: application/json' -d \<br /> '{<br /> &quot;type&quot;: &quot;fs&quot;,<br /> &quot;settings&quot;: {<br /> &quot;location&quot;: &quot;./&quot;,<br /> &quot;compress&quot;: true<br /> }<br /> }'<br /> <br /> e. Take the actual backup, and copy it to a safe place<br /> curl -X PUT &quot;localhost:9200/_snapshot/backup-repo/am_indexes_backup?wait_for_completion=true&quot;<br /> cp /var/lib/elasticsearch/backup-repo ~/elasticsearch-backup -rf<br /> <br /> <br /> <br /> 3. Check wich elasticsearch version is installed<br /> <br /> Redhat/Centos: rpm -qa | grep elasticsearch<br /> Ubuntu: dpkg -l | grep elasticsearch<br /> <br /> 4. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.6.tar.gz<br /> <br /> <br /> 5. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 6. Copy the backup you just created <br /> <br /> mkdir -p data<br /> cp /var/lib/elasticsearch/backup-repo data -rf # Use sudo if needed<br /> <br /> 7. Adjust file permissions<br /> <br /> sudo chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 8. Open other shell, and launch the temporary elasticsearch instance<br /> <br /> cd elasticsearch-1.7.5<br /> ES_JAVA_OPTS=&quot;-Xms2g -Xmx2g&quot; ./bin/elasticsearch -p elastic-tmp.pid -Des.http.port=9500 -Des.path.repo=data/backup-repo<br /> <br /> 9. Tell the temporary ElasticSearch instance about the backup files, and restore them:<br /> <br /> curl -XPUT &quot;localhost:9500/_snapshot/backup-repo&quot; -H 'Content-Type: application/json' -d \<br /> '{<br /> &quot;type&quot;: &quot;fs&quot;,<br /> &quot;settings&quot;: {<br /> &quot;location&quot;: &quot;./&quot;,<br /> &quot;compress&quot;: true<br /> }<br /> }'<br /> curl -X PUT &quot;localhost:9500/_snapshot/backup-repo/am_indexes_backup??wait_for_completion=true&quot;<br /> <br /> 10. Verify that the temporary ElasticSearch service has the same content than the system one:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 24816 1 175.4mb 175.4mb <br /> yellow open transfers 5 1 0 0 720b 720b <br /> <br /> <br /> 11. Remove system's ElasticSearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove --purge elasticsearch<br /> sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.5<br /> sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.5<br /> <br /> 12. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 13. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 14. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster -t 60 -s 10 http://localhost:9500<br /> &quot;;<br /> <br /> The command should finish with<br /> <br /> All reindex requests ended successfully!<br /> <br /> Else, you might need to tweak the timeout (-t) and size (-s) parameters.<br /> <br /> <br /> 15. Verify that the new indexes were created and populated:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open transferfiles jlu2d2yZQpWwpKKT1ZzMAA 5 1 0 0 1.1kb 1.1kb<br /> yellow open aips VAvONAByRBuVcWBYhlhUmw 5 1 116 0 66.9mb 66.9mb<br /> yellow open transfers OhfLVBzuRqCNUYRCSSTidg 5 1 0 0 1.1kb 1.1kb<br /> yellow open aipfiles clX1gcqnT9CTdwTm8jeatw 5 1 24700 0 70.6mb 70.6mb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.<br /> <br /> 16. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)<br /> <br /> 17. Restart all archivematica services<br /> <br /> service archivematica-dashboard restart<br /> service archivematica-mcp-server restart<br /> service archivematica-mcp-client restart<br /> service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch_snapshots&diff=12903 2019-02-27T14:11:12Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 24816 1 175.4mb 175.4mb <br /> yellow open transfers 5 1 0 0 720b 720b <br /> <br /> 2. Create an ElasticSearch backup (they are called snapshots ) using the following commands:<br /> <br /> a. Remove and recreate the folder that stores the backup<br /> <br /> sudo rm -rf /var/lib/elasticsearch/backup-repo/<br /> sudo mkdir -p /var/lib/elasticsearch/backup-repo/<br /> sudo chown elasticsearch:elasticsearch /var/lib/elasticsearch/backup-repo/<br /> <br /> b. Allow elasticsearch to write files to the backup<br /> <br /> echo 'path.repo: [&quot;/var/lib/elasticsearch/backup-repo&quot;]' |sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> <br /> c. Restart ElasticSearch and wait for it to start<br /> <br /> sudo service elasticsearch restart<br /> sleep 60s<br /> <br /> d. Configure the ES backup<br /> curl -XPUT &quot;localhost:9200/_snapshot/backup-repo&quot; -H 'Content-Type: application/json' -d \<br /> '{<br /> &quot;type&quot;: &quot;fs&quot;,<br /> &quot;settings&quot;: {<br /> &quot;location&quot;: &quot;./&quot;,<br /> &quot;compress&quot;: true<br /> }<br /> }'<br /> # Take the actual backup, and copy it to a safe place<br /> curl -X PUT &quot;localhost:9200/_snapshot/backup-repo/am_indexes_backup?wait_for_completion=true&quot;<br /> cp /var/lib/elasticsearch/backup-repo ~/elasticsearch-backup -rf<br /> <br /> <br /> <br /> 3. Check wich elasticsearch version is installed<br /> <br /> Redhat/Centos: rpm -qa | grep elasticsearch<br /> Ubuntu: dpkg -l | grep elasticsearch<br /> <br /> 4. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.6.tar.gz<br /> <br /> <br /> 5. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 6. Copy the backup you just created <br /> <br /> mkdir -p data<br /> cp /var/lib/elasticsearch/backup-repo data -rf # Use sudo if needed<br /> <br /> 7. Adjust file permissions<br /> <br /> sudo chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 8. Open other shell, and launch the temporary elasticsearch instance<br /> <br /> cd elasticsearch-1.7.5<br /> ES_JAVA_OPTS=&quot;-Xms2g -Xmx2g&quot; ./bin/elasticsearch -p elastic-tmp.pid -Des.http.port=9500 -Des.path.repo=data/backup-repo<br /> <br /> 9. Tell the temporary ElasticSearch instance about the backup files, and restore them:<br /> <br /> curl -XPUT &quot;localhost:9500/_snapshot/backup-repo&quot; -H 'Content-Type: application/json' -d \<br /> '{<br /> &quot;type&quot;: &quot;fs&quot;,<br /> &quot;settings&quot;: {<br /> &quot;location&quot;: &quot;./&quot;,<br /> &quot;compress&quot;: true<br /> }<br /> }'<br /> curl -X PUT &quot;localhost:9500/_snapshot/backup-repo/am_indexes_backup??wait_for_completion=true&quot;<br /> <br /> 10. Verify that the temporary ElasticSearch service has the same content than the system one:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 24816 1 175.4mb 175.4mb <br /> yellow open transfers 5 1 0 0 720b 720b <br /> <br /> <br /> 11. Remove system's ElasticSearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove --purge elasticsearch<br /> sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.5<br /> sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.5<br /> <br /> 12. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 13. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 14. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster -t 60 -s 10 http://localhost:9500<br /> &quot;;<br /> <br /> The command should finish with<br /> <br /> All reindex requests ended successfully!<br /> <br /> Else, you might need to tweak the timeout (-t) and size (-s) parameters.<br /> <br /> <br /> 15. Verify that the new indexes were created and populated:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open transferfiles jlu2d2yZQpWwpKKT1ZzMAA 5 1 0 0 1.1kb 1.1kb<br /> yellow open aips VAvONAByRBuVcWBYhlhUmw 5 1 116 0 66.9mb 66.9mb<br /> yellow open transfers OhfLVBzuRqCNUYRCSSTidg 5 1 0 0 1.1kb 1.1kb<br /> yellow open aipfiles clX1gcqnT9CTdwTm8jeatw 5 1 24700 0 70.6mb 70.6mb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.<br /> <br /> 16. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)<br /> <br /> 17. Restart all archivematica services<br /> <br /> service archivematica-dashboard restart<br /> service archivematica-mcp-server restart<br /> service archivematica-mcp-client restart<br /> service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch_snapshots&diff=12902 2019-02-27T14:10:46Z

<p>Scollazo: Created page with &quot;= ElasticSearch update = 1. Check that ES is up and running, and note the results: curl -X GET &quot;localhost:9200/_cat/indices?v&quot; health status index pri rep docs.count...&quot;</p> <hr /> <div>= ElasticSearch update =<br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 24816 1 175.4mb 175.4mb <br /> yellow open transfers 5 1 0 0 720b 720b <br /> <br /> 2. Create an ElasticSearch snapshot using the following commands:<br /> <br /> a. Remove and recreate the folder that stores the backup<br /> <br /> sudo rm -rf /var/lib/elasticsearch/backup-repo/<br /> sudo mkdir -p /var/lib/elasticsearch/backup-repo/<br /> sudo chown elasticsearch:elasticsearch /var/lib/elasticsearch/backup-repo/<br /> <br /> b. Allow elasticsearch to write files to the backup<br /> <br /> echo 'path.repo: [&quot;/var/lib/elasticsearch/backup-repo&quot;]' |sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> <br /> c. Restart ElasticSearch and wait for it to start<br /> <br /> sudo service elasticsearch restart<br /> sleep 60s<br /> <br /> d. Configure the ES backup<br /> curl -XPUT &quot;localhost:9200/_snapshot/backup-repo&quot; -H 'Content-Type: application/json' -d \<br /> '{<br /> &quot;type&quot;: &quot;fs&quot;,<br /> &quot;settings&quot;: {<br /> &quot;location&quot;: &quot;./&quot;,<br /> &quot;compress&quot;: true<br /> }<br /> }'<br /> # Take the actual backup, and copy it to a safe place<br /> curl -X PUT &quot;localhost:9200/_snapshot/backup-repo/am_indexes_backup?wait_for_completion=true&quot;<br /> cp /var/lib/elasticsearch/backup-repo ~/elasticsearch-backup -rf<br /> <br /> <br /> <br /> 3. Check wich elasticsearch version is installed<br /> <br /> Redhat/Centos: rpm -qa | grep elasticsearch<br /> Ubuntu: dpkg -l | grep elasticsearch<br /> <br /> 4. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.6.tar.gz<br /> <br /> <br /> 5. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 6. Copy the backup you just created <br /> <br /> mkdir -p data<br /> cp /var/lib/elasticsearch/backup-repo data -rf # Use sudo if needed<br /> <br /> 7. Adjust file permissions<br /> <br /> sudo chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 8. Open other shell, and launch the temporary elasticsearch instance<br /> <br /> cd elasticsearch-1.7.5<br /> ES_JAVA_OPTS=&quot;-Xms2g -Xmx2g&quot; ./bin/elasticsearch -p elastic-tmp.pid -Des.http.port=9500 -Des.path.repo=data/backup-repo<br /> <br /> 9. Tell the temporary ElasticSearch instance about the backup files, and restore them:<br /> <br /> curl -XPUT &quot;localhost:9500/_snapshot/backup-repo&quot; -H 'Content-Type: application/json' -d \<br /> '{<br /> &quot;type&quot;: &quot;fs&quot;,<br /> &quot;settings&quot;: {<br /> &quot;location&quot;: &quot;./&quot;,<br /> &quot;compress&quot;: true<br /> }<br /> }'<br /> curl -X PUT &quot;localhost:9500/_snapshot/backup-repo/am_indexes_backup??wait_for_completion=true&quot;<br /> <br /> 10. Verify that the temporary ElasticSearch service has the same content than the system one:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 24816 1 175.4mb 175.4mb <br /> yellow open transfers 5 1 0 0 720b 720b <br /> <br /> <br /> 11. Remove system's ElasticSearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove --purge elasticsearch<br /> sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.5<br /> sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.5<br /> <br /> 12. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 13. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 14. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster -t 60 -s 10 http://localhost:9500<br /> &quot;;<br /> <br /> The command should finish with<br /> <br /> All reindex requests ended successfully!<br /> <br /> Else, you might need to tweak the timeout (-t) and size (-s) parameters.<br /> <br /> <br /> 15. Verify that the new indexes were created and populated:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open transferfiles jlu2d2yZQpWwpKKT1ZzMAA 5 1 0 0 1.1kb 1.1kb<br /> yellow open aips VAvONAByRBuVcWBYhlhUmw 5 1 116 0 66.9mb 66.9mb<br /> yellow open transfers OhfLVBzuRqCNUYRCSSTidg 5 1 0 0 1.1kb 1.1kb<br /> yellow open aipfiles clX1gcqnT9CTdwTm8jeatw 5 1 24700 0 70.6mb 70.6mb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.<br /> <br /> 16. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)<br /> <br /> 17. Restart all archivematica services<br /> <br /> service archivematica-dashboard restart<br /> service archivematica-mcp-server restart<br /> service archivematica-mcp-client restart<br /> service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12901 2019-02-27T00:25:44Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> <br /> <br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 24816 1 175.4mb 175.4mb <br /> yellow open transfers 5 1 0 0 720b 720b <br /> <br /> Check also if there are any configured snapshot. If so, copy them to other place and remove them<br /> <br /> curl -XGET &quot;localhost:9200/_snapshot/_all?pretty&quot; <br /> curl -XDELETE &quot;localhost:9200/_snapshot/backup-repo?pretty&quot;<br /> <br /> <br /> 2. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> 3. Check wich elasticsearch version is installed<br /> <br /> Redhat/Centos: rpm -qa | grep elasticsearch<br /> Ubuntu: dpkg -l | grep elasticsearch<br /> <br /> 4. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 5. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 6. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> sudo cp /var/lib/elasticsearch data -rf<br /> <br /> 7. Adjust file permissions<br /> <br /> sudo chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 8. Launch the temporary elasticsearch instance<br /> <br /> ES_JAVA_OPTS=&quot;-Xms2g -Xmx2g&quot; ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500<br /> <br /> 9. Verify that the indexes were properly copied. Use the port specified in the previous command:<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 10. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove --purge elasticsearch<br /> sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.5<br /> sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.5<br /> <br /> 11. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 12. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 13. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster -t 60 -s 10 http://localhost:9500<br /> &quot;;<br /> <br /> The command should finish with<br /> <br /> All reindex requests ended successfully!<br /> <br /> Else, you might need to tweak the timeout (-t) and size (-s) parameters.<br /> <br /> <br /> 14. Verify that the new indexes were created and populated:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open transferfiles jlu2d2yZQpWwpKKT1ZzMAA 5 1 0 0 1.1kb 1.1kb<br /> yellow open aips VAvONAByRBuVcWBYhlhUmw 5 1 116 0 66.9mb 66.9mb<br /> yellow open transfers OhfLVBzuRqCNUYRCSSTidg 5 1 0 0 1.1kb 1.1kb<br /> yellow open aipfiles clX1gcqnT9CTdwTm8jeatw 5 1 24700 0 70.6mb 70.6mb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.<br /> <br /> 15. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)<br /> <br /> 16. Restart all archivematica services<br /> <br /> service archivematica-dashboard restart<br /> service archivematica-mcp-server restart<br /> service archivematica-mcp-client restart<br /> service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12900 2019-02-27T00:02:53Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> <br /> <br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 24816 1 175.4mb 175.4mb <br /> yellow open transfers 5 1 0 0 720b 720b <br /> <br /> Check also if there are any configured snapshot. If so, copy them to other place and remove them<br /> <br /> curl -XGET &quot;localhost:9200/_snapshot/_all?pretty&quot; <br /> curl -XDELETE &quot;localhost:9200/_snapshot/backup-repo?pretty&quot;<br /> <br /> <br /> 2. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> 3. Check wich elasticsearch version is installed<br /> <br /> Redhat/Centos: rpm -qa | grep elasticsearch<br /> Ubuntu: dpkg -l | grep elasticsearch<br /> <br /> 4. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 5. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 6. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> sudo cp /var/lib/elasticsearch data -rf<br /> <br /> 7. Adjust file permissions<br /> <br /> sudo chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 8. Launch the temporary elasticsearch instance<br /> <br /> ES_JAVA_OPTS=&quot;-Xms2g -Xmx2g&quot; ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500<br /> <br /> 9. Verify that the indexes were properly copied. Use the port specified in the previous command:<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 10. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove --purge elasticsearch<br /> sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.5<br /> sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.5<br /> <br /> 11. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 12. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 13. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster -t 60 http://localhost:9500<br /> &quot;;<br /> <br /> The command should finish with<br /> <br /> All reindex requests ended successfully!<br /> <br /> <br /> 14. Verify that the new indexes were created and populated:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open transferfiles jlu2d2yZQpWwpKKT1ZzMAA 5 1 0 0 1.1kb 1.1kb<br /> yellow open aips VAvONAByRBuVcWBYhlhUmw 5 1 116 0 66.9mb 66.9mb<br /> yellow open transfers OhfLVBzuRqCNUYRCSSTidg 5 1 0 0 1.1kb 1.1kb<br /> yellow open aipfiles clX1gcqnT9CTdwTm8jeatw 5 1 24700 0 70.6mb 70.6mb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.<br /> <br /> 15. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)<br /> <br /> 16. Restart all archivematica services<br /> <br /> service archivematica-dashboard restart<br /> service archivematica-mcp-server restart<br /> service archivematica-mcp-client restart<br /> service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12890 2019-02-19T18:49:22Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> <br /> <br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 24816 1 175.4mb 175.4mb <br /> yellow open transfers 5 1 0 0 720b 720b <br /> <br /> <br /> 2. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> 3. Check wich elasticsearch version is installed<br /> <br /> Redhat/Centos: rpm -qa | grep elasticsearch<br /> Ubuntu: dpkg -l | grep elasticsearch<br /> <br /> 4. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 5. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 6. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> sudo cp /var/lib/elasticsearch data -rf<br /> <br /> 7. Adjust file permissions<br /> <br /> sudo chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 8. Launch the temporary elasticsearch instance<br /> <br /> ES_JAVA_OPTS=&quot;-Xms2g -Xmx2g&quot; ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500<br /> <br /> 9. Verify that the indexes were properly copied. Use the port specified in the previous command:<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 10. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove --purge elasticsearch<br /> sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.5<br /> sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.5<br /> <br /> 11. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 12. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 13. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster -t 60 http://localhost:9500<br /> &quot;;<br /> <br /> The command should finish with<br /> <br /> All reindex requests ended successfully!<br /> <br /> <br /> 14. Verify that the new indexes were created and populated:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open transferfiles jlu2d2yZQpWwpKKT1ZzMAA 5 1 0 0 1.1kb 1.1kb<br /> yellow open aips VAvONAByRBuVcWBYhlhUmw 5 1 116 0 66.9mb 66.9mb<br /> yellow open transfers OhfLVBzuRqCNUYRCSSTidg 5 1 0 0 1.1kb 1.1kb<br /> yellow open aipfiles clX1gcqnT9CTdwTm8jeatw 5 1 24700 0 70.6mb 70.6mb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.<br /> <br /> 15. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)<br /> <br /> 16. Restart all archivematica services<br /> <br /> service archivematica-dashboard restart<br /> service archivematica-mcp-server restart<br /> service archivematica-mcp-client restart<br /> service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12889 2019-02-19T18:25:57Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> <br /> <br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 11 0 267.9kb 267.9kb <br /> yellow open transfers 5 1 24 0 35.2kb 35.2kb <br /> <br /> 2. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> 3. Check wich elasticsearch version is installed<br /> <br /> Redhat/Centos: rpm -qa | grep elasticsearch<br /> Ubuntu: dpkg -l | grep elasticsearch<br /> <br /> 4. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 5. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 6. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> sudo cp /var/lib/elasticsearch data -rf<br /> <br /> 7. Adjust file permissions<br /> <br /> sudo chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 8. Launch the temporary elasticsearch instance<br /> <br /> ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500<br /> <br /> 9. Verify that the indexes were properly copied. Use the port specified in the previous command:<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 10. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove --purge elasticsearch<br /> sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.5<br /> sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.5<br /> <br /> 11. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 12. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 13. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster http://localhost:9500<br /> &quot;;<br /> <br /> The command should finish with<br /> <br /> All reindex requests ended successfully!<br /> <br /> <br /> 14. Verify that the new indexes were created and populated:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open aips IL7lbAZFR9adKg-GpgepTw 5 1 1 0 135.8kb 135.8kb<br /> yellow open transferfiles hlG1GBwIRNCi1rI74LhQpQ 5 1 23 0 49.1kb 49.1kb<br /> yellow open transfers ebRiOvutR_e7N4dNGoNq4w 5 1 1 0 6kb 6kb<br /> yellow open aipfiles 2QFnLdr7Txi2qgG2C7-eJw 5 1 10 0 158.7kb 158.7kb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.<br /> <br /> 15. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)<br /> <br /> 16. Restart all archivematica services<br /> <br /> service archivematica-dashboard restart<br /> service archivematica-mcp-server restart<br /> service archivematica-mcp-client restart<br /> service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12888 2019-02-19T18:23:29Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> <br /> <br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 11 0 267.9kb 267.9kb <br /> yellow open transfers 5 1 24 0 35.2kb 35.2kb <br /> <br /> 2. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> 3. Check wich elasticsearch version is installed<br /> <br /> Redhat/Centos: rpm -qa | grep elasticsearch<br /> Ubuntu: dpkg -l | grep elasticsearch<br /> <br /> 4. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 5. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 6. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> sudo cp /var/lib/elasticsearch data -rf<br /> <br /> 7. Adjust file permissions<br /> <br /> sudo chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 8. Launch the temporary elasticsearch instance<br /> <br /> ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500<br /> <br /> 9. Verify that the indexes were properly copied<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 10. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove --purge elasticsearch<br /> sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.5<br /> sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.5<br /> <br /> 11. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 12. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 13. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster http://localhost:9500<br /> &quot;;<br /> <br /> The command should finish with<br /> <br /> All reindex requests ended successfully!<br /> <br /> <br /> 14. Verify that the new indexes were created and populated:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open aips IL7lbAZFR9adKg-GpgepTw 5 1 1 0 135.8kb 135.8kb<br /> yellow open transferfiles hlG1GBwIRNCi1rI74LhQpQ 5 1 23 0 49.1kb 49.1kb<br /> yellow open transfers ebRiOvutR_e7N4dNGoNq4w 5 1 1 0 6kb 6kb<br /> yellow open aipfiles 2QFnLdr7Txi2qgG2C7-eJw 5 1 10 0 158.7kb 158.7kb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.<br /> <br /> 15. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)<br /> <br /> 16. Restart all archivematica services<br /> <br /> service archivematica-dashboard restart<br /> service archivematica-mcp-server restart<br /> service archivematica-mcp-client restart<br /> service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12887 2019-02-19T18:19:57Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> <br /> <br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 11 0 267.9kb 267.9kb <br /> yellow open transfers 5 1 24 0 35.2kb 35.2kb <br /> <br /> 2. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> 3. Check wich elasticsearch version is installed<br /> <br /> Redhat/Centos: rpm -qa | grep elasticsearch<br /> Ubuntu: dpkg -l | grep elasticsearch<br /> <br /> 4. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 5. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 6. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> sudo cp /var/lib/elasticsearch data -rf<br /> <br /> 7. Adjust file permissions<br /> <br /> sudo chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 8. Launch the temporary elasticsearch instance<br /> <br /> ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500<br /> <br /> 9. Verify that the indexes were properly copied<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 10. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove elasticsearch<br /> sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.5<br /> sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.5<br /> <br /> 11. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 12. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 13. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster http://localhost:9500<br /> &quot;;<br /> <br /> The command should finish with<br /> <br /> All reindex requests ended successfully!<br /> <br /> <br /> 14. Verify that the new indexes were created and populated:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open aips IL7lbAZFR9adKg-GpgepTw 5 1 1 0 135.8kb 135.8kb<br /> yellow open transferfiles hlG1GBwIRNCi1rI74LhQpQ 5 1 23 0 49.1kb 49.1kb<br /> yellow open transfers ebRiOvutR_e7N4dNGoNq4w 5 1 1 0 6kb 6kb<br /> yellow open aipfiles 2QFnLdr7Txi2qgG2C7-eJw 5 1 10 0 158.7kb 158.7kb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.<br /> <br /> 15. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)<br /> <br /> 16. Restart all archivematica services<br /> <br /> service archivematica-dashboard restart<br /> service archivematica-mcp-server restart<br /> service archivematica-mcp-client restart<br /> service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12886 2019-02-19T17:43:26Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> <br /> <br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 11 0 267.9kb 267.9kb <br /> yellow open transfers 5 1 24 0 35.2kb 35.2kb <br /> <br /> 2. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> 3. Check wich elasticsearch version is installed<br /> <br /> Redhat/Centos: rpm -qa | grep elasticsearch<br /> Ubuntu: dpkg -l | grep elasticsearch<br /> <br /> 4. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 5. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 6. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> cp /var/lib/elasticsearch data -rf<br /> <br /> 7. Adjust file permissions<br /> <br /> chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 8. Launch the temporary elasticsearch instance<br /> <br /> ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500<br /> <br /> 9. Verify that the indexes were properly copied<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 10. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove elasticsearch<br /> sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.5<br /> sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.5<br /> <br /> 11. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 12. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 13. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster http://localhost:9500<br /> &quot;;<br /> <br /> The command should finish with<br /> <br /> All reindex requests ended successfully!<br /> <br /> <br /> 14. Verify that the new indexes were created and populated:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open aips IL7lbAZFR9adKg-GpgepTw 5 1 1 0 135.8kb 135.8kb<br /> yellow open transferfiles hlG1GBwIRNCi1rI74LhQpQ 5 1 23 0 49.1kb 49.1kb<br /> yellow open transfers ebRiOvutR_e7N4dNGoNq4w 5 1 1 0 6kb 6kb<br /> yellow open aipfiles 2QFnLdr7Txi2qgG2C7-eJw 5 1 10 0 158.7kb 158.7kb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.<br /> <br /> 15. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)<br /> <br /> 16. Restart all archivematica services<br /> <br /> service archivematica-dashboard restart<br /> service archivematica-mcp-server restart<br /> service archivematica-mcp-client restart<br /> service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12885 2019-02-19T17:41:48Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> <br /> <br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 11 0 267.9kb 267.9kb <br /> yellow open transfers 5 1 24 0 35.2kb 35.2kb <br /> <br /> 2. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> 3. Check wich elasticsearch version is installed<br /> <br /> Redhat/Centos: rpm -qa | grep elasticsearch<br /> Ubuntu: dpkg -l | grep elasticsearch<br /> <br /> 4. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 5. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 6. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> cp /var/lib/elasticsearch data -rf<br /> <br /> 7. Adjust file permissions<br /> <br /> chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 8. Launch the temporary elasticsearch instance<br /> <br /> ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500<br /> <br /> 9. Verify that the indexes were properly copied<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 10. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove elasticsearch<br /> rm -rf /var/lib/elasticsearch<br /> <br /> 11. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 12. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 13. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster http://localhost:9500<br /> &quot;;<br /> <br /> The command should finish with<br /> <br /> All reindex requests ended successfully!<br /> <br /> <br /> 14. Verify that the new indexes were created and populated:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open aips IL7lbAZFR9adKg-GpgepTw 5 1 1 0 135.8kb 135.8kb<br /> yellow open transferfiles hlG1GBwIRNCi1rI74LhQpQ 5 1 23 0 49.1kb 49.1kb<br /> yellow open transfers ebRiOvutR_e7N4dNGoNq4w 5 1 1 0 6kb 6kb<br /> yellow open aipfiles 2QFnLdr7Txi2qgG2C7-eJw 5 1 10 0 158.7kb 158.7kb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.<br /> <br /> 15. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)<br /> <br /> 16. Restart all archivematica services<br /> <br /> service archivematica-dashboard restart<br /> service archivematica-mcp-server restart<br /> service archivematica-mcp-client restart<br /> service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12884 2019-02-19T16:10:46Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> <br /> <br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 11 0 267.9kb 267.9kb <br /> yellow open transfers 5 1 24 0 35.2kb 35.2kb <br /> <br /> 2. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> <br /> 3. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 4. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 5. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> cp /var/lib/elasticsearch data -rf<br /> <br /> 6. Adjust file permissions<br /> <br /> chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 7. Launch the temporary elasticsearch instance<br /> <br /> ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500<br /> <br /> 8. Verify that the indexes were properly copied<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 9. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove elasticsearch<br /> rm -rf /var/lib/elasticsearch<br /> <br /> 10. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 11. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 12. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster http://localhost:9500<br /> &quot;;<br /> <br /> The command should finish with<br /> <br /> All reindex requests ended successfully!<br /> <br /> <br /> 13. Verify that the new indexes were created and populated:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open aips IL7lbAZFR9adKg-GpgepTw 5 1 1 0 135.8kb 135.8kb<br /> yellow open transferfiles hlG1GBwIRNCi1rI74LhQpQ 5 1 23 0 49.1kb 49.1kb<br /> yellow open transfers ebRiOvutR_e7N4dNGoNq4w 5 1 1 0 6kb 6kb<br /> yellow open aipfiles 2QFnLdr7Txi2qgG2C7-eJw 5 1 10 0 158.7kb 158.7kb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.<br /> <br /> 14. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)<br /> <br /> 15. Restart all archivematica services<br /> <br /> service archivematica-dashboard restart<br /> service archivematica-mcp-server restart<br /> service archivematica-mcp-client restart<br /> service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12883 2019-02-19T16:05:06Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> <br /> <br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 11 0 267.9kb 267.9kb <br /> yellow open transfers 5 1 24 0 35.2kb 35.2kb <br /> <br /> 2. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> <br /> 3. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 4. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 5. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> cp /var/lib/elasticsearch data -rf<br /> <br /> 6. Adjust file permissions<br /> <br /> chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 7. Launch the temporary elasticsearch instance<br /> <br /> ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500<br /> <br /> 8. Verify that the indexes were properly copied<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 9. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove elasticsearch<br /> rm -rf /var/lib/elasticsearch<br /> <br /> 10. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 11. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 12. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster http://localhost:9500<br /> &quot;;<br /> <br /> 13. Verify that the new indexes were created and populated:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open aips IL7lbAZFR9adKg-GpgepTw 5 1 1 0 135.8kb 135.8kb<br /> yellow open transferfiles hlG1GBwIRNCi1rI74LhQpQ 5 1 23 0 49.1kb 49.1kb<br /> yellow open transfers ebRiOvutR_e7N4dNGoNq4w 5 1 1 0 6kb 6kb<br /> yellow open aipfiles 2QFnLdr7Txi2qgG2C7-eJw 5 1 10 0 158.7kb 158.7kb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.<br /> <br /> 14. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)<br /> <br /> 15. Restart all archivematica services<br /> <br /> service archivematica-dashboard restart<br /> service archivematica-mcp-server restart<br /> service archivematica-mcp-client restart<br /> service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12882 2019-02-19T16:04:11Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> <br /> <br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 11 0 267.9kb 267.9kb <br /> yellow open transfers 5 1 24 0 35.2kb 35.2kb <br /> <br /> 2. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> <br /> 3. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 4. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 5. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> cp /var/lib/elasticsearch data -rf<br /> <br /> 6. Adjust file permissions<br /> <br /> chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 7. Launch the temporary elasticsearch instance<br /> <br /> ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500<br /> <br /> 8. Verify that the indexes were properly copied<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 9. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove elasticsearch<br /> rm -rf /var/lib/elasticsearch<br /> <br /> 10. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 11. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 12. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster http://localhost:9500<br /> &quot;;<br /> <br /> 13. Verify that the sum of aips and aipfiles in the new ES index, is the same than in the previous one: <br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot; <br /> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size<br /> yellow open aips IL7lbAZFR9adKg-GpgepTw 5 1 1 0 135.8kb 135.8kb<br /> yellow open transferfiles hlG1GBwIRNCi1rI74LhQpQ 5 1 23 0 49.1kb 49.1kb<br /> yellow open transfers ebRiOvutR_e7N4dNGoNq4w 5 1 1 0 6kb 6kb<br /> yellow open aipfiles 2QFnLdr7Txi2qgG2C7-eJw 5 1 10 0 158.7kb 158.7kb<br /> <br /> You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.<br /> <br /> 14. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)<br /> <br /> 15. Restart all archivematica services<br /> <br /> service archivematica-dashboard restart<br /> service archivematica-mcp-server restart<br /> service archivematica-mcp-client restart<br /> service archivematica-storage-service restart</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12881 2019-02-19T15:59:51Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> <br /> <br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 11 0 267.9kb 267.9kb <br /> yellow open transfers 5 1 24 0 35.2kb 35.2kb <br /> <br /> 2. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> <br /> 3. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 4. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 5. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> cp /var/lib/elasticsearch data -rf<br /> <br /> 6. Adjust file permissions<br /> <br /> chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 7. Launch the temporary elasticsearch instance<br /> <br /> ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500<br /> <br /> 8. Verify that the indexes were properly copied<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 9. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove elasticsearch<br /> rm -rf /var/lib/elasticsearch<br /> <br /> 10. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 11. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 12. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster http://localhost:9500<br /> &quot;;<br /> <br /> 13. Verify that the sum of aips and aipfiles in the new ES index, is the same than in the previous one: <br /> <br /> TBD<br /> <br /> 14. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12880 2019-02-19T15:59:05Z

<p>Scollazo: /* ElasticSearch update */</p> <hr /> <div>= ElasticSearch update =<br /> <br /> <br /> <br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 11 0 267.9kb 267.9kb <br /> yellow open transfers 5 1 24 0 35.2kb 35.2kb <br /> <br /> 2. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> <br /> 3. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 4. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 5. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> cp /var/lib/elasticsearch data -rf<br /> <br /> 6. Adjust file permissions<br /> <br /> chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 7. Launch the temporary elasticsearch instance<br /> <br /> ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500<br /> <br /> 8. Verify that the indexes were properly copied<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 9. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove elasticsearch<br /> rm -rf /var/lib/elasticsearch<br /> <br /> 10. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 11. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 12. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py \ <br /> reindex_from_remote_cluster http://localhost:9500<br /> &quot;;<br /> <br /> 13. Verify that the sum of aips and aipfiles in the new ES index, is the same than in the previous one: <br /> <br /> TBD<br /> <br /> 14. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12879 2019-02-19T15:42:22Z

<p>Scollazo: </p> <hr /> <div>= ElasticSearch update =<br /> <br /> <br /> <br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 11 0 267.9kb 267.9kb <br /> yellow open transfers 5 1 24 0 35.2kb 35.2kb <br /> <br /> 2. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> <br /> 3. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 4. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 5. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> cp /var/lib/elasticsearch data -rf<br /> <br /> 6. Adjust file permissions<br /> <br /> chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 7. Launch the temporary elasticsearch instance<br /> <br /> ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500<br /> <br /> 8. Verify that the indexes were properly copied<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 9. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove elasticsearch<br /> rm -rf /var/lib/elasticsearch<br /> <br /> 10. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 11. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 12. Migrate the indexes<br /> <br /> sudo -u archivematica bash -c &quot; \<br /> set -a -e -x<br /> source /etc/default/archivematica-dashboard || \<br /> source /etc/sysconfig/archivematica-dashboard \<br /> || (echo 'Environment file not found'; exit 1)<br /> cd /usr/share/archivematica/dashboard<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py manage.py \ <br /> reindex_from_remote_cluster http://localhost:9500<br /> &quot;;<br /> <br /> 13. Verify that the sum of aips and aipfiles in the new ES index, is the same than in the previous one: <br /> <br /> TBD<br /> <br /> 14. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12878 2019-02-19T10:44:31Z

<p>Scollazo: </p> <hr /> <div>= ElasticSearch update =<br /> <br /> <br /> <br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 11 0 267.9kb 267.9kb <br /> yellow open transfers 5 1 24 0 35.2kb 35.2kb <br /> <br /> 2. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> <br /> 3. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 4. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 5. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> cp /var/lib/elasticsearch data -rf<br /> <br /> 6. Adjust file permissions<br /> <br /> chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 7. Launch the temporary elasticsearch instance<br /> <br /> ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500<br /> <br /> 8. Verify that the indexes were properly copied<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 9. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove elasticsearch<br /> rm -rf /var/lib/elasticsearch<br /> <br /> 10. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 11. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 12. Migrate the indexes<br /> <br /> sudo su - archivematica -s /bin/bash<br /> export $(cat /etc/sysconfig/archivematica-dashboard)<br /> cd /usr/share/archivematica/dashboar<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster http://localhost:9500<br /> exit<br /> <br /> 13. Stop the temporary elasticsearch service<br /> <br /> kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12877 2019-02-19T10:37:23Z

<p>Scollazo: </p> <hr /> <div>= ElasticSearch update =<br /> <br /> <br /> <br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 11 0 267.9kb 267.9kb <br /> yellow open transfers 5 1 24 0 35.2kb 35.2kb <br /> <br /> 2. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> <br /> 3. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 4. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 5. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> cp /var/lib/elasticsearch data -rf<br /> <br /> 6. Adjust file permissions<br /> <br /> chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 7. Launch the temporary elasticsearch instance<br /> <br /> ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500<br /> <br /> 8. Verify that the indexes were properly copied<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 9. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove elasticsearch<br /> rm -rf /var/lib/elasticsearch<br /> <br /> 10. Upgrade archivematica and install elasticsearch 6<br /> <br /> ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src<br /> <br /> 11. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 12. Migrate the indexes<br /> <br /> sudo su - archivematica -s /bin/bash<br /> export $(cat /etc/sysconfig/archivematica-dashboard)<br /> cd /usr/share/archivematica/dashboar<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster http://localhost:9500<br /> exit<br /> <br /> 13. Stop the temporary elasticsearch service<br /> <br /> kill $(cat elasticsearch-1.7.5/elastic-tmp.pid)</div>

Scollazo https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&diff=12876 2019-02-19T10:27:47Z

<p>Scollazo: Created page with &quot;= ElasticSearch update = 1. Check that ES is up and running, and note the results: curl -X GET &quot;localhost:9200/_cat/indices?v&quot; health status index pri rep docs.co...&quot;</p> <hr /> <div>= ElasticSearch update =<br /> <br /> <br /> <br /> <br /> 1. Check that ES is up and running, and note the results:<br /> <br /> curl -X GET &quot;localhost:9200/_cat/indices?v&quot;<br /> health status index pri rep docs.count docs.deleted store.size pri.store.size <br /> yellow open aips 5 1 11 0 267.9kb 267.9kb <br /> yellow open transfers 5 1 24 0 35.2kb 35.2kb <br /> <br /> 2. Stop elasticsearch service<br /> <br /> sudo service elasticsearch stop<br /> <br /> <br /> 3. Download a temporary copy of elasticsearch (same version than intalled)<br /> <br /> wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz<br /> <br /> <br /> 4. Uncompress the downloaded tar.gz and go into it<br /> <br /> tar zxvf elasticsearch-1.7.5.tar.gz<br /> cd elasticsearch-1.7.5<br /> <br /> <br /> 5. Copy data from system wide installed elasticsearch, ot the temporary copy<br /> <br /> cp /var/lib/elasticsearch data -rf<br /> <br /> 6. Adjust file permissions<br /> <br /> chown &lt;your user&gt;:&lt;your group&gt; data -R<br /> <br /> 7. Launch the temporary elasticsearch instance<br /> <br /> ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500<br /> <br /> 8. Verify that the indexes were properly copied<br /> <br /> curl -X GET &quot;localhost:9500/_cat/indices?v&quot;<br /> <br /> 9. Remove elasticsearch 1.7.5 and it's files<br /> <br /> Redhat: sudo yum remove elasticsearch<br /> Ubuntu: sudo apt-get remove elasticsearch<br /> rm -rf /var/lib/elasticsearch<br /> <br /> 10. Upgrade archivematica and install elasticsearch 6<br /> <br /> 11. Configure ElasticSearch 6 to handle reindex from the temporary es <br /> <br /> echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml<br /> sudo service elasticsearch restart<br /> <br /> 12. Migrate the indexes<br /> <br /> sudo su - archivematica -s /bin/bash<br /> export $(cat /etc/sysconfig/archivematica-dashboard)<br /> cd /usr/share/archivematica/dashboar<br /> /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster http://localhost:9500<br /> exit<br /> <br /> 13. Stop the temporary elasticsearch service<br /> <br /> kill $(cat elasticsearch-1.7.5/elastic-tmp.pid)</div>

Scollazo https://wiki.archivematica.org/index.php?title=Release_Process&diff=12341 2018-03-26T14:40:58Z

<p>Scollazo: /* Debian packages */</p> <hr /> <div>This is an outline/checklist of the process to create Archivematica &amp; Storage service releases.<br /> <br /> == Overview ==<br /> <br /> # Merge new features<br /> # Test new features<br /> # [[#Update PRONOM]]<br /> # Write documentation<br /> # Update dependencies<br /> # [[#Build deb/rpm packages]]<br /> # Test packages for new installs and upgrades<br /> # [[#Tag Release]]<br /> # Update ansible roles<br /> # Announce release<br /> <br /> == Translations ==<br /> <br /> '''Needs to be improved!'''<br /> <br /> * Determine code freeze / call for translations process<br /> * Describe processes: push and pull - and when it needs to happen<br /> * We made a choice on how we're using Transifex to keep things simple: only one branch at a time pushed to Transifex. E.g. once SS 0.10.0 is released we have to decide if:<br /> *# We move Transifex to stable/0.10.x for a while so we can work on a minor release with translation fixes (e.g. 0.10.1), or<br /> *# We move to qa/0.11.x which would only make possible to bring new translations to SS 0.11.0.<br /> * Affected repositories<br /> ** archivematica-storage-service<br /> ** archivematica-workflow<br /> ** archivematica-dashboard<br /> *** Includes archivematica-fpr-admin<br /> *** Includes appraisal-tab<br /> <br /> == Update PRONOM ==<br /> <br /> PRONOM needs to be updated in our file identification tools, FIDO &amp; Siegfried, as well as in the FPR.<br /> <br /> === Update FIDO ===<br /> <br /> The FPR update currently use FIDO as a source for new PRONOM, since it is formatted nicer than PRONOM offers, so we depend on FIDO having updated their PRONOM. If that has not happened, we can generate a new formats-v##.xml by updating signatures manually. Artefactual can also update PRONOM and submit a PR to FIDO.<br /> <br /> # Checkout fido from https://github.com/openpreserve/fido<br /> # Update signatures<br /> #* Run &lt;code&gt;python2 -m fido.update_signatures&lt;/code&gt; from the fido repository root<br /> # Add:<br /> #* New signature file &lt;code&gt;fido/conf/DROID_SignatureFile-v##.xml&lt;/code&gt;<br /> #* New formats file &lt;code&gt;fido/conf/formats-v##.xml&lt;/code&gt;<br /> #* New PRONOM zip file &lt;code&gt;fido/conf/pronom-xml-v##.zip&lt;/code&gt;<br /> #* Updated &lt;code&gt;fido/conf/versions.xml&lt;/code&gt;<br /> # Remove:<br /> #* Old signature file<br /> #* Old formats file<br /> #* Old PRONOM zip file<br /> # Update &lt;code&gt;fido/fido.py&lt;/code&gt; to point to the new format_files<br /> # Create pull request. This may trigger a release from FIDO<br /> <br /> Also consider:<br /> * Update the container signatures (download here: https://www.nationalarchives.gov.uk/aboutapps/pronom/droid-signature-files.htm)<br /> * See this pull request submitted to update to PRONOM 90: https://github.com/openpreserve/fido/pull/110<br /> ** Notice that it also updates the container signatures<br /> * Remember to publish the wheel to pypi! https://pypi.python.org/pypi/opf-fido (see the Makefile)<br /> <br /> === Package FIDO ===<br /> <br /> First, clone the [https://github.com/artefactual-labs/am-packbuild/ am-packbuild] repo. The most recent work has been done in branch dev/packages-1.6-docker, and needs to be merged into main.<br /> <br /> Update the Makefiles available at &lt;code&gt;rpm/fido/Makefile&lt;/code&gt;, and &lt;code&gt;deb/fido/Makefile&lt;/code&gt; , and run &lt;code&gt;make&lt;/code&gt; in each directory to build the packages.<br /> <br /> === Package Siegfried ===<br /> <br /> First, clone the [https://github.com/artefactual-labs/am-packbuild/ am-packbuild] repo. The most recent work has been done in branch dev/packages-1.6-docker, and needs to be merged into main.<br /> <br /> Update the Makefiles available at &lt;code&gt;rpm/siegfried/Makefile&lt;/code&gt;, and &lt;code&gt;deb/siegfried/Makefile&lt;/code&gt; , and run &lt;code&gt;make&lt;/code&gt; in each directory to build the packages.<br /> <br /> === Update FPR ===<br /> <br /> &lt;div class=&quot;text-alert&quot;&gt;<br /> * In https://github.com/artefactual/archivematica-fpr-admin/pull/51, the devtools used here to generate the migration were included in archivematica-fpr-admin as django-admin management tools.<br /> * At the end of the process: '''remember''' to remove &quot;pk&quot;s from the final migration fixture! E.g. see PR #55<br /> * Examples<br /> ** PRONOM 92:<br /> *** https://github.com/artefactual/archivematica-fpr-admin/pull/63<br /> *** ???<br /> ** PRONOM 90:<br /> *** https://github.com/artefactual/archivematica-fpr-admin/pull/51<br /> *** https://github.com/artefactual/archivematica-fpr-admin/pull/55<br /> &lt;/div&gt;<br /> <br /> This process is currently more convoluted than it needs to be, and should be simplified.<br /> <br /> This depends on FIDO having updated their PRONOM files. See [[#Update FIDO]]<br /> <br /> # Install the latest stable Archivematica release<br /> # Generate a JSON with the current version of the FPR (for use later)<br /> #* &lt;code&gt;./src/dashboard/src/manage.py dumpdata fpr &gt; fpr-current.json&lt;/code&gt;<br /> # Install the latest development version of Archivematica<br /> # Generate a blank data migration in the FPR-admin module<br /> #* E.g. &lt;code&gt;./src/dashboard/src/manage.py makemigrations --empty --name pronom_89 fpr&lt;/code&gt;<br /> # Generate the FPR migration body. This also updates the local database's FPR with the new PRONOM IDs<br /> #* E.g. &lt;code&gt;am import-pronom-ids fido/fido/conf/formats-v89.xml --output-filename archivematica/pronom89.txt&lt;/code&gt;<br /> # Copy the output into the blank migration, add RunPython operation<br /> #* &lt;code&gt;migrations.RunPython(data_migration),&lt;/code&gt;<br /> # Due to a [https://projects.artefactual.com/issues/10466#change-46673 bug], fix the imports<br /> #* Remove the &lt;code&gt;apps.get_model&lt;/code&gt; lines<br /> #* Import the models directly with &lt;code&gt;from fpr.models import Format, FormatVersion, IDRule&lt;/code&gt;<br /> # Commit<br /> # Deploy on testing pipeline<br /> # (Analysts) Update the new entries<br /> #* '''Ask to edit ONLY entries added by the latest PRONOM update otherwise the fixture won't work properly'''<br /> #* Move new formats to a more appropriate category<br /> #* Create rules &amp; commands<br /> #* Test with data for new formats<br /> # Generate a JSON with the updated version of the FPR on the testing pipeline<br /> #* &lt;code&gt;./src/dashboard/src/manage.py dumpdata fpr &gt; fpr-updated.json&lt;/code&gt;<br /> # Get the updates as JSON<br /> #* E.g. &lt;code&gt;am get-fpr-changes fpr-current.json fpr-updated.json src/dashboard/src/fpr/migrations/pronom_89.json&lt;/code&gt;<br /> # Update the migration to load the JSON updates<br /> #* Replace the contents of the data migration function with<br /> #** &lt;code&gt;fixture_file = os.path.join(os.path.dirname(__file__), 'pronom_89.json')&lt;/code&gt;<br /> #** &lt;code&gt;call_command('loaddata', fixture_file, app_label='fpr')&lt;/code&gt;<br /> #* Remove the direct imports from the bug<br /> #* '''Remove the pk's from the entries in the JSON document.'''<br /> #* Improvement Note: Because this is using loaddata, this will have problems if the FPR models are changed. A possible solution is to update get-fpr-changes to generate a migration instead of JSON<br /> # Commit<br /> # Merge<br /> <br /> == Update dependencies ==<br /> <br /> === Python Packages ===<br /> <br /> [https://github.com/artefactual-labs/mets-reader-writer metsrw] and [https://github.com/artefactual-labs/agentarchives agentarchives] both have Makefiles that handle most of the packaging<br /> <br /> # Check for open PRs, merge as necessary<br /> # Update &lt;code&gt;setup.py&lt;/code&gt; with the new version, create a pull request, code review, [[Merging | merge]].<br /> # Tag new release, push tag<br /> #* &lt;code&gt;git push --tags&lt;/code&gt;<br /> # Run &lt;code&gt;make package&lt;/code&gt;<br /> #* This will build the package and upload it to PyPI. It will prompt for your PyPI username and password for the upload<br /> #* The Makefile was created from instructions at https://packaging.python.org/distributing/<br /> # &lt;code&gt;make clean&lt;/code&gt; will delete packaging related files<br /> <br /> == Build deb/rpm packages ==<br /> <br /> The am-packbuild repository has all the code related to building packages, except the building gpg keys. The steps to follow in order to build production production packages are as follow<br /> <br /> === Debian packages ===<br /> <br /> # Clone the [https://github.com/artefactual-labs/am-packbuild am-packbuild] repo. Latest work is available in master<br /> # Put your gpg private key into &lt;code&gt;debs/GPG-KEYS-REPOS&lt;/code&gt;. That's the place the Dockerfile looks for it when building the environment.<br /> # Update the makefile at &lt;code&gt;debs/archivematica/Makefile&lt;/code&gt; in order to reflect version/keys you want to use.<br /> # Run &lt;make&gt;, and the packages will be available in the &lt;code&gt;build&lt;/code&gt; once the building finishes.<br /> # Upload packages to public debian repository<br /> <br /> === Debian reposities ===<br /> <br /> There are two debian repositories, one for archivematica packages, and one for dependencies. The procedure in order to create new ones, or upload packages to them, is the same:<br /> <br /> #* Create folder for repo, and configuration file:<br /> &lt;code&gt;<br /> mkdir -p /path/to/repos/repo/conf<br /> <br /> cat &gt; /path/to/repos/repo/conf/distributions &lt;&lt; EOF<br /> <br /> Codename: trusty<br /> <br /> Components: main<br /> <br /> Architectures: amd64 source<br /> <br /> SignWith: &lt;short gpg keyid&gt; <br /> <br /> EOF<br /> &lt;/code&gt;<br /> <br /> #* Go inside the repo, and import the packages previously uploaded with:<br /> &lt;code&gt;<br /> cd /path/to/repos/repo/<br /> <br /> reprepro includedeb trusty /path/to/packages/*.deb<br /> <br /> reprepro includedsc trusty /path/to/packages/*.deb<br /> <br /> &lt;/code&gt;<br /> <br /> The current official repo is at packages.archivematica.org<br /> <br /> === RPM Packages ===<br /> <br /> # Package specs are available in [https://github.com/artefactual-labs/am-packbuild/tree/dev/packaging-1.6-docker/rpm am-packbuild/rpms]<br /> # There are vars in the Makefiles for version/release , so update them when needed<br /> # In order to build them, just go into the directory you want to build, and run “make”<br /> <br /> === RPM Repositories ===<br /> <br /> Once the packages are built, upload them to packages.archivematica.org/&lt;version&gt;/centos<br /> Sign the packages with rpm --addsign *.rpm (already signed packages will be skipped)<br /> <br /> Go inside that dir, and as user ohbot run:<br /> <br /> * ''rpm --addsign *.rpm'' (already signed packages will be skipped)<br /> <br /> * ''createrepo .'' (For packages other than archivematica , use “centos-extras” repository)<br /> <br /> * ''gpg --detach-sign --armor repodata/repomd.xml'' <br /> <br /> The first gpg command signs the rpms, and the later signs the repository content.<br /> <br /> === Development stage ===<br /> <br /> In the final stages of development, the repositories for the new releases are created, but packages are signed with a development key to avoid mistakes. Once the development stage finishes, all new packages need to be rebuild using the production keys.<br /> <br /> Development packages are built on each new commit to stable branches by Jenkins. Repositories are available at http://jenkins-ci.archivematica.org/repos/<br /> <br /> == Website ==<br /> <br /> '''Needs to be improved!'''<br /> <br /> ==== Homepage ====<br /> <br /> * Make changes in archivematica-web.git<br /> ** Update links<br /> ** Add new doc repos<br /> * Deploy<br /> ** Log in sites-pub as archivematica-web and run update-web.sh<br /> <br /> ==== Documentation ====<br /> <br /> * Deploy<br /> ** Log in sites-pub as archivematica-web and run update-docs.sh<br /> <br /> ==== Wiki ====<br /> <br /> * Release notes<br /> * Installation notes<br /> * ...<br /> <br /> == News ==<br /> <br /> * Twitter<br /> * Mailing list<br /> * News section in artefactual.com<br /> <br /> == Update ansible roles ==<br /> <br /> Check that the deploy-pub vars files for archivematica are updated, and the ansible-ansiblematica-src and ansible-archivematica-pkg roles are able to deploy the new version<br /> <br /> == Tag Release ==<br /> <br /> # Add release tags to repositories<br /> #* Archivematica<br /> #* Storage Service<br /> #* FPR-admin<br /> #* appraisal tab<br /> #* transfer browser<br /> #* Others?<br /> # Create stable/#.x branch<br /> # Delete qa/#.x branch if necessary<br /> <br /> [[Category:Process documentation]]</div>

Scollazo https://wiki.archivematica.org/index.php?title=Release_Process&diff=12340 2018-03-26T14:39:40Z

<p>Scollazo: /* Debian packages */</p> <hr /> <div>This is an outline/checklist of the process to create Archivematica &amp; Storage service releases.<br /> <br /> == Overview ==<br /> <br /> # Merge new features<br /> # Test new features<br /> # [[#Update PRONOM]]<br /> # Write documentation<br /> # Update dependencies<br /> # [[#Build deb/rpm packages]]<br /> # Test packages for new installs and upgrades<br /> # [[#Tag Release]]<br /> # Update ansible roles<br /> # Announce release<br /> <br /> == Translations ==<br /> <br /> '''Needs to be improved!'''<br /> <br /> * Determine code freeze / call for translations process<br /> * Describe processes: push and pull - and when it needs to happen<br /> * We made a choice on how we're using Transifex to keep things simple: only one branch at a time pushed to Transifex. E.g. once SS 0.10.0 is released we have to decide if:<br /> *# We move Transifex to stable/0.10.x for a while so we can work on a minor release with translation fixes (e.g. 0.10.1), or<br /> *# We move to qa/0.11.x which would only make possible to bring new translations to SS 0.11.0.<br /> * Affected repositories<br /> ** archivematica-storage-service<br /> ** archivematica-workflow<br /> ** archivematica-dashboard<br /> *** Includes archivematica-fpr-admin<br /> *** Includes appraisal-tab<br /> <br /> == Update PRONOM ==<br /> <br /> PRONOM needs to be updated in our file identification tools, FIDO &amp; Siegfried, as well as in the FPR.<br /> <br /> === Update FIDO ===<br /> <br /> The FPR update currently use FIDO as a source for new PRONOM, since it is formatted nicer than PRONOM offers, so we depend on FIDO having updated their PRONOM. If that has not happened, we can generate a new formats-v##.xml by updating signatures manually. Artefactual can also update PRONOM and submit a PR to FIDO.<br /> <br /> # Checkout fido from https://github.com/openpreserve/fido<br /> # Update signatures<br /> #* Run &lt;code&gt;python2 -m fido.update_signatures&lt;/code&gt; from the fido repository root<br /> # Add:<br /> #* New signature file &lt;code&gt;fido/conf/DROID_SignatureFile-v##.xml&lt;/code&gt;<br /> #* New formats file &lt;code&gt;fido/conf/formats-v##.xml&lt;/code&gt;<br /> #* New PRONOM zip file &lt;code&gt;fido/conf/pronom-xml-v##.zip&lt;/code&gt;<br /> #* Updated &lt;code&gt;fido/conf/versions.xml&lt;/code&gt;<br /> # Remove:<br /> #* Old signature file<br /> #* Old formats file<br /> #* Old PRONOM zip file<br /> # Update &lt;code&gt;fido/fido.py&lt;/code&gt; to point to the new format_files<br /> # Create pull request. This may trigger a release from FIDO<br /> <br /> Also consider:<br /> * Update the container signatures (download here: https://www.nationalarchives.gov.uk/aboutapps/pronom/droid-signature-files.htm)<br /> * See this pull request submitted to update to PRONOM 90: https://github.com/openpreserve/fido/pull/110<br /> ** Notice that it also updates the container signatures<br /> * Remember to publish the wheel to pypi! https://pypi.python.org/pypi/opf-fido (see the Makefile)<br /> <br /> === Package FIDO ===<br /> <br /> First, clone the [https://github.com/artefactual-labs/am-packbuild/ am-packbuild] repo. The most recent work has been done in branch dev/packages-1.6-docker, and needs to be merged into main.<br /> <br /> Update the Makefiles available at &lt;code&gt;rpm/fido/Makefile&lt;/code&gt;, and &lt;code&gt;deb/fido/Makefile&lt;/code&gt; , and run &lt;code&gt;make&lt;/code&gt; in each directory to build the packages.<br /> <br /> === Package Siegfried ===<br /> <br /> First, clone the [https://github.com/artefactual-labs/am-packbuild/ am-packbuild] repo. The most recent work has been done in branch dev/packages-1.6-docker, and needs to be merged into main.<br /> <br /> Update the Makefiles available at &lt;code&gt;rpm/siegfried/Makefile&lt;/code&gt;, and &lt;code&gt;deb/siegfried/Makefile&lt;/code&gt; , and run &lt;code&gt;make&lt;/code&gt; in each directory to build the packages.<br /> <br /> === Update FPR ===<br /> <br /> &lt;div class=&quot;text-alert&quot;&gt;<br /> * In https://github.com/artefactual/archivematica-fpr-admin/pull/51, the devtools used here to generate the migration were included in archivematica-fpr-admin as django-admin management tools.<br /> * At the end of the process: '''remember''' to remove &quot;pk&quot;s from the final migration fixture! E.g. see PR #55<br /> * Examples<br /> ** PRONOM 92:<br /> *** https://github.com/artefactual/archivematica-fpr-admin/pull/63<br /> *** ???<br /> ** PRONOM 90:<br /> *** https://github.com/artefactual/archivematica-fpr-admin/pull/51<br /> *** https://github.com/artefactual/archivematica-fpr-admin/pull/55<br /> &lt;/div&gt;<br /> <br /> This process is currently more convoluted than it needs to be, and should be simplified.<br /> <br /> This depends on FIDO having updated their PRONOM files. See [[#Update FIDO]]<br /> <br /> # Install the latest stable Archivematica release<br /> # Generate a JSON with the current version of the FPR (for use later)<br /> #* &lt;code&gt;./src/dashboard/src/manage.py dumpdata fpr &gt; fpr-current.json&lt;/code&gt;<br /> # Install the latest development version of Archivematica<br /> # Generate a blank data migration in the FPR-admin module<br /> #* E.g. &lt;code&gt;./src/dashboard/src/manage.py makemigrations --empty --name pronom_89 fpr&lt;/code&gt;<br /> # Generate the FPR migration body. This also updates the local database's FPR with the new PRONOM IDs<br /> #* E.g. &lt;code&gt;am import-pronom-ids fido/fido/conf/formats-v89.xml --output-filename archivematica/pronom89.txt&lt;/code&gt;<br /> # Copy the output into the blank migration, add RunPython operation<br /> #* &lt;code&gt;migrations.RunPython(data_migration),&lt;/code&gt;<br /> # Due to a [https://projects.artefactual.com/issues/10466#change-46673 bug], fix the imports<br /> #* Remove the &lt;code&gt;apps.get_model&lt;/code&gt; lines<br /> #* Import the models directly with &lt;code&gt;from fpr.models import Format, FormatVersion, IDRule&lt;/code&gt;<br /> # Commit<br /> # Deploy on testing pipeline<br /> # (Analysts) Update the new entries<br /> #* '''Ask to edit ONLY entries added by the latest PRONOM update otherwise the fixture won't work properly'''<br /> #* Move new formats to a more appropriate category<br /> #* Create rules &amp; commands<br /> #* Test with data for new formats<br /> # Generate a JSON with the updated version of the FPR on the testing pipeline<br /> #* &lt;code&gt;./src/dashboard/src/manage.py dumpdata fpr &gt; fpr-updated.json&lt;/code&gt;<br /> # Get the updates as JSON<br /> #* E.g. &lt;code&gt;am get-fpr-changes fpr-current.json fpr-updated.json src/dashboard/src/fpr/migrations/pronom_89.json&lt;/code&gt;<br /> # Update the migration to load the JSON updates<br /> #* Replace the contents of the data migration function with<br /> #** &lt;code&gt;fixture_file = os.path.join(os.path.dirname(__file__), 'pronom_89.json')&lt;/code&gt;<br /> #** &lt;code&gt;call_command('loaddata', fixture_file, app_label='fpr')&lt;/code&gt;<br /> #* Remove the direct imports from the bug<br /> #* '''Remove the pk's from the entries in the JSON document.'''<br /> #* Improvement Note: Because this is using loaddata, this will have problems if the FPR models are changed. A possible solution is to update get-fpr-changes to generate a migration instead of JSON<br /> # Commit<br /> # Merge<br /> <br /> == Update dependencies ==<br /> <br /> === Python Packages ===<br /> <br /> [https://github.com/artefactual-labs/mets-reader-writer metsrw] and [https://github.com/artefactual-labs/agentarchives agentarchives] both have Makefiles that handle most of the packaging<br /> <br /> # Check for open PRs, merge as necessary<br /> # Update &lt;code&gt;setup.py&lt;/code&gt; with the new version, create a pull request, code review, [[Merging | merge]].<br /> # Tag new release, push tag<br /> #* &lt;code&gt;git push --tags&lt;/code&gt;<br /> # Run &lt;code&gt;make package&lt;/code&gt;<br /> #* This will build the package and upload it to PyPI. It will prompt for your PyPI username and password for the upload<br /> #* The Makefile was created from instructions at https://packaging.python.org/distributing/<br /> # &lt;code&gt;make clean&lt;/code&gt; will delete packaging related files<br /> <br /> == Build deb/rpm packages ==<br /> <br /> The am-packbuild repository has all the code related to building packages, except the building gpg keys. The steps to follow in order to build production production packages are as follow<br /> <br /> === Debian packages ===<br /> <br /> # Clone the [https://github.com/artefactual-labs/am-packbuild am-packbuild] repo. Latest work is available in master<br /> # Put your gpg private key into &lt;code&gt;debs/GPG-KEYS-REPOS&lt;/code&gt;. That's the place the Dockerfile looks for it when building the environment.<br /> # Update the makefile at &lt;code&gt;debs/archivematica/Makefile&lt;/code&gt; in order to reflect version/keys you want to use. If you want to build the storage service, you need also to change the packbuild.py line to reflect that (TODO: create service-storage package folder)<br /> This dockerfile uses packbuild.py script, whose function is downloadig the code from github, creating a debian/changelog file for each ubuntu version with the expected version name and description of last commit included, and build the package aftewards. It can be run on their own too, without the need of a docker environment.<br /> # Run &lt;make&gt;, and the packages will be available in the &lt;code&gt;build&lt;/code&gt; once the building finishes.<br /> # Upload packages to public debian repository<br /> <br /> === Debian reposities ===<br /> <br /> There are two debian repositories, one for archivematica packages, and one for dependencies. The procedure in order to create new ones, or upload packages to them, is the same:<br /> <br /> #* Create folder for repo, and configuration file:<br /> &lt;code&gt;<br /> mkdir -p /path/to/repos/repo/conf<br /> <br /> cat &gt; /path/to/repos/repo/conf/distributions &lt;&lt; EOF<br /> <br /> Codename: trusty<br /> <br /> Components: main<br /> <br /> Architectures: amd64 source<br /> <br /> SignWith: &lt;short gpg keyid&gt; <br /> <br /> EOF<br /> &lt;/code&gt;<br /> <br /> #* Go inside the repo, and import the packages previously uploaded with:<br /> &lt;code&gt;<br /> cd /path/to/repos/repo/<br /> <br /> reprepro includedeb trusty /path/to/packages/*.deb<br /> <br /> reprepro includedsc trusty /path/to/packages/*.deb<br /> <br /> &lt;/code&gt;<br /> <br /> The current official repo is at packages.archivematica.org<br /> <br /> === RPM Packages ===<br /> <br /> # Package specs are available in [https://github.com/artefactual-labs/am-packbuild/tree/dev/packaging-1.6-docker/rpm am-packbuild/rpms]<br /> # There are vars in the Makefiles for version/release , so update them when needed<br /> # In order to build them, just go into the directory you want to build, and run “make”<br /> <br /> === RPM Repositories ===<br /> <br /> Once the packages are built, upload them to packages.archivematica.org/&lt;version&gt;/centos<br /> Sign the packages with rpm --addsign *.rpm (already signed packages will be skipped)<br /> <br /> Go inside that dir, and as user ohbot run:<br /> <br /> * ''rpm --addsign *.rpm'' (already signed packages will be skipped)<br /> <br /> * ''createrepo .'' (For packages other than archivematica , use “centos-extras” repository)<br /> <br /> * ''gpg --detach-sign --armor repodata/repomd.xml'' <br /> <br /> The first gpg command signs the rpms, and the later signs the repository content.<br /> <br /> === Development stage ===<br /> <br /> In the final stages of development, the repositories for the new releases are created, but packages are signed with a development key to avoid mistakes. Once the development stage finishes, all new packages need to be rebuild using the production keys.<br /> <br /> Development packages are built on each new commit to stable branches by Jenkins. Repositories are available at http://jenkins-ci.archivematica.org/repos/<br /> <br /> == Website ==<br /> <br /> '''Needs to be improved!'''<br /> <br /> ==== Homepage ====<br /> <br /> * Make changes in archivematica-web.git<br /> ** Update links<br /> ** Add new doc repos<br /> * Deploy<br /> ** Log in sites-pub as archivematica-web and run update-web.sh<br /> <br /> ==== Documentation ====<br /> <br /> * Deploy<br /> ** Log in sites-pub as archivematica-web and run update-docs.sh<br /> <br /> ==== Wiki ====<br /> <br /> * Release notes<br /> * Installation notes<br /> * ...<br /> <br /> == News ==<br /> <br /> * Twitter<br /> * Mailing list<br /> * News section in artefactual.com<br /> <br /> == Update ansible roles ==<br /> <br /> Check that the deploy-pub vars files for archivematica are updated, and the ansible-ansiblematica-src and ansible-archivematica-pkg roles are able to deploy the new version<br /> <br /> == Tag Release ==<br /> <br /> # Add release tags to repositories<br /> #* Archivematica<br /> #* Storage Service<br /> #* FPR-admin<br /> #* appraisal tab<br /> #* transfer browser<br /> #* Others?<br /> # Create stable/#.x branch<br /> # Delete qa/#.x branch if necessary<br /> <br /> [[Category:Process documentation]]</div>

Scollazo https://wiki.archivematica.org/index.php?title=Release_Process&diff=12289 2018-03-09T12:00:23Z

<p>Scollazo: /* RPM Repositories */</p> <hr /> <div>This is an outline/checklist of the process to create Archivematica &amp; Storage service releases.<br /> <br /> == Overview ==<br /> <br /> # Merge new features<br /> # Test new features<br /> # [[#Update PRONOM]]<br /> # Write documentation<br /> # Update dependencies<br /> # [[#Build deb/rpm packages]]<br /> # Test packages for new installs and upgrades<br /> # [[#Tag Release]]<br /> # Update ansible roles<br /> # Announce release<br /> <br /> == Translations ==<br /> <br /> '''Needs to be improved!'''<br /> <br /> * Determine code freeze / call for translations process<br /> * Describe processes: push and pull - and when it needs to happen<br /> * We made a choice on how we're using Transifex to keep things simple: only one branch at a time pushed to Transifex. E.g. once SS 0.10.0 is released we have to decide if:<br /> *# We move Transifex to stable/0.10.x for a while so we can work on a minor release with translation fixes (e.g. 0.10.1), or<br /> *# We move to qa/0.11.x which would only make possible to bring new translations to SS 0.11.0.<br /> * Affected repositories<br /> ** archivematica-storage-service<br /> ** archivematica-workflow<br /> ** archivematica-dashboard<br /> *** Includes archivematica-fpr-admin<br /> *** Includes appraisal-tab<br /> <br /> == Update PRONOM ==<br /> <br /> PRONOM needs to be updated in our file identification tools, FIDO &amp; Siegfried, as well as in the FPR.<br /> <br /> === Update FIDO ===<br /> <br /> The FPR update currently use FIDO as a source for new PRONOM, since it is formatted nicer than PRONOM offers, so we depend on FIDO having updated their PRONOM. If that has not happened, we can generate a new formats-v##.xml by updating signatures manually. Artefactual can also update PRONOM and submit a PR to FIDO.<br /> <br /> # Checkout fido from https://github.com/openpreserve/fido<br /> # Update signatures<br /> #* Run &lt;code&gt;python2 -m fido.update_signatures&lt;/code&gt; from the fido repository root<br /> # Add:<br /> #* New signature file &lt;code&gt;fido/conf/DROID_SignatureFile-v##.xml&lt;/code&gt;<br /> #* New formats file &lt;code&gt;fido/conf/formats-v##.xml&lt;/code&gt;<br /> #* New PRONOM zip file &lt;code&gt;fido/conf/pronom-xml-v##.zip&lt;/code&gt;<br /> #* Updated &lt;code&gt;fido/conf/versions.xml&lt;/code&gt;<br /> # Remove:<br /> #* Old signature file<br /> #* Old formats file<br /> #* Old PRONOM zip file<br /> # Update &lt;code&gt;fido/fido.py&lt;/code&gt; to point to the new format_files<br /> # Create pull request. This may trigger a release from FIDO<br /> <br /> Also consider:<br /> * Update the container signatures (download here: https://www.nationalarchives.gov.uk/aboutapps/pronom/droid-signature-files.htm)<br /> * See this pull request submitted to update to PRONOM 90: https://github.com/openpreserve/fido/pull/110<br /> ** Notice that it also updates the container signatures<br /> * Remember to publish the wheel to pypi! https://pypi.python.org/pypi/opf-fido (see the Makefile)<br /> <br /> === Package FIDO ===<br /> <br /> First, clone the [https://github.com/artefactual-labs/am-packbuild/ am-packbuild] repo. The most recent work has been done in branch dev/packages-1.6-docker, and needs to be merged into main.<br /> <br /> Update the Makefiles available at &lt;code&gt;rpm/fido/Makefile&lt;/code&gt;, and &lt;code&gt;deb/fido/Makefile&lt;/code&gt; , and run &lt;code&gt;make&lt;/code&gt; in each directory to build the packages.<br /> <br /> === Package Siegfried ===<br /> <br /> First, clone the [https://github.com/artefactual-labs/am-packbuild/ am-packbuild] repo. The most recent work has been done in branch dev/packages-1.6-docker, and needs to be merged into main.<br /> <br /> Update the Makefiles available at &lt;code&gt;rpm/siegfried/Makefile&lt;/code&gt;, and &lt;code&gt;deb/siegfried/Makefile&lt;/code&gt; , and run &lt;code&gt;make&lt;/code&gt; in each directory to build the packages.<br /> <br /> === Update FPR ===<br /> <br /> &lt;div class=&quot;text-alert&quot;&gt;<br /> * In https://github.com/artefactual/archivematica-fpr-admin/pull/51, the devtools used here to generate the migration were included in archivematica-fpr-admin as django-admin management tools.<br /> * At the end of the process: '''remember''' to remove &quot;pk&quot;s from the final migration fixture! E.g. see PR #55<br /> * Examples<br /> ** PRONOM 92:<br /> *** https://github.com/artefactual/archivematica-fpr-admin/pull/63<br /> *** ???<br /> ** PRONOM 90:<br /> *** https://github.com/artefactual/archivematica-fpr-admin/pull/51<br /> *** https://github.com/artefactual/archivematica-fpr-admin/pull/55<br /> &lt;/div&gt;<br /> <br /> This process is currently more convoluted than it needs to be, and should be simplified.<br /> <br /> This depends on FIDO having updated their PRONOM files. See [[#Update FIDO]]<br /> <br /> # Install the latest stable Archivematica release<br /> # Generate a JSON with the current version of the FPR (for use later)<br /> #* &lt;code&gt;./src/dashboard/src/manage.py dumpdata fpr &gt; fpr-current.json&lt;/code&gt;<br /> # Install the latest development version of Archivematica<br /> # Generate a blank data migration in the FPR-admin module<br /> #* E.g. &lt;code&gt;./src/dashboard/src/manage.py makemigrations --empty --name pronom_89 fpr&lt;/code&gt;<br /> # Generate the FPR migration body. This also updates the local database's FPR with the new PRONOM IDs<br /> #* E.g. &lt;code&gt;am import-pronom-ids fido/fido/conf/formats-v89.xml --output-filename archivematica/pronom89.txt&lt;/code&gt;<br /> # Copy the output into the blank migration, add RunPython operation<br /> #* &lt;code&gt;migrations.RunPython(data_migration),&lt;/code&gt;<br /> # Due to a [https://projects.artefactual.com/issues/10466#change-46673 bug], fix the imports<br /> #* Remove the &lt;code&gt;apps.get_model&lt;/code&gt; lines<br /> #* Import the models directly with &lt;code&gt;from fpr.models import Format, FormatVersion, IDRule&lt;/code&gt;<br /> # Commit<br /> # Deploy on testing pipeline<br /> # (Analysts) Update the new entries<br /> #* '''Ask to edit ONLY entries added by the latest PRONOM update otherwise the fixture won't work properly'''<br /> #* Move new formats to a more appropriate category<br /> #* Create rules &amp; commands<br /> #* Test with data for new formats<br /> # Generate a JSON with the updated version of the FPR on the testing pipeline<br /> #* &lt;code&gt;./src/dashboard/src/manage.py dumpdata fpr &gt; fpr-updated.json&lt;/code&gt;<br /> # Get the updates as JSON<br /> #* E.g. &lt;code&gt;am get-fpr-changes fpr-current.json fpr-updated.json src/dashboard/src/fpr/migrations/pronom_89.json&lt;/code&gt;<br /> # Update the migration to load the JSON updates<br /> #* Replace the contents of the data migration function with<br /> #** &lt;code&gt;fixture_file = os.path.join(os.path.dirname(__file__), 'pronom_89.json')&lt;/code&gt;<br /> #** &lt;code&gt;call_command('loaddata', fixture_file, app_label='fpr')&lt;/code&gt;<br /> #* Remove the direct imports from the bug<br /> #* '''Remove the pk's from the entries in the JSON document.'''<br /> #* Improvement Note: Because this is using loaddata, this will have problems if the FPR models are changed. A possible solution is to update get-fpr-changes to generate a migration instead of JSON<br /> # Commit<br /> # Merge<br /> <br /> == Update dependencies ==<br /> <br /> === Python Packages ===<br /> <br /> [https://github.com/artefactual-labs/mets-reader-writer metsrw] and [https://github.com/artefactual-labs/agentarchives agentarchives] both have Makefiles that handle most of the packaging<br /> <br /> # Check for open PRs, merge as necessary<br /> # Update &lt;code&gt;setup.py&lt;/code&gt; with the new version, create a pull request, code review, [[Merging | merge]].<br /> # Tag new release, push tag<br /> #* &lt;code&gt;git push --tags&lt;/code&gt;<br /> # Run &lt;code&gt;make package&lt;/code&gt;<br /> #* This will build the package and upload it to PyPI. It will prompt for your PyPI username and password for the upload<br /> #* The Makefile was created from instructions at https://packaging.python.org/distributing/<br /> # &lt;code&gt;make clean&lt;/code&gt; will delete packaging related files<br /> <br /> == Build deb/rpm packages ==<br /> <br /> The am-packbuild repository has all the code related to building packages, except the building gpg keys. The steps to follow in order to build production production packages are as follow<br /> <br /> === Debian packages ===<br /> <br /> # Clone the [https://github.com/artefactual-labs/am-packbuild am-packbuild] repo. Latest work is available in dev/packages-1.6-docker<br /> # Put your gpg private key into &lt;code&gt;debs/GPG-KEYS-REPOS&lt;/code&gt;. That's the place the Dockerfile looks for it when building the environment.<br /> # Update the makefile at &lt;code&gt;debs/archivematica/Makefile&lt;/code&gt; in order to reflect version/keys you want to use. If you want to build the storage service, you need also to change the packbuild.py line to reflect that (TODO: create service-storage package folder)<br /> This dockerfile uses packbuild.py script, whose function is downloadig the code from github, creating a debian/changelog file for each ubuntu version with the expected version name and description of last commit included, and build the package aftewards. It can be run on their own too, without the need of a docker environment.<br /> # Run &lt;make&gt;, and the packages will be available in the &lt;code&gt;build&lt;/code&gt; once the building finishes.<br /> # Upload packages to public debian repository<br /> <br /> === Debian reposities ===<br /> <br /> There are two debian repositories, one for archivematica packages, and one for dependencies. The procedure in order to create new ones, or upload packages to them, is the same:<br /> <br /> #* Create folder for repo, and configuration file:<br /> &lt;code&gt;<br /> mkdir -p /path/to/repos/repo/conf<br /> <br /> cat &gt; /path/to/repos/repo/conf/distributions &lt;&lt; EOF<br /> <br /> Codename: trusty<br /> <br /> Components: main<br /> <br /> Architectures: amd64 source<br /> <br /> SignWith: &lt;short gpg keyid&gt; <br /> <br /> EOF<br /> &lt;/code&gt;<br /> <br /> #* Go inside the repo, and import the packages previously uploaded with:<br /> &lt;code&gt;<br /> cd /path/to/repos/repo/<br /> <br /> reprepro includedeb trusty /path/to/packages/*.deb<br /> <br /> reprepro includedsc trusty /path/to/packages/*.deb<br /> <br /> &lt;/code&gt;<br /> <br /> The current official repo is at packages.archivematica.org<br /> <br /> === RPM Packages ===<br /> <br /> # Package specs are available in [https://github.com/artefactual-labs/am-packbuild/tree/dev/packaging-1.6-docker/rpm am-packbuild/rpms]<br /> # There are vars in the Makefiles for version/release , so update them when needed<br /> # In order to build them, just go into the directory you want to build, and run “make”<br /> <br /> === RPM Repositories ===<br /> <br /> Once the packages are built, upload them to packages.archivematica.org/&lt;version&gt;/centos<br /> Sign the packages with rpm --addsign *.rpm (already signed packages will be skipped)<br /> <br /> Go inside that dir, and as user ohbot run:<br /> <br /> * ''rpm --addsign *.rpm'' (already signed packages will be skipped)<br /> <br /> * ''createrepo .'' (For packages other than archivematica , use “centos-extras” repository)<br /> <br /> * ''gpg --detach-sign --armor repodata/repomd.xml'' <br /> <br /> The first gpg command signs the rpms, and the later signs the repository content.<br /> <br /> === Development stage ===<br /> <br /> In the final stages of development, the repositories for the new releases are created, but packages are signed with a development key to avoid mistakes. Once the development stage finishes, all new packages need to be rebuild using the production keys.<br /> <br /> Development packages are built on each new commit to stable branches by Jenkins. Repositories are available at http://jenkins-ci.archivematica.org/repos/<br /> <br /> == Website ==<br /> <br /> '''Needs to be improved!'''<br /> <br /> ==== Homepage ====<br /> <br /> * Make changes in archivematica-web.git<br /> ** Update links<br /> ** Add new doc repos<br /> * Deploy<br /> ** Log in sites-pub as archivematica-web and run update-web.sh<br /> <br /> ==== Documentation ====<br /> <br /> * Deploy<br /> ** Log in sites-pub as archivematica-web and run update-docs.sh<br /> <br /> ==== Wiki ====<br /> <br /> * Release notes<br /> * Installation notes<br /> * ...<br /> <br /> == News ==<br /> <br /> * Twitter<br /> * Mailing list<br /> * News section in artefactual.com<br /> <br /> == Update ansible roles ==<br /> <br /> Check that the deploy-pub vars files for archivematica are updated, and the ansible-ansiblematica-src and ansible-archivematica-pkg roles are able to deploy the new version<br /> <br /> == Tag Release ==<br /> <br /> # Add release tags to repositories<br /> #* Archivematica<br /> #* Storage Service<br /> #* FPR-admin<br /> #* appraisal tab<br /> #* transfer browser<br /> #* Others?<br /> # Create stable/#.x branch<br /> # Delete qa/#.x branch if necessary<br /> <br /> [[Category:Process documentation]]</div>

Scollazo https://wiki.archivematica.org/index.php?title=Packages&diff=12103 2017-10-06T16:07:38Z

<p>Scollazo: /* Package development */</p> <hr /> <div><br /> == Archivematica packaging ==<br /> <br /> Some Archivematica dependencies can't currently be obtained from the official Ubuntu/Centos package repositories, so we build them and make them publicly available in the archivematica-externals repository. <br /> <br /> Those external dependencies, and also the official Archivematica packages for Ubuntu and Redhat/Centos can be built using the scripts provided by the am-packbuild repository.<br /> <br /> <br /> Roughly, the workflow for debian package building is as follow:<br /> - build docker image<br /> - clone code<br /> - tweak code when needed<br /> - update changelog<br /> - build debian/rpm package<br /> <br /> <br /> There is a Jenkins job triggered each time a new commit is made to the qa/1.x and qa/0.x. The job is workflow is based on the am-packbuil repo, and each time a new commit is made to the qa/1.x branch, the deb and rpm packages are built and uploaded to a development repo that can be configured as follow:<br /> <br /> RH/Centos:<br /> http://jenkins-ci.archivematica.org/repos/rpm/release/<br /> <br /> Ubuntu 14.04 (Trusty)<br /> deb http://jenkins-ci.archivematica.org/repos/apt/release /<br /> <br /> Ubuntu 16.04 (Xenial)<br /> deb http://jenkins-ci.archivematica.org/repos/apt/release-xenial /<br /> <br /> The development key can be installed on Ubuntu systems with:<br /> wget -O - http://jenkins-ci.archivematica.org/repos/devel.key | sudo apt-key add - <br /> <br /> The jenkins job trigger gets the release version and number using the git describe --tags (for 1.7, the VERSION values are hardcoded to 1.7.0/0.11.0), and builds the package with<br /> make VERSION=&lt;version number&gt; RELEASE=&lt;release number&gt; BRANCH=qa/1.x, and then signs them with the archivematica development key.<br /> <br /> This is a first step working towards a full git integration, where a new official releases are triggered by the creation of a tag in the repo, and don't need manual intervention besides configuring the gpg production signing keys in advance.For now, for official releases, the build job can be configured with the gpg id of the production key, in order to build packages that can be added to the production repo.<br /> <br /> == Package development ==<br /> The Makefiles available in the debs/ and rpm/ folders cover all the steps noted in at the beggining. Some of them can be triggered individually by running an specific make task (make build-docker-image, or make build)<br /> <br /> For adhoc packages (using a different branch than qa/1.x), jenkins jobs can be triggered using the needed parameters for version, branch, and package repository.<br /> <br /> == GPG Key management ==<br /> <br /> There are two keys being used, one in the development stage while the other is only for production ready packages. The reason behind that is avoiding misshappenings with development packages reaching production systems.<br /> <br /> To avoid exposing the private gpg keys needed to build the packges, there archivematica makefiles have two methods for injecting them. One is a file called GPG-KEYS in the root directory, useful for local development, and the other loads the environment var $GPG_KEY, and imports it, so we can inject the key to the jenkins build process without exposing it.</div>

Scollazo https://wiki.archivematica.org/index.php?title=Packages&diff=12102 2017-10-06T15:38:14Z

<p>Scollazo: initial draft</p> <hr /> <div><br /> == Archivematica packaging ==<br /> <br /> Some Archivematica dependencies can't currently be obtained from the official Ubuntu/Centos package repositories, so we build them and make them publicly available in the archivematica-externals repository. <br /> <br /> Those external dependencies, and also the official Archivematica packages for Ubuntu and Redhat/Centos can be built using the scripts provided by the am-packbuild repository.<br /> <br /> <br /> Roughly, the workflow for debian package building is as follow:<br /> - build docker image<br /> - clone code<br /> - tweak code when needed<br /> - update changelog<br /> - build debian/rpm package<br /> <br /> <br /> There is a Jenkins job triggered each time a new commit is made to the qa/1.x and qa/0.x. The job is workflow is based on the am-packbuil repo, and each time a new commit is made to the qa/1.x branch, the deb and rpm packages are built and uploaded to a development repo that can be configured as follow:<br /> <br /> RH/Centos:<br /> http://jenkins-ci.archivematica.org/repos/rpm/release/<br /> <br /> Ubuntu 14.04 (Trusty)<br /> deb http://jenkins-ci.archivematica.org/repos/apt/release /<br /> <br /> Ubuntu 16.04 (Xenial)<br /> deb http://jenkins-ci.archivematica.org/repos/apt/release-xenial /<br /> <br /> The development key can be installed on Ubuntu systems with:<br /> wget -O - http://jenkins-ci.archivematica.org/repos/devel.key | sudo apt-key add - <br /> <br /> The jenkins job trigger gets the release version and number using the git describe --tags (for 1.7, the VERSION values are hardcoded to 1.7.0/0.11.0), and builds the package with<br /> make VERSION=&lt;version number&gt; RELEASE=&lt;release number&gt; BRANCH=qa/1.x, and then signs them with the archivematica development key.<br /> <br /> This is a first step working towards a full git integration, where a new official releases are triggered by the creation of a tag in the repo, and don't need manual intervention besides configuring the gpg production signing keys in advance.For now, for official releases, the build job can be configured with the gpg id of the production key, in order to build packages that can be added to the production repo.<br /> <br /> == Package development ==<br /> The Makefiles available in the debs/ and rpm/ folders cover all the steps noted in at the beggining. Some of them can be triggered individually by running an specific make task (make build-docker-image, or make build)<br /> <br /> == GPG Key management ==<br /> <br /> There are two keys being used, one in the development stage while the other is only for production ready packages. The reason behind that is avoiding misshappenings with development packages reaching production systems.<br /> <br /> To avoid exposing the private gpg keys needed to build the packges, there archivematica makefiles have two methods for injecting them. One is a file called GPG-KEYS in the root directory, useful for local development, and the other loads the environment var $GPG_KEY, and imports it, so we can inject the key to the jenkins build process without exposing it.</div>

Scollazo https://wiki.archivematica.org/index.php?title=Release_Process&diff=11841 2017-06-14T16:13:03Z

<p>Scollazo: /* Development stage */</p> <hr /> <div>This is an outline/checklist of the process to create Archivematica &amp; Storage service releases.<br /> <br /> == Overview ==<br /> <br /> # Merge new features<br /> # Test new features<br /> # [[#Update PRONOM]]<br /> # Write documentation<br /> # Update dependencies<br /> # [[#Build deb/rpm packages]]<br /> # Test packages for new installs and upgrades<br /> # [[#Tag Release]]<br /> # Update ansible roles<br /> # Announce release<br /> <br /> == Translations ==<br /> <br /> '''Needs to be improved!'''<br /> <br /> * Determine code freeze / call for translations process<br /> * Describe processes: push and pull - and when it needs to happen<br /> * We made a choice on how we're using Transifex to keep things simple: only one branch at a time pushed to Transifex. E.g. once SS 0.10.0 is released we have to decide if:<br /> *# We move Transifex to stable/0.10.x for a while so we can work on a minor release with translation fixes (e.g. 0.10.1), or<br /> *# We move to qa/0.11.x which would only make possible to bring new translations to SS 0.11.0.<br /> * Affected repositories<br /> ** archivematica-storage-service<br /> ** archivematica-workflow<br /> ** archivematica-dashboard<br /> *** Includes archivematica-fpr-admin<br /> *** Includes appraisal-tab<br /> <br /> == Update PRONOM ==<br /> <br /> PRONOM needs to be updated in our file identification tools, FIDO &amp; Siegfried, as well as in the FPR.<br /> <br /> === Update FIDO ===<br /> <br /> The FPR update currently use FIDO as a source for new PRONOM, since it is formatted nicer than PRONOM offers, so we depend on FIDO having updated their PRONOM. If that has not happened, we can generate a new formats-v##.xml by updating signatures manually. Artefactual can also update PRONOM and submit a PR to FIDO.<br /> <br /> # Checkout fido from https://github.com/openpreserve/fido<br /> # Update signatures<br /> #* Run &lt;code&gt;python -m fido.update_signatures&lt;/code&gt; from the fido repository root<br /> # Add:<br /> #* New signature file &lt;code&gt;fido/conf/DROID_SignatureFile-v##.xml&lt;/code&gt;<br /> #* New formats file &lt;code&gt;fido/conf/formats-v##.xml&lt;/code&gt;<br /> #* New PRONOM zip file &lt;code&gt;fido/conf/pronom-xml-v##.zip&lt;/code&gt;<br /> #* Updated &lt;code&gt;fido/conf/versions.xml&lt;/code&gt;<br /> # Remove:<br /> #* Old signature file<br /> #* Old formats file<br /> #* Old PRONOM zip file<br /> # Update &lt;code&gt;fido/fido.py&lt;/code&gt; to point to the new format_files<br /> # Create pull request. This may trigger a release from FIDO<br /> <br /> === Package FIDO ===<br /> <br /> First, clone the [https://github.com/artefactual-labs/am-packbuild/ am-packbuild] repo. The most recent work has been done in branch dev/packages-1.6-docker, and needs to be merged into main.<br /> <br /> Update the Makefiles available at &lt;code&gt;rpm/fido/Makefile&lt;/code&gt;, and &lt;code&gt;deb/fido/Makefile&lt;/code&gt; , and run &lt;code&gt;make&lt;/code&gt; in each directory to build the packages.<br /> <br /> === Package Siegfried ===<br /> <br /> First, clone the [https://github.com/artefactual-labs/am-packbuild/ am-packbuild] repo. The most recent work has been done in branch dev/packages-1.6-docker, and needs to be merged into main.<br /> <br /> Update the Makefiles available at &lt;code&gt;rpm/siegfried/Makefile&lt;/code&gt;, and &lt;code&gt;deb/siegfried/Makefile&lt;/code&gt; , and run &lt;code&gt;make&lt;/code&gt; in each directory to build the packages.<br /> <br /> === Update FPR ===<br /> <br /> This process is currently more convoluted than it needs to be, and should be simplified.<br /> <br /> This depends on FIDO having updated their PRONOM files. See [[#Update FIDO]]<br /> <br /> # Install the latest stable Archivematica release<br /> # Generate a JSON with the current version of the FPR (for use later)<br /> #* &lt;code&gt;./src/dashboard/src/manage.py dumpdata fpr &gt; fpr-current.json&lt;/code&gt;<br /> # Install the latest development version of Archivematica<br /> # Generate a blank data migration in the FPR-admin module<br /> #* E.g. &lt;code&gt;./src/dashboard/src/manage.py makemigrations --empty --name pronom_89 fpr&lt;/code&gt;<br /> # Generate the FPR migration body. This also updates the local database's FPR with the new PRONOM IDs<br /> #* E.g. &lt;code&gt;am import-pronom-ids fido/fido/conf/formats-v89.xml --output-filename archivematica/pronom89.txt&lt;/code&gt;<br /> # Copy the output into the blank migration, add RunPython operation<br /> #* &lt;code&gt;migrations.RunPython(data_migration),&lt;/code&gt;<br /> # Due to a [https://projects.artefactual.com/issues/10466#change-46673 bug], fix the imports<br /> #* Remove the &lt;code&gt;apps.get_model&lt;/code&gt; lines<br /> #* Import the models directly with &lt;code&gt;from fpr.models import Format, FormatVersion, IDRule&lt;/code&gt;<br /> # Commit<br /> # Deploy on testing pipeline<br /> # (Analysts) Update the new entries<br /> #* Move new formats to a more appropriate category<br /> #* Create rules &amp; commands<br /> #* Test with data for new formats<br /> # Generate a JSON with the updated version of the FPR on the testing pipeline<br /> #* &lt;code&gt;./src/dashboard/src/manage.py dumpdata fpr &gt; fpr-updated.json&lt;/code&gt;<br /> # Get the updates as JSON<br /> #* E.g. &lt;code&gt;am get-fpr-changes fpr-current.json fpr-updated.json src/dashboard/src/fpr/migrations/pronom_89.json&lt;/code&gt;<br /> # Update the migration to load the JSON updates<br /> #* Replace the contents of the data migration function with<br /> #** &lt;code&gt;fixture_file = os.path.join(os.path.dirname(__file__), 'pronom_89.json')&lt;/code&gt;<br /> #** &lt;code&gt;call_command('loaddata', fixture_file, app_label='fpr')&lt;/code&gt;<br /> #* Remove the direct imports from the bug<br /> #* Improvement Note: Because this is using loaddata, this will have problems if the FPR models are changed. A possible solution is to update get-fpr-changes to generate a migration instead of JSON<br /> # Commit<br /> # Merge<br /> <br /> == Update dependencies ==<br /> <br /> === Python Packages ===<br /> <br /> [https://github.com/artefactual-labs/mets-reader-writer metsrw] and [https://github.com/artefactual-labs/agentarchives agentarchives] both have Makefiles that handle most of the packaging<br /> <br /> # Check for open PRs, merge as necessary<br /> # Update &lt;code&gt;setup.py&lt;/code&gt; with the new version, create a pull request, code review, [[Merging | merge]].<br /> # Tag new release, push tag<br /> #* &lt;code&gt;git push --tags&lt;/code&gt;<br /> # Run &lt;code&gt;make package&lt;/code&gt;<br /> #* This will build the package and upload it to PyPI. It will prompt for your PyPI username and password for the upload<br /> #* The Makefile was created from instructions at https://packaging.python.org/distributing/<br /> # &lt;code&gt;make clean&lt;/code&gt; will delete packaging related files<br /> <br /> == Build deb/rpm packages ==<br /> <br /> The am-packbuild repository has all the code related to building packages, except the building gpg keys. The steps to follow in order to build production production packages are as follow<br /> <br /> === Debian packages ===<br /> <br /> # Clone the [https://github.com/artefactual-labs/am-packbuild am-packbuild] repo. Latest work is available in dev/packages-1.6-docker<br /> # Put your gpg private key into &lt;code&gt;debs/GPG-KEYS-REPOS&lt;/code&gt;. That's the place the Dockerfile looks for it when building the environment.<br /> # Update the makefile at &lt;code&gt;debs/archivematica/Makefile&lt;/code&gt; in order to reflect version/keys you want to use. If you want to build the storage service, you need also to change the packbuild.py line to reflect that (TODO: create service-storage package folder)<br /> This dockerfile uses packbuild.py script, whose function is downloadig the code from github, creating a debian/changelog file for each ubuntu version with the expected version name and description of last commit included, and build the package aftewards. It can be run on their own too, without the need of a docker environment.<br /> # Run &lt;make&gt;, and the packages will be available in the &lt;code&gt;build&lt;/code&gt; once the building finishes.<br /> # Upload packages to public debian repository<br /> <br /> === Debian reposities ===<br /> <br /> There are two debian repositories, one for archivematica packages, and one for dependencies. The procedure in order to create new ones, or upload packages to them, is the same:<br /> <br /> #* Create folder for repo, and configuration file:<br /> &lt;code&gt;<br /> mkdir -p /path/to/repos/repo/conf<br /> <br /> cat &gt; /path/to/repos/repo/conf/distributions &lt;&lt; EOF<br /> <br /> Codename: trusty<br /> <br /> Components: main<br /> <br /> Architectures: amd64 source<br /> <br /> SignWith: &lt;short gpg keyid&gt; <br /> <br /> EOF<br /> &lt;/code&gt;<br /> <br /> #* Go inside the repo, and import the packages previously uploaded with:<br /> &lt;code&gt;<br /> cd /path/to/repos/repo/<br /> <br /> reprepro includedeb trusty /path/to/packages/*.deb<br /> <br /> reprepro includedsc trusty /path/to/packages/*.deb<br /> <br /> &lt;/code&gt;<br /> <br /> The current official repo is at packages.archivematica.org<br /> <br /> === RPM Packages ===<br /> <br /> # Package specs are available in [https://github.com/artefactual-labs/am-packbuild/tree/dev/packaging-1.6-docker/rpm am-packbuild/rpms]<br /> # There are vars in the Makefiles for version/release , so update them when needed<br /> # In order to build them, just go into the directory you want to build, and run “make”<br /> <br /> === RPM Repositories ===<br /> <br /> Once the packages are built, upload them to packages.archivematica.org/&lt;version&gt;/centos<br /> Go inside that dir, and run “createrepo .” (For packages other than archivematica , use “centos-extras” repository)<br /> <br /> === Development stage ===<br /> <br /> In the final stages of development, the repositories for the new releases are created, but packages are signed with a development key to avoid mistakes. Once the development stage finishes, all new packages need to be rebuild using the production keys.<br /> <br /> Development packages are built on each new commit to stable branches by Jenkins. Repositories are available at http://jenkins-ci.archivematica.org/repos/<br /> <br /> == Website ==<br /> <br /> '''Needs to be improved!'''<br /> <br /> ==== Homepage ====<br /> <br /> * Make changes in archivematica-web.git<br /> ** Update links<br /> ** Add new doc repos<br /> * Deploy<br /> ** Log in sites-pub as archivematica-web and run update-web.sh<br /> <br /> ==== Documentation ====<br /> <br /> * Deploy<br /> ** Log in sites-pub as archivematica-web and run update-docs.sh<br /> <br /> ==== Wiki ====<br /> <br /> * Release notes<br /> * Installation notes<br /> * ...<br /> <br /> == News ==<br /> <br /> * Twitter<br /> * Mailing list<br /> * News section in artefactual.com<br /> <br /> == Update ansible roles ==<br /> <br /> Check that the deploy-pub vars files for archivematica are updated, and the ansible-ansiblematica-src and ansible-archivematica-pkg roles are able to deploy the new version<br /> <br /> == Tag Release ==<br /> <br /> # Add release tags to repositories<br /> #* Archivematica<br /> #* Storage Service<br /> #* FPR-admin<br /> #* appraisal tab<br /> #* transfer browser<br /> #* Others?<br /> # Create stable/#.x branch<br /> # Delete qa/#.x branch if necessary<br /> <br /> [[Category:Process documentation]]</div>

Scollazo https://wiki.archivematica.org/index.php?title=Release_Process&diff=11813 2017-04-11T17:34:46Z

<p>Scollazo: /* Build deb/rpm packages */</p> <hr /> <div>This is an outline/checklist of the process to create Archivematica &amp; Storage service releases.<br /> <br /> == Overview ==<br /> <br /> # Merge new features<br /> # Test new features<br /> # [[#Update PRONOM]]<br /> # Write documentation<br /> # Update dependencies<br /> # [[#Build deb/rpm packages]]<br /> # Test packages for new installs and upgrades<br /> # [[#Tag Release]]<br /> # Update ansible roles<br /> # Announce release<br /> <br /> == Translations ==<br /> <br /> '''Needs to be improved!'''<br /> <br /> * Determine code freeze / call for translations process<br /> * Describe processes: push and pull - and when it needs to happen<br /> * We made a choice on how we're using Transifex to keep things simple: only one branch at a time pushed to Transifex. E.g. once SS 0.10.0 is released we have to decide if:<br /> *# We move Transifex to stable/0.10.x for a while so we can work on a minor release with translation fixes (e.g. 0.10.1), or<br /> *# We move to qa/0.11.x which would only make possible to bring new translations to SS 0.11.0.<br /> * Affected repositories<br /> ** archivematica-storage-service<br /> ** archivematica-workflow<br /> ** archivematica-dashboard<br /> *** Includes archivematica-fpr-admin<br /> *** Includes appraisal-tab<br /> <br /> == Update PRONOM ==<br /> <br /> PRONOM needs to be updated in our file identification tools, FIDO &amp; Siegfried, as well as in the FPR.<br /> <br /> === Update FIDO ===<br /> <br /> The FPR update currently use FIDO as a source for new PRONOM, since it is formatted nicer than PRONOM offers, so we depend on FIDO having updated their PRONOM. If that has not happened, we can generate a new formats-v##.xml by updating signatures manually. Artefactual can also update PRONOM and submit a PR to FIDO.<br /> <br /> # Checkout fido from https://github.com/openpreserve/fido<br /> # Update signatures<br /> #* Run &lt;code&gt;python -m fido.update_signatures&lt;/code&gt; from the fido repository root<br /> # Add:<br /> #* New signature file &lt;code&gt;fido/conf/DROID_SignatureFile-v##.xml&lt;/code&gt;<br /> #* New formats file &lt;code&gt;fido/conf/formats-v##.xml&lt;/code&gt;<br /> #* New PRONOM zip file &lt;code&gt;fido/conf/pronom-xml-v##.zip&lt;/code&gt;<br /> #* Updated &lt;code&gt;fido/conf/versions.xml&lt;/code&gt;<br /> # Remove:<br /> #* Old signature file<br /> #* Old formats file<br /> #* Old PRONOM zip file<br /> # Update &lt;code&gt;fido/fido.py&lt;/code&gt; to point to the new format_files<br /> # Create pull request. This may trigger a release from FIDO<br /> <br /> === Package FIDO ===<br /> <br /> First, clone the [https://github.com/artefactual-labs/am-packbuild/ am-packbuild] repo. The most recent work has been done in branch dev/packages-1.6-docker, and needs to be merged into main.<br /> <br /> Update the Makefiles available at &lt;code&gt;rpm/fido/Makefile&lt;/code&gt;, and &lt;code&gt;deb/fido/Makefile&lt;/code&gt; , and run &lt;code&gt;make&lt;/code&gt; in each directory to build the packages.<br /> <br /> === Package Siegfried ===<br /> <br /> First, clone the [https://github.com/artefactual-labs/am-packbuild/ am-packbuild] repo. The most recent work has been done in branch dev/packages-1.6-docker, and needs to be merged into main.<br /> <br /> Update the Makefiles available at &lt;code&gt;rpm/siegfried/Makefile&lt;/code&gt;, and &lt;code&gt;deb/siegfried/Makefile&lt;/code&gt; , and run &lt;code&gt;make&lt;/code&gt; in each directory to build the packages.<br /> <br /> === Update FPR ===<br /> <br /> This process is currently more convoluted than it needs to be, and should be simplified.<br /> <br /> This depends on FIDO having updated their PRONOM files. See [[#Update FIDO]]<br /> <br /> # Install the latest stable Archivematica release<br /> # Generate a JSON with the current version of the FPR (for use later)<br /> #* &lt;code&gt;./src/dashboard/src/manage.py dumpdata fpr &gt; fpr-current.json&lt;/code&gt;<br /> # Install the latest development version of Archivematica<br /> # Generate a blank data migration in the FPR-admin module<br /> #* E.g. &lt;code&gt;./src/dashboard/src/manage.py makemigrations --empty --name pronom_89 fpr&lt;/code&gt;<br /> # Generate the FPR migration body. This also updates the local database's FPR with the new PRONOM IDs<br /> #* E.g. &lt;code&gt;am import-pronom-ids fido/fido/conf/formats-v89.xml --output-filename archivematica/pronom89.txt&lt;/code&gt;<br /> # Copy the output into the blank migration, add RunPython operation<br /> #* &lt;code&gt;migrations.RunPython(data_migration),&lt;/code&gt;<br /> # Due to a [https://projects.artefactual.com/issues/10466#change-46673 bug], fix the imports<br /> #* Remove the &lt;code&gt;apps.get_model&lt;/code&gt; lines<br /> #* Import the models directly with &lt;code&gt;from fpr.models import Format, FormatVersion, IDRule&lt;/code&gt;<br /> # Commit<br /> # Deploy on testing pipeline<br /> # (Analysts) Update the new entries<br /> #* Move new formats to a more appropriate category<br /> #* Create rules &amp; commands<br /> #* Test with data for new formats<br /> # Generate a JSON with the updated version of the FPR on the testing pipeline<br /> #* &lt;code&gt;./src/dashboard/src/manage.py dumpdata fpr &gt; fpr-updated.json&lt;/code&gt;<br /> # Get the updates as JSON<br /> #* E.g. &lt;code&gt;am get-fpr-changes fpr-current.json fpr-updated.json src/dashboard/src/fpr/migrations/pronom_89.json&lt;/code&gt;<br /> # Update the migration to load the JSON updates<br /> #* Replace the contents of the data migration function with<br /> #** &lt;code&gt;fixture_file = os.path.join(os.path.dirname(__file__), 'pronom_89.json')&lt;/code&gt;<br /> #** &lt;code&gt;call_command('loaddata', fixture_file, app_label='fpr')&lt;/code&gt;<br /> #* Remove the direct imports from the bug<br /> #* Improvement Note: Because this is using loaddata, this will have problems if the FPR models are changed. A possible solution is to update get-fpr-changes to generate a migration instead of JSON<br /> # Commit<br /> # Merge<br /> <br /> == Update dependencies ==<br /> <br /> === Python Packages ===<br /> <br /> [https://github.com/artefactual-labs/mets-reader-writer metsrw] and [https://github.com/artefactual-labs/agentarchives agentarchives] both have Makefiles that handle most of the packaging<br /> <br /> # Check for open PRs, merge as necessary<br /> # Update &lt;code&gt;setup.py&lt;/code&gt; with the new version, create a pull request, code review, [[Merging | merge]].<br /> # Tag new release, push tag<br /> #* &lt;code&gt;git push --tags&lt;/code&gt;<br /> # Run &lt;code&gt;make package&lt;/code&gt;<br /> #* This will build the package and upload it to PyPI. It will prompt for your PyPI username and password for the upload<br /> #* The Makefile was created from instructions at https://packaging.python.org/distributing/<br /> # &lt;code&gt;make clean&lt;/code&gt; will delete packaging related files<br /> <br /> == Build deb/rpm packages ==<br /> <br /> The am-packbuild repository has all the code related to building packages, except the building gpg keys. The steps to follow in order to build production production packages are as follow<br /> <br /> === Debian packages ===<br /> <br /> # Clone the [https://github.com/artefactual-labs/am-packbuild am-packbuild] repo. Latest work is available in dev/packages-1.6-docker<br /> # Put your gpg private key into &lt;code&gt;debs/GPG-KEYS-REPOS&lt;/code&gt;. That's the place the Dockerfile looks for it when building the environment.<br /> # Update the makefile at &lt;code&gt;debs/archivematica/Makefile&lt;/code&gt; in order to reflect version/keys you want to use. If you want to build the storage service, you need also to change the packbuild.py line to reflect that (TODO: create service-storage package folder)<br /> This dockerfile uses packbuild.py script, whose function is downloadig the code from github, creating a debian/changelog file for each ubuntu version with the expected version name and description of last commit included, and build the package aftewards. It can be run on their own too, without the need of a docker environment.<br /> # Run &lt;make&gt;, and the packages will be available in the &lt;code&gt;build&lt;/code&gt; once the building finishes.<br /> # Upload packages to public debian repository<br /> <br /> === Debian reposities ===<br /> <br /> There are two debian repositories, one for archivematica packages, and one for dependencies. The procedure in order to create new ones, or upload packages to them, is the same:<br /> <br /> #* Create folder for repo, and configuration file:<br /> &lt;code&gt;<br /> mkdir -p /path/to/repos/repo/conf<br /> <br /> cat &gt; /path/to/repos/repo/conf/distributions &lt;&lt; EOF<br /> <br /> Codename: trusty<br /> <br /> Components: main<br /> <br /> Architectures: amd64 source<br /> <br /> SignWith: &lt;short gpg keyid&gt; <br /> <br /> EOF<br /> &lt;/code&gt;<br /> <br /> #* Go inside the repo, and import the packages previously uploaded with:<br /> &lt;code&gt;<br /> cd /path/to/repos/repo/<br /> <br /> reprepro includedeb trusty /path/to/packages/*.deb<br /> <br /> reprepro includedsc trusty /path/to/packages/*.deb<br /> <br /> &lt;/code&gt;<br /> <br /> The current official repo is at packages.archivematica.org<br /> <br /> === RPM Packages ===<br /> <br /> # Package specs are available in [https://github.com/artefactual-labs/am-packbuild/tree/dev/packaging-1.6-docker/rpm am-packbuild/rpms]<br /> # There are vars in the Makefiles for version/release , so update them when needed<br /> # In order to build them, just go into the directory you want to build, and run “make”<br /> <br /> === RPM Repositories ===<br /> <br /> Once the packages are built, upload them to packages.archivematica.org/&lt;version&gt;/centos<br /> Go inside that dir, and run “createrepo .” (For packages other than archivematica , use “centos-extras” repository)<br /> <br /> === Development stage ===<br /> <br /> In the final stages of development, the repositories for the new releases are created, but packages are signed with a development key to avoid mistakes. Once the development stage finishes, all new packages need to be rebuild using the production keys.<br /> <br /> == Website ==<br /> <br /> '''Needs to be improved!'''<br /> <br /> ==== Homepage ====<br /> <br /> * Make changes in archivematica-web.git<br /> ** Update links<br /> ** Add new doc repos<br /> * Deploy<br /> ** Log in sites-pub as archivematica-web and run update-web.sh<br /> <br /> ==== Documentation ====<br /> <br /> * Deploy<br /> ** Log in sites-pub as archivematica-web and run update-docs.sh<br /> <br /> ==== Wiki ====<br /> <br /> * Release notes<br /> * Installation notes<br /> * ...<br /> <br /> == News ==<br /> <br /> * Twitter<br /> * Mailing list<br /> * News section in artefactual.com<br /> <br /> == Update ansible roles ==<br /> <br /> Check that the deploy-pub vars files for archivematica are updated, and the ansible-ansiblematica-src and ansible-archivematica-pkg roles are able to deploy the new version<br /> <br /> == Tag Release ==<br /> <br /> # Add release tags to repositories<br /> #* Archivematica<br /> #* Storage Service<br /> #* FPR-admin<br /> #* appraisal tab<br /> #* transfer browser<br /> #* Others?<br /> # Create stable/#.x branch<br /> # Delete qa/#.x branch if necessary<br /> <br /> [[Category:Process documentation]]</div>

Scollazo https://wiki.archivematica.org/index.php?title=Release_Process&diff=11777 2017-03-30T21:11:59Z

<p>Scollazo: </p> <hr /> <div>This is an outline/checklist of the process to create Archivematica &amp; Storage service releases.<br /> <br /> == Overview ==<br /> <br /> # Merge new features<br /> # Test new features<br /> # [[#Update PRONOM]]<br /> # Write documentation<br /> # [[#Build deb/rpm packages]]<br /> # [[#Tag Release]]<br /> # Update ansible roles<br /> # Announce release<br /> <br /> <br /> == Update PRONOM ==<br /> <br /> PRONOM needs to be updated in our file identification tools, FIDO &amp; Siegfried, as well as in the FPR.<br /> <br /> === Update FIDO ===<br /> <br /> The FPR update currently use FIDO as a source for new PRONOM, since it is formatted nicer than PRONOM offers, so we depend on FIDO having updated their PRONOM. If that has not happened, we can generate a new formats-v##.xml by updating signatures manually. Artefactual can also update PRONOM and submit a PR to FIDO.<br /> <br /> # Checkout fido from https://github.com/openpreserve/fido<br /> # Update signatures<br /> #* Run &lt;code&gt;python -m fido.update_signatures&lt;/code&gt; from the fido repository root<br /> # Add:<br /> #* New signature file &lt;code&gt;fido/conf/DROID_SignatureFile-v##.xml&lt;/code&gt;<br /> #* New formats file &lt;code&gt;fido/conf/formats-v##.xml&lt;/code&gt;<br /> #* New PRONOM zip file &lt;code&gt;fido/conf/pronom-xml-v##.zip&lt;/code&gt;<br /> #* Updated &lt;code&gt;fido/conf/versions.xml&lt;/code&gt;<br /> # Remove:<br /> #* Old signature file<br /> #* Old formats file<br /> #* Old PRONOM zip file<br /> # Update &lt;code&gt;fido/fido.py&lt;/code&gt; to point to the new format_files<br /> # Create pull request. This may trigger a release from FIDO<br /> <br /> === Package FIDO ===<br /> <br /> First, clone the [https://github.com/artefactual-labs/am-packbuild/ am-packbuild] repo. The most recent work has been done in branch dev/packages-1.6-docker, and needs to be merged into main.<br /> <br /> Update the Makefiles available at &lt;code&gt;rpm/fido/Makefile&lt;/code&gt;, and &lt;code&gt;deb/fido/Makefile&lt;/code&gt; , and run &lt;code&gt;make&lt;/code&gt; in each directory to build the packages.<br /> <br /> === Package Siegfried ===<br /> <br /> First, clone the [https://github.com/artefactual-labs/am-packbuild/ am-packbuild] repo. The most recent work has been done in branch dev/packages-1.6-docker, and needs to be merged into main.<br /> <br /> Update the Makefiles available at &lt;code&gt;rpm/siegfried/Makefile&lt;/code&gt;, and &lt;code&gt;deb/siegfried/Makefile&lt;/code&gt; , and run &lt;code&gt;make&lt;/code&gt; in each directory to build the packages.<br /> <br /> === Update FPR ===<br /> <br /> This process is currently more convoluted than it needs to be, and should be simplified.<br /> <br /> This depends on FIDO having updated their PRONOM files. See [[#Update FIDO]]<br /> <br /> # Install the latest stable Archivematica release<br /> # Generate a JSON with the current version of the FPR (for use later)<br /> #* &lt;code&gt;./src/dashboard/src/manage.py dumpdata fpr &gt; fpr-current.json&lt;/code&gt;<br /> # Install the latest development version of Archivematica<br /> # Generate a blank data migration in the FPR-admin module<br /> #* E.g. &lt;code&gt;./src/dashboard/src/manage.py makemigrations --empty --name pronom_89 fpr&lt;/code&gt;<br /> # Generate the FPR migration body. This also updates the local database's FPR with the new PRONOM IDs<br /> #* E.g. &lt;code&gt;am import-pronom-ids fido/fido/conf/formats-v89.xml --output-filename archivematica/pronom89.txt&lt;/code&gt;<br /> # Copy the output into the blank migration, add RunPython operation<br /> #* &lt;code&gt;migrations.RunPython(data_migration),&lt;/code&gt;<br /> # Due to a [https://projects.artefactual.com/issues/10466#change-46673 bug], fix the imports<br /> #* Remove the &lt;code&gt;apps.get_model&lt;/code&gt; lines<br /> #* Import the models directly with &lt;code&gt;from fpr.models import Format, FormatVersion, IDRule&lt;/code&gt;<br /> # Commit<br /> # Deploy on testing pipeline<br /> # (Analysts) Update the new entries<br /> #* Move new formats to a more appropriate category<br /> #* Create rules &amp; commands<br /> #* Test with data for new formats<br /> # Generate a JSON with the updated version of the FPR on the testing pipeline<br /> #* &lt;code&gt;./src/dashboard/src/manage.py dumpdata fpr &gt; fpr-updated.json&lt;/code&gt;<br /> # Get the updates as JSON<br /> #* E.g. &lt;code&gt;am get-fpr-changes fpr-current.json fpr-updated.json src/dashboard/src/fpr/migrations/pronom_89.json&lt;/code&gt;<br /> # Update the migration to load the JSON updates<br /> #* Replace the contents of the data migration function with<br /> #** &lt;code&gt;fixture_file = os.path.join(os.path.dirname(__file__), 'pronom_89.json')&lt;/code&gt;<br /> #** &lt;code&gt;call_command('loaddata', fixture_file, app_label='fpr')&lt;/code&gt;<br /> #* Remove the direct imports from the bug<br /> #* Improvement Note: Because this is using loaddata, this will have problems if the FPR models are changed. A possible solution is to update get-fpr-changes to generate a migration instead of JSON<br /> # Commit<br /> # Merge<br /> <br /> <br /> == Build deb/rpm packages ==<br /> <br /> The am-packbuild repository has all the code related to building packages, except the building gpg keys. The steps to follow in order to build production production packages are as follow<br /> <br /> === Debian packages ===<br /> <br /> # Clone the [https://github.com/artefactual-labs/am-packbuild am-packbuild] repo. Latest work is available in dev/packages-1.6-docker<br /> # Put your gpg private key into &lt;code&gt;debs/GPG-KEYS-REPOS&lt;/code&gt;. That's the place the Dockerfile looks for it when building the environment.<br /> # Update the makefile at &lt;code&gt;debs/archivematica/Makefile&lt;/code&gt; in order to reflect version/keys you want to use. If you want to build the storage service, you need also to change the packbuild.py line to reflect that (TODO: create service-storage package folder)<br /> This dockerfile uses packbuild.py script, whose function is downloadig the code from github, creating a debian/changelog file for each ubuntu version with the expected version name and description of last commit included, and build the package aftewards. It can be run on their own too, without the need of a docker environment.<br /> # Run &lt;make&gt;, and the packages will be available in the &lt;code&gt;build&lt;/code&gt; once the building finishes.<br /> # Upload packages to public debian repository<br /> <br /> === Debian reposities ===<br /> <br /> There are two debian repositories, one for archivematica packages, and one for dependencies. The procedure in order to create new ones, or upload packages to them, is the same:<br /> <br /> #* Create folder for repo, and configuration file:<br /> &lt;code&gt;<br /> mkdir -p /path/to/repos/repo/conf<br /> <br /> cat &gt; /path/to/repos/repo/conf/distributions &lt;&lt; EOF<br /> <br /> Codename: trusty<br /> <br /> Components: main<br /> <br /> Architectures: amd64 source<br /> <br /> SignWith: &lt;short gpg keyid&gt; <br /> <br /> EOF<br /> &lt;/code&gt;<br /> <br /> #* Go inside the repo, and import the packages previously uploaded with:<br /> &lt;code&gt;<br /> cd /path/to/repos/repo/<br /> <br /> reprepro includedeb trusty /path/to/packages/*.deb<br /> <br /> reprepro includedsc trusty /path/to/packages/*.deb<br /> <br /> &lt;/code&gt;<br /> <br /> The current official repo is at packages.archivematica.org<br /> <br /> === RPM Packages ===<br /> <br /> # Package specs are available in [https://github.com/artefactual-labs/am-packbuild/tree/dev/packaging-1.6-docker/rpm am-packbuild/rpms]<br /> # There are vars in the Makefiles for version/release , so update them when needed<br /> # In order to build them, just go into the directory you want to build, and run “make”<br /> <br /> === RPM Repositories ===<br /> <br /> Once the packages are built, upload them to packages.archivematica.org/&lt;version&gt;/centos<br /> Go inside that dir, and run “createrepo .” (For packages other than archivematica , use “centos-extras” repository)<br /> <br /> <br /> == Update ansible roles ==<br /> <br /> Check that the deploy-pub vars files for archivematica are updated, and the ansible-ansiblematica-src and ansible-archivematica-pkg roles are able to deploy the new version<br /> <br /> == Tag Release ==<br /> <br /> # Add release tags to repositories<br /> #* Archivematica<br /> #* Storage Service<br /> #* FPR-admin<br /> #* appraisal tab<br /> #* transfer browser<br /> #* Others?<br /> # Create stable/#.x branch<br /> # Delete qa/#.x branch if necessary<br /> <br /> [[Category:Process documentation]]</div>

Scollazo https://wiki.archivematica.org/index.php?title=Release_Process&diff=11775 2017-03-30T19:03:52Z

<p>Scollazo: Add packaging info</p> <hr /> <div>This is an outline/checklist of the process to create Archivematica &amp; Storage service releases.<br /> <br /> == Overview ==<br /> <br /> # Merge new features<br /> # Test new features<br /> # [[#Update PRONOM]]<br /> # Write documentation<br /> # Create packages<br /> # [[#Tag Release]]<br /> # Announce release<br /> <br /> <br /> == Update PRONOM ==<br /> <br /> PRONOM needs to be updated in our file identification tools, FIDO &amp; Siegfried, as well as in the FPR.<br /> <br /> === Update FIDO ===<br /> <br /> The FPR update currently use FIDO as a source for new PRONOM, since it is formatted nicer than PRONOM offers, so we depend on FIDO having updated their PRONOM. If that has not happened, we can generate a new formats-v##.xml by updating signatures manually. Artefactual can also update PRONOM and submit a PR to FIDO.<br /> <br /> # Checkout fido from https://github.com/openpreserve/fido<br /> # Update signatures<br /> #* Run &lt;code&gt;python -m fido.update_signatures&lt;/code&gt; from the fido repository root<br /> # Add:<br /> #* New signature file &lt;code&gt;fido/conf/DROID_SignatureFile-v##.xml&lt;/code&gt;<br /> #* New formats file &lt;code&gt;fido/conf/formats-v##.xml&lt;/code&gt;<br /> #* New PRONOM zip file &lt;code&gt;fido/conf/pronom-xml-v##.zip&lt;/code&gt;<br /> #* Updated &lt;code&gt;fido/conf/versions.xml&lt;/code&gt;<br /> # Remove:<br /> #* Old signature file<br /> #* Old formats file<br /> #* Old PRONOM zip file<br /> # Update &lt;code&gt;fido/fido.py&lt;/code&gt; to point to the new format_files<br /> # Create pull request. This may trigger a release from FIDO<br /> <br /> === Package FIDO ===<br /> <br /> First, clone the [https://github.com/artefactual-labs/am-packbuild/ am-packbuild] repo. The most recent work has been done in branch dev/packages-1.6-docker, and needs to be merged into main.<br /> <br /> Update the Makefiles available at &lt;code&gt;rpm/fido/Makefile&lt;/code&gt;, and &lt;code&gt;deb/fido/Makefile&lt;/code&gt; , and run &lt;code&gt;make&lt;/code&gt; in each directory to build the packages.<br /> <br /> === Package Siegfried ===<br /> <br /> First, clone the [https://github.com/artefactual-labs/am-packbuild/ am-packbuild] repo. The most recent work has been done in branch dev/packages-1.6-docker, and needs to be merged into main.<br /> <br /> Update the Makefiles available at &lt;code&gt;rpm/siegfried/Makefile&lt;/code&gt;, and &lt;code&gt;deb/siegfried/Makefile&lt;/code&gt; , and run &lt;code&gt;make&lt;/code&gt; in each directory to build the packages.<br /> <br /> === Update FPR ===<br /> <br /> This process is currently more convoluted than it needs to be, and should be simplified.<br /> <br /> This depends on FIDO having updated their PRONOM files. See [[#Update FIDO]]<br /> <br /> # Install the latest stable Archivematica release<br /> # Generate a JSON with the current version of the FPR (for use later)<br /> #* &lt;code&gt;./src/dashboard/src/manage.py dumpdata fpr &gt; fpr-current.json&lt;/code&gt;<br /> # Install the latest development version of Archivematica<br /> # Generate a blank data migration in the FPR-admin module<br /> #* E.g. &lt;code&gt;./src/dashboard/src/manage.py makemigrations --empty --name pronom_89 fpr&lt;/code&gt;<br /> # Generate the FPR migration body. This also updates the local database's FPR with the new PRONOM IDs<br /> #* E.g. &lt;code&gt;am import-pronom-ids fido/fido/conf/formats-v89.xml --output-filename archivematica/pronom89.txt&lt;/code&gt;<br /> # Copy the output into the blank migration, add RunPython operation<br /> #* &lt;code&gt;migrations.RunPython(data_migration),&lt;/code&gt;<br /> # Due to a [https://projects.artefactual.com/issues/10466#change-46673 bug], fix the imports<br /> #* Remove the &lt;code&gt;apps.get_model&lt;/code&gt; lines<br /> #* Import the models directly with &lt;code&gt;from fpr.models import Format, FormatVersion, IDRule&lt;/code&gt;<br /> # Commit<br /> # Deploy on testing pipeline<br /> # (Analysts) Update the new entries<br /> #* Move new formats to a more appropriate category<br /> #* Create rules &amp; commands<br /> #* Test with data for new formats<br /> # Generate a JSON with the updated version of the FPR on the testing pipeline<br /> #* &lt;code&gt;./src/dashboard/src/manage.py dumpdata fpr &gt; fpr-updated.json&lt;/code&gt;<br /> # Get the updates as JSON<br /> #* E.g. &lt;code&gt;am get-fpr-changes fpr-current.json fpr-updated.json src/dashboard/src/fpr/migrations/pronom_89.json&lt;/code&gt;<br /> # Update the migration to load the JSON updates<br /> #* Replace the contents of the data migration function with<br /> #** &lt;code&gt;fixture_file = os.path.join(os.path.dirname(__file__), 'pronom_89.json')&lt;/code&gt;<br /> #** &lt;code&gt;call_command('loaddata', fixture_file, app_label='fpr')&lt;/code&gt;<br /> #* Remove the direct imports from the bug<br /> #* Improvement Note: Because this is using loaddata, this will have problems if the FPR models are changed. A possible solution is to update get-fpr-changes to generate a migration instead of JSON<br /> # Commit<br /> # Merge<br /> <br /> == Tag Release ==<br /> <br /> # Add release tags to repositories<br /> #* Archivematica<br /> #* Storage Service<br /> #* FPR-admin<br /> #* appraisal tab<br /> #* transfer browser<br /> #* Others?<br /> # Create stable/#.x branch<br /> # Delete qa/#.x branch<br /> <br /> == Build deb/rpm packages ==<br /> <br /> The am-packbuild repository has all the code related to building packages, except the building gpg keys. The steps to follow in order to build production production packages are as follow<br /> <br /> === Debian packages ===<br /> <br /> # Clone the [https://github.com/artefactual-labs/am-packbuild am-packbuild] repo. Latest work is available in dev/packages-1.6-docker<br /> # Put your gpg private key into &lt;code&gt;debs/GPG-KEYS-REPOS&lt;/code&gt;. That's the place the Dockerfile looks for it when building the environment.<br /> # Update the makefile at &lt;code&gt;debs/archivematica/Makefile&lt;/code&gt; in order to reflect version/keys you want to use. If you want to build the storage service, you need also to change the packbuild.py line to reflect that (TODO: create service-storage package folder)<br /> This dockerfile uses packbuild.py script, whose function is downloadig the code from github, creating a debian/changelog file for each ubuntu version with the expected version name and description of last commit included, and build the package aftewards. It can be run on their own too, without the need of a docker environment.<br /> # Run &lt;make&gt;, and the packages will be available in the &lt;code&gt;build&lt;/code&gt; once the building finishes.<br /> # Upload packages to public debian repository<br /> <br /> === Debian reposities ===<br /> <br /> There are two debian repositories, one for archivematica packages, and one for dependencies. The procedure in order to create new ones, or upload packages to them, is the same:<br /> <br /> #* Create folder for repo, and configuration file:<br /> &lt;code&gt;<br /> mkdir -p /path/to/repos/repo/conf<br /> <br /> cat &gt; /path/to/repos/repo/conf/distributions &lt;&lt; EOF<br /> <br /> Codename: trusty<br /> <br /> Components: main<br /> <br /> Architectures: amd64 source<br /> <br /> SignWith: &lt;short gpg keyid&gt; <br /> <br /> EOF<br /> &lt;/code&gt;<br /> <br /> #* Go inside the repo, and import the packages previously uploaded with:<br /> &lt;code&gt;<br /> cd /path/to/repos/repo/<br /> <br /> reprepro includedeb trusty /path/to/packages/*.deb<br /> <br /> reprepro includedsc trusty /path/to/packages/*.deb<br /> <br /> &lt;/code&gt;<br /> <br /> The current official repo is at packages.archivematica.org<br /> <br /> === RPM Packages ===<br /> <br /> # Package specs are available in [https://github.com/artefactual-labs/am-packbuild/tree/dev/packaging-1.6-docker/rpm am-packbuild/rpms]<br /> # There are vars in the Makefiles for version/release , so update them when needed<br /> # In order to build them, just go into the directory you want to build, and run “make”<br /> <br /> === RPM Repositories ===<br /> <br /> Once the packages are built, upload them to packages.archivematica.org/&lt;version&gt;/centos<br /> Go inside that dir, and run “createrepo .” (For packages other than archivematica , use “centos-extras” repository)<br /> <br /> <br /> <br /> [[Category:Process documentation]]</div>

Scollazo https://wiki.archivematica.org/index.php?title=Release_Process&diff=11771 2017-03-30T16:33:48Z

<p>Scollazo: /* Package Siegfried */</p> <hr /> <div>This is an outline/checklist of the process to create Archivematica &amp; Storage service releases.<br /> <br /> == Overview ==<br /> <br /> # Merge new features<br /> # Test new features<br /> # [[#Update PRONOM]]<br /> # Write documentation<br /> # Create packages<br /> # [[#Tag Release]]<br /> # Announce release<br /> <br /> <br /> == Update PRONOM ==<br /> <br /> PRONOM needs to be updated in our file identification tools, FIDO &amp; Siegfried, as well as in the FPR.<br /> <br /> === Update FIDO ===<br /> <br /> The FPR update currently use FIDO as a source for new PRONOM, since it is formatted nicer than PRONOM offers, so we depend on FIDO having updated their PRONOM. If that has not happened, we can generate a new formats-v##.xml by updating signatures manually. Artefactual can also update PRONOM and submit a PR to FIDO.<br /> <br /> # Checkout fido from https://github.com/openpreserve/fido<br /> # Update signatures<br /> #* Run &lt;code&gt;python -m fido.update_signatures&lt;/code&gt; from the fido repository root<br /> # Add:<br /> #* New signature file &lt;code&gt;fido/conf/DROID_SignatureFile-v##.xml&lt;/code&gt;<br /> #* New formats file &lt;code&gt;fido/conf/formats-v##.xml&lt;/code&gt;<br /> #* New PRONOM zip file &lt;code&gt;fido/conf/pronom-xml-v##.zip&lt;/code&gt;<br /> #* Updated &lt;code&gt;fido/conf/versions.xml&lt;/code&gt;<br /> # Remove:<br /> #* Old signature file<br /> #* Old formats file<br /> #* Old PRONOM zip file<br /> # Update &lt;code&gt;fido/fido.py&lt;/code&gt; to point to the new format_files<br /> # Create pull request. This may trigger a release from FIDO<br /> <br /> === Package FIDO ===<br /> <br /> First, clone the [https://github.com/artefactual-labs/am-packbuild/ am-packbuild] repo. The most recent work has been done in branch dev/packages-1.6-docker, and needs to be merged into main.<br /> <br /> Update the Makefiles available at &lt;code&gt;rpm/fido/Makefile&lt;/code&gt;, and &lt;code&gt;deb/fido/Makefile&lt;/code&gt; , and run &lt;code&gt;make&lt;/code&gt; in each directory to build the packages.<br /> <br /> === Package Siegfried ===<br /> <br /> First, clone the [https://github.com/artefactual-labs/am-packbuild/ am-packbuild] repo. The most recent work has been done in branch dev/packages-1.6-docker, and needs to be merged into main.<br /> <br /> Update the Makefiles available at &lt;code&gt;rpm/siegfried/Makefile&lt;/code&gt;, and &lt;code&gt;deb/siegfried/Makefile&lt;/code&gt; , and run &lt;code&gt;make&lt;/code&gt; in each directory to build the packages.<br /> <br /> === Update FPR ===<br /> <br /> This process is currently more convoluted than it needs to be, and should be simplified.<br /> <br /> This depends on FIDO having updated their PRONOM files. See [[#Update FIDO]]<br /> <br /> # Install the latest stable Archivematica release<br /> # Generate a JSON with the current version of the FPR (for use later)<br /> #* &lt;code&gt;./src/dashboard/src/manage.py dumpdata fpr &gt; fpr-current.json&lt;/code&gt;<br /> # Install the latest development version of Archivematica<br /> # Generate a blank data migration in the FPR-admin module<br /> #* E.g. &lt;code&gt;./src/dashboard/src/manage.py makemigrations --empty --name pronom_89 fpr&lt;/code&gt;<br /> # Generate the FPR migration body. This also updates the local database's FPR with the new PRONOM IDs<br /> #* E.g. &lt;code&gt;am import-pronom-ids fido/fido/conf/formats-v89.xml --output-filename archivematica/pronom89.txt&lt;/code&gt;<br /> # Copy the output into the blank migration, add RunPython operation<br /> #* &lt;code&gt;migrations.RunPython(data_migration),&lt;/code&gt;<br /> # Due to a [https://projects.artefactual.com/issues/10466#change-46673 bug], fix the imports<br /> #* Remove the &lt;code&gt;apps.get_model&lt;/code&gt; lines<br /> #* Import the models directly with &lt;code&gt;from fpr.models import Format, FormatVersion, IDRule&lt;/code&gt;<br /> # Commit<br /> # Deploy on testing pipeline<br /> # (Analysts) Update the new entries<br /> #* Move new formats to a more appropriate category<br /> #* Create rules &amp; commands<br /> #* Test with data for new formats<br /> # Generate a JSON with the updated version of the FPR on the testing pipeline<br /> #* &lt;code&gt;./src/dashboard/src/manage.py dumpdata fpr &gt; fpr-updated.json&lt;/code&gt;<br /> # Get the updates as JSON<br /> #* E.g. &lt;code&gt;am get-fpr-changes fpr-current.json fpr-updated.json src/dashboard/src/fpr/migrations/pronom_89.json&lt;/code&gt;<br /> # Update the migration to load the JSON updates<br /> #* Replace the contents of the data migration function with<br /> #** &lt;code&gt;fixture_file = os.path.join(os.path.dirname(__file__), 'pronom_89.json')&lt;/code&gt;<br /> #** &lt;code&gt;call_command('loaddata', fixture_file, app_label='fpr')&lt;/code&gt;<br /> #* Remove the direct imports from the bug<br /> #* Improvement Note: Because this is using loaddata, this will have problems if the FPR models are changed. A possible solution is to update get-fpr-changes to generate a migration instead of JSON<br /> # Commit<br /> # Merge<br /> <br /> == Tag Release ==<br /> <br /> # Add release tags to repositories<br /> #* Archivematica<br /> #* Storage Service<br /> #* FPR-admin<br /> #* appraisal tab<br /> #* transfer browser<br /> #* Others?<br /> # Create stable/#.x branch<br /> # Delete qa/#.x branch<br /> <br /> <br /> [[Category:Process documentation]]</div>

Scollazo https://wiki.archivematica.org/index.php?title=Release_Process&diff=11770 2017-03-30T16:32:58Z

<p>Scollazo: /* Package FIDO */</p> <hr /> <div>This is an outline/checklist of the process to create Archivematica &amp; Storage service releases.<br /> <br /> == Overview ==<br /> <br /> # Merge new features<br /> # Test new features<br /> # [[#Update PRONOM]]<br /> # Write documentation<br /> # Create packages<br /> # [[#Tag Release]]<br /> # Announce release<br /> <br /> <br /> == Update PRONOM ==<br /> <br /> PRONOM needs to be updated in our file identification tools, FIDO &amp; Siegfried, as well as in the FPR.<br /> <br /> === Update FIDO ===<br /> <br /> The FPR update currently use FIDO as a source for new PRONOM, since it is formatted nicer than PRONOM offers, so we depend on FIDO having updated their PRONOM. If that has not happened, we can generate a new formats-v##.xml by updating signatures manually. Artefactual can also update PRONOM and submit a PR to FIDO.<br /> <br /> # Checkout fido from https://github.com/openpreserve/fido<br /> # Update signatures<br /> #* Run &lt;code&gt;python -m fido.update_signatures&lt;/code&gt; from the fido repository root<br /> # Add:<br /> #* New signature file &lt;code&gt;fido/conf/DROID_SignatureFile-v##.xml&lt;/code&gt;<br /> #* New formats file &lt;code&gt;fido/conf/formats-v##.xml&lt;/code&gt;<br /> #* New PRONOM zip file &lt;code&gt;fido/conf/pronom-xml-v##.zip&lt;/code&gt;<br /> #* Updated &lt;code&gt;fido/conf/versions.xml&lt;/code&gt;<br /> # Remove:<br /> #* Old signature file<br /> #* Old formats file<br /> #* Old PRONOM zip file<br /> # Update &lt;code&gt;fido/fido.py&lt;/code&gt; to point to the new format_files<br /> # Create pull request. This may trigger a release from FIDO<br /> <br /> === Package FIDO ===<br /> <br /> First, clone the [https://github.com/artefactual-labs/am-packbuild/ am-packbuild] repo. The most recent work has been done in branch dev/packages-1.6-docker, and needs to be merged into main.<br /> <br /> Update the Makefiles available at &lt;code&gt;rpm/fido/Makefile&lt;/code&gt;, and &lt;code&gt;deb/fido/Makefile&lt;/code&gt; , and run &lt;code&gt;make&lt;/code&gt; in each directory to build the packages.<br /> <br /> === Package Siegfried ===<br /> <br /> Add packaging instructions here...<br /> <br /> === Update FPR ===<br /> <br /> This process is currently more convoluted than it needs to be, and should be simplified.<br /> <br /> This depends on FIDO having updated their PRONOM files. See [[#Update FIDO]]<br /> <br /> # Install the latest stable Archivematica release<br /> # Generate a JSON with the current version of the FPR (for use later)<br /> #* &lt;code&gt;./src/dashboard/src/manage.py dumpdata fpr &gt; fpr-current.json&lt;/code&gt;<br /> # Install the latest development version of Archivematica<br /> # Generate a blank data migration in the FPR-admin module<br /> #* E.g. &lt;code&gt;./src/dashboard/src/manage.py makemigrations --empty --name pronom_89 fpr&lt;/code&gt;<br /> # Generate the FPR migration body. This also updates the local database's FPR with the new PRONOM IDs<br /> #* E.g. &lt;code&gt;am import-pronom-ids fido/fido/conf/formats-v89.xml --output-filename archivematica/pronom89.txt&lt;/code&gt;<br /> # Copy the output into the blank migration, add RunPython operation<br /> #* &lt;code&gt;migrations.RunPython(data_migration),&lt;/code&gt;<br /> # Due to a [https://projects.artefactual.com/issues/10466#change-46673 bug], fix the imports<br /> #* Remove the &lt;code&gt;apps.get_model&lt;/code&gt; lines<br /> #* Import the models directly with &lt;code&gt;from fpr.models import Format, FormatVersion, IDRule&lt;/code&gt;<br /> # Commit<br /> # Deploy on testing pipeline<br /> # (Analysts) Update the new entries<br /> #* Move new formats to a more appropriate category<br /> #* Create rules &amp; commands<br /> #* Test with data for new formats<br /> # Generate a JSON with the updated version of the FPR on the testing pipeline<br /> #* &lt;code&gt;./src/dashboard/src/manage.py dumpdata fpr &gt; fpr-updated.json&lt;/code&gt;<br /> # Get the updates as JSON<br /> #* E.g. &lt;code&gt;am get-fpr-changes fpr-current.json fpr-updated.json src/dashboard/src/fpr/migrations/pronom_89.json&lt;/code&gt;<br /> # Update the migration to load the JSON updates<br /> #* Replace the contents of the data migration function with<br /> #** &lt;code&gt;fixture_file = os.path.join(os.path.dirname(__file__), 'pronom_89.json')&lt;/code&gt;<br /> #** &lt;code&gt;call_command('loaddata', fixture_file, app_label='fpr')&lt;/code&gt;<br /> #* Remove the direct imports from the bug<br /> #* Improvement Note: Because this is using loaddata, this will have problems if the FPR models are changed. A possible solution is to update get-fpr-changes to generate a migration instead of JSON<br /> # Commit<br /> # Merge<br /> <br /> == Tag Release ==<br /> <br /> # Add release tags to repositories<br /> #* Archivematica<br /> #* Storage Service<br /> #* FPR-admin<br /> #* appraisal tab<br /> #* transfer browser<br /> #* Others?<br /> # Create stable/#.x branch<br /> # Delete qa/#.x branch<br /> <br /> <br /> [[Category:Process documentation]]</div>

Scollazo https://wiki.archivematica.org/index.php?title=Improvements/CentOS-RedHat_support/Installation&diff=11468 2016-09-06T11:08:25Z

<p>Scollazo: /* Known bugs / Caveats */</p> <hr /> <div>'''IMPORTANT NOTE''' These instructions will create an installation of a QA version of Archivematica and is not recommended for production use yet.<br /> <br /> = Quick install using [https://www.vagrantup.com/ Vagrant] =<br /> <br /> This method will have you up and running in no time, using a virtual machine provisioned with vagrant.<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, clone the am-packbuild repo:&lt;/p&gt;<br /> &lt;pre&gt;git clone https://github.com/artefactual-labs/am-packbuild/&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;cd into the rpm-testing directory&lt;/p&gt;<br /> &lt;pre&gt;cd am-packbuild/rpm-testing/&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;run vagrant&lt;/p&gt;<br /> &lt;pre&gt;vagrant up&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> After the install, the Archivematica Dashboard will be avaliable on port 81 of the vagrant deployed box, and the Storage Service, in port 8001.<br /> <br /> You can log in the vm over ssh running &lt;code&gt;vagrant ssh&lt;/code&gt;<br /> <br /> = Step by step Install =<br /> <br /> == Prerequisites ==<br /> <br /> === Extra repos ===<br /> <br /> Some repositories need to be installed in order to fullfill the installation procedure:<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;Extra packages for enterprise linux&lt;/p&gt;<br /> &lt;pre&gt;sudo yum install -y epel-release&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Elasticsearch&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch<br /> sudo -u root bash -c 'cat &amp;lt;&amp;lt; EOF &amp;gt; /etc/yum.repos.d/elasticsearch.repo<br /> [elasticsearch-1.7]<br /> name=Elasticsearch repository for 1.7 packages<br /> baseurl=https://packages.elastic.co/elasticsearch/1.7/centos<br /> gpgcheck=1<br /> gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch<br /> enabled=1<br /> EOF'&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Archivematica&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root bash -c 'cat &amp;lt;&amp;lt; EOF &amp;gt; /etc/yum.repos.d/archivematica.repo<br /> [archivematica]<br /> name=archivematica<br /> baseurl=https://packages.archivematica.org/1.5.x/centos<br /> gpgcheck=0<br /> enabled=1<br /> EOF'&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> === Service depencencies. ===<br /> <br /> Common services like elasticsearch, mariadb and gearmand should be installed and enabled before the archivematica install. It can be done with:<br /> <br /> &lt;pre&gt;sudo -u root yum install -y java-1.8.0-openjdk-headless elasticsearch mariadb-server gearmand<br /> sudo -u root systemctl enable elasticsearch<br /> sudo -u root systemctl start elasticsearch<br /> sudo -u root systemctl enable mariadb<br /> sudo -u root systemctl start mariadb<br /> sudo -u root systemctl enable gearmand<br /> sudo -u root systemctl start gearmand&lt;/pre&gt;<br /> == Installing Archivematica Storage Service ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, we install the packages:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y python-pip archivematica-storage-service&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;After the package is installed, we need to populate the sqlite database, and collect some static files used by django. Those tasks must be run as “archivematica” user.&lt;/p&gt;<br /> &lt;pre&gt;sudo -u archivematica bash -c &amp;quot; \<br /> set -a -e -x<br /> source /etc/sysconfig/archivematica-storage-service<br /> cd /usr/share/archivematica/storage-service<br /> /usr/lib/python2.7/archivematica/storage-service/bin/python manage.py migrate<br /> /usr/lib/python2.7/archivematica/storage-service/bin/python manage.py collectstatic --noinput<br /> &amp;quot;;&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;And now, we enable and start the archivematica-storage-service and it’s nginx frontend&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-storage-service<br /> sudo -u root systemctl start archivematica-storage-service<br /> sudo -u root systemctl enable nginx<br /> sudo -u root systemctl start nginx&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> The storage service will be avaliable at http://&lt;ip&gt;:8001<br /> <br /> == Installing Archivematica Dashboard and MCP Server ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, install the pacakges:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y archivematica-common archivematica-mcp-server archivematica-dashboard&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Create user and mysql database with:&lt;/p&gt;<br /> &lt;pre&gt;sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;DROP DATABASE IF EXISTS MCP; CREATE DATABASE MCP CHARACTER SET utf8 COLLATE utf8_unicode_ci;&amp;quot;<br /> sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;CREATE USER 'archivematica'@'localhost' IDENTIFIED BY 'demo';&amp;quot;<br /> sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;GRANT ALL ON MCP.* TO 'archivematica'@'localhost';&amp;quot;&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;And as archivematica user, run migrations:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u archivematica bash -c &amp;quot; \<br /> set -a -e -x<br /> source /etc/sysconfig/archivematica-dashboard<br /> cd /usr/share/archivematica/dashboard<br /> /usr/lib/python2.7/archivematica/dashboard/bin/python manage.py syncdb --noinput<br /> &amp;quot;;<br /> &lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Start and enable services:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-mcp-server<br /> sudo -u root systemctl start archivematica-mcp-server<br /> sudo -u root systemctl enable archivematica-dashboard<br /> sudo -u root systemctl start archivematica-dashboard&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Reload nginx in order to load the dashboard config file:&lt;br /&gt;<br /> &lt;pre&gt;<br /> sudo -u root systemctl reload nginx<br /> &lt;/pre&gt;<br /> &lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> The dashboard will be avaliable at http://ip:81<br /> <br /> == Installing Archivematica MCP Client ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, we need to add some extra repos with the MCP Client dependencies:&lt;/p&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Archivematica supplied external packages:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root bash -c 'cat &lt;&lt; EOF &gt; /etc/yum.repos.d/archivematica-extras.repo<br /> [archivematica-extras]<br /> name=archivematica-extras<br /> baseurl=https://packages.archivematica.org/1.5.x/centos-extras<br /> gpgcheck=0<br /> enabled=1<br /> EOF'<br /> &lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Nux multimedia repo&lt;/p&gt;<br /> &lt;pre&gt;sudo rpm -Uvh https://li.nux.ro/download/nux/dextop/el7/x86_64/nux-dextop-release-0-5.el7.nux.noarch.rpm&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Forensic tools repo&lt;/p&gt;<br /> &lt;pre&gt;sudo rpm -Uvh https://forensics.cert.org/cert-forensics-tools-release-el7.rpm&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Then, install the package:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y archivematica-mcp-client&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;The MCP Client expect some programs in certain paths, so we put things in place:&lt;/p&gt;<br /> &lt;pre&gt;sudo cp /usr/bin/clamscan /usr/bin/clamdscan<br /> sudo ln -s /usr/bin/7za /usr/bin/7z&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;After that, we can enable and start services&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-mcp-client<br /> sudo -u root systemctl start archivematica-mcp-client<br /> sudo -u root systemctl enable fits-nailgun<br /> sudo -u root systemctl start fits-nailgun&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> == Finalizing Installation ==<br /> <br /> The dashboard will be available on port 81, and the storage service on port 8001. You will need to complete the installation by opening up the dashboard in a web browser, and filling in the form you are presented with. On the 2nd page of the installer, you are asked for information about the storage service. You will need to log into the storage service and find the api key that was generated for your user (in admin-&gt;users).<br /> <br /> == Configuration ==<br /> <br /> Each service have a configuration file in /etc/sysconfig/archivematica-''packagename''<br /> <br /> == Known bugs / Caveats ==<br /> <br /> * If IPv6 is disabled, Nginx may refuse to start. If that is the case make sure that the listen directives used under /etc/nginx are not using IPv6 addresses like [::]:80.<br /> * In RHEL 7 , you might need to enable repo rhel-7-server-eus-optional-rpms with<br /> &lt;pre&gt;<br /> sudo subscription-manager repos --enable rhel-7-server-eus-optional-rpms<br /> &lt;/pre&gt;<br /> * If you find a bug, please let us know [https://github.com/artefactual-labs/am-packbuild/issues here]</div>

Scollazo https://wiki.archivematica.org/index.php?title=Improvements/CentOS-RedHat_support/Installation&diff=11467 2016-09-06T11:04:56Z

<p>Scollazo: /* Known bugs / Caveats */</p> <hr /> <div>'''IMPORTANT NOTE''' These instructions will create an installation of a QA version of Archivematica and is not recommended for production use yet.<br /> <br /> = Quick install using [https://www.vagrantup.com/ Vagrant] =<br /> <br /> This method will have you up and running in no time, using a virtual machine provisioned with vagrant.<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, clone the am-packbuild repo:&lt;/p&gt;<br /> &lt;pre&gt;git clone https://github.com/artefactual-labs/am-packbuild/&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;cd into the rpm-testing directory&lt;/p&gt;<br /> &lt;pre&gt;cd am-packbuild/rpm-testing/&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;run vagrant&lt;/p&gt;<br /> &lt;pre&gt;vagrant up&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> After the install, the Archivematica Dashboard will be avaliable on port 81 of the vagrant deployed box, and the Storage Service, in port 8001.<br /> <br /> You can log in the vm over ssh running &lt;code&gt;vagrant ssh&lt;/code&gt;<br /> <br /> = Step by step Install =<br /> <br /> == Prerequisites ==<br /> <br /> === Extra repos ===<br /> <br /> Some repositories need to be installed in order to fullfill the installation procedure:<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;Extra packages for enterprise linux&lt;/p&gt;<br /> &lt;pre&gt;sudo yum install -y epel-release&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Elasticsearch&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch<br /> sudo -u root bash -c 'cat &amp;lt;&amp;lt; EOF &amp;gt; /etc/yum.repos.d/elasticsearch.repo<br /> [elasticsearch-1.7]<br /> name=Elasticsearch repository for 1.7 packages<br /> baseurl=https://packages.elastic.co/elasticsearch/1.7/centos<br /> gpgcheck=1<br /> gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch<br /> enabled=1<br /> EOF'&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Archivematica&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root bash -c 'cat &amp;lt;&amp;lt; EOF &amp;gt; /etc/yum.repos.d/archivematica.repo<br /> [archivematica]<br /> name=archivematica<br /> baseurl=https://packages.archivematica.org/1.5.x/centos<br /> gpgcheck=0<br /> enabled=1<br /> EOF'&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> === Service depencencies. ===<br /> <br /> Common services like elasticsearch, mariadb and gearmand should be installed and enabled before the archivematica install. It can be done with:<br /> <br /> &lt;pre&gt;sudo -u root yum install -y java-1.8.0-openjdk-headless elasticsearch mariadb-server gearmand<br /> sudo -u root systemctl enable elasticsearch<br /> sudo -u root systemctl start elasticsearch<br /> sudo -u root systemctl enable mariadb<br /> sudo -u root systemctl start mariadb<br /> sudo -u root systemctl enable gearmand<br /> sudo -u root systemctl start gearmand&lt;/pre&gt;<br /> == Installing Archivematica Storage Service ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, we install the packages:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y python-pip archivematica-storage-service&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;After the package is installed, we need to populate the sqlite database, and collect some static files used by django. Those tasks must be run as “archivematica” user.&lt;/p&gt;<br /> &lt;pre&gt;sudo -u archivematica bash -c &amp;quot; \<br /> set -a -e -x<br /> source /etc/sysconfig/archivematica-storage-service<br /> cd /usr/share/archivematica/storage-service<br /> /usr/lib/python2.7/archivematica/storage-service/bin/python manage.py migrate<br /> /usr/lib/python2.7/archivematica/storage-service/bin/python manage.py collectstatic --noinput<br /> &amp;quot;;&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;And now, we enable and start the archivematica-storage-service and it’s nginx frontend&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-storage-service<br /> sudo -u root systemctl start archivematica-storage-service<br /> sudo -u root systemctl enable nginx<br /> sudo -u root systemctl start nginx&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> The storage service will be avaliable at http://&lt;ip&gt;:8001<br /> <br /> == Installing Archivematica Dashboard and MCP Server ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, install the pacakges:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y archivematica-common archivematica-mcp-server archivematica-dashboard&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Create user and mysql database with:&lt;/p&gt;<br /> &lt;pre&gt;sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;DROP DATABASE IF EXISTS MCP; CREATE DATABASE MCP CHARACTER SET utf8 COLLATE utf8_unicode_ci;&amp;quot;<br /> sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;CREATE USER 'archivematica'@'localhost' IDENTIFIED BY 'demo';&amp;quot;<br /> sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;GRANT ALL ON MCP.* TO 'archivematica'@'localhost';&amp;quot;&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;And as archivematica user, run migrations:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u archivematica bash -c &amp;quot; \<br /> set -a -e -x<br /> source /etc/sysconfig/archivematica-dashboard<br /> cd /usr/share/archivematica/dashboard<br /> /usr/lib/python2.7/archivematica/dashboard/bin/python manage.py syncdb --noinput<br /> &amp;quot;;<br /> &lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Start and enable services:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-mcp-server<br /> sudo -u root systemctl start archivematica-mcp-server<br /> sudo -u root systemctl enable archivematica-dashboard<br /> sudo -u root systemctl start archivematica-dashboard&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Reload nginx in order to load the dashboard config file:&lt;br /&gt;<br /> &lt;pre&gt;<br /> sudo -u root systemctl reload nginx<br /> &lt;/pre&gt;<br /> &lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> The dashboard will be avaliable at http://ip:81<br /> <br /> == Installing Archivematica MCP Client ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, we need to add some extra repos with the MCP Client dependencies:&lt;/p&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Archivematica supplied external packages:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root bash -c 'cat &lt;&lt; EOF &gt; /etc/yum.repos.d/archivematica-extras.repo<br /> [archivematica-extras]<br /> name=archivematica-extras<br /> baseurl=https://packages.archivematica.org/1.5.x/centos-extras<br /> gpgcheck=0<br /> enabled=1<br /> EOF'<br /> &lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Nux multimedia repo&lt;/p&gt;<br /> &lt;pre&gt;sudo rpm -Uvh https://li.nux.ro/download/nux/dextop/el7/x86_64/nux-dextop-release-0-5.el7.nux.noarch.rpm&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Forensic tools repo&lt;/p&gt;<br /> &lt;pre&gt;sudo rpm -Uvh https://forensics.cert.org/cert-forensics-tools-release-el7.rpm&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Then, install the package:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y archivematica-mcp-client&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;The MCP Client expect some programs in certain paths, so we put things in place:&lt;/p&gt;<br /> &lt;pre&gt;sudo cp /usr/bin/clamscan /usr/bin/clamdscan<br /> sudo ln -s /usr/bin/7za /usr/bin/7z&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;After that, we can enable and start services&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-mcp-client<br /> sudo -u root systemctl start archivematica-mcp-client<br /> sudo -u root systemctl enable fits-nailgun<br /> sudo -u root systemctl start fits-nailgun&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> == Finalizing Installation ==<br /> <br /> The dashboard will be available on port 81, and the storage service on port 8001. You will need to complete the installation by opening up the dashboard in a web browser, and filling in the form you are presented with. On the 2nd page of the installer, you are asked for information about the storage service. You will need to log into the storage service and find the api key that was generated for your user (in admin-&gt;users).<br /> <br /> == Configuration ==<br /> <br /> Each service have a configuration file in /etc/sysconfig/archivematica-''packagename''<br /> <br /> == Known bugs / Caveats ==<br /> <br /> * If IPv6 is disabled, Nginx may refuse to start. If that is the case make sure that the listen directives used under /etc/nginx are not using IPv6 addresses like [::]:80.<br /> * In RHEL 7 , you might need to enable repo rhel-7-server-eus-optional-rpms with<br /> &lt;pre&gt;<br /> subscription-manager repos --enable rhel-7-server-eus-optional-rpms<br /> &lt;/pre&gt;<br /> * If you find a bug, please let us know [https://github.com/artefactual-labs/am-packbuild/issues here]</div>

Scollazo https://wiki.archivematica.org/index.php?title=Improvements/CentOS-RedHat_support/Installation&diff=11466 2016-09-06T10:47:42Z

<p>Scollazo: /* Installing Archivematica Dashboard and MCP Server */</p> <hr /> <div>'''IMPORTANT NOTE''' These instructions will create an installation of a QA version of Archivematica and is not recommended for production use yet.<br /> <br /> = Quick install using [https://www.vagrantup.com/ Vagrant] =<br /> <br /> This method will have you up and running in no time, using a virtual machine provisioned with vagrant.<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, clone the am-packbuild repo:&lt;/p&gt;<br /> &lt;pre&gt;git clone https://github.com/artefactual-labs/am-packbuild/&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;cd into the rpm-testing directory&lt;/p&gt;<br /> &lt;pre&gt;cd am-packbuild/rpm-testing/&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;run vagrant&lt;/p&gt;<br /> &lt;pre&gt;vagrant up&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> After the install, the Archivematica Dashboard will be avaliable on port 81 of the vagrant deployed box, and the Storage Service, in port 8001.<br /> <br /> You can log in the vm over ssh running &lt;code&gt;vagrant ssh&lt;/code&gt;<br /> <br /> = Step by step Install =<br /> <br /> == Prerequisites ==<br /> <br /> === Extra repos ===<br /> <br /> Some repositories need to be installed in order to fullfill the installation procedure:<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;Extra packages for enterprise linux&lt;/p&gt;<br /> &lt;pre&gt;sudo yum install -y epel-release&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Elasticsearch&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch<br /> sudo -u root bash -c 'cat &amp;lt;&amp;lt; EOF &amp;gt; /etc/yum.repos.d/elasticsearch.repo<br /> [elasticsearch-1.7]<br /> name=Elasticsearch repository for 1.7 packages<br /> baseurl=https://packages.elastic.co/elasticsearch/1.7/centos<br /> gpgcheck=1<br /> gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch<br /> enabled=1<br /> EOF'&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Archivematica&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root bash -c 'cat &amp;lt;&amp;lt; EOF &amp;gt; /etc/yum.repos.d/archivematica.repo<br /> [archivematica]<br /> name=archivematica<br /> baseurl=https://packages.archivematica.org/1.5.x/centos<br /> gpgcheck=0<br /> enabled=1<br /> EOF'&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> === Service depencencies. ===<br /> <br /> Common services like elasticsearch, mariadb and gearmand should be installed and enabled before the archivematica install. It can be done with:<br /> <br /> &lt;pre&gt;sudo -u root yum install -y java-1.8.0-openjdk-headless elasticsearch mariadb-server gearmand<br /> sudo -u root systemctl enable elasticsearch<br /> sudo -u root systemctl start elasticsearch<br /> sudo -u root systemctl enable mariadb<br /> sudo -u root systemctl start mariadb<br /> sudo -u root systemctl enable gearmand<br /> sudo -u root systemctl start gearmand&lt;/pre&gt;<br /> == Installing Archivematica Storage Service ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, we install the packages:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y python-pip archivematica-storage-service&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;After the package is installed, we need to populate the sqlite database, and collect some static files used by django. Those tasks must be run as “archivematica” user.&lt;/p&gt;<br /> &lt;pre&gt;sudo -u archivematica bash -c &amp;quot; \<br /> set -a -e -x<br /> source /etc/sysconfig/archivematica-storage-service<br /> cd /usr/share/archivematica/storage-service<br /> /usr/lib/python2.7/archivematica/storage-service/bin/python manage.py migrate<br /> /usr/lib/python2.7/archivematica/storage-service/bin/python manage.py collectstatic --noinput<br /> &amp;quot;;&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;And now, we enable and start the archivematica-storage-service and it’s nginx frontend&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-storage-service<br /> sudo -u root systemctl start archivematica-storage-service<br /> sudo -u root systemctl enable nginx<br /> sudo -u root systemctl start nginx&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> The storage service will be avaliable at http://&lt;ip&gt;:8001<br /> <br /> == Installing Archivematica Dashboard and MCP Server ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, install the pacakges:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y archivematica-common archivematica-mcp-server archivematica-dashboard&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Create user and mysql database with:&lt;/p&gt;<br /> &lt;pre&gt;sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;DROP DATABASE IF EXISTS MCP; CREATE DATABASE MCP CHARACTER SET utf8 COLLATE utf8_unicode_ci;&amp;quot;<br /> sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;CREATE USER 'archivematica'@'localhost' IDENTIFIED BY 'demo';&amp;quot;<br /> sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;GRANT ALL ON MCP.* TO 'archivematica'@'localhost';&amp;quot;&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;And as archivematica user, run migrations:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u archivematica bash -c &amp;quot; \<br /> set -a -e -x<br /> source /etc/sysconfig/archivematica-dashboard<br /> cd /usr/share/archivematica/dashboard<br /> /usr/lib/python2.7/archivematica/dashboard/bin/python manage.py syncdb --noinput<br /> &amp;quot;;<br /> &lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Start and enable services:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-mcp-server<br /> sudo -u root systemctl start archivematica-mcp-server<br /> sudo -u root systemctl enable archivematica-dashboard<br /> sudo -u root systemctl start archivematica-dashboard&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Reload nginx in order to load the dashboard config file:&lt;br /&gt;<br /> &lt;pre&gt;<br /> sudo -u root systemctl reload nginx<br /> &lt;/pre&gt;<br /> &lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> The dashboard will be avaliable at http://ip:81<br /> <br /> == Installing Archivematica MCP Client ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, we need to add some extra repos with the MCP Client dependencies:&lt;/p&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Archivematica supplied external packages:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root bash -c 'cat &lt;&lt; EOF &gt; /etc/yum.repos.d/archivematica-extras.repo<br /> [archivematica-extras]<br /> name=archivematica-extras<br /> baseurl=https://packages.archivematica.org/1.5.x/centos-extras<br /> gpgcheck=0<br /> enabled=1<br /> EOF'<br /> &lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Nux multimedia repo&lt;/p&gt;<br /> &lt;pre&gt;sudo rpm -Uvh https://li.nux.ro/download/nux/dextop/el7/x86_64/nux-dextop-release-0-5.el7.nux.noarch.rpm&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Forensic tools repo&lt;/p&gt;<br /> &lt;pre&gt;sudo rpm -Uvh https://forensics.cert.org/cert-forensics-tools-release-el7.rpm&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Then, install the package:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y archivematica-mcp-client&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;The MCP Client expect some programs in certain paths, so we put things in place:&lt;/p&gt;<br /> &lt;pre&gt;sudo cp /usr/bin/clamscan /usr/bin/clamdscan<br /> sudo ln -s /usr/bin/7za /usr/bin/7z&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;After that, we can enable and start services&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-mcp-client<br /> sudo -u root systemctl start archivematica-mcp-client<br /> sudo -u root systemctl enable fits-nailgun<br /> sudo -u root systemctl start fits-nailgun&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> == Finalizing Installation ==<br /> <br /> The dashboard will be available on port 81, and the storage service on port 8001. You will need to complete the installation by opening up the dashboard in a web browser, and filling in the form you are presented with. On the 2nd page of the installer, you are asked for information about the storage service. You will need to log into the storage service and find the api key that was generated for your user (in admin-&gt;users).<br /> <br /> == Configuration ==<br /> <br /> Each service have a configuration file in /etc/sysconfig/archivematica-''packagename''<br /> <br /> == Known bugs / Caveats ==<br /> <br /> * If IPv6 is disabled, Nginx may refuse to start. If that is the case make sure that the listen directives used under /etc/nginx are not using IPv6 addresses like [::]:80.<br /> * If you find a bug, please let us know [https://github.com/artefactual-labs/am-packbuild/issues here]</div>

Scollazo https://wiki.archivematica.org/index.php?title=Improvements/CentOS-RedHat_support/Installation&diff=11465 2016-09-06T10:47:01Z

<p>Scollazo: /* Installing Archivematica Dashboard and MCP Server */</p> <hr /> <div>'''IMPORTANT NOTE''' These instructions will create an installation of a QA version of Archivematica and is not recommended for production use yet.<br /> <br /> = Quick install using [https://www.vagrantup.com/ Vagrant] =<br /> <br /> This method will have you up and running in no time, using a virtual machine provisioned with vagrant.<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, clone the am-packbuild repo:&lt;/p&gt;<br /> &lt;pre&gt;git clone https://github.com/artefactual-labs/am-packbuild/&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;cd into the rpm-testing directory&lt;/p&gt;<br /> &lt;pre&gt;cd am-packbuild/rpm-testing/&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;run vagrant&lt;/p&gt;<br /> &lt;pre&gt;vagrant up&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> After the install, the Archivematica Dashboard will be avaliable on port 81 of the vagrant deployed box, and the Storage Service, in port 8001.<br /> <br /> You can log in the vm over ssh running &lt;code&gt;vagrant ssh&lt;/code&gt;<br /> <br /> = Step by step Install =<br /> <br /> == Prerequisites ==<br /> <br /> === Extra repos ===<br /> <br /> Some repositories need to be installed in order to fullfill the installation procedure:<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;Extra packages for enterprise linux&lt;/p&gt;<br /> &lt;pre&gt;sudo yum install -y epel-release&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Elasticsearch&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch<br /> sudo -u root bash -c 'cat &amp;lt;&amp;lt; EOF &amp;gt; /etc/yum.repos.d/elasticsearch.repo<br /> [elasticsearch-1.7]<br /> name=Elasticsearch repository for 1.7 packages<br /> baseurl=https://packages.elastic.co/elasticsearch/1.7/centos<br /> gpgcheck=1<br /> gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch<br /> enabled=1<br /> EOF'&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Archivematica&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root bash -c 'cat &amp;lt;&amp;lt; EOF &amp;gt; /etc/yum.repos.d/archivematica.repo<br /> [archivematica]<br /> name=archivematica<br /> baseurl=https://packages.archivematica.org/1.5.x/centos<br /> gpgcheck=0<br /> enabled=1<br /> EOF'&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> === Service depencencies. ===<br /> <br /> Common services like elasticsearch, mariadb and gearmand should be installed and enabled before the archivematica install. It can be done with:<br /> <br /> &lt;pre&gt;sudo -u root yum install -y java-1.8.0-openjdk-headless elasticsearch mariadb-server gearmand<br /> sudo -u root systemctl enable elasticsearch<br /> sudo -u root systemctl start elasticsearch<br /> sudo -u root systemctl enable mariadb<br /> sudo -u root systemctl start mariadb<br /> sudo -u root systemctl enable gearmand<br /> sudo -u root systemctl start gearmand&lt;/pre&gt;<br /> == Installing Archivematica Storage Service ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, we install the packages:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y python-pip archivematica-storage-service&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;After the package is installed, we need to populate the sqlite database, and collect some static files used by django. Those tasks must be run as “archivematica” user.&lt;/p&gt;<br /> &lt;pre&gt;sudo -u archivematica bash -c &amp;quot; \<br /> set -a -e -x<br /> source /etc/sysconfig/archivematica-storage-service<br /> cd /usr/share/archivematica/storage-service<br /> /usr/lib/python2.7/archivematica/storage-service/bin/python manage.py migrate<br /> /usr/lib/python2.7/archivematica/storage-service/bin/python manage.py collectstatic --noinput<br /> &amp;quot;;&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;And now, we enable and start the archivematica-storage-service and it’s nginx frontend&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-storage-service<br /> sudo -u root systemctl start archivematica-storage-service<br /> sudo -u root systemctl enable nginx<br /> sudo -u root systemctl start nginx&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> The storage service will be avaliable at http://&lt;ip&gt;:8001<br /> <br /> == Installing Archivematica Dashboard and MCP Server ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, install the pacakges:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y archivematica-common archivematica-mcp-server archivematica-dashboard&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Create user and mysql database with:&lt;/p&gt;<br /> &lt;pre&gt;sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;DROP DATABASE IF EXISTS MCP; CREATE DATABASE MCP CHARACTER SET utf8 COLLATE utf8_unicode_ci;&amp;quot;<br /> sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;CREATE USER 'archivematica'@'localhost' IDENTIFIED BY 'demo';&amp;quot;<br /> sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;GRANT ALL ON MCP.* TO 'archivematica'@'localhost';&amp;quot;&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;And as archivematica user, run migrations:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u archivematica bash -c &amp;quot; \<br /> set -a -e -x<br /> source /etc/sysconfig/archivematica-dashboard<br /> cd /usr/share/archivematica/dashboard<br /> /usr/lib/python2.7/archivematica/dashboard/bin/python manage.py syncdb --noinput<br /> &amp;quot;;<br /> &lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Start and enable services:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-mcp-server<br /> sudo -u root systemctl start archivematica-mcp-server<br /> sudo -u root systemctl enable archivematica-dashboard<br /> sudo -u root systemctl start archivematica-dashboard&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Reload nginx in order to load the dashboard config file:&lt;br /&gt;<br /> &lt;pre&gt;<br /> sudo -u root systemctl reload nginx&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;<br /> &lt;/pre&gt;<br /> The dashboard will be avaliable at http://ip:81<br /> <br /> == Installing Archivematica MCP Client ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, we need to add some extra repos with the MCP Client dependencies:&lt;/p&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Archivematica supplied external packages:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root bash -c 'cat &lt;&lt; EOF &gt; /etc/yum.repos.d/archivematica-extras.repo<br /> [archivematica-extras]<br /> name=archivematica-extras<br /> baseurl=https://packages.archivematica.org/1.5.x/centos-extras<br /> gpgcheck=0<br /> enabled=1<br /> EOF'<br /> &lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Nux multimedia repo&lt;/p&gt;<br /> &lt;pre&gt;sudo rpm -Uvh https://li.nux.ro/download/nux/dextop/el7/x86_64/nux-dextop-release-0-5.el7.nux.noarch.rpm&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Forensic tools repo&lt;/p&gt;<br /> &lt;pre&gt;sudo rpm -Uvh https://forensics.cert.org/cert-forensics-tools-release-el7.rpm&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Then, install the package:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y archivematica-mcp-client&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;The MCP Client expect some programs in certain paths, so we put things in place:&lt;/p&gt;<br /> &lt;pre&gt;sudo cp /usr/bin/clamscan /usr/bin/clamdscan<br /> sudo ln -s /usr/bin/7za /usr/bin/7z&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;After that, we can enable and start services&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-mcp-client<br /> sudo -u root systemctl start archivematica-mcp-client<br /> sudo -u root systemctl enable fits-nailgun<br /> sudo -u root systemctl start fits-nailgun&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> == Finalizing Installation ==<br /> <br /> The dashboard will be available on port 81, and the storage service on port 8001. You will need to complete the installation by opening up the dashboard in a web browser, and filling in the form you are presented with. On the 2nd page of the installer, you are asked for information about the storage service. You will need to log into the storage service and find the api key that was generated for your user (in admin-&gt;users).<br /> <br /> == Configuration ==<br /> <br /> Each service have a configuration file in /etc/sysconfig/archivematica-''packagename''<br /> <br /> == Known bugs / Caveats ==<br /> <br /> * If IPv6 is disabled, Nginx may refuse to start. If that is the case make sure that the listen directives used under /etc/nginx are not using IPv6 addresses like [::]:80.<br /> * If you find a bug, please let us know [https://github.com/artefactual-labs/am-packbuild/issues here]</div>

Scollazo https://wiki.archivematica.org/index.php?title=Improvements/CentOS-RedHat_support/Installation&diff=11464 2016-09-06T10:46:10Z

<p>Scollazo: /* Installing Archivematica Dashboard and MCP Server */</p> <hr /> <div>'''IMPORTANT NOTE''' These instructions will create an installation of a QA version of Archivematica and is not recommended for production use yet.<br /> <br /> = Quick install using [https://www.vagrantup.com/ Vagrant] =<br /> <br /> This method will have you up and running in no time, using a virtual machine provisioned with vagrant.<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, clone the am-packbuild repo:&lt;/p&gt;<br /> &lt;pre&gt;git clone https://github.com/artefactual-labs/am-packbuild/&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;cd into the rpm-testing directory&lt;/p&gt;<br /> &lt;pre&gt;cd am-packbuild/rpm-testing/&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;run vagrant&lt;/p&gt;<br /> &lt;pre&gt;vagrant up&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> After the install, the Archivematica Dashboard will be avaliable on port 81 of the vagrant deployed box, and the Storage Service, in port 8001.<br /> <br /> You can log in the vm over ssh running &lt;code&gt;vagrant ssh&lt;/code&gt;<br /> <br /> = Step by step Install =<br /> <br /> == Prerequisites ==<br /> <br /> === Extra repos ===<br /> <br /> Some repositories need to be installed in order to fullfill the installation procedure:<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;Extra packages for enterprise linux&lt;/p&gt;<br /> &lt;pre&gt;sudo yum install -y epel-release&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Elasticsearch&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch<br /> sudo -u root bash -c 'cat &amp;lt;&amp;lt; EOF &amp;gt; /etc/yum.repos.d/elasticsearch.repo<br /> [elasticsearch-1.7]<br /> name=Elasticsearch repository for 1.7 packages<br /> baseurl=https://packages.elastic.co/elasticsearch/1.7/centos<br /> gpgcheck=1<br /> gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch<br /> enabled=1<br /> EOF'&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Archivematica&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root bash -c 'cat &amp;lt;&amp;lt; EOF &amp;gt; /etc/yum.repos.d/archivematica.repo<br /> [archivematica]<br /> name=archivematica<br /> baseurl=https://packages.archivematica.org/1.5.x/centos<br /> gpgcheck=0<br /> enabled=1<br /> EOF'&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> === Service depencencies. ===<br /> <br /> Common services like elasticsearch, mariadb and gearmand should be installed and enabled before the archivematica install. It can be done with:<br /> <br /> &lt;pre&gt;sudo -u root yum install -y java-1.8.0-openjdk-headless elasticsearch mariadb-server gearmand<br /> sudo -u root systemctl enable elasticsearch<br /> sudo -u root systemctl start elasticsearch<br /> sudo -u root systemctl enable mariadb<br /> sudo -u root systemctl start mariadb<br /> sudo -u root systemctl enable gearmand<br /> sudo -u root systemctl start gearmand&lt;/pre&gt;<br /> == Installing Archivematica Storage Service ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, we install the packages:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y python-pip archivematica-storage-service&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;After the package is installed, we need to populate the sqlite database, and collect some static files used by django. Those tasks must be run as “archivematica” user.&lt;/p&gt;<br /> &lt;pre&gt;sudo -u archivematica bash -c &amp;quot; \<br /> set -a -e -x<br /> source /etc/sysconfig/archivematica-storage-service<br /> cd /usr/share/archivematica/storage-service<br /> /usr/lib/python2.7/archivematica/storage-service/bin/python manage.py migrate<br /> /usr/lib/python2.7/archivematica/storage-service/bin/python manage.py collectstatic --noinput<br /> &amp;quot;;&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;And now, we enable and start the archivematica-storage-service and it’s nginx frontend&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-storage-service<br /> sudo -u root systemctl start archivematica-storage-service<br /> sudo -u root systemctl enable nginx<br /> sudo -u root systemctl start nginx&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> The storage service will be avaliable at http://&lt;ip&gt;:8001<br /> <br /> == Installing Archivematica Dashboard and MCP Server ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, install the pacakges:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y archivematica-common archivematica-mcp-server archivematica-dashboard&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Create and populate the mysql database with:&lt;/p&gt;<br /> &lt;pre&gt;sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;DROP DATABASE IF EXISTS MCP; CREATE DATABASE MCP CHARACTER SET utf8 COLLATE utf8_unicode_ci;&amp;quot;<br /> sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;CREATE USER 'archivematica'@'localhost' IDENTIFIED BY 'demo';&amp;quot;<br /> sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;GRANT ALL ON MCP.* TO 'archivematica'@'localhost';&amp;quot;&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;And as archivematica user, run migrations:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u archivematica bash -c &amp;quot; \<br /> set -a -e -x<br /> source /etc/sysconfig/archivematica-dashboard<br /> cd /usr/share/archivematica/dashboard<br /> /usr/lib/python2.7/archivematica/dashboard/bin/python manage.py syncdb --noinput<br /> &amp;quot;;<br /> &lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Start and enable services:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-mcp-server<br /> sudo -u root systemctl start archivematica-mcp-server<br /> sudo -u root systemctl enable archivematica-dashboard<br /> sudo -u root systemctl start archivematica-dashboard&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Reload nginx in order to load the dashboard config file:&lt;br /&gt;<br /> &lt;pre&gt;<br /> sudo -u root systemctl reload nginx&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;<br /> &lt;/pre&gt;<br /> The dashboard will be avaliable at http://ip:81<br /> <br /> == Installing Archivematica MCP Client ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, we need to add some extra repos with the MCP Client dependencies:&lt;/p&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Archivematica supplied external packages:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root bash -c 'cat &lt;&lt; EOF &gt; /etc/yum.repos.d/archivematica-extras.repo<br /> [archivematica-extras]<br /> name=archivematica-extras<br /> baseurl=https://packages.archivematica.org/1.5.x/centos-extras<br /> gpgcheck=0<br /> enabled=1<br /> EOF'<br /> &lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Nux multimedia repo&lt;/p&gt;<br /> &lt;pre&gt;sudo rpm -Uvh https://li.nux.ro/download/nux/dextop/el7/x86_64/nux-dextop-release-0-5.el7.nux.noarch.rpm&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Forensic tools repo&lt;/p&gt;<br /> &lt;pre&gt;sudo rpm -Uvh https://forensics.cert.org/cert-forensics-tools-release-el7.rpm&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Then, install the package:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y archivematica-mcp-client&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;The MCP Client expect some programs in certain paths, so we put things in place:&lt;/p&gt;<br /> &lt;pre&gt;sudo cp /usr/bin/clamscan /usr/bin/clamdscan<br /> sudo ln -s /usr/bin/7za /usr/bin/7z&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;After that, we can enable and start services&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-mcp-client<br /> sudo -u root systemctl start archivematica-mcp-client<br /> sudo -u root systemctl enable fits-nailgun<br /> sudo -u root systemctl start fits-nailgun&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> == Finalizing Installation ==<br /> <br /> The dashboard will be available on port 81, and the storage service on port 8001. You will need to complete the installation by opening up the dashboard in a web browser, and filling in the form you are presented with. On the 2nd page of the installer, you are asked for information about the storage service. You will need to log into the storage service and find the api key that was generated for your user (in admin-&gt;users).<br /> <br /> == Configuration ==<br /> <br /> Each service have a configuration file in /etc/sysconfig/archivematica-''packagename''<br /> <br /> == Known bugs / Caveats ==<br /> <br /> * If IPv6 is disabled, Nginx may refuse to start. If that is the case make sure that the listen directives used under /etc/nginx are not using IPv6 addresses like [::]:80.<br /> * If you find a bug, please let us know [https://github.com/artefactual-labs/am-packbuild/issues here]</div>

Scollazo https://wiki.archivematica.org/index.php?title=Improvements/CentOS-RedHat_support/Installation&diff=11463 2016-09-06T10:45:39Z

<p>Scollazo: /* Installing Archivematica MCP Client */</p> <hr /> <div>'''IMPORTANT NOTE''' These instructions will create an installation of a QA version of Archivematica and is not recommended for production use yet.<br /> <br /> = Quick install using [https://www.vagrantup.com/ Vagrant] =<br /> <br /> This method will have you up and running in no time, using a virtual machine provisioned with vagrant.<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, clone the am-packbuild repo:&lt;/p&gt;<br /> &lt;pre&gt;git clone https://github.com/artefactual-labs/am-packbuild/&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;cd into the rpm-testing directory&lt;/p&gt;<br /> &lt;pre&gt;cd am-packbuild/rpm-testing/&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;run vagrant&lt;/p&gt;<br /> &lt;pre&gt;vagrant up&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> After the install, the Archivematica Dashboard will be avaliable on port 81 of the vagrant deployed box, and the Storage Service, in port 8001.<br /> <br /> You can log in the vm over ssh running &lt;code&gt;vagrant ssh&lt;/code&gt;<br /> <br /> = Step by step Install =<br /> <br /> == Prerequisites ==<br /> <br /> === Extra repos ===<br /> <br /> Some repositories need to be installed in order to fullfill the installation procedure:<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;Extra packages for enterprise linux&lt;/p&gt;<br /> &lt;pre&gt;sudo yum install -y epel-release&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Elasticsearch&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch<br /> sudo -u root bash -c 'cat &amp;lt;&amp;lt; EOF &amp;gt; /etc/yum.repos.d/elasticsearch.repo<br /> [elasticsearch-1.7]<br /> name=Elasticsearch repository for 1.7 packages<br /> baseurl=https://packages.elastic.co/elasticsearch/1.7/centos<br /> gpgcheck=1<br /> gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch<br /> enabled=1<br /> EOF'&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Archivematica&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root bash -c 'cat &amp;lt;&amp;lt; EOF &amp;gt; /etc/yum.repos.d/archivematica.repo<br /> [archivematica]<br /> name=archivematica<br /> baseurl=https://packages.archivematica.org/1.5.x/centos<br /> gpgcheck=0<br /> enabled=1<br /> EOF'&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> === Service depencencies. ===<br /> <br /> Common services like elasticsearch, mariadb and gearmand should be installed and enabled before the archivematica install. It can be done with:<br /> <br /> &lt;pre&gt;sudo -u root yum install -y java-1.8.0-openjdk-headless elasticsearch mariadb-server gearmand<br /> sudo -u root systemctl enable elasticsearch<br /> sudo -u root systemctl start elasticsearch<br /> sudo -u root systemctl enable mariadb<br /> sudo -u root systemctl start mariadb<br /> sudo -u root systemctl enable gearmand<br /> sudo -u root systemctl start gearmand&lt;/pre&gt;<br /> == Installing Archivematica Storage Service ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, we install the packages:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y python-pip archivematica-storage-service&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;After the package is installed, we need to populate the sqlite database, and collect some static files used by django. Those tasks must be run as “archivematica” user.&lt;/p&gt;<br /> &lt;pre&gt;sudo -u archivematica bash -c &amp;quot; \<br /> set -a -e -x<br /> source /etc/sysconfig/archivematica-storage-service<br /> cd /usr/share/archivematica/storage-service<br /> /usr/lib/python2.7/archivematica/storage-service/bin/python manage.py migrate<br /> /usr/lib/python2.7/archivematica/storage-service/bin/python manage.py collectstatic --noinput<br /> &amp;quot;;&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;And now, we enable and start the archivematica-storage-service and it’s nginx frontend&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-storage-service<br /> sudo -u root systemctl start archivematica-storage-service<br /> sudo -u root systemctl enable nginx<br /> sudo -u root systemctl start nginx&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> The storage service will be avaliable at http://&lt;ip&gt;:8001<br /> <br /> == Installing Archivematica Dashboard and MCP Server ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, install the pacakges:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y archivematica-common archivematica-mcp-server archivematica-dashboard&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Create and populate the mysql database with:&lt;/p&gt;<br /> &lt;pre&gt;sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;DROP DATABASE IF EXISTS MCP; CREATE DATABASE MCP CHARACTER SET utf8 COLLATE utf8_unicode_ci;&amp;quot;<br /> sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;CREATE USER 'archivematica'@'localhost' IDENTIFIED BY 'demo';&amp;quot;<br /> sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;GRANT ALL ON MCP.* TO 'archivematica'@'localhost';&amp;quot;&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;And as archivematica user, run migrations:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u archivematica bash -c &amp;quot; \<br /> set -a -e -x<br /> source /etc/sysconfig/archivematica-dashboard<br /> cd /usr/share/archivematica/dashboard<br /> /usr/lib/python2.7/archivematica/dashboard/bin/python manage.py syncdb --noinput<br /> &amp;quot;;<br /> &lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Start and enable services:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-mcp-server<br /> sudo -u root systemctl start archivematica-mcp-server<br /> sudo -u root systemctl enable archivematica-dashboard<br /> sudo -u root systemctl start archivematica-dashboard&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Reload nginx in order to load the dashboard config file:&lt;br /&gt;<br /> sudo -u root systemctl reload nginx&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> The dashboard will be avaliable at http://ip:81<br /> <br /> == Installing Archivematica MCP Client ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, we need to add some extra repos with the MCP Client dependencies:&lt;/p&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Archivematica supplied external packages:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root bash -c 'cat &lt;&lt; EOF &gt; /etc/yum.repos.d/archivematica-extras.repo<br /> [archivematica-extras]<br /> name=archivematica-extras<br /> baseurl=https://packages.archivematica.org/1.5.x/centos-extras<br /> gpgcheck=0<br /> enabled=1<br /> EOF'<br /> &lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Nux multimedia repo&lt;/p&gt;<br /> &lt;pre&gt;sudo rpm -Uvh https://li.nux.ro/download/nux/dextop/el7/x86_64/nux-dextop-release-0-5.el7.nux.noarch.rpm&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Forensic tools repo&lt;/p&gt;<br /> &lt;pre&gt;sudo rpm -Uvh https://forensics.cert.org/cert-forensics-tools-release-el7.rpm&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Then, install the package:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y archivematica-mcp-client&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;The MCP Client expect some programs in certain paths, so we put things in place:&lt;/p&gt;<br /> &lt;pre&gt;sudo cp /usr/bin/clamscan /usr/bin/clamdscan<br /> sudo ln -s /usr/bin/7za /usr/bin/7z&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;After that, we can enable and start services&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-mcp-client<br /> sudo -u root systemctl start archivematica-mcp-client<br /> sudo -u root systemctl enable fits-nailgun<br /> sudo -u root systemctl start fits-nailgun&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> == Finalizing Installation ==<br /> <br /> The dashboard will be available on port 81, and the storage service on port 8001. You will need to complete the installation by opening up the dashboard in a web browser, and filling in the form you are presented with. On the 2nd page of the installer, you are asked for information about the storage service. You will need to log into the storage service and find the api key that was generated for your user (in admin-&gt;users).<br /> <br /> == Configuration ==<br /> <br /> Each service have a configuration file in /etc/sysconfig/archivematica-''packagename''<br /> <br /> == Known bugs / Caveats ==<br /> <br /> * If IPv6 is disabled, Nginx may refuse to start. If that is the case make sure that the listen directives used under /etc/nginx are not using IPv6 addresses like [::]:80.<br /> * If you find a bug, please let us know [https://github.com/artefactual-labs/am-packbuild/issues here]</div>

Scollazo https://wiki.archivematica.org/index.php?title=Improvements/CentOS-RedHat_support/Installation&diff=11462 2016-09-06T10:44:09Z

<p>Scollazo: /* Extra repos */</p> <hr /> <div>'''IMPORTANT NOTE''' These instructions will create an installation of a QA version of Archivematica and is not recommended for production use yet.<br /> <br /> = Quick install using [https://www.vagrantup.com/ Vagrant] =<br /> <br /> This method will have you up and running in no time, using a virtual machine provisioned with vagrant.<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, clone the am-packbuild repo:&lt;/p&gt;<br /> &lt;pre&gt;git clone https://github.com/artefactual-labs/am-packbuild/&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;cd into the rpm-testing directory&lt;/p&gt;<br /> &lt;pre&gt;cd am-packbuild/rpm-testing/&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;run vagrant&lt;/p&gt;<br /> &lt;pre&gt;vagrant up&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> After the install, the Archivematica Dashboard will be avaliable on port 81 of the vagrant deployed box, and the Storage Service, in port 8001.<br /> <br /> You can log in the vm over ssh running &lt;code&gt;vagrant ssh&lt;/code&gt;<br /> <br /> = Step by step Install =<br /> <br /> == Prerequisites ==<br /> <br /> === Extra repos ===<br /> <br /> Some repositories need to be installed in order to fullfill the installation procedure:<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;Extra packages for enterprise linux&lt;/p&gt;<br /> &lt;pre&gt;sudo yum install -y epel-release&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Elasticsearch&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch<br /> sudo -u root bash -c 'cat &amp;lt;&amp;lt; EOF &amp;gt; /etc/yum.repos.d/elasticsearch.repo<br /> [elasticsearch-1.7]<br /> name=Elasticsearch repository for 1.7 packages<br /> baseurl=https://packages.elastic.co/elasticsearch/1.7/centos<br /> gpgcheck=1<br /> gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch<br /> enabled=1<br /> EOF'&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Archivematica&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root bash -c 'cat &amp;lt;&amp;lt; EOF &amp;gt; /etc/yum.repos.d/archivematica.repo<br /> [archivematica]<br /> name=archivematica<br /> baseurl=https://packages.archivematica.org/1.5.x/centos<br /> gpgcheck=0<br /> enabled=1<br /> EOF'&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> === Service depencencies. ===<br /> <br /> Common services like elasticsearch, mariadb and gearmand should be installed and enabled before the archivematica install. It can be done with:<br /> <br /> &lt;pre&gt;sudo -u root yum install -y java-1.8.0-openjdk-headless elasticsearch mariadb-server gearmand<br /> sudo -u root systemctl enable elasticsearch<br /> sudo -u root systemctl start elasticsearch<br /> sudo -u root systemctl enable mariadb<br /> sudo -u root systemctl start mariadb<br /> sudo -u root systemctl enable gearmand<br /> sudo -u root systemctl start gearmand&lt;/pre&gt;<br /> == Installing Archivematica Storage Service ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, we install the packages:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y python-pip archivematica-storage-service&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;After the package is installed, we need to populate the sqlite database, and collect some static files used by django. Those tasks must be run as “archivematica” user.&lt;/p&gt;<br /> &lt;pre&gt;sudo -u archivematica bash -c &amp;quot; \<br /> set -a -e -x<br /> source /etc/sysconfig/archivematica-storage-service<br /> cd /usr/share/archivematica/storage-service<br /> /usr/lib/python2.7/archivematica/storage-service/bin/python manage.py migrate<br /> /usr/lib/python2.7/archivematica/storage-service/bin/python manage.py collectstatic --noinput<br /> &amp;quot;;&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;And now, we enable and start the archivematica-storage-service and it’s nginx frontend&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-storage-service<br /> sudo -u root systemctl start archivematica-storage-service<br /> sudo -u root systemctl enable nginx<br /> sudo -u root systemctl start nginx&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> The storage service will be avaliable at http://&lt;ip&gt;:8001<br /> <br /> == Installing Archivematica Dashboard and MCP Server ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, install the pacakges:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y archivematica-common archivematica-mcp-server archivematica-dashboard&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Create and populate the mysql database with:&lt;/p&gt;<br /> &lt;pre&gt;sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;DROP DATABASE IF EXISTS MCP; CREATE DATABASE MCP CHARACTER SET utf8 COLLATE utf8_unicode_ci;&amp;quot;<br /> sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;CREATE USER 'archivematica'@'localhost' IDENTIFIED BY 'demo';&amp;quot;<br /> sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;GRANT ALL ON MCP.* TO 'archivematica'@'localhost';&amp;quot;&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;And as archivematica user, run migrations:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u archivematica bash -c &amp;quot; \<br /> set -a -e -x<br /> source /etc/sysconfig/archivematica-dashboard<br /> cd /usr/share/archivematica/dashboard<br /> /usr/lib/python2.7/archivematica/dashboard/bin/python manage.py syncdb --noinput<br /> &amp;quot;;<br /> &lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Start and enable services:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-mcp-server<br /> sudo -u root systemctl start archivematica-mcp-server<br /> sudo -u root systemctl enable archivematica-dashboard<br /> sudo -u root systemctl start archivematica-dashboard&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Reload nginx in order to load the dashboard config file:&lt;br /&gt;<br /> sudo -u root systemctl reload nginx&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> The dashboard will be avaliable at http://ip:81<br /> <br /> == Installing Archivematica MCP Client ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, we need to add some extra repos with the MCP Client dependencies:&lt;/p&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Archivematica supplied external packages:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root bash -c 'cat &lt;&lt; EOF &gt; /etc/yum.repos.d/archivematica-extras.repo<br /> [archivematica-extras]<br /> name=archivematica-extras<br /> baseurl=https://packages.archivematica.org/1.5.x/centos-extras<br /> gpgcheck=0<br /> enabled=1<br /> EOF'<br /> &lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Nux multimedia repo&lt;/p&gt;<br /> &lt;pre&gt;rpm -Uvh https://li.nux.ro/download/nux/dextop/el7/x86_64/nux-dextop-release-0-5.el7.nux.noarch.rpm&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Forensic tools repo&lt;/p&gt;<br /> &lt;pre&gt;rpm -Uvh https://forensics.cert.org/cert-forensics-tools-release-el7.rpm&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Then, install the package:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y archivematica-mcp-client&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;The MCP Client expect some programs in certain paths, so we put things in place:&lt;/p&gt;<br /> &lt;pre&gt;sudo cp /usr/bin/clamscan /usr/bin/clamdscan<br /> sudo ln -s /usr/bin/7za /usr/bin/7z&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;After that, we can enable and start services&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-mcp-client<br /> sudo -u root systemctl start archivematica-mcp-client<br /> sudo -u root systemctl enable fits-nailgun<br /> sudo -u root systemctl start fits-nailgun&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> == Finalizing Installation ==<br /> <br /> The dashboard will be available on port 81, and the storage service on port 8001. You will need to complete the installation by opening up the dashboard in a web browser, and filling in the form you are presented with. On the 2nd page of the installer, you are asked for information about the storage service. You will need to log into the storage service and find the api key that was generated for your user (in admin-&gt;users).<br /> <br /> == Configuration ==<br /> <br /> Each service have a configuration file in /etc/sysconfig/archivematica-''packagename''<br /> <br /> == Known bugs / Caveats ==<br /> <br /> * If IPv6 is disabled, Nginx may refuse to start. If that is the case make sure that the listen directives used under /etc/nginx are not using IPv6 addresses like [::]:80.<br /> * If you find a bug, please let us know [https://github.com/artefactual-labs/am-packbuild/issues here]</div>

Scollazo https://wiki.archivematica.org/index.php?title=Improvements/CentOS-RedHat_support&diff=10989 2016-03-31T14:44:05Z

<p>Scollazo: /* Status */</p> <hr /> <div><br /> == User Story ==<br /> <br /> As a systems administrator, I would like to be able to run Archivematica on an rpm based version of linux like CentOS or RedHat.<br /> <br /> <br /> == Status ==<br /> <br /> Sponsored development began in early February 2016. <br /> <br /> The instructions in order to install archivematica on centos are avaliable at [[Improvements/CentOS-RedHat_Install]] <br /> <br /> <br /> There is also an ansible playbook capable of getting Archivematica running on CentOS here: https://github.com/artefactual-labs/ansible-role-archivematica-src/tree/dev/centos . This is a work in progress, not ready to be tested fully yet.<br /> <br /> == Interest ==<br /> <br /> Please feel free to add your organizations name to this list, if you have an interest in this improvement.<br /> <br /> Artefactual would like to see this improvement developed. We are able to do the development work, for a fee. We are also willing to assist others to complete all or part of the work required, in order to reduce the scope down to a level where Artefactual can complete the process as part of our existing commitment to provide new packages with each release.<br /> <br /> The National Library of Wales would like to see this developed. We have been building Archivematica on RHEL for a couple of years now, and would like to see an official rpm distribution. We have contacted Artefactual regarding the work.<br /> <br /> == Analysis: ==<br /> <br /> === The Current Situation ===<br /> <br /> Currently, Archivematica only works on LTS versions of Ubuntu (14.04 and 12.04). This is not due to any specific limitations in the Archivematica codebase, it is entirely a matter of packaging and bundling the large number of dependencies required in an Archivematica installation.<br /> <br /> Archivematica is comprised of a set of 5 separate packages, written primarily in python. Artefactual currently builds .deb packages for Ubuntu of all 5 packages, and makes them available via launchpad (e.g, the most current stable version is at https://launchpad.net/~archivematica/+archive/ubuntu/1.4).<br /> <br /> It is not necessary to use those .deb packages to install Archivematica. For example see these instructions for installing Archivematica using Ansible and Vagrant (https://wiki.archivematica.org/Getting_started#Installation). In that example, the 5 Archivematica packages are installed directly from github, as source, not from .deb packages. However, the longer list of external dependencies are still installed from .deb packages. <br /> <br /> Some of those .deb packages are available in the standard Ubuntu repositories, like apache and mysql. Others, like jhove, are in the Ubuntu multiverse repository, which is not always enabled by default in Ubuntu, but is easy to enable. Others are either not available from Ubuntu repositories, or the versions available there are too old to work in Archivematica. For these packages, Artefactual has taken on the responsibility for building .deb packages, and hosting them on launchpad. There are several listed here for example https://launchpad.net/~archivematica/+archive/ubuntu/1.4, like bagit, bulk_extractor, ffmpeg and others.<br /> <br /> ==== List of Archivematica's Ubuntu Package Dependencies ====<br /> <br /> This list is based on the packages installed by the Ansible role ( https://github.com/artefactual-labs/ansible-role-archivematica-src )<br /> <br /> Note that it is planned to move some package based dependencies to pip ( https://github.com/artefactual/archivematica/pull/398 )<br /> <br /> ===== Installed Using Packages =====<br /> <br /> ====== Storage Service ======<br /> <br /> - &quot;python-lxml&quot;<br /> - &quot;nginx&quot;<br /> - &quot;unar&quot;<br /> - &quot;uwsgi&quot;<br /> - &quot;uwsgi-plugin-python&quot;<br /> - &quot;python-virtualenv&quot;<br /> - &quot;python-dev&quot;<br /> - &quot;libxml2-dev&quot;<br /> - &quot;libxslt1-dev&quot;<br /> - &quot;libz-dev&quot;<br /> - &quot;libffi-dev&quot;<br /> - &quot;libssl-dev&quot;<br /> <br /> ====== Pipeline (dashboard, MCP Server, MCP Client) ======<br /> <br /> - &quot;python&quot;<br /> - &quot;python-pip&quot;<br /> - &quot;python2.7-elementtree&quot;<br /> - &quot;python-mimeparse&quot;<br /> - &quot;python-dateutil&quot;<br /> <br /> - &quot;apache2-mpm-prefork&quot;<br /> - &quot;libapache2-mod-wsgi&quot;<br /> - &quot;python-pip&quot;<br /> - &quot;python-gearman&quot;<br /> - &quot;python-simplejson&quot;<br /> <br /> - &quot;dbconfig-common&quot;<br /> - &quot;logapp&quot;<br /> - &quot;python-pyinotify&quot;<br /> - &quot;python-gearman&quot;<br /> - &quot;python-mysqldb&quot;<br /> - &quot;python-lxml&quot;<br /> - &quot;uuid&quot;<br /> <br /> - &quot;atool&quot;<br /> - &quot;bagit&quot; (*)<br /> - &quot;bulk-extractor&quot; (*)<br /> - &quot;clamav&quot;<br /> - &quot;clamav-daemon&quot;<br /> - &quot;ffmpeg&quot; (*)<br /> - &quot;fits&quot; (*)<br /> - &quot;gearman&quot;<br /> - &quot;imagemagick&quot;<br /> - &quot;inkscape&quot;<br /> - &quot;jhove&quot;<br /> - &quot;libimage-exiftool-perl&quot; (*)<br /> - &quot;libxml2-utils&quot;<br /> - &quot;logapp&quot;<br /> - &quot;md5deep&quot;<br /> - &quot;mediainfo&quot;<br /> - &quot;nfs-common&quot;<br /> - &quot;openjdk-7-jre-headless&quot;<br /> - &quot;p7zip-full&quot;<br /> - &quot;pbzip2&quot;<br /> - &quot;postfix&quot;<br /> - &quot;python-fido&quot; (*)<br /> - &quot;python-gearman&quot;<br /> - &quot;python-lxml&quot;<br /> - &quot;python-mysqldb&quot;<br /> - &quot;python-pyicu&quot;<br /> - &quot;python-unidecode&quot;<br /> - &quot;readpst&quot;<br /> - &quot;rsync&quot;<br /> - &quot;siegfried&quot; (*)<br /> - &quot;sleuthkit&quot; (*)<br /> - &quot;tesseract-ocr&quot;<br /> - &quot;tika&quot; (*)<br /> - &quot;tree&quot;<br /> - &quot;ufraw&quot;<br /> - &quot;unrar-free&quot;<br /> - &quot;uuid&quot;<br /> <br /> Packages with an (*) have been built by Artefactual, as they do not exist in the public Ubuntu repositories, or the versions found in the public repositories were not recent enough for Archivematica to work properly.<br /> <br /> === RPM Support ===<br /> <br /> This analysis is based on the use of CentOS/Redhat 7, for a port to CentOS 6, there would be more work than is outlined here.<br /> <br /> In order to get Archivematica running on an rpm based linux distribution, a set of 5 rpm packages for the different Archivematica applications would need to be created. RPM's for about 30 different dependencies would also have to be created.<br /> <br /> Archivematica Packages:<br /> <br /> Archivematica Common<br /> Dashboard<br /> MCP Server<br /> MCP Client<br /> Archivematica Storage Service<br /> <br /> optional (not all of these are currently being packaged as .debs): <br /> MCP rpc Client<br /> automation-tools<br /> archivematica-devtools<br /> <br /> Other Open Source applications required in rpm packages for Archivematica:<br /> <br /> * atool<br /> * bagit<br /> * bulk_extractor<br /> * exiftool (repoforge has this, but we want to package a specific version)<br /> * ffmpeg<br /> ** libasound2<br /> ** jack<br /> ** openjpeg - 1.5.x is packaged, might be new enough to avoid packaging<br /> ** libraw1394 - we might be able to live without this<br /> ** libvpx<br /> * fido<br /> * fits<br /> * gearman / python-gearman<br /> * jhove<br /> * mediainfo<br /> * nailgun / nailgun-client<br /> * nfs - should work without extra packages, but requires testing<br /> * python-elementtree<br /> * python-mimeparse<br /> * siegfried<br /> ** go 1.4<br /> * sleuthkit<br /> ** afflib<br /> ** libbfio<br /> ** libewf (version in repoforge is 3 years older)<br /> * tika<br /> * ufraw<br /> * unar<br /> * unidecode<br /> * unrar-free<br /> <br /> All of the remaining required dependencies are either included in the stock RHEL/CentOS 7 repositories, or in the third-party Repoforge repository.<br /> (TODO: list which repo each dependency is maintained in).<br /> <br /> Artefactual already produces Ubuntu packages for most of the packages listed here, and we have some tooling in place to automate some of this process. <br /> See: https://github.com/artefactual-labs/am-packbuild for an example set of python scripts used to create Ubuntu packages for the 5 Archivematica packages.<br /> (TODO: move ubuntu packaging scripts to github).<br /> <br /> == Scope ==<br /> <br /> Artefactual has done some analysis of the effort required, and provided quotes for this work to some institutions, at their request. Feel free to add additional comments or another piece to this section if you have more information about the possible scope of work required.<br /> <br /> === Proposed Approach ===<br /> <br /> # Check if the Ubuntu package dependencies have equivalent RPMs in CentOS. We estimate that Ubuntu packages provided by standard Ubuntu repositories have a CentOS equivalent in the standard CentOS repos, and that all the Ubuntu packages built by Artefactual are missing in CentOS and require to be built as well <br /> # Build missing CentOS RPMs package dependencies<br /> # Fix the ansible role in https://github.com/artefactual-labs/ansible-role-archivematica-src to be able to install Archivematica / Storage Service in CentOS<br /> # Repeat steps above as required until we obtain a working CentOS installation<br /> <br /> === Artefactual Estimate ===<br /> <br /> We have estimated that it would take about 16-20 hours to create an initial set of packages for all of the required applications.<br /> <br /> Another 12-16 hours of testing would be required, at a minimum, to confirm that all the packages are working properly, and to do very basic tests of Archivematica on CentOS 7. Further testing would be desirable, any testing that could be done by the community would help improve the outcome of this work. <br /> <br /> We would probably need another 8 hours of work for project administration and documentation.<br /> <br /> Based on these estimates, it would probably take about 40 hours of work to create the required packages and test and document the CentOS 7 installation procedure.<br /> <br /> Artefactual provides development services on either a fixed fee or a time and materials basis. We offer a less expensive rate for time and materials contracts. Our rates are listed here:<br /> <br /> https://www.artefactual.com/services/<br /> <br /> == Alternative approaches ==<br /> <br /> It's also possible to ship the Archivematica packages and its dependencies as container images. Docker has donated its container format and runtime to the [https://www.opencontainers.org/ Open Container Initiative]. These images are composable and they can run via the Docker or rkt container runtimes or independently via [https://github.com/opencontainers/runc runC]. Images are easily distributed via services like [https://hub.docker.com/ Docker Hub] or [https://github.com/docker/distribution Docker Distribution].<br /> <br /> A first attempt at this approach has been published [https://hub.docker.com/r/sevein/archivematica-mcp-client-base/]. This is a docker image containing all of the dependencies of the Archivematica MCP Client package. The image is built on top of an ubuntu base image, but it can be deployed on any linux distribution, not just CentOS. Docker images can also be run on a windows host, via virtualbox on windows 8/10, and via Windows Containers on Windows Server 2016.<br /> <br /> It is possible to produce rpm packages, that contain docker images, for example: [https://github.com/alanfranz/docker-rpm-builder]. <br /> <br /> <br /> More on Docker from RedHat: [http://developerblog.redhat.com/2014/05/15/practical-introduction-to-docker-containers/]</div>

Scollazo https://wiki.archivematica.org/index.php?title=Improvements/CentOS-RedHat_support/Installation&diff=10988 2016-03-31T14:39:54Z

<p>Scollazo: Created page with &quot;= Quick install using [https://www.vagrantup.com/ Vagrant] = This method will have you up and running in no time, using a virtual machine provisioned with vagrant. &lt;ul&gt; &lt;li&gt;...&quot;</p> <hr /> <div>= Quick install using [https://www.vagrantup.com/ Vagrant] =<br /> <br /> This method will have you up and running in no time, using a virtual machine provisioned with vagrant.<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, clone the am-packbuild repo:&lt;/p&gt;<br /> &lt;pre&gt;git clone https://github.com/artefactual-labs/am-packbuild/&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;cd into the rpm-testing directory&lt;/p&gt;<br /> &lt;pre&gt;cd am-packbuild/rpm-testing/&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;run vagrant&lt;/p&gt;<br /> &lt;pre&gt;vagrant up&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> After the install, the Archivematica Dashboard will be avaliable on port 81 of the vagrant deployed box, and the Storage Service, in port 8001.<br /> <br /> You can log in the vm over ssh running &lt;code&gt;vagrant ssh&lt;/code&gt;<br /> <br /> = Step by step Install =<br /> <br /> == Prerequisites ==<br /> <br /> === Extra repos ===<br /> <br /> Some repositories need to be installed in order to fullfill the instalation procedure:<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;Extra packages for enterprise linux&lt;/p&gt;<br /> &lt;pre&gt;yum install -y epel-release&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Elasticsearch&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch<br /> sudo -u root bash -c 'cat &amp;lt;&amp;lt; EOF &amp;gt; /etc/yum.repos.d/elasticsearch.repo<br /> [elasticsearch-1.7]<br /> name=Elasticsearch repository for 1.7 packages<br /> baseurl=https://packages.elastic.co/elasticsearch/1.7/centos<br /> gpgcheck=1<br /> gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch<br /> enabled=1<br /> EOF'&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Archivematica&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root bash -c 'cat &amp;lt;&amp;lt; EOF &amp;gt; /etc/yum.repos.d/archivematica.repo<br /> [archivematica]<br /> name=archivematica<br /> baseurl=https://packages.archivematica.org/1.x/centos<br /> gpgcheck=0<br /> enabled=1<br /> EOF'&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> === Service depencencies. ===<br /> <br /> Common services like elasticsearch, mariadb and gearmand should be installed and enabled before the archivematica install. It can be done with:<br /> <br /> &lt;pre&gt;sudo -u root yum install -y java-1.8.0-openjdk-headless elasticsearch mariadb-server gearmand<br /> sudo -u root systemctl enable elasticsearch<br /> sudo -u root systemctl start elasticsearch<br /> sudo -u root systemctl enable mariadb<br /> sudo -u root systemctl start mariadb<br /> sudo -u root systemctl enable gearmand<br /> sudo -u root systemctl start gearmand&lt;/pre&gt;<br /> == Installing Archivematica Storage Service ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, we install the packages:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y python-pip archivematica-storage-service&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;After the package is installed, we need to populate the sqlite database, and collect some static files used by django. Those tasks must be run as “archivematica” user.&lt;/p&gt;<br /> &lt;pre&gt;sudo -u archivematica bash -c &amp;quot; \<br /> set -a -e -x<br /> source /etc/sysconfig/archivematica-storage-service<br /> cd /usr/share/archivematica/storage-service<br /> /usr/lib/python2.7/archivematica/storage-service/bin/python manage.py migrate<br /> /usr/lib/python2.7/archivematica/storage-service/bin/python manage.py collectstatic --noinput<br /> &amp;quot;;&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;And now, we enable and start the archivematica-storage-service and it’s nginx frontend&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-storage-service<br /> sudo -u root systemctl start archivematica-storage-service<br /> sudo -u root systemctl enable nginx<br /> sudo -u root systemctl start nginx&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> The storage service will be avaliable at http://&lt;ip&gt;:8001<br /> <br /> == Installing Archivematica Dashboard and MCP Server ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, install the pacakges:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y archivematica-common archivematica-mcp-server archivematica-dashboard&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Create and populate the mysql database with:&lt;/p&gt;<br /> &lt;pre&gt;sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;DROP DATABASE IF EXISTS MCP; CREATE DATABASE MCP CHARACTER SET utf8 COLLATE utf8_unicode_ci;&amp;quot;<br /> sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;CREATE USER 'archivematica'@'localhost' IDENTIFIED BY 'demo';&amp;quot;<br /> sudo -H -u root mysql -hlocalhost -uroot -e &amp;quot;GRANT ALL ON MCP.* TO 'archivematica'@'localhost';&amp;quot;<br /> sudo -H -u root mysql -hlocalhost -uroot MCP &amp;lt; /usr/share/archivematica/MCPServer/mysql&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;And as archivematica user, run migrations:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u archivematica bash -c &amp;quot; \<br /> set -a -e -x<br /> source /etc/sysconfig/archivematica-dashboard<br /> cd /usr/share/archivematica/dashboard<br /> /usr/lib/python2.7/archivematica/dashboard/bin/python manage.py syncdb --noinput<br /> &amp;quot;;<br /> sudo -u root /usr/share/archivematica/MCPServer/mysql_dev.sh MCP&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Start and enable services:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-mcp-server<br /> sudo -u root systemctl start archivematica-mcp-server<br /> sudo -u root systemctl enable archivematica-dashboard<br /> sudo -u root systemctl start archivematica-dashboard&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Reload nginx in order to load the dashboard config file:&lt;br /&gt;<br /> sudo -u root systemctl reload nginx&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> The dashboard will be avaliable at http://ip:81<br /> == Installing Archivematica MCP Client ==<br /> <br /> &lt;ul&gt;<br /> &lt;li&gt;&lt;p&gt;First, we need to add some extra repos with the MCP Client dependencies:&lt;/p&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Nux multimedia repo&lt;/p&gt;<br /> &lt;pre&gt; rpm -Uvh https://li.nux.ro/download/nux/dextop/el7/x86_64/nux-dextop-release-0-5.el7.nux.noarch.rpm&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Forensic tools repo&lt;/p&gt;<br /> &lt;pre&gt; rpm -Uvh https://forensics.cert.org/cert-forensics-tools-release-el7.rpm&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;Then, install the package:&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root yum install -y archivematica-mcp-client&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;The MCP Client expect some programs in certain paths, so we put things in place:&lt;/p&gt;<br /> &lt;pre&gt;sudo cp /usr/bin/clamscan /usr/bin/clamdscan<br /> sudo ln -s /usr/bin/7za /usr/bin/7z&lt;/pre&gt;&lt;/li&gt;<br /> &lt;li&gt;&lt;p&gt;After that, we can enable and start services&lt;/p&gt;<br /> &lt;pre&gt;sudo -u root systemctl enable archivematica-mcp-client<br /> sudo -u root systemctl start archivematica-mcp-client<br /> sudo -u root systemctl enable fits-nailgun<br /> sudo -u root systemctl start fits-nailgun&lt;/pre&gt;&lt;/li&gt;&lt;/ul&gt;<br /> <br /> == Configuration ==<br /> <br /> Each service have a configuration file in /etc/sysconfig/archivematica-''packagename''<br /> <br /> == Known bugs / Caveats ==<br /> <br /> * If IPv6 is disabled, Nginx may refuse to start. If that is the case make sure that the listen directives used under /etc/nginx are not using IPv6 addresses like [::]:80.<br /> * If you find a bug, please let us know [https://github.com/artefactual-labs/am-packbuild/issues here]</div>

Scollazo