Difference between revisions of "Update ElasticSearch"
Line 10: | Line 10: | ||
yellow open aips 5 1 24816 1 175.4mb 175.4mb | yellow open aips 5 1 24816 1 175.4mb 175.4mb | ||
yellow open transfers 5 1 0 0 720b 720b | yellow open transfers 5 1 0 0 720b 720b | ||
+ | |||
+ | Check also if there are any configured snapshot. If so, copy them to other place and remove them | ||
+ | |||
+ | curl -XGET "localhost:9200/_snapshot/_all?pretty" | ||
+ | curl -XDELETE "localhost:9200/_snapshot/backup-repo?pretty" | ||
Revision as of 19:02, 26 February 2019
ElasticSearch update
1. Check that ES is up and running, and note the results:
curl -X GET "localhost:9200/_cat/indices?v" health status index pri rep docs.count docs.deleted store.size pri.store.size yellow open aips 5 1 24816 1 175.4mb 175.4mb yellow open transfers 5 1 0 0 720b 720b
Check also if there are any configured snapshot. If so, copy them to other place and remove them
curl -XGET "localhost:9200/_snapshot/_all?pretty" curl -XDELETE "localhost:9200/_snapshot/backup-repo?pretty"
2. Stop elasticsearch service
sudo service elasticsearch stop
3. Check wich elasticsearch version is installed
Redhat/Centos: rpm -qa | grep elasticsearch Ubuntu: dpkg -l | grep elasticsearch
4. Download a temporary copy of elasticsearch (same version than intalled)
wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz
5. Uncompress the downloaded tar.gz and go into it
tar zxvf elasticsearch-1.7.5.tar.gz cd elasticsearch-1.7.5
6. Copy data from system wide installed elasticsearch, ot the temporary copy
sudo cp /var/lib/elasticsearch data -rf
7. Adjust file permissions
sudo chown <your user>:<your group> data -R
8. Launch the temporary elasticsearch instance
ES_JAVA_OPTS="-Xms2g -Xmx2g" ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500
9. Verify that the indexes were properly copied. Use the port specified in the previous command:
curl -X GET "localhost:9500/_cat/indices?v"
10. Remove elasticsearch 1.7.5 and it's files
Redhat: sudo yum remove elasticsearch Ubuntu: sudo apt-get remove --purge elasticsearch sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.5 sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.5
11. Upgrade archivematica and install elasticsearch 6
ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src
12. Configure ElasticSearch 6 to handle reindex from the temporary es
echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml sudo service elasticsearch restart
13. Migrate the indexes
sudo -u archivematica bash -c " \ set -a -e -x source /etc/default/archivematica-dashboard || \ source /etc/sysconfig/archivematica-dashboard \ || (echo 'Environment file not found'; exit 1) cd /usr/share/archivematica/dashboard /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster -t 60 http://localhost:9500 ";
The command should finish with
All reindex requests ended successfully!
14. Verify that the new indexes were created and populated:
curl -X GET "localhost:9200/_cat/indices?v" health status index uuid pri rep docs.count docs.deleted store.size pri.store.size yellow open transferfiles jlu2d2yZQpWwpKKT1ZzMAA 5 1 0 0 1.1kb 1.1kb yellow open aips VAvONAByRBuVcWBYhlhUmw 5 1 116 0 66.9mb 66.9mb yellow open transfers OhfLVBzuRqCNUYRCSSTidg 5 1 0 0 1.1kb 1.1kb yellow open aipfiles clX1gcqnT9CTdwTm8jeatw 5 1 24700 0 70.6mb 70.6mb
You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.
15. Stop the temporary elasticsearch service
kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)
16. Restart all archivematica services
service archivematica-dashboard restart service archivematica-mcp-server restart service archivematica-mcp-client restart service archivematica-storage-service restart