Difference between revisions of "Update ElasticSearch"

From Archivematica
Jump to navigation Jump to search
 
(7 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
= ElasticSearch update =
 
= ElasticSearch update =
  
 +
1. Backup elasticsearch data directory
  
 +
  sudo service elasticsearch stop
 +
  sudo tar cfz var_lib_elasticsearch_$(date +%y%m%d).tgz  /var/lib/elasticsearch
 +
  sudo service elasticsearch start
  
−
 
+
2. Check that ES is up and running, and note the results:
−
1. Check that ES is up and running, and note the results:
 
  
 
   curl -X GET "localhost:9200/_cat/indices?v"
 
   curl -X GET "localhost:9200/_cat/indices?v"
 
   health status index    pri rep docs.count docs.deleted store.size pri.store.size  
 
   health status index    pri rep docs.count docs.deleted store.size pri.store.size  
−
   yellow open  aips        5  1         11           0   267.9kb       267.9kb
+
   yellow open  aips        5  1     24816           1   175.4mb       175.4mb
−
   yellow open  transfers  5  1         24           0     35.2kb        35.2kb
+
   yellow open  transfers  5  1         0           0       720b          720b
 +
 
 +
Check also if there are any configured snapshot:
 +
 
 +
  curl -XGET "localhost:9200/_snapshot/_all?pretty"
 +
 
 +
3. Remove Snapshots
 +
 
 +
If there are any snapshot (step 1), please remove it:
  
−
2. Stop elasticsearch service
+
  curl -XDELETE "localhost:9200/_snapshot/backup-repo?pretty"
 +
 
 +
4. Stop elasticsearch service
  
 
   sudo service elasticsearch stop
 
   sudo service elasticsearch stop
  
−
3. Check wich elasticsearch version is installed
+
5. Check wich elasticsearch version is installed
  
−
   Redhat/Centos: rpm -qa | grep elasticsearch
+
   Redhat/Centos: rpm -q elasticsearch
−
   Ubuntu: dpkg -l | grep elasticsearch
+
   Ubuntu: dpkg -l elasticsearch
  
−
4. Download a temporary copy of elasticsearch (same version than intalled)
+
6. Download a temporary copy of elasticsearch (same version than intalled)
  
−
   wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz
+
   wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.6.tar.gz
  
  
−
5. Uncompress the downloaded tar.gz and go into it
+
7. Uncompress the downloaded tar.gz and go into it
  
−
   tar zxvf elasticsearch-1.7.5.tar.gz
+
   tar zxvf elasticsearch-1.7.6.tar.gz
−
   cd elasticsearch-1.7.5
+
   cd elasticsearch-1.7.6
  
  
−
6. Copy data from system wide installed elasticsearch, ot the temporary copy
+
8. Copy data from system wide installed elasticsearch, ot the temporary copy
  
 
   sudo cp /var/lib/elasticsearch data -rf
 
   sudo cp /var/lib/elasticsearch data -rf
  
−
7. Adjust file permissions
+
9. Adjust file permissions
  
 
   sudo chown <your user>:<your group> data -R
 
   sudo chown <your user>:<your group> data -R
  
−
8. Launch the temporary elasticsearch instance
+
10. Launch the temporary elasticsearch instance
  
−
   ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500
+
   ES_JAVA_OPTS="-Xms2g -Xmx2g" ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500 -Des.discovery.zen.ping.multicast.enabled=false
  
−
9. Verify that the indexes were properly copied. Use the port specified in the previous command:
+
11. Verify that the indexes were properly copied. Use the port specified in the previous command:
  
 
   curl -X GET "localhost:9500/_cat/indices?v"
 
   curl -X GET "localhost:9500/_cat/indices?v"
  
−
10. Remove elasticsearch 1.7.5 and it's files
+
12. Remove elasticsearch 1.7.6 and it's files
  
 
   Redhat: sudo yum remove elasticsearch
 
   Redhat: sudo yum remove elasticsearch
 
   Ubuntu: sudo apt-get remove --purge elasticsearch
 
   Ubuntu: sudo apt-get remove --purge elasticsearch
−
   sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.5
+
   sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.6
−
   sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.5
+
   sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.6
  
−
11. Upgrade archivematica and install elasticsearch 6
+
13. Upgrade archivematica and install elasticsearch 6
  
 
   ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src
 
   ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src
  
−
12. Configure ElasticSearch 6 to handle reindex from the temporary es  
+
14. Configure ElasticSearch 6 to handle reindex from the temporary es  
  
 
   echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml
 
   echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml
 
   sudo service elasticsearch restart
 
   sudo service elasticsearch restart
  
−
13. Migrate the indexes
+
15. Migrate the indexes
  
 
   sudo -u archivematica bash -c " \
 
   sudo -u archivematica bash -c " \
Line 71: Line 84:
 
         || (echo 'Environment file not found'; exit 1)
 
         || (echo 'Environment file not found'; exit 1)
 
   cd /usr/share/archivematica/dashboard
 
   cd /usr/share/archivematica/dashboard
−
   /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster http://localhost:9500
+
   /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster -t 60 -s 1 http://localhost:9500
 
   ";
 
   ";
  
Line 78: Line 91:
 
   All reindex requests ended successfully!
 
   All reindex requests ended successfully!
  
 +
Else, you might need to tweak the timeout (-t) and size (-s) parameters.
 +
 +
 +
16. Verify that the new indexes were created and populated:
  
−
14. Verify that the new indexes were created and populated:
 
−
 
 
 
   curl -X GET "localhost:9200/_cat/indices?v"  
 
   curl -X GET "localhost:9200/_cat/indices?v"  
 
   health status index        uuid                  pri rep docs.count docs.deleted store.size pri.store.size
 
   health status index        uuid                  pri rep docs.count docs.deleted store.size pri.store.size
−
   yellow open  aips          IL7lbAZFR9adKg-GpgepTw   5  1          1           0   135.8kb        135.8kb
+
   yellow open  transferfiles jlu2d2yZQpWwpKKT1ZzMAA   5  1          0           0     1.1kb          1.1kb
−
   yellow open  transferfiles hlG1GBwIRNCi1rI74LhQpQ   5  1         23           0    49.1kb         49.1kb
+
   yellow open  aips          VAvONAByRBuVcWBYhlhUmw   5  1       116           0    66.9mb         66.9mb
−
   yellow open  transfers    ebRiOvutR_e7N4dNGoNq4w   5  1          1           0       6kb            6kb
+
   yellow open  transfers    OhfLVBzuRqCNUYRCSSTidg   5  1          0           0     1.1kb          1.1kb
−
   yellow open  aipfiles      2QFnLdr7Txi2qgG2C7-eJw   5  1         10           0   158.7kb        158.7kb
+
   yellow open  aipfiles      clX1gcqnT9CTdwTm8jeatw   5  1     24700           0     70.6mb        70.6mb
  
−
You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.
+
You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.6. The same happens with the transfers/transferfiles indexes.
  
−
15. Stop the temporary elasticsearch service
+
17. Stop the temporary elasticsearch service
  
−
   kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)
+
   kill $(cat ~/elasticsearch-1.7.6/elastic-tmp.pid)
  
−
16. Restart all archivematica services
+
18. Restart all archivematica services
 
    
 
    
−
   service archivematica-dashboard restart
+
   sudo service archivematica-mcp-server restart
−
   service archivematica-mcp-server restart
+
   sudo service archivematica-mcp-client restart
−
   service archivematica-mcp-client restart
+
   sudo service archivematica-dashboard restart
−
   service archivematica-storage-service restart
+
   sudo service archivematica-storage-service restart

Latest revision as of 20:37, 28 February 2019

ElasticSearch update[edit]

1. Backup elasticsearch data directory 

 sudo service elasticsearch stop
 sudo tar cfz var_lib_elasticsearch_$(date +%y%m%d).tgz  /var/lib/elasticsearch
 sudo service elasticsearch start

2. Check that ES is up and running, and note the results: 

 curl -X GET "localhost:9200/_cat/indices?v"
 health status index     pri rep docs.count docs.deleted store.size pri.store.size 
 yellow open   aips        5   1      24816            1    175.4mb        175.4mb 
 yellow open   transfers   5   1          0            0       720b           720b

Check also if there are any configured snapshot: 

 curl -XGET "localhost:9200/_snapshot/_all?pretty"

3. Remove Snapshots

If there are any snapshot (step 1), please remove it: 

 curl -XDELETE "localhost:9200/_snapshot/backup-repo?pretty"

4. Stop elasticsearch service 

 sudo service elasticsearch stop

5. Check wich elasticsearch version is installed 

 Redhat/Centos: rpm -q elasticsearch
 Ubuntu: dpkg -l elasticsearch

6. Download a temporary copy of elasticsearch (same version than intalled) 

 wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.6.tar.gz


7. Uncompress the downloaded tar.gz and go into it 

 tar zxvf elasticsearch-1.7.6.tar.gz
 cd elasticsearch-1.7.6


8. Copy data from system wide installed elasticsearch, ot the temporary copy 

 sudo cp /var/lib/elasticsearch data -rf

9. Adjust file permissions 

 sudo chown <your user>:<your group> data -R

10. Launch the temporary elasticsearch instance 

 ES_JAVA_OPTS="-Xms2g -Xmx2g" ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500  -Des.discovery.zen.ping.multicast.enabled=false

11. Verify that the indexes were properly copied. Use the port specified in the previous command: 

 curl -X GET "localhost:9500/_cat/indices?v"

12. Remove elasticsearch 1.7.6 and it's files 

 Redhat: sudo yum remove elasticsearch
 Ubuntu: sudo apt-get remove --purge elasticsearch
 sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.6
 sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.6

13. Upgrade archivematica and install elasticsearch 6 

 ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src

14. Configure ElasticSearch 6 to handle reindex from the temporary es 

 echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml
 sudo service elasticsearch restart

15. Migrate the indexes 

 sudo -u archivematica bash -c " \
 set -a -e -x
 source /etc/default/archivematica-dashboard || \
 source /etc/sysconfig/archivematica-dashboard \
       || (echo 'Environment file not found'; exit 1)
 cd /usr/share/archivematica/dashboard
 /usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster -t 60 -s 1 http://localhost:9500
 ";

The command should finish with 

 All reindex requests ended successfully!

Else, you might need to tweak the timeout (-t) and size (-s) parameters.


16. Verify that the new indexes were created and populated: 

 curl -X GET "localhost:9200/_cat/indices?v" 
 health status index         uuid                   pri rep docs.count docs.deleted store.size pri.store.size
 yellow open   transferfiles jlu2d2yZQpWwpKKT1ZzMAA   5   1          0            0      1.1kb          1.1kb
 yellow open   aips          VAvONAByRBuVcWBYhlhUmw   5   1        116            0     66.9mb         66.9mb
 yellow open   transfers     OhfLVBzuRqCNUYRCSSTidg   5   1          0            0      1.1kb          1.1kb
 yellow open   aipfiles      clX1gcqnT9CTdwTm8jeatw   5   1      24700            0     70.6mb         70.6mb

You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.6. The same happens with the transfers/transferfiles indexes.

17. Stop the temporary elasticsearch service 

 kill $(cat ~/elasticsearch-1.7.6/elastic-tmp.pid)

18. Restart all archivematica services 

 sudo service archivematica-mcp-server restart
 sudo service archivematica-mcp-client restart
 sudo service archivematica-dashboard restart
 sudo service archivematica-storage-service restart
Retrieved from "https://wiki.archivematica.org/index.php?title=Update_ElasticSearch&oldid=12912"