Archivematica - User contributions [en]

Update mapping and reindex Elasticsearch indices

2020-12-23T21:25:49Z

Mamedin: /* Run script to reindex and use new mappings */

= Backup Elasticsearch data =

The easiest way to backup the Elasticsearch data is copying the data directory:

sudo service elasticsearch stop
tar cvfz var_lib_elasticsearch.tgz /var/lib/elasticsearch
sudo service elasticsearch start

= List indices to resize the Elasticsearch heap size when needed =

Use the following command to list indices:

curl -s -X GET 'http://localhost:9200/_cat/indices/%2A?v=&s=index:desc'

The output should show something like this:

root@archivematica-test-server:~# curl -s -X GET 'http://localhost:9200/_cat/indices/%2A?v=&s=index:desc'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open transfers lYqkYjwZRy2XG8CP_3S3PQ 5 1 0 0 1.2kb 1.2kb
yellow open transferfiles K5gnDZyOQz2JdIeZ6adJsQ 5 1 0 0 1.2kb 1.2kb
yellow open aips yAyK_koXThaZcWsBYfzN7w 5 1 17 0 101.4mb 101.4mb
yellow open aipfiles TVrrX8jkRhWWxGfvK_M6zg 5 1 11987 0 2.9gb 2.9gb

Take the elasticsearch heap size from ''/etc/default/elasticsearch'' (Ubuntu) or ''/etc/sysconfig/elasticsearch'' (CentOS):

root@ny-gclibrary-test-release-1:~# grep ES_JAVA_OPTS= /etc/default/elasticsearch
#ES_JAVA_OPTS=
ES_JAVA_OPTS="-Xms2g -Xmx2g"

The heap size for the example is 2G.

Ensure your Elasticsearch heap size is greater than the max ''store.size'' in the indices list. For our example, it should be greater than 3GB.

* Edit ''/etc/default/elasticsearch'' or ''/etc/sysconfig/elasticsearch''.
* Change ''ES_JAVA_OPTS'' to a bigger value, in our example:
ES_JAVA_OPTS="-Xms3g -Xmx3g".
* Restart Elasticsearch service for the changes to take effect (''sudo service elasticsearch restart'')

= Run script to reindex and use new mappings =

Use the following script:

<nowiki>
#!/bin/bash

es_url="http://localhost:9200"

index_list='aips aipfiles transfers transferfiles'

echo -e "\nIndex list before reindexing:\n"
curl -s -X GET "${es_url}/_cat/indices/%2A?v=&s=index:desc"
echo -e "\n"

#Clone indices with _reindex API call:
for index in $index_list;do
echo "Reindex ${index} in ${index}_new..."
curl -s -X POST \
${es_url}/_reindex \
-H 'Content-Type: application/json' \
-d '{
"source": {
"index": "'"${index}"'"
},
"dest": {
"index": "'"${index}_new"'"
}
}' > /dev/null
done

echo -e "\n\n"

echo -e "Index list after tmp indices creation\n"
indices_output=$(curl -s -X GET "${es_url}/_cat/indices/%2A?v=&s=index:desc")
curl -s -X GET "${es_url}/_cat/indices/%2A?v=&s=index:desc"
echo -e "\n"

#Delete old indices
for index in $index_list;do
echo "Deleting ${index}..."
curl -s -X DELETE ${es_url}/${index} > /dev/null
done

#Restart archivematica-dashboard to create indices with new mappings
echo -e "\nRestarting archivematica-dashboard"
sudo service archivematica-dashboard restart

#Wait 30 seconds
echo "Wait 30 seconds to ensure dashboard has created the empty indices with new mapping"
sleep 30
echo -e "\n"

#When index has no docs the reindex doesn't create the new index (typically transferfiles index)
#There's a check to ensure the new index has been create before reindexing.
#Reindex fron *_new indices:
for index in $index_list;do
if echo "$indices_output" | grep ${index}_new >/dev/null; then
echo "Indexing ${index} using ${index}_new ..."
curl -s -X POST \
${es_url}/_reindex \
-H 'Content-Type: application/json' \
-d '{
"source": {
"index": "'"${index}_new"'"
},
"dest": {
"index": "'"${index}"'"
}
}' > /dev/null
fi
done

echo -e "\n"

#Delete tmp indices
for index in $index_list;do
if echo "$indices_output" | grep ${index}_new >/dev/null; then
echo "Deleting ${index}_new..."
curl -s -X DELETE ${es_url}/${index}_new > /dev/null
fi
done

echo -e "\n\nReindexing done:\n"
curl -s -X GET "${es_url}/_cat/indices/%2A?v=&s=index:desc"
echo -e "\n"
</nowiki>

For our example it takes 11 minutes, and this is the output:

<nowiki>
root@archivematica-test-server:~# time ./script_reindex_new_map.sh

Index list before reindexing:

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open transfers lYqkYjwZRy2XG8CP_3S3PQ 5 1 3 0 11.6kb 11.6kb
yellow open transferfiles K5gnDZyOQz2JdIeZ6adJsQ 5 1 0 0 1.2kb 1.2kb
yellow open aips yAyK_koXThaZcWsBYfzN7w 5 1 17 0 101.4mb 101.4mb
yellow open aipfiles TVrrX8jkRhWWxGfvK_M6zg 5 1 12905 0 2.6gb 2.6gb

Reindex aips in aips_new...
Reindex aipfiles in aipfiles_new...
Reindex transfers in transfers_new...
Reindex transferfiles in transferfiles_new...

Index list after tmp indices creation

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open transfers_new gdFevH8yRdiNTdrPcfo8Lg 5 1 0 0 460b 460b
yellow open transfers lYqkYjwZRy2XG8CP_3S3PQ 5 1 3 0 11.6kb 11.6kb
yellow open transferfiles K5gnDZyOQz2JdIeZ6adJsQ 5 1 0 0 1.2kb 1.2kb
yellow open aips_new uJ-ehaYLTfe_1lOSErfu3Q 5 1 17 0 96.8mb 96.8mb
yellow open aips yAyK_koXThaZcWsBYfzN7w 5 1 17 0 101.4mb 101.4mb
yellow open aipfiles_new 00Xxu7v2QvWsq92gM247xQ 5 1 12905 0 3.1gb 3.1gb
yellow open aipfiles TVrrX8jkRhWWxGfvK_M6zg 5 1 12905 0 2.6gb 2.6gb

Deleting aips...
Deleting aipfiles...
Deleting transfers...
Deleting transferfiles...

Restarting archivematica-dashboard
Wait 30 seconds to ensure dashboard has created the empty indices with new mapping

Indexing aips using aips_new ...
Indexing aipfiles using aipfiles_new ...
Indexing transfers using transfers_new ...

Deleting aips_new...
Deleting aipfiles_new...
Deleting transfers_new...

Reindexing done:

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open transfers FC7aSVPmSmmCc_LTv1AQRA 5 1 3 0 1.2kb 1.2kb
yellow open transferfiles 5JMAft3FQwmosZQFi7eJNw 5 1 0 0 1.2kb 1.2kb
yellow open aips EtwXG3-4SO2Px-4QMRufXA 5 1 17 0 102.1mb 102.1mb
yellow open aipfiles -PFuzslgTeWJ4CWny8VZoA 5 1 12905 0 3gb 3gb

real 10m47.114s
user 0m0.068s
sys 0m0.032s
</nowiki>

'''NOTE''': The script could fail because JAVA heap size out of memory (please, check /var/log/elascticsearch.log). In this case the indices will be empty, so restore /var/lib/elasticsearch from backup, increase Elasticsearch JAVA heap size and try again.

The script uses the elasticsearch API and makes the following actions:

* Reindex the ''transfers'', ''transferfiles'', ''aips'' and ''aipfiles'' indices in new temporary indices
* Delete original indices
* Restart ''archivematica-dashboard'' service to create empty indices with new mappings
* Reindex from temporary indices
* Delete temporary indices

= Restore Elasticsearch heap size when needed=

* Edit ''/etc/default/elasticsearch'' or ''/etc/sysconfig/elasticsearch'' when needed.
* Change ''ES_JAVA_OPTS'' when needed.
* Restart Elasticsearch service when needed (''sudo service elasticsearch restart'')

Update mapping and reindex Elasticsearch indices

2020-12-23T21:25:14Z

Mamedin: /* Run script to reindex and use new mappings */

= Backup Elasticsearch data =

The easiest way to backup the Elasticsearch data is copying the data directory:

sudo service elasticsearch stop
tar cvfz var_lib_elasticsearch.tgz /var/lib/elasticsearch
sudo service elasticsearch start

= List indices to resize the Elasticsearch heap size when needed =

Use the following command to list indices:

curl -s -X GET 'http://localhost:9200/_cat/indices/%2A?v=&s=index:desc'

The output should show something like this:

root@archivematica-test-server:~# curl -s -X GET 'http://localhost:9200/_cat/indices/%2A?v=&s=index:desc'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open transfers lYqkYjwZRy2XG8CP_3S3PQ 5 1 0 0 1.2kb 1.2kb
yellow open transferfiles K5gnDZyOQz2JdIeZ6adJsQ 5 1 0 0 1.2kb 1.2kb
yellow open aips yAyK_koXThaZcWsBYfzN7w 5 1 17 0 101.4mb 101.4mb
yellow open aipfiles TVrrX8jkRhWWxGfvK_M6zg 5 1 11987 0 2.9gb 2.9gb

Take the elasticsearch heap size from ''/etc/default/elasticsearch'' (Ubuntu) or ''/etc/sysconfig/elasticsearch'' (CentOS):

root@ny-gclibrary-test-release-1:~# grep ES_JAVA_OPTS= /etc/default/elasticsearch
#ES_JAVA_OPTS=
ES_JAVA_OPTS="-Xms2g -Xmx2g"

The heap size for the example is 2G.

Ensure your Elasticsearch heap size is greater than the max ''store.size'' in the indices list. For our example, it should be greater than 3GB.

* Edit ''/etc/default/elasticsearch'' or ''/etc/sysconfig/elasticsearch''.
* Change ''ES_JAVA_OPTS'' to a bigger value, in our example:
ES_JAVA_OPTS="-Xms3g -Xmx3g".
* Restart Elasticsearch service for the changes to take effect (''sudo service elasticsearch restart'')

= Run script to reindex and use new mappings =

Use the following script:

<nowiki>
#!/bin/bash

es_url="http://localhost:9200"

index_list='aips aipfiles transfers transferfiles'

echo -e "\nIndex list before reindexing:\n"
curl -s -X GET "${es_url}/_cat/indices/%2A?v=&s=index:desc"
echo -e "\n"

#Clone indices with _reindex API call:
for index in $index_list;do
echo "Reindex ${index} in ${index}_new..."
curl -s -X POST \
${es_url}/_reindex \
-H 'Content-Type: application/json' \
-d '{
"source": {
"index": "'"${index}"'"
},
"dest": {
"index": "'"${index}_new"'"
}
}' > /dev/null
done

echo -e "\n\n"

echo -e "Index list after tmp indices creation\n"
indices_output=$(curl -s -X GET "${es_url}/_cat/indices/%2A?v=&s=index:desc")
curl -s -X GET "${es_url}/_cat/indices/%2A?v=&s=index:desc"
echo -e "\n"

#Delete old indices
for index in $index_list;do
echo "Deleting ${index}..."
curl -s -X DELETE ${es_url}/${index} > /dev/null
done

#Restart archivematica-dashboard to create indices with new mappings
echo -e "\nRestarting archivematica-dashboard"
sudo service archivematica-dashboard restart

#Wait 30 seconds
echo "Wait 30 seconds to ensure dashboard has created the empty indices with new mapping"
sleep 30
echo -e "\n"

#When index has no docs the reindex doesn't create the new index (typically transferfiles index)
#There's a check to ensure the new index has been create before reindexing.
#Reindex fron *_new indices:
for index in $index_list;do
if echo "$indices_output" | grep ${index}_new >/dev/null; then
echo "Indexing ${index} using ${index}_new ..."
curl -s -X POST \
${es_url}/_reindex \
-H 'Content-Type: application/json' \
-d '{
"source": {
"index": "'"${index}_new"'"
},
"dest": {
"index": "'"${index}"'"
}
}' > /dev/null
fi
done

echo -e "\n"

#Delete tmp indices
for index in $index_list;do
if echo "$indices_output" | grep ${index}_new >/dev/null; then
echo "Deleting ${index}_new..."
curl -s -X DELETE ${es_url}/${index}_new > /dev/null
fi
done

echo -e "\n\nReindexing done:\n"
curl -s -X GET "${es_url}/_cat/indices/%2A?v=&s=index:desc"
echo -e "\n"
</nowiki>

For our example it takes 11 minutes, and this is the output:

<nowiki>
root@archivematica-test-server:~# time ./script_reindex_new_map.sh

Index list before reindexing:

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open transfers lYqkYjwZRy2XG8CP_3S3PQ 5 1 3 0 11.6kb 11.6kb
yellow open transferfiles K5gnDZyOQz2JdIeZ6adJsQ 5 1 0 0 1.2kb 1.2kb
yellow open aips yAyK_koXThaZcWsBYfzN7w 5 1 17 0 101.4mb 101.4mb
yellow open aipfiles TVrrX8jkRhWWxGfvK_M6zg 5 1 12905 0 2.6gb 2.6gb

Reindex aips in aips_new...
Reindex aipfiles in aipfiles_new...
Reindex transfers in transfers_new...
Reindex transferfiles in transferfiles_new...

Index list after tmp indices creation

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open transfers_new gdFevH8yRdiNTdrPcfo8Lg 5 1 0 0 460b 460b
yellow open transfers lYqkYjwZRy2XG8CP_3S3PQ 5 1 3 0 11.6kb 11.6kb
yellow open transferfiles K5gnDZyOQz2JdIeZ6adJsQ 5 1 0 0 1.2kb 1.2kb
yellow open aips_new uJ-ehaYLTfe_1lOSErfu3Q 5 1 17 0 96.8mb 96.8mb
yellow open aips yAyK_koXThaZcWsBYfzN7w 5 1 17 0 101.4mb 101.4mb
yellow open aipfiles_new 00Xxu7v2QvWsq92gM247xQ 5 1 12905 0 3.1gb 3.1gb
yellow open aipfiles TVrrX8jkRhWWxGfvK_M6zg 5 1 12905 0 2.6gb 2.6gb

Deleting aips...
Deleting aipfiles...
Deleting transfers...
Deleting transferfiles...

Restarting archivematica-dashboard
Wait 30 seconds to ensure dashboard has created the empty indices with new mapping

Indexing aips using aips_new ...
Indexing aipfiles using aipfiles_new ...
Indexing transfers using transfers_new ...

Deleting aips_new...
Deleting aipfiles_new...
Deleting transfers_new...

Reindexing done:

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open transfers FC7aSVPmSmmCc_LTv1AQRA 5 1 3 0 1.2kb 1.2kb
yellow open transferfiles 5JMAft3FQwmosZQFi7eJNw 5 1 0 0 1.2kb 1.2kb
yellow open aips EtwXG3-4SO2Px-4QMRufXA 5 1 17 0 102.1mb 102.1mb
yellow open aipfiles -PFuzslgTeWJ4CWny8VZoA 5 1 12905 0 3gb 3gb

real 10m47.114s
user 0m0.068s
sys 0m0.032s
</nowiki>

**NOTE**: The script could fail because JAVA heap size out of memory (pleease, check /var/log/elascticsearch.log). In this case the indices will be empty, so restore /var/lib/elasticsearch from backup, increase Elasticsearch JAVA heap size and try again.

The script uses the elasticsearch API and makes the following actions:

* Reindex the ''transfers'', ''transferfiles'', ''aips'' and ''aipfiles'' indices in new temporary indices
* Delete original indices
* Restart ''archivematica-dashboard'' service to create empty indices with new mappings
* Reindex from temporary indices
* Delete temporary indices

= Restore Elasticsearch heap size when needed=

* Edit ''/etc/default/elasticsearch'' or ''/etc/sysconfig/elasticsearch'' when needed.
* Change ''ES_JAVA_OPTS'' when needed.
* Restart Elasticsearch service when needed (''sudo service elasticsearch restart'')

Update mapping and reindex Elasticsearch indices

2020-12-21T16:49:20Z

Mamedin: /* Run script to reindex and use new mappings */

= Backup Elasticsearch data =

The easiest way to backup the Elasticsearch data is copying the data directory:

sudo service elasticsearch stop
tar cvfz var_lib_elasticsearch.tgz /var/lib/elasticsearch
sudo service elasticsearch start

= List indices to resize the Elasticsearch heap size when needed =

Use the following command to list indices:

curl -s -X GET 'http://localhost:9200/_cat/indices/%2A?v=&s=index:desc'

The output should show something like this:

root@archivematica-test-server:~# curl -s -X GET 'http://localhost:9200/_cat/indices/%2A?v=&s=index:desc'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open transfers lYqkYjwZRy2XG8CP_3S3PQ 5 1 0 0 1.2kb 1.2kb
yellow open transferfiles K5gnDZyOQz2JdIeZ6adJsQ 5 1 0 0 1.2kb 1.2kb
yellow open aips yAyK_koXThaZcWsBYfzN7w 5 1 17 0 101.4mb 101.4mb
yellow open aipfiles TVrrX8jkRhWWxGfvK_M6zg 5 1 11987 0 2.9gb 2.9gb

Take the elasticsearch heap size from ''/etc/default/elasticsearch'' (Ubuntu) or ''/etc/sysconfig/elasticsearch'' (CentOS):

root@ny-gclibrary-test-release-1:~# grep ES_JAVA_OPTS= /etc/default/elasticsearch
#ES_JAVA_OPTS=
ES_JAVA_OPTS="-Xms2g -Xmx2g"

The heap size for the example is 2G.

Ensure your Elasticsearch heap size is greater than the max ''store.size'' in the indices list. For our example, it should be greater than 3GB.

* Edit ''/etc/default/elasticsearch'' or ''/etc/sysconfig/elasticsearch'' when needed.
* Change ''ES_JAVA_OPTS'' when needed.
* Restart Elasticsearch service when needed (''sudo service elasticsearch restart'')

= Run script to reindex and use new mappings =

Use the following script:

<nowiki>
#!/bin/bash

es_url="http://localhost:9200"

index_list='aips aipfiles transfers transferfiles'

echo -e "\nIndex list before reindexing:\n"
curl -s -X GET "${es_url}/_cat/indices/%2A?v=&s=index:desc"
echo -e "\n"

#Clone indices with _reindex API call:
for index in $index_list;do
echo "Reindex ${index} in ${index}_new..."
curl -s -X POST \
${es_url}/_reindex \
-H 'Content-Type: application/json' \
-d '{
"source": {
"index": "'"${index}"'"
},
"dest": {
"index": "'"${index}_new"'"
}
}' > /dev/null
done

echo -e "\n\n"

echo -e "Index list after tmp indices creation\n"
indices_output=$(curl -s -X GET "${es_url}/_cat/indices/%2A?v=&s=index:desc")
curl -s -X GET "${es_url}/_cat/indices/%2A?v=&s=index:desc"
echo -e "\n"

#Delete old indices
for index in $index_list;do
echo "Deleting ${index}..."
curl -s -X DELETE ${es_url}/${index} > /dev/null
done

#Restart archivematica-dashboard to create indices with new mappings
echo -e "\nRestarting archivematica-dashboard"
sudo service archivematica-dashboard restart

#Wait 30 seconds
echo "Wait 30 seconds to ensure dashboard has created the empty indices with new mapping"
sleep 30
echo -e "\n"

#When index has no docs the reindex doesn't create the new index (typically transferfiles index)
#There's a check to ensure the new index has been create before reindexing.
#Reindex fron *_new indices:
for index in $index_list;do
if echo "$indices_output" | grep ${index}_new >/dev/null; then
echo "Indexing ${index} using ${index}_new ..."
curl -s -X POST \
${es_url}/_reindex \
-H 'Content-Type: application/json' \
-d '{
"source": {
"index": "'"${index}_new"'"
},
"dest": {
"index": "'"${index}"'"
}
}' > /dev/null
fi
done

echo -e "\n"

#Delete tmp indices
for index in $index_list;do
if echo "$indices_output" | grep ${index}_new >/dev/null; then
echo "Deleting ${index}_new..."
curl -s -X DELETE ${es_url}/${index}_new > /dev/null
fi
done

echo -e "\n\nReindexing done:\n"
curl -s -X GET "${es_url}/_cat/indices/%2A?v=&s=index:desc"
echo -e "\n"
</nowiki>

For our example it takes 11 minutes, and this is the output:

<nowiki>
root@archivematica-test-server:~# time ./script_reindex_new_map.sh

Index list before reindexing:

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open transfers lYqkYjwZRy2XG8CP_3S3PQ 5 1 3 0 11.6kb 11.6kb
yellow open transferfiles K5gnDZyOQz2JdIeZ6adJsQ 5 1 0 0 1.2kb 1.2kb
yellow open aips yAyK_koXThaZcWsBYfzN7w 5 1 17 0 101.4mb 101.4mb
yellow open aipfiles TVrrX8jkRhWWxGfvK_M6zg 5 1 12905 0 2.6gb 2.6gb

Reindex aips in aips_new...
Reindex aipfiles in aipfiles_new...
Reindex transfers in transfers_new...
Reindex transferfiles in transferfiles_new...

Index list after tmp indices creation

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open transfers_new gdFevH8yRdiNTdrPcfo8Lg 5 1 0 0 460b 460b
yellow open transfers lYqkYjwZRy2XG8CP_3S3PQ 5 1 3 0 11.6kb 11.6kb
yellow open transferfiles K5gnDZyOQz2JdIeZ6adJsQ 5 1 0 0 1.2kb 1.2kb
yellow open aips_new uJ-ehaYLTfe_1lOSErfu3Q 5 1 17 0 96.8mb 96.8mb
yellow open aips yAyK_koXThaZcWsBYfzN7w 5 1 17 0 101.4mb 101.4mb
yellow open aipfiles_new 00Xxu7v2QvWsq92gM247xQ 5 1 12905 0 3.1gb 3.1gb
yellow open aipfiles TVrrX8jkRhWWxGfvK_M6zg 5 1 12905 0 2.6gb 2.6gb

Deleting aips...
Deleting aipfiles...
Deleting transfers...
Deleting transferfiles...

Restarting archivematica-dashboard
Wait 30 seconds to ensure dashboard has created the empty indices with new mapping

Indexing aips using aips_new ...
Indexing aipfiles using aipfiles_new ...
Indexing transfers using transfers_new ...

Deleting aips_new...
Deleting aipfiles_new...
Deleting transfers_new...

Reindexing done:

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open transfers FC7aSVPmSmmCc_LTv1AQRA 5 1 3 0 1.2kb 1.2kb
yellow open transferfiles 5JMAft3FQwmosZQFi7eJNw 5 1 0 0 1.2kb 1.2kb
yellow open aips EtwXG3-4SO2Px-4QMRufXA 5 1 17 0 102.1mb 102.1mb
yellow open aipfiles -PFuzslgTeWJ4CWny8VZoA 5 1 12905 0 3gb 3gb

real 10m47.114s
user 0m0.068s
sys 0m0.032s
</nowiki>

The script uses the elasticsearch API and makes the following actions:

* Reindex the ''transfers'', ''transferfiles'', ''aips'' and ''aipfiles'' indices in new temporary indices
* Delete original indices
* Restart ''archivematica-dashboard'' service to create empty indices with new mappings
* Reindex from temporary indices
* Delete temporary indices

= Restore Elasticsearch heap size when needed=

* Edit ''/etc/default/elasticsearch'' or ''/etc/sysconfig/elasticsearch'' when needed.
* Change ''ES_JAVA_OPTS'' when needed.
* Restart Elasticsearch service when needed (''sudo service elasticsearch restart'')

Update ElasticSearch

2019-02-27T17:20:04Z

Mamedin: /* ElasticSearch update */

= ElasticSearch update =

1. Backup elasticsearch data directory

sudo service elasticsearch stop
sudo tar cfz var_lib_elasticsearch_$(date +%y%m%d).tgz /var/lib/elasticsearch
sudo service elasticsearch start

2. Check that ES is up and running, and note the results:

curl -X GET "localhost:9200/_cat/indices?v"
health status index pri rep docs.count docs.deleted store.size pri.store.size
yellow open aips 5 1 24816 1 175.4mb 175.4mb
yellow open transfers 5 1 0 0 720b 720b

Check also if there are any configured snapshot:

curl -XGET "localhost:9200/_snapshot/_all?pretty"

3. Remove Snapshots

If there are any snapshot (step 1), please remove it:

curl -XDELETE "localhost:9200/_snapshot/backup-repo?pretty"

4. Stop elasticsearch service

sudo service elasticsearch stop

5. Check wich elasticsearch version is installed

Redhat/Centos: rpm -q elasticsearch
Ubuntu: dpkg -l elasticsearch

6. Download a temporary copy of elasticsearch (same version than intalled)

wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz

7. Uncompress the downloaded tar.gz and go into it

tar zxvf elasticsearch-1.7.5.tar.gz
cd elasticsearch-1.7.5

8. Copy data from system wide installed elasticsearch, ot the temporary copy

sudo cp /var/lib/elasticsearch data -rf

9. Adjust file permissions

sudo chown <your user>:<your group> data -R

10. Launch the temporary elasticsearch instance

ES_JAVA_OPTS="-Xms2g -Xmx2g" ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500

11. Verify that the indexes were properly copied. Use the port specified in the previous command:

curl -X GET "localhost:9500/_cat/indices?v"

12. Remove elasticsearch 1.7.5 and it's files

Redhat: sudo yum remove elasticsearch
Ubuntu: sudo apt-get remove --purge elasticsearch
sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.5
sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.5

13. Upgrade archivematica and install elasticsearch 6

ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src

14. Configure ElasticSearch 6 to handle reindex from the temporary es

echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml
sudo service elasticsearch restart

15. Migrate the indexes

sudo -u archivematica bash -c " \
set -a -e -x
source /etc/default/archivematica-dashboard || \
source /etc/sysconfig/archivematica-dashboard \
|| (echo 'Environment file not found'; exit 1)
cd /usr/share/archivematica/dashboard
/usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster -t 60 -s 10 http://localhost:9500
";

The command should finish with

All reindex requests ended successfully!

Else, you might need to tweak the timeout (-t) and size (-s) parameters.

16. Verify that the new indexes were created and populated:

curl -X GET "localhost:9200/_cat/indices?v"
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open transferfiles jlu2d2yZQpWwpKKT1ZzMAA 5 1 0 0 1.1kb 1.1kb
yellow open aips VAvONAByRBuVcWBYhlhUmw 5 1 116 0 66.9mb 66.9mb
yellow open transfers OhfLVBzuRqCNUYRCSSTidg 5 1 0 0 1.1kb 1.1kb
yellow open aipfiles clX1gcqnT9CTdwTm8jeatw 5 1 24700 0 70.6mb 70.6mb

You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.

17. Stop the temporary elasticsearch service

kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)

18. Restart all archivematica services

service archivematica-mcp-server restart
service archivematica-mcp-client restart
service archivematica-dashboard restart
service archivematica-storage-service restart

Update ElasticSearch

2019-02-27T17:16:15Z

Mamedin: /* ElasticSearch update */

= ElasticSearch update =

1. Backup elasticsearch data directory

sudo service elasticsearch stop
sudo tar cfz var_lib_elasticsearch_$(date).tgz /var/lib/elasticsearch
sudo service elasticsearch start

2. Check that ES is up and running, and note the results:

curl -X GET "localhost:9200/_cat/indices?v"
health status index pri rep docs.count docs.deleted store.size pri.store.size
yellow open aips 5 1 24816 1 175.4mb 175.4mb
yellow open transfers 5 1 0 0 720b 720b

Check also if there are any configured snapshot:

curl -XGET "localhost:9200/_snapshot/_all?pretty"

3. Remove Snapshots

If there are any snapshot (step 1), please remove it:

curl -XDELETE "localhost:9200/_snapshot/backup-repo?pretty"

4. Stop elasticsearch service

sudo service elasticsearch stop

5. Check wich elasticsearch version is installed

Redhat/Centos: rpm -q elasticsearch
Ubuntu: dpkg -l elasticsearch

6. Download a temporary copy of elasticsearch (same version than intalled)

wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz

7. Uncompress the downloaded tar.gz and go into it

tar zxvf elasticsearch-1.7.5.tar.gz
cd elasticsearch-1.7.5

8. Copy data from system wide installed elasticsearch, ot the temporary copy

sudo cp /var/lib/elasticsearch data -rf

9. Adjust file permissions

sudo chown <your user>:<your group> data -R

10. Launch the temporary elasticsearch instance

ES_JAVA_OPTS="-Xms2g -Xmx2g" ./bin/elasticsearch -d -p elastic-tmp.pid -Des.http.port=9500

11. Verify that the indexes were properly copied. Use the port specified in the previous command:

curl -X GET "localhost:9500/_cat/indices?v"

12. Remove elasticsearch 1.7.5 and it's files

Redhat: sudo yum remove elasticsearch
Ubuntu: sudo apt-get remove --purge elasticsearch
sudo mv /var/lib/elasticsearch /var/lib/elasticsearch-1.7.5
sudo mv /etc/elasticsearch /etc/elasticsearch-1.7.5

13. Upgrade archivematica and install elasticsearch 6

ansible-playbook -i hosts singlenode.yml --tags=elasticsearch,archivematica-src

14. Configure ElasticSearch 6 to handle reindex from the temporary es

echo 'reindex.remote.whitelist: localhost:9500' | sudo tee -a /etc/elasticsearch/elasticsearch.yml
sudo service elasticsearch restart

15. Migrate the indexes

sudo -u archivematica bash -c " \
set -a -e -x
source /etc/default/archivematica-dashboard || \
source /etc/sysconfig/archivematica-dashboard \
|| (echo 'Environment file not found'; exit 1)
cd /usr/share/archivematica/dashboard
/usr/share/archivematica/virtualenvs/archivematica-dashboard/bin/python manage.py reindex_from_remote_cluster -t 60 -s 10 http://localhost:9500
";

The command should finish with

All reindex requests ended successfully!

Else, you might need to tweak the timeout (-t) and size (-s) parameters.

16. Verify that the new indexes were created and populated:

curl -X GET "localhost:9200/_cat/indices?v"
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open transferfiles jlu2d2yZQpWwpKKT1ZzMAA 5 1 0 0 1.1kb 1.1kb
yellow open aips VAvONAByRBuVcWBYhlhUmw 5 1 116 0 66.9mb 66.9mb
yellow open transfers OhfLVBzuRqCNUYRCSSTidg 5 1 0 0 1.1kb 1.1kb
yellow open aipfiles clX1gcqnT9CTdwTm8jeatw 5 1 24700 0 70.6mb 70.6mb

You should have 4 indexes now, and the sum of the aips and aipfiles docs.count column should be equal to the doc.count for the aip index in ElasticSearch 1.7.5. The same happens with the transfers/transferfiles indexes.

17. Stop the temporary elasticsearch service

kill $(cat ~/elasticsearch-1.7.5/elastic-tmp.pid)

18. Restart all archivematica services

service archivematica-mcp-server restart
service archivematica-mcp-client restart
service archivematica-dashboard restart
service archivematica-storage-service restart