Scalability testing

From Archivematica
Revision as of 14:11, 18 April 2012 by Evelyn (talk | contribs)
Jump to navigation Jump to search

Main Page > Development roadmap > Scalability testing

Test File Sets

Test Documents

Test design

Maximums to test for:

  • Max number of SIPS - 10
  • Max number of files in SIP - 10,000
  • Max size of individual file - 30 GiB
  • Max size of SIP - 100 GiB

Baseline amounts:

  • number of SIPS - 1
  • number of files in SIP - 10
  • size of individual file - 1 MiB
  • size of SIP - 100 MiB
Test No. of SIPs No. of files in SIP Max size of individual file Max size of SIP
1. Baseline Test 1 10 1 MiB 100 MiB
2. No. of SIPs 10 10 1 MiB 100 MiB
3. No. of files 1 10,000 1 MiB 100 MiB
4. Max file size 1 10 30 GiB 100 MiB
5. Max SIP size 1 10 1 MiB 100 GiB
...
  • Other tests: combination of maximums

CVA tests

System setup:

  • Bare-metal install, 1 processor
  • 2 cores
  • 4GB ram 9 GB swap
  • xubuntu

Note: excludes store AIP and upload DIP micro-services except where noted


Test date No. transfers/SIPs No. files Total file size Largest file size AIP size Total time Comments
2011/11/10 1/1 1,000 12.1 GB 60 MB
  • Failed at prepareAIP due to max Bag size: Issue 785
  • Failed at uploadDIP due to max post size limit in ica-atom (8M).
2011/11/10 1/1 1 2.7 GB 2.7 GB Failed at prepareAIP due to max Bag size: Issue 785
2011/11/18 1/1 1,000 12.1 GB 60 MB 7.2 GB 4 hrs 30 mins Access normalization only
2011/12/02 2/2 1,998 13 GB 21 MB Access normalization only
2011/12/11 1/1 1,000 6.51 GB 21 MB 3.5 GB Access normalization only
2011/12/11 2/2 1,996 13.8 GB 27 MB 7.2 GB Access normalization only
2011/12/13 3/3 2,974 18.6 GB 20 MB 10.3 GB 3 hrs 19 mins Access normalization only
2011/12/14 4/4 3,993 24.6 GB 22 MB 13.2 GB 3 hrs 16 mins Access normalization only
2011/12/15 4/4 3,982 43 GB 12 MB 15 GB 3 hrs 30 mins Access normalization only
2011/12/15 6/6 5,113 34.1 GB 38 MB 19.8 GB 4 hrs 2 mins Access normalization only
2012/01/04 6/6 5,845 42.4 GB 33 MB 24 GB 3 hrs 52 mins Access normalization only
2012/01/05 3/3 2,957 20.9 GB 45 MB 13.6 GB 4 hrs Access normalization only
2012/01/05 6/6 5,947 33 GB 52 MB 19.2 GB 4 hrs 47 mins Access normalization only
2012/01/12 6/6 4,847 38.5 GB 58 MB 23.2 GB 4 hrs 43 mins Access normalization only
2012/01/13 6/6 5,912 101.6 GB 175 MB 63.8 GB 8 hrs 53 mins Access normalization only
2012/01/17 1/1 1 1.4 GB 1.4 GB 0.6 GB 25 mins Access normalization only
2012/01/17 5/5 23 19.7 GB 2.1 GB 19 GB 4 hrs 1 min Access normalization only
2012/01/18 2/2 2 3.8 GB 2.1 GB 3.7 GB 1 hr 11 mins Access normalization only
2012/01/20 6/6 14 6.1 GB 1.3 GB 5.9 GB 48 mins Access normalization only
2012/02/07 5/5 5 56.7 GB 25.4 GB 55.5 GB 4 hrs 51 mins No normalization
2012/02/08 5/5 10 124.4 GB 23.8 GB 122.2 GB 8 hrs 21 mins No normalization
2012/02 1/1 1044 7.5 GB 12.4 MB 32.8 GB >16 hrs Preservation and access normalization
2012/02 1/1 104 611.6 MB 7.1 MB 2.58 GB <2 hrs Preservation and access normalization
2012/02 1/1 2125 47.1 GB 35.9 MB 46.2 GB >24 hrs Preservation and access normalization
2012/03 1/1 1654 7.9 GB 11.7 MB 37.7 GB >16 hrs Preservation and access normalization
2012/03 1/1 1195 5.7 GB 9.9 MB 26.8 GB >12 hrs Preservation and access normalization
2012/03/22 1/1 11.0 GB 246.3 MB GB Preservation and access normalization
2012/03/22 1/1 6.7 GB 9.7 MB GB Preservation and access normalization
2012/03/26 1/1 6.6 GB 14.3 MB GB Preservation and access normalization
2012/03 1/1 18.1 GB 11.7 MB Preservation and access normalization


Multi-processor testing

Problem statement

  • Does the amount of processing time decrease for each additional processing station added?
  • If yes, by how much?

Constants and variables

Constants:

  • Ram amount
  • Ram speed
  • Disk size
  • Cpu frequency

Variables:

  • Number of processors
  • Number of transfer(s)
  • Size of transfer(s)
  • Number of files(s)

Ideal network for testing network consists of 6nodes+ each with dual core processor, 2GB+ memory, and 6GB+ disk space. Due to limited disk capacity, current tests are running with 5 nodes.

Testing data

  • All testing data will be be preserved for analysis. Select data will be reported on this wiki.

Network setup

Unknowns: network saturation.

To gather seperatly: Supported cpu instruction set (included in 'sudo cat /proc/cpuinfo').

To gather at end of each test: Dump and reset the database. Store the dump.

HOSTNAME Processor Memory Ram speed/timing Disk/s Size Shared directory disk write speed Shared directory disk read speed IP Filesystem Services Network Connection Speed
test01server 4x500mhz 2048mb 6GB+35GB 10.10.0.1 ext4 MCPServer,MySQL,NFS,MCPClient
test01client01 2x500mhz 1024mb 6GB 10.10.0.11 ext4,NFS MCPClient
test01client02 2x500mhz 1024mb 6GB 10.10.0.12 ext4,NFS MCPClient
test01client03 2x500mhz 1024mb 6GB 10.10.0.12 ext4,NFS MCPClient
test01client04 2x500mhz 1024mb 6GB 10.10.0.14 ext4,NFS MCPClient

Testing metrics

Our results are derived from running 000.zip through the archivematica pipe line, and then extracting MYSQL- timing views from the database. This gives us a clearer picture of productivity of clients.

To extract testing data, run the following on the command line(test0001.log being the output file name): 

mysql MCP -e "select * from PDI_by_unit; select * from processingDurationInformation;" > test0001.log

you should get a file(test0001.log) containing something like: 

SIP_OR_TRANSFER_UUID    unitType        Total time processing   Number_of_tasks Average time per task   total file size number of files count( DISTINCT  FilesByUnit.fileUUID)  average file size KB    average file size MB    time per task per MB    currentLocation currentPath
032dc1ee-de1e-486b-b43e-d5602b4bc986    Transfer        838:59:59       4928    00:47:39        758373178       980     980     773.85018163    0.77385018      01:01:35        %sharedPath%watchedDirectories/SIPCreation/completedTransfers/000-88-032dc1ee-de1e-486b-b43e-d5602b4bc986/      NULL
ca229fa8-073b-4bd3-980e-ced7e810c5d3    SIP     133:07:00       991     00:08:04        758373178       980     980     773.85018163    0.77385018      00:10:25        NULL    %sharedPath%watchedDirectories/workFlowDecisions/createDip/000-88-ca229fa8-073b-4bd3-980e-ced7e810c5d3/
SIP_OR_TRANSFER_UUID    client  Time_spent_in_system    Time_spent_processing   Number_of_tasks currentLocation currentPath
09210bef-9053-44b4-8a95-e1a4022826eb    server_1        00:00:01        00:00:01        1       NULL    NULL
4c500821-6e7c-4bc2-8087-73b6e7002078    server_1        00:00:00        00:00:00        1       NULL    NULL
def33d3e-1bc8-4a14-ab5f-c06d3e2475a5    server_4        00:00:00        00:00:00        1       NULL    NULL
032dc1ee-de1e-486b-b43e-d5602b4bc986    server_1        01:16:23        03:59:35        4928    %sharedPath%watchedDirectories/SIPCreation/completedTransfers/000-88-032dc1ee-de1e-486b-b43e-d5602b4bc986/      NULL
ca229fa8-073b-4bd3-980e-ced7e810c5d3    client04_1      00:01:55        00:08:09        991     NULL    %sharedPath%watchedDirectories/workFlowDecisions/createDip/000-88-ca229fa8-073b-4bd3-980e-ced7e810c5d3/

Test results

Retrieved from "https://wiki.archivematica.org/index.php?title=Scalability_testing&oldid=5753"