Scalability testing

From Archivematica
Revision as of 10:31, 4 May 2011 by Austin (talk | contribs)
Jump to navigation Jump to search

Main Page > Development roadmap > Scalability testing

Maximums to test for:

  • Max number of SIPS - 10
  • Max number of files in SIP - 10,000
  • Max size of individual file - 30 GiB
  • Max size of SIP - 100 GiB

Baseline amounts:

  • number of SIPS - 1
  • number of files in SIP - 10
  • size of individual file - 1 MiB
  • size of SIP - 100 MiB
Test No. of SIPs No. of files in SIP Max size of individual file Max size of SIP
1. Baseline Test 1 10 1 MiB 100 MiB
2. No. of SIPs 10 10 1 MiB 100 MiB
3. No. of files 1 10,000 1 MiB 100 MiB
4. Max file size 1 10 30 GiB 100 MiB
5. Max SIP size 1 10 1 MiB 100 GiB
...

Other tests: combination of maximums

Test Metrics:

  • Success: completed/not completed
  • Time: by (a) each micro-service or by (b) receiveSIP to completion of storeAIP/uploadDIP?
  • Number of errors
  • ?
  • ?


Setup:

  • dual core AMD 2.2ghz
  • 2gb of ram
  • 2x2TB in raid 0
  • archivematica installed bare metal

Findings when running a 10,000 object sip through:

  • Sanitize names hung For a very long time, MCPServer and MCPClient seemed fine however /usr/lib/archivematica/MCPClient/clientScripts/sanitizeObjectNames.py
  • transcoder extract packages was producing ALOT of log, causing the logs to rotate every few seconds.