Difference between revisions of "Scalability testing"

From Archivematica
Jump to navigation Jump to search
Line 175: Line 175:
 
|-
 
|-
 
!Test date
 
!Test date
!System setup
 
 
!No. transfers
 
!No. transfers
 
!No. SIPs
 
!No. SIPs
Line 185: Line 184:
 
!Comments
 
!Comments
 
|-
 
|-
 +
|11/10/11
 +
|1
 +
|1
 +
|1,000
 
|
 
|
|
+
|60 MB
|
+
|Fail at UploadDIP
|
 
|
 
|
 
|
 
|
 
 
|
 
|
 
|
 
|

Revision as of 16:34, 10 November 2011

Main Page > Development roadmap > Scalability testing

Test File Sets

Test Documents

Test design

Maximums to test for:

  • Max number of SIPS - 10
  • Max number of files in SIP - 10,000
  • Max size of individual file - 30 GiB
  • Max size of SIP - 100 GiB

Baseline amounts:

  • number of SIPS - 1
  • number of files in SIP - 10
  • size of individual file - 1 MiB
  • size of SIP - 100 MiB
Test No. of SIPs No. of files in SIP Max size of individual file Max size of SIP
1. Baseline Test 1 10 1 MiB 100 MiB
2. No. of SIPs 10 10 1 MiB 100 MiB
3. No. of files 1 10,000 1 MiB 100 MiB
4. Max file size 1 10 30 GiB 100 MiB
5. Max SIP size 1 10 1 MiB 100 GiB
...
  • Other tests: combination of maximums

Test results

Transfer

Test date System setup No. transfers No. files Total file size Largest file size Pass/fail Total time Comments


Ingest

Test date System setup No. SIPS No. files Total file size Largest file size Pass/fail Total time Comments


CVA tests

System setup:

  • 4 systems, bare-metal install.
  • 2 cores
  • 4GB ram 9 GB swap
  • xubuntu
  • Dev Nov 10 updated to 1871


Test date No. transfers No. SIPs No. files Total file size Largest file size Pass/fail Total time Comments
11/10/11 1 1 1,000 60 MB Fail at UploadDIP


Old test findings

Findings when running a 2,000 and 10,000 object sip through (May 2011):

  • the dashboard would lose contact with the MCP intermittently this maybe fixed with more cores in a processor
  • transcoder extract packages was producing ALOT of log, causing the logs to rotate every few seconds. Is a less verbose log possible, with option for debug mode?
  • Sanitize (/usr/lib/sanitizeNames.py) names hung For a very long time, MCPServer and MCPClient seemed fine however. My 2,000 object sip contained ~1,000 objects with spaces in the name, the 10,000 object sip contained ~9,000 objects with spaces in the name. On both the 2,000 object sip and the 10,000 object SIP the processing failed to complete.