Difference between revisions of "Scalability testing"

From Archivematica
Jump to navigation Jump to search
Line 161: Line 161:
 
</br>
 
</br>
  
 +
= CVA tests =
  
−
== Old test findings ==
+
System setup:
 +
 
 +
== Transfer ==
 +
 
 +
{| border="1" cellpadding="10" cellspacing="0" width=90%
 +
|-
 +
! Test date
 +
!System setup
 +
!No. transfers
 +
!No. files
 +
!Total file size
 +
!Largest file size
 +
!Pass/fail
 +
!Total time
 +
!Comments
 +
|-
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
|}
 +
</br>
 +
 
 +
== Ingest ==
 +
 
 +
{| border="1" cellpadding="10" cellspacing="0" width=90%
 +
|-
 +
! Test date
 +
!System setup
 +
!No. SIPS
 +
!No. files
 +
!Total file size
 +
!Largest file size
 +
!Pass/fail
 +
!Total time
 +
!Comments
 +
|-
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
|}
 +
</br>
 +
 
 +
= Old test findings =
  
 
Findings when running a 2,000 and 10,000 object sip through (May 2011):
 
Findings when running a 2,000 and 10,000 object sip through (May 2011):

Revision as of 16:18, 10 November 2011

Main Page > Development roadmap > Scalability testing

Test File Sets

Test Documents

Test design

Maximums to test for:

  • Max number of SIPS - 10
  • Max number of files in SIP - 10,000
  • Max size of individual file - 30 GiB
  • Max size of SIP - 100 GiB

Baseline amounts:

  • number of SIPS - 1
  • number of files in SIP - 10
  • size of individual file - 1 MiB
  • size of SIP - 100 MiB
Test No. of SIPs No. of files in SIP Max size of individual file Max size of SIP
1. Baseline Test 1 10 1 MiB 100 MiB
2. No. of SIPs 10 10 1 MiB 100 MiB
3. No. of files 1 10,000 1 MiB 100 MiB
4. Max file size 1 10 30 GiB 100 MiB
5. Max SIP size 1 10 1 MiB 100 GiB
...
  • Other tests: combination of maximums

Test results

Transfer

Test date System setup No. transfers No. files Total file size Largest file size Pass/fail Total time Comments


Ingest

Test date System setup No. SIPS No. files Total file size Largest file size Pass/fail Total time Comments


CVA tests

System setup:

Transfer

Test date System setup No. transfers No. files Total file size Largest file size Pass/fail Total time Comments


Ingest

Test date System setup No. SIPS No. files Total file size Largest file size Pass/fail Total time Comments


Old test findings

Findings when running a 2,000 and 10,000 object sip through (May 2011):

  • the dashboard would lose contact with the MCP intermittently this maybe fixed with more cores in a processor
  • transcoder extract packages was producing ALOT of log, causing the logs to rotate every few seconds. Is a less verbose log possible, with option for debug mode?
  • Sanitize (/usr/lib/sanitizeNames.py) names hung For a very long time, MCPServer and MCPClient seemed fine however. My 2,000 object sip contained ~1,000 objects with spaces in the name, the 10,000 object sip contained ~9,000 objects with spaces in the name. On both the 2,000 object sip and the 10,000 object SIP the processing failed to complete.
Retrieved from "https://wiki.archivematica.org/index.php?title=Scalability_testing&oldid=4821"