Difference between revisions of "Scalability testing"

From Archivematica
Jump to navigation Jump to search
Line 78: Line 78:
 
= Test results =
 
= Test results =
  
−
{| class="wikitable"
+
 
 +
 
 +
 
 +
{| border="1" cellpadding="10" cellspacing="0" width=90%
 
|-
 
|-
 
! Test date
 
! Test date
 
!System setup
 
!System setup
−
!No. of files
+
!No. files
 
!Total file size
 
!Total file size
 
!Largest file size
 
!Largest file size
−
!Maximum directory depth
+
!Max. directory depth
−
!No. of transfers
+
!No. transfers
−
!No. of SIPs
+
!No. SIPs
 
!Pass/Fail
 
!Pass/Fail
 
!Total time
 
!Total time
 
!Comments
 
!Comments
 +
|-
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|-
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 +
|
 
|-
 
|-
 
|  
 
|  

Revision as of 18:02, 8 November 2011

Main Page > Development roadmap > Scalability testing

Test design

Maximums to test for:

  • Max number of SIPS - 10
  • Max number of files in SIP - 10,000
  • Max size of individual file - 30 GiB
  • Max size of SIP - 100 GiB

Baseline amounts:

  • number of SIPS - 1
  • number of files in SIP - 10
  • size of individual file - 1 MiB
  • size of SIP - 100 MiB
Test No. of SIPs No. of files in SIP Max size of individual file Max size of SIP
1. Baseline Test 1 10 1 MiB 100 MiB
2. No. of SIPs 10 10 1 MiB 100 MiB
3. No. of files 1 10,000 1 MiB 100 MiB
4. Max file size 1 10 30 GiB 100 MiB
5. Max SIP size 1 10 1 MiB 100 GiB
...
  • Other tests: combination of maximums

Test metrics

  • Success: completed/not completed
  • Time: by (a) each micro-service or by (b) receiveSIP to completion of storeAIP/uploadDIP?
  • Number of errors
  • ?
  • ?


Setup:

  • dual core AMD 2.2ghz
  • 2gb of ram
  • 2x2TB in raid 0
  • archivematica installed bare metal

Test results

Test date System setup No. files Total file size Largest file size Max. directory depth No. transfers No. SIPs Pass/Fail Total time Comments


Findings when running a 2,000 and 10,000 object sip through:

  • the dashboard would loose contact with the MCP intermittently this maybe fixed with more cores in a processor
  • transcoder extract packages was producing ALOT of log, causing the logs to rotate every few seconds. Is a less verbose log possible, with option for debug mode?
  • Sanitize (/usr/lib/sanitizeNames.py) names hung For a very long time, MCPServer and MCPClient seemed fine however. My 2,000 object sip contained ~1,000 objects with spaces in the name, the 10,000 object sip contained ~9,000 objects with spaces in the name. On both the 2,000 object sip and the 10,000 object SIP the processing failed to complete.
Retrieved from "https://wiki.archivematica.org/index.php?title=Scalability_testing&oldid=4812"