Scalability testing

Main Page > Development roadmap > Scalability testing

Test File Sets

Test Documents

Test design

Maximums to test for:

• Max number of SIPS - 10
• Max number of files in SIP - 10,000
• Max size of individual file - 30 GiB
• Max size of SIP - 100 GiB

Baseline amounts:

• number of SIPS - 1
• number of files in SIP - 10
• size of individual file - 1 MiB
• size of SIP - 100 MiB

Test	No. of SIPs	No. of files in SIP	Max size of individual file	Max size of SIP
1. Baseline Test	1	10	1 MiB	100 MiB
2. No. of SIPs	10	10	1 MiB	100 MiB
3. No. of files	1	10,000	1 MiB	100 MiB
4. Max file size	1	10	30 GiB	100 MiB
5. Max SIP size	1	10	1 MiB	100 GiB
...

• Other tests: combination of maximums

Test results

Transfer

Test date	System setup	No. transfers	No. files	Total file size	Largest file size	Pass/fail	Total time	Comments

Ingest

Test date	System setup	No. SIPS	No. files	Total file size	Largest file size	Pass/fail	Total time	Comments

Old test findings

Findings when running a 2,000 and 10,000 object sip through (May 2011):

• the dashboard would lose contact with the MCP intermittently this maybe fixed with more cores in a processor
• transcoder extract packages was producing ALOT of log, causing the logs to rotate every few seconds. Is a less verbose log possible, with option for debug mode?
• Sanitize (/usr/lib/sanitizeNames.py) names hung For a very long time, MCPServer and MCPClient seemed fine however. My 2,000 object sip contained ~1,000 objects with spaces in the name, the 10,000 object sip contained ~9,000 objects with spaces in the name. On both the 2,000 object sip and the 10,000 object SIP the processing failed to complete.