Difference between revisions of "Scalability testing"
Jump to navigation
Jump to search
| Line 78: | Line 78: | ||
= Test results = | = Test results = | ||
| − | + | == Transfer == | |
| − | |||
{| border="1" cellpadding="10" cellspacing="0" width=90% | {| border="1" cellpadding="10" cellspacing="0" width=90% | ||
| Line 85: | Line 84: | ||
! Test date | ! Test date | ||
!System setup | !System setup | ||
| + | !No. transfers | ||
!No. files | !No. files | ||
!Total file size | !Total file size | ||
!Largest file size | !Largest file size | ||
| − | + | !Pass/fail | |
| − | |||
| − | |||
| − | !Pass/ | ||
!Total time | !Total time | ||
!Comments | !Comments | ||
|- | |- | ||
| − | |||
| − | |||
| | | | ||
| | | | ||
| Line 107: | Line 102: | ||
| | | | ||
|- | |- | ||
| − | |||
| − | |||
| | | | ||
| | | | ||
| Line 119: | Line 112: | ||
| | | | ||
|- | |- | ||
| − | |||
| − | |||
| | | | ||
| | | | ||
| Line 131: | Line 122: | ||
| | | | ||
|- | |- | ||
| − | | | + | |} |
| − | + | </br> | |
| − | + | ||
| − | + | == Ingest == | |
| − | + | ||
| − | + | {| border="1" cellpadding="10" cellspacing="0" width=90% | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | | | ||
|- | |- | ||
| − | + | ! Test date | |
| − | + | !System setup | |
| − | + | !No. SIPS | |
| − | + | !No. files | |
| − | + | !Total file size | |
| − | + | !Largest file size | |
| − | + | !Pass/fail | |
| − | + | !Total time | |
| − | + | !Comments | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
|- | |- | ||
| − | |||
| − | |||
| | | | ||
| | | | ||
| Line 179: | Line 149: | ||
| | | | ||
|- | |- | ||
| − | |||
| − | |||
| | | | ||
| | | | ||
| Line 191: | Line 159: | ||
| | | | ||
|- | |- | ||
| − | |||
| − | |||
| | | | ||
| | | | ||
| Line 204: | Line 170: | ||
|- | |- | ||
|} | |} | ||
| + | </br> | ||
| + | </br> | ||
== Old test findings == | == Old test findings == | ||
Revision as of 18:18, 8 November 2011
Main Page > Development roadmap > Scalability testing
Test design
Maximums to test for:
- Max number of SIPS - 10
- Max number of files in SIP - 10,000
- Max size of individual file - 30 GiB
- Max size of SIP - 100 GiB
Baseline amounts:
- number of SIPS - 1
- number of files in SIP - 10
- size of individual file - 1 MiB
- size of SIP - 100 MiB
| Test | No. of SIPs | No. of files in SIP | Max size of individual file | Max size of SIP |
|---|---|---|---|---|
| 1. Baseline Test | 1 | 10 | 1 MiB | 100 MiB |
| 2. No. of SIPs | 10 | 10 | 1 MiB | 100 MiB |
| 3. No. of files | 1 | 10,000 | 1 MiB | 100 MiB |
| 4. Max file size | 1 | 10 | 30 GiB | 100 MiB |
| 5. Max SIP size | 1 | 10 | 1 MiB | 100 GiB |
| ... |
- Other tests: combination of maximums
Test metrics
- Success: completed/not completed
- Time: by (a) each micro-service or by (b) receiveSIP to completion of storeAIP/uploadDIP?
- Number of errors
- ?
- ?
Setup:
- dual core AMD 2.2ghz
- 2gb of ram
- 2x2TB in raid 0
- archivematica installed bare metal
Test results
Transfer
| Test date | System setup | No. transfers | No. files | Total file size | Largest file size | Pass/fail | Total time | Comments |
|---|---|---|---|---|---|---|---|---|
Ingest
| Test date | System setup | No. SIPS | No. files | Total file size | Largest file size | Pass/fail | Total time | Comments |
|---|---|---|---|---|---|---|---|---|
Old test findings
Findings when running a 2,000 and 10,000 object sip through:
- the dashboard would loose contact with the MCP intermittently this maybe fixed with more cores in a processor
- transcoder extract packages was producing ALOT of log, causing the logs to rotate every few seconds. Is a less verbose log possible, with option for debug mode?
- Sanitize (/usr/lib/sanitizeNames.py) names hung For a very long time, MCPServer and MCPClient seemed fine however. My 2,000 object sip contained ~1,000 objects with spaces in the name, the 10,000 object sip contained ~9,000 objects with spaces in the name. On both the 2,000 object sip and the 10,000 object SIP the processing failed to complete.