Difference between revisions of "Scalability testing"
Jump to navigation
Jump to search
| Line 183: | Line 183: | ||
!Comments | !Comments | ||
|- | |- | ||
| − | |11/10 | + | |2011/11/10 |
|1 | |1 | ||
|1 | |1 | ||
| Line 195: | Line 195: | ||
*Failed at uploadDIP due to max post size limit in ica-atom (8M). | *Failed at uploadDIP due to max post size limit in ica-atom (8M). | ||
|- | |- | ||
| − | |11/10 | + | |2011/11/10 |
|1 | |1 | ||
|1 | |1 | ||
| Line 205: | Line 205: | ||
|Failed at prepareAIP due to max Bag size: Issue 785 | |Failed at prepareAIP due to max Bag size: Issue 785 | ||
|- | |- | ||
| − | |11/18 | + | |2011/11/18 |
|1 | |1 | ||
|1 | |1 | ||
| Line 218: | Line 218: | ||
*Did not upload DIP | *Did not upload DIP | ||
|- | |- | ||
| − | |12/02 | + | |2011/12/02 |
|2 | |2 | ||
|2 | |2 | ||
| Line 228: | Line 228: | ||
| | | | ||
*Did not normalize to preservation copies because all files were uncompressed TIFF | *Did not normalize to preservation copies because all files were uncompressed TIFF | ||
| − | |||
*Did not upload DIP | *Did not upload DIP | ||
|- | |- | ||
| + | |2011/12/11 | ||
| + | | | ||
|} | |} | ||
</br> | </br> | ||
Revision as of 14:21, 14 December 2011
Main Page > Development roadmap > Scalability testing
Test File Sets
Test design
Maximums to test for:
- Max number of SIPS - 10
- Max number of files in SIP - 10,000
- Max size of individual file - 30 GiB
- Max size of SIP - 100 GiB
Baseline amounts:
- number of SIPS - 1
- number of files in SIP - 10
- size of individual file - 1 MiB
- size of SIP - 100 MiB
| Test | No. of SIPs | No. of files in SIP | Max size of individual file | Max size of SIP |
|---|---|---|---|---|
| 1. Baseline Test | 1 | 10 | 1 MiB | 100 MiB |
| 2. No. of SIPs | 10 | 10 | 1 MiB | 100 MiB |
| 3. No. of files | 1 | 10,000 | 1 MiB | 100 MiB |
| 4. Max file size | 1 | 10 | 30 GiB | 100 MiB |
| 5. Max SIP size | 1 | 10 | 1 MiB | 100 GiB |
| ... |
- Other tests: combination of maximums
Test results
Transfer
| Test date | System setup | No. transfers | No. files | Total file size | Largest file size | Pass/fail | Total time | Comments |
|---|---|---|---|---|---|---|---|---|
Ingest
| Test date | System setup | No. SIPS | No. files | Total file size | Largest file size | Pass/fail | Total time | Comments |
|---|---|---|---|---|---|---|---|---|
CVA tests
System setup:
- 4 systems, bare-metal install.
- 2 cores
- 4GB ram 9 GB swap
- xubuntu
| Test date | No. transfers | No. SIPs | No. files | Total file size | Largest file size | Pass/fail | Total time | Comments |
|---|---|---|---|---|---|---|---|---|
| 2011/11/10 | 1 | 1 | 1,000 | 12.1 GB | 60 MB | Fail |
| |
| 2011/11/10 | 1 | 1 | 1 | 2.7 GB | 2.7 GB | Fail | Failed at prepareAIP due to max Bag size: Issue 785 | |
| 2011/11/18 | 1 | 1 | 1,000 | 12.1 GB | 60 MB | Pass | 4.5 hours |
|
| 2011/12/02 | 2 | 2 | 1,998 | 13 GB | 21 MB | Pass |
| |
| 2011/12/11 |
Old test findings
Findings when running a 2,000 and 10,000 object sip through (May 2011):
- the dashboard would lose contact with the MCP intermittently this maybe fixed with more cores in a processor
- transcoder extract packages was producing ALOT of log, causing the logs to rotate every few seconds. Is a less verbose log possible, with option for debug mode?
- Sanitize (/usr/lib/sanitizeNames.py) names hung For a very long time, MCPServer and MCPClient seemed fine however. My 2,000 object sip contained ~1,000 objects with spaces in the name, the 10,000 object sip contained ~9,000 objects with spaces in the name. On both the 2,000 object sip and the 10,000 object SIP the processing failed to complete.