Meeting 20111102

Development

• Joseph has been working on METS implementation, Evelyn is going to test
• Scalability issues are paramount right now because of CVA deployment
• Jesus has finished the SWORD protocol work and DIPs now upload to Qubit trunk
• Jesus is working on Transfer tab
• Review of roadmap and major issues shows a tight timeline for 0.8 dev

Deployment

• Austin deployed dev at cva on October 27. Austin thinks that one of CVA's routers lost its forwarding information, it may need some IT intervention
• Joseph updated cov of 1808 yesterday. Updating a distributed enviroment isn't the cleanest thing.
• We are not planning any other pre-release installs

Testing

• Austin will conduct scalability testing today, Evelyn and Austin will work on capturing results on Scalability testing page

Chat log

(10:34:09 AM) peterVG: wazup Archivematica?
(10:34:13 AM) epmclellan: hi
(10:34:58 AM) djjuhasz: epmclellan: on phone
(10:35:02 AM) berwin22: Criticals List???:
(10:35:02 AM) berwin22: large number of files in a sip crashes the MCP server
(10:35:02 AM) berwin22: unicode causes crashes in archivematica
(10:35:02 AM) berwin22: sips/transfers hidden in the dashboard, re-appear on restart
(10:35:02 AM) berwin22: Metadata
(10:35:04 AM) berwin22: -rights
(10:35:04 AM) berwin22: -DC
(10:35:24 AM) epmclellan: off phone now
(10:35:44 AM) peterVG: ouch
(10:35:51 AM) epmclellan: our most serious critical is probably the scalability issue
(10:36:01 AM) epmclellan: in terms of bugs
(10:36:05 AM) peterVG: yes definitely but that's why we're doing on site testing
(10:36:12 AM) epmclellan: yes
(10:36:19 AM) berwin22: yeah... I'm going to pound my head on that again today. I got a really massive head-ache yesterday from it
(10:36:22 AM) peterVG: can we recreate in office on ciaus (when/if its back up)
(10:36:38 AM) peterVG: berwin22: sounds like we may have reinforcements soon
(10:36:41 AM) epmclellan: CVA is only interested in scalability testing at the moment
(10:36:42 AM) berwin22: we can re-create on any archivematica machine
(10:36:43 AM) peterVG: but hang in there
(10:36:51 AM) ARTi: I setup a xubuntu vm on caius, to test the dev env
(10:36:51 AM) epmclellan: reinforcements?
(10:37:03 AM) peterVG: how come ARTi didn't catch earlier with 10,000 file tests?
(10:37:16 AM) epmclellan: it worked for him
(10:37:17 AM) ARTi: alot of changes were made since then
(10:37:25 AM) epmclellan: when did you do the tests?
(10:37:44 AM) ARTi: it was still the old directory structures
(10:37:55 AM) peterVG: okay, was thinking about this yesterday. need epmclellan to organize scalability tests, just like functional tests from this point forward to 0.8 release
(10:38:16 AM) epmclellan: can I just command ARTi to do some scalability tests?
(10:38:21 AM) epmclellan: :)
(10:38:25 AM) epmclellan: ARTi: do some tests
(10:38:30 AM) JessicaB: *command*
(10:38:31 AM) epmclellan: please
(10:39:33 AM) ARTi: ./runARTiRun.sh
(10:39:33 AM) peterVG: i.e. so we can keep re-running same scalability tests and have them documented as part of test plan. 
(10:39:33 AM) peterVG: epmclellan yes but i want you to help with documenting the results
(10:39:33 AM) peterVG: we need to put more process/structure into iterative scalablity testing
(10:39:33 AM) peterVG: this is not just because of CVA yesterday, is part of long-term plan to get to 1.0
(10:39:39 AM) epmclellan: ok
(10:39:44 AM) peterVG: but now right time to get serious about it
(10:39:44 AM) berwin22: I think it would help to have some more structured testing
(10:39:44 AM) berwin22: "It use to work" doesn't help me much
(10:39:44 AM) berwin22: when did it break?
(10:40:23 AM) peterVG: right, let's discuss testing plan in office tomorrow. austin can join on Skype/chat
(10:40:27 AM) epmclellan: ARTi: I can document the results
(10:40:27 AM) djjuhasz: lol
(10:40:39 AM) epmclellan: are you available for testing today?
(10:40:58 AM) peterVG: berwin22: pinning issue reports to specific trunk r# 's will be a good start
(10:40:59 AM) ARTi: yes, Ive set up a VM on caius todo bulk tests
(10:41:07 AM) peterVG: cool
(10:41:09 AM) epmclellan: ok, let's keep in touch in the chatroom
(10:41:10 AM) epmclellan: thanks
(10:41:28 AM) peterVG: we've got all the basics in place, we just need to be on same page on how to implement coherently/cosistently
(10:41:51 AM) epmclellan: I'll make up a table on the wiki page, something that can easily capture tests and results
(10:42:04 AM) epmclellan: for scalability, I mean
(10:42:06 AM) peterVG: epmclellan ARTi and I have already started one
(10:42:10 AM) peterVG: please work from that
(10:42:13 AM) epmclellan: ok
(10:42:13 AM) ***peterVG looking for link
(10:42:31 AM) epmclellan: I think I know the ones you mean
(10:42:43 AM) peterVG: http://archivematica.org/wiki/index.php?title=Scalability_testing
(10:43:07 AM) epmclellan: yes, I'm familiar with that one
(10:43:11 AM) berwin22: I like the idea of a spreadsheet, with a list of issues/features. 
(10:43:11 AM) berwin22: each time the tester tests, they do it against a revision, and stores it in one of our shared folders. Issues would still need to be created.
(10:43:14 AM) epmclellan: hey, is anyone taking notes?
(10:43:30 AM) epmclellan: I'll take notes
(10:43:38 AM) peterVG: thx epmclellan
(10:43:40 AM) berwin22: thank you epmclellan
(10:44:49 AM) epmclellan: can we have update on dev?
(10:45:00 AM) epmclellan: Sevein: and berwin22 have been quite busy
(10:45:16 AM) berwin22: I made some progress on this: Issue 756:One-step SIP creation when one transfer = 1 SIP
(10:45:30 AM) Sevein: yep, the new qubit-upload.php file is working well so far, I am working in the transfer tab right now
(10:45:31 AM) epmclellan: yes, and I've tested and provided feedback
(10:45:42 AM) epmclellan: re One-step SIP
(10:45:51 AM) epmclellan: Sevein: that's great
(10:46:02 AM) berwin22: did anyone have anything to add to the list of criticals I posted earlier?
(10:46:08 AM) epmclellan: I'm taking a close look at the METS implementation today
(10:46:23 AM) peterVG: btw, we need to save 'critical' priority for post code freeze
(10:46:36 AM) peterVG: everthing else that 'must' make the 0.8 release should be 'high'
(10:46:47 AM) peterVG: we've got close to 30 of those now
(10:46:49 AM) epmclellan: ok,
(10:46:56 AM) berwin22: That's new
(10:47:00 AM) epmclellan: we have a number of new features that have yet to be started
(10:47:05 AM) epmclellan: such as the access tab, etc
(10:47:13 AM) berwin22: thought no changes to the code post code freeze
(10:47:13 AM) epmclellan: basically, everything on the roadmap is high
(10:47:17 AM) berwin22: all testing
(10:47:24 AM) peterVG: berwin22: we've been doing that for past couple of Qubit releases
(10:47:47 AM) peterVG: berwin22: right, which then may reveal a 'critical' bug(s)
(10:47:53 AM) berwin22: yes, and similar for archivematica, all critical, post feature freeze
(10:48:14 AM) berwin22: same thing, but with feature freeze**
(10:48:19 AM) peterVG: berwin22: right, sorry
(10:48:29 AM) djjuhasz: peterVG: I think application breaking bugs should always be "Critical"
(10:48:35 AM) peterVG: so those are in fact 0.8 criticals
(10:48:40 AM) berwin22: I think we're in feature freeze now, are we not?
(10:48:42 AM) peterVG: djjuhasz: agreed
(10:48:44 AM) peterVG: berwin22: yes
(10:48:52 AM) djjuhasz: It's just a matter of not using "Critical" for new features
(10:49:10 AM) peterVG: fair enough. don't mind me. carry on...
(10:49:27 AM) berwin22: Criticals:
(10:49:27 AM) berwin22: large number of files in a sip crashes the MCP server
(10:49:27 AM) berwin22: unicode causes crashes in archivematica
(10:49:27 AM) berwin22: sips/transfers hidden in the dashboard, re-appear on restart
(10:49:27 AM) berwin22: Metadata - METS
(10:49:28 AM) berwin22: -rights
(10:49:28 AM) berwin22: -DC
(10:49:32 AM) peterVG: the point i was going to get to is that we have circa 30 'high' 0.8 issues right now
(10:49:39 AM) berwin22: was there anything else to add to the list? visualizations?
(10:49:43 AM) epmclellan: access tab
(10:49:51 AM) epmclellan: full-text indexing
(10:49:57 AM) epmclellan: visualization report
(10:49:59 AM) peterVG: transfer tab
(10:50:16 AM) peterVG: ----- and that ^ is it! -----
(10:50:48 AM) peterVG: ...or maybe not?
(10:51:04 AM) berwin22: ... and whatever other bumps we find 
(10:51:05 AM) epmclellan: checking roadmap...
(10:51:06 AM) djjuhasz: phew, at least it's looks like a small list... </sarcasm>
(10:51:09 AM) ARTi: I have been working on deploying ACE, but thats kinda sidenote
(10:51:29 AM) epmclellan: ARTi: important, though
(10:51:42 AM) peterVG: ARTi: yes, imp't
(10:52:03 AM) epmclellan: I think we've covered everything on the roadmap
(10:52:41 AM) peterVG: phew
(10:52:57 AM) epmclellan: it's a lot, we have pretty tight timeline here
(10:53:15 AM) berwin22: Alright, we have the crit list; go go gadget programmer; just let me eat some spinach
(10:53:42 AM) epmclellan: deployment now?
(10:54:52 AM) berwin22: I updated cov of 1808 yesterday. Updating a distributed enviroment isn't the cleanest thing. Never has been.
(10:55:53 AM) ARTi: yep, deployed dev at cva last thursday
(10:56:13 AM) peterVG: berwin22: inspector Popeye Gadget
(10:56:59 AM) peterVG: ARTi: so we can ssh in now to sync their trunk with our dev over next couple of weeks?
(10:57:03 AM) epmclellan: peterVG: are we planning any other pre-release installs? eg at UBC Library?
(10:57:09 AM) peterVG: epmclellan nope
(10:57:12 AM) epmclellan: k
(10:57:29 AM) ARTi: peterVG: I think one of there routers lost its forwarding information
(10:57:32 AM) ARTi: their*
(10:57:39 AM) ARTi: so we need some IT intervention
(10:58:35 AM) ***peterVG finishes reading berwin22 's debried
(10:58:36 AM) berwin22: presently port 22 is forwarded from the internal router, with an insecure pw
(10:58:57 AM) peterVG: but that sits behind another firewall which we don't have admin privilige on, right?
(10:59:06 AM) peterVG: so we have to ask them to okay the port 22 forward?
(10:59:08 AM) berwin22: correct
(10:59:16 AM) peterVG: okay, will ask today
(10:59:44 AM) berwin22: we'll also need a static ip on the internal network
(11:00:21 AM) peterVG: then moving DIPs from that local CVA network to ICA-AtoM (CVA02) VM on CVA VMware server is another issue but will discuss offline
(11:01:34 AM) peterVG: berwin22: right, otherwise we can't find the network right?
(11:03:23 AM) epmclellan: Should we move onto testing?
(11:03:36 AM) peterVG: okay guys it's past 11- shall we book a time to discuss testing tomorrow?
(11:03:40 AM) epmclellan: sure