Meeting 20120502

Development

• Joseph working on bagit ingest
• Austin moved off of avconv back to ffmpeg (Issue 985)
• Austin updating dependecies since 12.04 Xubuntu release
• Mike finished the source directory picker (/administration/sources/ ) and made some browser performance enhancements to the file browser/directory picker JavaScript
• Mike worked on microservices grouping
• Joseph adding the transcoder to MCP this week

Deployment

• Joseph patched CVA deployment because of issues with filenames (duplication and periods)
• Austin rebuilt CVA packages for conflicts and msql scalability views
• Joseph is testing big transfers that falied at CVA

Testing

• Courtney to start testing and doing outstanding QA from issues list this week
• Evelyn testing normalization tools in MCP - will create wiki page with structured testing including significant characterics analysis
• Austin has 6 nodes current and ready to test on openhosting but hit a snag with the dashboard (?) - not passing MCP approval (Mike and Joseph to help troubleshoot)

Documentation

• Courtney updated wiki overview page

Chat log

(10:30:38 AM) courtney: archivematica - i can take notes
(10:30:40 AM) djjuhasz: yep
(10:30:54 AM) courtney: Dev?
(10:31:03 AM) berwin22: Lots of work on bagit.
(10:31:23 AM) courtney: i'm doing a mockup for that today, too
(10:31:30 AM) berwin22: Added obtain/release mysql lock to the MCP poll for jobsAwaitingApproval.
(10:31:30 AM) berwin22: -The reason this hasn't been done in the past was that large outuputs of microservices, being logged to the database, may delay the responce and briefly show a not connected in the dashboard.
(10:31:30 AM) berwin22: -The reason for doing this now, is it's a strong indicator of whether there's been an interal problem in the MCP. This would then be reflected in the dashboard (better user feedback).
(10:31:47 AM) djjuhasz: (Qubit notes: http://qubit-toolkit.org/wiki/index.php?title=Meeting_20120502)
(10:32:03 AM) Sevein: thank you djjuhasz 
(10:32:03 AM) ARTi: I moved off of avconv back to ffmpeg for http://code.google.com/p/archivematica/issues/detail?id=985
(10:32:12 AM) Sevein: JessicaB: I can do the VM setup for fiver 
(10:32:17 AM) ARTi: leaving it open until tested
(10:32:22 AM) JessicaB: Sevein: thanks that would be great.
(10:32:24 AM) epmclellan: thanks ARTi, I'll test it
(10:32:33 AM) djjuhasz: thanks Sevein :)
(10:33:30 AM) berwin22: That's it for dev for me
(10:33:40 AM) ARTi:  Ive had to catch up a number of dependencies since the official 12.04 release,  everything seems to be installing with out conflict now
(10:33:46 AM) berwin22: mcantelon - archivematica dev?
(10:34:04 AM) mcantelon: Mike C finished the source directory picker (/administration/sources/ ) and made some browser performance enhancements to the file browser/directory picker JavaScript
(10:34:22 AM) mcantelon: Mike C did a bit of work on microservice grouping too
(10:34:23 AM) peterVG: ARTi: great
(10:34:23 AM) Sevein: fiver: please download virtualbox and install it, it's pretty straightforward - http://download.virtualbox.org/virtualbox/4.1.14/VirtualBox-4.1.14-77440-Win.exe
(10:34:43 AM) peterVG: mcantelon: cool
(10:34:56 AM) epmclellan: exciting to have mcantelon working on dev now
(10:35:05 AM) mcantelon: :D
(10:35:36 AM) courtney: deployment?
(10:36:02 AM) epmclellan: CVA ran into a bunch of trouble with filename issues
(10:36:14 AM) epmclellan: periods in filenames, identical filenames for different objects, etc
(10:36:28 AM) epmclellan: berwin22 submitted a patch for the periods in the filenames
(10:36:52 AM) ARTi: austin rebuilt packages (for conflicts, and mysql scalability views)
(10:37:43 AM) courtney: Joseph is testing a few giant transfers that kept failing at CVA
(10:38:15 AM) berwin22: I have a copy of them, testing them is on my todo list
(10:38:16 AM) ARTi: I have 6 nodes current and ready to test on openhosting but hit a snag with the dashboard
(10:38:51 AM) ARTi: I think its the dashboard anyways.. does not seem to be passing approval to the mcp
(10:39:05 AM) epmclellan: normalization tools in LTS 12.04 need more testing, I think
(10:39:21 AM) epmclellan: I'll keep testing them
(10:39:43 AM) peterVG: ARTi: can mike or joseph help troubleshoot that?
(10:40:02 AM) berwin22: I sent out an email about the status of various sponsored issues, and I think I'll be able to get enough time to get the transcoder into the MCP. So I'm pushing that this week.
(10:40:07 AM) ARTi: yes, I think so.  epmclellan pass any issues on to me that need to go to ubuntu 
(10:40:15 AM) epmclellan: will do
(10:40:30 AM) ARTi: mcantelon maybe we can look at it after the meeting?
(10:40:36 AM) mcantelon: Sounds good
(10:40:54 AM) peterVG: epmclellan: can we introduce some structure to our normalization testing as well, so that we can repeat over course of future tool updates or addition of new tools?
(10:41:02 AM) berwin22: ARTi: is the MCPrpcClient available?
(10:41:07 AM) epmclellan: peterVG: yes
(10:41:19 AM) epmclellan: I'll start a page
(10:41:20 AM) peterVG: i am assuming similar to our significant characteristics tests? 
(10:41:42 AM) peterVG: with info on paramaters, settings, etc
(10:41:43 AM) epmclellan: tests should include significant characteristics analysis, yes
(10:41:44 AM) peterVG: epmclellan: thx
(10:42:03 AM) epmclellan: also batch testing, because some files fail, like some bmp files did this week
(10:42:06 AM) epmclellan: while others pass
(10:42:07 AM) epmclellan: :(
(10:43:05 AM) epmclellan: eg http://code.google.com/p/archivematica/issues/detail?id=987
(10:43:10 AM) courtney: berwin22 is setting me up to test some outstanding QA issues assigned to me today
(10:43:22 AM) epmclellan: great
(10:43:36 AM) epmclellan: there will be a lot of testing needed over the next couple of months
(10:43:51 AM) courtney: and now i'm here almost always!
(10:43:59 AM) epmclellan: :D
(10:44:08 AM) ARTi: berwin22: installing
(10:44:13 AM) fiver1 [~user@70.71.0.109] entered the room.
(10:44:44 AM) djjuhasz: sweet! @ courtney working 4 days for artefactual :)
(10:44:46 AM) berwin22: ARTi: will help determine if it's a dashboard thing, or a MCP thing
(10:45:01 AM) peterVG: courtney: woohoo
(10:45:06 AM) djjuhasz: lol
(10:45:15 AM) courtney: testing?
(10:45:20 AM) peterVG: one sec
(10:45:23 AM) courtney: i meant docs
(10:45:32 AM) djjuhasz: peterVG: without punctuation that's looks like the lamest, flatest woo-hoo ever :P
(10:45:44 AM) epmclellan: woohoo...sigh
(10:45:47 AM) djjuhasz: lol
(10:45:56 AM) peterVG: wooho...o...whatever
(10:46:06 AM) epmclellan: no docs from me
(10:46:10 AM) peterVG: issues mtg last week
(10:46:25 AM) peterVG: more work needed on that list?
(10:46:27 AM) epmclellan: oh yeah, had blotted that from my memory
(10:46:30 AM) courtney: oh yeah, we had an issues meeting last week... 
(10:46:34 AM) epmclellan: separate meeting re dashboard issues
mcantelon Misty 
(10:46:47 AM) epmclellan: I think courtney sent out an invite?
(10:47:00 AM) courtney: Sevein and mcantelon need to chat and figure out which of Sevein's issues are still active and then which need to transfer to mcantelon
(10:47:06 AM) peterVG: okay
(10:47:20 AM) mcantelon: cool
(10:47:25 AM) ARTi: berwin22: works! thanks
(10:47:29 AM) courtney: we need a sep mtg wrt dashboard issues?
(10:47:31 AM) peterVG: more work on prioritizing 0.9 issues as well or is the current 'high' list accurate?
(10:47:46 AM) epmclellan: think it's pretty accurate now
(10:47:48 AM) courtney: peterVG: we updated it - should be accurate
(10:47:50 AM) berwin22: ARTi: sudo apache2ctl restart ?
(10:47:52 AM) Sevein: courtney: yep, that would be easy
(10:47:55 AM) peterVG: great
(10:48:08 AM) peterVG: berwin22: mentioned his one concern on that list is the mbox normalization
(10:48:14 AM) peterVG: currently no tool identified for this?
(10:48:19 AM) ARTi: berwin22: Ive already reboot the server,  so I dont think that would be it
(10:48:29 AM) epmclellan: peterVG: mbox normalization?
(10:48:35 AM) mcantelon: courtney: yeah meeting sounds good
(10:48:39 AM) berwin22: Extracting attachments
(10:48:44 AM) ARTi: berwin22: had the same behaviour.. was unresponsive to approval,  now its buzzing through with the cli
(10:49:00 AM) epmclellan: i.e. getting attachments out of maildir
(10:49:06 AM) epmclellan: right?
(10:49:17 AM) berwin22: yes.
(10:49:50 AM) epmclellan: so maybe have to convert to mbox, then extract attachments?
(10:50:47 AM) fiver left the room (quit: Read error: Operation timed out).
(10:50:54 AM) berwin22: I don't like the way we are currently extracting attachments from mbox. that could do with some improvement:
(10:50:55 AM) berwin22: extract just attachments, using perl library functions
(10:50:55 AM) berwin22: rather than individual files
(10:51:25 AM) epmclellan: how do we link the attachments back to their messages?
(10:52:01 AM) berwin22: currently, or with the proposed idea?
(10:52:10 AM) epmclellan: with the proposed idea
(10:52:42 AM) berwin22: pull whatever identifiers we want to use from the email: I believe emails contain a UUID.
(10:52:57 AM) epmclellan: actually, yes, they have all kinds of individual identifiers
(10:53:01 AM) berwin22: not sure what version
(10:53:07 AM) epmclellan: we can talk more later
(10:53:08 AM) peterVG: if not we can assign them one, assign one also to each extracting attachment. make the connection in the METS file
(10:53:33 AM) peterVG: use hierarchical representation so it can be imported as parent (email) and children (attachment) levels into Qubit?
(10:53:47 AM) berwin22: peterVG: I don't see how we would assign emails, not extracted identifiers
(10:54:20 AM) peterVG: i meant assign a file UUID to the email file
(10:54:35 AM) peterVG: sounds like we need more discussion on this post-meeting?
(10:54:36 AM) berwin22: Also: I need to have a look at the tool ARTi was suggesting for access normalization:
(10:54:37 AM) berwin22: I think it may strip directory structure.
(10:54:55 AM) epmclellan: peterVG: good idea, let's go over this separately
(10:55:23 AM) ARTi: afiak mbox does not have directory structure..  so it removes them all  ^^
(10:55:23 AM) peterVG: completing email ingest--to-access is a priority for 0.9 though so we should have a clear strategy quickly if that is currently not the case
(10:56:22 AM) berwin22: ARTi: I think you're right
(10:56:27 AM) epmclellan: peterVG: let's meet when you're available
(10:56:33 AM) epmclellan: and berwin22
(10:56:40 AM) berwin22: k
(10:56:49 AM) berwin22: let me set courney up first
(10:56:58 AM) epmclellan: sure
(10:57:30 AM) epmclellan: meeting finished?
(10:57:54 AM) courtney: meeting finished
(10:58:00 AM) berwin22: yup
(10:58:03 AM) peterVG: later