Meeting 20110504

Development

• Austin added http://code.google.com/p/archivematica/source/browse/trunk/src/SIPCreationTools/bin/transferSip?spec=svn1363&r=1363 which should help with http://code.google.com/p/archivematica/issues/detail?id=455. Also may be work around for http://code.google.com/p/archivematica/issues/detail?id=482
• Joseph added groups to db, so the transcoder database has the ability to associate file extensions with groups
• Austin submitted some package fixes yesterday, made them a bit smarter when reinstalling or updating: http://code.google.com/p/archivematica/source/detail?r=1358
• Issue 482 <http://code.google.com/p/archivematica/issues/detail?id=482>:Receive SIP watch directory behavior. - Rename move etc. Joseph is going to take a crack at hacking this to work in the MCP
• Evelyn has spent time working with City of Vancouver Archives on video preservation
• Peter is planning to work on AIP structure review/design over next couple of weeks. He will try to highlight any dev requirements as early as next week (i.e. using Bagit as SIPs).
  • Joseph: that will lead into the MCP dbase/data model review which we can discuss further and finally pin-down at ArtefactualCon2011

Deployment

• Daily packages are working so far.. still a few things to fix here and there.. but so far so good

Testing

• Scalability testing (http://archivematica.org/wiki/index.php?title=Scalability_testing): /usr/lib/sanitizeNames.py seems to choke with lots of objects with lots of spaces
• Evelyn has been testing metadata extraction from video files
  • FITS doesn't recognize MKV video files, which is City of Vancouver Archives' preferred wrapper for video files

Chat log

(10:34:18 AM) epmclellan: I'll take Archivematica notes
(10:34:29 AM) peterVG: dev
(10:35:27 AM) david.juhasz: epmclellan: thanks
(10:35:27 AM) Austin: I just added http://code.google.com/p/archivematica/source/browse/trunk/src/SIPCreationTools/bin/transferSip?spec=svn1363&r=1363  which should help with  http://code.google.com/p/archivematica/issues/detail?id=455
(10:35:44 AM) david.juhasz: Tim: can you please send me a link to the physical storage discussion?
(10:35:47 AM) david.juhasz: oops
(10:35:49 AM) david.juhasz: too late :)
(10:35:51 AM) peterVG: Austin: nice
(10:36:01 AM) Austin: also maybe work around for http://code.google.com/p/archivematica/issues/detail?id=482
(10:36:08 AM) Austin: may need to use this at ubc tomorrow
(10:36:49 AM) peterVG: cool
(10:38:11 AM) epmclellan: berwin22: ping
(10:38:31 AM) berwin22: hi
(10:38:37 AM) epmclellan: Archivematica dev news?
(10:39:37 AM) berwin22: been working on small time consuming stuff
(10:39:44 AM) berwin22: no major changes that I can think of
(10:39:59 AM) epmclellan: what about submissionDocumentation workflow
(10:40:02 AM) berwin22: added groups to db
(10:40:05 AM) epmclellan: and manual normalization workflow?
(10:40:11 AM) berwin22: I thought that was last week
(10:40:14 AM) epmclellan: oh
(10:41:05 AM) berwin22: so the transcoder database has the ability to associate file extensions with groups
(10:41:12 AM) Austin: I submitted some package fixes yesturday, made em a bit smarter when reinstalling or updating 
(10:41:23 AM) Austin: http://code.google.com/p/archivematica/source/detail?r=1358
(10:42:17 AM) berwin22: http://code.google.com/p/archivematica/issues/detail?id=558 this needs discussion on action we want to take
(10:42:17 AM) Austin: all on dev from me
(10:42:38 AM) epmclellan: berwin22: yes, we can discuss that this afternoon
(10:36:08 AM) Austin: may need to use this at ubc tomorrow
(10:36:49 AM) peterVG: cool
(10:37:10 AM) epmclellan: berwin22: ping
(10:37:30 AM) berwin22: hi
(10:37:36 AM) epmclellan: Archivematica dev news?
(10:38:37 AM) berwin22: been working on small time consuming stuff
(10:38:43 AM) berwin22: no major changes that I can think of
(10:38:58 AM) epmclellan: what about submissionDocumentation workflow
(10:39:01 AM) berwin22: added groups to db
(10:39:04 AM) epmclellan: and manual normalization workflow?
(10:39:11 AM) berwin22: I thought that was last week
(10:39:13 AM) epmclellan: oh
(10:40:04 AM) berwin22: so the transcoder database has the ability to associate file extensions with groups
(10:40:11 AM) Austin: I submitted some package fixes yesturday, made em a bit smarter when reinstalling or updating 
(10:40:22 AM) Austin: http://code.google.com/p/archivematica/source/detail?r=1358
(10:41:16 AM) berwin22: http://code.google.com/p/archivematica/issues/detail?id=558 this needs discussion on action we want to take
(10:41:16 AM) Austin: all on dev from me
(10:41:37 AM) epmclellan: berwin22: yes, we can discuss that this afternoon
(10:43:20 AM) epmclellan: any deployment news?
(10:43:53 AM) peterVG: ubc workstation install tomorrow?
(10:43:56 AM) epmclellan: Joseph, what about that issue with the virtualBox version on Windows - i.e. the problem we ran into at UBC lab for the workshop
(10:44:06 AM) epmclellan: is that a deployment issue?
(10:44:30 AM) Austin: daily packages are working so far.. still a few things to fix here and there.. but so far so good
(10:44:32 AM) epmclellan: peterVG: yes, tomorrow afternoon at UBC
(10:44:45 AM) berwin22: Issue 482 <http://code.google.com/p/archivematica/issues/detail?id=482>:Receive SIP watch directory behavior. - Rename move etc.
I'm going to take a crack at hacking this to work in the MCP
(10:46:43 AM) epmclellan: documentation?
(10:47:15 AM) epmclellan: nothing from me, by the way, this past week I've been mainly preparing and delivering a workshop + working with CVA re video preservation
(10:47:39 AM) peterVG: i'm doing an Archivematica webinar tomorrow for Lyrasis
(10:40:04 AM) berwin22: so the transcoder database has the ability to associate file extensions with groups
(10:40:11 AM) Austin: I submitted some package fixes yesturday, made em a bit smarter when reinstalling or updating 
(10:40:22 AM) Austin: http://code.google.com/p/archivematica/source/detail?r=1358
(10:41:16 AM) berwin22: http://code.google.com/p/archivematica/issues/detail?id=558 this needs discussion on action we want to take
(10:41:16 AM) Austin: all on dev from me
(10:41:37 AM) epmclellan: berwin22: yes, we can discuss that this afternoon
(10:42:20 AM) epmclellan: any deployment news?
(10:42:52 AM) peterVG: ubc workstation install tomorrow?
(10:42:55 AM) epmclellan: Joseph, what about that issue with the virtualBox version on Windows - i.e. the problem we ran into at UBC lab for the workshop
(10:43:05 AM) epmclellan: is that a deployment issue?
(10:43:29 AM) Austin: daily packages are working so far.. still a few things to fix here and there.. but so far so good
(10:43:31 AM) epmclellan: peterVG: yes, tomorrow afternoon at UBC
(10:43:44 AM) berwin22: Issue 482 <http://code.google.com/p/archivematica/issues/detail?id=482>:Receive SIP watch directory behavior. - Rename move etc.
I'm going to take a crack at hacking this to work in the MCP
(10:45:42 AM) epmclellan: documentation?
(10:46:14 AM) epmclellan: nothing from me, by the way, this past week I've been mainly preparing and delivering a workshop + working with CVA re video preservation
(10:46:38 AM) peterVG: i'm doing an Archivematica webinar tomorrow for Lyrasis
(10:48:08 AM) peterVG: http://www.lyrasis.org/Products-and-Services/Digital-Services/Staying-on-TRAC.aspx
(10:48:35 AM) Austin: peterVG: cool
(10:48:38 AM) epmclellan: neat
(10:48:53 AM) peterVG: I'm planning to work on AIP structure review/design over next couple of weeks
(10:49:27 AM) berwin22: cool: I'm looking forward to seeing the results of that peter
(10:49:28 AM) Jessica  Bushey: The workshop Evelyn gave at UBC was excellent. I received VERY POSITIVE feedback from a colleague who attended the workshop.
(10:49:28 AM) peterVG: but will try to highlight any dev requirements as early as next week (i.e. using Bagit as SIPs)
(10:49:44 AM) epmclellan: thanks Jessica
(10:50:10 AM) peterVG: berwin22: yes, and then that will lead into the MCP dbase/data model review which we can discuss further and finally pin-down at ArtefactualCon2011
(10:51:13 AM) peterVG: Jessica  Bushey: Yes epmclellan's slides are great
(10:51:21 AM) epmclellan: thanks :)
(10:51:29 AM) peterVG: she's basically spelled out our missing authenticity requirements
(10:52:21 AM) peterVG: epmclellan: remember when we were doing the InterPARES analysis with Joe and we talked about the hypothetical 'Certify Authenticity' process? Well, now we can start to fill in that blank...
(10:52:41 AM) Jessica  Bushey: Are the slides available on the wiki?
(10:52:46 AM) epmclellan: that will be interesting, it will come from the AIP structure review?
(10:52:52 AM) epmclellan: Jessica  Bushey: I sent them to the AABC
(10:52:56 AM) peterVG: ^ that was all to impress Luciana who didn't show. At least that didn't go to waste :-)
(10:53:00 AM) epmclellan: should be posted on their website
(10:50:12 AM) peterVG: Jessica  Bushey: Yes epmclellan's slides are great
(10:50:20 AM) epmclellan: thanks :)
(10:50:28 AM) peterVG: she's basically spelled out our missing authenticity requirements
(10:51:20 AM) peterVG: epmclellan: remember when we were doing the InterPARES analysis with Joe and we talked about the hypothetical 'Certify Authenticity' process? Well, now we can start to fill in that blank...
(10:51:40 AM) Jessica  Bushey: Are the slides available on the wiki?
(10:51:44 AM) epmclellan: that will be interesting, it will come from the AIP structure review?
(10:51:51 AM) epmclellan: Jessica  Bushey: I sent them to the AABC
(10:51:55 AM) peterVG: ^ that was all to impress Luciana who didn't show. At least that didn't go to waste :-)
(10:51:59 AM) epmclellan: should be posted on their website
(10:53:10 AM) Jessica  Bushey: thnks
(10:53:10 AM) epmclellan: ah
(10:53:15 AM) epmclellan: right
(10:53:31 AM) epmclellan: I talked a little more about authenticity than I had originally planned to
(10:53:37 AM) epmclellan: but then Luciana didn't show
(10:53:45 AM) epmclellan: prolly a good thing
(10:53:46 AM) peterVG: we can post a copy as well at artefactual.com/downloads and link from the archivematica.org News section
(10:53:53 AM) epmclellan: sure
(10:54:26 AM) peterVG: there's already a line for AABC workshop but right now it just points to AABC conference page. Just swap that for the slide download
(10:54:37 AM) epmclellan: ok, I can do that
(10:55:21 AM) peterVG: anymore deployment or docs?
(10:55:27 AM) epmclellan: not from me
(10:55:43 AM) Austin: testing?
(10:56:13 AM) epmclellan: Austin: how is the scalability testing going?
(10:56:16 AM) Austin: added some notes to the bottom of the page here
(10:56:42 AM) Austin: /usr/lib/sanitizeNames.py seems to choke with lots of objects with lots of spaces
(10:57:07 AM) Austin: woops
(10:57:18 AM) Austin: link - http://archivematica.org/wiki/index.php?title=Scalability_testing
(10:57:33 AM) epmclellan: thanks
(10:57:48 AM) epmclellan: you were also having FITS problems?
(10:52:45 AM) peterVG: we can post a copy as well at artefactual.com/downloads and link from the archivematica.org News section
(10:52:52 AM) epmclellan: sure
(10:53:25 AM) peterVG: there's already a line for AABC workshop but right now it just points to AABC conference page. Just swap that for the slide download
(10:53:36 AM) epmclellan: ok, I can do that
(10:54:20 AM) peterVG: anymore deployment or docs?
(10:54:26 AM) epmclellan: not from me
(10:54:42 AM) Austin: testing?
(10:55:12 AM) epmclellan: Austin: how is the scalability testing going?
(10:55:15 AM) Austin: added some notes to the bottom of the page here
(10:55:41 AM) Austin: /usr/lib/sanitizeNames.py seems to choke with lots of objects with lots of spaces
(10:56:06 AM) Austin: woops
(10:56:17 AM) Austin: link - http://archivematica.org/wiki/index.php?title=Scalability_testing
(10:56:32 AM) epmclellan: thanks
(10:56:47 AM) epmclellan: you were also having FITS problems?
(10:58:10 AM) Austin: got past that, was caused by http://code.google.com/p/archivematica/issues/detail?id=531
(10:58:17 AM) epmclellan: oh, nice
(10:58:51 AM) epmclellan: I've been testing metadata extraction from video files
(10:58:57 AM) epmclellan: for Vancouver Archives
(10:59:05 AM) epmclellan: video preservation is going to kill me
(10:59:32 AM) Austin: with the office sip everything is working up to normalization... so I know smaller sips are getting passed sanitize
(10:59:32 AM) epmclellan: peterVG: I'll update you this afternoon
(10:59:56 AM) Misty: It will kill all of us, I'm sure. ;) Sorry to intrude, but what metadata are you looking at extracting?
(11:00:03 AM) epmclellan: hi Misty!
(11:00:07 AM) Misty: Hi!
(11:00:08 AM) epmclellan: We can chat after the meeting
(11:00:31 AM) epmclellan: which will be in a minute or two, we're almost done I think
(11:00:34 AM) Misty: OK, great. I should probably leave for lunch soon anyway - I'll message you later this afternoon.
(11:00:37 AM) peterVG: Austin: great start on the testing. some interesting findings already
(11:00:38 AM) epmclellan: sure
(11:00:56 AM) berwin22: Re "video preservation is going to kill me" is it the preservation, or the metadata extraction?
(11:01:17 AM) epmclellan: the whole dang thing, but am currently working on metadata extraction in Archivematica
(11:01:27 AM) epmclellan: FITS doesn't have a clue about mkv video files
(11:01:41 AM) epmclellan: which is City of Vancouver's first choice for a video wrapper
(11:01:46 AM) Austin: yeah, Ill try a 10,000 object file with out any spaces in the file name, to see if I can get by sanatizenames
(11:01:57 AM) epmclellan: Austin: that sounds like fun
(11:02:16 AM) berwin22: directories are the killer for spaces in sanitized names
(11:02:44 AM) berwin22: espcially directories with many levels beneath them
(11:02:50 AM) Austin: the SIP had a space in the name.. however no other directories
(10:59:07 AM) epmclellan: We can chat after the meeting
(10:59:30 AM) epmclellan: which will be in a minute or two, we're almost done I think
(10:59:33 AM) Misty: OK, great. I should probably leave for lunch soon anyway - I'll message you later this afternoon.
(10:59:36 AM) peterVG: Austin: great start on the testing. some interesting findings already
(10:59:37 AM) epmclellan: sure
(10:59:55 AM) berwin22: Re "video preservation is going to kill me" is it the preservation, or the metadata extraction?
(11:00:16 AM) epmclellan: the whole dang thing, but am currently working on metadata extraction in Archivematica
(11:00:26 AM) epmclellan: FITS doesn't have a clue about mkv video files
(11:00:40 AM) epmclellan: which is City of Vancouver's first choice for a video wrapper
(11:00:45 AM) Austin: yeah, Ill try a 10,000 object file with out any spaces in the file name, to see if I can get by sanatizenames
(11:00:56 AM) epmclellan: Austin: that sounds like fun
(11:01:15 AM) berwin22: directories are the killer for spaces in sanitized names
(11:01:43 AM) berwin22: espcially directories with many levels beneath them
(11:01:49 AM) Austin: the SIP had a space in the name.. however no other directories
(11:03:07 AM) Austin: just a single folder full of objects
(11:03:48 AM) berwin22: no zipped, or other packaged things?
(11:04:35 AM) Austin: a few gziped things
(11:04:40 AM) Austin: yeah
(11:05:02 AM) Austin: containing single files
(11:05:05 AM) epmclellan: think meeting is done?
(11:05:16 AM) peterVG: yup