Meeting 20110817

Development

• Josephs been working on a "preIngest area" where transfers are done, and SIPs are created. info, moving towards
• Evelyns worked on SIP creation and transfer mockups
• Peter added some notes on indexing & visualization requirements
• Joseph worked on re-implementing the old workflow
• Lots of dashboard brainstorming going on, to be contained here

Deployment

• David got Misty's awesome ruby script for extracting video metadata running on Evelyn's workstation
• Evelyn's been trying to extract and measure significant properties of video files

Documentation

• Evelyn updated video preservation pages and Transfer/Create SIP page

TODO

• Evelyn + Peter - dev meeting about relationship with accession medata
• Evelyn to begin testing soon, which would mean the dashboard would need to be working (Sevien?).
• Evelyn + Peter to sort out dashboard requirements
• Austin + Joseph to help RAC with PST files

Chat log

(10:31:07 AM) epmclellan: Archivematica meeting?
(10:31:07 AM) JessicaB left the room (quit: Read error: Connection reset by peer).
(10:31:27 AM) ARTi: WWEEEE
(10:31:30 AM) JessicaB [~Jessica@S01060018e789c0a0.vs.shawcable.net] entered the room.
(10:31:35 AM) ARTi: ?
(10:31:40 AM) ARTi: I can take notes
(10:31:56 AM) peterVG: thanks ARTi
(10:32:00 AM) peterVG: dev?
(10:32:00 AM) epmclellan: thanks ARTi
(10:32:20 AM) ***ARTi dosnt have much, diving into my 0.7.2 issues finally this morning
(10:32:27 AM) berwin22: thanks ARTi :)
(10:32:30 AM) berwin22: I've been working on a "preIngest area" where transfers are done, and SIPs are created.
(10:32:30 AM) berwin22: http://code.google.com/p/archivematica/source/browse/trunk#trunk%2Fsrc%2FMCPServer%2FsharedDirectoryStructure%2FwatchedDirectories%2FpreIngest
(10:32:30 AM) berwin22: When the user says an active transfer is completed, some processing is done on the transfer, then it is moved to completed transfers. 
(10:32:52 AM) berwin22: This is working towards: http://archivematica.org/wiki/index.php?title=Transfer_and_SIP_creation
(10:33:01 AM) epmclellan: nice
(10:33:57 AM) epmclellan: peterVG: did you get some feedback about transfer & SIP creation at curatecamp?
(10:34:16 AM) peterVG: epmclellan: yes
(10:34:38 AM) peterVG: thanks very much for getting those mockups up in time
(10:34:39 AM) Sevein: peterVG: Barcelona - Madrid 2pm PT
(10:34:43 AM) epmclellan: sure, np
(10:34:50 AM) peterVG: you'll note that I added some notes during the session about indexing & visualization requirements
(10:34:50 AM) Sevein: (offtopic, sorry)
(10:34:59 AM) epmclellan: yes, I did
(10:35:04 AM) peterVG: courtney also took some additional notes with more suggestions from the room
(10:35:10 AM) epmclellan: we should have another dev meeting about it
(10:35:15 AM) epmclellan: and to talk about metadata
(10:35:32 AM) peterVG: epmclellan: yes, to talk about the relationship with accession medata in particular right?
(10:35:37 AM) epmclellan: yup
(10:35:43 AM) peterVG: transfer md <-> sip md <-> accession md
(10:36:11 AM) peterVG: read your email about that but needed more time to think it through. shall we schedule a meeting on Thu or Fri?
(10:36:19 AM) epmclellan: sounds good to me
(10:36:40 AM) berwin22: ok
(10:36:46 AM) epmclellan: After that I can work out the requirements in detail
(10:36:53 AM) peterVG: great
(10:37:37 AM) berwin22: Worked on re-implementing the old workflow.
(10:37:38 AM) berwin22: Archivematica is extremely brittle at this time. I would like to get epmclellan testing soon, which would mean the dashboard would need to be working. 
(10:37:38 AM) berwin22: Not heavy testing, but it would be handy to know which major areas i should focus on first when I start to fix things. I know extracting packages is probably broken.
(10:38:05 AM) epmclellan: ok, as soon as the dashboard is working I can do some testing
(10:38:20 AM) Sevein: I could try to make those updates
(10:38:27 AM) berwin22: peterVG/Sevein what's the schedule like on that
(10:38:30 AM) Sevein: along this week
(10:39:39 AM) Sevein: peterVG: ?
(10:39:39 AM) JessicaB left the room (quit: Read error: Connection reset by peer).
(10:39:41 AM) peterVG: when Sevein finishes up with Qubit rights he can move over the Dashboard dev
(10:39:50 AM) peterVG: which would be later this week?
(10:39:54 AM) Sevein: hopefully!
(10:39:57 AM) peterVG: okay
(10:40:07 AM) berwin22: sounds great to me
(10:40:14 AM) Sevein: ok, noted down
(10:40:16 AM) JessicaB [~Jessica@S01060018e789c0a0.vs.shawcable.net] entered the room.
(10:40:19 AM) ARTi: so far http://archivematica.org/wiki/index.php?title=Meeting_20110817
(10:40:49 AM) peterVG: berwin22: epmclellan and I will try to have all our new Dashboard requirements sorted out by then as well
(10:41:02 AM) epmclellan: yes, I think we're getting pretty close
(10:42:12 AM) epmclellan: finished dev?
(10:42:12 AM) JessicaB left the room (quit: Read error: Connection reset by peer).
(10:42:20 AM) berwin22: brainstorm of dashboard requirements: editing normalization rules, editing MCP workflow, access to transfer area, adding another layer of abstraction for traditional microservices to group jobs under, 
(10:42:32 AM) JessicaB [~Jessica@S01060018e789c0a0.vs.shawcable.net] entered the room.
(10:42:49 AM) Sevein: nice
(10:42:50 AM) berwin22: editing metadata in the dashboard
(10:43:11 AM) berwin22: access to information now stored in db, like premis events on a file.
(10:43:16 AM) Sevein: exciting features! :)
(10:44:00 AM) peterVG: display of ES indexing results 
(10:44:03 AM) djjuhasz: ooh, ooh, i have deployment news for Achivematica!
(10:44:15 AM) djjuhasz: is it deployment yet?
(10:44:15 AM) peterVG: display of ES/Protovis visualizations (will get MJ's help on these)
(10:44:23 AM) peterVG: djjuhasz: go for it
(10:44:29 AM) ARTi: is there dashboard requirements wiki page?
(10:44:42 AM) peterVG: sounds like there should be
(10:44:49 AM) berwin22: search for text in transfers... with elasticsearch?
(10:44:54 AM) djjuhasz: I got Misty's awesome ruby script for extracting video metadata running on Evelyn's workstation :)
(10:44:55 AM) peterVG: berwin22: yes
(10:45:07 AM) ARTi: djjuhasz: nice one!
(10:45:10 AM) djjuhasz: It's my first contribution to the Archivematica project
(10:45:27 AM) JessicaB: cross dresser!
(10:45:27 AM) epmclellan: yes, and that was just the beginning!!
(10:45:28 AM) peterVG: berwin22: privacy and security keywords, pattern matching (for SIN #'s, credit card #'s, etc)
(10:45:35 AM) ***djjuhasz feels so proud
(10:45:39 AM) epmclellan: Dashboard dev page: http://www.archivematica.org/wiki/index.php?title=Dashboard
(10:45:44 AM) JessicaB: pride is just the beginning david
(10:46:24 AM) epmclellan: David helped me because I've been trying to extract and measure significant properties of video files
(10:46:25 AM) peterVG: berwin22: see notes here http://archivematica.org/wiki/index.php?title=Transfer_and_SIP_creation#Create_SIPs
(10:46:27 AM) djjuhasz: Linkage for Misty's metadater script: https://github.com/mistydemeo/metadater
(10:46:31 AM) berwin22: what's the license on that script djjuhasz
(10:46:36 AM) peterVG: djjuhasz: right on thanks
(10:46:48 AM) djjuhasz: License  License is forthcoming. It will be permissive ;)
(10:46:56 AM) Misty: berwin22: Planning on BSD or smth BSD-like.
(10:47:00 AM) djjuhasz: ^ quote from Misty's readme
(10:47:01 AM) Sevein: Misty: o/
(10:47:05 AM) Misty: \o
(10:47:13 AM) Sevein: hehe
(10:47:32 AM) epmclellan: Here is sample test page with before and after significant characteristics: http://www.archivematica.org/wiki/index.php?title=MOV_to_MKV_using_FFmpeg
(10:47:52 AM) epmclellan: Updated for video conversion to ffv1/mkv
(10:48:24 AM) berwin22: Cool, thanks Misty
(10:48:46 AM) peterVG: thanks Misty
(10:48:51 AM) epmclellan: ditto that
(10:49:18 AM) Misty: :D No problem. Glad it's useful for other people.
(10:49:30 AM) epmclellan: it is, definitely
(10:50:27 AM) epmclellan: deployment?
(10:50:47 AM) epmclellan: I have deployment issue, actually
(10:51:00 AM) epmclellan: at RAC, efforts to ingest large PST files failed
(10:51:13 AM) epmclellan: we need some remote trouble-shooting
(10:51:27 AM) epmclellan: ARTi: will you have some time to spend on that?
(10:51:34 AM) epmclellan: maybe with help from berwin22?
(10:51:57 AM) ARTi: yes
(10:52:01 AM) ARTi: totally
(10:52:08 AM) epmclellan: the PST file got to normalization, then the process ground to a halt
(10:52:10 AM) berwin22: sure, if I can
(10:52:14 AM) epmclellan: thanks guys
(10:52:21 AM) ARTi: weird.. not bigger then the imf ones though
(10:52:36 AM) ARTi: notes so far : http://archivematica.org/wiki/index.php?title=Meeting_20110817
(10:52:38 AM) epmclellan: The IMF ones didn't get normalized
(10:52:43 AM) ***djjuhasz hates pst files
(10:52:52 AM) ARTi: epmclellan: it was the same issue we were having when we left, no?
(10:52:54 AM) epmclellan: because they were running through 0.7
(10:53:09 AM) epmclellan: yes, just wanted to remind you since I know it's been a while
(10:53:30 AM) epmclellan: djjuhasz: they're not as bad as CAD files, those are the worst
(10:53:38 AM) berwin22: if they look in the preservation planning tab, show advanced details, do they have any frozen/processing?
(10:54:05 AM) epmclellan: berwin22: I think we'll have to set up a trouble-shooting session, I'll email Marisa
(10:54:09 AM) djjuhasz: I've never had to try and transfer Sibylle's *CAD* files from one computer to another :(
(10:54:17 AM) djjuhasz: *bitter*
(10:55:10 AM) epmclellan: any more deployment news?
(10:55:20 AM) berwin22: pst is an acronym for probably somewhat corrupted
(10:55:35 AM) epmclellan: probably screwed completely
(10:55:49 AM) djjuhasz: lol
(10:56:06 AM) djjuhasz: "probably screwed completely" ftw!
(10:56:11 AM) epmclellan: heh
(10:56:23 AM) epmclellan: docs?
(10:57:21 AM) epmclellan: I've covered most of my docs news, I guess: updates of video preservation pages, plus Transfer/Create SIP page
(10:57:34 AM) epmclellan: that's it from me
(10:58:48 AM) epmclellan: meeting ended?
(10:58:48 AM) JessicaB left the room (quit: Read error: Connection reset by peer).
(10:58:51 AM) peterVG: thanks again for getting that reqs page with mockups up in a hurry Evelyn. 
(10:58:52 AM) berwin22: that's all folks
(10:59:03 AM) epmclellan: sure, np peterVG
(10:59:18 AM) JessicaB [~Jessica@S01060018e789c0a0.vs.shawcable.net] entered the room.