Development
- Joseph has been working on moving the transcoder into the MCP, he may be passing it to QA at the end of the week: it might need some tweeks in terms of how the jobs are grouped.
- Mike C worked on the DIP upload destination selection feature, attempting to implement this through microservice job flow. Joseph has been a big help orienting him to how the microservice job flow is defined in the database.
- Peter, Evelyn, Courtney and Mike will be meeting to discuss AIP indexing
Deployment
- Courtney has been helping UBC Rare Books and Special Collections to accession a large body of digital photographs: she has made hardware recommendations and provided questions to ask the donor
- Evelyn has been helping RAC work with its vendor to structure digitization output for ingest
Testing
- Austin: most recent test data was here.. started noticing how much extra time fits is actually taking.. http://66.151.32.163/testdata/test03/. FITS is sucking up huge amounts of time and resources.
- Courtney is going to look into streamlining FITS output and processes post release 0.9.
Documentation
Chat log
(10:30:41 AM) epmclellan: Archivematica meeting?
(10:30:45 AM) berwin22: sure
(10:30:47 AM) epmclellan: I can take notes
(10:30:50 AM) berwin22: thanks
(10:30:58 AM) mcantelon: Thanks!
(10:31:00 AM) epmclellan: np
(10:31:02 AM) epmclellan: dev news?
(10:31:32 AM) berwin22: I've been working on moving the transcoder into the MCP.
(10:31:50 AM) mcantelon: Mike C worked on the DIP upload destination selection feature, attempting to implement this through microservice job flow. Joseph has been a big help orienting me to how the microservice job flow is defined in the database.
(10:32:22 AM) peterVG: cool
(10:32:24 AM) courtney: i mocked up that dip upload stuff last week
(10:32:39 AM) courtney: excited to see that mcantelon is already on it!
(10:32:47 AM) mcantelon: Woot!
(10:32:53 AM) peterVG: go DJ Mike
(10:33:01 AM) courtney: evelyn - can i start on issue 971?
(10:33:11 AM) epmclellan: courtney: link to dip upload page?
(10:33:37 AM) epmclellan: have you tested bagit ingest?
(10:34:07 AM) epmclellan: I would hold off on PREMIS rights stuff, the revised standard is still draft
(10:34:20 AM) courtney: not yet, qa this afternoon and tmrw/most of next week
(10:34:25 AM) courtney: lots of qa issues assigned to me
(10:34:40 AM) epmclellan: Good idea to work on qa issues first, I think
(10:35:13 AM) courtney: http://www.archivematica.org/wiki/index.php?title=Upload_DIP scroll down for mockup of export dip dialogue
(10:35:22 AM) epmclellan: thx
(10:35:35 AM) epmclellan: nice
(10:35:48 AM) epmclellan: we have the world's best mockups, I swear
(10:36:44 AM) mcantelon: Yeah, nice.
(10:36:50 AM) courtney: thank you. i'm new.
(10:36:56 AM) epmclellan: berwin22: how far into moving the transcoder into MCP are you?
(10:37:00 AM) epmclellan: just for sake of the notes
(10:37:31 AM) courtney: See also configurable DIP export locations in the mockup for the Administration tab. http://archivematica.org/wiki/index.php?title=File_Browser_Requirements#ADMINISTRATION_TAB
(10:37:44 AM) berwin22: I'm looking at passing it to qa at the end of the week
(10:37:51 AM) epmclellan: great!
(10:38:02 AM) courtney: sweet! more qa fun
(10:38:12 AM) epmclellan: courtney: this is just the beginning
(10:38:13 AM) berwin22: it might need some tweeks in terms of how the jobs are grouped.
(10:39:33 AM) courtney: one more dev - peter, mike and i are chatting about aip indexing this pm
(10:40:08 AM) epmclellan: I think I'm at that meeting to, right?
(10:40:11 AM) epmclellan: too
(10:40:40 AM) courtney: yeah. i did keep the invite lists small so as not to take up too many ppl's time, but any interested parties are welcome
(10:40:49 AM) epmclellan: oh, I thought you sent me an invite
(10:40:53 AM) courtney: i did
(10:40:59 AM) epmclellan: ok, thx
(10:41:50 AM) epmclellan: any more dev news?
(10:42:13 AM) berwin22: a discussion list request was made for subtitles on: http://youtu.be/haotj_NlbX0
(10:42:29 AM) epmclellan: I noticed
(10:42:36 AM) courtney: austin sent me a tool to use for the 0.9 screencast
(10:42:43 AM) berwin22: not sure what to reply to that
(10:42:44 AM) epmclellan: how much time/effort?
(10:42:52 AM) courtney: since that is soon, i don't know if it is necessary to go backwards at this point
(10:42:56 AM) courtney: but good for future
(10:43:05 AM) berwin22: never done subtitles before
(10:43:12 AM) courtney: berwin22: can you tell him that 0.9 screencast will include subtitles?
(10:43:22 AM) epmclellan: let's say we'll look into it
(10:43:25 AM) ARTi: with subtitle editor..
(10:43:25 AM) epmclellan: but not commit to it
(10:43:29 AM) ARTi: you basicly play the video in a window..
(10:43:29 AM) berwin22: it's not the screencast courtney
(10:43:33 AM) ARTi: and attach words to timestamps
(10:43:37 AM) courtney: oh, oops
(10:43:44 AM) epmclellan: it's Joseph's video
(10:43:46 AM) courtney: still. i think i should do it for 0.9
(10:43:54 AM) ARTi: http://home.gna.org/subtitleeditor/
(10:44:18 AM) berwin22: ARTi: I think youtube has it's own mechanism
(10:44:36 AM) ARTi: ahh yeah, I think you can cc stuff huh
(10:44:43 AM) mcantelon: Yeah, it does.
(10:44:48 AM) ARTi: never used youtube other then to watch..
(10:45:05 AM) peterVG: don't think it worth too much of our time at this stage
(10:45:12 AM) peterVG: given all our other responsbilities
(10:45:16 AM) berwin22: peterVG agreed
(10:45:46 AM) peterVG: maybe suggest that volunteers add subtittles if they wabt
(10:45:49 AM) peterVG: *want*
(10:45:59 AM) ARTi: volunteers++
(10:46:06 AM) berwin22: cowd source
(10:46:34 AM) courtney: Canada: where our cows can type
(10:46:36 AM) epmclellan: ok, "wabt"... ""cowd source": you two are not doing the subtitles
(10:46:39 AM) mcantelon: Haha
(10:47:21 AM) berwin22: courtney... odly enough... that explains a few things
(10:47:42 AM) epmclellan: any deployment news?
(10:48:08 AM) courtney: i helped UBC special collections with digital photo acquisition strategy
(10:48:08 AM) epmclellan: or testing? ARTi, I think you have testing news
(10:48:19 AM) epmclellan: right. how did that go?
(10:48:29 AM) ARTi: yesp
(10:48:32 AM) courtney: great. i recommended some hardware for acquisitions
(10:48:39 AM) courtney: and i gave them some questions to ask the donor
(10:48:44 AM) epmclellan: excellent
(10:48:51 AM) peterVG: cool. do you have notes/docs?
(10:48:51 AM) courtney: bases on AIMS and Paradigm questionairre
(10:48:54 AM) courtney: I do
(10:49:01 AM) ARTi: most recent test data was here.. started noticing how much extra time fits is actually taking.. http://66.151.32.163/testdata/test03/
(10:49:12 AM) courtney: grrrr. fits.
(10:49:12 AM) epmclellan: courtney: if you send them to me I can stick them in the project report
(10:49:15 AM) ARTi: so since then Ive played with both fits configs and working on building a new copy
(10:49:19 AM) courtney: epmclellan: will do
(10:49:32 AM) epmclellan: yeah, FITS... our big fat processing problem
(10:49:36 AM) mcantelon: FITS just uses file extensions for figuring out content type?
(10:49:40 AM) peterVG: great, can you send to archivematica@artefa.. Hope to have GoogleDocs for Business up and runnning asap to simplify doc sharing
(10:50:32 AM) peterVG: mcantelon: no it uses a series of tools to figure out content type, then we just use file extension because FITS gives conflicting results
(10:50:48 AM) mcantelon: peterVG: Ah, makes sense.
(10:50:48 AM) peterVG: a pretty big problem still that we need to work on
(10:50:52 AM) ARTi: peterVG: what do you want to send to google docs?
(10:50:56 AM) epmclellan: we use file extension for normalization
(10:51:14 AM) peterVG: ARTi: huh?
(10:51:32 AM) epmclellan: mcantelon: the normalization path for each object is based on its file extension
(10:51:48 AM) ARTi: peterVG: great, can you send to archivematica@artefa.. Hope to have
(10:51:48 AM) mcantelon: epmclellan: Ah, cool.
(10:52:03 AM) ARTi: was that @ courtney
(10:52:04 AM) ARTi: ?
(10:52:07 AM) epmclellan: well, not so much, really, but currently not much choice
(10:52:08 AM) peterVG: ARTi: sorry, that was @courtney
(10:52:37 AM) ARTi: kk
(10:52:51 AM) courtney: i intend to dedicate a lot of time to fits post 0.9 release
(10:52:55 AM) peterVG: ARTi: you saw this morn's email about moving ahead with jaz drive/disk purchase for RAC?
(10:52:59 AM) courtney: no way to have a solution prior to that
(10:53:02 AM) courtney: a lot of work
(10:53:04 AM) ARTi: Im going to see how this FITS package changes things, then move ahead with scalability testing
(10:53:21 AM) peterVG: courtney: understood, thanks for putting/keeping on radar
(10:53:21 AM) epmclellan: courtney: looking forward to that
(10:53:26 AM) epmclellan: and glad you're here!
(10:53:42 AM) mcantelon: courtney: Yeah, definitely a good dragon to slay.
(10:53:43 AM) ARTi: peterVG: yes, got it. So do we know what versions of jazz drives we need?
(10:54:03 AM) peterVG: no, can you pls follow-up with sibyl (cc me and evelyn)
(10:54:37 AM) ARTi: ok
(10:55:17 AM) epmclellan: re deployment, I've been helping RAC work with its vendor to structure digitization output for ingest
(10:55:39 AM) epmclellan: I forget about that stuff because it's for clients but I should include them in these meetings I guess
(10:55:50 AM) epmclellan: the vendor is packaging the output into bags
(10:55:58 AM) epmclellan: which is why we need bagit ingest in 0.9
(10:56:34 AM) ARTi: dunno if rfp counts as deployment.. but wow! nice work team. was neat to see collaborative edit in action
(10:56:54 AM) epmclellan: yeah, googledocs plus chatroom was working pretty well there!
(10:56:58 AM) peterVG: thanks everyone for pitching in
(10:57:12 AM) epmclellan: ARTi and Sevein were filling in the doc then pinging me when a requirement was answered
(10:57:19 AM) epmclellan: it was kinda crazy
(10:57:24 AM) peterVG: turned into quite a heroic effort/sprint in the end
(10:57:35 AM) peterVG: epmclellan took the brunt of it, thx!
(10:57:36 AM) epmclellan: the RFP was over 100 pages long
(10:57:41 AM) courtney: good job folks
(10:57:42 AM) epmclellan: you're welcome
(10:57:53 AM) ARTi: epmclellan: ++
(10:58:00 AM) epmclellan: thanks guys
(10:58:05 AM) mcantelon: epmcellan: Whoa! :o
(10:58:13 AM) epmclellan: I know, I know!
(10:58:21 AM) epmclellan: who's gonna read all that stuff?
(10:58:29 AM) epmclellan: maybe they'll just look at the pictures!
(10:58:37 AM) ARTi: srsly.
(10:59:00 AM) courtney: quick question - even though 0.9 won't have a full answer to the fits situation, are we at least going to stop basing the normalization on extension?
(10:59:12 AM) epmclellan: not for 0.9
(10:59:13 AM) berwin22: no
(10:59:17 AM) courtney: ok
(10:59:22 AM) epmclellan: too complicated, no sponsor
(10:59:55 AM) berwin22: that's probably like a 4 week project
(11:00:02 AM) epmclellan: groan
(11:00:10 AM) peterVG: i think we're going to have to bite the bullet on this one ourselves for 1.0. btw, my assumption is that the primary FITS situation is figuring out how to use it to stop relying on file extension for normalization
(11:00:27 AM) epmclellan: yes, they're definitely related
(11:00:30 AM) peterVG: other than FITS scalability what other FITS issue are there?
(11:00:46 AM) epmclellan: that's it, I think. FITS is a hog
(11:00:47 AM) courtney: turning "off" some of the tools for particular file formats
(11:00:53 AM) epmclellan: yes
(11:01:03 AM) courtney: configuring it might make it less of a hog
(11:01:10 AM) epmclellan: exactly
(11:01:18 AM) epmclellan: maybe turn it off for digitization output
(11:01:20 AM) courtney: no way to know for sure unless we configure it
(11:01:29 AM) epmclellan: if the output formats are standardized eg on TIFF 6.0
(11:01:49 AM) epmclellan: or truncate the output
(11:01:54 AM) epmclellan: etc.
(11:01:57 AM) courtney: yeah, that too
(11:03:02 AM) epmclellan: finis?
(11:03:18 AM) courtney: think so
(11:03:27 AM) ARTi: umm..
(11:03:35 AM) ARTi: I added a few docs
(11:03:41 AM) ARTi: cleaned up the docs for gathering testing data http://www.archivematica.org/wiki/index.php?title=Scalability_testing#Testing_metrics
(11:03:49 AM) epmclellan: notes at http://www.archivematica.org/wiki/index.php?title=Meeting_20120509
(11:03:52 AM) ARTi: added some info from sevein's communication with ubc on upload dip http://www.archivematica.org/wiki/index.php?title=Upload_DIP
(11:03:57 AM) epmclellan: oops, sorry
(11:04:13 AM) epmclellan: thanks ARTi
(11:04:30 AM) epmclellan: updating notes
(11:04:40 AM) ARTi: ^^ cheers