https://wiki.archivematica.org/index.php?action=history&feed=atom&title=Meeting_20110721Meeting 20110721 - Revision history2026-06-02T18:00:58ZRevision history for this page on the wikiMediaWiki 1.35.4https://wiki.archivematica.org/index.php?title=Meeting_20110721&diff=4569&oldid=prevJoseph at 20:16, 21 September 20112011-09-21T20:16:02Z<p></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 20:16, 21 September 2011</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l92" >Line 92:</td>
<td colspan="2" class="diff-lineno">Line 92:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>(10:59:40 AM) berwin22: k</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>(10:59:40 AM) berwin22: k</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div></pre></div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div></pre></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">[[Category:meetings]]</ins></div></td></tr>
<!-- diff cache key mwiki_am:diff::1.12:old-4241:rev-4569 -->
</table>Josephhttps://wiki.archivematica.org/index.php?title=Meeting_20110721&diff=4241&oldid=prevJoseph at 18:07, 21 July 20112011-07-21T18:07:19Z<p></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 18:07, 21 July 2011</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l1" >Line 1:</td>
<td colspan="2" class="diff-lineno">Line 1:</td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">= Development =</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">* Joseph/Austin</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">** Look at ceph; possible problems with watched directories (inotify)</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">* Joseph</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">** Work on MCP with gearman.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">= Deployment =</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">* Austin is at RAC NY with Archivematica 0.7.1</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">** Going well, 0.7.1 is fast.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">** got 5.25 floppy md5summing on win95 working</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">= Testing =</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">= Documentation =</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">= Chat log =</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div><pre></div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div><pre></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>(10:36:17 AM) berwin22: Dev:</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>(10:36:17 AM) berwin22: Dev:</div></td></tr>
<!-- diff cache key mwiki_am:diff::1.12:old-4240:rev-4241 -->
</table>Josephhttps://wiki.archivematica.org/index.php?title=Meeting_20110721&diff=4240&oldid=prevJoseph: Created page with '<pre> (10:36:17 AM) berwin22: Dev: (10:36:17 AM) berwin22: I've been working with gearman. It's looking good, I'll need to discuss some stuff about getting task information about...'2011-07-21T18:02:37Z<p>Created page with '<pre> (10:36:17 AM) berwin22: Dev: (10:36:17 AM) berwin22: I've been working with gearman. It's looking good, I'll need to discuss some stuff about getting task information about...'</p>
<p><b>New page</b></p><div><pre><br />
(10:36:17 AM) berwin22: Dev:<br />
(10:36:17 AM) berwin22: I've been working with gearman. It's looking good, I'll need to discuss some stuff about getting task information about what's processing with jesus.<br />
(10:36:17 AM) berwin22: Specifically, gearman has it's own api for getting information on queued/processing events. I think we may want to get information directly from that AIP, rather than pulling it through the MCP server, to lighten the load on the server.<br />
(10:36:19 AM) JessicaB: thanks david and jesus - see you in the office tomorrow.<br />
(10:36:40 AM) djjuhasz: See you tommorrow JB<br />
(10:36:44 AM) djjuhasz: :)<br />
(10:37:01 AM) Sevein: djjuhasz: qubit notes - http://www.qubit-toolkit.org/wiki/index.php?title=Meeting_20110721, please paste the chat log, it is difficult with a shell client<br />
(10:37:09 AM) berwin22: Had a more in-depth look at ceph yesterday. <br />
(10:37:09 AM) berwin22: I started doing so, because I wanted to know if we could easily identify which files were located on which machine in a distributed enviroment, and have the mcp assign jobs prioriting queue based on where files are located. (assign tasks to workers that already have the file, so they don't have to fetch it over the network). Gearman won't support this, but I might like to contact both gearman and ceph, to see if we might be able to collaborate on this feature.<br />
(10:37:26 AM) Sevein: who is taking notes?<br />
(10:38:40 AM) Sevein: ping broadcast<br />
(10:38:59 AM) berwin22: I'm not sure how much there is today, I was just going to dump the chat log in the wiki.<br />
(10:39:12 AM) djjuhasz: nice<br />
(10:39:14 AM) Sevein: ok, sounds good<br />
(10:40:11 AM) berwin22: I was also talking with Austin (ARTi) about the risk of ceph not working with inotify.<br />
(10:40:37 AM) djjuhasz: hmm, why ceph no like inotify?<br />
(10:40:58 AM) berwin22: inotify relies on hooks at the kernel level<br />
(10:40:59 AM) ARTi: same reason nfs dosnt<br />
(10:41:20 AM) ARTi: sshfs does though<br />
(10:41:32 AM) ARTi: so maybe the ceph-fuse mount would be a option<br />
(10:42:32 AM) berwin22: inotify may not be critical for the MCP watched directories, but would be for the arrangement section in thunar/archivematica<br />
(10:44:07 AM) berwin22: While looking at that stuff, I also tried to document what were some of the bottlenecks in archivematica.<br />
(10:44:08 AM) berwin22: http://archivematica.org/wiki/index.php?title=Bottlenecks<br />
(10:44:42 AM) Sevein: nice docs berwin22 :)<br />
(10:45:23 AM) berwin22: any other dev?<br />
(10:45:31 AM) djjuhasz: yes, great notes<br />
(10:46:41 AM) berwin22: deployment - ARTi<br />
(10:47:54 AM) djjuhasz: Sounds like ARTi is in the thick of deployment :)<br />
(10:47:59 AM) ARTi left the room (quit: Quit: Leaving.).<br />
(10:48:02 AM) berwin22: agreed<br />
(10:48:32 AM) berwin22: @RAC NY with Archivematica 0.7.1<br />
(10:48:49 AM) djjuhasz: sweet :)<br />
(10:49:25 AM) ARTi [~austin@static-71-167-174-3.nycmny.fios.verizon.net] entered the room.<br />
(10:50:17 AM) ***ARTi thinks VM / package building is destroying his HD<br />
(10:50:35 AM) berwin22: ssd?<br />
(10:50:50 AM) ARTi: its one of those combo drives<br />
(10:51:02 AM) berwin22: deployment - ARTi ?<br />
(10:51:09 AM) ARTi: umm. RAC is going rad<br />
(10:51:14 AM) ARTi: they are running 7.1<br />
(10:51:40 AM) ARTi: got 5.25 floppy md5summing on win95 working<br />
(10:51:53 AM) ARTi: dont let me near any sharp objects ;]<br />
(10:52:12 AM) ARTi: 7.1 has been super fast<br />
(10:52:24 AM) ARTi: have 5 workers and one masterServer<br />
(10:52:42 AM) djjuhasz: wow, nice<br />
(10:53:00 AM) berwin22: yeah... that should be a bit of a power house<br />
(10:53:06 AM) epmclellan [~epmclella@static-71-167-174-3.nycmny.fios.verizon.net] entered the room.<br />
(10:53:08 AM) ARTi: yeah, its rocking out<br />
(10:53:41 AM) berwin22: Fits and normalization are probably the two slowest microservices, and they are distributed well<br />
(10:53:41 AM) ARTi: but Ive got to get back to work..<br />
(10:53:44 AM) ARTi: yeah<br />
(10:53:56 AM) berwin22: cheers<br />
(10:53:58 AM) ARTi: fits crashed once.. on one of the workers.. but we couldnt get it to repeat<br />
(10:54:17 AM) ARTi: then the submission Docs normalization also crashed once<br />
(10:54:22 AM) ARTi: but couldnt duplicated<br />
(10:54:31 AM) ARTi: atleast nothing has brought the system down<br />
(10:54:38 AM) ARTi: way to go berwin22 :]<br />
(10:54:44 AM) berwin22: did submission docs crash on it trying to normalize a file that didn't exist?<br />
(10:54:54 AM) ARTi: no, it was a pdf..<br />
(10:55:01 AM) ARTi: actually I remember.. was a ghostscript error<br />
(10:55:08 AM) ARTi: thought it might be the MS issue..<br />
(10:55:14 AM) ARTi: but it worked the next few times around<br />
(10:55:31 AM) berwin22: mind grabbing the crash info you can for me?<br />
(10:55:31 AM) berwin22: logs and screen shots, idea<br />
(10:55:38 AM) ARTi: made a 100 copies of the same pdf and ran it through.. just to make sure all the workers were tested<br />
(10:56:04 AM) ARTi: berwin22.. Ill see if I can find.. sec<br />
(10:56:38 AM) berwin22: Fail or crash... did it hang?<br />
(10:57:04 AM) berwin22: I hoped we'd got rid of things hanging in 0.7.1 (for the most part)<br />
(10:57:11 AM) ARTi: nope didnt hang<br />
(10:57:12 AM) ARTi: fail<br />
(10:57:18 AM) berwin22: Thanks ARTi, I know you're busy busy busy<br />
(10:57:21 AM) ARTi: yeah.. ++ :]<br />
(10:57:36 AM) ARTi: http://i.imgur.com/aMqkx.png<br />
(10:57:39 AM) berwin22: Alright... any other business?<br />
(10:58:02 AM) ARTi: berwin22^ err<br />
(10:58:02 AM) epmclellan left the room (quit: Read error: Connection reset by peer).<br />
(10:59:00 AM) ARTi: not from me.. evelyn is pretty busy atm<br />
(10:59:40 AM) berwin22: k<br />
</pre></div>Joseph