Evelyn McLellan: /* Chat log */

2011-11-09T19:02:22Z

Chat log

Sevein at 19:00, 9 November 2011

2011-11-09T19:00:11Z

Sevein: Created page with '== Development == * Joseph has been testing and such with scalability issues Successes with a large number of files He will build today the path around the watched directo...'

2011-11-09T18:56:32Z

Created page with '== Development == * Joseph has been testing and such with scalability issues ** Successes with a large number of files ** He will build today the path around the watched directo...'

New page

== Development ==

* Joseph has been testing and such with scalability issues
** Successes with a large number of files
** He will build today the path around the watched directories for SIP creation
** He thinks he still need to look at the thread creation (SIP creation)... looks like some threads spawn and aren't removed. Deciding how to manage that, might get bumped to 0.9.
* Peter suggests to have a scalability refactor discussion very early on in the 0.9 development schedule.
* There are lot of remaining issues, a Monday code freeze is not realistic
* We can bump code freeze by a week, that then still leaves 2 weeks between code freeze and release.
* Evelyn to update the issue list and mark more criticals now that we're getting close
* Austin: ace is up and monitoring ~1000 documents
* '''Meeting scheduled on Mon, Nov 13 11am'''

== Documentation ==

Evelyn updated [[Significant characteristics of raster images]] wiki page

== Deployment ==

* Tomorrow, CVA and UBC
* Evelyn and Austin will do the RAC install the week of Dec 12
** That oculd get us an extra week in the 0.8 dev schedule

== Chat log ==

<pre>

</pre>
[[Category:meetings]]

← Older revision		Revision as of 19:02, 9 November 2011
Line 25:		Line 25:

	<pre>		<pre>
		+	(10:34:09 AM) peterVG: okay, let's roll
		+	(10:34:29 AM) berwin22: Been testing and such with scalability issues
		+	(10:34:44 AM) berwin22: Successes with a large number of files
		+	(10:34:51 AM) epmclellan: nice!
		+	(10:35:02 AM) epmclellan: that's been a big issue this past week
		+	(10:35:16 AM) epmclellan: Sevein has been doing all kinds of dashboard dev
		+	(10:35:20 AM) berwin22: today I'll build the path around the watched directories for SIP creation
		+	(10:35:41 AM) peterVG: okay
		+	(10:36:03 AM) peterVG: so the background story to ^ this is that we're running into what appears to be a Python threading limitation
		+	(10:36:16 AM) Sevein: yep, I did a big refactor in the dashboard pages using twitter bootstrap and lot of new code to manage transfer and sips individually
		+	(10:36:26 AM) berwin22: I think I still need to look at the thread creation there (sip creation)... looks like some threads spawn and aren't removed. Deciding how to manage that. Might get bumped to 0.9
		+	(10:36:30 AM) peterVG: which means we have a small, hard cap on total number of files that can be processed at once
		+	(10:37:14 AM) peterVG: ...and we'll have to skip/have ability to turn off Transfer SIP creation for 0.8 as it (appears to) spawn a thread for each file in the Transfer area
		+	(10:37:14 AM) djjuhasz: :(
		+	(10:37:45 AM) peterVG: so we'll have to have a scalibility refactor discussion very early on in the 0.9 development schedule.
		+	(10:37:47 AM) berwin22: I did some more testing on that. I think it's for each SIP/Transfer
		+	(10:37:57 AM) djjuhasz: sounds like some reasonable compromise
		+	(10:38:03 AM) berwin22: I may be able to get around it, spent some time looking into it yesterday.
		+	(10:38:18 AM) peterVG: berwin22: fingers crossed
		+	(10:38:31 AM) berwin22: I'm finding it challenging to debug python threads.
		+	(10:39:20 AM) berwin22: Looking at the calendar... Monday is code freeze, and tomorrow we're at CVA
		+	(10:39:40 AM) epmclellan: I updated the scalability testing page: http://www.archivematica.org/wiki/index.php?title=Scalability_testing
		+	(10:39:42 AM) berwin22: I think there are other issues we wanted me to work on before release
		+	(10:39:59 AM) epmclellan: there are a lot of remaining issues, a Monday code freeze is not realistic
		+	(10:40:04 AM) peterVG: agreed
		+	(10:40:07 AM) epmclellan: quite a few METS issues
		+	(10:40:08 AM) ARTi2: cva&ubc?
		+	(10:40:21 AM) epmclellan: ARTi2: I'll be at CVA
		+	(10:40:27 AM) epmclellan: youdon't really need me at UBC
		+	(10:40:36 AM) ARTi2: ok
		+	(10:40:47 AM) epmclellan: Lisa will be there, she has been experiencing the problems
		+	(10:40:53 AM) epmclellan: and can explain/test them
		+	(10:41:07 AM) epmclellan: anyway, yes, we have quite a few outstanding issues
		+	(10:41:09 AM) ARTi2: great
		+	(10:41:15 AM) epmclellan: and some more will crop as dashboard changes are made
		+	(10:41:37 AM) epmclellan: because sometimes issues don't become apparent until you use the new dashboard features
		+	(10:41:44 AM) epmclellan: but I'll try to be realistic :)
		+	(10:41:51 AM) peterVG: epmclellan & ARTi2 would you guys be able to do the RAC install the week of Dec 12?
		+	(10:41:56 AM) epmclellan: yes
		+	(10:42:05 AM) peterVG: that could get us an extra week in the 0.8 dev schedule
		+	(10:42:06 AM) ARTi2: yes
		+	(10:42:12 AM) peterVG: okay, good.
		+	(10:42:14 AM) epmclellan: assuming we can get flights
		+	(10:42:19 AM) epmclellan: it's the busy season
		+	(10:42:25 AM) epmclellan: we'd better book asap
		+	(10:42:33 AM) peterVG: hmm, yes. let's get on that today then. first i'll confirm dates with RAC
		+	(10:42:37 AM) epmclellan: don't wanna go to New York via the Yukon
		+	(10:42:41 AM) peterVG: for sure
		+	(10:42:52 AM) ARTi2: ;-)
		+	(10:42:54 AM) peterVG: but let's assume that's the plan. we can bump code freeze by a week
		+	(10:43:00 AM) epmclellan: peterVG: and with the UN?
		+	(10:43:20 AM) epmclellan: dates, I mean
		+	(10:43:21 AM) peterVG: that then still leaves 2 weeks between code freeze and release
		+	(10:43:36 AM) peterVG: epmclellan: yes, I'll confirm those as well. tag a day onto RAC
		+	(10:43:41 AM) epmclellan: ok, thx
		+	(10:44:49 AM) epmclellan: ARTi2: re scalability testing
		+	(10:44:58 AM) epmclellan: can we have zip of Corpora files on wiki?
		+	(10:45:04 AM) epmclellan: For testing purposes
		+	(10:45:24 AM) peterVG: so do we have the list of criticals for code freeze then? http://code.google.com/p/archivematica/issues/list?can=2&q=milestone=Release-0.8&sort=priority&colspec=ID%20Milestone%20Component%20Type%20Status%20Priority%20Owner%20Summary
		+	(10:45:40 AM) epmclellan: I think I need to mark more criticals
		+	(10:45:44 AM) epmclellan: now that we're getting close
		+	(10:45:54 AM) epmclellan: I've marked a lot of issues as high that are probably critical
		+	(10:45:56 AM) peterVG: is there an issue(s) for the transfer scalability & SIP scalablity problems?
		+	(10:46:03 AM) epmclellan: hmm
		+	(10:46:12 AM) epmclellan: I think so
		+	(10:46:25 AM) ARTi2: yes @ digicopora files
		+	(10:46:30 AM) epmclellan: thanks
		+	(10:46:59 AM) epmclellan: berwin22: I believe you filed the scalability issue? Just looking for it
		+	(10:47:00 AM) peterVG: ok, well we now have a hard deadline for the 0.8 code freeze so that will be the determinant on how many 'highs' we can accomodate as 'criticals'
		+	(10:47:21 AM) peterVG: should we book a mtg next week to review this list?
		+	(10:47:25 AM) ARTi2: ace is up and monitoring ~1000 documents
		+	(10:47:31 AM) berwin22: I filed one on our internal issues, not sure about google
		+	(10:47:33 AM) peterVG: ARTi2: v. cool!
		+	(10:47:41 AM) epmclellan: yes, it's very cool
		+	(10:47:54 AM) epmclellan: berwin22: can you put it on google too?
		+	(10:48:18 AM) epmclellan: peterVG: I'll assign what I think are the criticals, we can review next week
		+	(10:48:30 AM) peterVG: okay, let's schedule a mtg now
		+	(10:48:52 AM) peterVG: joseph isn't here on mon
		+	(10:48:57 AM) peterVG: i'm at ubc on tue
		+	(10:49:07 AM) peterVG: wed the 16th?
		+	(10:49:08 AM) epmclellan: berwin22: you're planning to come in on Monday, yes?
		+	(10:49:19 AM) peterVG: oh, right, instead of the Fri?
		+	(10:49:23 AM) peterVG: that would work well
		+	(10:49:28 AM) epmclellan: Monday would work for me
		+	(10:49:58 AM) epmclellan: berwin22: ?
		+	(10:50:14 AM) berwin22: yes, monday in
		+	(10:50:17 AM) berwin22: is the plan
		+	(10:50:18 AM) epmclellan: great
		+	(10:51:03 AM) peterVG: cool
		+	(10:51:04 AM) Sevein: Evelyn updated "Significant characteristics of raster images" wiki page
		+	(10:51:06 AM) peterVG: 11am?
		+	(10:51:10 AM) Sevein: on docs
		+	(10:51:11 AM) berwin22: Issue 388:Stress tests with multiple clients < what's the status of this issue
		+	(10:51:27 AM) epmclellan: thanks Sevein
		+	(10:51:40 AM) epmclellan: 11:00 is fine
		+	(10:52:11 AM) peterVG: ARTi2: may be our last chance to be all together prior to 0.8 release so would be good if you could make it so we cover all logistics for upcoming weeks
		+	(10:52:12 AM) berwin22: http://code.google.com/p/archivematica/issues/detail?id=388
		+	(10:52:23 AM) epmclellan: thanks berwin22
		+	(10:52:48 AM) ARTi2: sounds good.
		+	(10:53:02 AM) ARTi2: be there Monday.
		+	(10:53:20 AM) peterVG: okay, great. i've just put in calendar for 11am.
		+	(10:53:41 AM) berwin22: btw... I have plans the third, probably won't make it
		+	(10:54:08 AM) peterVG: oh no!
		+	(10:55:28 AM) ARTi2: were talking the next Monday right?
		+	(10:55:46 AM) epmclellan: yeah, the 14th
		+	(10:56:07 AM) ARTi2: thanks
		+	(10:56:47 AM) JessicaB1 [~Jessica@70.71.0.109] entered the room.
		+	(10:56:47 AM) JessicaB left the room (quit: Read error: Connection reset by peer).
		+	(10:57:30 AM) Sevein: http://archivematica.org/wiki/index.php?title=Meeting_20111109
		+	(10:57:35 AM) Sevein: could someone paste the chat log, please?
		+	(10:57:41 AM) epmclellan: I can do that
		+	(10:57:46 AM) Sevein: thank you!
		+	</pre>
		+

−	~~</pre>~~
	[[Category:meetings]]		[[Category:meetings]]

@@ Line 10: / Line 10: @@
 * Evelyn to update the issue list and mark more criticals now that we're getting close
 * Austin: ace is up and monitoring ~1000 documents
-* '''Meeting scheduled on Mon, Nov 13 11am'''
+* '''Meeting scheduled on Mon, Nov 14 11am'''
 == Documentation ==
@@ Line 20: / Line 20: @@
 * Tomorrow, CVA and UBC
 * Evelyn and Austin will do the RAC install the week of Dec 12
-** That oculd get us an extra week in the 0.8 dev schedule
+** That could get us an extra week in the 0.8 dev schedule
 == Chat log ==

Meeting 20111109 - Revision history

Evelyn McLellan: /* Chat log */

Sevein at 19:00, 9 November 2011

Sevein: Created page with '== Development == * Joseph has been testing and such with scalability issues ** Successes with a large number of files ** He will build today the path around the watched directo...'

Sevein: Created page with '== Development == * Joseph has been testing and such with scalability issues Successes with a large number of files He will build today the path around the watched directo...'