14:05:10 #startmeeting 2012-03-08 - Developer Meeting 14:05:10 Meeting started Thu Mar 8 14:05:10 2012 US/Eastern. The chair is bshum. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:05:10 Useful Commands: #action #agreed #help #info #idea #link #topic. 14:05:25 #topic Who's here? 14:05:33 #info Ben Shum, Bibliomation 14:05:41 #info Thomas Berezansky, MVLC 14:05:42 #info Bill Erickson, Equinox 14:05:45 #info James Fournie, BC Libraries Cooperative / Sitka 14:06:01 #info Jason Stephenson, MVLC 14:06:10 #info Lebbeous Fogle-Weekley, Equinox 14:06:13 #info Kathy Lussier, MassLNC 14:06:46 #info Anoop Atre, PALS 14:06:47 #info Mike Rylander, Equinox 14:06:52 berick++ 14:07:07 #info Dan Scott, Laurentian University 14:08:27 Okay, so meeting over? 14:08:37 Heh, no 14:08:39 #info Chris Sharp, GPLS 14:08:39 Okay, then 14:08:42 Moving on 14:08:50 #topic Review past action items 14:09:04 #info All of them appear to be done or mostly done. 14:09:26 #action moodaepo (and others) to reply to rsoullierre's questions about DIG contributions to the release team. 14:09:34 Agenda > http://evergreen-ils.org/dokuwiki/doku.php?id=dev:meetings:2012-03-08 14:09:54 kmlussier++ for helping to setup doodle polls and agenda for this meeting 14:10:11 * tsbere notes the agenda says today is Monday 14:10:15 #info Action item 2 leads to next topic. 14:10:35 #topic Evergreen 2.2 alpha2/alpha3/beta 14:10:47 #info Michael Peters, Evergreen Indiana 14:10:52 #info tsbere was to cut 2.2 alpha2 last week 14:11:17 I cut it. Then it was tweaked to make the upgrade script work. Then....it stalled out and never got any further. 14:11:20 he did, but it didn't get to the web site, so nobody but us chickens knew 14:12:05 tsbere: oops. That's what I get for copying and pasting. Will fix it. 14:12:29 So I guess other than me, did anyone else from IRC get to try out 2.1-2.2? Or I guess we need a round of announcements to get the word out. 14:12:46 I believe shopkins was having issues with it 14:13:25 Hmm, sounds like we could consider alpha3 round, this time with a little more publicity. Email to the lists, etc. 14:13:35 +1 14:13:48 +1 14:14:04 List, blog post, update website for downloads page 14:14:11 like a release process! 14:14:13 I'll craft an action item on that. 14:14:32 I'll cut alpha3 whenever we decide to, provided I know what our procedure should be with the upgrade script. Not going anywhere near cutting anything until we figure that out ;) 14:14:40 denials++ 14:14:54 http://evergreen-ils.org/dokuwiki/doku.php?id=dev:evergreen:release_checklist is looking pretty moribund when it comes to ownership :) 14:14:54 tsbere++ 14:15:10 tsbere++ 14:15:27 #action tsbere to cut alpha3 once upgrade script procedure decided (see next topic) 14:15:38 #action bshum to update website download pages for alpha3 14:15:57 Volunteer for blog post or lists? (I guess tsbere can do the list when he's finished) 14:15:57 tsbere: if "someone builds one by hand" isn't figured enough ;) then the slate is clean and open for any ideas 14:16:27 bshum: Might want to update the release checklist page with your name also 14:16:35 moodaepo: Good point. 14:16:46 I'd volunteered so now we have backups : ) 14:16:51 eeevil: There we run into "who" and other issues, like how to deal with "we built one, now we are expanding upon it" type issues ;) 14:16:52 #action mrpeters-isl will install Alpha 3 to testing.evergreen.lib.in.us for public testing 14:16:54 #help Revise release checklist with your roles. 14:16:55 tsbere: I think an automated approach with a bit of hand tailoring is fine for the first alpha, but after that it should be updated by hand 14:17:30 e.g. we fixed the same issue with is_json() twice 14:17:35 #action moodaepo to do the blog post 14:18:48 Okay. 14:18:57 So I guess we're already discussing the next topic 14:18:58 * denials also thinks the point-to-point version upgrade script should live in master once it is created and edited there from that point on 14:19:02 I'm personally fine with finding the last upgrade contained in a release and using cat to push everything after that into on file ... I'd be even more fine with an expansion of update_db.sh that just applied whatever was missing 14:19:04 * denials needs to run 14:19:14 #topic Upgrade scripts and what we are *supposed* to be doing in making/updating/whatever them. 14:19:35 both of those could be automated at tar-building time 14:20:15 I am under the impression that we want a single transaction for the whole mess, though, and we have found that fails wonderfully when a table has new rows added and is the subject of an ALTER TABLE later. 14:20:16 and not stored in git (just like we don't store i18n-ized files past the .po's) 14:20:49 what about savepoints? 14:21:07 * tsbere isn't sure how savepoints would help there 14:21:18 well you could have one big transaction with savepoints to rollback to the last upgrade 14:22:38 so that way if one fails it's a bit easier to troubleshoot 14:22:43 depending on what happens 14:23:53 would it be possible to use the upgrade filenames to determine if they need to be committed? ie if it has schema in it? 14:24:04 well, stop_on_error will let us do that now, with small transactions, assuming we aren't lazily shoving maybe-dups in outside a transaction so it's ok to fail (which, of course, we've done plenty of in the past 14:24:45 jamesrf: that's actually the idea of the numbers on the names being recorded in the upgrade log 14:25:02 unless I'm misunderstanding you 14:25:10 which I think I am. NM 14:25:33 like: 0628.schema.acq_fund_view_repairs.sql 14:25:39 vs 0629.data.jedi-template.sql 14:25:41 I think part of the problem is we don't get a complete list of all upgrade numbers we need to worry about in release builds. We just get the highest number that was in as of the release on a clean DB load. 14:26:04 jamesrf: The data chunks are just as, if not more, important thatn the schema chunks sometimes. 14:26:30 no but my original question is could that be used to determine if we need to commit it or not 14:26:37 tsbere: that's where son-of-update_db.sh comes in, and does things based on current state 14:29:06 * denials somewhat wonders if eeevil is fine with the "concat the upgrades together" approach because he has a support business to keep busy - it could work, in a perfect world, but thus far we're not perfect ... 14:29:13 tsbere: something that does this: 1) look at the oldest in the db 2) find all upgrade scripts after that making a to-apply list 3) drop the ones that are in the db already from the to-apply list 4) (optional) use deprecation/supercession info within the scripts to further drop some from to-apply 5) run to-apply's in number order 14:29:38 * denials has his tongue very much in cheek 14:30:26 eeevil: Still fails on clean DBs, I think. Install rel_2_1 as it currently stands, then upgrade via that method. I bet it misses a pile of changes. 14:30:31 denials: that's your foot, not your tongue :) 14:31:13 tsbere: why? part of the baseline schema is shoving the 4-digit number of the reified schema into the upgrade log 14:31:40 the "concat the upgrades together" worked very well for 2.0.5-> 2.0.11 for us, but i can see there could be problems going between major releases 14:31:59 eeevil: Which works wonderfully until 9 DB changes in master go by before someone backports the 10th, bumping the baseline number up past those other 9. 14:32:30 * csharp can attest that going between major releases (or 2) is a big ol' deal for huge consortia 14:33:15 tsbere: not following, sorry. that's what deprication/supercession is for (though we're not using it now), and even without that, a forward-ported placehold should take care of things 14:33:23 tsbere: I had do do that recently 14:33:36 (a forward-ported placeholder) 14:34:29 no more releases, then. everyone must run master from now on. :) 14:34:41 eeevil: I think you are looking at things differently than I am. Example: Master gets update number 1000-1009, then someone gives master 1010 *and* backports that to rel_2_1. rel_2_1 now has a baseline of 1010, but we are missing 1000-1009 on an upgrade script based on that. 14:34:42 these are not insurmountable problems, and they're even detectable at upgrade time by a script that can refuse to /start/ if there's something wonky going on 14:35:15 So... can we talk about what we want to do in the next week or two, given existing options, vs. what we would like to do some point in the future? 14:35:38 tsbere: you missed (1): look at 14:35:46 arg ... look at the oldest in the db 14:36:20 eeevil: On a clean load of rel_2_1 the 1010 *will* be the oldest in that case 14:36:23 tsbere: wait. lightbulb. got it 14:36:31 i'm curious have we identified the main problems with the upgrade scripts? sounds like backports are a major problem? 14:36:43 * denials has sent email about this very problem in the past 14:37:21 * bshum agrees with denials, we do have a release to worry about and if it's a bigger solution we need, we can discuss further on lists / post 2.2 solutions? 14:37:36 my "build the upgrade script automatically but without spotting many common issues" code right now does a "look at the old branch, then the new, and grab any numbered script in the new but not in the old" via git trickery. 14:38:46 let's just bump versions across the board when backporting? 14:39:20 tsbere: of course, that just means my other suggestion -- stop reifying the baseline and ONLY use upgrades within a version number. we call 2.2 3.0, ever touch the baseline schema again until 4.0, and simply upgrade through the 3.x series with small scripts 14:40:06 and we NEVER backport across version boundaries (or, rather, the upgrade scripts RESTART at version boundaries, so their numbers are distinct and unrelated) 14:40:06 I am going to vote for "figure out what to do for 2.2, and re-do the entirety of how we handle upgrade scripts for 3.0 later" for now. 14:41:08 We have an in-progress upgrade script for 2.2, I can keep building on top of that for 2.2 by hand. However, I still want to know if I should be keeping the copy in master up to date, and if so can I do so without getting someone else's sign-off? 14:41:24 #info Decide approach for 2.2 now, then discuss/come up with new upgrade script handling in 3.0+ 14:41:24 tsbere: what your script does now is an approximation of what I did by hand before, so as mentioned above, I'm fine with that 14:41:46 * denials votes "yes" for upgrade script updating without sign-off 14:41:55 tsbere: +1, do it 14:41:57 +1 14:41:58 to it 14:42:07 (for the build-cutter, for getting 2.2 out the door) 14:42:13 * Dyrcona abstains from voting 14:43:11 +1 imo, we can always tweak issues as they arise. 14:43:26 Which is what we did during 2.0 alphas if I recall 14:43:44 * denials thinks others could & should still push branches that contain updates to the upgrade script during 2.2 for any related issues, if that helps the build cutter 14:43:57 +1 to that 14:44:10 The build cutter in this case doesn't have old pre-2.1 databases kicking around to test on ;) 14:44:20 heh 14:44:34 sitka will be testing on pre-2.1 databases at some point 14:44:47 Ok. Poking backwards a little bit, when do we want alpha3 cut? Tomorrow? Sunday? Monday? 14:45:06 (yea, I volunteered sunday. I am doing an update cycle for MVLC *anyway* then....) 14:45:08 Well, last week, we talked about a 2 week testing period for this next cut (be it alpha3/beta1, but since it's alpha3) 14:45:35 One other question on upgrade script policy: updating default config settings to fix bugs - in this case, removing a normalizer for ISSNs - should we just change the default schema and warn about it in release notes? Or force an upgrade & reindex? 14:46:17 (sorry, just queuing up upgrade-script-related issues - here's the bug in question: https://bugs.launchpad.net/evergreen/+bug/932540 ) 14:46:17 Launchpad bug 932540 in Evergreen "Indexing multiple ISSNs in a single bib results in identifier|issn lookup failure" (affected: 1, heat: 6) [Undecided,New] 14:46:32 denials: Personally I like seeing stuff like that in release notes for reading's sake, but then I'd also want the upgrade SQL to handle it for me if it can. 14:46:39 So maybe both? 14:46:50 I vote for change default and warn for things like Templates, but for things like indexes and such I say include it in the upgrade scripts. 14:46:54 * bshum envisions the upgrade SQL taking longer 14:46:56 denials: it would be nice to have a way to warn about it in the release notes and the end of the upgrade SQL, and then a separate script that does it that you run manually 14:47:21 +1 to jamesrf's proposal actually 14:47:21 just also because i'm sure with 2.0.5 -> 2.0.10 we actually reingested URIs twice 14:47:26 jamesrf++ 14:47:32 jamesrf++ # good idea 14:48:04 tsbere: Is there anything waiting in the wings for an alpha3 cut in LP (haven't checked lately) 14:48:22 Showstoppers I mean 14:48:29 LP still only has a target for alpha2 14:48:30 * tsbere plans on pushing at least one more branch today, and wouldn't object to the auditor boost branch being pushed, but isn't aware of showstoppers 14:48:36 Ack 14:48:53 #action Core team to update LP with alpha3 target 14:49:05 #action Bug wranglers to help retarget things appropriately. 14:51:13 Well, let's aim for an alpha3 sometime by Monday for announcement purposes. 14:51:29 Announcing on Friday/weekend seems... annoying 14:51:49 I'd like a fresh week to start testing alpha3 and beyond, but that's just my opinion. 14:52:34 Anyone else have suggestions for tsbere? 14:52:38 +1 to that. 14:53:24 I can do "by Monday" just fine. Most likely provided I do things Sunday night. ;) 14:53:33 * tsbere finds post-update mondays tend to be busy mornings 14:54:51 +1 announcing on Monday 14:54:56 So Monday 3/12 is expected 2.2 alpha3. We'll aim to test throughout next week, hopefully others can help there too. Then set for another dev meeting week starting 3/19 to discuss either alpha4/beta1? 14:55:07 (1 week testing enough or do we want more time?) 14:55:13 (assuming we announce it for realz) 14:55:57 mrpeters-isl++ # for offering further community testing as well for next alpha 14:57:16 #info Monday 3/12 is expected 2.2 alpha3. We'll aim to test throughout next week, hopefully others can help there too. Then set for another dev meeting week starting 3/19 to discuss either alpha4/beta1. 14:57:16 bshum: Sounds good to me 14:57:31 tsbere++ #release-cutting 14:57:54 testing.evergreen.lib.in.us has alpha2 on it right now, as of a few minutes ago 14:58:12 Clients: http://testing.evergreen.lib.in.us/updates/manualupdate.html 14:58:32 So, next topic? 14:58:33 loading up a bunch of bibs at the moment 14:58:47 bshum: please, sir 14:58:48 #topic Time zone proposal (jamesrf) 14:59:00 mrpeters-isl++ 14:59:11 just looking for some feedback, or if none, a note that feedback would be appreciated 14:59:55 jamesrf: to clarify, your proposal is to CREATE TABLE config.timezones AS SELECT * FROM pg_timezone_abbrevs; and use the name column as an fkey target from actor.org_unit, yes? 15:00:09 (or, perhaps, as an org_unit_setting?) 15:00:21 no, it would be the "link" datatype in org_unit_setting_type 15:00:25 put it in fieldmapper 15:00:50 Which is the same thing, and would likely be needed with the fkey anyway 15:01:10 jamesrf: right, perfect 15:01:16 and pg_timezone_names.name not abbrevs 15:01:23 just as it's more sane for the end user 15:01:30 imo anyway 15:01:32 agreed 15:01:37 +1 15:02:00 the other part of that is just what do do with manually specified due date times 15:02:12 since they no longer get automagically rolled to 23:59 without the db trigger 15:02:26 I'm pretty sure it won't cover everything TZ related, but it will get the most important data correct (circs 15:03:09 i guess my question is that currently if you manually set the due date/time it will sometimes be pushed to 23:59, but was this intended or even desireable in the first place? 15:03:28 it was intended 15:03:41 imo because there's a time selector, the user should expect that to be the due time 15:03:41 for any loans that have loan periods measured in days, IIRC 15:04:01 jamesrf: right, I just wanted to clarify how the TZ and the org unit would be linked. the rest I'm clear on. I'm totally in favor of the smart date/timestamp seletor in the staff client as well 15:04:05 denials: right 15:04:31 right i understand and obviously not good to change current behaviour, so hence the proposal to change the selector entirely 15:04:40 +1 15:05:36 great now that i know there's some support i'll see what i can mock up to that end thanks 15:05:45 Cool, so further questions/developments to come on the dev-list. Thanks jamesrf! 15:05:46 * denials vaguely recalls reading something about jamesrf's proposal 15:06:05 If someone wrangles a link, we can include it in the notes. 15:06:17 jamesrf: while we're on the subject of URI reingest (hah) https://bugs.launchpad.net/evergreen/+bug/918020 could use a poke 15:06:17 Launchpad bug 918020 in Evergreen 2.2 "2.0-2.1 upgrade script has wrong version of biblio.extract_located_uris" (affected: 1, heat: 6) [High,New] 15:06:45 (also, in general, in a future meeting we need to think about rel_2_0 and rel_2_1 releases again) 15:06:50 #link http://www.mail-archive.com/open-ils-general@list.georgialibraries.org/msg05855.html 15:06:56 jamesrf++ 15:07:20 denials: yeah saw that i'll poke it 15:07:25 Last new agenda topic was the GSoC update, which I saw denials wrote something on earlier. 15:07:29 #topic GSoC update 15:08:06 #info from earlier: (01:18:58 PM) ***denials offers a GSoC update: nobody has touched the "Summer of Coding" page at http://open-ils.org/dokuwiki/doku.php?id=dev:summer_of_coding_ideas since Feb. 27, maybe we should not apply this year. 15:09:35 What do we lack presently? Just need more ideas for things? More volunteers? 15:09:54 Or just overall, it's not going to pan out before the deadline (when is that btw?) 15:10:36 deadline is tomorrow 15:10:39 bshum: I don't see any signs of enthusiasm on the part of would-be mentors, and I'm not sure that's a good environment to bring students into 15:11:54 Hmm, alright. Well unless would-be mentors have any other plans/intentions, I suppose that's that? 15:13:04 would-be mentors as of last meeting included dbwells, senator, and tsbere (in case they're looking at something else atm) 15:13:31 tsbere senator dbwells and denials signed up as mentors so question to them would be can we just take a few of the items completed items off the list and use the rest? 15:14:38 you want mentors to put their names by the project ideas 15:14:48 obvious, upon scrutinizing the wiki page 15:14:55 I can through the search/relevance related stuff that came up recently on the ideas page 15:14:57 missed the call to action before now 15:15:00 moodaepo: For me, I guess it's less about the page as it stands, as the lack of attention that anyone else has paid to the GSoC 15:15:11 and if some of that gets picked up, mentor it 15:15:33 Mentoring a student isn't a last-minute rush kind of job, it's a commitment over a summer 15:16:18 denials: I just want to make sure your work prepping for this year's GSoC doesn't get wasted : ) 15:16:23 i'm aware of that. but mentoring a specific gsoc project and leading the whole evergreen gsoc effort are distinct things 15:16:25 denials: fair enough ... if you doubt my commitment to sparkle motion, I'll not try to keep my foot in the door and let it close 15:16:47 So like senator asked do they need to signup to particular projects? 15:17:11 Yes, students propose particular projects 15:17:17 gmcharlt & denials are 'leading' the effort 15:19:11 We said we'd put the application together. But it doesn't make sense to apply if the mentors are feeling guilted into being mentors. Maybe that's just my impression, I just don't sense real enthusiasm for it. 15:19:46 If I'm wrong, and people really want to be mentors, great! 15:19:49 i think we're just unaware of what it is we're supposed to be doing at this stage. 15:19:57 i'm puttin my name on specific projects now, fwiw 15:20:03 senator++ 15:20:05 denials: I feel guilted /out/ of it ATM ... anyway, once the page is unlocked for editing I'll add some stuff 15:20:14 eeevil++ 15:20:43 * tsbere can poke around on it after others have to throw his name on things too 15:21:06 * bshum needs to step away, transferring meetbot chair to moodaepo 15:21:32 #chair moodaepo 15:21:32 Current chairs: bshum moodaepo 15:21:43 I guess I expected that when I linked to the Google Summer of Code page and mentorship docs in http://markmail.org/message/74sql5dogh5aufhb that people might read them? 15:22:18 (I mean, to understand what the program is and how it works) 15:23:03 I had issues getting navigation to work on some of the pages I tried looking at. With or without javascript disabled. 15:24:19 Okay. http://en.flossmanuals.net/GSoCMentoring/ seems to be working fine now 15:25:08 I get no response from next/previous links on that page, for example. 15:25:45 tsbere: Same here with/without js 15:25:50 tsbere: try the PDF? 15:26:18 The whole thing works for me, HTML navigation, everything. 15:26:20 * csharp is able to navigate it using Chromium 15:26:27 Hmm 15:27:32 eeevil: so are you saying you want to mentor for 2012? 15:27:55 * denials wasn't clear what eeevil was talking about with "sparkle motion" 15:28:16 * moodaepo thinks he just wanted to help with ideas 15:29:14 denials: yes 15:29:20 denials: In any case I personally think the mentors are still interested and could also be backup mentors for say 2 students 15:29:26 eeevil++ 15:30:17 * moodaepo proposes we go ahead with the application (especially since he is doing jack : ) 15:31:23 okay. application deadline is tomorrow, we'll go ahead and apply (I'll work with Galen as previously indicated). please keep polishing up the ideas page! 15:32:01 if people want a PDF or ePub version of the mentoring guide, I can supply them 15:32:05 #action denials & gmcharlt will go ahead with GSoC application for this year. 15:32:31 Next topic? 15:32:53 Thursday afternoons work for me 15:33:02 #topic Developer Meeting Schedule - Should we move to Thursday afternoons or continue polling for meeting dates? 15:33:18 +1 15:34:09 Thursday afternoons going once 15:34:36 denials: please foward a pdf, that'd be handy 15:34:51 eeevil: Thursday afternoons still ok for you? 15:35:22 * tsbere dropped the PDF from the site there onto his droid for mobile reading 15:35:28 * bshum suggests putting it on the list to make a permanent move to Thursday afternoons, but generally supports it +1 15:35:32 * bshum is back 15:36:00 #chair bshum 15:36:00 Current chairs: bshum moodaepo 15:36:35 I kinda like the reminder-ish-ness of the weekly poll 15:36:50 but I'll go with whatever (that isn't noon) 15:36:50 eeevil: wget http://en.flossmanuals.net/_booki/GSoCMentoring/GSoCMentoring.pdf ? 15:37:05 (http://en.flossmanuals.net/_booki/GSoCMentoring/GSoCMentoring.epub for those so inclined) 15:37:08 The poll also lets us adjust around events, internal meetings, holidays, etc more easily 15:37:15 The doodle poll has been kind of nice. And gives us a more weekly flow 15:37:33 Fine in that case any volunteers to do the doodle poll? 15:37:41 I can do it again 15:37:48 kmlussier: ? being out resident doodle poll expert? 15:37:52 kmlussier++ 15:37:57 *our 15:38:00 Have to be an expert in something! :-) 15:38:09 I suggest the week starting 3/19th for the polling dates 15:38:09 For next week? 15:38:25 To give us next week to devote to really testing alpha3 and getting more community eyes on it. 15:38:28 #info kmlussier will setup/send out doodle poll for next week's dev meeting 15:38:29 gotcha 15:38:44 I guess that should have been an action 15:39:50 #action kmlussier will setup/send out doodle poll for the next dev meeting for the week starting March 12th 15:39:50 kmlussier: Either way works though if majority rules. 15:40:15 Next topic 15:40:21 Was that week of March 12th or 19th? Thought it was the 19th. 15:42:27 Ugh kmlussier it should have been 19th 15:43:19 #action Update previous action for kmlussier to the week starting March 19th 15:43:30 kmlussier++ 15:43:34 #topic Announcements 15:43:56 * moodaepo looks around 15:44:08 Think we're good. 15:44:36 bshum: That is a good announcement! 15:44:52 Closing the meeting then. 15:45:00 Thanks for attending this meeting. And thanks moodaepo for taking the reigns at the end. 15:45:01 #endmeeting