13:00:29 #startmeeting 13:00:29 Meeting started Wed Feb 6 13:00:29 2013 US/Eastern. The chair is gmcharlt. Information about MeetBot at http://wiki.debian.org/MeetBot. 13:00:29 Useful Commands: #action #agreed #help #info #idea #link #topic. 13:00:45 #info Agenda is http://evergreen-ils.org/dokuwiki/doku.php?id=dev:meetings:future_of_staff_client&#agenda 13:00:57 #topic quick round of intros 13:01:03 * gmcharlt = Galen Charlton, Equinox Software 13:01:08 er, 13:01:15 #info gmcharlt = Galen Charlton, Equinox Software 13:01:21 #info tsbere = Thomas Berezansky, MVLC 13:01:25 #info bshum = Ben Shum, Bibliomation 13:01:27 #info berick Bill Erickson, Equinox Software 13:01:27 #info csharp = Chris Sharp, GPLS 13:01:31 #info kmlussier = Kathy Lussier, MassLNC 13:01:33 #info Dyrcona = Jason Stephenson, MVLC 13:01:34 #info senator = Lebbeous Fogle-Weekley, Equinox Software 13:01:35 #info jeff = Jeff Godin, Traverse Area District Library (TADL) 13:01:35 #info DPearl = Dan Pearl, C/W MARS 13:01:36 #info jamesrf = James Fournie, BC Libraries Coop 13:01:38 #info ktomita = Kyle Tomita, Catalyst IT Services 13:01:42 #info krvmga = Jim Keenan C/W MARS 13:01:45 #info Rogan_ = Rogan Hamby, SCLENDS 13:01:56 #info paxed = Pasi Kallinen, Regional Library of Joensuu 13:01:59 #info davidboyle - David Boyle, Catalyst IT Services 13:02:04 #info dbwells = Dan Wells, Calvin College 13:02:17 #info Colin Campbell, PTFS-Europe Ltd 13:02:24 #info alexlazar = Aleksey Lazar PALS MN 13:03:13 #info eeevil = Mike Rylander / ESI (slow, phone, driving) 13:03:19 #info Tim Spindler, C/W MARS 13:03:23 ok, folks can continue to introduce themselves, but let's get started 13:03:28 eeevil: eyes on the road! ;) 13:03:31 #topic Do we still need a staff client? 13:03:39 #info denials = Dan Scott / Laurentian 13:03:40 to set the stage, this special meeting came out of the previous dev meeting 13:03:50 #info phasefx = Jason Etheridge / ESI 13:03:57 in particular, the discussion surronding bug 1086458 13:03:57 Launchpad bug 1086458 in Evergreen 2.3 "Staff client memory leaks in 2.3 and later" (affected: 4, heat: 34) [High,Triaged] https://launchpad.net/bugs/1086458 13:04:06 #info bradl = Brad LaJeunesse, Equinox 13:04:11 #shopkins = sue Hopkins C/W MARS 13:04:19 and while phasefx and I and others have been making some progress tackling the specific memory leaks 13:04:36 I thought it a good idea for us to step back and think about the future of the staff client 13:04:46 the agenda lists various related proposals 13:05:25 so I'm first going to list them all so that we can decide which ones we want to focus on 13:05:38 so the proposals so far are 13:05:49 #info go (almost) entirely web-based 13:05:58 #info have a minimal staff client for things like printing 13:06:11 #info keep the staff client, but move as much as possible into chrome 13:06:17 #info remove Dojo from the staff client 13:06:22 #info implement websockets 13:06:39 #info perform a third-party performance analsysis 13:06:56 as you can see, some of these proposals are mutually exclusive 13:06:59 some are not 13:07:05 I disagree with the first two 13:07:10 originally, back when EG was just on the whiteboard, the basic idea was to have a staff client that could handle network connectivity issues (even automagically go to standalone mode), but it obviously never got there. 13:07:51 is it fair to say that the two main factors that make a staff client are offline mode and receipt printing? 13:08:05 there was also a librarian prejudice against web apps. I think it may be a split opinion these days 13:08:34 Jason Boyer from Indiana - sorry I skipped an intro. I wanted to ask though, is there a specific reason that real, native clients aren't a viable option? 13:08:39 offline mode, receipt printing, local data storage concerns, and RFID integration are my thoughts for why we should be sticking with a staff client instead of web app 13:08:46 offline mode could be implemented today in a pure web app (offline/manifest) 13:09:15 in principle, a staff client can do some things much faster than any web app can ... but does anybody actually disagree with the statement that most of the staff client isn't actually very quick to respond? 13:09:17 we also get about 8MB of local/session storage in modern browsers (not sure if that's enough, but..) 13:09:31 jasonb: Portability is one major reason that native apps are not a consideration. 13:09:49 another con of staff clients is the fact that you have to deploy them and manage upgrades ... although of course that's gotten a lot easier with tsbere's work on client auto-update 13:10:05 the staff client feels very sluggish to me, at least when looking at the dojo tables and such 13:10:12 gmcharlt: does the Koha world have any regrets about being web-based only? 13:10:12 One major concern I have about a pure web app is browsers. Both keeping up with them and the fact that people change them. Printing in a pure web app is pretty much impossible to do without prompts now as well. 13:10:16 almost entirely web-based client would be OS-agnostic, possibly allow libraries to run cheaper hardware (Chrome OS) and would not require local system admin access 13:10:43 for those who are not aware, c/wmars is experiencing some extreme problems with slowness 13:10:45 bradl: not really, except for label printing; offline circulation is managed via a couple options -- a standalone php app or a Firefox extension 13:11:02 one option for web-based is to not be browser agnostic, which defeats some of the point 13:11:17 label printing in EG is not awesome either 13:11:41 phasefx: and that's certainly true in Koha -- the general notion is that the OPAC should run everywhere, but the staff web interface has always favored Firefox and more recently Chrome over IE 13:11:43 chsarp is absolutely right about that, an ongoing complaint almost 4 years in for us 13:12:12 I don't think the current state of the staff client makes anyone jump for joy :) 13:12:19 I think realistically, if we go web only, we will need to target a single browser. We just don't have enough resources. 13:12:48 dbwells: I think targeting FF and Chrome would be reasonable, based on Koha's experience 13:12:59 another benefit of web-only is that it could work on tablets 13:13:01 dbwells++ 13:13:04 nope. Personally, I'm interested in a web based staff client but my mind is far from made up. 13:13:08 targetting IE as well depencds on somebody championing it 13:13:14 dbwells: unless targeting more than one browser is free, becasue they support the same HTML5 and other standards 13:13:20 dbwells++ 13:13:21 dbwells++ 13:13:38 from the i18n viewpoint, going web-only would make translations a bit easier, no need to reinstall the staff client to every workstation when the translation strings change. 13:13:39 "the same HTML5" and "standards" - hahaha :) 13:13:41 and of course, one thing to keep in mind is that a bunch of staff client interfaces *already* work in web browsers sans XUL 13:14:06 i think if things were to go web-only targeting firefox makes a lot of sense in terms of transition 13:14:17 those are also some of the most clunky, perhaps because of our ancient dojo 13:14:28 I think if it does go web only there will be mutiny when people have to answer printer prompts all the time in circ ;) 13:14:30 OK, I'm going to give another couple minutes for this topic (I'm planning to run through them all pretty quickly, then leave time at the end for more in-depth discussions) 13:14:35 How do you handle offline data? 13:14:49 phasefx++ 13:15:02 jamesrf: targeting an open-source browser in particular (firefox or chromium) 13:15:04 tsbere: losing all of the receipt context functoinality will also hurt. 13:15:24 wolf29: HTML(5) offers several different local storage possibilities, depending on the browser 13:15:39 denials beat me to it. 13:15:40 but if we went with FF, couldn't receipt printing, etc. be done via addons? 13:15:43 wolf29: and offline application execution 13:15:52 so it seems like not a lot of support for going the xul/chrome route? 13:15:55 if we are going to start using addons, why are we moving away from xul? 13:16:02 Note that web only we also lose top level menus and toolbars 13:16:03 jamesrf: we haven't discussed it 13:16:19 thanks. just wanted to get it in the record.. 13:16:32 #topic Move much as possible to chrome 13:16:40 tsbere: it could be a question of "a little bit of xul or NaCl" versus "scads" 13:16:51 I listed a cuople pros and cons in the agenda 13:17:16 where "chrome" means "local XUL client", not "Chrome the browser" 13:17:25 in case anyone is confused :) 13:17:28 I think moving things away from remote xul would help 13:17:45 biggest pro I see is signficantly improving responsiveness for loading UI elements and reducing how much we have to think about certain times of caching 13:17:55 A *lot* of requests for otherwise identical files happen right now in the system due to query strings being used to pass arguments. Moving those files local will reduce network traffic. 13:18:25 biggest con I see is that dropping remote XUL is going to increase the compiile/build/test cycle for staff client development 13:18:36 in general, the less that we stray from the mainstream (see tsbere's tremendous effort to keep remote xul working) the better we can focus on value-add dev, IMO 13:18:44 removing query strings in and of itself could also help with the staff client performance re: caching 13:18:48 denials++ exactly 13:19:06 another pro I see: moving away from remote XUL, while not a completely mechanical thing, does lend itself to a bunch of devs working on it in parallel 13:19:08 tsbere++ libraries with weak network bandwidth will benefit 13:19:35 denials comment (re mainstream) is a great reason to avoid xul entirely, imo 13:19:48 berick ++ 13:19:48 moving away from remote xul also lets us make a less server dependent staff client 13:19:50 berick++ 13:20:27 is anyone doing fancy tricks with remote xul and ip-based redirection for customization? 13:20:30 I don't see ditching xul as a good idea unless we are ditching the xulrunner staff client 100% and replacing it with an entirely different one 13:21:07 And while some people would like to see us ditch xulrunner in favor of other staff client platforms I don't see that happening anytime soon 13:21:22 If we are using xulrunner we should be using xul, basically 13:21:46 yeah, I see a balancing act 13:22:04 XULrunner is not a priority for Mozilla, but we have a lot invested in it 13:22:09 there will be transition (of an undefined time / form), and moving the xulrunner staff client away from remote xul to local xul could be part of that. 13:22:32 it would likely breath more life into / revive the xulrunner based staff client. 13:22:34 and sticking with it is potentially a lower hurdle for us in the medium term than adopting another platform 13:22:38 gmcharlt: also balance that with the amount of time you and phasefx have spent plugging memory leak after memory leak 13:23:00 we're going to have the staff client for a while regardless 13:23:03 most of the memory leaks I have seen being plugged can be blamed on coding styles, I think 13:23:09 not xulrunner or xul 13:23:18 bradl: yes, we've discovered that we're very much at the mercy of XULRunner's garbage collector ... and not because of Dojo, by the way 13:23:19 javascript closures are problematic, in particular 13:23:29 plugging memory leaks is a worthwhile effort thank you 13:23:34 indeed, memory leaks can happen anywhere 13:24:01 and by anywhere, I mean IE, of coure 13:24:08 berick++ 13:24:11 berick++ 13:24:15 berick++ 13:24:26 and those interested in experiments such as implementing more staff features in web interfaces would gain the immediate benefits they seek from that (such as, say... support on platforms where xulrunner does not run), and said experiments would result in more data / insight as we further evaluate the status of the staff client / staff interfaces. 13:24:43 berick++ 13:25:27 jeff++ 13:25:27 ok, moving on 13:25:30 #topic Removal of Dojo from staff client (proposal by Thomas Berezansky) 13:25:48 I'm in favour of a web-based staff-client; I presume less memory leaks and more likely to run on my Rasberry Pi - but it's years away 13:26:36 is dojo the for-certain culprit in poor performance in circ and patron edit screens? 13:26:43 My main arguments for removal of dojo (and not replacing it with another javascript framework) is the fact that these frameworks aren't designed with xul in mind, they are designed with html in mind. This includes how browsers treat web pages security-wise, which is very different from how xul is treated. 13:26:53 krvmga: nope, the leak profiler doesn't bear that out 13:27:26 Other arguments are the fact that almost every interface I have trouble with, and most of the ones I hear people complain heavily about, are dojo-based interfaces 13:27:39 and the fact that they just don't look right alongside the xul ones 13:27:51 I would say the question of removing Dojo is dependent on the decision to keep the xulrunner client or to go with a web client. 13:28:26 I think the web sockets question is orthogonal to both. 13:28:47 agreed; and I think the JavaScript framework is *also* orthogonal 13:28:49 Dyrcona: agreed on both pointt 13:29:25 i can understand how in a pure-xul environment, dojo (etc) makes less sense 13:29:50 The only arguments for dojo I have seen are "it lets us quickly make interfaces" but it isn't that hard to build on top of xul for most of those same uses 13:30:13 I think we ought to make a decision concerning xulrunner vs. web, then. 13:30:30 so what happens if Mozilla stops developing/supporting xul? 13:30:37 xulrunner, even 13:30:39 Dyrcona++ 13:30:52 csharp: i think that's a valid concern 13:30:55 csharp: like they have abandoned xul on mobile, fer instance? 13:30:55 csharp: that's one of the not-so-invsible elephants in the room 13:30:59 yes 13:31:00 csharp: Given that xulrunner is supposedly the base of firefox I imagine mozilla goes away entirely? 13:31:44 but the xulrunner-not-as-a-component-of-firefox project is... tenuous (at least to me) 13:31:44 Or, they pivot on their mobile efforts for firefox.next, dropping xul. who knows? 13:31:47 unless mozilla focuses their considerably-larger-than-Evergreen development resources toward changing/replacing xulrunner in a way / with something that is no longer suitable for Evergreen's use. 13:31:53 tsbere: I think that's not so likely as Mozilla deciding that they only care about XUL for their own projects and Firefox extensions 13:32:33 gmcharlt++ 13:32:37 gmcharlt: (such as changes to remote xul support) 13:32:44 jeff: indeed 13:32:52 In any case, rewriting all of the Dojo UIs we currently rely on in the staff client is not a trivial effort 13:33:11 but deciding to stop making new ones would help on that front 13:33:25 denials++ 13:33:28 tsbere++ 13:33:43 tsbere++ 13:33:44 that's a tough choice, too 13:33:51 tsbere: you mentioned a desire to re-create some of the "let us quickly make interfaces" bits in local xul? 13:34:25 Most of the interfaces we can "quickly create" are simple config interfaces. Those should be trivial to make compatible with a fully xul interface. 13:34:26 tsbere: sans a framework, building a framework ourselves, i take it? 13:34:52 The question becomes what other things are we thinking dojo lets us make? 13:35:20 It's currently used in the autosuggest UI in the TPAC. 13:35:42 * berick doesn't foresee anything trivial coming out of this discussion, regardless of the outcome 13:35:43 I am not looking to remove dojo from Evergreen 100%. Just from the staff client itself. The catalog can keep using it for autosuggest, IMO. 13:36:29 I guess I'm not convinced that xulrunner is a safe bet for the future 13:36:34 (feel free to indicate that this is not a question relevant to the current topic) Assuming dojo in the staff client stays, what efforts are required to take advantage of some of the (GSoC?) work done to bring us "more current" with recent Dojo releases? 13:36:47 (says one who manages "we can no longer print" tickets after upgrades) 13:36:59 csharp: I agree with you, FWIW 13:37:01 jeff: Re-write most to all of our dojo modules, I think. 13:37:16 So then we just need to pull the fieldmapper-parsing logic from openils.fieldmapper into a non-Dojo library and rely on that directly to generate XUL bits & control perms and such? 13:37:20 Which is another argument against dojo for me. If we have to re-write a lot of stuff, in whole or in part, why not just stop using the framework? 13:37:33 xulrunner losing support for text-only printer drivers on windows was a pain for everyone, and not much of a pain for mozilla, thus another bug/change where xulrunner's focus left us behind. 13:37:37 I came up with a non-dojo fieldmapper already, for the record. 13:37:38 at least dojo is an active project with its own momentum, for instance 13:38:08 tsbere: oh? might have been good to link to that in the meeting notes 13:38:16 rewrite is perhaps stronger than warranted 13:39:05 eeevil: The entire loading mechanism for dojo has changed significantly, just to start. 13:39:08 dojo 1.7 introduced a different loading mechanism, but I'm with eeevil 13:39:33 csharp: web based rather than xulrunner has been identified as having challenges with: 1) offline (probably do-able now with modern browsers), 2) printing (i would like to investigate a native print client that knows how to print well, and only print well), but I'm also a bit worried about the cataloging interfaces. 13:39:36 but not the logic 13:39:53 just the wrapper 13:39:55 csharp: implementing circ staff functions would be where i think of starting web-based experimentation. 13:40:05 jeff: what about the cataloging interfaces? 13:40:11 anything in particular? 13:40:22 that's a start, but seeing something complete and concrete beyond vague assertions that it's not suited for XUL would be better 13:40:55 berick: sorry, i was doing a poor job of saying "implementing the marc editing functions in a web interface seems daunting to me at this point in time" 13:41:06 eh just replace the cataloging interface with an XML editor 13:41:09 berick: but not impossible, and likely with benefit if done well, etc. :-) 13:41:13 jeff: aww, you don't like openils.marc ? 13:41:33 jeff: ah. yeah, doesn't really scare me 13:41:41 nor I 13:41:45 excellent. 13:41:56 berick++ 13:41:59 gmcharlt++ 13:42:04 On the "dojo isn't suited for xul" front: Dojo is an HTML JavaScript framework. Why would we expect it to work with non-HTML well? All of our significant dojo interfaces are HTML, not XUL, as it is. 13:42:07 considering almost every interface I use on a daily basis is web-based, i think we can manage our little corner of the world 13:42:27 berick++ 13:42:30 tsbere: because XUL is XML and Dojo deals with XML 13:42:45 berick++ 13:43:05 OK, we can get back to this, but moving on so that we touch on everything... 13:43:10 #topic Websockets 13:43:16 I honestly think the discussion about Dojo in the staff client is moot. 13:43:26 * csharp will need to read berick's posts 13:43:29 websockets++ 13:43:34 If we go web-based I think we can skip websockets. We don't need 30 websockets per client open. 13:43:38 berick++ websockets posts and proof of concept 13:43:47 tsbere: it would be 1 open connection 13:43:58 tsbere: curious, where do you get the 30 number? 13:44:06 berick: One open connection per open window. Cross-window communication isn't that good on web browsers. 13:44:15 jeff: The number of open tabs I have seen whenever I visit a library. 13:44:36 for starters, we can design a UI that does not require piles of tabs 13:44:40 like every other web site 13:44:42 in the world 13:44:45 heh 13:44:46 berick++ 13:44:53 berick++ 13:45:01 berick: "Doesn't require" and "staff don't open them" are two different things. 13:45:10 fwiw, I rarely see our staff having more than a couple of tabs open - cataloging or circ 13:45:13 We already don't need 15 catalog tabs open at a circ desk. I still see it. 13:45:56 Yeah we applied a limit to our tabs for the same reason. 13:46:08 lots of different workflows in PINES - several use multiple tabs, others just two or so 13:46:25 tabs were a much lauded feature 13:47:11 anyway - websockets - perhaps in parallel with websockets, would it be possible to make the client (web or xul) less chatty - do more work on the server and return more per request? 13:47:22 denials: +1 13:47:39 i have wondered if there would be benefit to do more fieldmappering on the server 13:47:47 I _think_ that would be a performance win :/ 13:47:53 denials: That's a topic for another day, I think: Backend Overhaul. :p 13:48:07 we cannot make progress by avoiding new technologies due to concerns about possible un-recommended use of the software. if 30 tabs is common and is a resource drain, we can either recommend against, or fall back to typical ajax / non-persistent connections after the first X tabs, etc. 13:48:22 jeff: How do we count tabs? 13:48:23 Dyrcona: I don't think it would be much of an overhaul, just move more logic from staff client to server 13:48:34 or do UI work to address whatever drives somebody to open so many tabs at the circ desk 13:48:35 could be entirely new methods that just wrap what's there 13:48:37 I don't know of any way to determine how many tabs of a given website I have open from the website. 13:48:49 gmcharlt++ # right 13:48:55 gmcharlt: exactly. 13:49:11 tsbere: a detail that is probably too specific for the scope of this meeting, but counting open websockets for a session token comes to mind as one possibility. 13:49:12 @who wants to hire a UI pro? :) 13:49:13 denials: Beyond here be dragons. 13:49:24 ... and of course, as we all know, the answer for the circ desk is green screen terminals! ;) 13:49:47 gmcharlt++ 13:49:48 As I said, if we go web based I think we need to ditch websockets. If we stay with an actual client I am all for them. 13:49:50 They are heavy and useful for blunt force 13:50:15 jamesrf: i've toyed with the idea of a "thick" gateway / api endpoint to make external integration easier for those who look at the fieldmapper and go "nevermind, we'll just screen scrape" 13:50:24 tsbere: what is your main objection to web sockets on the web based? 13:50:28 and if we do a hybrid, web for ui, add-on for printing (and websockets)? 13:50:39 jamesrf: Too many open from a single workstation. I already said that. ;) 13:50:49 tsbere: I don't think they're mutual exclusive ... a web-only interface could still be heavily dependent on a lot of AJAX 13:50:52 tsbere: my understanding of websockets in a web based interface is that they can be an optional enhancement, with fallback. i could be wrong on all counts. 13:50:53 tsbere: right but i'm sure others have encountered this in using web sockets? 13:51:04 phasefx: I disagree with hybrid, in general, but websockets could be used to communicate *to* the printing client in that case. 13:51:56 OK, moving on for the moment 13:51:59 #topic Objective third-party performance analysis to identify bottlenecks in the system. 13:52:15 kmlussier: could you expand on this? 13:52:17 After the previous discussion at the last dev meeting, I thought it might be useful if the community had third-party software peformance analysis to identify where the most severe bottlenecks are in the system. 13:52:22 incidentally, the reason for going with xulrunner in the first place was to leverage web-based technologies under the hood while disguised as a local app, plus offline mode and printing 13:52:25 And while this analysis might include the staff client, it wouldn't be limited to it. 13:52:39 The main benefit being that any decision we make on changing the staff client or other pieces of the architecutre would be based on evidence generated by this analysis. 13:53:02 kmlussier: i'm interested to know if there was anything in particular that led to the use of "third party" and "objective" in the proposal 13:53:34 haha, not to speak for kmlussier, but we developers can't always smell our own... well it's a meeting so i'll avoid rough language 13:53:44 Well, I think everyone here has certain biases and particular technologies they are tied too. 13:53:56 senator++ 13:53:57 nor do we often eat our own dogfood 13:54:05 * denials supports anything, as long as it runs on Fedora 13:54:07 kmlussier++ for being polite 13:54:09 senator++ 13:54:11 kmlussier++ 13:54:17 * Dyrcona eats his own dog food, quite literally. 13:54:20 We have a lot of libraries that are anxious for performance improvement. 13:54:22 kmlussier++ 13:54:26 kmlussier++ 13:54:28 kmlussier: i would be in favor of more performance analysis / staff interface instrumentation / etc, but (other than "there has not been a comprehensive effort by the community yet" (perhaps unfair, especially in light of recent memory leak work), is there a reason you think it would need to be "objective third party"? 13:54:33 And my main concern is that we promise that a solution is coming 6 months down the road. 13:54:33 kmlussier++ 13:54:35 kmlussier++ 13:54:54 But, when it's implemented, it leads to very little improvement becaue there is a bigger issue behind the scenes that has nothing to do with the staff client. 13:54:55 kmlussier: okay, so developer biases for favorite methods/tools... anything else? :-) 13:54:55 jeff: 1) avoids biases 2) avoids getting derailed in "oh another bug to fix" 13:55:11 jeff: my own understanding is just to have it done by someone who doesn't have a dog in the fight, so to speak. 13:55:12 we could probably get a lot of win by hiring Mozilla developers to poke at our use of xul, but it wouldn't be all that objective 13:55:13 cool. thanks for addressing my curiosity. 13:55:44 kmlussier++ 13:55:51 it probably goes without saying that "third party objective" == money? do you have anyone in mind at this point? 13:56:09 No, I don't have anyone in mind. 13:56:24 but what third party exists that wouldn't have a bias for any particular technology or solution but *would* have an interest in Evergreen? 13:56:31 I think a massive profiling of the server code while serving client requests would help as much as anything. The machines I'm using can't be made this slow by any sane combination of JavaScript, but a server that doesn't respond immediately makes the client seem slower than it really is. 13:56:31 I know MassLNC could contribute *some* funds, but I think it's important that if this were to happen, it's a community project, not a masslnc thing. 13:57:15 But I'm in no position to actually perform that, just mentioning it. :/ 13:57:27 I agree with the fact that some of the staff client issues are very likely actually server side code issues. 13:57:39 I think finding/hiring such a person or entity would have to be handled very carefully to make sure that the results (a) got buy-in from whoever would implement the recommendations and (b) actually participate in the community, at least during the course of their project 13:57:40 and redundant network calls to the server 13:57:40 Sitka/BCLC would support such a project as well 13:57:41 I would like to have more instrumentation overall (we already have some examples, like the timing debug in tpac) for evaluating new code / code changes with performance in mind. 13:57:43 gmcharlt: fair enough, we might get a recommendation that SQL Server would fix all of our problems :) 13:58:13 perhaps we could preceed any 3rd-party invovlement with our own list of ideas? i'm sure we have them and we'll probably find a lot of agreement 13:58:16 that said, I don't disagree with the concept of getting help from folks who specialize in (say) UI or performance issues 13:58:20 denials: or oracle 13:58:50 ("test cases are hard, performance test cases doubly s[l]o[w]"?) 13:59:04 and certainly things like third-party security audits (thanks denials and Conifer!) are useful 13:59:17 But 1) I don't think we have the capacity to do this kind of work in our existing community, heroic weekend efforts that lead to memory leak fixes aside 13:59:53 2) We might also lack the expertise to do a good job 14:00:00 denials++ 14:00:09 Evergreen has a huge learning curve 14:00:20 they might spend 6 months just learning how it works 14:00:26 csharp: indeed 14:00:28 csharp: true 14:00:33 csharp: agreed 14:00:36 csharp++ 14:00:38 csharp++ 14:00:42 * paxed looks around and pretends he knows something about evergreen. 14:00:48 I wonder if it would be worth dedicating a whole release cycle to performance issues, rather than new features. So 2.5 (for example) doesn't get any new features. 14:00:50 paxed++ 14:00:53 ideally we would come out of this with a set up that would be repeatable 14:01:14 dbwells: I find that an interesting idea ... but would there be community support and funding? 14:01:14 dbwells: not a bad idea 14:01:18 (if the proposal went forward) so that we could measure progress 14:01:26 dbwells: I think that makes some sense too, a lot of work done on adding features but not addressing long term problems 14:01:41 dbwells: it would be a really tough sell :/ 14:02:04 no one wants to pay for "bug" fixes if you can call it that 14:02:24 a performance test suite would be something the entire community benefits from 14:02:36 Bugs aren't working properly. Performance improvements are a different thing. 14:02:47 I used the term loosely 14:02:51 tspindler: the argument is likely that no devs get paid if they're not writing new features 14:03:20 dbwells: I like the idea too, but, looking at my list of wanted enhancements, it would indeed be a tough sell. 14:03:31 performance improvements you can pitch and get paid for, but saying "no new features" would be tough 14:03:34 mrpetersisl: unless they are specifically charged with clean-up 14:03:35 But I'm also concerned that the enhancements we keep sponsoring are just making performance worse. 14:03:41 which does tie into a general oversight committee discussion about fundraising, including for general infrastructure work 14:03:53 who charges them to only work on patches, wolf29? 14:04:04 in theory, you could pitch "unit tests, regression tests, system tests, performance tests" and get paid for that too 14:04:15 denials++ 14:05:14 mrpeters-isl: if you hire someone to do a job, you pay them to do it, is all I am saying. :-) 14:05:20 ktomita: http://coffeecode.net/archives/228-Got-funds-for-enhancing-Evergreen-Looking-for-places-to-spend-it.html :) 14:05:38 I can't speak for the oversight committee obviously but I do like the idea of organizing fund raising for things like this and QA. 14:05:38 to suggest a possible action item, does somebody (or somebodies) want to research concrete people and/or entities offering indepndent experitise? 14:05:47 The low hanging fruit for performance is databse queries. Cataylst and Commandprompt just finshed fixing holds counts for KCLS. We took 4-5 second queries and turning them into 4-5ms queries 14:06:12 smyers_: great! was there a launchpad bug with that issue and fix? sounds quite promising. 14:06:17 gmcharlt: I could work on it, but I think I would need help. 14:06:18 smyers_++ I think you are on the right track 14:06:25 Halting feature development for refining is an easy sell to my users in terms of being a good idea but I don't know if they'll put funds directly into it (they will my time though!). 14:06:27 smyers_++ 14:06:40 jeff++ launchpad_entries++ 14:06:43 kmlussier: an evidence-based analysis could be done by the community, not necessarily by a 'third-party'; I think the key is to use the relevant tools to gather the evidence. 14:07:09 smyers_++ # yep - bug entry would be nice as mentioned! 14:07:15 I liked berick's ideas of coming up with a list of ideas before finding somebody. 14:07:27 smyers_: did you add coverage for all the hold types, then? 14:07:53 We are looking at how to implement the fix into latest as KCLS is 2.1...2.2 14:08:01 stevenyvr2: i do like the idea of the community doing (at least) some initial ground work in advance of hiring someone and saying "make this faster" 14:09:05 jeff: I think kmlussier's proposal was to have the 3rd party say "Here are the bottlenecks" - and stop there 14:09:39 that was my understanding as well 14:09:42 hiring someone to fix those bottlenecks, or fixing them within existing community resources, would be a different matter 14:09:42 ah, right. i guess revised that would be "tell us where this is slow" :-) 14:09:46 We may be looking at more than one person. There are a lot of fronts to attack here. 14:10:03 Yes, exactly. I think we would need to make it clear up front that they won't be getting further development work out of it. 14:10:15 * denials said 3rd party there to enable it to be an organization :) 14:10:36 DPearl++ 14:10:40 That's my kind of party! 14:10:49 client, opensrf, database, ah, only three fronts :) 14:10:53 denials: yeah, "someone" in my usage above was not meant to exclude organizations :-) 14:11:12 3rd-party + community input -> list o things -> launchpad 14:11:22 * gmcharlt is wondering what we can do to increase expertise *within* the community about performance and UI 14:11:29 berick++ 14:11:34 berick++ 14:11:36 denials: you forgot Windows antivirus / endpoint protection products and proxy servers! ;-) 14:11:38 gmcharlt: Study? 14:11:40 berick++ 14:12:07 I think part of the problem is the lack of test suites for the software, not just for performance, but for functionality. We need test suites in a test framework that can be repeated. 14:12:10 gmcharlt: expand time? 14:12:17 Dyrcona: indeed, we certainly all have a responsible to do that ... but it may be helped by also investigating training options? 14:12:21 stevenyvr2: I _knew_ I liked you. 14:13:09 If there were test suites/test framework that have good coverage, we would have all the evidence we want. 14:13:31 gmcharlt: knowledge sharing? study groups? more developers writing documentation? lots of possibilities, but short of denials's time dilation idea i'm not sure of how to encourage said possible options. 14:13:43 stevenyvr2: Testing suites like that would be a good argument for web-based rather than staff client 14:13:44 Hello Evergreeners, I have a question. I just created noncataloged types (paperback, magazines etc) in the editor in Local Admin. I then went to check out and don't see my new ones there. When I go back to the editor, my items are there. How do I get them to appear under checkout? 14:14:15 JennB: There's a meeting in progress. 14:14:18 jeff: right, but I do think we need to think about it; after all, things like test suites don't happen without time commitments either 14:14:18 JennB: hi! not to pile on, but we're in the middle of a meeting -- if you could stick around and ask your question after the meeting, or post to the evergreen mailing list for an answer... 14:14:55 Oh Im very sorry! I didnt realize. 14:15:31 so dragging us back to the agenda 14:15:32 test suites would be time consuming but the benefits would be great 14:15:46 ktomita++ 14:15:49 I think we've had a good discussion of various options 14:16:04 2.5: All performance; all test coverage; all honing of our awesome developer skills 14:16:12 denials++ 14:16:18 Test suites could be built up over time; the key is to have a good framework. 14:16:19 but I'm now opening the floor to proposals for action items 14:16:20 denials++ especially on the last one 14:16:31 denials++ yes! 14:16:59 JennB: check PMs 14:17:07 mrpeters-isl++ 14:17:28 #topic Next steps? 14:18:23 jeff: [since it fits in with next steps, I'll hit enter...] entities writing features or fixing issues (bugs, performance, otherwise) could be strongly encouraged at the point of a commit bit to provide more documentation earlier. there's a not-insubstantial cost (effort, money, delay) to that, of course, regardless of the entity producing the code 14:18:46 one idea might be to use cucumber to leverage the super awesome docs group to allow them to write some test cases 14:19:12 Work Together. Work in Public. I'm interested in several aspects of this, and have some ability to devote time. It would be good to know others interested in similar bits, and work / study together to further those goals/interests. 14:20:00 have we veered away from the point of the meeting, though, of "future of the staff client"? 14:20:02 * denials seems to recall joelewis writing a bunch of unit tests for his dojo.next work... 14:20:02 jamesrf: interesting 14:20:19 jeff: evergreen is a community after all. 14:21:02 eeevil: we have indeed, a bit, given that performance in general has been discussed as a important issue 14:21:19 eeevil: I would say that if the future of the staff client requires decisions based on objective criteria, and based on the level of skill / capacity of the community to overcome performance problems / UI problems, then it's related to the topic 14:21:30 but that raises a question: of the specific *staff client* proposals made during this meeting, is there a consensus on any approach? 14:21:39 Java 14:21:42 * bradl runs 14:21:44 denials: it's certainly related, yes 14:22:23 gmcharlt: based on my understanding of the "camps", I'd say no. for some it seems black or white. 14:22:51 i'm interested in experiments with browser-based, clientless staff interfaces. i'd like to work with others who might share those same interests. i don't suggest that this mean "xulrunner staff client is dead, the new web-based UI will be ready in a few months" 14:22:52 but we have more cards on the table now 14:23:17 jeff: ditto. 14:23:37 jeff: I am interested in that as well 14:23:41 * phasefx wonders if folks really would tolerate a green screen 14:23:42 i really fear the browser imcompatibility issues if we mvoe to a web based UI 14:23:54 my sense is that we're at a point where experimentation, hopefully with some concrete code, is in order on various fronts 14:24:01 i like the idea of moving toward less remote xul (for many of the reasons stated earlier) in the staff client, but i do not have strong xul chops (other than chopping out unused interfaces), so i can't say i'll be much help there, but as i mentioned earlier it seems like a good direction to move toward, since nobody is saying "kill the xulrunner client TODAY" 14:24:07 movement to a dojo-less client? 14:24:09 yeah, we can tell libraries to install a particular browser/version but it's much easier to say "install this client" 14:24:27 krvmga: i've no strong opinion on dojo in the staff client. 14:24:42 mrpeters-isl: I think it's not as horrible as you imagine 14:24:53 we would never be able to enforce any kind of browser standard in our network 14:24:53 krvmga: there's "remote xul" and "local xul", and i think the reasons for shifting toward more "local xul" bear further investigation. 14:24:56 with browser incompatibilities 14:25:00 would it be fair to say that there is a consensus that moving await from remote XUL is a good idea in the short term? 14:25:03 mrpeters-isl: there are tools like modernizr that could help and we could always put a big alert saying "your browser isn't the best" 14:25:03 we've got 100+ libraries and now we have to support their web browser? no thanks 14:25:09 never matches my sweet imagination :) 14:25:13 gmcharlt: I would agree with that 14:25:21 gmcharlt+1 14:25:40 gmcharlt: +1 moving away from remote xul 14:25:45 mrpeters-isl: ditto here 14:26:16 +1 moving away from remote xul 14:26:26 gmcharlt: +1 (despite the debugging pain) 14:26:32 OK, so I'd like to call for the sense of the room on the following statement: in the short term, Evergreen devs are encouraged to start moving staff client interfaces away from remote XUL 14:26:43 gmcharlt: what is the benefit 14:26:51 i missed much of this talk 14:26:56 just speed? 14:26:58 mrpeters-isl: loading time decreased 14:27:06 mrpeters-isl: Less network traffic downloading the xul files, especially ones that get loaded with query string arguments 14:27:07 mrpeters-isl: scrollback for more, but performance and less hackery 14:27:57 i guess i have no objection then 14:28:17 in addition to traffic/performance/caching benefits, "remote XUL" is far less supported than it used to be in xulrunner, requiring (denials just said it) hackery. 14:28:26 yeah, i recall that part 14:29:12 why was remote xul chosen in initially? just wondering why we didn't go for local xul in the beginning. 14:29:29 just trying to understand what has changed the philosophy 14:29:41 mrpeters-isl: mozilla stopped supporting remote xul based on security concerns 14:29:49 ease/speed of development. We were making changes nightly for PINES 14:30:02 thats what i was looking for, thanks phasefx 14:30:06 mrpeters-isl: because it gave us one of the main benefits of a web app -- remote deployment 14:30:06 that changed as remote xul became a red-headed stepchild for Mozilla 14:30:15 sure, i understand now 14:30:25 in the long-ago past, we (our library system, not the larger community) used remote XUL to have one stock client behave differently in certain interfaces based on server host/hostname that it was connecting to, and to deploy changes within the staff client without a client install / update being required. 14:31:07 since we're at an hour and a half, i'll just do this 14:31:10 #action Evergreen devs are encouraged to start moving staff client interfaces away from remote XUL 14:31:48 and unless I hear any other items for the formal part of the meeting, I'll plan on ending in a minute 14:31:58 gmcharlt++ 14:32:03 obviously, it's clear that this is going to be a long-running discussion 14:33:06 gmcharlt++ 14:33:07 #endmeeting