2011-01-07T01:33:54 *** mrpeters-isl has quit IRC 2011-01-07T03:24:02 *** youdonotexist has quit IRC 2011-01-07T03:24:43 *** youdonotexist has joined #evergreen 2011-01-07T06:24:41 *** Dmagick has joined #evergreen 2011-01-07T08:15:11 *** alxp has joined #evergreen 2011-01-07T08:21:11 *** mjg_ has joined #evergreen 2011-01-07T08:21:19 *** mjg_ has left #evergreen 2011-01-07T08:35:51 *** rsinger has quit IRC 2011-01-07T08:50:18 *** Meliss has joined #evergreen 2011-01-07T08:52:21 *** Meliss has quit IRC 2011-01-07T08:54:04 *** finnapz has left #evergreen 2011-01-07T09:00:15 *** Melissa has joined #evergreen 2011-01-07T09:08:05 *** mrpeters-isl has joined #evergreen 2011-01-07T09:08:12 *** Dyrcona has joined #evergreen 2011-01-07T09:10:29 *** jenny has joined #evergreen 2011-01-07T09:14:22 *** bshum has joined #evergreen 2011-01-07T09:20:26 *** rsinger has joined #evergreen 2011-01-07T09:23:21 *** r123 has joined #evergreen 2011-01-07T09:37:11 *** ebyr has joined #evergreen 2011-01-07T09:43:21 *** finnapz has joined #evergreen 2011-01-07T09:57:06 *** rjackson-isl has joined #evergreen 2011-01-07T10:13:12 *** bshum has quit IRC 2011-01-07T10:15:51 anyone who might be using SIP found a way to have the logs written elsewhere than /var/log/SIP.log? perhaps to a syslog-ng? 2011-01-07T10:17:44 ^^ ignore me i found some info 2011-01-07T10:21:28 *** bshum has joined #evergreen 2011-01-07T10:24:14 *** youdonotexist has quit IRC 2011-01-07T10:43:08 *** kmlussier has joined #evergreen 2011-01-07T10:43:59 *** Ghidorah has quit IRC 2011-01-07T10:47:15 *** Christinger has joined #evergreen 2011-01-07T10:49:19 I'm trying to get Evergreen to run on Amazon EC2, using Ubuntu 10.04. I can get the Web interface up, but nothing else and I'm getting the following error message: Exception: OpenSRF::EX::Session 2011-01-07T14:09:40 Net::Server::PreFork 2011-01-07T10:49:19 /usr/share/perl5/Net/Server/PreFork.pm:271 Session Error: 2011-01-07T10:49:19 opensrf@private.localhost/client_at_ip-10-195-51-95.ec2.internal_22179 IS NOT CONNECTED TO THE NETWORK!!! 2011-01-07T10:49:19 How do I address this problem? 2011-01-07T10:50:53 Is it a matter of resetting /etc/hosts to point to the elastic IPs? 2011-01-07T10:56:31 Christinger: It's possible. I've never run Evergreen on the cloud before. 2011-01-07T10:56:45 Someone else might know more exactly what needs to happen. 2011-01-07T10:58:29 Christinger: Same boat as bshum never tried the 'cloud' but I've been planning on trying EG out on the cloud so your progress or lack of is useful. Any logs you can look at? 2011-01-07T10:59:23 That error sounds like OpenSRF isn't started correctly. MIght want to check to make sure that it's running right or if a restart of things might help. 2011-01-07T11:01:17 *** moodaepo has quit IRC 2011-01-07T11:02:30 Yes, I've got lots of logs. 2011-01-07T11:02:35 *** parsr has joined #evergreen 2011-01-07T11:03:46 I'm new to Evergreen and OpenSRF, but I think OpenSRF started properly. Or at least it was running with errors before I installed Evergreen. 2011-01-07T11:04:30 Christinger: a common problem is that when people install Evergreen, they forget to change opensrf.xml and opensrf_core.xml appropriately 2011-01-07T11:07:05 some local developer asked if there is url exposed that returns the raw xml of a search request? Is this available via the opensearch api or other method? I'm not sure why they'd need that when evergreen has such hackable urls for search parameters. 2011-01-07T11:07:31 I did change them, so I don't think that's the issue. It looks to me like it has something to do with the internal addressing for an AWS instance and the elastic IP or external address for the same virtual machine. But I'm confused, because the Web interface comes up on the public Web. 2011-01-07T11:11:14 I restarted OpenSRF and the autogen process. Now, it's telling me that the settings server is going to sleep. What does this mean? It was trying to update Fieldmapper before it went to sleep. 2011-01-07T11:15:11 parsr: a good way to approach your/their question is to ask what they think they are trying to achieve. There's usually another way to do it. 2011-01-07T11:19:36 Christinger: we've seen weird networking issues on some VM systems in the psat 2011-01-07T11:19:45 Dyrcona: agreed. I know what they're trying to do, but I don't know why they're looking at this option since I believe we've already given them good options (vis a vis our recommended options to crawl the catalogue). Instead, they're looking at some angle to federate their search tool with our indexes I believe. I'll follow up with them (but painful since they have "a hammer" toolbox and... 2011-01-07T11:19:47 ...want to make everything a "nail" imho) . I believe we gave them some good options but they're looking at other angles. 2011-01-07T11:20:51 Any sense of how to get the network functioning? Do you thinking change the settings in /etc/hosts to reflect the elastic IP from AWS will make any difference? 2011-01-07T11:21:04 parsr: SRU isn't good enough for them, eh? Sound like a case of not-invented-here. :) 2011-01-07T11:23:20 Christinger: are you using localhost (and osrf_ctl.sh with the "-l" flag) currently? 2011-01-07T11:23:49 I am. 2011-01-07T11:25:29 Hmm. Might be worth a shot. Before that, though, let's try starting the services individually 2011-01-07T11:25:38 osrf_ctl.sh -l -a stop_all 2011-01-07T11:25:47 osrf_ctl.sh -l -a start_router 2011-01-07T11:26:02 Then confirm that you've got two "OpenSRF Router" processes running 2011-01-07T11:26:06 then 2011-01-07T11:26:15 osrf_ctl.sh -l -a start_perl 2011-01-07T11:26:36 and confirm that you've got some "opensrf.settings" processes running 2011-01-07T11:26:39 Dyrcona: yes.. Originally, they wanted me to write the database to a single XML document so that their crawler could index the "document" (oh, man!). I'm trying to steer them towards the harvest/crawl model using supercat / sru / opensearch etc. (our original plan with the PREVIOUS developer), but new crew have some weird ideas about finishing the job. 2011-01-07T11:26:44 Thanks. I'll give it a try. 2011-01-07T11:26:44 and then 2011-01-07T11:26:49 osrf_ctl.sh -l -a start_c 2011-01-07T11:27:01 and confirm that you've got some "open-ils.cstore" processes running 2011-01-07T11:27:18 just in case there's a race condition that's super-exacerbated by cloudiness :) 2011-01-07T11:28:18 parsr: huh, so they need the XML instead of the JSON from gateway requests? 2011-01-07T11:28:51 parsr: They didn't expect your server to serve them this XML document on demand did they? Depending on the size of your database it could be well over 20GB in XML! 2011-01-07T11:29:02 parsr: something like http://evergreen-ils.org/dokuwiki/doku.php?id=osrfhttp:opensrf_gateway perhaps 2011-01-07T11:29:30 parsr: It does sound like all they have is a XML hammer. 2011-01-07T11:29:32 parsr: are they looking to just get record IDs back that they can then request individually? 2011-01-07T11:30:11 * dbs still isn't clear what exactly is desired 2011-01-07T11:30:30 dbs: I'm not sure the developer is, either. :) 2011-01-07T11:30:46 You could run the marc_export script on a regular basis and drop the MARCXML in a web-accessible location if they really want the whole damn thing 2011-01-07T11:31:13 I just added a "--all" option to enable fools to export the complete set of records, rather than having to input the record IDs individually :) 2011-01-07T11:32:29 Followed the instructions above, everything started, but now apache won't restart. 2011-01-07T11:32:44 dbs: record id's option is where I was hoping to move them towards (oai_pmh style direction) - using id's to manage the changes / updates /deletions. All pretty straight forward imho. But no... 2011-01-07T11:33:04 Christinger: how much RAM does your cloud instance have? 2011-01-07T11:33:25 2 GB. 2011-01-07T11:33:28 I'm wondering if the OOM killer is getting active 2011-01-07T11:33:34 oh, that should be fine 2011-01-07T11:33:42 alright, let's not worry about apache 2011-01-07T11:33:59 autogen.sh doesn't use apache, so let's see what happens 2011-01-07T11:34:52 christinger: 2 GB of RAM is OK. I have that on a development machine that runs everything. 2011-01-07T11:36:25 And I said it should be fine because I run VMs with 1.5 GB of RAM. 2011-01-07T11:36:34 When I run autogen, this is what I get: Updating Evergreen organization tree and IDL using '/openils/conf/opensrf_core.xml'Updating fieldmapper No Response from settings server...going to sleep 2011-01-07T11:37:47 Christinger: okay, something else to try 2011-01-07T11:38:34 update the timeout values in /usr/local/share/perl5/OpenSRF/Utils/SettingsClient.pm to 300 instead of 10 2011-01-07T11:38:46 (you might need to adjust the path for your distro) 2011-01-07T11:39:03 then restart the Perl services: osrf_ctl.sh -l -a restart_perl 2011-01-07T11:39:43 Christinger: watch for errors when you do the latter. 2011-01-07T11:40:27 *** atz__ has joined #evergreen 2011-01-07T11:42:46 *** atz_ has quit IRC 2011-01-07T11:43:39 Okay. I will try that. 2011-01-07T11:44:23 Dmagick: Poking at your recent blog post about zip codes and trying to get it working. Once we follow the steps you've outlined, does this automatically appear for the patron registration? Or is it used somewhere else? 2011-01-07T11:45:30 Oooh, search 2011-01-07T11:45:32 Nvm 2011-01-07T11:45:46 bshum: :) 2011-01-07T11:45:59 Had to look at the openils portion to finally click that. 2011-01-07T11:46:00 Too bad 2011-01-07T11:46:02 :) 2011-01-07T11:46:14 It would have been cooler if something like that could populate during patron registration. 2011-01-07T11:46:29 *** brian_f has joined #evergreen 2011-01-07T11:48:24 Oh well. Fun exercise all the same. 2011-01-07T11:48:32 Dmagick: Thanks for the knowledge share! 2011-01-07T11:49:12 *** youdonotexist has joined #evergreen 2011-01-07T11:49:41 bshum: no problem. I'd like that, too. Worth an enhancement request, anyway. 2011-01-07T11:49:57 (if it's not already in...) 2011-01-07T11:50:14 I'm starting a list of "things" the 2.0 patron editor needs that we had in 1.6 and before. 2011-01-07T11:50:26 Or potential bugs 2011-01-07T11:50:58 bshum++ 2011-01-07T11:50:58 Changing the timeout value allowed fieldmapper to update, but still not connecting. Will continue to work on this later. Thanks! 2011-01-07T11:51:00 The thing that's stumped me recently has been how the profile dropdown lists every profile in the system instead of showing only those you can actually register. 2011-01-07T11:51:01 *** Christinger has quit IRC 2011-01-07T11:51:35 So you can find yourself in the awkward position of trying to register someone to a usergroup that you don't have permission to register for. 2011-01-07T11:51:38 Hmm. Whatever call is being made should probably be made perm-aware 2011-01-07T11:51:52 And it fails with an unhappy message. 2011-01-07T11:52:26 bshum: please do bug that 2011-01-07T11:53:08 dbs: Will do. 2011-01-07T11:54:08 Christinger: I've got Evergreen running on EC2, and was able to leave the etc/hosts just as the instructions said... 2011-01-07T11:55:02 *** moodaepo has joined #evergreen 2011-01-07T12:00:16 @later tell Christinger to reach brian_f or bfeifarek at q . com for more things to try to get Evergreen running on Amazon 2011-01-07T12:00:16 brian_f: The operation succeeded. 2011-01-07T12:03:09 @later tell Christinger to make sure OpenSRF *.pid files are cleared out of /openils/var/run directory before starting OpenSRF. ejabberd also can get messed up when Amazon's IP address changes-- I had to wrestle with that 2011-01-07T12:03:09 brian_f: The operation succeeded. 2011-01-07T12:10:06 brian_f: any EC2-specific details that you can provide would be awesome - do you have a wiki account? 2011-01-07T12:13:36 bshum and Dmagick: I'm confused. Where does the zip code info show up? I had been thinking it was in patron registration. 2011-01-07T12:14:10 kmlussier: Me too unfortunately 2011-01-07T12:14:20 If I read things correctly, and I can't confirm yet 2011-01-07T12:14:30 Is that it would populate search strings in the patron search screen 2011-01-07T12:14:40 So that if you keyed in a zip code there 2011-01-07T12:14:43 No, it should be in the patron editor 2011-01-07T12:14:54 If it's supposed to be there, then there's something wrong :) 2011-01-07T12:14:58 Let me look at this again and make sure I didn't miss something. 2011-01-07T12:15:05 bshum: kmlussier: the ZIP code information file resides on the app server 2011-01-07T12:15:07 as Dmagick writes: "there is a way to automate some address information for a patron when they give you a zipcode during an account creation or update" 2011-01-07T12:15:14 check in opensrf.xml for a zips file 2011-01-07T12:15:16 Huh 2011-01-07T12:15:35 gmcharlt: We got that part. 2011-01-07T12:15:39 s/zips file/setting the points to a zipcode info file/ 2011-01-07T12:15:48 * dbs hasn't tried using that with Canadian postal codes, so doesn't know whether 1) that works for canucks or 2) whether it's working in the 2.0 editor 2011-01-07T12:15:53 *** AaronZ has joined #evergreen 2011-01-07T12:16:05 I should point out that my tests were on a 1.6.1.4 server. 2011-01-07T12:16:07 Yes. That path has to be correct and pointed to your zips, with a restart of opensrf svcs to accommodate. 2011-01-07T12:16:10 not 2.0 2011-01-07T12:16:20 dbs: I do have a wiki account. I'll put some notes up when I get some free moments 2011-01-07T12:16:25 brian_f++ 2011-01-07T12:16:57 dbs: now that I think of it, I had to do some zip work up there, and the hash works. There was a slight glitch, but I'll have to test and find it again. 2011-01-07T12:17:04 Meaning CA 2011-01-07T12:17:17 At first, I thought it was because of changes we had made to our editors, so I went back to defaults. 2011-01-07T12:18:01 bshum: that's what I used for this post... 1.6.1.4 2011-01-07T12:18:16 Dmagick: Hmm, then maybe I really screwed something else up :) 2011-01-07T12:18:33 I did fiddle with the zips.txt file that I created for the occasion, maybe there's something wrong with the way I styled it. 2011-01-07T12:19:00 kmlussier: back to you, if things work as intended, it *should* apply to patron registration. Guess I mucked it up. 2011-01-07T12:19:59 yes, it's the same grouping of code if I recall correctly that grabs the default state for your drivers license, so ptron registration... 2011-01-07T12:20:14 Good to know. That's what I had been hoping for. :-) 2011-01-07T12:20:24 kmlussier: Me too, I just gave up too quickly :( 2011-01-07T12:20:28 Dmagick: do you have an RSS feed that provides the complete content rather than just a summary? 2011-01-07T12:22:35 dbs: just fixed it. :) 2011-01-07T12:23:14 Dmagick++ 2011-01-07T12:26:01 *** b_bonner has joined #evergreen 2011-01-07T12:26:34 So, in playing with trunk, we have found that admin can see org unit trees in circ policies, and nobody else we have in the system can. 2011-01-07T12:27:57 This includes those with "everything at consortium depth" 2011-01-07T12:36:19 *** alxp has quit IRC 2011-01-07T12:40:09 * tsbere dislikes the way permissions work right now 2011-01-07T12:41:02 tsbere: probably a missing explicit permission 2011-01-07T12:42:02 tsbere: you're free to dislike things, as long as you understand them and can propose alternatives - which you're good at doing 2011-01-07T12:42:16 *** parsr has quit IRC 2011-01-07T12:42:18 this is probably also a result of almost all testing being done with the 'admin' user 2011-01-07T12:42:20 dbs: I have an alternative that needs fleshing out. And implementing. 2011-01-07T12:43:07 * tsbere feels that being able to assign differing permissions per org unit is needed, as is being able to deny (not just remove) a permission 2011-01-07T12:46:38 Sure, sounds like a decent refinement of the current approach. So what you dislike is the limits of the current implementation 2011-01-07T12:47:19 Well, that, and the hoops you have to jump through. 2011-01-07T12:48:56 My current "fleshing out" problem is "how do I NOT break the old permissions and way of doing things in the process?" 2011-01-07T12:50:34 legacy circ scripts vs in-db circ? ;-) 2011-01-07T12:55:08 *** pmplett has joined #evergreen 2011-01-07T12:55:38 jeff: Technically, I keep trying to add to the functionality set of both. 2011-01-07T12:55:50 So, bad example. :P 2011-01-07T12:56:16 oh, i just mean that it's one possible example of parallel architectures with one being destined for deprecation 2011-01-07T12:56:36 *** AaronZ has quit IRC 2011-01-07T12:56:37 i wasn't trying to imply that something in in-db had broken circ scripts 2011-01-07T12:56:49 Still a bad example. I want the old method to basically work as the underpinning for the new method, while the new method adds a pile of fun new things. 2011-01-07T12:57:08 Thus, the new method is more of a "new pieces overlay" of the old one 2011-01-07T12:57:28 * tsbere just can't figure out the best way to DO that 2011-01-07T12:59:13 step 1: write tests for the current permissions :) 2011-01-07T13:00:07 Question on potentially denying permissions: Should a "deny" applied to the "Everything" permission just stop the everything permission (such as from a group) or should it literally deny *everything* permissions-wise? 2011-01-07T13:04:20 jeff++ 2011-01-07T13:05:34 jeff: I just posted the promised "let's brainstorm stuff what we should think about after 2.0" email to the list; jumping in with "tests, dammit!" would be a good addition 2011-01-07T13:06:12 *** rickd_ has quit IRC 2011-01-07T13:06:38 * tsbere wonders if he should change his terminology from "deny" to "revoke" 2011-01-07T13:07:52 *** kmlussier has quit IRC 2011-01-07T13:13:36 *** AaronZ has joined #evergreen 2011-01-07T13:14:16 *** b_bonner has quit IRC 2011-01-07T13:15:04 *** kmlussier has joined #evergreen 2011-01-07T13:17:40 *** b_bonner has joined #evergreen 2011-01-07T13:26:08 *** rsoulliere has joined #evergreen 2011-01-07T13:35:54 Dmagick: So when you enter a zip code in the patron editor, it automatically fills the City/State/County entries? (based on your example) 2011-01-07T13:36:22 we have that too 2011-01-07T13:36:26 talk to atheos 2011-01-07T13:36:35 he will probably share his script for generating a zips.txt file 2011-01-07T13:36:37 Were there anything changes that need to be made to the stock ue_config.js for that? I'm so confused :( 2011-01-07T13:36:47 bshum: dont think so 2011-01-07T13:36:56 let me try to reach atheos and see if i can share 2011-01-07T13:36:58 im sure he wont care 2011-01-07T13:36:58 I have a sample zips.txt that I rewrote with CT zips 2011-01-07T13:37:07 But maybe I formatted it wrong? 2011-01-07T13:37:09 did you edit opensrf.xml 2011-01-07T13:37:16 Right, to remove the comments from the zip file 2011-01-07T13:37:59 2011-01-07T13:37:59 /openils/var/data/zips.txt 2011-01-07T13:37:59 2011-01-07T13:37:59 2011-01-07T13:38:08 in opensrf.xml 2011-01-07T13:38:17 Yep, that's the one. 2011-01-07T13:38:28 Removed the comment around the zips_file portion 2011-01-07T13:38:33 here is a line from my file 2011-01-07T13:38:47 |IN|Alexandria|46001|1||| 2011-01-07T13:39:06 That might be the problem then 2011-01-07T13:39:16 ||CT|Avon|06001|1||Hartford|| 2011-01-07T13:39:20 Mine is shifted one over 2011-01-07T13:39:24 perhaps! 2011-01-07T13:39:32 According to the code 2011-01-07T13:39:45 I think the extra | is a problem 2011-01-07T13:39:46 Means the image capture from Dmagick is of slightly. Or I did it wrong... hmm 2011-01-07T13:39:57 *off 2011-01-07T13:40:06 Or Dmagick did it wrong. 2011-01-07T13:40:10 Yeah. Mine came from GPLS, and they use the first block. Could be customized. Try removing the first pipe and it would then work. 2011-01-07T13:40:10 http://dmagick.files.wordpress.com/2011/01/zips.png 2011-01-07T13:40:11 { print "|$state|$city|$zip|1|||\n"; } 2011-01-07T13:40:14 Yeah his image is a bit off. 2011-01-07T13:40:20 thats how atheos has his formatted 2011-01-07T13:40:41 Thanks for catching it. I'll put the caveat in... 2011-01-07T13:41:02 The code block (http://dmagick.files.wordpress.com/2011/01/data-code.png) implies that the first | shouldn't be there in the zips.png file 2011-01-07T13:41:14 I don't even know where my script is located mrpeters-isl - I'm guessing you do though 2011-01-07T13:41:20 i do :) 2011-01-07T13:41:30 cool, if someone can use it, shar eit 2011-01-07T13:41:34 share it even 2011-01-07T13:41:50 i dont knwo how to works so you will have to fill that in :) 2011-01-07T13:42:09 http://git.evergreen.lib.in.us/git/?p=evergreen.git/.git;a=blob_plain;f=Evergreen-ILS-1.6.0.0/zipcode/generate_zips.pl;h=d42a75386cf4192a059e39dc867c66da05565294;hb=6032d7a086ff96b23d295fbe9fea218ed5811516 2011-01-07T13:42:15 *** Melissa has left #evergreen 2011-01-07T13:42:26 hey look! more git users! 2011-01-07T13:42:32 haha 2011-01-07T13:42:38 we got git 2011-01-07T13:42:45 i'm a major git novice 2011-01-07T13:42:54 now we need tiny url 2011-01-07T13:43:09 phasefx: yeah probably would have been better 2011-01-07T13:43:18 phasefx: tinygit? 2011-01-07T13:43:19 http://git.evergreen.lib.in.us doesn't load for me -- maybe internal-only? 2011-01-07T13:43:28 there's a supy plugin for that, but I doubt pinesol will ever get such love :) 2011-01-07T13:43:29 jeff: crap, i may not have the firewall open to it 2011-01-07T13:43:42 let me jsut paste it 2011-01-07T13:44:01 http://pastie.org/1437950 2011-01-07T13:44:39 atheos: maybe you can write a README for this? 2011-01-07T13:44:41 dbs: with my accent, we could call it done-got 2011-01-07T13:44:44 mrpeters-isl that should run as is, so long as the zip range is correct 2011-01-07T13:44:55 i think it relies on some perl modules that aren't stock though? 2011-01-07T13:44:57 the api code is mine, but it should work fine 2011-01-07T13:45:19 Can't locate Geo/Coder/Google.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.10.0 /usr/local/share/perl/5.10.0 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl .) at ./generate_zips.pl line 3. 2011-01-07T13:45:19 BEGIN failed--compilation aborted at ./generate_zips.pl line 3. 2011-01-07T13:45:48 perl -MCPAN -e shell; install Geo::Coder::Google ? 2011-01-07T13:46:15 http://search.cpan.org/~miyagawa/Geo-Coder-Google-0.06/lib/Geo/Coder/Google.pm 2011-01-07T13:46:16 or "cpan Geo::Coder::Google" :) 2011-01-07T13:46:37 bshum: fixed on the post now. Thanks for digging. I didn't realize at the time that the two were different. 2011-01-07T13:46:39 ++dbs for getting straight to the point 2011-01-07T13:47:22 cool. that seems to be the only one 2011-01-07T13:47:50 people love atheos for this feature 2011-01-07T13:48:29 they do? cool, that script was a good 15 minute investment then 2011-01-07T13:49:40 haha man, if ever an address is missing they are quick to let me know! 2011-01-07T13:49:55 they rely on it like crazy 2011-01-07T13:51:33 *** bshum has quit IRC 2011-01-07T13:52:38 atheos++ 2011-01-07T13:53:25 atheos: nice little script! 2011-01-07T13:53:59 thanks guys, glad I can be useful 2011-01-07T13:55:58 atheos: willing to throw a GPL v2-or-later header on it and submit it with a DCO for Open-ILS/src/extras? 2011-01-07T13:56:29 yes, I'll do that. mrpeters-isl, which box is this script located on? 2011-01-07T13:56:52 all of them 2011-01-07T13:56:58 in /openils/var/data 2011-01-07T13:57:06 1) might want to remove your API key, 2) isn't this use forbidden by the google TOS? 2011-01-07T13:58:23 jeff, yes, I'd imagine that to be the case. 2011-01-07T13:58:37 jeff: ah, API keys, c'mon, librarywebchic shared hers yesterday, it's the trend 2011-01-07T13:59:31 A script that violates the terms of service of the service it uses won't be welcome, so that would be good to verify. 2011-01-07T13:59:35 seems likely, sadly 2011-01-07T13:59:47 bummer 2011-01-07T14:00:16 dbs jeff, it wasn't intentionally left in there, the script hasn't been sanitized for public consumption yet. 2011-01-07T14:00:25 that's our production script :) 2011-01-07T14:00:38 atheos: *nod* -- you didn't share before sanitizing, that was all mrpeters-isl's fault ;-) 2011-01-07T14:01:08 mrpeters-isl: kidding :) 2011-01-07T14:01:13 all your fault mrpeters-isl 2011-01-07T14:01:19 haha always is 2011-01-07T14:01:30 * dbs looks at http://www.populardata.com/postalcode_database.html 2011-01-07T14:02:49 there are lots of options out there -- i've used a few different ones. USPS has some nice data products that you can obtain for a fee, and I've looked at what it would take to use them. 2011-01-07T14:03:02 dbs, you could probably use the usps api to easily retrieve this info 2011-01-07T14:03:37 We run into situations where we have a renumbering take place, and I'd love to be able to NOT bother every patron "do you have one of the new zip codes?" 2011-01-07T14:03:51 atheos: yeah, that download I looked at doesn't provide streets, but hey... fair enough 2011-01-07T14:04:20 to be fair, 90% of our data will be bulk-loaded anyway so it's not really an issue for us 2011-01-07T14:04:29 *** Melissa has joined #evergreen 2011-01-07T14:05:08 most of the collections i've seen have similar disclaimers: "This database is a few years old and I've had reports of a missing postal code now and then." (from the populardata url) 2011-01-07T14:07:32 * dbs wonders if a live call to Google geolocation API when the postal code is entered in the staff client would meet the ToS; no idea what the ToS is 2011-01-07T14:09:13 Most of google's map TOS says "if you aren't using this to put a dot on a google map and ONLY to put a dot on a google map then it isn't allowed" 2011-01-07T14:10:11 Ah, then definitely out 2011-01-07T14:10:37 caching and archiving are also listed as "only to facilitate later putting on a google map", I think. 2011-01-07T14:11:13 yep, makes sense 2011-01-07T14:11:21 similar to amazon 2011-01-07T14:11:52 I looked up using google's API in the past for some of this. 2011-01-07T14:13:54 http://pastie.org/1438039 2011-01-07T14:14:06 dbs - do I send this DCO to the mailing list? 2011-01-07T14:15:21 atheos: Your script technically violates google's TOS, I don't think it would be good to check it into the official repos 2011-01-07T14:16:39 well, that's a buzzkill 2011-01-07T14:17:01 * tsbere dislikes the TOS in this case 2011-01-07T14:17:32 atheos: sorry, that's what I was saying at "A script that violates the terms of service of the service it uses won't be welcome, so that would be good to verify" 2011-01-07T14:17:35 atheos: PM plz 2011-01-07T14:17:47 sorry about that atheos :( 2011-01-07T14:18:20 so, are you guys saying that this violates googles TOS too? http://search.cpan.org/~miyagawa/Geo-Coder-Google-0.06/lib/Geo/Coder/Google.pm 2011-01-07T14:22:45 atheos: not necessarily. If you use it to create a Google map that complies with the TOS, then it's just a means to an end - but I am not a lawyer 2011-01-07T14:23:06 The TOS says things like "no grabbing bulk information", "no pre-fetch, cache, or storing of content except to improve performance of your API-using code", "no using it with anything that isn't being shown in a web browser", "no using it without a google map (unless you have written permission from google to do so)" 2011-01-07T14:23:22 * dbs looks at http://code.google.com/apis/maps/terms.html 2011-01-07T14:23:25 I'm reading through the TOS right now, but <-- not a lawyer either 2011-01-07T14:24:04 heh, I'll request written permission from google. 2011-01-07T14:24:37 You would still be in violation of bulk operations and lack of web browser, and caching/storing for that matter. 2011-01-07T14:25:13 staff client is sorta a web-browser :P 2011-01-07T14:25:23 but caching/storing, you got me. 2011-01-07T14:26:00 the example "you must not use geocodes obtained through the Service except in conjunction with a Google map" appears to be fairly clear 2011-01-07T14:27:38 what it says, and the spirit in which it's written might be relevant (which is why I'm going to contact them). They may mean "you cannot use google API for your commerical GPS device or product that competes with google maps" 2011-01-07T14:28:10 worse case, they just say no 2011-01-07T14:28:15 ^^ this 2011-01-07T14:28:16 atheos: definitely worth a shot! 2011-01-07T14:28:19 some portions of their TOS are there because of restrictions placed by who THEY get the data from. 2011-01-07T14:28:51 i.e., "We here at GeoData R Us give you Google these rights but you have to keep people from using it for general geocoding purposes" 2011-01-07T14:28:55 sure, Tele Atlas Content et al 2011-01-07T14:29:00 *** bshum has joined #evergreen 2011-01-07T14:29:12 *** brian_f has quit IRC 2011-01-07T14:29:19 * tsbere thinks it may be better to get permission from the USPS for US zip codes 2011-01-07T14:29:24 jeff, the very small amount of information I'm using google api to mine should really be public information (IMO) 2011-01-07T14:29:29 "we don't want our customer to become a competitor too quickly" 2011-01-07T14:29:37 i say we just tell everyoen to email atheos and make him run the script for every state :) 2011-01-07T14:29:40 there's nothing proprietary there 2011-01-07T14:29:52 mrpeters-isl: Tried the new formatting you guys suggested and it's beautiful :) 2011-01-07T14:29:57 cool! 2011-01-07T14:30:05 Didn't get to poke at the script though, our internet went down 2011-01-07T14:30:08 thank atheos, not me...i just noticed your extra delimter 2011-01-07T14:30:11 I blame it on the insane snow headed our way 2011-01-07T14:30:18 * dbs buys his zipcodes from atheos-bulk-zips.com 2011-01-07T14:30:22 *** Melissa has left #evergreen 2011-01-07T14:30:36 atheos++ mrpeters-isl++ 2011-01-07T14:30:41 yea, there's probably a better way to do this. I'll go brush up on the usps API. I gotta study it for an up-coming project anyway 2011-01-07T14:31:05 atheos: Take note, the USPS api requires you get permission from USPS before you can call it. 2011-01-07T14:31:09 * tsbere was already looking into it 2011-01-07T14:31:15 maybe there is a way, as a community, to get Google or USPS or someone interested in helping to get this feature right into Evergreen 2011-01-07T14:31:30 especially USPS, since it might help with addresses being more accurate 2011-01-07T14:31:33 thanks tsbere - note taken 2011-01-07T14:32:25 Canada Post is mostly in the business of selling their postal code data to ad mail firms, it seems 2011-01-07T14:32:37 http://www.canadapost.ca/tools/pg/customerguides/CGintro-e.asp#1379264 2011-01-07T14:32:37 *** pmplett has quit IRC 2011-01-07T14:32:41 mrpeters-isl maybe we can find out what bureaucratic department is responsible for maintain this information and we bug the crap out of them for the info 2011-01-07T14:32:43 thanks, canada post! 2011-01-07T14:33:06 usps has "you can buy the information on DVD, or we have web APIs, but you have to ask us before doing ZIP based lookups" 2011-01-07T14:34:04 They don't require you ask before doing delivery info, rate calcs, shipping labels, or carrier pickup requests, though. 2011-01-07T14:34:26 dbs: I applied the zips.txt file to our 2.0 rc1 server and it's populating the 2.0 patron editor very nicely too. 2011-01-07T14:36:29 bshum++ # good to know 2011-01-07T14:40:40 *** rickd_ has joined #evergreen 2011-01-07T14:42:00 Hmm, call for conference proposals closes in one week. I'm thinking about proposing a talk on Authorities in 2.0, but would be willing to talk about whatever people think I should talk about. 2011-01-07T14:42:10 Consider this a call for a call for proposals for dbs :) 2011-01-07T14:43:28 *** mrpeters-isl has quit IRC 2011-01-07T14:43:36 *** mrpeters-isl has joined #evergreen 2011-01-07T14:43:59 * Dyrcona will not be going this year to allow others to attend the conference. 2011-01-07T14:45:30 Dyrcona: dang - you could have done a talk about speeding up migration import time 2011-01-07T14:45:45 He will be too busy trying to ensure migration happens, I think. 2011-01-07T14:45:51 yep. 2011-01-07T14:46:06 The conference is a bit close to our migration date. 2011-01-07T14:46:21 hey, we can use perl + curl + http://zip4.usps.com/zip4/citytown_zip.jsp = zips.txt :) 2011-01-07T14:46:24 heh, that's a good reason 2011-01-07T14:46:38 dbs: I was hoping to stay for post-conference activities as you were talking on the mailing list, but our CLA (yes, OURS) conference is the Monday right after. 2011-01-07T14:46:55 bshum: boooo 2011-01-07T14:48:01 to speed up loads though, the magic bullet is pgtune. 2011-01-07T14:48:29 and using JDBC batch inserts. it seems to be faster than COPY. 2011-01-07T14:50:08 that last bit still doesn't make sense to me :) 2011-01-07T14:52:01 dbs: Could be that with the batch inserts I'm doing everything from Java with threads. When using COPY, I was using processes not spawned from Java. 2011-01-07T14:52:45 ah, COPY = serialized vs. JDBC = multithreaded, ergo PostgreSQL can use parallel processes 2011-01-07T14:52:48 that makes sense 2011-01-07T14:53:24 yes. in both cases, i'm running 7 simultaneous batches or copies. 2011-01-07T14:53:26 I split our load into multiple parts and parallelized it that way 2011-01-07T14:53:41 so, really, parallelization is the key 2011-01-07T14:54:02 the technology to accomplish it is left up to the discretion of the implementer :) 2011-01-07T14:55:05 i also had issues with SIGPIPE failures when parallelizing the COPY commands from C. 2011-01-07T14:56:49 *** bshum has quit IRC 2011-01-07T15:05:51 *** AaronZ has quit IRC 2011-01-07T15:07:18 *** cbandito has quit IRC 2011-01-07T15:14:26 *** rsoulliere has quit IRC 2011-01-07T15:17:00 *** cbandito has joined #evergreen 2011-01-07T15:28:44 * tsbere discovers permission checking code in the C code, and isn't sure how he wants to handle that 2011-01-07T15:28:59 *** brian_f has joined #evergreen 2011-01-07T15:31:07 *** Dmagick has quit IRC 2011-01-07T15:41:54 *** pmplett has joined #evergreen 2011-01-07T15:44:48 mmm, official canadian postal codes available for $4,000 / year at http://www.canadapost.ca/cpo/mc/business/productsservices/atoz/postalcodeproducts.jsf 2011-01-07T15:45:57 *** pmplett_ has joined #evergreen 2011-01-07T15:46:22 *** AaronZ has joined #evergreen 2011-01-07T15:48:26 *** pmplett_ has left #evergreen 2011-01-07T15:52:44 * tsbere finds the USPS docs lacking in a number of places. Such as not using file:// paths. 2011-01-07T16:02:55 *** b_bonner has quit IRC 2011-01-07T16:13:11 *** rjackson-isl has quit IRC 2011-01-07T16:40:29 *** r123 has left #evergreen 2011-01-07T17:13:21 *** AaronZ has quit IRC 2011-01-07T17:19:58 *** cbandito is now known as dave-esi-away 2011-01-07T17:23:10 *** kmlussier has quit IRC 2011-01-07T17:27:18 *** Dmagick-home has quit IRC 2011-01-07T17:30:59 *** Dyrcona has quit IRC 2011-01-07T17:51:34 *** jenny has left #evergreen 2011-01-07T19:27:54 *** brian_f has quit IRC 2011-01-07T19:34:30 *** Christinger has joined #evergreen 2011-01-07T19:36:03 I have Evergreen partly running on an Amazon EC2 instance running Ubuntu 10.04. When I run the autogen script, I get the following message: Updating Evergreen organization tree and IDL using '/openils/conf/opensrf_core.xml' 2011-01-07T19:36:03 Updating fieldmapper 2011-01-07T19:36:04 Updating web_fieldmapper 2011-01-07T19:36:04 Updating OrgTree 2011-01-07T19:36:04 removing OrgTree from the cache for locale hy-AM... 2011-01-07T19:36:05 removing OrgTree from the cache for locale cs-CZ... 2011-01-07T19:36:07 Attempting to build a client session as a server Session ID [1294446730.95809371.206989825217], remote_id 2011-01-07T19:36:12 [opensrf@private.localhost/opensrf.settings_drone_at_localhost_19354] at /usr/local/share/perl/5.10.1/OpenSRF/ 2011-01-07T19:36:15 AppSession.pm line 98. 2011-01-07T19:36:17 removing OrgTree from the cache for locale en-CA... 2011-01-07T19:36:19 removing OrgTree from the cache for locale en-US... 2011-01-07T19:36:21 removing OrgTree from the cache for locale fr-CA... 2011-01-07T19:36:23 removing OrgTree from the cache for locale ru-RU... 2011-01-07T19:36:25 Updating OrgTree HTML 2011-01-07T19:36:27 Attempting to build a client session as a server Session ID [1294446758.70344836.86739813378], remote_id 2011-01-07T19:36:30 [opensrf@private.localhost/opensrf.settings_drone_at_localhost_19355] at /usr/local/share/perl/5.10.1/OpenSRF/ 2011-01-07T19:36:33 AppSession.pm line 98. 2011-01-07T19:36:35 Exception: OpenSRF::EX::Session 2011-01-08T00:32:54 OpenSRF::Transport /usr/local/share/perl/5.10.1/OpenSRF/ 2011-01-07T19:36:38 Transport.pm:120 Session Error: Transport::handler(): No AppSession object returned from server_build() 2011-01-07T19:36:42 How do I correct the problem that's causing this error? 2011-01-07T19:38:17 *** pmplett has quit IRC 2011-01-07T19:49:14 Christinger: I'm not the best one to give you troubleshooting advice on that particular error, but have you successfully used srfsh and issued a successful login command? 2011-01-07T19:49:39 On another topic, is there any place where there's troubleshooting advice on the SIPServer? 2011-01-07T19:52:19 I'm apparently getting it to start but it seems like it's not staying around--at least none of the connection tests work. 2011-01-07T19:53:21 Ah, there's no such process as the PID file contains--so it's not running, clearly. 2011-01-07T19:55:40 srfsh login worked, but autogen has to run without errors to fully outfit the Web server. I can run a search from the Web interface, but I can't use advanced search or get a client to log in. 2011-01-07T20:00:37 christinger: Sorry; that one's not one I've encountered (at least not that I remember). 2011-01-07T20:02:34 I'm seeing some errors in /var/log/SIP_evergreen.log now, so I've got some things to look at. 2011-01-07T20:03:22 christinger: You might take a look at osrfsys.log and see if there's anything useful in there (but it sounds like you might have done that already). 2011-01-07T20:04:34 *** Christinger has quit IRC 2011-01-07T20:13:25 OK. SIP server seems to be happy now. Need to verify something about the directions and then make some mods (got caught by an ambiguity). 2011-01-07T20:15:56 (Well, if not happy, at least responding with error messages I can work with.) 2011-01-07T20:19:51 *** rickd_ has quit IRC 2011-01-07T20:23:15 *** youdonotexist has quit IRC 2011-01-07T20:52:26 *** tpham has joined #evergreen