[09:13] Justin Clark-Casey: hi mic, hi rob [09:13] Robert Adams: morning all [09:13] Rob Willis: Gday [09:13] Doug Osborn: aloha all [09:14] Justin Clark-Casey: hello doug [09:14] Doug Osborn: hey justin :) [09:14] Robert Adams: mic usually disappears about now to refresh his caffene [09:15] Rob Willis: i should do that too [09:15] Fei Yeh is Online [09:16] Mic Bowman: g'morning everyone [09:16] Doug Osborn: aloha Mic [09:16] Robert Adams: morning Mic [09:17] Fei Yeh: mornin' [09:17] Mic Bowman: i will be putting the transcript of our discussion this morning on the wiki [09:17] Mic Bowman: as with others... [09:17] Doug Osborn: Thanks [09:17] Mic Bowman: big things happening in opensim development over the last couple weeks [09:18] Mic Bowman: we are currently testing the head of the dev tree internally at intel [09:18] Mic Bowman: with the expectation that we will upgrade sciencesim regions on 10/26 [09:18] Mic Bowman: with a version we find to be appropriately stable [09:18] Doug Osborn: I am concerned about a mantis report i saw that link distances are "artically restrained" now as of 6.7 [09:19] Doug Osborn: artifically [09:19] Mic Bowman: there are BIG changes in the new code (better texture handling, much better network handling, etc) [09:19] Mic Bowman: doug: what is your concern? i haven't seen that mantis? [09:19] Doug Osborn: i have been trying to find it again..... [09:20] Doug Osborn: it looks like melanie acted on mantis - 0001209: There are currently no constraints on link set size or prim count. [09:20] Mic Bowman: one more note on the expectations for 10/26... as a test case, i'm going to try to include extsim in our distribution at that point [09:20] Mic Bowman: doug: there used to be a config variable that let you set the distance and delay for script exectuion [09:20] Mic Bowman: was that removed? [09:21] Justin Clark-Casey: an externally provided distribution or internally to sciencesim? [09:21] Doug Osborn: i dont know [09:21] Mic Bowman: the sciencesim git repository that we currently provide [09:21] Doug Osborn: the mantis dodint make it sound like that [09:21] Doug Osborn: im still digging through the wiki [09:22] Doug Osborn: but it soundled like a hard distance limit on linksets and a limit on the prim count ( "t prevent griefing" [09:22] Doug Osborn: to prevent griefing) [09:22] Mic Bowman: oh those are new [09:22] Doug Osborn: someone with a "rezzer product submitted it as a bug [09:22] Mic Bowman: is there a config variable? [09:22] Doug Osborn: i havent heard wind of one [09:23] Mic Bowman: well... [09:23] Justin Clark-Casey: mic: do you have a url for that git tree? [09:23] Mic Bowman: i know that we're running HEAD from yesterday internally... and i've had no problems with one of my high prim count regions [09:23] Mic Bowman: justin: just a second [09:23] Justin Clark-Casey: repository, I should say [09:23] Mic Bowman: http://island.sciencesim.com/src/opensim-scisim [09:24] Justin Clark-Casey: thanks [09:24] Mic Bowman: i can have jhurliman check on the limits [09:24] Mic Bowman: the limits are fine so long as they are configurable [09:24] Robert Adams: that git tree is what's running on scisim at the moment [09:25] Robert Adams: the conversion on 10/26 will mean a protocol bump and the need for all simulators to upgrade [09:25] Doug Osborn: :) [09:25] Mic Bowman: justin: the scisim repository is opensim + some performance patches that are now in HEAD plus some fixes that should be there... [09:26] Justin Clark-Casey: fixes? Unapplied patches in the opensimualator mantis? [09:26] Mic Bowman: rob: in theory... the upgrade should improve your memory issues [09:26] Mic Bowman: justin: yes... [09:27] Mic Bowman: one in particular is about the way chat is handled by the IRC module [09:27] Mic Bowman: it just makes it possible to have personal IRC communicators [09:27] Justin Clark-Casey: so feature rather than bug? [09:27] Justin Clark-Casey accepted your inventory offer. [09:27] Rob Willis: great [09:27] Mic Bowman: features yes [09:28] Mic Bowman: or maybe "change in behavior" [09:28] Rob Willis: im trying to remove things but not seeing any difference [09:28] Rob Willis: this time its under cable beach asset [09:28] Mic Bowman: rob: what asset cache are you using? [09:28] Rob Willis: how can i check? [09:28] Mic Bowman: i've converted all of our regions to use the flotsam disk cache [09:29] Mic Bowman: depending on how you've set up your config files... it should be specified in GridCommon.ini [09:29] Rob Willis: yeah the every second its logging that and cablebeach asset [09:29] Rob Willis: flotsam [09:29] Justin Clark-Casey: mic: you might have to give me that object again. I just tried to rez it here.. and lost it instead. [09:29] Mic Bowman: its a HUD [09:29] Mic Bowman: hang on [09:29] Justin Clark-Casey accepted your inventory offer. [09:29] Justin Clark-Casey: thanks [09:30] Doug Osborn: OK I found it and I dont see linkset size restrictions just a limit on the link distance.... mantis 4228...... [09:30] Mic Bowman: is the linkset size a limitation of the viewer or the simulator? [09:30] Doug Osborn: well is seems to be the simulator now [09:30] Mic Bowman: sorry... is the NUMBER of items in a linkset a limitation of the viewer or the siulator? [09:31] Mic Bowman: there is an optimization we've been considering that would link all he static items in a region together so there would only be one update [09:31] Rob Willis: cable beach assets: failed to fetch asset metadata [09:31] Doug Osborn: i dont know.... I've linked 3K primatives before and at disatances of almost 20-30 meters in OPensim [09:31] Rob Willis: ... [09:31] Mic Bowman: rob: that's a problem with old textures in your j2k cache [09:32] Doug Osborn: the mantis indicates the link distance is now restricted to less than 10M [09:32] Rob Willis: j2k? [09:32] Mic Bowman: it's not a problem... just over reporting and is fixed in HEAD [09:32] Mic Bowman: jpeg2000 [09:32] Mic Bowman: the internal format that SL & opensim use for images [09:32] Rob Willis: ok [09:32] Rob Willis: and that causes a leak? [09:32] Mic Bowman: doug: is there a config ooption? [09:32] Mic Bowman: rob: no [09:33] Rob Willis: so some other problem then.... [09:33] Doug Osborn: I dont know mic I havent had a chance to upgrade to 6.7 [09:33] Mic Bowman: rob: i don't know what's causing your particular leak... it could be client connections (that's the most likely) or a script that's gone crazy [09:34] Mic Bowman: but the CB asset messages occur when the j2k decoder cache tries to clean up an asset that is no longer stored on the client (like dynamic textures or map images) [09:34] Rob Willis: there is another error. waves dont be evil......the scene reused a disposed physactor [09:35] Doug Osborn: Shen and I have already agreed to dedicate the Grid2 server to a cable beach distribution though, but I have a feeling that link distance limit is going to be an issue [09:35] Mic Bowman: if you clean out the j2k decoder cache (can't remember what the name of the directory is... but its obvious) you'll stop getting those messages [09:35] Mic Bowman: rob: that message is generated by ODE (the physics engine)... not sure what causes it but it is usually not a problem [09:36] Mic Bowman: the biggest cause of memory leaks in the simulator (in the version we're running) are client connections [09:36] Rob Willis: it would only be me connecting [09:36] Rob Willis: never see any1 else [09:37] Mic Bowman: ok... [09:37] Mic Bowman: the only way i know to debug that particular leak is to monitor closely the memory that is being used and correlate it with some behavior [09:38] Mic Bowman: justin? [09:38] Justin Clark-Casey: woah, that's trippy [09:38] Rob Willis: oh its a webserver too, but not a busy one [09:39] Mic Bowman: that shouldn't be a problem [09:39] Mic Bowman: justin are you playing with animations? [09:39] Justin Clark-Casey: yes, I'll stop [09:39] Mic Bowman: the frog one was pretty interesting [09:39] Mic Bowman: :-) [09:40] Mic Bowman: rob: if there is any way that you can get a graph of the memory being used over time (at like 10 minute intervals to start with) that would be VERY helpful [09:40] Doug Osborn: i was gonna say mic you been holding out on the sitting animations :) i wanna frog chair too :) [09:40] Rob Willis: sure just let me know what to do [09:40] Mic Bowman: you've seen the graphs we generate from the ganglia code? [09:40] Mic Bowman: http://www.sciencesim.com/status2 [09:41] Mic Bowman: i'm not suggesting you install ganglia... but that's the kind of data we've used to find the memory leaks [09:41] Justin Clark-Casey: /shake [09:42] Rob Willis: ok [09:42] Mic Bowman: take a look at this page... it shows the virtual memory that is allocated with new users... [09:42] Mic Bowman: http://www.sciencesim.com/status2/?c=M2&h=run.kepler&m=cpu_used&r=hour&s=descending&hc=4 [09:42] Mic Bowman: that's the correlation we're looking for to debug your leak [09:43] Kite Hallison: hello [09:43] Mic Bowman: doug: the 10m limit... [09:43] Justin Clark-Casey: Hello Kite [09:43] Mic Bowman: hi kite! [09:43] Justin Clark-Casey: b [09:43] Rob Willis: on windoze? [09:43] Mic Bowman: i'll take a look at some point... that should definitely be configurable [09:44] Mic Bowman whispers: rob: ? [09:44] Rob Willis: ganglia [09:44] Mic Bowman: all of our simulators are running on linux [09:44] Mic Bowman: robert: do you know if there is a ganglia for windows? [09:44] Doug Osborn: yes well I agree... completely [09:45] Robert Adams: no I don't... we got our ganglia by building from sources on Linux [09:45] Mic Bowman: here's the ganglia link: http://ganglia.sourceforge.net/ [09:45] Mic Bowman: you don't necessarily need ganglia [09:45] Mic Bowman: that would certainly be helpful... [09:46] Mic Bowman: but a shell script that wrote the memory size every minute would be good [09:46] Mic Bowman: as a starting point [09:46] Rob Willis: ok [09:46] Rob Willis: so nothing specific, just the opensim process? [09:47] Mic Bowman: the opensim process should be sufficient [09:47] Robert Adams: on windows, perfmon can do some recording of memory usage [09:47] Mic Bowman: true... and you should be able to capture a graph of the memory usage [09:48] Rob Willis: ok [09:48] Mic Bowman: if you can get that... it would really help [09:48] Rob Willis: there isnt any functionality like that in the simulator? [09:48] Mic Bowman: the simulator does not natively have the ability to track memory usage over time [09:49] Mic Bowman: there is a "show stats" command that can show you what it thinks its using [09:49] Rob Willis: ah ok.... [09:49] Mic Bowman: which under mono is much different than what its actually using... [09:49] Mic Bowman: i haven't watched it under windows [09:50] Mic Bowman: any other issues? [09:50] Doug Osborn: not for me [09:50] Mic Bowman: i think jhurliman just started the commits for the scene prioritization patches we've been working on [09:50] Doug Osborn: :) [09:50] Doug Osborn: that stuff will be way cool [09:50] Mic Bowman: that will be goodness... much faster loads and loading the "right" stuff [09:51] Mic Bowman: i'm really excited about the recent opensim changes [09:52] Mic Bowman: getting the configuration details worked out was... *non-trivial* [09:52] Doug Osborn: :) [09:52] Rob Willis: this remote console, would it act like a service? [09:52] Rob Willis: in the new 0.6.7 [09:52] Justin Clark-Casey: a service [09:52] Mic Bowman: rob: i don't have any experience with the remote console... [09:52] Justin Clark-Casey: ? [09:53] Rob Willis: with the console commands [09:53] Rob Willis: like shutdown [09:53] Mic Bowman: as i understand it... you can put the simulator into background execution and use the remote console to interact with it through REST calls [09:53] Mic Bowman: over an http interface [09:54] Mic Bowman: security is through a identity/password that is defined in the opensim.ini configuration file [09:54] Rob Willis: i was looking into how to make a service whereby through programmatical calls i could start/shutdown/restart etc [09:55] Mic Bowman: the shutdown should work... the start probably won't because the simulator itself is the "host" of the remote console [09:55] Rob Willis: ok [09:55] Mic Bowman: part of the reason we moved away from windows for hosting the scisim services is that its much easier to do remote admnistration [09:55] Doug Osborn: shutdown needds to be a call that just sets the simulator to a heartbeat coindition and clrears all the regions related stuff.... [09:56] Rob Willis: ok [09:56] Mic Bowman: justin's on the hot seat [09:56] Justin Clark-Casey: ah, that's toasty [09:56] Doug Osborn: fortunately the FRI clothes and hair are Non-flammable :) [09:56] Mic Bowman: :-) [09:57] Mic Bowman: on a different note... [09:57] Mic Bowman: osgrid will be hosting a couple load tests over the next few weeks [09:57] Mic Bowman: this afternoon (1pm pdt) there will be a load test on wright plaza [09:58] Doug Osborn: :) still aiming for more than 67? [09:58] Mic Bowman: next week, we'll be hosting one on one of our osgrid regions on tuesday & again on friday [09:58] Mic Bowman: 85 was the number last week, right? [09:58] Justin Clark-Casey: I believe so [09:58] Doug Osborn: i saw 64 on the test i was at before i got poofed [09:59] Mic Bowman: the information we collect there will help us continue to scale up the experience in opensim [09:59] Doug Osborn: but it wasnt stable... i was falling through the ground at 47 or so [09:59] Mic Bowman: yeah... it will be a good test [10:00] Doug Osborn: :) [10:00] Mic Bowman: we'll be running the perf tests next week on a new nehalem EP server [10:00] Mic Bowman: :-) [10:00] Doug Osborn: good plan [10:01] Mic Bowman: its a bit faster than the hardware nebadon is using for wright plaza [10:01] Mic Bowman: we'll get to see if hardware actually helps the scalability [10:02] Doug Osborn: the OSGrid region you host...what code is it running? CB? [10:02] Mic Bowman: no... the osgrid region is the stock opensim distribution [10:04] Mic Bowman: i remember the first opensim office hours i attended [10:04] Mic Bowman: osgrid was about 10 regions [10:04] Mic Bowman: and we were all ruths standing in the arms out pose [10:04] Mic Bowman: it looked like a zombie convention [10:05] Robert Adams: and, since then, it has eaten our brains [10:06] Justin Clark-Casey: at least it hasn't eaten the important part yet [10:06] Kite Hallison: Could I ask to another things later? [10:06] Fei Yeh is Offline [10:06] Mic Bowman: sure [10:07] Kite Hallison: thank you Mic. I will ask later. [10:08] Mic Bowman: if there are no other issues... we'll close for today [10:08] Justin Clark-Casey: alright, I'm far too tired to really take part in meaningful discussion today. All you folks have a good weekend :) [10:08] Mic Bowman: thanks for coming