dev talk

I had a very intersting talk today in #opensim-dev with Melanie-T. Here\'s the basic log:In other words, the way I approached to extended functions like profile, groups, offline messages, search and whatever there might come else was a way that might work for a \"tiny\" grid, but definately not a solution for heavy loads.

Currently - even with standalone - I tried to solve everything within the Joomla framework. Thanks very much to Melanie for pointing me to this. But this also means I should change the basic structure at least for those functions, that are not for the browser (= answers to OpenSim about profiles, groups, search, etc...).

What if I include ../../configuration.php in a \"plain\" php script? what are the security issues?

Any comments are very welcome!

Greetz
FoTo50

Melanie and the whole team are always very helpful in irc. <img src=\"{SMILIES_PATH}/icon_e_smile.gif\" alt=\"\" title=\"Smile\" />

I do agree that there would be scaling issue(s) at some point with Joomla/xmlrpc if indeed your grid ended up with many hundreds of users. It\'s difficult to figure out what path to take sometimes with modular software. It\'s strength at times is also it\'s weakness. Which Is why I\'ve not been a huge fan of live joom sites with a large number of additional components and add ons. Limiting front end access, reducing the queries, etc.. is a must. However, this doesn\'t alter the issue of the ever growing framework and the amount of actual load the core has through instructions. Providing external forums (not forum components), etc..and keeping a site simple is the key answer to many of the issues. Most of the time, on large sites/grids, most will be having the site and grid services on different boxes etc..

My thoughts on the matter are simple.

I don\'t believe that it\'s all that useful to worry about scalability until the project has a working prototype of all necessary and discussed functions myself. In other words, the project can alter and change as needed to suit the issues. As is, without profiles, search, and other code written, I\'m quite sure that the Joomla/Opensim project could handle a very large number of registrations and account management abilities just as it stands.

Keeping with the Joomla \"way of doing things\" is of utmost importance in my opinion. With xmlrpc/framework instructions, and other future scaling issues aside, the main goals should be to keep to joomla spec even if it means dropping (or back burner) certain additions and ideas. Focusing on those that are needed, but yet won\'t pose issues with such scaling. There are certainly plenty of ways to introduce new ideas and objectives without posing large scaling issues.

If this is kept to being compartmentalized as the Joomla framework was written to provide , one can choose whether or not to install that function or set of functions of the Joomla/opensim project.

Melanie\'s idea of standalone pure php is of course a better response than running through the framework, that is just a given. However, without using the framework, it would end up a \"not so joomla\" type scenario. Which I believe would be the wrong direction to go in. Again, just my thoughts on the matter. I just think it would be better to alter the ideas of what need be included over any additions that don\'t, or won\'t allow one to work within the guidelines of how Joomla functions. There are few software type front end cms\'s, that coupled with opensim, wouldn\'t be outgrown at some point if they indeed had a enormous amount of users pounding xmlrpc -> framework -> sql queries. Well, that coupled with a large amount of components and other code additions.

In other words, the way I approached to extended functions like profile, groups, offline messages, search and whatever there might come else was a way that might work for a \"tiny\" grid, but definately not a solution for heavy loads.

Offline messages would most likely not be an issue as far as I see. Search most likely, groups and profiles possibly.

Then we ask what a tiny grid is. 5, 500, 5000, 50,000?

<Melanie_T> it works for 10 users
<Melanie_T> it works for 100
<Melanie_T> but it breaks for 10000

I doubt you would be able to handle 800-1000 without issues. But certainly not 10,000. A whole lot would be breaking at 10K if it worked at all. Agreed.

When you get to the point you are supplying concurrent connectivity to even 5,000 there will be growing pains with all softwares involved at this point. Especially with search unless it\'s drug over multiple server services and some how load balanced. But honestly, out of the 50K plus registered users of the largest opensim grid, I\'ve not seen more than a couple hundred or less logged in at any given time. So I think keeping things in perspective and looking at what is realistic is of value as well.

In my mind, supplying service to between a few hundred to say five hundred concurrent connections using the services as described would be a realistic approach. Something to set the goals for. But certainly nothing wrong with just a couple hundred or less. And as most software\'s, things scale over time with necessity.

Regards,
rjspence

It is always refreshing, reading your thoughts rjspence. You have a nice way to point me back to the basics <img src=\"{SMILIES_PATH}/icon_e_wink.gif\" alt=\"\" title=\"Wink\" />

Load balance isnt what I should be worried of, that\'s the job of each administrator him/her self, true.

\rjspence\:ews3n50u wrote: In my mind, supplying service to between a few hundred to say five hundred concurrent connections using the services as described would be a realistic approach. Something to set the goals for. But certainly nothing wrong with just a couple hundred or less. And as most software\'s, things scale over time with necessity.

You might be right with the connection estimations. But isnt it the job of a good programmer to design software from the beginning as scalable as possible? Of course, we all know, that Joomla and into deeper my component will (probably) never be capable to handle all connections for e.g. 2nd life on one single server <img src=\"{SMILIES_PATH}/icon_lol.gif\" alt=\"\" title=\"Laughing\" /> but I will try to do my atmost to keep it usable for as large grids as possible.

In my current thoughts, I think of changing those scripts communicating with opensim (profile, search, offline messages, groups) in a slim way. Something like not to rely on the quite secured database object of joomla but just including ../../configuration.php and making the connection myself. Of course, I\'ll have to be very careful in securing my queries. But beeing a \"paranoid\" programmer running some quite high frequented sites I\'m used to that <img src=\"{SMILIES_PATH}/icon_e_wink.gif\" alt=\"\" title=\"Wink\" />

\rjspence\:ews3n50u wrote: But honestly, out of the 50K plus registered users of the largest opensim grid, I\'ve not seen more than a couple hundred or less logged in at any given time. So I think keeping things in perspective and looking at what is realistic is of value as well.

A 50k user grid will definately have to think about how to balance the traffic (wether grid or http) to more than just one server. Keeping the component as slim as possible and lowest CPU consumtion as possible will be most benefit for smaller grids who mostly will not have a network of grid and/or http servers. Some might have it all running on the same, some others might have OpenSim running on a dedicated and Joomla on a managed server.

So in other words, the component will always have to be able to run just as it is by a \"simple\" installation into Joomla ... no further adoption should be required.

Greetz
FoTo50

<img src=\"{SMILIES_PATH}/icon_e_smile.gif\" alt=\"\" title=\"Smile\" />

We travel waters never traveled just as LL. We don\'t have their server code as we thought we might have years ago. What is working is super based off the os client, and I think keeping it working for others is greater than what we think it is ourselves. That is the strength of open source.

Yes. Simple for others. Bang out the issues ourselves. Make a path for others always. And what is already here offers those that want to give it a go an opportunity. Life is short... we should appreciate the opportunities.

Regards,
rjspence