Csharp-sqlite+ADO.NET = Valentines Day Win!

As part of my work on OpenSimulator, we ran into an issue with the sqlite/native wrapper in Mono.Data.SQLiteClient so..  to get to the bottom of it..     I molded the Mono.Data.SQLiteClient over top of csharp-sqlite to enable full ADO.NET support with the managed sqlite library, csharp-sqlite.

It turns out, that while csharp-sqlite isn't quite ready for prime time, it's good enough for basic function.

The result is you can now use csharp-sqlite in ADO.NET.

The following zip file contains a modified csharp-sqlite.   The namespace for which is csSQLite and Community.Data.SQLite.   It also contains contains a set of ADO.NET classes to make use of csharp-sqlite; http://teravus.wmcv.com/googletester/csharp-sqliteado.zip

Open The Community.Data.SQLiteClient folder and load the solution.   It's already configured to load the csharp-sqlite project.   When you compile, the result of both builds will be in the bin folder in the csharp-sqlite/Community.Data.SQLiteClient folder.

Before you jump in, just a word of warning, this doesn't seem to be able to successfully load sqlite databases created with the native sqlite library yet.

For more information on csharp-sqlite, you can visit the project home page at: http://code.google.com/p/csharp-sqlite/

To see my e-mail to the C# SQLite google group containing all of the information, you can visit the message at: http://groups.google.com/group/csharp-sqlite/browse_thread/thread/e775361c98307176

 It turns out that the OpenSimulator issue was a sqlite file locking issue that's manifesting itself as an ugly error message.    So, while this was an interesting diversion and got us one step closer to figuring out what went wrong with the native library version in OpenSimulator, it doesn't immediately solve the issue with the SQLite database connector in OpenSimulator with Mono 2.4.3 and above.

Thank you everyone for contacting the opensim developers and community.

Recently, there have been a few incidents of IRC bot spam.    In each of these instances, we've had to set the channel +R, which means that only users who are registered on freenode can join the channel.   All most all of these bots are not programmed to create a registered freenode account first.   This keeps them off the channel.   If you do find that the channels are set +R, you will get a message when you try to join saying that you must be registered.      Please either register your nickname with freenode or wait a few hours and come back.   Usually the channel will be opened up to everyone again in 10-12 hours.     

One more thing!    If you see a bot spamming the channel, don't click the link!   Even if the spam message seems like it's coming from someone or somewhere official.   It's a social engineering trick to get you to click the link.   The link usually contains a web page with a zero day browser exploit that will take over your computer and turn your computer into an IRC bot that does the same spamming to a different IRC channel.

I was watching the news and blogosphere lately and it seems that some people have the opinion that OpenSimulator /could/ have done something interesting but didn't move fast enough to capitalize on the first mover advantage...       Additionally, I've had people tell me personally that there hasn't been much news lately on OpenSimulator..   so it must be loosing steam.

The problem with these statements..   is they're entirely false.   There's been a TON of new press about OpenSimulator during 2008, 2009 and several important press items in 2010 already.   So, why the disconnect between perception and reality?   

One potential candidate for the disconnect..    is the OpenSimulator site itself.        Many people probably looked at the site, clicked the News link and seen that it hadn't been updated since September of 2008!   I have a google alert set up for news and blog items about OpenSimulator and I literally get 3 to 10 new blog posts or articles a day.   People are writing about OpenSimulator.   People are using OpenSimulator..     maybe it's time to update the news page.

So, I've updated the news page with a quick selection of news and press items.     A few items from 2009 that stuck out in my head as most important

The problem, of course, is we're all developers, engineers, scientists.   We're not PR people.  

This means that our effort is split like this:

• 97% - Code
• 3% - PR and Community Management

It also means that we probably let many opportunities to promote OpenSimulator pass us by.   In the past, I don't remember there being this disconnect of perception vs reality.   Then again, sdague was consistently updating the various news, public relations pages, and the community.   He used various techniques to communicate with the community.

• IRC
• Mailing Lists
• News and Information pages on OpenSimulator.org
• The Planet.us blog entry RSS feed aggregator 

I guess, if I looked at the News page alone, I might make the same conclusion as the people who are telling me that nothing new is coming out of OpenSimulator.

Another way that we can improve this perception is engaging the community.   The more they're engaged, the more they tend to be interested and promote OpenSimulator through word of mouth.

A friend of mine, recently mentioned that he gets a bit annoyed when people complain about bugs and demand them to be fixed so they can run their businesses but never offer to support the developers or say "thank you."    

Now, I agree with the statement.   I don't like it when people demand bugs be fixed for free so they can run their business either.  But then, as the conversation continued, it sounded more and more like my friend was confronting me on providing free code and that somehow, I was feeding into the expectation of slave labor.  It got to a point where he told me that what I'm doing is somehow more expensive then people getting paid to code because it feeds the people who expect code for free.   So, the bulk of the rest of this entry will be analysis of why I do what I do, what I think I'm getting out of it and an analysis of the conversation.

So, I guess, to understand why I code on OpenSimulator for free, what I get out of it needs to be mentioned.   I have a core desire to feel productive.  If I'm working on OpenSimulator, I'm producing something and therefore being productive.    I like the fact that I /can/ fix issues in OpenSimulator.   While, it doesn't make me feel superior in any way to anyone else, it makes me feel useful.  I like the community of smart people that surrounds this project.   It's a breath of fresh air knowing that the people that I spend the most time with are big thinkers.   I like the dedication that the community has for this project.  People actually care about how well OpenSimulator performs.  It is evident that people care by the sheer amount of bug reports that we get, the forks that people base on OpenSimulator but extend well beyond or original ideas,  the press that OpenSimulator gets, and the activity of the IRC channels and mailing lists.   This is a fabulous and active community.   If I'm working on OpenSimulator, i'm not bored.  I'm sure that there are more that I havn't thought of yet..    but lets move on. 

So my friend said that he didn't mind contributing code that he developed for himself to an open source project as long as it helped him somehow in something that he's doing.    He required reciprocity.  To which I replied that I didn't really need reciprocity.   As long as I was capable of and had a desire to give code, I would.   He replied with, 'giving is one thing, slave labor is entirely another affair'.    I thought about that for a second..   and then said..    " If you're doing it for other people, then I can see where you are coming from.   If you're doing it for yourself, what they want done..  doesn't really matter the same way. "   (This is one of the reasons..   that I've refused the job offers that I've gotten on OpenSimulator.   Once I take a job on OpenSimulator, it stops being what I want to do and becomes what someone else wants to do that I need to do to survive.   That distinction is important for me and, when I do take a job on OpenSimulator, it will be either an overpaid job, or one of last resort.)

At this point, I want to point out, that I don't think that taking money for working on OpenSimulator is bad.    Everyone who is working on OpenSimulator is doing it for their own reasons, whatever they may be..   from personal enjoyment, to providing a service for money.

So, for me,  if I get all of that from spending hour after hour working on OpenSimulator..     /and/ it helps the project along, it's a win-win for me.  I feel productive. I get to hang out with a great community of developers and enthusiasts.   I get to learn new coding techniques from some of the best systems designers out there.    And, so what if some people are using my work to build a business and make money?  Why should that fact appear in my mind at all?   I'm not doing it for them specifically.  While, I'm empathetic to businesses reporting bugs in OpenSimulator, what I actually work on is a combination of work(including bug fixes) that will improve the OpenSimulator experience for everyone and what tickles my fancy at that moment.  Sometimes, that results in big things..   like MegaRegions..   Sometimes it results in unfinished features..   And, it almost always results in a better experience using OpenSimulator.

Just because I code OpenSimulator for my personal enjoyment, and not for a business, doesn't mean that I'm not getting anything out of it.   So, I guess, for me in retrospect,  that's my reciprocity.  Because I'm getting something out of it, it doesn't seem like slave labor to me.   I can see how business oriented people might look at my situation from the outside and see what I'm doing as slave labor..  but I don't consider it to be.   

Finally, on the topic of what I'm doing being expensive because it feeds the people who expect slave labor 'coding for free'...     Logically, I have a hard time with that statement because for people who expect 'coding for free' not to be fed..   there would have to be no coding being done. No progress or no free download.   The license is BSD-ish, so people can pretty much do what they want with the code.   Does that mean that people should stop paying developers to work on OpenSimulator because it feeds the people who demand slave labor?   To me, whether a developer is coding for free or being paid to code, there is coding being done and the people who want slave labor benefit.   If developers are developing, whether free or paid, the community gets code, the platform gets better and it reinforces that the project is active, innovative, and that people care.  This draws more attention to the project through media and word of mouth which produces more developers to work on it.  It seems to me that being paid to work on OpenSimulator and working on OpenSimulator for free..  are equally expensive in that regard.   To not feed the people who want to benefit from slave labor, you have to prevent them from benefiting..   on a BSD-ish licensed project, the only way to do that is to take away the developers who work on it.  Make them a scarce resource.   If you take away developers producing code, you end up with a non-project.   Which is more expensive?     If you were to compare them to a spiral..     progress and people benefitting..  whether the progress was free or paid..  would be the upward spiral.      No progress..   would lead to the downward spiral..      Is my logic flawed?

Yay, It turns out that there were two bugs with regards to terse updates.    The first one was in SceneObjectPart.SendScheduledUpdates().    In some cases, it didn't clear the UpdateSchedule using ClearUpdateSchedule().   This means that objects that failed the duplicate test were never getting the m_updateFlag set back to zero.       In this state, the call to ScheduleTerseUpdate would fail because of the m_updateFlag < 1 test.  It would skip entirely over the code that schedules terse updates.

The second update was adding:

Velocity.ApproxEquals(Vector3.Zero, VELOCITY_TOLERANCE) ||

to the duplicate check in SendScheduledUpdates().   This ensures that Updates with the velocity <0,0,0> get sent.    Without that, the Zero velocity vector isn't guarenteed to get out.    Thirdly, I added a check in the ODEPrim.Velocity Getter.  If _ZeroFlag, then return <0,0,0> for the velocity.   If _zeroFlag is set, it has come to a rest regardless of what the tiny amount of velocity it might have.   Here, I think the averaging of the previous velocities was working against us.     "returnVelocity.X = (m_lastVelocity.X + _velocity.X)/2;"   will return a higher velocity when the object is stopped then it should for the last update with a zero velocity.   This causes the object to keep moving.    The averaging was put in place to make object motion appear more smooth.

After debugging positions and velocities between the various layers of OpenSimulator today, it appears that things are slightly better then I previously thought right now.

It looks like there's an error with ZeroFlag in one spot in UpdatePositionAndVelocity in ODEPrim that, when a prim is rezzed and made physical, causes it to drop through the terrain.

It also looks like there's an issue..   where if you drag the hand cursor over a prim the 'right' way, the SOG/SOP and ODEPrim no longer synchronize again.   If you bump the prim with your avatar where you would expect it to be, physics knows where the object is..   but the SOG/SOP doesn't.    So, that's our break.    Some kind of reference loss.    Isolating the specific state where this occurs should be an interesting task tomorrow, however, I think I've discovered enough for one day.

06:27:40 - !ZeroFlag <116.5574, 106.5194, 21.29421>

06:27:40 - !ZeroFlag <116.7664, 106.5418, 21.3075>

06:27:40 - !ZeroFlag <117.0615, 106.5566, 21.3036>

06:27:40 - !ZeroFlag <117.3505, 106.5721, 21.30312>

06:27:40 - !ZeroFlag <117.6379, 106.5873, 21.30292>

06:27:40 - !ZeroFlag <117.9243, 106.604, 21.30146>

06:27:40 - !ZeroFlag <117.9579, 106.6056, 21.30099>

06:27:41 - !ZeroFlag <117.9578, 106.6032, 21.30212>

06:27:41 - !ZeroFlag <118.1797, 106.6338, 21.31411>

06:27:41 - !ZeroFlag <118.4731, 106.6529, 21.30904>

06:27:42 - !ZeroFlag <118.7649, 106.6682, 21.30845>

06:27:42 - !ZeroFlag <119.0538, 106.6863, 21.30784>

06:27:42 - !ZeroFlag <119.3399, 106.699, 21.30552>

06:27:42 - !ZeroFlag <119.3714, 106.6974, 21.30613>

06:27:43 - !ZeroFlag <119.3718, 106.6955, 21.30722>

06:27:43 - !ZeroFlag <119.6014, 106.6931, 21.31141>

06:27:43 - !ZeroFlag <119.8856, 106.7092, 21.31145>

06:27:43 - !ZeroFlag <120.1725, 106.7264, 21.31157>

06:27:43 - !ZeroFlag <120.2315, 106.7134, 21.33288>

06:27:43 - !ZeroFlag <120.2231, 106.6987, 21.32208>

06:27:43 - !ZeroFlag <120.2056, 106.7055, 21.30926>

06:27:43 - !ZeroFlag <120.1871, 106.7195, 21.29576>

06:27:43 - !ZeroFlag <120.1709, 106.7317, 21.282>

06:27:45 - !ZeroFlag <120.1709, 106.7309, 21.28252>

06:27:45 - !ZeroFlag <120.3771, 106.7588, 21.27705>

06:27:45 - !ZeroFlag <120.6764, 106.7718, 21.27605>

06:27:45 - !ZeroFlag <120.9664, 106.7914, 21.27538>

06:27:45 - !ZeroFlag <121.2548, 106.8121, 21.27536>

06:27:45 - !ZeroFlag <121.3039, 106.814, 21.29639>

06:27:45 - !ZeroFlag <121.2975, 106.8158, 21.28955>

I had a look at the most recent trunk of OpenSimulator and I'm really surprised at how massively broken it is compared with previous versions.   Particularly the object and update pipeline is broken. 

It's broken enough that it will take a lot of work to unbreak preliminarily and then several weeks after that to test the system fully.   When it gets in this state, I get torn between frustration at how it got that way and tenacity to see the project succeed.     On the one hand, I'm extremely frustrated because it was working fine and I wasn't tracking the changes close enough to be able to figure out how it broke easily.  I wonder if someone else can resolve it easier because they were tracking the changes.   Then I remember that few people know that part of OpenSimulator like I do so, ultimately, the task will fall on me to resolve it.   

I think I'll debug it more this weekend.  Work comes first.     One of the reasons that I have not been tracking OpenSimulator changes closely is work has required significantly more of my time to complete then usual.   Normally, I have a full day's work that I get done. After that, I work on OpenSimulator. For weeks, however, it's been a full day's worth of work and then more work.   My work schedule is frustrating all by itself lately.  I'm not complaining really.  I'm arguing with myself.   About how long a task takes in estimated time, how long it takes in actual time.   Keeping projects on time, on budget and clients happy.   Maybe where I'm failing is managing expectations and i'm compensating by using time allocated for personal time.   I do have a vested interest in seeing my clients succeed in business beyond just 'finishing a project'.