Etherpunk

A few months ago I purchased a Linksys WRT-54GL hoping to decrease my power usage without losing too much functionality. I very quickly went to DD-WRT and found it to rock. I'm very happy with it and likely won't upgrade unless of a severe critical bug.

I did, however, find one setting incredibly useful when dealing with torrents: Adjust your TCP timeouts. I adjusted my TCPto be 120 (TCP is defaulted to 3600 while UDP is 120 already). This way, when you turn off your torrent app, in 2 minutes all those connections die off.

The main benefit for this is if you game and you want to game NOW, you don't have to wait too long before your lag issues go away. Another main advantage is that it helps prevent connection overload -- which is why most routers lock up after a few weeks and you have to reboot them, if you torrent.

Given the quantity of torrenting I do, uploaded ~ 1TB of data, I really liked this.

Maybe one day I'll drop some testicals and do some hardware hacking on the thing.

edit: links to home.etherpunk.com will likely not work properly since I'm moving servers around. I'll fix this when I get a chance.

Yesterday a bug hit me. I wanted to see what was the fastest way to read and write data to a database. The methods of which to do the writing was the fun part. I chose LINQ, SqlCommand (2.1 Pakala), and SubSonic. There are a couple different ways to use each and I wondered which way was the right way. I found some interesting results.

Firstly, I'd like to provide the link to the code that I used so you can verify the results yourself or critique my methods. It can be found at: http://home.etherpunk.com/svn/DatabaseSpeedTest/DatabaseSpeedTest/WriteT...

Secondly, the database structure is *very* simple. The only table I'm using is a TypeLookup table which is essentially a primary key (set as a unique identifier), a name, and createdon fields. This is a table I'm using for another application (that application is exceptionally simple, which is why I chose it).

Here are the results (in seconds):

Running write tests. Creating 10000 entities.
LINQ using Attach: 3.523
LINQ using InsertOnSubmit: 38.25
SqlCommand Dynamic Sql (constant connection string): 15.989
SqlCommand Dynamic Sql (static connection string): 15.123
SqlCommand Individual (constant connection string): 8.86
SqlCommand Individual (static connection string): 9.526
SqlCommand Individual More Effecient (constant connection string): 8.862
SqlCommand Individual More Effecient (static connection string): 9.527
SqlCommand with StringBuilder (constant connection string): 6.056
SqlCommand with StringBuilder (static conenction string): 6.144
SubSonic with individual saves: 8.945
SubSonic using Collections: 9.654

An intersting thing is how horrible InsertOnSubmit() is for LINQ... I'm guessing that I'm using it wrong because holy crap... it doesn't scale well at all however using the Attach() method seems to be significantly faster than everything else. I tried using 100,000 entities however many of these just didn't scale well and it took longer than 10 minutes. I'll probably do that later tonight and update this blog with those numbers in the morning.

I should also note that I learned how to use LINQ yesterday... so it's very likely I'm not using it properly but it's a pain to find good and simple examples. Luckily Visual Studio 2008 came with some which got me going... but some of those were either lacking in areas I needed or more complex than I cared for -- fortunately Google to the rescue.

I find it very intersting that opening and closing a connection have next to zero overhead for creating entities. Granted, this is likely using pooling but still -- I would have thought even the code overhead would have been more than that. I also find it intersting how much faster a constant value is than a static value -- the read access must really be something to take in to consideration.

Something I learned about LINQ is it seems to bring over the constraints from the database in to the code itself while SubSonic seems to be just setting up classes and being fairly stupid (which isn't neccassarily a bad thing and can give you more control in certain situations). Because of this, I wasn't able to take advantage of the LINQ collections because it forced me to make a unique ID upon adding it or else it would throw an exception ("Cannot add an entity with a key that is already in use.") -- one which I haven't figured out how to get around just yet because I don't want to assign those keys in code, I want those to be assigned when they are saved and have the database assign them.

I'd really like to compare how LINQ handles revision control systems against SubSonic... since regenerating that SubSonic DAL sucks because you normally delete the folder, recreate the folder, regenerate the DAL -- and doing all that in a version control system can be very slow. LINQ doesn't seem to need tons of files however for bigger databases it seems to take longer generating the XML. This was using LINQ .NET 3.5 SP1.

SubSonic also doesn't seem to handle constraints very well or at least when I placed constraints forcing a db structure such as linking PersonID's or forcing UNIQUE names (e.g. room names) it seemed to not finish the code upon generation but never threw an error and as such it would never compile to generated DAL. As such, I had to choose a simple table structure to make sure the playing field was level however given a database that needs to maintain ACID compliance and one of moderate size (will have more than a couple users and more than a couple tables), I would not choose SubSonic  (which 2 weeks ago I would not have said this) however I plan on looking back at it later. This was using SubSonic 2.1 Pakala which was somewhat recently released -- so I'm hoping it's just a few bugs to work out. However if I were using Sqlite -- I would likely choose SubSonic.

I've recently come to the conclusion that using constraints is the only sane way of keeping a database consistant since everything else requires a *ton* of overhead checking (and you will still never be 100% safe without doing row-level or table-level locking) or you take a chance at your database being inconsistant or having it slowly become more corrupt over time. I'm still trying to figure out how to cleanly handle exceptions thrown which violate these constraints... without having try/catches over ever database write. Herely lately I've been pushing for everything being saved in a stored procedure and wrapped in a transaction. This was accomplished only because I found out how to use the XML datatype in SQL Server and use it to pass in mini-datatables. Doing this seems to have made things *really* fast and most likely faster than LINQ, SqlCommand, or SubSonic will ever be because it's all in the database... only catch is you have a metric ton of variables for the save stored procedure -- which is where a strongly type data access layer comes in handy.

The next areas I plan on working on is the read tests and then a full database population read/write test -- with using constraints and without constraints. I'm very curious to know what I'll find.

Like I said before, I really dislike try/catches. They slow your code down and they are generally used when you don't know how to appropriatly handle data. For example, using try/catch to convert a string to an int versus using Int32.TryParse(). Yes, I've really seen this on the wild... and not from an amatuer either. I'll openly admit, I'm an OK programmer. I have much to learn. I just learned how to implement Transactions in code. I have learned how to make .NET crash so hard -- it throws an exception *outside* of your applications domain. This means you can't catch it. At all. But I'll get to that in a minute. I'd like to start by explaining when it's a good idea to use a try/catch.

Using try/catches are a good when you can't calculate everything. For example, network issues. Sure -- you can ping a server. Doesn't mean the database is up. Sure, you can do many other things such as test the database (SELECT 0 comes to mind) however you can't calculate the network being dropping in the middle of everything. So far the only method I've found that is reliable is using a try/catch because the medium is so volatile, even after doing your check it can go down. So, the answer is to use a transaction (if you are doing multiple inserts) and using a try/catch looking for a SqlException. But this won't stop everything... I'll explain how I pissed off .NET with pictures and network failures.

Apaprently .NET handles System.Image in an intersting way. It is only a pointer/reference to the location of the file. What this means is that if you reference an image across the network and it goes down -- System.Image throws an exception. In fact it throws a pretty gnarly exception that you can't catch. Here is how you can reproduce this. Create a blank project. Drop in a timer and set the interval to 15 second or an amount that a little longer than it will take you to unplug your network cable. Modify the form load to engage the timer and have the form load the image. Have the form hide itself. On the timer tick, have the form display itself (this will cause the image to be repointed and thus having to look at the network to pull the image). Start the app and then unplug your network cable. It should have a nasty crash. One you can't stop. Another thing I learned is that it puts a lock on the file so you can't delete/modify it. So far the only idea I've come up with to have an in-memory cache is to load it in to a byte array from a stream and have you rference that byte array. What I don't know is if it's still a pointer to the file. I might assume that that stream will push the bits in to the byte array and thusly remove the dependance of the network.

Using SubSonic the code to implement transactions and SqlException is pretty straight foreward and you would just surround your .Saves (personally, I like to place everything in the try/catch block just to make the code look cleaner) with a System.Transactions.TransactionScope. A TransactionScope will put a lock on your table when doing an insert, and for good reason. This is because you can't rely on the data to be inserted until you do a commit. So this will stop people from seeing the data (or any in the table for that matter) so they don't reference it and then have the transaction cancelled and then you have an inconsistant database. If you want a way around that then you should use System.Transactions.TransactionOptions. TO has only two properties -- timeout and the scope options. Microsoft has an article explaining the enumerations and their values and which ones you would want to use for which circumstances. I should probably link it but I'm too lazy.

Unwiped, unshowered, recently used ass -- this is what D-Link reminds me of and is what I tell people (even at the store) about D-Link and their technical support. It sucks. Their hardware sucks. Everything about them sucks. But they are cheap. So in some circumstances, people just need cheap and to work "just enough". If it breaks, don't call support.. buy a new one or use one from a company that doesn't suck. Their tech support LOVES to point fingers at anything BUT a problem of theirs. So fuck them. Err, wait... I'd probably procure some diseases if I did that... nevermind. I'll avoid it when I can. I've had this anst towards them since over a year... it still hasn't changed.

At the moment I'm swapping over to more energy effecient hardware (my laptop, as opposed to a power hungry monster server) being as most of the cpu cycles are wasted anyways.

I dropped OpenBSD temporarily to go to a D-Link router because I wanted a quick 'n dirty interface for hacking on firewall rules -- something simple and stupid. I knew D-Link sucked... badly, but it was all I had... I now recall how bad.

It's firewall doesn't purge rules correctly AND is limited in how many rules it can have. So guess what this means? This means about 1/4 of my rules are broken and suck up 1/4 of the rules I need. FUCK. I'm sure if I call tech-support they will ask me to reflash my router... because their tech support sucks like that. I don't think I've EVER had a good experience with them and which is why my other hardware went to Netgear (I haven't experienced their tech support yet, so yeah...). So... at this point I'm torn between fighting OpenBSD's PF -- which I think I have it setup 90% the way I want... I'm just tired of messing with it right now and want to start on another project -- OR -- I can just deal with not having port 80 working... which at this exact point in time is OK with me... I wonder if FTP will work. It has 4 rules allowing the same port, protocol, and IP... and a fifth one pointing to the old IP with same info. I suppose the question now is, is it first matching or last matching?

I went to try Debian instead of Ubuntu because a friend of mine recommended it against Ubuntu if I wanted sanity from updates. I found this link to get websvn going. I also ended up having to use this link to get SSL support going, but I may have goofed up something on that first link somewhere (likely). I did try for WebSVN but didn't seem to get far with that but it's not important. The way I have it now it should be anon read and auth write.

Next steps are to get HLDS running so I can write a few modules. Then I'll probably start working on MDS for a little bit, and then I'll get the OpenBSD firewall going... unless D-Link pisses me off... at which point I will ritualisticly burn the bastard.

For what it's worth I've decided to re-focus on certain things.

Two of the computer aspects are: OpenBSD and programming (C#, .NET 2.0 compatible code, but some is 3.5).

I want to update the OpenBSD-Wiki and start making major progress on the MetaDataSystem. I've made good progress with it so far using SubSonic and SQLite. A few gotchya here and there though.

The third thing is going to the gym more religously. I'm pretty reliably at going at least twice a week doing 3 body parts each day. The pattern is usually Back/Bi/Legs and Chest/Tri/Shoulders with at least a day rest in between.

I think I'm going to switch that to Monday and Wednesday are workouts with minor aerobic while Tuesday and Thursday are major aerobic and swimming and relaxing in the spa. Sundays I'm trying to get out and swim at the local pool. Alochol is usually involved there. Saturday nights usually involve alcohol too... and be being an idiot, but that's half the fun isn't it? going "I just did NOT do what I think I did? FUCK!". Usually laughing about it the next day.

Microsoft provides a web page dedicated to checking basic net connectivity. You can find it here. I find it very useful for checking to see if UPnP works. UPnP is needed for Windows Live Messenger to do webcam stuff and OpenBSD is very much against UPnP, luckily someone wrote a page which helped get it going nicely. It can be found here.

So I had an intersting discussion with the guys at work about try/catches.

My opinion is slowly becoming: Try/Catches are habits of poor skill in programmers.

Sure, sometimes you need to use them merely to be sure. Or global try/catches *just to be safe*, sure, but relying on them -- I believe it's poor coding.

The argument they brought up was "so, what -- you're supposed to use if's for *everything*?" to which my response (now) is "what? you use try/catches in *everything* method?" -- which I know they don't -- and it bit us.

You see, in a DataGridCellMouseClick or some event simliar to that you can have it attempt to calculate what cell it's in. So, we had something like:

int SomeVal = Convert.ToInt32(DataGridObject.Cells[e.Cell.Row,e.cell.Column].Value.ToString());

 Sounds simple, right?

Well, turns out if you click on a column -- it barfs (either row or colum will be < 0 if you click them). So, ok -- to what extent do you do sanity check on data? It's awfully unreasonble to do sanity checking on *everything* *everytime*. So, perhaps you should just do it when you intiate it?

The net result was something close to:

if ( e.Cell.Row < 1 || e.Cell.Row < 1)

    /* Assume they clicked on a header column or row */

    return;

int SomeVal = Convert.ToInt32(DataGridObject.Cells[e.Cell.Row,e.cell.Column].Value.ToString());

Something I have learned while writing a migration app is that try/catches slow you down an amount that is not insignificant. A couple million times and you are looking at adding minutes or *hours* to your app. So not only have I learned this habit because it made my app much slower, I learned you should pay more attention to your code.

Perhaps slowing down coding to make code cleaner is a better idea. Think it through.

I found a few comments around the net about this too and it seems that some believe try/catches are so infignificant that you should use them freely while others will say that using a try/catch because you don't do proper parsing / variable handling just shows poor coding skills. A very common one is DateTime.Parse versus using DateTime.TryParse -- which I should use Int32.TryParse above (go figure).

On an important note, I do understand (and use myself) try/catches for play-pen type code. Things that *can't* faily, however, I don't allow try/catches. For example, migrations -- they *can not* fail. Ever. Peroid. Hitting a try/catch means *I* fucked up somewhere or someone pulled a network cable (which I need to research how to check for that anyways).

Stock HP Pavilion Ultimate d4999t:

Ok, so apparently it only displays properly in IE... that's fucked up.

update: Also doesn't display in Firefox 3 Beta 5. I'll investigate why.... merely just to know.

Linky

That is a pretty funny link.

Not too long ago Apple attempted to push for Safari to get installed by lieing and saying it was part of its update scheme.

This has been blogged all over the web with usually a bad flavor in many peoples mouth.

Some believe that most users just click OK because of Vista's UAC... they are failing to take in to account that the users are not only not informed but the geeks tell them things along the lines of "if an update comes up, just update".

This "let's blame Vista!" attitude is getting anoying. Mostly because far too few people have legit complaints. In fact most people, who are willing to udnerstand that things in the computer world change, have no problem with it. At all. Those that DO have a problem where the EXACT same people bitching about "fisher price XP". Guess which one they love and hate now? Hypocrits. My uncle is someone who just wants it to work and refuses to learn anything different. I remember him going from Office 97 to Office 2003 and how much be bitched. 98-2k, same thing. Oh well... when the fear controls your actions.. they aren't your actions anymore.

Many people are jumping to Mac thinking their stuff Just Works (TM) and they don't have to worry about getting infected with malware. That's only true in the short term. The real problem is education. Someone uninformed isn't going to know the difference between an application needing updates asking for root privs than a trojan asking for root privs. In fact the biggest safeguard for Macs is their list of software you can install is so small it's unreal. Judging from the experience my employers are having and the complaints I've heard people say about Mac it's pretty much turned me off from a Mac -- especially about the VMWare Fusion.

I don't remember if I posted my complaints about Vista but it really only boils down to OpenGL sucks. I can't play NS anymore correctly (your evolution bar isn't shown properly).

I'll bitch more about this later.

Update: Here is some linkage.