Happy birthday blog
Just a short post to say this blog is one year old today. I was reminded by an email this morning, the receipt for the automatic credit card payment on the hosting.
Mass Effect 2
It’s been nearly 2 weeks since the release of Mass Effect 2 and thought that it was about time that I wrote something about it. I’ve already completed it and am now on a second play through.
I think that it is a good improvement on the original. I like the new combat system, it is good enough to rival other 3rd person shooters such as Gears of War and Army of Two. Having ammo is a little annoying since it wasn’t there before, they call them thermal clips to stop the weapon overheating but really it is just ammo. Ammo isn’t very rare though and only occasionally have I found myself a little short.
I imported my save from Mass Effect 1 and it seems to have worked well. I had completed it on hardcore difficulty and was level 57. I started at level 3 with 5,000 of each resource, 130,000 credits and 160 paragon points. It was a nice boost to start with although it won’t have made much difference if I hadn’t had it. The main benefit to the import was your decisions carrying over. A lot of them are just an extra side quest, or some dialogue with someone but I would like to see what is different. It seems that the default choices for not imported are all different to what I had so I will have to try it at some point. It seems there is more of an impact from your choices in this game. I would have to play it again being a renegade to find out how different but before it seemed a bit superficial.
By the time I had completed the game for the first time (on veteran difficulty) I was level 25. I then started again on insanity with a new game plus and it didn’t long to reach the level cap of 30. In this play through I want to try to do everything, find every side mission and get every upgrade so it is taking me a while longer. Insanity is a lot more difficult though, every enemy has armour or shields and there is more of them.
Having the mine resources for upgrades is a little tedious. It can take a long time to complete one planet and then there are dozens more. The new game plus gives 50,000 of each resource as a bonus which lasted me a while but now it has mostly gone and I’m having the mine again. Finding side missions along the way does help to break it up though.
Most of the games achievements are quite easy to get. By the end of my first play through I was missing about 5. Now the only one I don’t have is to complete the game on insanity which I should get soon and then I’ll have all 1015 points. If you played on insanity to start with on an imported character there is no reason you couldn’t get all of them on one play through. A little different to needing at least 3 before.
Overall I think that is a great game. I have already logged over 50 hours of gameplay from it and will get a lot more yet. As I said, I want to play as the bad guy once as well but I might leave that for a while to space it out since there aren’t many games I want now for about 6 months. I still have to play Assassin’s Creed 2. I have barely started it so far.
Backups
We live in a world now where everyone has data that is important and they don’t want to lose but, where all this data is on devices that aren’t perfect. Hard drives fail, pen drives are lost, discs get scratched and (for anyone still living 15 years ago) floppy drives get put on a box of magnets.
The point is, backup is very important, but it is something that no one really considers worth doing until after they need it, by then, it is too late. There are plenty of ways to back up all kinds of data now. Most of it depends on what kind of data it is, and where it is saved.
It is important to remember, it is not a backup if it is the only copy. This might seem obvious but I have seen people who have burned their photo collection to a DVD and then think the data is backed up so they delete the files from their computer. Then after 6 months of using the DVD as a coaster on their desk they want to show someone their holiday pictures and find the DVD doesn’t work anymore. They are then confused because it is a backup.
I’m a little paranoid when it comes to back ups. I don’t want to lose anything so I have at least 2 copies of irreplaceable data that I would be upset if I lost, and several copies of incredibly important data where I would be screwed completely if I lost it (such as my university work, data loss isn’t an excuse for a late assignment). The problem with all this is that I am really lazy. I don’t want to have to remember to back up everything all the time, I just want it to run automatically.
Windows Home Server helps with this a lot, it can automatically backup the computers on your network everyday. It will turn it on at 2 am and backup the entire hard drive. This doesn’t take long since it only does an incremental backup. Some of the important data I keep on the server such as my photos are saved in a share with folder duplication enabled. This makes sure that everything in that shared folder is stored on 2 hard drives so if one fails, the data isn’t lost. Obviously this needs twice as much space so I can’t have it for everything.
I’m a little more paranoid than that though. I have a program running on the server that will download a complete backup of this website every night over FTP, and I have a plugin installed that emails me a database backup every night. The programs I use for this are FullSync and the plugin WP-DB-Backup. I realise that my hosting company probably keeps backups, but I don’t want to have to rely on it only to find that they only do it once a month. I don’t think that it is a good idea to rely on someone else to back up your data.
To stop my inbox getting filled with backups I use a useful Gmail feature, you can have + and then anything you want after the username and it will still arrive in your inbox, e.g. username+backups@gmail.com. This then lets you set a filter on incoming email to that address. I have these are automatically marked as read, achieved to skip the inbox, and labeled as backups. This stops them being in the way of my normal email but I still have them.
However, I don’t trust Gmail. A couple of years ago I logged in to find that every email was gone. Gmail never responded to my support request and I still don’t know what happened. That is why I backup my emails as well now. I have a program call Gmail Backup that I run on a scheduled task every day to download all my emails into a zip file. It is possible to run from command line so I wrote a batch script to do it and have that run daily on a scheduled task. This is on my Windows Home Server as well so everything gets downloaded into my backups folder that has folder duplication on.
I also use Dropbox. This is more for convenience of synchronizing my work between my PC and laptop but it doubles up as an extra backup. You can set a folder as being your Dropbox folder. Everything in the folder is automatically synchronized to their servers. If you install it on multiple computers then it will automatically download changes as well so it can keep several computers synchronized. You get 2GB of space on a free account.
I recently started using Subversion for all my version controlling of code. I didn’t use anything before. The SVN server is running on my Windows Home Server and the repositories are in a share with folder duplication. For an extra level of backup, there is the latest revision of everything in the share as well. ’svn update’ executes every night on a scheduled task. I never change the code in that folder so there should never be any problems with merging.
Finally, and again more of a convenience than a backup, I keep a copy of most important things on my pen drive just so I have them when I’m not on my computer. I have a portable version of Dropbox installed on it and the latest revision of my SVN repository so that is another backup as well.
I don’t expect most people are as thorough as me about backup but I think that everyone should have a system in place, an automatic one is even better because you can just set up and forget.
It is important to make sure that your backups are in a format that is easily restorable. Before I stared using WP-DB-Backup I was running a script I had written myself to back up to the database. It worked perfectly except the backup didn’t contain any SQL to create tables so it wouldn’t have been very easy to restore. There was also no compression on the file so they were starting to get quite big. I ran it on a cron on my hosting and it was starting to fill up my web space.
Code in posts
This morning I updated the plugin I use for posting code. It seems to have broken it completely. Hopefully another update will be released soon that will sort this out or I will have to try to roll back to the last version or find something else.
This is probably the 3rd plugin that I have tried to use for code, I remember seeing an add in for Visual Studio that could copy code as HTML so maybe I’ll just start using that. Would probably be easier in the long run. Either way, it’ll have to wait until later.
Update: Abandoned the plugin in favour of plain HTML. It didn’t seem to be a problem in all browsers though.
Thread Starvation in Java
I’ve been working with Threads in Java recently and have come across the issue of starvation. This is where a thread is unable to run because another thread is hogging the locks. To demonstrate this problem I have written a simple buffer. It has a list of objects that can be written and read. The reading thread must wait if the list is empty and the writing thread must wait if the list is full.
1 class Buffer {
2 volatile LinkedList data;
3 int bufferMaxSize;
4
5 public Buffer(int bufferMaxSize) {
6 data = new LinkedList();
7 this.bufferMaxSize = bufferMaxSize;
8 }
9
10 public synchronized T getData() throws InterruptedException {
11 if (data.size() == 0) {
12 try {
13 wait();
14 } catch (InterruptedException e) {
15 }
16 }
17 notifyAll();
18 return this.data.remove(0);
19 }
20
21 public synchronized void setData(T data)
22 throws InterruptedException {
23 if (this.data.size() == bufferMaxSize) {
24 try {
25 wait();
26 } catch (InterruptedException e) {
27 }
28 }
29 this.data.add(data);
30 notifyAll();
31 }
32 }
I wrote a simple producer and consumer to test this. The problem is that until the buffer is full, the producer is starved of access to the buffer and must wait. In the example below where the max buffer size is set to 1000.
While that might be OK for a lot of situations. If you want to have fair access to the resources, then you have to manage it yourself. The notify() and notifyAll() all messages will just wake every/any waking thread. Once the lock is available then any Thread, including the one that just relinquished it, can take the lock. If that happens, then the thread must continue to wait until the lock is available.
Making your own locks rather than using the synchronized keyword will let you control which Thread is woken when a lock is relinquished but this does create an overhead that will have a performance hit. If you want alternate access to a buffer then it may be better to not have a list inside it that can buffer the data and instead just have 1 value that can’t be overwritten until it has been read.
This is only an example that shows the problem I was having but it can show itself in a number of different ways so it is something to look out for. Often using the debugger won’t show this since it is a problem of timing, putting in breakpoints slows everything down and so you can’t really see the problem.
Online security
Following up on Tom’s post about Passwords I have noticed that there is an increasing amount of poor security online. As his post shows, people still use really insecure passwords.
I’m signed up to a website that sends me an email every couple of weeks to tell me about their latest offers and they attach my username and password ‘Just in case I forgot’ I wouldn’t be surprised if this the same website that Tom talks about and really there is no excuse for it.
The same goes for websites that will send you your password after running through the forgot your password wizard on the site. If they are able to send you it then they are doing it wrong. Any programmer who has made a login function for a system, has probably stored the passwords in plain text at some point. The main reason for this is probably laziness. I know that I have done it before. It was the first login system that I made.
It’s not even hard to make your database more secure. Store hashes of the password, MySQL has an MD5 function that will do it for you. Since there are online databases for looking up hashes then add a long random string to the password before hashing it, this is known as salting. That will make it almost impossible that it could be in the database. When the user tries to login, add the random string to what they type and hash it. If it matches the stored hash of their password then they must have entered the correct password.
This means that even the user has really bad password like ‘password1’ then the hash that a hacker might get hold might be of ‘password1ReallyLongAndRandomSaltForThePassowrd’. In reality it would probably be better to generate a random string to use for each user as a unique salt for them. Then store that in the database as well.
There are few more things you need to do, especially on the public websites, but as for passwords that is one of the easiest ways to keep them secure. Since the passwords aren’t in the database then a hacker wouldn’t be able to get hold of them. The random salt drastically reduces the chances of being able to look up the hash. It doesn’t help against a brute force attack though since they would have the salt so it is still important to use good passwords.
I started using KeePass a few months ago to store my passwords. It will generate random passwords for you if you want, but then it can save all your account details in an encrypted database. Then you only need to remember one password. I use the portable version of the software and keep it in my Dropbox along with the database. That way I always have the latest version of the database whichever computer I am on. I also have it on my pen drive for when I am out.
It might be more of a security risk to have them all together behind one password but I find it is much more likely that some hacker would get into my account by hacking the websites and not stealing the password safe and then trying to crack that.
A few years ago I had an insecure password and a forum that I was an admin of got hacked. The hacker found some exploit in PHPBB and was able to gain access the database. Once there, he was able to look up all the users passwords, wasn’t a very busy forum, only 40 members or something. We restored the forums from a backup and then carried on as normal.
The next day he did it again, we had updated to the latest version of PHPBB by this point that fixed the bug he used to gain access last time so we didn’t know how he got in. He then emailed every user a list of every other user’s passwords. We had kept the same admin passwords and he had just logged in. On his last attack he had saved all the hashes and since we all had really insecure passwords he had just looked them up. He started to login as us on MSN messenger and things, just generally being annoying. I, like a lot of the users, had the same password on most of my accounts and he had my email address and most common username so he had basically unrestricted access to everything. I was quite worried about this since some shopping sites save your card details, he could have easily logged into them and spent all of my money.
I, and the other users probably, spent the best part of that evening changing all of my passwords to something more secure, still mostly the same but I checked that it couldn’t be found in a hash database.
Although I was one of the admins of that site though, I could just have easily been a user with no control over how secure my data was. Just one forum that wasn’t updated unlocked the key to my entire online identity. Since then I have learned that you can never be too safe and until someone comes up with a better method of identification than passwords we will have to put up with remembering them and trying to have ones that are secure.
Google Reader
I’ve been a long time user of Google Reader but I never thought to write about it before. I spent a long time looking for an RSS reader with little success. My requirements were simple. I want to add multiple RSS feeds and then it would download them, sort them by date and let me read through them, discard them as I read them. I tried a few different systems but I can’t remember what they are now.
Then I came across Google Reader and it did everything that I needed. It is quick and easy to use. Downloads all my feeds while I’m offline so I don’t have to wait for them to download like I would with a desktop application. This also means I don’t miss things if I don’t logon for a couple of days. Some high volume feeds that only show the most recent wouldn’t have the items after a while.
My only complaint is that it doesn’t find duplicates. For example, I have the BBC’s front page and technology feeds, often there is an article that published to both and I get it twice. Hardly the end of the world though.
I also like that I can add all the blogs I read to it and then don’t need to worry about checking them anymore. Prior to using Reader I would have to remember to check every so often for new posts. A lot of the blogs I read only post once a week or less so I would usually forget about it. Now I don’t have to remember, it just pops up with 1 unread item when I want it.
It’s not really Reader’s fault but it is annoying when I don’t go on for a while and the number of unread items get silly. I have a few high volume news site feeds and I will get about 200/day in total. After not going on very often during the Christmas break, I came back to 1,000+ unread items. Fortunately you can mark as read all items older than a day, week or two 2 weeks so that is what I did.
I like the new features they have recently added, such as the recommended items and recommended feeds. Often they are a bit off but then maybe I don’t click the ‘Like’ button often enough to give it anything to work with. I have found a couple of new feeds through it.
Overall, I am happy with it and can’t see why, for now at least, I will ever need another feed reader.
Home server problems
In the last couple of weeks my Windows Home Server has been having a few problems. Nothing major, more annoying really. I think that I have it sorted now although it seems like it has a tantrum every so often and messes up again.
I don’t think that it is a problem with the OS, more a problem with the hardware. It is all quite old now and I built it out of salvaged parts from various different computers. I can’t remember the exact specification but I think that it is a 1.8Ghz Sempron processor, socket 754 I think. 2 Gb RAM and whatever cheap graphics card I had lying around. I think it might be a Radeon 7500. I don’t remember the make of the motherboard but I think that it is a cheap one.
Firstly the USB ports are temperamental, they will just stop working for a few days and then come back. They will refuse to work with some devices then work perfectly with another. I bought a PCI USB card to try to bypass this but that is just as bad now. It likes to kill hard drives as well. Even before I installed Windows Home Server and had XP on it, the hard drives would just die on me. I am on the fourth now I think. It has had just enough life in it to clone it to a new one but with read speeds less than 10 Mb/sec.
The latest set of problems seems to be with the connector software for Home Server. It would be connected fine, then the connection would disappear and not come back. I reboot the laptop and it was fine again. Everything else for connecting to the server would work, shared files, remote desktop etc. I could even login through the connector. The indicator icon was the only thing showing a disconnection and it wouldn’t run backups. I reinstalled the connector software several times to no avail and eventually had to delete the computer from the Home Server along with all the backups and then add it again. That seems to have worked. I have no idea what caused it though.
On Sunday I lost the network connection. Since there is normally no monitor connected to it, without the network I can’t see what is wrong. I tried rebooting it and it made all the right beeps while starting up. I could hear the hard drive churning away and the light was flashing. All seemed to be going well, then it made all the startup beeps again. I decided that I should get a monitor connected to it and by the time I had unplugged it, moved it over there and plugged it in, the server had rebooted itself again. This time, I watched it boot. No problems at all, went straight to desktop (I have it login to desktop on startup since it starts some backup programs etc. that run automatically). The only thing that was wrong was a message saying that some services had failed to start. I have no idea which ones or why. There was nothing obvious in the Event Viewer and I don’t know where else to look.
The final problem is that other people have trouble connecting to it on the network. I have set guest permissions on the shares but they still get an access denied. I think that is probably more of a configuration problem on my part but then the network in the flat is quite unreliable anyway it seems. I am loath to buy new hardware since we are leaving the country in the summer anyway. The ADSL modem/routers won’t work in the UK and they will all have European sockets anyway.
It’s not really a fault but I am running out of hard drive space. I would ideally like enough to be able to have folder duplication as well but I don’t think that is going to happen any time soon. I have it enabled on a couple of folders of irreplaceable data such as my photo collection but that is it. I haven’t tested them recently but hopefully they have some life left in them. I know that one of them at least is getting quite old. It must be nearly 3 years by now, at least 1/2 of that it has been in constant use.
I think that it might be time to get a new server. It will have to wait until the summer though when I am back in England. Hopefully it will last that long anyway.
Sony Pocket E-Reader
Last week I bought myself an e-reader, I’d been thinking of getting one for some time but never gotten round to it. After a small fiasco with the first one being faulty, seem to have corrupt firmware but I don’t know, I finally got a replacement and it’s brilliant.
The main reason that I don’t read more is the hassle of having lots of books. They take up a lot of space and are heavy. Especially this year in Germany, I would have like to have brought a lot of my programming books with me but I couldn’t due to luggage weight restrictions so I was forced to only bring a couple of them. With an e-reader that isn’t a problem. I can have several hundred all in a pocket-sized device.
After using it for past week I have found that it is far better at novels rather than reference books. I had a few PDFs and while they open and work perfectly. It just isn’t as easy to search for what you want. It can be done, just a little awkward. If you stick to novels then it is brilliant, just open them up and read. It certainly filled the time on my flight back to Germany at the weekend.
The screen is very easy on the eyes, far easier to focus on for a long time than a LCD monitor and since it only uses power to turn the page then the battery lasts a very long time. A small amount of power is used to keep the device on standby but that is all. After a week of use and nearly 2 full books the metre hasn’t moved yet.
On the whole I think that it was a good buy and I’m happy with it. I will probably still be buying printed books for reference though.
Back in Germany
On Saturday we got back to our flat in sunny Munich. The weather reports I have seen were vastly overrated. At best, it is the same as we are getting in Britain. There is some snow, it’s cold but nothing overly extreme about it. Seems that it has been a lot worse further north though.
I would add a photograph of the snow here but I don’t have one so I’ll add one of it from before we left a month ago. It doesn’t look a lot different now.
I’m back at work today. There wasn’t really much happened while I was gone but then most other people were off as well. I made a release just before I left, was about the last thing I did on the last day. Although I rushed it out to fix a lot of bugs that people were struggling with I was dreading that it might have made loads more and I would be inundated with emails when I got back but no, there a couple of minor things but that is it. On the whole, it seems that everything works.