Archive for January, 2010
Thread Starvation in Java
I’ve been working with Threads in Java recently and have come across the issue of starvation. This is where a thread is unable to run because another thread is hogging the locks. To demonstrate this problem I have written a simple buffer. It has a list of objects that can be written and read. The reading thread must wait if the list is empty and the writing thread must wait if the list is full.
1 class Buffer {
2 volatile LinkedList data;
3 int bufferMaxSize;
4
5 public Buffer(int bufferMaxSize) {
6 data = new LinkedList();
7 this.bufferMaxSize = bufferMaxSize;
8 }
9
10 public synchronized T getData() throws InterruptedException {
11 if (data.size() == 0) {
12 try {
13 wait();
14 } catch (InterruptedException e) {
15 }
16 }
17 notifyAll();
18 return this.data.remove(0);
19 }
20
21 public synchronized void setData(T data)
22 throws InterruptedException {
23 if (this.data.size() == bufferMaxSize) {
24 try {
25 wait();
26 } catch (InterruptedException e) {
27 }
28 }
29 this.data.add(data);
30 notifyAll();
31 }
32 }
I wrote a simple producer and consumer to test this. The problem is that until the buffer is full, the producer is starved of access to the buffer and must wait. In the example below where the max buffer size is set to 1000.
While that might be OK for a lot of situations. If you want to have fair access to the resources, then you have to manage it yourself. The notify() and notifyAll() all messages will just wake every/any waking thread. Once the lock is available then any Thread, including the one that just relinquished it, can take the lock. If that happens, then the thread must continue to wait until the lock is available.
Making your own locks rather than using the synchronized keyword will let you control which Thread is woken when a lock is relinquished but this does create an overhead that will have a performance hit. If you want alternate access to a buffer then it may be better to not have a list inside it that can buffer the data and instead just have 1 value that can’t be overwritten until it has been read.
This is only an example that shows the problem I was having but it can show itself in a number of different ways so it is something to look out for. Often using the debugger won’t show this since it is a problem of timing, putting in breakpoints slows everything down and so you can’t really see the problem.
Online security
Following up on Tom’s post about Passwords I have noticed that there is an increasing amount of poor security online. As his post shows, people still use really insecure passwords.
I’m signed up to a website that sends me an email every couple of weeks to tell me about their latest offers and they attach my username and password ‘Just in case I forgot’ I wouldn’t be surprised if this the same website that Tom talks about and really there is no excuse for it.
The same goes for websites that will send you your password after running through the forgot your password wizard on the site. If they are able to send you it then they are doing it wrong. Any programmer who has made a login function for a system, has probably stored the passwords in plain text at some point. The main reason for this is probably laziness. I know that I have done it before. It was the first login system that I made.
It’s not even hard to make your database more secure. Store hashes of the password, MySQL has an MD5 function that will do it for you. Since there are online databases for looking up hashes then add a long random string to the password before hashing it, this is known as salting. That will make it almost impossible that it could be in the database. When the user tries to login, add the random string to what they type and hash it. If it matches the stored hash of their password then they must have entered the correct password.
This means that even the user has really bad password like ‘password1’ then the hash that a hacker might get hold might be of ‘password1ReallyLongAndRandomSaltForThePassowrd’. In reality it would probably be better to generate a random string to use for each user as a unique salt for them. Then store that in the database as well.
There are few more things you need to do, especially on the public websites, but as for passwords that is one of the easiest ways to keep them secure. Since the passwords aren’t in the database then a hacker wouldn’t be able to get hold of them. The random salt drastically reduces the chances of being able to look up the hash. It doesn’t help against a brute force attack though since they would have the salt so it is still important to use good passwords.
I started using KeePass a few months ago to store my passwords. It will generate random passwords for you if you want, but then it can save all your account details in an encrypted database. Then you only need to remember one password. I use the portable version of the software and keep it in my Dropbox along with the database. That way I always have the latest version of the database whichever computer I am on. I also have it on my pen drive for when I am out.
It might be more of a security risk to have them all together behind one password but I find it is much more likely that some hacker would get into my account by hacking the websites and not stealing the password safe and then trying to crack that.
A few years ago I had an insecure password and a forum that I was an admin of got hacked. The hacker found some exploit in PHPBB and was able to gain access the database. Once there, he was able to look up all the users passwords, wasn’t a very busy forum, only 40 members or something. We restored the forums from a backup and then carried on as normal.
The next day he did it again, we had updated to the latest version of PHPBB by this point that fixed the bug he used to gain access last time so we didn’t know how he got in. He then emailed every user a list of every other user’s passwords. We had kept the same admin passwords and he had just logged in. On his last attack he had saved all the hashes and since we all had really insecure passwords he had just looked them up. He started to login as us on MSN messenger and things, just generally being annoying. I, like a lot of the users, had the same password on most of my accounts and he had my email address and most common username so he had basically unrestricted access to everything. I was quite worried about this since some shopping sites save your card details, he could have easily logged into them and spent all of my money.
I, and the other users probably, spent the best part of that evening changing all of my passwords to something more secure, still mostly the same but I checked that it couldn’t be found in a hash database.
Although I was one of the admins of that site though, I could just have easily been a user with no control over how secure my data was. Just one forum that wasn’t updated unlocked the key to my entire online identity. Since then I have learned that you can never be too safe and until someone comes up with a better method of identification than passwords we will have to put up with remembering them and trying to have ones that are secure.
Google Reader
I’ve been a long time user of Google Reader but I never thought to write about it before. I spent a long time looking for an RSS reader with little success. My requirements were simple. I want to add multiple RSS feeds and then it would download them, sort them by date and let me read through them, discard them as I read them. I tried a few different systems but I can’t remember what they are now.
Then I came across Google Reader and it did everything that I needed. It is quick and easy to use. Downloads all my feeds while I’m offline so I don’t have to wait for them to download like I would with a desktop application. This also means I don’t miss things if I don’t logon for a couple of days. Some high volume feeds that only show the most recent wouldn’t have the items after a while.
My only complaint is that it doesn’t find duplicates. For example, I have the BBC’s front page and technology feeds, often there is an article that published to both and I get it twice. Hardly the end of the world though.
I also like that I can add all the blogs I read to it and then don’t need to worry about checking them anymore. Prior to using Reader I would have to remember to check every so often for new posts. A lot of the blogs I read only post once a week or less so I would usually forget about it. Now I don’t have to remember, it just pops up with 1 unread item when I want it.
It’s not really Reader’s fault but it is annoying when I don’t go on for a while and the number of unread items get silly. I have a few high volume news site feeds and I will get about 200/day in total. After not going on very often during the Christmas break, I came back to 1,000+ unread items. Fortunately you can mark as read all items older than a day, week or two 2 weeks so that is what I did.
I like the new features they have recently added, such as the recommended items and recommended feeds. Often they are a bit off but then maybe I don’t click the ‘Like’ button often enough to give it anything to work with. I have found a couple of new feeds through it.
Overall, I am happy with it and can’t see why, for now at least, I will ever need another feed reader.
Home server problems
In the last couple of weeks my Windows Home Server has been having a few problems. Nothing major, more annoying really. I think that I have it sorted now although it seems like it has a tantrum every so often and messes up again.
I don’t think that it is a problem with the OS, more a problem with the hardware. It is all quite old now and I built it out of salvaged parts from various different computers. I can’t remember the exact specification but I think that it is a 1.8Ghz Sempron processor, socket 754 I think. 2 Gb RAM and whatever cheap graphics card I had lying around. I think it might be a Radeon 7500. I don’t remember the make of the motherboard but I think that it is a cheap one.
Firstly the USB ports are temperamental, they will just stop working for a few days and then come back. They will refuse to work with some devices then work perfectly with another. I bought a PCI USB card to try to bypass this but that is just as bad now. It likes to kill hard drives as well. Even before I installed Windows Home Server and had XP on it, the hard drives would just die on me. I am on the fourth now I think. It has had just enough life in it to clone it to a new one but with read speeds less than 10 Mb/sec.
The latest set of problems seems to be with the connector software for Home Server. It would be connected fine, then the connection would disappear and not come back. I reboot the laptop and it was fine again. Everything else for connecting to the server would work, shared files, remote desktop etc. I could even login through the connector. The indicator icon was the only thing showing a disconnection and it wouldn’t run backups. I reinstalled the connector software several times to no avail and eventually had to delete the computer from the Home Server along with all the backups and then add it again. That seems to have worked. I have no idea what caused it though.
On Sunday I lost the network connection. Since there is normally no monitor connected to it, without the network I can’t see what is wrong. I tried rebooting it and it made all the right beeps while starting up. I could hear the hard drive churning away and the light was flashing. All seemed to be going well, then it made all the startup beeps again. I decided that I should get a monitor connected to it and by the time I had unplugged it, moved it over there and plugged it in, the server had rebooted itself again. This time, I watched it boot. No problems at all, went straight to desktop (I have it login to desktop on startup since it starts some backup programs etc. that run automatically). The only thing that was wrong was a message saying that some services had failed to start. I have no idea which ones or why. There was nothing obvious in the Event Viewer and I don’t know where else to look.
The final problem is that other people have trouble connecting to it on the network. I have set guest permissions on the shares but they still get an access denied. I think that is probably more of a configuration problem on my part but then the network in the flat is quite unreliable anyway it seems. I am loath to buy new hardware since we are leaving the country in the summer anyway. The ADSL modem/routers won’t work in the UK and they will all have European sockets anyway.
It’s not really a fault but I am running out of hard drive space. I would ideally like enough to be able to have folder duplication as well but I don’t think that is going to happen any time soon. I have it enabled on a couple of folders of irreplaceable data such as my photo collection but that is it. I haven’t tested them recently but hopefully they have some life left in them. I know that one of them at least is getting quite old. It must be nearly 3 years by now, at least 1/2 of that it has been in constant use.
I think that it might be time to get a new server. It will have to wait until the summer though when I am back in England. Hopefully it will last that long anyway.
Sony Pocket E-Reader
Last week I bought myself an e-reader, I’d been thinking of getting one for some time but never gotten round to it. After a small fiasco with the first one being faulty, seem to have corrupt firmware but I don’t know, I finally got a replacement and it’s brilliant.
The main reason that I don’t read more is the hassle of having lots of books. They take up a lot of space and are heavy. Especially this year in Germany, I would have like to have brought a lot of my programming books with me but I couldn’t due to luggage weight restrictions so I was forced to only bring a couple of them. With an e-reader that isn’t a problem. I can have several hundred all in a pocket-sized device.
After using it for past week I have found that it is far better at novels rather than reference books. I had a few PDFs and while they open and work perfectly. It just isn’t as easy to search for what you want. It can be done, just a little awkward. If you stick to novels then it is brilliant, just open them up and read. It certainly filled the time on my flight back to Germany at the weekend.
The screen is very easy on the eyes, far easier to focus on for a long time than a LCD monitor and since it only uses power to turn the page then the battery lasts a very long time. A small amount of power is used to keep the device on standby but that is all. After a week of use and nearly 2 full books the metre hasn’t moved yet.
On the whole I think that it was a good buy and I’m happy with it. I will probably still be buying printed books for reference though.
Back in Germany
On Saturday we got back to our flat in sunny Munich. The weather reports I have seen were vastly overrated. At best, it is the same as we are getting in Britain. There is some snow, it’s cold but nothing overly extreme about it. Seems that it has been a lot worse further north though.
I would add a photograph of the snow here but I don’t have one so I’ll add one of it from before we left a month ago. It doesn’t look a lot different now.
I’m back at work today. There wasn’t really much happened while I was gone but then most other people were off as well. I made a release just before I left, was about the last thing I did on the last day. Although I rushed it out to fix a lot of bugs that people were struggling with I was dreading that it might have made loads more and I would be inundated with emails when I got back but no, there a couple of minor things but that is it. On the whole, it seems that everything works.
Snow
So we’ve had a little bit of snow here in Britain over the last week. It seems that it has caused the entire country to grind to a halt. We aren’t fortunate enough to live on a road that the council deems important enough to grit. I can’t blame them for it though since there are probably less than 200 people living past the point in the village that they do grit too.
Living at the bottom of a hill doesn’t help, it can sometimes be a challenge to get up it and since the bottom is dead-end then it doesn’t help matters for getting out. Still, we seem to have managed it. I’m going back to Munich on Saturday to start work again on Monday and I don’t see there being any problems over there with the snow even though they have undoubtedly had more, but they are used to it.
Below are a few photos from around the house. The snow is about 2” deep. We keep having snow showers occasionally but nothing major. We seem to be in a bit of a black spot here for weather and miss out on all of it, whatever it is. It is never as cold, warm, wet or windy as the surrounding area. Go 10 miles in any direction and there is a foot or more of snow.
I hope that the airport is open on Saturday, I don’t really fancy being stuck in the airport for a few hours waiting for my flight because the runway has been closed. It is quite late in the afternoon so it may be the following morning before we go if it is delayed.
Despite all this, the weather in Munich looks to be much worse. It is colder and has more snow than here.