Programming
Code in posts
This morning I updated the plugin I use for posting code. It seems to have broken it completely. Hopefully another update will be released soon that will sort this out or I will have to try to roll back to the last version or find something else.
This is probably the 3rd plugin that I have tried to use for code, I remember seeing an add in for Visual Studio that could copy code as HTML so maybe I’ll just start using that. Would probably be easier in the long run. Either way, it’ll have to wait until later.
Update: Abandoned the plugin in favour of plain HTML. It didn’t seem to be a problem in all browsers though.
Thread Starvation in Java
I’ve been working with Threads in Java recently and have come across the issue of starvation. This is where a thread is unable to run because another thread is hogging the locks. To demonstrate this problem I have written a simple buffer. It has a list of objects that can be written and read. The reading thread must wait if the list is empty and the writing thread must wait if the list is full.
1 class Buffer {
2 volatile LinkedList data;
3 int bufferMaxSize;
4
5 public Buffer(int bufferMaxSize) {
6 data = new LinkedList();
7 this.bufferMaxSize = bufferMaxSize;
8 }
9
10 public synchronized T getData() throws InterruptedException {
11 if (data.size() == 0) {
12 try {
13 wait();
14 } catch (InterruptedException e) {
15 }
16 }
17 notifyAll();
18 return this.data.remove(0);
19 }
20
21 public synchronized void setData(T data)
22 throws InterruptedException {
23 if (this.data.size() == bufferMaxSize) {
24 try {
25 wait();
26 } catch (InterruptedException e) {
27 }
28 }
29 this.data.add(data);
30 notifyAll();
31 }
32 }
I wrote a simple producer and consumer to test this. The problem is that until the buffer is full, the producer is starved of access to the buffer and must wait. In the example below where the max buffer size is set to 1000.
While that might be OK for a lot of situations. If you want to have fair access to the resources, then you have to manage it yourself. The notify() and notifyAll() all messages will just wake every/any waking thread. Once the lock is available then any Thread, including the one that just relinquished it, can take the lock. If that happens, then the thread must continue to wait until the lock is available.
Making your own locks rather than using the synchronized keyword will let you control which Thread is woken when a lock is relinquished but this does create an overhead that will have a performance hit. If you want alternate access to a buffer then it may be better to not have a list inside it that can buffer the data and instead just have 1 value that can’t be overwritten until it has been read.
This is only an example that shows the problem I was having but it can show itself in a number of different ways so it is something to look out for. Often using the debugger won’t show this since it is a problem of timing, putting in breakpoints slows everything down and so you can’t really see the problem.
Online security
Following up on Tom’s post about Passwords I have noticed that there is an increasing amount of poor security online. As his post shows, people still use really insecure passwords.
I’m signed up to a website that sends me an email every couple of weeks to tell me about their latest offers and they attach my username and password ‘Just in case I forgot’ I wouldn’t be surprised if this the same website that Tom talks about and really there is no excuse for it.
The same goes for websites that will send you your password after running through the forgot your password wizard on the site. If they are able to send you it then they are doing it wrong. Any programmer who has made a login function for a system, has probably stored the passwords in plain text at some point. The main reason for this is probably laziness. I know that I have done it before. It was the first login system that I made.
It’s not even hard to make your database more secure. Store hashes of the password, MySQL has an MD5 function that will do it for you. Since there are online databases for looking up hashes then add a long random string to the password before hashing it, this is known as salting. That will make it almost impossible that it could be in the database. When the user tries to login, add the random string to what they type and hash it. If it matches the stored hash of their password then they must have entered the correct password.
This means that even the user has really bad password like ‘password1’ then the hash that a hacker might get hold might be of ‘password1ReallyLongAndRandomSaltForThePassowrd’. In reality it would probably be better to generate a random string to use for each user as a unique salt for them. Then store that in the database as well.
There are few more things you need to do, especially on the public websites, but as for passwords that is one of the easiest ways to keep them secure. Since the passwords aren’t in the database then a hacker wouldn’t be able to get hold of them. The random salt drastically reduces the chances of being able to look up the hash. It doesn’t help against a brute force attack though since they would have the salt so it is still important to use good passwords.
I started using KeePass a few months ago to store my passwords. It will generate random passwords for you if you want, but then it can save all your account details in an encrypted database. Then you only need to remember one password. I use the portable version of the software and keep it in my Dropbox along with the database. That way I always have the latest version of the database whichever computer I am on. I also have it on my pen drive for when I am out.
It might be more of a security risk to have them all together behind one password but I find it is much more likely that some hacker would get into my account by hacking the websites and not stealing the password safe and then trying to crack that.
A few years ago I had an insecure password and a forum that I was an admin of got hacked. The hacker found some exploit in PHPBB and was able to gain access the database. Once there, he was able to look up all the users passwords, wasn’t a very busy forum, only 40 members or something. We restored the forums from a backup and then carried on as normal.
The next day he did it again, we had updated to the latest version of PHPBB by this point that fixed the bug he used to gain access last time so we didn’t know how he got in. He then emailed every user a list of every other user’s passwords. We had kept the same admin passwords and he had just logged in. On his last attack he had saved all the hashes and since we all had really insecure passwords he had just looked them up. He started to login as us on MSN messenger and things, just generally being annoying. I, like a lot of the users, had the same password on most of my accounts and he had my email address and most common username so he had basically unrestricted access to everything. I was quite worried about this since some shopping sites save your card details, he could have easily logged into them and spent all of my money.
I, and the other users probably, spent the best part of that evening changing all of my passwords to something more secure, still mostly the same but I checked that it couldn’t be found in a hash database.
Although I was one of the admins of that site though, I could just have easily been a user with no control over how secure my data was. Just one forum that wasn’t updated unlocked the key to my entire online identity. Since then I have learned that you can never be too safe and until someone comes up with a better method of identification than passwords we will have to put up with remembering them and trying to have ones that are secure.
Starting external processes in Java
This is mainly for my own reference in the future but thought that I would post it here as well as I couldn’t really find anything on this while looking it up.
Launching an external process in Java is done through the ProcessBuilder class. Runtime.exec can be used as well but this just calls ProcessBuilder itself.
The constructor for ProcessBuilder takes a String vararg of the executable and the arguments. There is also the method ProcessBuilder.directory(File directory) to set the directory the process is launched from. ProcessBuilder.start() starts the process and then returns a Process object for the process.
ProcessBuilder builder =
new ProcessBuilder("exe", "arg 1", "arg 2", "arg 3");
builder.directory(new File("/path/to/working/dir"));
Process process = builder.start();
The part that I found the be awkward is the way that arguments are handled. You have to have 1 element per argument, as in the example above, having spaces doesn’t matter, the program will still treat it as one argument. Passing it as “\”arg 1\”" is unnecessary, the program will receive it as “arg 1″ with speech marks included. The example where I was having problems involved start ksh and getting it to run a program and pipe that into another. From command line it was:
ksh -c “exec arg | tee exec.log”
However, to run this from ProcessBuilder I needed
ProcessBuilder builder =
new ProcessBuilder("ksh", "-c", "exec arg | tee exec.log");
Where the value of the -c parameter was passed as one argument. Some things I tried that didn’t work were:
ProcessBuilder builder =
new ProcessBuilder("ksh", "-c", "\"exec arg | tee exec.log\"");
ProcessBuilder builder =
new ProcessBuilder("ksh", "-c", "exec", "arg", "|", "tee", "exec.log");
ProcessBuilder builder =
new ProcessBuilder("ksh", "-c exec arg | tee exec.log");
There were a few others as well. The problem is that with the first one of them, ksh would execute “exec arg | tee exec.log” including the speech marks so it wouldn’t find it, just as if you had typed that into a terminal. The second one, ksh would execute exec but then arg would be passed as an argument to ksh rather than exec and the same for all the others after it. The final one, ksh would receive the whole thing as single argument and so wouldn’t recognize it as it is expecting [-c] [command] as 2 arguments rather than one.
Back in Newark
I’ve now moved out of Stafford, need to go back this weekend to get the last of my things but that will be it then. As Tom mentioned in his blog we are all starting our placements soon. He is moving back across the country tomorrow to start on Monday, we don’t start for just over 3 weeks and moving to Munich in 2. It’s strange to think that we won’t be back at uni for a year. I’m still not spending my time doing anything constructive really, I probably should but I don’t really know what to do. I need to complete some games so I might do that, although I am needed to help build some steps in the garden so that might fill a few days.
A while ago I posted about Project Euler, I haven’t done any of the problems on there since about then so I might do that to get me back into a programming mind set before I start work. I haven’t really done any programming at all since we finished our assignments just after Easter, that’s nearly 2 months ago now.
I still intend to get myself a camera and take plenty of pictures while in Munich so some might find their way on here.
Project Euler
I have recently been working through some of the problems on Project Euler. The site has a series of mathematical problems to solve. Each can be solved by writing a program to calculate the answer. An answer of one of the simpler problems is “Find the difference between the sum of the squares of the first one hundred natural numbers and the square of the sum.” and the solution is quite simple shown here in Python.
squareTotal=0
sumTotal=0
for num in range (1,101):
sumTotal += num
squareTotal += num*num
sumSquare = sumTotal*sumTotal
print sumSquare-squareTotal
Some of the problems are much more difficult however, they have a rule that all of the problems should be solvable on a modest PC in under a minute of computation. There are some that I have written where my first attempt would have taken several months to run but I have refined it down to about 10 minutes. There is a forum that you get access to once you have solved each problem. There are posts with other peoples’ solutions in a vast array of languages which can be quite interesting. That particular problem had a solution that ran in under 10 seconds.
There is well over 200 problems and rising so it should be enough to keep anyone busy for a while. I will post any interesting solutions as I come across them.