Web Browser

It's been a bit of a gap since my last post, I managed to get sidetracked to doing some work on a website I setup last year .. which involved lots of php, mysql and javascript. So now I'm back ready for some 'proper' coding I needed to do a refresher on c++, so I've decided to have a quick go at a second version of a web browser I wrote a couple of years ago.

The 'skanky sea dog' web browser is just a bit of fun really .. I'm just doing it as a little learning project so I can learn the details of how html / web servers work. The first version was very simple, it downloaded the html for a page, and did some incredibly basic processing of the html to show some text on the screen, and download some of the images (more a proof of concept).

This time, after learning a bit of css and javascript, I have a bit better understanding of how the html DOM (document object model) works, and as it is a tree structure, and I have a fair bit of experience at dealing with tree structures (they seem to crop up everywhere in coding), I thought i'd give it a go at parsing the html into a tree of c++ objects, with different types for different html tags.

The parsing was actually not all that tricky, I had earlier written an xml parser, so I modified it to get some useful functions for html parsing, then allowed the tree to 'build itself' by parsing the html - i.e. it encounters an [html] tag, it creates an html node, and begins parsing for child objects within this node. When it finally encounters the [/html] tag, it is finished with the node, and moves up the the higher node on the tree (in this case the document node), until all the document has been processed.

For rendering, I knew that I had to somehow implement a version of the html 'box model', i.e. child elements determining the size of parent elements, or parents constraining the size of child elements etc. I have no idea how firefox and IE handle this, but I have done it using the old trick of traversing the tree.

I do several passes, down the tree to the leaf nodes, then up again to the root, doing different operations on each pass, gradually refining the box models for each element (things like widths, heights, minimum width, desired width, offsets etc).

This seems to be working, and now I'm doing some refinements for tables to allow the columns and rows to line up.



For rendering, I wanted ideally to make things cross platform, but as I'm only familiar with windows, I've tried to separate things a bit for the rendering code. Each html element has it's own win32 window. Whether that's a good or bad idea I don't know yet .. I basically had the choice to draw everything manually, or rely on the win32 techniques. While I like doing everything manually, win32 normally makes it an enormous pain the ass to do anything manually, so I'll go with the flow for now and see what happens.

Incidently finding the sizes of image elements is easy .. text elements is more tricky. I used things like getting the text metrics and finding the pixels used by each word in turn to determine the minimum widths and desired widths for text elements. In fact in some elements (e.g. [div] or [body]) you can have text elements intermingled with images etc so it is slightly more complex determining widths etc but it is very doable.

I get the feeling that while the overall structure of the browser code is quite well laid out, the intricasies could quickly become a bit of a rats nest because I think there will be so many 'special cases'. This may be part of the explanation for the differences in behaviour between Internet Explorer and Firefox, however I think there is probably some fundamental difference in their box model calculation method which leads to their 'quirks'.

Egor videos

After a long gap, I've finally got back to doing some more work on Egor. It's actually been really good having a rest, and coming back with a fresh perspective.

Anyway I did a couple of videos this morning, showing some of the basics, hope you like!

The Solution to Spam

I was just reading today on the register about how spammers have defeated the CAPTCHA protection to stop automated registrations with the major email providers.

http://www.theregister.co.uk/2008/03/11/global_spam_trends/

Some time ago it struck me that there may be quite a simple solution to the whole spam issue, in fact I think my current ideas are based on a suggestion by none-other-than Bill Gates(!), who I think suggested having a 'stamp' or small cost associated with sending an email.

The idea is that if you can introduce a cost to sending email, no matter how small, it will deter spammers because when you are sending millions of emails, these costs rapidly add up and make the whole thing unprofitable.

I think this idea is really good, the variation I currently am thinking might be successful (I can't remember whether I read this or just came up with it after Bill's suggestion) is as follows:

Instead of having a fixed cost per-email, have every person creating an email account make a DEPOSIT. That is, a deposit of good faith to indicate that they are not going to use the account for spamming. Now the idea is, when someone receives an email that they regard as spam, they mark it as so, and through 'an undetermined mechanism', the sender loses their deposit.

There are 2 obvious variations here:

1) The deposit is large (say 10 pounds) and the sender loses access to their account on confirmation of spam activity.

2) The deposit is small (say 1 pound) and the sender loses the ability to send mail until they REPLACE the deposit.

The obvious problem is the problem of abuse by the receiver of the message. If they don't like you, or want to play a joke on you, they could classify your mail as spam and e.g. lose you a pound, which is unfair. So in a way you would need an impartial human to vet the reported spam and check it before giving the penalty.

Anything where there are humans involved becomes more costly. BUT WAIT!! If the spammer is losing their deposit, you can actually use this money to pay the checkers!! :)



It has occurred to me you could use a similar system for telephone spam too, if you receive an unwanted call, you simply press your SPAM button on the phone, and the caller loses their deposit.

Of course the issue is that a new email protocol would need to be used in order to prevent spoofing the sender, but if we bear in mind how long the existing system has been used, is it unreasonable to expect a revision, based on the experiences of use over the past 30 years? Any system undergoing widespread use usually shows flaws which can be corrected in a revision to the standard to make it more robust.

There is actually no reason why a new email standard could not be used in parallel to the existing system for a number of years, and if the benefits are significant, the market will adapt to using it.

Music Composer App

As a little side project I've been writing a music composing application. I've been planning to for a few years now, just never got round to it! Actually I've learned a bit more win32 programming since I last did a music app around 10 years ago, so things are coming along quite quickly. Having said that, although I've done a lot of programming I haven't done a lot of windows GUI type coding, partly because lots of flashy user interface stuff doesn't interest me as much as the things going on underneath... so the user interface so far is pretty basic lol! Of course once I have the basics working I can photoshop up some graphics and make it look more flashy.

My aim is to produce something like fruityloops (FL studio) in functionality, but more geared towards composition. In the past I've found many sequencers very annoying because although they tend to be very versatile, and let you program any music, they tend to make it a very tedious process that takes many hours. I instead want a system designed for rapid composing, with lots of helpful tools and systems that are 'composer friendly' instead of being 'tech friendly'.



The other thing I used to find really annoying back in the old days of MIDI and multitrack tapes was the nightmare of getting everything in sync. I completely solve this problem in my app by placing samples / instruments in precisely calculated accuracy, sample accurate so accurate to 1/44100th of a second typically. Of course to get bang on sync, you also need to make sure your samples start on the B of the Bang, so I have tools for simplifying this. The interesting case is for instruments with a slow attack (such as a slow bowed string sound). To get this tight, should you chop off the attack, or start playing the sound BEFORE the 'note start'? Some interesting questions that a human player would do naturally, but a computer needs a plan of action for this type of thing.

The saving culture

http://news.bbc.co.uk/1/hi/health/7214709.stm

Elderly and disabled people in England are increasingly being denied social services, a report says.

The Commission for Social Care Inspection said councils were tightening their criteria which determines who is eligible for care.

The watchdog said the situation meant there were 275,000 people in need of help receiving none while another 450,000 suffered shortfalls in care.

The BBC have on this topic run a 'have your say' section so people can put forward their thoughts. Most have complained that the government should do more to look after old people, and how they have paid their taxes etc.

While I think it's right people complain (and they should, to bring attention to such issues), from what I have read the problem from the post war years is that the proportion of the population that are elderly is increasing as we are leading longer lives with medical advances.

The big question seems to be - who should be responsible for looking after an individual when they reach an age when they cannot look after themselves? Should it be the state? Should it be their family? Or should they have saved over their lifetime to provide finance for their care in old age?

My own belief is that in the UK the great thing that is lacking is the saving mindset. Those that are financially successful longterm tend to have the saving mindset, and those that end up depending on state benefits tend (on average) to not be so good in this area.

For myself I have been lucky in this respect ... firstly for being brought up by my parents as a saver, and secondly having their financial support. I guess as everyone 'comes from somewhere' it makes it hard to have unbiased views on the topic. But objectively speaking I find it sad that some of my friends that run into financial difficulty, are in a sense 'addicted' to spending what money they do have.

Over a lifetime, in a vaguely capitalist system, I believe that people should be encouraged to save and build up their savings, so that they can form a buffer to look after themselves and their loved ones in difficult circumstances. The most unfortunate thing, coupled with the mindset problem mentioned earlier, is that governments (particularly the UK government) routinely penalize people for saving.



People who have very little and depend to a certain extent on extra benefits are hugely discouraged from saving. The moment they start to put away money each month for their longterm benefit, the government will correspondingly reduce any state benefits they receive (childcare etc). This means in practice that saved money is wasted money - the individual sees no advantage. Instead if someone has 200 pounds left over in any given month, they are better off spending it on a TV set or some asset, because this is not 'counted against' the individual in the decision as to state benefit rates.

Hence you get the bizarre situation where thousands of people living off benefits have council houses stocked up with the latest hi tech gadetry, playstation 3s, xbox 360s, plasma TVs, sky etc etc... to the extent that they have more gadetry than many middle income self supporting familys!! It is bizarre but makes total sense, given the benefit system.

Another example of such a problem is the situation faced by many single mothers. They can end up in a situation where financially they are better off not working, than working! Many examples I have found where single mothers do work, is not in order to increase income, but in order to feel as though they aren't dependent on a state handout. Indeed often until a single mother is earning quite a considerable amount (well over 20K) there is no significant financial benefit to working!!

While state benefits should provide a backup solution to help people in need, there is clearly something very broken in the UK system, where individuals see no personal advantage in getting themselves off the breadline. Capitalism works only when people will see a reward for their effort. If you want to get the millions on the breadline making some contribution to their own welfare, you simply have to make it worth their while.

Straight A's no longer enough for top universities

I see they are introducing a new 'A*' grade for A levels, because so many pupils are getting A that the universities can't select on that basis:

LONDON (Reuters) - Achieving three A grades at A-level will no longer be enough to ensure a place at a top university, academics warned on Wednesday.

From September sixth-formers will begin studying A-level exams which will include a higher grade of A* for those getting marks of 90 percent or above in their papers.

http://uk.news.yahoo.com/rtrs/20080123/tuk-uk-britain-education-exams-fa6b408_3.html

Of course it's not anything to do with the schools being ranked on their grade performance, obviously teaching has got orders of magnitude better than when we were at school *sarcasm*. Such a magnitude of change is unlikely to be due to genetics, so must either be due to the environment, or the marking system.

It's almost inevitable that this will continue to happen, given that schools are 'marked' in league tables on this basis. What they should do (in addition perhaps) is introduce a mark similar to the IQ mark:

While IQ tests vary considerably, there is a built in 'normalization' for the result. That is, if you give 10,000 people an IQ test, the average mark will always be 100, BY DEFINITION. If it's an easy test, you'll STILL get the same people tending to score above 100, and similar people scoring below 100. So the actual mark in the test is passed through a mathematical function which compensates for the population result, to give a more standardized result (the IQ).

The same process can be applied to any exam, and if applied to A level results would give a clear, fair and consistent means for universities to select. An additional benefit is that this process could be used to correct for the inherent 'easyness' of some subject choices over others.

Thus the current A, B, C etc scale could be used as an ABSOLUTE measure of performance (poor as it is), and a normalized scale similar to IQ could be used as a RELATIVE measure of performance (more suited for selection).

Tesco Online Website

Tonight I'm going to talk about a subject dear to my heart. The tesco online website.

Oh dear.

Those of us in the UK will be familiar with Tesco, one of, if not the biggest UK food superstore chain. Many of us in the UK shop often shop online for our food rather than going to the store. Alright I'm lazy .. but I also don't drive, and carting back a huge shopping haul on a bike is not pleasant. So I am more than happy to pay the 5 quid or so delivery charge to have someone do this for me.

I first started using sainsburys online service, and was quite happy with it. However, I find sainsburys tend to be a bit expensive overall (for myself, I'm not so bothered about paying for premium quality) so I moved over to tesco.

I am usually very impressed by the way tesco picks my food in the store for me and delivers it with nice drivers, doing what can't be a super pleasant job. I am very happy with the service, apart from in one area - the website!

The tesco online website is SO BAD, it's almost beyond belief. Alright it looks very nice, but the problem is, it DOESN'T WORK. Now I'm not a complete newbie to website design myself, I've written several, and have a reasonable knowledge of web technology such as html, css, php and sql.

I am currently stuck tonight with no food, having for the umpteenth time spent over an hour attempting to shop at the website. My main web browser is firefox (like 20% or so of web surfers), and I have a totally up to date install and disabled all plugins (to give tesco online the benefit of the doubt).

If I'm lucky, I can login to the site. However, when I click on a link, for example to show 'my favourites' in order to place my order, 99% of the time the website just hangs. If I click the link again sometimes the site gets very confused indeed, and tries to download an .aspx file to my computer. No, I don't want an .aspx file, I would like to see the website, thank you very much.

After 15 mins trying this and getting nowhere I give up and fireup internet explorer, which I keep for situations like this. I have version 6 of IE (perhaps this is where I am slipping up, not being interested in updating more microsoft bloatware). With all the security turned down to minimum, internet explorer fails to even load the front page :( . Sometimes I have got further with IE on the tesco site, but not tonight.

I went back to firefox. By a stroke of luck I managed to be able to add some items to my shopping cart. My prebooked delivery slot had long since disappeared, probably due to me having to log in multiple times, and delete my cache repeatedly to get anything to load.

So my message to Tesco, the company would be simple. Whoever is in charge of your website, fire them. They are guilty of gross incompetance. A child could build a more servicable website. The problem is probably in part due to the use of .NET type microsoft software in combination with incompetant website design. If you please could, given the huge amount of profit you make every day, please please please hire some competant web designers to make you a working website.

I would advise writing one that doesn't rely on proven unworkable technology, and instead opt for something more commonly used and proven to be scalable. If you really can't do it, I'd probably write the website for you, free of charge, or gladly instruct your website team on the basics of software development.

Here's hoping.

P.S. I may end up having to go back to sainsburys or a competitor who has a website that works. I know this means nothing in the grand scheme of things, and probably tesco online sales are a very small proportion of their operation, but I really like the rest of their system, it just makes me incredibly sad that a large corporation with so many resources can get something so horribly, horribly wrong. :(