Thinking rationally about big numbers

One difficulty you see in both politics and everyday life is in comprehending the meaning of numbers; things like "doubling your risk", "as many as 2 million British residents affected", or even "just 50p a day" get thrown around, and you are expected to somehow understand what these numbers mean to you, and whether they should or should not influence your behaviour. I'm not going to look at how these numbers get abused; I recommend Ben Goldacre's Bad Science and Mark Chu-Carroll's Good Math, Bad Math for that.

Instead, I'm going to discuss techniques you can use to help turn overwhelming numbers that you can't get a good grip on into numbers you can understand intuitively. What do I mean by intuitive? In this case, it means that you've got them into the range of numbers that you encounter every day and that you can reason about - this means small fractions (nothing much less than 0.05, or one twentieth), and small numbers (nothing more than a few dozen). It means bringing numbers that refer to big groups down to talking about the group of people you know personally. It means that you've got numbers where your instinctive feel for what a number means actually works properly.

So, let's start with an example; in their manifesto, the BNP claim that there are 300,000 to 500,000 third world immigrants to Britain every year. This is too big a number for me to comprehend instinctively, so my first reaction is to shrink it. The population of the UK is around 60 million, so the BNP's number is less than 1% of the population.

This, however, is still an awkward number to think about; I don't normally think about 1% of a person. So, let's look at this a different way; in a typical week, how many people do you interact with who you'll still recognise a week later (including things like the helpful checkout assistant at the supermarket, whose name you'll never remember, but who you'll recognise by sight)? I reckon there's under 50 of those in a week for me; so, the BNP statistic reduces down to half a person extra in my week who I might recognise in future. Even thinking about the number of people I interact with in a year who I'd still recognise later, I struggle to get to 100 people, which reduces the BNP statistic to "less than one person a year in the size of group I understand".

Once you reduce the scary big number to that, it's not so scary - you can now get into a more meaningful consideration of the number - is an extra person a year in the group you interact with something the country can absorb? Are they being absorbed into the general ebb and flow of British citizenry, or are you seeing Third World ghettos appear in your neighbourhood?

For a different example, imagine newspaper headlines telling you that getting up at 6am and going to bed at 6pm halves your chances of dying from a heart attack before you're 60. It sounds like you should change your habits to match the study; a 50% reduction is huge. However, on further reading, it's not nearly as impressive as it sounds: 94,000 people die of a heart attack in the UK each year, in a population of 60 million, which works out to around 0.16% of the population, or 1 person in every 600, each year. This includes people who die of heart attacks while over 60, which biases the analysis a little, but we'll continue with the known-faulty figure. A useful way to think of this is that in a typical group of (say) 10 people - yourself and your closest friends and family - you'd expect to get through 60 years before one of the group had a heart attack. If you make the group a bit bigger (say 30 people), you now only expect to get through 20 years before someone has a heart attack.

A halving of risk in this case doubles the time to expected heart attack, so our big group now has 40 years between heart attacks, not 20. And, of course, this figure is based on a known-bad assumption; we assumed that under-60s were as likely to have a heart attack as over-60s. However, we know have a more useful way to think about the headline claim - it means that a group of 30 people who follow the advice go from losing a member every 20 years due to heart attacks, to a member every 40 years. Thinking about it this way, you may decide you prefer not to worry about making drastic lifestyle changes.

Two examples, two different sets of reasoning; what's the link? In both cases, I started with a number I couldn't intuitively handle; either too large, or too abstract. I used external statistics to convert it from a raw figure to a probability or proportion I could apply to a group. Finally, I applied the probability to a group that I could picture in my head (all the people I deal with in a week, a group of my closest friends), instead of to an abstract group that I can't really think about. Once I'd done this, I had a number I could reason about rationally, and where my intuition about what it meant matched cold logic, instead of a scary number that I couldn't handle.

On freeloaders in the benefits system, and reducing the offence they cause

One complaint that appears in the popular press from time to time is that the welfare state is too generous to undeserving people, while not being generous enough to hard-working citizens.

It's an unfortunate reality that no matter how careful you are in designing your welfare state, you're guaranteed one of three outcomes:

1. Some people freeload and get away with it, because there's nothing in the system to stop them.
2. Some deserving people don't get the help they need, because the system says no.
3. The system costs a huge amount to administer, and is much more expensive overall than a system that permits freeloading - you can easily end up spending hundreds of pounds on administration for each pound of freeloading prevented.

I hope that people would agree with me that the third option is absolutely insane - there comes a point where pragmatism says that you shouldn't spend hundreds of pounds just to guarantee that a few pennies don't get given to people who don't need them. This leaves us with just two choices; I would personally prefer that deserving people don't miss out, which means that I have to tolerate freeloaders.

Given this state of affairs, what can be done to ensure that freeloaders don't upset people? First, we need an understanding of why freeloaders upset people; I believe that, in large part, people get upset because they work hard "for what they have", while freeloaders appear to get more by not working at all. We therefore need to ensure that freeloading isn't going to leave you better off than working.

Of course, I have an idea to solve this; simply put, we have plenty of benefits that could easily be paid to everyone, regardless of need, with those who don't need them paying them back in taxes. For example, the highest rate of Jobseeker's Allowance is £65.45 per week per person (around £3,500 per year). The basic rate of income tax is currently 20%, but by removing the personal allowance completely (replacing it with what used to be JSA), you claim back £1,500 from basic rate tax payers; you could then increase the basic rate to claim back the remaining £2,000, or accept that by pushing more people into the higher rate tax band, you're getting more income tax. Further, because people continue to get the same rate of benefits whether they work or not, you encourage people who don't work because they're scared of losing their benefits, or who can't stick out a job for more than a few months at a time to work, and thus pay taxes.

As a second upside, you reduce the amount of administration needed; if you look at all means-tested benefits, and remove those that can fairly be paid to everyone and reclaimed via tax, you reduce the number of people you need to administer the benefits system. There will still be exceptional cases (disability benefits for one), but they're fewer and further between.

There are, of course, downsides. For one, you need to be good at catching out tax dodgers; if people hide from the tax system in the black market, the maths stops working out. You still have a social problem; while you're now always better off working for a living, people will still resent carrying people who don't work, even though they should. I do, however, believe that this system will lead to less resentment, and even, possibly, more employment overall (it becomes possible to work for a couple of hours a week, without losing out on your "bennies").

A proposal for Parliamentary reform in the UK

Anyone in the UK will be aware that we've got a general election coming up, and thus that our parliament is suspended until after the election; one of the many things we were promised that hasn't materialised is full reform of the upper chamber (the House of Lords). This, then, is my modest proposal for reforming Parliament to better reflect the modern UK.

First, a brief look at what we have now. Parliament is split into two houses; Commons, the elected MPs, and Lords, who are appointed for life (in theory by the monarch, in practice by the Commons). New legislation can be proposed in either house; it's debated and amended by the house in which it was proposed. Once it's approved, it's sent to the other house, where the process is repeated by a different group of people; eventually, both houses approve the legislation without amendment, and it becomes law.

There is a tie-break process in place (the Parliament Act) to ensure that the Lords cannot completely block legislation. If the Commons can agree to a proposed bill three times without amending it themselves or accepting amendments from the Lords, the Lords are not given the chance to reject or amend the legislation again.

By and large, this setup works. The Lords act to balance the Commons' short-term thinking, but cannot hold back a Commons that is determined to pass a particular piece of legislation. The back and forth process between the two houses ensures that most legislation gets discussed properly, and that it's possible for Commons MPs (who are under pressure from voters) to say "we tried to get your knee-jerk legislation in place, but it got watered down by the Lords".

There are, however, two issues that I see with it:

1. The Lords is often reactionary and disconnected from the populace; this leads to media discontent with the Lords. I'm not entirely sure that this is an issue worth solving, as it's more about perception than reality.
2. By and large, the flow of legislation is entirely one way; the statute book grows, but does not normally shrink. Ignorance of the law is no defence in English law, so as time goes on, it becomes harder to remain a law-abiding citizen.

Naturally, I have proposals for fixing both of these; I will start with the second, as I see this as a bigger issue.

Firstly, we need to think about why the statute book keeps growing; there are many causes for this:

• Statute starts to cover more and more things that were previously not a problem. For example, there are no 17th century laws regulating driving on the motorway, or mobile telephony.
• Statutes get introduced to cover things that are a concern at the time, but which are no longer relevant; for example, we still have statute covering the militias, which have long been replaced by the modern Army.
• Statutes are introduced to cover a very specific problem; after a while, we work out that the specific problem is just one example of a more general problem, we legislate to cover the more general problem, but the specific legislation remains.
• Statute covers something that, at the time of introduction, is believed to be a problem. As time goes by, and society adjusts, we stop enforcing statute, as we discover that there is no problem; however, the statute remains, and can be suddenly enforced at any time. Enforcement of the statute will surprise people.

It's clear that some of this can't be avoided - we need new legislation to handle problem that never existed before, and it's not surprising that general legislation overtakes specific legislation as the true scope of a problem becomes clear. However, some of it is simple waste; once we have the general legislation, we don't need the specific legislation. We don't need legislation that refers to a time gone by, and that's now irrelevant. We should lose legislation that's no longer enforced.

So, on to my modest proposal to fix this: all legislation should have a built-in expiry date. If it's not renewed by the expiry date, it comes off the statute books, and, if it's still needed, it must be reintroduced. In order to stop important legislation falling off the books, I would put a duty on the Lords to review all legislation that's approaching expiry.

Obviously, there are details to work out; what expiry date should be put on legislation? If it's too short, the Lords wastes time renewing legislation again and again; if it's too long, it doesn't keep the statute book clear. I would suggest that the ideal unit of time is the lifetime of a parliament (typically 5 years, although it can be shorter). By making the expiry the end of a parliament, we give the Lords time to renew legislation before it expires. In order to keep statute changes flowing through, I would suggest setting the expiry based on the vote that passed the legislation, according to the table below:

Size of majority in Commons	Size of majority in Lords	Expiry of legislation
50% or more of the MPs who voted	50% or more of the MPs who voted	End of the next parliament (circa 5 years)
75% or more of the MPs who voted, and at least 50% of all MPs voting	50% or more of the MPs who voted	End of the parliament after next (circa 10 years)
50% or more of the MPs who voted	75% or more of the MPs who voted, and at least 50% of all MPs voting	End of the parliament after next (circa 10 years)
75% or more of the MPs who voted, and at least 50% of all MPs voting	75% or more of the MPs who voted, and at least 50% of all MPs voting	End of the 4th parliament after this one (circa 20 years)

This ensures that legislation which MPs can be persuaded to care about lasts longer than legislation that doesn't particularly appeal to their sense of duty; for example, murder laws would almost certainly get renewed for close to 20 years at a time, as would a new bill protecting people from unlawful killing, while "special interest" legislation is unlikely to last much more than 5 years at a time.

The renewal process would work much like the process for introducing new legislation; someone brings the bill for renewal before the house, it's debated, voted on, and eventually, passes; the size of majority affects when it next comes up for renewal, just like new legislation.

I would expect this to gradually reduce the size of the statute book; because many laws are non-controversial (and thus will get renewed without question), it reduces the time available to produce new laws. Laws that should be repealed will face fresh scrutiny on a regular basis, and, with any luck, the resulting reduction in rate of change of legislation will enable any citizen of the UK to memorise all the statutes that affect them.

So, onto the less important reform; making the Lords democratically accountable. There are 646 seats in the House of Commons. I would make the Lords the same size as the Commons, but, instead of electing each seat using first past the post, I would group Commons seats into 38 groups, each of which returns 16 members to the Lords. These members would be elected by the constituents of the 17 seats they represent using single transferable vote; a system in which electors mark their preferences in order, and which aims to elect the least objectionable of the candidates. Elected members survive 4 parliaments, and each constituency would be expected to elect in groups of 4 at a time at the same time as they elect their Commons MP.

This results in members of the Lords being able to take a much more long-term view than Commons MPs; they can expect to be in office for 20 years at a time, to the 5 years of a Commons MP. It also reduces the turnover, resulting in a lot of continuity in the Lords. STV also tends to prevent tactical voting - you can afford to put the virtually unelectable candidate you agree with before the very electable candidate you just about could live with, knowing that if you're right, and they don't get elected, your vote still helps prevent the candidate you couldn't live with from winning.

This leaves 38 seats empty, as against the Commons. I would increase this by 2, to get 2 more Lords than Commons. Again, these 40 would survive 4 parliaments at a time, and again, I would replace in groups of 5, so that there's a slow turnover. However, these 40 would be appointed by the incoming Commons administration; their role is to ensure that there are people in the Lords who can act as a link between the administration, including past administrations whose laws are up for renewal, and the Lords, to explain the rationale behind Government bills in Lords debate.

As you may have gathered from this ramble, I do have an interest in politics. I would be interested to hear other people's views on reforming UK politics - including explanations of why it's not needed.

How a 1980s telecoms compromise helped set the Bluray video format.

I've mentioned the ATM committee's weird decisions before, when talking about ADSL; this post is meant to make you think about how seemingly small committee decisions can have long term impact.

To recap; in the 1980s, telecoms engineers were setting standards for "fast" data links (45MBit/s and above), to be used to carry voice, fax and data; the decision was made to use small cells, so that even slow links could use the same standard. The resulting standard, ATM, has a 48 byte payload and a 5 byte header on every cell.

There's only one standard set worth considering if you're interested in serious video; the MPEG standards. Both DVD and Bluray are built on MPEG; DVD uses MPEG-2 exclusively, carrying video and audio in an MPEG-2 program stream, extended to add timecode to packets, resulting in the "VOB" file format. This has limitations when it comes to seeking; when an optical disk player seeks, it moves the read head to a location that's approximately right, then reads the disk until it finds the timecode it's after, then either moves the heads again (to a better estimate of the correct place), or resumes playback. Because program stream packets are variable-length, the player can end up reading a significant amount of data, only to discover that the timecode tells it that it's badly off, and it has to seek again.

Bluray escapes this by using MPEG-2 transport streams as the container, but with a 4-byte timecode added to each transport packet. Transport stream packets are fixed in length at 188 bytes, so a Bluray player never needs read more than 192 bytes after seeking before it can decide whether it needs to seek again to a better guess at the correct location, or whether it can just read to the right point.

188 bytes is a rather unusual number; the header is 4 bytes long, and the payload length of 184 bytes is neither a nice number for humans, nor is it a nice number for computers to deal with. So, why did the MPEG-2 committee choose 188 bytes for transport stream packet size? It all comes back round to ATM; when MPEG-2 was being designed, ATM was the telecommunications networking technology of choice, and it was considered important that you should be able to easily carry MPEG-2 transport packets in ATM.

There are two sensible ATM Adaptation Layers for MPEG-2; AAL1, meant for constant bit rate services (such as carrying a multiplex from BBC headquarters to the Freeview transmission sites around the country), and AAL5, for services that can cope with a long delay. AAL1 takes 1 byte from every cell's payload, leaving 47 bytes for the user; 47 times 4 is 188, which is where MPEG-2 gets 188 bytes from. It also works well for AAL5; AAL5 uses 8 bytes from every group of up to 1366 cells, leaving 40 bytes in that cell and 48 bytes in the remaining cells in the group for user data; two 188 byte packets plus 8 bytes of AAL5 overhead fit precisely in 8 cells with no padding.

So, to interoperate well with ATM, the MPEG-2 guys chose 188 bytes for their fixed-packet-length container. MPEG has been wildly successful, so that the Bluray guys didn't want to design their own container format. As a result, a compromise between France (who wanted 32 bytes payload), and the USA (who wanted 64 bytes payload) has influenced the design of the most modern consumer video format to date. Next time you're compromising on a technical issue, think hard; your compromise may live on longer than you expected, and affect more people than you thought it would.

Ada Lovelace Day - Valerie Aurora

So, it's Ada Lovelace Day. The day on which geeks everywhere are asked to point out geeky women who've inspired them. I would love for this to be an irrelevance, and for no-one to care that much; unfortunately, the geek community is still male-dominated. I've therefore picked on Valerie Aurora as a geek who's work inspires me, and who happens to be female.

Why do I find Val's work inspiring? In part, it's inspiring because she tackles important problems; not necessarily interesting problems to the general population, but problems that need to be dealt with now. However, there are lots of programmers who do that; what makes this one special?

The difference in Val's work is the clarity of explanation she produces. Most programmers struggle to explain their work to other programmers in the same field; if you read any of the publications linked on her homepage, you'll note that she writes clearly and concisely in English, not just in the usual set of programming languages. Combine that with the high technical standards maintained in the publications and in Val's work on things like the Linux kernel, and I'm impressed.

To top all this off, she's written a very good document aimed at encouraging women to break out of unfair social constraints: HOWTO negotiate your salary. Often, all it takes to break a bad cycle is someone to notice it, and point it out to the people involved; this document is a perfect example of how to make a difference.

It's often said (with good cause) that when members of a previously underrepresented group enter a field, they have to be not just good, but noticeably above average. Val's work fits this stereotype, and sets a standard that I hope to be able to meet one day.

DIY garden replacement

People who know me well know that about 3 years ago, my wife and I bought a house in need of a lot of TLC. This week's decision has been that we should redo the garden before hay fever season makes it horrendous to go outside.

We've got a lot ahead of us. The grass isn't appropriate for a lawn, there's stones everywhere, and Rachael wants a vegetable patch. The current plan of campaign is to use two weekends to do it.

Weekend 1 plans

• Hire a sod cutter and remove the entire top surface of the garden.
• Tear down the remains of the shed.
• Place new turf over the lawn area.
• Cover the area that's going to be vegetable garden with plastic sheeting.
• Cover the area that'll be used to extend the patio with plastic sheeting.
• Cover the area that'll be used for a new shed with plastic sheeting.

Weekend 2 plans

• Buy a new shed, cement, and a taller pole for the satellite dish.
• Cement in the tall pole.
• Move the satellite dish and recable.
• Lay the shed foundations.
• Put up the new shed (just for storage space, no need for power/light).
• Lay extra patio.
• Start putting in raised beds (large) for the vegetable garden.

As you can see, this is a fairly hefty work plan; I hope we're up to it...

Calculating the number of VoIP channels you can fit on an ADSL line

A question that occasionally comes up in the #A&A IRC channel is "how many VoIP channels can I fit on one ADSL line?" The answer is quite simple to calculate, and you can do similar arithmetic for other realtime traffic:

We start with some general knowledge; people who read my previous post on ADSL overheads will be aware that ADSL sync rates are the number of ATM cell bits you can fit down the line. ATM cells are 53 bytes long, so you divide your sync rate by 424 (bits per cell) to get the cell rate. So, all we need to know is how many cells per second in a VoIP call. The table below has details of all the bits in a single 20 millisecond packet of G.711 (a-law or µ-law) VoIP:

Bytes	Reason
8	AAL5 trailer
2	PPP framing for PPPoA
20	IPv4 header
8	UDP header
12	RTP header
160	G.711 voice data

This adds up to 210 bytes; cells carry 48 bytes each, and can only be used for one packet, so we need 5 cells, with 30 bytes padding. There are 50 of these packets in a second of VoIP, so each call needs 250 cells/second, or 106 kbit/s of sync rate.

Because we have 30 bytes of padding, we can deduce that the extra 20 bytes overhead of PPPoE won't hurt us; nor will the extra 20 bytes header size of IPv6. However, if we use both IPv6 and PPPoE, we will need one more cell per packet, making it 6 cells for one packet, or 300 cells/second, or 127.2 kbit/s of sync rate per call.

We know we need a little spare capacity for signalling and monitoring, but from this, we can deduce that a 448kbit/s ADSL upstream (IP Stream Standard) can support 4 calls, or 3 if you're using both PPPoE and IPv6. An 832kbit/s upstream supports 8 calls, or 6 if you're using both PPPoE and IPv6. The 2MBit/s I get on Annex M supports 15 calls with PPPoE and IPv6, or 18 calls with either IPv4 or PPPoA.