Testing Times

What is the point of exams?

Is it to test the candidate's knowledge of the subject or is it to test the candidate's skill in passing exams? In theory at least, it should be the former but when there is a lot riding on the result there is a tendency towards the latter with students placing more emphasis on training to pass the exam rather than gaining a deep understanding of the subject. The student may gain some knowledge of the subject, but only those parts that will gain most marks in the exam as part of a strategy to maximise the result for minimum effort.

This doesn't only apply to academic exams.  Similar strategies can also be adopted for all sorts of tests.  For example, if a car maker wants their car to get a good fuel efficiency rating they can work on scoring better in the test, rather than improving the car's performance.  They can use non-standard high performance lubricants, disconnect the alternator, over inflate the tyres, remove door mirrors and even tape up joints in body panels to improve the test results. Car manufacturers have got so good at performing well in tests that the average gap between quoted fuel efficiency and actual real world fuel efficiency has grown from 7% in 2001 to 23% in 2011 according to figures from Germany.  In fact the real world fuel consumption has been shown to be as much as 50% higher than the official test figures for some vehicles.("Mind the Gap:Why official car fuel economy figures don't match reality")

While no rules or regulations are broken by manipulating the test results, there is certainly an intent to deceive or mislead the consumers and governments (some of which set taxation levels depending on the results of the tests).  This approach to testing taken by car makers could account for as much as half of the fuel efficiency improvements between 2002 and 2010 according to a study undertaken for the European Commission. This makes it difficult for car buyers to make an informed judgement about which models are more efficient. There is, apparently, further scope to improve test results without improving the performance of the production car.

Once you've bought a shiny new supposedly fuel efficient car it is fairly easy to compare your real world fuel consumption with that advertised.  To calculate your fuel consumption:

1) Fill the tank completely and note down the odometer (mileage) reading;

2) Fill the tank again after a few hundred miles, note the odometer reading and the amount of fuel used to fill the tank;

3) Calculate fuel consumption in milers per gallon (mpg):

       [(2nd Odometer reading) - (1st Odometer reading)]/(fuel used in litres/4.55) = mpg

Once you have calculated your fuel consumption and found it to be significantly greater than advertised, what recourse do you have? Under trading standards regulations, you would expect recourse if the product differed from that advertised and advertising standards, manufacturer's must be able to justify their advertised claims.  Unfortunately the consumer loses out in both cases as the manufacturer has testing data to justify their claims and they do not advertise the real world performance or the performance that you should expect, only the performance under a set of loosely defined test criteria.

The Future of Fuel Savings

The European Union is considering future emissions from new cars including setting a target to 95 g CO2/km on average for new cars by 2020.  According to some reports motorists will save £350 per year on fuel thanks to the new emission levels, which will pay for the additional cost of the new car in three years.  Presumably this is based on average mileage and real world fuel efficiency improving at the same rate as the quoted tested efficiency.  If real efficiency savings are half of the test values then the pay back time would double and, for those who drive less,  there may never be enough savings of fuel to cover the additional capital cost of the car.  This is not the incentive we need to reduce fuel consumption.

Media Perception

The New York Times recently reported on a road test of a Tesla electric car, in which it highlighted major shortcomings in its energy use performance, specifically its range.  This clearly of much greater significance than the fuel efficiency of a petrol or diesel car as they can be quickly and easily be topped up with more fuel  at one of many filling stations unlike the electric car which takes time to top up the battery at much less common charging points.  Nevertheless, the emphasis on fuel consumption seems disproportionate with the media generally accepting manufacturer's performance claims, perhaps accepting that real world fuel consumption may be a bit higher than test conditions but certainly not emphasising the growing disparity between quoted performance and actual performance.  One may suspect that they do not want to challenge the car makers to much in case the supply of shiny new toys to test drive is curtailed...

Conclusion

In conclusion, we need:

1. A more accurate test that better simulates real driving conditions and is based on sample cars off the production line or from the forecourt without any modifications.

2. More robust challenging of fuel efficiency claims across all car types by the motoring press.

3. Genuine increases in fuel efficiency.

In addition, we must also focus on alternatives to the car, such as active travel, public transport, private group travel and reducing demand for travel, some of which will be covered in further posts.