Do our state Electoral Commissions have enough to do the job?

Last week, while we were all distracted by the NSW election, the Tasmanian Electoral Commission (TEC) made a submission to a parliamentary inquiry warning that cuts to its funding may affect its ability to administer Tasmanian elections.

The TEC has been hit by funding cuts from the Tasmanian state government, down $35,000 in the last year. The organisation also relies on fees from local councils to fund its operations, and these fees have been reduced due to council terms being extended from three years to four.

Australian elections have genuinely been conducted to a very high standard, and a big part of this standard has been the administration of elections by independent, competent electoral commissions at a state and federal level. Yet government cost-cutting threatens this high-quality standard. Electoral commissions need to not only be independent, but have the resources to not just do the bare minimum when running an election, but to ensure that redundancies are built in to the system to catch problems, that staff are well-trained, that mistakes are not made, and that the entire process can be conducted in a way which is above reproach.

When you cut budgets, you threaten this capacity. Sure, electoral commissions will continue to do the basics – set up polling booths, take votes and count them – but the things that will take a hit will be the things you don’t always see, and they could well be the things that cause problems when everything doesn’t go to plan, which happens quite regularly in such a big operation as running a large election.

We’ve seen the problems with the NSW iVote system in recent weeks: errors on the ballot paper, security problems, and problems with small parts of the ballot paper being visible on the screen. I’ll return to iVote tomorrow, but it is part of the story of cost-cutting. iVote was in part very popular amongst overseas voters thanks to the closure of numerous overseas polling places in major cities.

While the standard of elections in Australia is still very good in most cases, there’s a bunch of small ways in which our state Electoral Commissions fall short of the Australian Electoral Commission that I certainly notice when attempting to use electoral data for this website.

Of course, it should be noted that in my experience most of the professional full-time staff at state Electoral Commissions are very professional, take their jobs very seriously and generally do good work, and from time to time you see some good innovations by state electoral commissions. The implementation of limited electronic voting in the ACT seems to have been handled well, and the ACT has also been successful in using optical character recognition (OCR) scanning of handwritten ballots to eliminate counting errors and speed up the counting process for large ballots of the type that take so long to count for the Senate and various Legislative Councils. Further down I’ll also discuss one new exciting innovation from the NSW Electoral Commission at this election.

If you’re someone looking to access electoral data at a federal level, it’s very easy. In addition to results data in a format easy for casual viewers to read, the AEC provides a ‘results downloads’ page (here is the page for the House of Representatives in 2013).

This data includes basically anything you might want to analyse the results. Full lists of candidates, full lists of polling places, including latitude and longitude for mapping purposes, enrolment statistics, and election results broken down in a bunch of ways – primary votes, two-candidate-preferred, two-party-preferred (there’s a difference), by booth, by vote type, by division, by state.

If only the same was available for each state election.

In most cases, it’s possible to get most of the same information, but there are usually gaps in the data. The most common problem is that data is only publicly available in an HTML format designed for casual readers – this includes keeping booth results on a separate page for each seat, so if you want the whole state’s data you need to use some kind of web-scraping tool or spend all day with copy and paste, and even then it’s a big hassle to combine data from different seats.

Quite often, data that is posted at the time of the election is taken down afterwards, making it hard to reconstruct results. In at least two jurisdictions (Victoria and South Australia) I had to email the electoral commission to get the full list of booths, and these didn’t include latitudes and longitudes, and in the Victorian case the data was organised in a way that required a lot of reorganisation before it was ready to be analysed. Other states do it better – the ECQ has their data on their website, and the NSW Electoral Commission always publishes a CSV file of all booths, including their estimates of how many voters are expected at each booth (here is the 2015 page, although who knows where this data will reside when the temporary 2015 election minisite is reorganised).

Even if booth lists are available, there’s often a problem matching lists of booths to the actual results. If you want to make a map, you need to match each individual booth on a list of booth addresses (and possibly geolocations) to the results of the election in each of those booths. Usually this is done by matching up a “unique name” – if there is more than one booth in a suburb, you add qualifying words, such as “Blacktown North” and “Blacktown South”, or sometimes the booths are named after the booth premises “Sackville Street Public School”, or the road the booth is on. The key is each booth has a unique name.

In the case of previous Tasmanian Electoral Commission booth lists, for both Legislative Council and House of Assembly elections, it has been difficult to find a booth list that actually matches the list of addresses to the results. Thankfully this seems to not always be the case – I have just downloaded the 2009 Legislative Council booth lists to prepare for May’s elections, and this list provides everything I need.

There are also big problems when it comes to actual vote figures.

Again, most state jurisdictions provide this data as a single HTML table on a separate page for each seat, which is accessible but time-consuming if you’re interested in numerous seats.

In Tasmania, this data is even less accessible. For statewide House of Assembly elections, the data is posted as large PDF files, which requires a substantial effort to clean up before you can begin analysing. For the Legislative Council it’s even worse: results are posted as PNG image files (such as this one). In order to use this data in a spreadsheet, to calculate percentages or add up votes for multiple booths, it’s necessary to painstakingly data-enter these results into a new spreadsheet, which is time-consuming and completey unnecessary.

In other states, there are various issues, many of which revolve around the provision of the appropriate two-candidate-preferred counts for each booth.

For federal elections, the Australian Electoral Commission provides accurate two-candidate-preferred (2CP) counts for every booth in every seat. Where the two-party-preferred count is different (where either of the top two candidates is not Labor or Coalition), they also provide the 2PP count by booth as well. Where the AEC wrongly guesses the top two candidates before election day, a fresh 2CP count is completed as soon as possible.

In various states, these counts aren’t as comprehensive. In Queensland, all two-candidate-preferred counts by booth are pulled down only a few days after the election and are not subsequently available. Even if you save this data before it’s taken down, in most seats the data does not exist for special votes, such as absentee, postal and pre-poll.

All electoral commissions need to guess which two candidates they think will come in the top two before election day, so that vote counters can proceed straight to the preference count after counting primary votes, but sometimes this guess is wrong. It’s not uncommon that booth counts are simply never conducted if the guess is wrong, and we are left guessing about the two-candidate-preferred count until the final distribution of preferences. In the seat of Prahran in Victoria in 2014, we only have a two-party-preferred booth count between the losing Labor and Liberal candidates, but no candidate including the successful Greens candidate.

The NSW Electoral Commission has had similar problems in the past – in a bunch of seats in NSW in 2011 there exists no count between Greens and Coalition by booth, which would reflect the top two parties. In many of these seats, such as Ballina, Lismore and Vaucluse, the Greens maintained their top-two finish but still the count was conducted between Labor and Liberal/Nationals.

In seats like Vaucluse there’s no particular urgency in producing the corrected figures, but in seats like Ballina and Lismore it was substantially harder to predict the result because the NSWEC chose not to conduct a fresh 2CP count between the correct candidates.

The NSWEC is solving this problem in the long-term thanks to their new process whereby all ballots are data-entered into the computer. This will allow the NSWEC to eventually release two-candidate-preferred counts for all conceivable combinations in each seat, which will give even more information than is provided currently by the AEC, and we’ve been promised that data in coming weeks. But it’s still a problem about how they rectify the problem where the NSWEC conducts the wrong 2CP count in a close race.

Of course, there are also problems with how the Australian Electoral Commission conducts elections. In addition to the unusual circumstances of lost ballot papers that triggered the Senate re-election in 2014, the 2013 Senate recount revealed the number of votes changing in a large number of booths across Western Australia. It’s an ongoing problem about how to improve the accuracy of vote-counting, and ensure that temporary election staff are well-trained.

Whatever you think is the cause of these problems, imposing tighter budgets on our electoral commissions won’t help them solve their problems.

I’m considering taking all of the candidate, booth and vote data I have collected over the years and putting them together into a central data repository in a format easy to use for data analysts. Let me know in comments if you would find this useful.

Liked it? Take a second to support the Tally Room on Patreon!