University Entrance: Part VI - University Grading and Outcomes

This post is part of a multi-part series on University Entrance and whether it is set at the right standard. For the previous part, click here - Universities and University Students.

Are students with UE actually prepared enough for university?

In theory, every student who enters university should be sufficiently capable to complete a degree. This is important to understand, because if a significant proportion of students are failing university, then perhaps the university entrance standard is in the wrong place. There are plenty of reasons why a student might fail, so we shouldn’t expect a 100% pass rate, but similarly a 10% pass rate would be worrying. Unfortunately, the data that we need to answer this question is not easily available.

Luckily, last year I asked all of the universities under the Official Information Act (OIA) for pass rate data for all of their papers, and now have a(n incomplete) dataset to work with. I’ll document the OIA request and processes in an appendix post (here). I’ll note here that I did receive data from University of Otago, but it did not include class sizes so I can’t use it for the below analysis. In all of the following tables, courses with fewer than 10 students are excluded. So with data from three out of eight universities, we can have a look at the average pass rate across all papers (weighted for number of students per paper) from 2011 to 2013:

Weighted average pass rate	2011		2012		2013
	Pass %	Courses	Pass %	Courses	Pass %	Courses
Lincoln University	81.5	395	83.5	402	83.9	393
University of Auckland	88	2,699	89	2,721	88	2,692
Victoria University of Wellington	88	1,372	88	1,359	86	1,397

Around 10-15% of students failing is probably reasonable and to be expected. There are many reasons to fail papers beyond preparedness for university. However, these statistics are across all papers at the university, including in some cases Foundational or Honours papers – the picture looks different when we only consider 100-level (commonly, although not exclusively, 1^st year) papers:

Weighted average pass rate (100-level papers only)	2011		2012		2013
	Pass %	Courses	Pass %	Courses	Pass %	Courses
Lincoln University	75.2	49	76.3	51	75.4	52
University of Auckland	83	378	84	372	82	367
Victoria University of Wellington	78.3	205	80	200	79.6	202

Between 5-10% more students fail when we look at only 100-level papers. So it would be fair to say that the average first year is more likely to fail than the average student at a university, which is probably not a ground-breaking conclusion; the question is if that failure rate is too high, too low, or just right? However, rather than looking at averages, let’s look at the data another way. In each of the following graphs, only 100-level papers with at least 10 students enrolled are shown:

Note: The 0% pass rate papers at the bottom-left of the VUW graph are papers that students going on exchange are enrolled in for administrative purposes.

Beyond averages, it’s important to look at the spreads and see that there is broad variation from the mean. In particular, there are some courses with quite low pass rates - as low as 40%. Generally, failing one paper is enough to derail a degree and force students to have to take an extra semester (or more) to finish their course. Additionally, it’s important to recognise the sizes of the classes and appreciate the scope of the issue.

For example, let’s take the right-most point on the UoA graph. This course, with 287.69 EFTS enrolments, which equates to roughly 2,300 students (assuming that it’s a 15-point course), has a pass rate of 75%. One-quarter of students who take this 100-level paper fail. That’s roughly 575 students who failed that course in 2012, and are going to either repeat the course, try to do another paper instead if they’re allowed to (this one happens to be a pre-requisite for a number of majors), switch degrees, or drop out of university.

Now, with the caveat that students who fail one paper are likely to have failed other papers as well and therefore be double counted, we can look at the total number of course failures for 100-level courses at each university:

Total Number of Course Failures (100-level papers only)	2011	2012	2013
Lincoln University	1,534	1,266	1,293
University of Auckland	14,598	14,066	15,218
Victoria University of Wellington	8,866	8,139	8,182

Making the critical assumption that the universities did a perfect job in teaching their students and preparing them for their exams as best they could and that the assessment standards are at the right places [cough cough], this table shows a worryingly high number of course failures. Taking a somewhat educated stab in the dark, we might see roughly 50,000 course failures across all eight universities per year. The question then becomes this – is this number acceptable?

Why are so many students failing?

There are two main reasons why a grade distribution might look the way it does – either the students are assessed at a standard and the distribution reflects that ability, or the assessors have an expected distribution and scale the marks to meet that distribution. As described in the previous part of this series, at university it’s likely a combination of both reasons. As several of the universities reminded me in their responses to OIA requests, there are many factors that influence pass rates, and similarly there are many factors that influence failure rates, some of which are difficult to pin down. However, one of those factors is that an increasing number of students are insufficiently prepared for university.

A report by the Tertiary Education Commission in 2013 (obtained by the Listener under the OIA) stated that most students who got an “achieved” grade at university would have had a “fail” grade under the School Certificate/Bursary system. Universities said that students are reaching them under-prepared and with a poor work ethic. Dale Carnegie at Victoria University said that students were able to “game” NCEA in a way that they couldn’t at university, giving the perception that they are more capable and prepared than they actually are. It suggests that if we continue to push more and more students who have reached the stated minimum entry requirement but are ultimately unprepared for university, we can only expect the number of students who fail papers to increase.

When an unprepared student enters university, fails one or more papers, and drops out of university, we end up with a lose-lose-lose situation. The government wastes money funding education and student loans, the university wastes time and resources catering to their needs, and worst of all the student wastes time that they could have spent working and wastes money that they could have spent on more useful things.

At the end of the day it’s about whether or not young people are in the right places for them and their future. University simply may not be the right place for everyone, and pushing students towards university when it’s not the right path for them has negative consequences. This effect has been seen in the United Kingdom, where university drop-out rates have soared to the 20% range; the University and College Union secretary suggested that the source of the problem is rising fees forcing students into courses that are cheaper but “do not suit their abilities”. In the United States, 33% of freshmen (first-year) students don’t make it to second-year. A lack of self-directed and disciplined learning, an inability to move away from rote learning towards conceptual thinking, and being motivated at university for the wrong reasons (such as just to please friends and family) are identified as key factors that drive students away from completing their first year of studies. When a young person is pigeonholed into a system like a university where they really should not be, maybe we are setting them up to fail.

What does this mean for University Entrance?

The reasonably high failure rates indicate that University Entrance does not accurately reflect adequate preparedness for degree-level study. So assuming that the universities do not strictly stick to their existing expected distributions, it will be interesting to see what happens over the next couple of years with a raised UE standard. In theory, the number of failing students should decrease, and the pass rates should rise. Ultimately, this is a good thing. At that exact moment when a student finds out that they missed out on getting into university, it can be a real struggle to appreciate that. But maybe they’ve avoided wasting time and money on struggling through a university education. And of course, if they really want to get into university, there are bridging and foundational courses that were developed to ensure that the students really are prepared and capable for university life, to give them a much higher chance of success at degree-level study.

Ultimately this all ties into broader ideological arguments about whether or not every person should have access to university (or tertiary) education. It leads to arguments about equality vs. elitism, which is not necessarily an argument that can be won. There are strong advocates on both sides, and I can see the merits of both sides, and I think the answer is probably a balance somewhere inbetween. So I’m going to wimp out and let believers from both sides duke it out in the comments below if they want to.

In this section, we looked at what happens after students obtain University Entrance and actually get into university, pass rates and their associated failure rates, discussed why failure rates are so high, and touched on how UE can affect these failure rates. In the next section, we’ll try to wrap everything up.

As an aside, for an interesting read of how the “failure” problem has continued over decades, this article published in Salient (the Victoria University student magazine) in 1966 discusses why students fail university courses and how that failure that can be anticipated and avoided. One line near the beginning sticks out: “students who do well at school are not necessarily as successful at university”.

Part VII of this series - Conclusions, is available here.