Asylum seekers with legal representation are seven times more likely to succeed before the government tribunal tasked with reviewing refugee cases than those who represent themselves.

Where refugees come from and which individual member is reviewing their case on the Administrative Appeals Tribunal (AAT) may also significantly influence their odds of success.

These are some of the major findings from our research into the decision-making patterns of the overburdened AAT in refugee cases.

The data, obtained from a freedom of information request, cover 18,196 cases decided by the AAT between January 2015 and December 2019. Our research only looked at asylum seekers who arrived by plane and had access to a review by the AAT.

The analysis is part of larger project, run in collaboration with my research student, Keyvan Dorostkar, collating and studying quantitative data at all stages of Australia’s asylum process.

How the visa approval and review process works

For refugees and asylum seekers applying for protection visas in Australia, the process is lengthy and arduous.

The initial assessment of a protection visa application is carried out by the Department of Home Affairs. If this is denied, the options for review then depend on how they arrived in Australia.

Those who arrived by plane can seek review at the AAT, where they are given a fresh hearing assessing the merits of their claim for protection. Those who arrived by boat without authorisation can only access a much more limited form of review before the Immigration Assessment Authority (IAA).

Read more: Refugees without secure visas have poorer mental health – but the news isn't all bad

If the asylum seeker’s claims fail at the IAA or AAT, they can then seek judicial review at the Federal Circuit Court, but only on the very narrow grounds of there having been some serious legal error.

Overall, we found asylum seekers received favourable outcomes before the AAT in just 13% of cases. This includes instances where a visa has been granted or the matter was sent back to the department for reconsideration.

In the remaining 87% of cases, the original decision to refuse a visa was affirmed by the AAT or the application was withdrawn.

Why legal representation matters

Our analysis of the data reveals much more about the factors that tend to lead to a successful or unsuccessful review.

One of the most striking findings relates to the potential influence of professional migration advice from a lawyer or migration agent.

We found that only 4% of unrepresented applicants were successful at the AAT. This figure rose to 28% when an asylum seeker had legal representation.

Read more: We don't know how many asylum seekers are turned away at Australian airports

These statistics suggest the government’s decision to restrict public funding for free legal advice services may be severely disadvantaging applicants who cannot secure representation.

This is all the more concerning given our data show that just over half (52%) of all applicants do not have representation when they appear before the AAT.

Country of origin plays a huge role in success

There are also stark differences in the success rates of applicants from different countries.

Of the countries that had 20 or more applications during the period we studied, applicants from Libya (91%), Afghanistan (76%), Ethiopia (61%), stateless individuals (43%), Iraq (53%) and Iran (47%) were the most likely to succeed with their reviews.

While some variation is to be expected in these cases, the very high rates of decisions being overturned for certain countries raises concerns about the quality of the initial decisions being made on visas by the Department of Home Affairs.

Why is the department getting it wrong 90% of the time for Libyan applicants? Or more than 75% of the time for applicants from Afghanistan?

Read more: Sri Lankan asylum seekers are being deported from Australia despite fears of torture

At a time when the AAT is facing a record backlog of applications, it’s vital to understand why this is happening so that some of the pressure might be alleviated.

At the other end, the success rates for visa reviews for those from Ireland and Tonga were 0%, followed by Taiwan and South Korea (1%) and Malaysia (3%).

The Malaysian applicants are significant as they made up more than one-third of the entire caseload for the period (6,488 applications). The large numbers and low success rates among this group significantly skew the overall data. When the Malaysian applications are removed, the success rate for all asylum seekers increases from 13% to 19%.

A parliamentary inquiry found that people smugglers and illegal labour hire companies may be bringing workers into the country on travel visas and then applying for protection visas. This concern was raised particularly with respect to Malaysians.

However, there is no strong evidence that backs up claims around the systematic involvement of people smugglers and organised crime.

Regardless, the only incentive to put in an unmeritorious asylum claim is that it can buy you more time living and working in Australia.

If exploitation is a concern, the best way to ensure the integrity of the system is to reduce delays and invest more resources to boost the capacity for high-quality decision-making at both the department and AAT.

Which tribunal member hears the case also matters

In our research, we also found significant differences in the success rates for refugee visa reviews, depending which tribunal member hears the case.

We only examined members who had decided 50 or more cases to ensure the sample is large enough to be statistically relevant.

Two members did not find in favour of a single asylum seeker applicant, and another 16 had approval rates of less than 5%.

At the other end, one member decided in favour of the asylum seeker in 86% of cases, while another three members had approval rates over 40%.

Read more: There's no airport border 'crisis', only management failure of the Home Affairs department

It is important to caution against drawing inferences as to the cause of this variation.

While this could be a result of the individual preferences or biases of tribunal members, it could also be explained by the way cases are allocated. Members generally have expertise in specific types of claims from specific countries, which influences the cases they are assigned.

In response to questions about this, the AAT said

to construct any meaningful comparison concerning the variation of outcomes across individual members, there should be more analysis of the nature of the reviews undertaken by those members sampled.

For instance, the country of origin of applicants and the nature of the claims made by those applicants are generally the most significant factors in determining the outcome of a review.

We will examine these factors in more detail in future research.

The AAT is under enormous pressure with its record backlog of cases and associated delays.

We believe making data on decision-making patterns publicly available for analysis can lead to better ideas for improving the efficiency and fairness of the process. And this would be in the interest of both refugees and the government.

Macquarie student Keyvan Dorostkar provided research assistance for this article.

Daniel Ghezelbash is a member of the management committee of Refugee Advice and Casework Services and is a Special Counsel at the National Justice Project.