Policymakers have long grappled with the best way to engage a seemingly apathetic public into paying more attention to their superannuation.

The introduction of a low-cost and simple default option, MySuper, for people who do not nominate a specific superannuation investment fund, was meant to protect consumers from imprudent fund management and high fees and help people compare funds. A key feature of MySuper is a simplified dashboard using a standardised structure that radically shortens product disclosure statements.

It was meant to facilitate comparison across MySuper offerings by different providers. Experimental research I have been undertaking with colleagues, Ben Newell, Hazel Bateman, Isabella Dobrescu, and Susan Thorp, that we are in the process of writing up, suggest that the recommended information items on MySuper dashboards might not be effective as hoped.

Many plan members do not understand key items such investment risk, although the dashboard template was designed for ordinary people to use. We conjecture that this finding is due to both insufficient financial literacy and the complexity of the dashboard itself.

Using a simplified design we have demonstrated how understanding of the template can be improved. Our findings suggest that the process of how to test well-intentioned innovations of this kind is in need of a re—think. Focus groups and in-depth interviews may just not be good enough.

In earlier work that has just been published, we investigated experimentally the related phenomenon of mandated short-form product disclosure forms for retirement investment funds.

Retirement investment providers are now required to present in such a form for each of their investment options five pieces of information - a name and description, a list of asset classes (such as shares, cash, or property, for instance), investment return objective, minimum time frame required to hold the investment, and summary risk level of the investment.

A prominent feature of most currently used investment disclosures is a pie chart that is meant to illustrate the way the assets being invested in are allocated (called strategic asset allocation), an important decision on how well the overall investment will perform on total returns. To visualise some such pie chart, imagine a round pie being cut up in pieces where each of the pieces represents a different asset class, say a high-risk international equity investment fund and low-risk domestic money-market fund.

We tested in the laboratory whether the information contained in these prescribed investment disclosures was used by our subjects – mostly students with superannuation accounts but also staff – in the way that the regulator has intended.

We presented our subjects with repeated pair-wise choices between investment options with different risk-return characteristics.

Then we analysed statistically how choices were determined through the various pieces of information that we provided, as well as the format used (for example, a pie chart or a table for the strategic asset allocation).

Simply speaking, it was the regulator’s intention to focus superannuation participants on matching risk-adjusted returns to their own risk profile.

But one of our key findings is that most of our subjects do not make their decisions in this manner.

Instead, when they were given the pie chart information, for example, they typically went for those investment options that had more slices and that had the more uniform distribution of the slices. This is known in the literature as 1/n strategy and can be best visualized by a pie with n slices where each slice is about 1/n of the pie, with n identifying the number of pieces, or asset classes.

It may turn out not to be a good strategy. For instance, someone early in their life might want to invest in more risky growth assets because over time they pay on average a risk premium over money-market funds that pay comparatively little. Our results suggest that some such person would invest in too many slices that have low risk but also low returns, meaning that s/he would not accumulate as much early in the retirement game than arguably they should.

The 1/n strategy (which previously had been identified as a reasonable one for individual assets), when applied to already highly diversified investment options, can thus result in outcomes that are not informed by appropriate risk-return trade-offs.

We confirmed our finding by running a treatment where asset allocation was omitted, and by demonstrating that participants in that treatment traded off risks and returns appropriately. We also demonstrated that none of the five prescribed information items was significant in explaining individual choices of more than 35% of participants.

These findings also highlight that information contained in prescribed investment disclosures might not be used in the manner intended by the regulator.

While the regulator’s intention was laudable in facilitating superannuation members' retirement investment decisions, our results suggest strongly that the current process of evaluating behavioural policy innovations, at least in this case, is wanting and that well-intentioned innovation of this kind would benefit from better pre-release testing. Focus groups and in-depth interviews may just not be good enough.

Andreas Ortmann receives funding from the Australian Research Council through Linkage grants with UniSuper.