A federal inquiry into AI is examining the impact of the technology on government departments.
Rebecca Fawcett, the new leader of the Community and Public Service Union, fronted the inquiry on Friday alongside union delegate Ikini Frape.
The union officials struck a cautionary tone when delivering their opening remarks before the committee, warning that AI adoption in the public sector must include workers being consulted.
Ms Frape, referencing an MIT study from the US, said that AI use could erode critical thinking skills, the ability to problem solve issues, and the potential increases in dementia risk.
"Sometimes those administrative tasks are really essential for giving the brain a break when you're doing very complex work," she said.
Concerns of AI being used to automate decisions within the public service were also raised, following the fallout of the robodebt scandal.
Ms Fawcett has warned of governments misinterpreting increased capacity through AI augmentation as budget savings.
"Public servants, with their deep understanding of government systems, service delivery, and the needs of people who rely on them, must have a genuine role in shaping how AI is designed, introduced, and used," she said.
Ms Frape aired a number of concerns about data security and sovereignty, bias, the impact on workers' wellbeing, and the risks of cultural and intellectual First Nations knowledge being used without consent and in ways that could damage livelihoods while perpetuating colonisation.
Speaking to automated decision making, Ms Frape said public trust was broken after the robodebt scandal and staff were apprehensive of such technology.
The robodebt scandal was an unlawful scheme that falsely accused welfare recipients of owing the government money based off automated tax office data.
"If AI is used to make decisions, will staff and customers be able to understand how those decisions were made, and how it will be reviewed to ensure accuracy?" Ms Frape told the inquiry.
"If there is a mistake, who is responsible? And how can customers seek to have the decision corrected? What steps can staff take to speak up when the system is wrong, as they tried to do during robodebt?"Â
This was echoed by Ms Fawcett, who said a 400-member strong town hall meeting in recent months saw 98 per cent of attendees voting in favour of pausing the use of automated decision-making in complex decisions, until a "clear and mandatory" framework was implemented by the government.
The union's submission to the inquiry highlighted recent UK government analysis that estimated billions of dollars in savings through automating public service delivery without considering if existing AI models were capable of delivering.
Uncertainty on return on investment - another concern of the union - was raised by Treasury officials, who told the inquiry no formal calculation had been done of the net national benefit of investing in AI infrastructure due to inherent uncertainties of the rapidly evolving technology.
Jeremy Lawson, the department's first assistant secretary, argued that while a specific net figure would be "too rubbery" to base decision-making on alone, the long-term economic upside remained substantial.
Lifeline 13 11 14
beyondblue 1300 22 4636
