A group of almost 40 people from Pakistan and Bangladesh were found at Beagle Bay north of Broome late last week.
The men have since been flown to an offshore detention centre on the Pacific island of Nauru.
Opposition Leader Peter Dutton accused the Albanese government of beginning to "lose control again of our borders".
Abul Rizvi, a former deputy secretary of the immigration department, said the language used by politicians could lead to more boat arrivals.
"If people smugglers are able to convince potential people who may wish to come to Australia by boat, that the difference in the rhetoric is something they can take advantage of ... we may if that happens, see a rise in boats," Dr Rizvi told ABC radio.
"I would counsel politicians on either side not to inflame the rhetoric in this space."
Anthony Albanese has criticised the opposition for playing politics with boat arrivals. (Dean Lewins/AAP PHOTOS)
Prime Minister Anthony Albanese reaffirmed his government would not allow asylum seekers to settle in Australia if they arrived by boat.
"What is important is how the response occurs and that message go out there at unauthorised arrivals," he told ABC radio.
"That was our commitment that we will keep those Operation Sovereign Borders arrangements in place. That's precisely what we have done."
In response to accusations he was jeopardising national security, Mr Dutton said the government, not his comments, was enticing people smugglers.
"Nobody's politicising the issue. We're pointing out the fact the government is making significant errors here," he told Sunrise on Monday.
"They have closed down one of the important legs of Operation Sovereign Borders, which is the temporary protection visas."
Dr Rizvi said he had not seen any evidence there had been a reduction in resourcing for the Australian Border Force to patrol the nation's shores.
"The statistics and the allocation of resources in this space is not entirely transparent and it would be good if the government were to explain what has happened in terms of resourcing," he said.
Asylum seekers who arrived in remote WA have been flown to an offshore detention centre on Nauru. (HANDOUT/ABC TV)
Mr Albanese confirmed he spoke with Operation Sovereign Borders commander Rear Admiral Brett Sonter on Sunday, after three groups of boat arrivals had been found.
The commander also cautioned against politicising the boat arrivals.
Mr Dutton said the government needed to explain how many boats had arrived and how they were able to evade detection.
