With more than one million votes counted early on Tuesday, 77 per cent supported remaining in Canada, while 23 per cent backed beginning a process toward separation, according to unofficial results from Elections Alberta.
The vote was not a direct referendum on independence but would have set oil-rich Alberta on a path towards a future vote on separation from Canada.
Alberta Premier Danielle Smith welcomed the early results, saying they showed widespread support for remaining in Canada.
"Albertans have voted for a stronger Alberta in a united Canada," Smith posted on social media.
Smith, who called the referendum amid pressure from separatists within her governing United Conservative Party despite opposing independence, said she would address the province Tuesday evening about the results and Alberta's future.
"We are a solid country that's united from coast to coast to coast," Thomas Lukaszuk, a former Alberta deputy premier who led the Forever Canadian campaign against separation, told supporters.
Results were delayed after late openings forced more than 100 polling stations to extend voting hours on Monday.
Alberta, home to the world's fourth-largest proven oil reserves, is cowboy country and a conservative stronghold whose political culture has long had more in common with the American West than much of Canada.
Resentment towards Ottawa runs deep, but has not traditionally translated into widespread support for independence.
Despite the proposal's apparent failure, separatist leaders have said they turned a fringe cause into a movement that will not disappear.
Carney earlier urged Albertans to deliver a decisive vote for unity.
"Vote like Canada's future depends on it because it does," he told a rally Friday in Edmonton, where he grew up.
Separatist parties traditionally attracted only one per cent or two per cent of the vote, former premier Jason Kenney said.
Anger over COVID-19 restrictions helped mobilise activists who later embraced the separatist movement, he said.
Trump's trade war and threats to make Canada the 51st US state have helped Carney rally support in Alberta, where Liberal prime ministers have historically been unpopular.
Separatist advocate Jeff Rath said he would be thrilled if 30 per cent of Albertans voted to begin the separation process.
He wants Smith ousted as leader of her governing United Conservative Party and said the party could splinter, potentially creating a new pro-independence party.