Based on the first 152,000 votes counted of an eligible 270,000, 50.1 per cent were against the government's proposal to restart negotiations, while 49.9 per cent were in favour, broadcaster RUV reported.
Voting ended at 10pm on Saturday (8am AEST Sunday), kicking off a complex operation to bring ballots from towns and rural polling stations by plane, boat and car.
On Friday, a Gallup poll indicated a narrow majority opposed the government's proposal of reopening negotiations, a shift from earlier surveys that pointed to a narrow lead for supporters of renewed talks.
"Yes" campaigners backing talks have said joining the bloc could ease high interest rates and offer more security in a world unsettled by war in Ukraine and US President Donald Trump's rhetoric about annexing fellow Arctic island Greenland.
"No" campaigners have argued that EU membership would threaten the island's sovereignty and fishing waters.
The referendum campaign has fuelled debate about Iceland's place in the world, Prime Minister Kristrun Frostadottir told reporters after casting her ballot in the capital on Saturday.
"This is a great day, we've been waiting for this for years to be able to have a vote," Frostadottir said, adding that her government would continue its work regardless of the outcome.
Though a "yes" vote would open up discussions with Brussels, under Icelandic law, there would have to be a second referendum, likely several years later, on whether to join the EU.
Iceland first applied to join the EU in 2009 amid a financial crisis that hit the country's small, exposed economy. A Eurosceptic government ended talks in 2013.
Geopolitical turmoil including war in the Middle East and U.S. import tariffs has brought urgency to an issue Icelanders have debated on and off for decades.
Debate inside Iceland has focused more on interest rates, fishing rights and the cost of living than on security.
Iceland has grappled with persistently high inflation rates and interest rates that have made mortgages expensive.
Supporters of a "yes" vote have said EU membership, and possibly adoption of the euro, could bring relief. Iceland's central bank interest rate is 8 per cent, compared with 2.25 per cent at the European Central Bank.
"It would not solve Iceland's structural faults, but it could become a catalyst to a more healthy economy," said chief economist Vilhjalmur Hilmarsson at Viska, one of Iceland's largest labour unions.
Opposition Leader Gudrun Hafsteinsdottir, who has led the "no" campaign, has argued the economic case is overstated.
"These are not problems that Brussels will solve for us. These are problems that Icelandic politicians must solve," she said.
10°C