The boot of replacement Santiago Carreras proved the difference as he kicked over three second-half penalties on Saturday (local time), leaving all four teams in the competition with one win apiece after two rounds of the Southern Hemisphere Championship.
Earlier on Saturday, South Africa had beaten Australia in Cape Town.
Billy Proctor and Fletcher Newell scored first-half tries to give New Zealand a 13-6 lead, but scores from Juan Martin Gonzalez and Gonzalo Garcia helped Argentina fight back to win after losing 41-24 to the All Blacks in Cordoba last week.
They had lost all 15 previous home Tests against New Zealand, stretching back to 1985.
"First of all we believe that we can win. We've believed that for a few years now," Argentina captain Julian Montoya said in a television interview.
"We came back from a tough loss last game and I think that we kept in the fight and we threw a few punches.
"We took a few punches as well, but it's how you come back from those errors."
The two sides traded early penalties as Argentina went ahead 6-3 in the 15th minute, before New Zealand scored two quick tries to lead 13-6.
The All Blacks had a try with their first attacking chance in the 19th minute as Codie Taylor, marking his 100th appearance, went close on the right.
A quick shift to the other side saw Rieko Ioane held up before the line, but the ball came out quickly for Proctor to take a gap in the defence and go over.
Six minutes later, prop forward Newell barged through at pace to crash down in the corner and extend the visitors' lead.
But after Will Jordan was yellow-carded, Argentina made use of the numerical advantage and scored their first try from Gonzalez, with Carreras adding the conversion to level matters.
New Zealand then had flanker Tupou Vaa'i yellow-carded to be down to 13 players, and Argentina came close to taking the lead before halftime, but a handling error denied Bautista Delguy a try as he spilled the ball on the line.
The visitors' discipline did not improve after the break, handing penalty opportunities to Argentina, which Carreras kicked over in the 51st minute and again five minutes later to swing the momentum.
Victory looked in sight when Pablo Matera broke off the back of the scrum in the 59th minute and, while he was held up short of line, scrumhalf Garcia pounced on the ball to dart over for a try that put Argentina 26-13 in front.
New Zealand had shown their strength in the set-pieces in the closing stages in Cordoba last week and threatened the same again when replacement hooker Samisoni Taukei'aho scored from a line-out maul with 12 minutes remaining.
However, any hopes of a dramatic comeback ended when Sevu Reece was penalised, and yellow-carded, for sticking out his hand to block the ball, allowing Carreras to kick over another penalty that ensured the victory.
with Associated Press
16°C