The sides cancelled each other out in a tight opening at a blustery LNER Community Stadium on Saturday (Sunday AEST), but the Knights went ahead when Paul McShane's speculative kick was allowed to bounce in goal and David Nofoaluma pounced to touch down.
Mathieu Cozza passed up an opportunity to drag the Giants back into it with 20 minutes gone when he spilled the ball over the line, but winger Lewis Jagger's neat footwork and strength took him over within six minutes to reduce the deficit.
With scores locked at 6-6 at half time, the visitors went in front seven minutes into the second half when Lolohea chased down his own kick ahead and got the touch as he and defender Toa Mata'afa dived in.
York were back in it again when Scott Galeano forced his way over in the corner for an unconverted try, but Lolohea put Connor Wrench in to give Huddersfield breathing space.
They increased their lead to 24-10 after Zac Woolford scored, but Jordan Thompson and Tom Inman both crossed in quick succession, with Jon Bennison taking the home side to within two points as a dramatic fightback came up just short.
Speaking after his first win as Giants head coach, Jim Lenihan told Sky Sports: "I'm just happy for the players, happy for the supporters, happy for the club just to get a win. We just managed to believe in ourselves to get there.
"We deserved an opportunity to win the game. We got through the sets – it got a bit hairy at the back end there, we lost our way a touch, but steadied the boat, got to the end of the set and won the game, which is good."
Elsewhere, in-form Leigh saw off Warrington 28-12 to close in on the top four.
Leopards head coach Adrian Lam praised the way his side came through, despite having to make changes during the match because of injuries.
"I thought we were resilient today and hung in there and played really well," said Lam, who confirmed Frankie Halton suffered a suspected torn pectoral muscle which could rule him out for the rest of the season.
"The first half was awesome and we left some tries out there as well, but I though we were physical and we were up for it."
Warrington boss Sam Burgess felt his team deserved more.
"It is one of those games. I thought the scoreline probably didn't reflect the game or how tight the game was," Burgess said on Sky Sports.
"In that second half I thought we started with great spirit. We just couldn't get any points."
In Toulouse, Tommy Makinson scored two tries as John Cartwright's Catalans Dragons ended a four-match losing run with a scrappy 18-16 win away.
