The 61-year-old talk show host's program will end with the current season on May 21, but the TV star insisted he's not quite ready to let go of anything just yet.
Oprah Winfrey was a guest on his show on Tuesday and asked if there was anything he wanted to "release" as the end approaches.
He said: "I still have a white-knuckle grip on all these people who I love, who I've worked with all these years, including those people over there."
Colbert - who is writing new Lord of the Rings film Shadow of the Past - is still close to the band, camera crew, light and sound teams, and more behind the scenes.
Meanwhile, he and Oprah both agreed that the audience is such an important part of a talk show.
He explained: "The audience is very pure in their engagement of our conversation ... The audience members are listening and reacting."
Earlier in 2026, Colbert admitted his Late Show cancellation - which comes after 10 years fronting the program - was finally starting to feel real.
Appearing on Late Night with Seth Myers, he told his friend: "It feels real now. It did not feel ... I mean, I know it was real, but now, there's four months left."
Colbert confirmed the show's fate in July 2025, when he told the audience the devastating news.
He said in his monologue at the time: "Before we start the show, I want you to know something that I found out just last night. Next year will be our last season. The network will be ending The Late Show in May ...
"It's not just the end of our show, but it's the end of The Late Show on CBS. I'm not being replaced. This is all just going away."
The announcement came shortly after he criticised parent company Paramount's $US16 million ($A23 million) settlement with US President Donald Trump and during a time when the studio was waiting for approval for its merger with Skydance.
While CBS insisted the decision to axe the show was "purely financial" and not related to "other matters happening at Paramount", many speculated that that wasn't the case, though Colbert himself has refused to "engage in that speculation".
The Late Show airs on Channel 10 in Australia.