Mr Carroll arrived at a Labor caucus meeting flanked by police minister Anthony Carbines, who is also from the right faction.
Multiple media reports say he has decided to challenge leadership favourite Jacinta Allan for the position of premier.
Ms Allan is from the left faction and became deputy premier in June 2022
She nominated herself for the top job hours after Mr Andrews resigned.
A Labor caucus meeting was meant to start at midday but Mr Andrews and Ms Allan have not appeared for it as of 12.30pm.
Mr Carroll was seen leaving the meeting room while talking on his phone and refused to answer questions about whether or not he is standing for leader.
If Mr Carroll does challenge then the new premier will not be chosen on Wednesday afternoon, as party rules stipulate members are entitled to take part in a leadership ballot.
MPs remained tight-lipped as they entered the caucus room, with Environment Minister Lily D'Ambrosio telling journalists the party had a process to go through.
"There's caucus meeting this morning, I'm looking forward to joining the caucus with my colleagues and letting the usual caucus process run its course," Ms D'Ambrosio told reporters as she walked into parliament earlier in the day.
When asked if she had the support of her Labor colleagues, Ms Allan said she would respect them and let the usual process run its course.
If successful, Ms Allan would be the state's first female premier since Joan Kirner stepped down in 1992.
Rules stipulate nominations for leadership have to be open for three days but it's uncertain if they will be followed if there is only one nomination.
Attorney-General Jaclyn Symes, who is from the right faction, told reporters she expected Ms Allan would run unopposed and believed a new premier would be chosen on Wednesday regardless.
Treasurer Tim Pallas arrived at parliament and did not rule out a tilt for the deputy role.
"I'm still consulting with my colleagues about that and of course, it's important to see who the leader is and that person get the opportunity to pick the person they want to be deputy," he told reporters.
Mr Pallas said it was up to the caucus to determine whether the three-day period needs to be observed.
Former federal opposition leader Bill Shorten said if the Bendigo East MP was successful she would ensure all Victorians were represented on Spring Street.
"She's a very highly successful, accomplished woman leader in her own right," Mr Shorten told ABC Radio National.
"She comes from the regions, she's a very experienced politician."
Mr Andrews announced his decision to step down at a press conference on Tuesday.
The outgoing premier cited thoughts about life after the top job, hinting he could work in the private sector after taking time off.
Mr Andrews became premier in November 2014, booting out a first-term Liberal government, and led Labor to victories in 2018 and last year.
During the 2022 campaign and the months after, he repeatedly said he would stay for the full four-year term.
His change of heart was a recent decision, he said.
Prime Minister Anthony Albanese told ABC Radio Melbourne on Wednesday Mr Andrews had been a strong leader and passionate Victorian.
35°C