Conductors, train controllers, ticket officers and customer service staff stopped work between 3am and 4pm on Friday due to industrial action led by the Rail, Tram and Bus Union (RTBU).
The strike marks a major escalation in a long-running dispute between rail operator V/Line and the workers' union over a new enterprise agreement.
It comes on the eve of Victoria's Labor Day long weekend
Buses will partially replace trains on the expansive network which operates throughout Victoria's regions.
But V/Line chief executive Matt Carrick said the industrial action meant the vast majority of the trains' usual passengers could not access public transport.
"Normally on a day like today you could expect about 70,000 people to use our network, today we have had about 3000," he said.
"These delays will go on well into the evening which is why we encourage people to make new transport arrangements."
V/Line workers are seeking the same pay as staff on Melbourne's Metro network who received a 17 per cent increase over four years.
The union said other main sticking points include boosting wages in line with inflation and conditions such as shift length.
RTBU Branch Secretary Vik Sharma urged the state government to get serious at the bargaining table to resolve the pay dispute.
"The state wages policy is being used to hold back regional workers and create a two class transport network by paying Metro workers more than their regional counterparts," he said.
"Into the tenth month of bargaining, the Allan Government has endorsed a dud deal that would leave regional workers worse off than their metropolitan counterparts and have sat on their hands since notification of this action was given."
Mr Carrick said any pay increase for V/Line staff was hindered by a range of circumstances.
"We're a public sector agency so we're bound by government wages and industrial relations policy. It's really clear in its rules, it does not discriminate where people are based," he said.
Trains will progressively return on the Geelong, Ballarat, Bendigo and Traralgon lines from 5pm but significant delays will be expected.
Train drivers are not part of the strike.