Each side sent representatives to the ceremony, in the border town of Kaesong.
Last month, the two Koreas carried out a joint inspection of parts of the North's railway system, which they hope to connect someday with the South. The goal is to link the countries' railroads and thereby increase economic ties.
The move is one of several efforts agreed to by North Korean leader Kim Jong Un and the president of South Korea, Moon Jae-in.
Progress linked to nuclear talks
Any progress on the project, however, will require changes to United States-led economic actions against North Korea. They are designed to punish the North Korean government for its nuclear activities.
The railway inspection had been delayed for months because of concerns about United Nations sanctions on the North. The U.N. Security Council gave the two Koreas special permission for the study.
North Korea has voiced displeasure with the slow progress on projects aimed at building ties between the two sides. South Korea hopes to carry out additional inspections before further details of the project are released. Work depends on progress in nuclear talks with the North.
The United States and its allies have called for restrictions to remain in place until North Korea takes clear steps toward giving up its nuclear weapons and missiles.
For the ground-breaking ceremony on Wednesday, a South Korean train traveled to Kaesong. It was carrying about 100 people, including government officials and lawmakers. Members of families displaced by the Korean War in the early 1950s also were on the train.
South Korea's unification ministry reported that officials from the United Nations, China, Russia and Mongolia joined the North Korean delegation.
After the event, North and South Korea agreed to carry out additional railway inspections. Both Koreas said they would work closely with the United States and the U.N. to gain support for the project and deal with concerns over sanctions.
Lee Eugene, a spokeswoman for the unification ministry, spoke with reporters. "We plan to hold detailed negotiations with the North to coordinate the specific levels we want to achieve in the modernization of railways and roads and how to carry out the project," she said.
However, experts say investment in the project could take time even with steps toward nuclear weapons removal and easing of restrictions.
Jung Dae-jin is a professor with the Ajou Institute of Unification in South Korea. He noted that the project would increase exchanges between the two governments.
But he said the future of Moon's plans for connecting the railways rests with North Korea.
"We need to see the New Year's address by Kim Jong Un," Jung said. He added that the global community needs to see measures taken by the North toward removing nuclear weapons for progress on railways to go forward.
I'm Mario Ritter, Jr.