The pope was ushered into the basilica through a side door and brought to a chapel where he prayed for more than half an hour in front of an icon of the Madonna, the Vatican said in a statement.
"Since the basilica was open to the public, and numerous faithful were present, the pope, before leaving, stopped in front of the central altar" where he asked the faithful to follow his journey "with prayer, faith and penance," the Vatican said. The entire visit lasted a little more than an hour.
St. Mary Major is the same basilica where the pope went for an early morning prayer the day after being elected pontiff.
Earlier, the pope reached out over social media to young followers gearing up for World Youth Day, wishing young Catholics en route to Rio a safe journey.
Francis departs Monday for Brazil, where more than a million young Catholics are expected to celebrate their new pope.
The 76-year-old Argentine became the church's first pontiff from the Americas in March, and the trip to Brazil is his first international journey since becoming pope.
Catholic youth festivals are meant to reinvigorate the faithful, and Francis is expected to inspire young people with his humble ways.