The anniversary comes as tensions over North Korea's rogue weapons programme and increasingly bellicose rhetoric from United States president Donald Trump rattle the region and put nuclear threats in the spotlight.
"A strong sense of anxiety is spreading across the globe that in the not too distant future these weapons could actually be used again," said Mayor Tomihisa Taue at a ceremony at Nagasaki Peace Park.
"Nagasaki must be the last place to suffer an atomic bombing," he said.
A bell tolled as thousands of people, including ageing survivors and relatives of victims, observed a minute's silence at 11:02 am (0202 GMT), the exact moment that the blast struck on August 9, 1945 in the closing days of World War II.
Japan is the only country to have suffered atomic attacks.
The US dropped the first atomic bomb on Hiroshima on August 6, 1945, killing around 140,000 people. The toll includes those who survived the explosion itself but died soon after from severe radiation exposure.
Three days later, the US dropped a plutonium bomb on the port city of Nagasaki, killing some 74,000 people.
Japan announced its surrender in World War II on August 15, 1945.
Wednesday's ceremony took place amid regional concerns over North Korea, with Trump sharpening the rhetoric against Pyongyang late Tuesday.
"They will be met with fire and fury like the world has never seen," the US president said at his golf club in New Jersey.
The security fears over North Korea have recently been cited by nuclear powers Britain, France and the US to oppose a UN treaty banning atomic weapons. It was a position supported by Japan.
Taue lambasted Tokyo over its stance on the treaty as "incomprehensible to those of us living in the cities that suffered atomic bombings".
Japanese officials routinely argue that they abhor nuclear weapons, but the nation's defence is firmly set under the US nuclear umbrella.
Many in Japan feel the attacks amount to war crimes because they targeted civilians and due to the unprecedented destructive nature of the weapons.
But many Americans believe they hastened the end of a bloody conflict, and ultimately saved lives, thus justifying the bombings.
Barack Obama became the first sitting US president to visit Hiroshima in May last year, paying moving tribute to victims of the devastating bomb.
