Nemtsov, a reformer, democrat and former deputy PM, was a fierce critic of President Vladimir Putin.
Some marchers chanted "Russia will be free!" and "Putin is war!"
Five Chechen men are on trial for the killing. They deny the charges and Nemtsov's relatives fear whoever ordered the murder will never be found.
Separately on Sunday, anti-Kremlin activist Ildar Dadin, the first person jailed under new laws targeting protests, was released from a Siberian prison. The Supreme Court had overturned his conviction on Wednesday.
He says he was tortured by staff while in prison, an accusation officials deny.
Nemtsov, who was 55, was shot in the back as he walked home from a restaurant with his Ukrainian girlfriend late at night near the Kremlin on 27 February 2015.
He had earlier been at the radio station Ekho Moskvy, calling on listeners to join a protest. He had accused Russia's president of launching an illegal war with Ukraine, prompting Western sanctions and an economic crisis.
He had also been planning to publish a file on Russian involvement in eastern Ukraine.
Defiance and remembrance - Sarah Rainsford, BBC News, Moscow
The slogans were sharp: "I'm against the annexation of Crimea"; "Give us back our elections, you rats!"; "For Russia, without Putin". ' They were all quotes from Boris Nemtsov, held high by the crowd alongside photographs of the politician and the Russian flag.
Two years on, it's still not clear who ordered his murder but his supporters are certain he was killed for his critical, political views. So joining this march through central Moscow was an act of defiance for many, as well as remembrance.
That's also what the unofficial shrine to Mr Nemtsov has become. Flowers and photographs first appeared at the spot where he was killed straight after the murder.
The site beside the Kremlin has been guarded round the clock every day ever since, from vandals who repeatedly attack it. But today, there was a long queue to reach the shrine, all the way along the bridge and back to Red Square as Russians came to lay fresh flowers and remember a man who they say stood up for their freedom, and for democracy.
One of the marchers, Galina Zolina, told Agence France-Presse: "We came to pay tribute to the honesty and bravery of Boris Nemtsov. We want to show the authorities that we haven't forgotten."
Organiser Ilya Yashin told Reuters: "We gathered here to demand the bringing of Boris Nemtsov's killers to justice, not only its performers but also its organisers and those who ordered it."
The rally appeared peaceful, although a green fluid was thrown into the face of Putin critic and former PM Mikhail Kasyanov.
Rallies also took place in St Petersburg and other cities.
Nemtsov's political allies believe the killing was meant to terrify them into silence.
Opposition activist Vladimir Kara-Murza earlier urged people to join the anniversary rallies.
Mr Kara-Murza is recovering abroad after an illness that left him in a medically induced coma. His symptoms were similar to a near-fatal illness in 2015 that he believed was the result of deliberate poisoning.
The trial of the five Chechens for Nemtsov's murder began in October.
The suspect who investigators say carried out the killing, Zaur Dadayev, was an officer under the command of pro-Moscow's Ramzan Kadyrov, the leader of Russia's Chechen Republic in the volatile North Caucasus.
The Chechen leader has denied any link to the killing.
Boris Nemtsov murder: Who are the suspects?
President Putin called the murder "vile and cynical" and vowed that those responsible would be held to account.
Russia has seen several killings of high-profile politicians and journalists.
But the country has a long history of prosecuting alleged hit-men and then failing to follow the chain of command upwards to discover who ordered the murder or why, our correspondent says.