He made the admission at the opening of a memorial conference on the sad chapter of the world body's operations by accepting institutional and personal blame for the slaughter.
The killings were initially ignored by world leaders.
"The international community is guilty of sins of omission," said Annan, who was head of the United Nations peacekeeping agency at the time and had asked countries to provide troops.
"I believed at the time that I was doing my best. But I realised after the genocide that there was more that I could and should have done to sound the alarm and rally support," Annan said in a speech to open the "Memorial Conference on the Rwanda Genocide."
It was not the first time that the secretary-general had criticised the United Nations and his own mistakes. But he said the painful memory of Rwanda and ethnic cleansing in Bosnia-Herzegovina in the mid-1990s "has influenced much of my thinking, and many of my actions" as head of the world body.
The small central African country was plunged into ethnic butchery in April 1994 after a plane carrying President Juvenal Habyarimana was shot down over Kigali.
Some 800,000 Tutsis and Hutu moderates were slain in about 100 days by Hutu extremists and their followers, armed with machetes, garden hoes and spiked clubs. They were spurred on by hateful radio broadcasts.
The genocide was halted when Tutsi-led rebels overthrew the Hutu extremists, many of whom fled to neighbouring Zaire, now called the Democratic Republic of Congo. The rebels went on to form Rwanda's government.
In April 2000, the U.N. Security Council admitted responsibility for failing to stop the Rwanda killings. A U.N.-commissioned report in December 1999 accused the world body of being timid, disorganised and misguided before the massacres and failing to intervene once the killing had started.
Canada, which has been a leading organiser of much of the U.N.'s self-examination over Rwanda, said yesterday that the international community had not yet learnt how to build structures capable of withstanding such brutality next time.
"Or, to put it more starkly, we have learned what we need to do but I suggest, colleagues, we lack the political will to achieve the necessary agreement on how to put in place the type of measures that will prevent a future Rwanda from ever happening again," Canadian Foreign Affairs Minister Bill Graham told the memorial conference.
Canadian Lt.-Gen. Romeo Dallaire, head of a small U.N. force, was driven to the brink of suicidal depression after returning home from Kigali because the Security Council did not send in reinforcements as Rwandans begged him for help.
The one-day conference heard from witnesses of the slaughter and officials' views of what lessons were to be drawn and better ways to respond to any future genocide.
Annan has designated April 7 the "International Day of Reflection on the Genocide of Rwanda" and supported a request from the Rwandan government that the world observe one minute of silence on that day at 12 noon in each time zone.
"None of us must ever forget, or be allowed to forget, that genocide did take place in Rwanda, or that it was highly organised, or that it was carried out in broad daylight," Annan said.