The 39-year-old American, who made her Games debut at Sydney in 2000, made the announcement in her pre-Wimbledon media conference on Sunday.
“I’m actually not on the Olympic list – not that I’m aware of. If so, then I shouldn’t be on it,” she said.
Williams did not want to elaborate when asked more about the decision.
The 23-time Grand Slam singles champion had already indicated she did not want to travel to Tokyo because she could not take three-year-old daughter Olympia with her under Games restrictions.
“There are lots of reasons that I made my Olympic decision. I don’t feel like going into them today. Maybe another day. Sorry,” she said.
Williams won gold in the singles and women’s doubles at London 2012 with her two other medals coming in the doubles at Sydney 2000 and Beijing 2008.
She begins her bid for an eighth Wimbledon singles title and a record-equalling 24th Grand Slam singles title against Aliaksandra Sasnovich of Belarus.