"Her people have been trying to pin it on me," Powell, 79, told PEOPLE Saturday night at the Apollo in the Hamptons 2016 Night of Legends fête in East Hampton, New York.
"The truth is, she was using [the private email server] for a year before I sent her a memo telling her what I did," Powell added.
Why does the former diplomat believe this to be the case?
"Why do you think?" he said. "It doesn't bother me. But it's okay; I'm free."
The email scandal has dogged presidential hopeful Clinton for more than a year. But federal officials decided not to pursue criminal charges after a three-and-a-half-hour interview, which was when the Democratic nominee disclosed her alleged conversation with Powell.
The reported conversation was first brought to light in journalist Joe Conason's upcoming Bill Clinton biography, Man of the World: The Further Endeavors of Bill Clinton, in which the writer details a dinner party held by Clinton and attended by Powell, Madeleine Albright, Henry Kissinger and Condoleeza Rice.
"Toward the end of the evening, over dessert, Albright asked all of the former secretaries to offer one salient bit of counsel to the nation's next top diplomat," Conason wrote. "Powell told her to use her own email, as he had done, except for classified communications, which he had sent and received via a State Department computer ... [Powell] confirmed a decision she had made months earlier ‐ to keep her personal account and use it for most messages."
Powell's office later released a statement to NBC News, saying he "has no recollection of the dinner conversation." However, "He did write former Secretary Clinton an email memo describing his use of his personal AOL email account for unclassified messages and how it vastly improved communications within the State Department."