William Jefferson of his committee assignment on Thursday after the Louisiana lawmaker rebuffed repeated calls to step aside pending completion of a federal bribery investigation. Jefferson, at the center of an increasingly complicated legal and political controversy with racial overtones, insisted he would not go quietly. "I can guarantee" he will not voluntarily step aside, said Melanie Roussell, his spokeswoman.The congressman has not been indicted and has repeatedly denied wrongdoing. He is a member of the Ways and Means Committee, with authority over taxes, Medicare, trade, Social Security and more. Several officials said Jefferson had rebuffed a final appeal to step aside in a meeting with three fellow Democrats dispatched by party leaders. They spoke on condition of anonymity, citing the delicate nature of the conversation. Within hours, the same leadership group met and voted to remove him from the panel, at least temporarily, according to officials familiar with the deliberations. These officials said the vote was not unanimous, reflecting at least some disagreement about the case. The entire Democratic rank and file was summoned to an evening meeting to ratify the action, but they put off a decision until next week.