Sending “Pot of Greed” to the Graveyard is a cost.
If you have your opponent’s “Pot of Greed” in your hand because of “Exchange” or “Graverobber”, you CAN send your opponent’s “Pot of Greed” to the Graveyard to activate your “Avatar of the Pot”’s effect. But if you did this with “Graverobber”, you take 2000 points of damage to do so.
You cannot activate “Avatar of the Pot”’s effect while “Banisher of the Light” is in play.