Third Party Contract on Ledger

When we create a third party contract, that third party is automatically added to the ledger. However, when a third party contract is created in error and we delete it, it does not get deleted from the ledger. We would like the ledger for the corresponding contract to also be removed.