How do I *completely* remove an IIS virtual directory?

I'm using the Microsoft.Web.Administration API to delete a virtual directory:

$mgr = new ServerManager()
$site = $mgr.Sites | Where-Object { $_.Name -eq 'MySite' }
$rootApp = $site.Applications | Where-Object { $_.Path -eq '/' }
$vdir = $rootApp.VirtualDirectories | Where-Object { $_.Path -eq 'MyVDir' }
$rootApp.VirtualDirectories.Remove($vdir)
$mgr.CommitChanges()

However, if someone has configured custom settings for that virtual directory, they remain in applicationHost.config:

<location path="MySite/MyVDir"><system.webServer><httpProtocol><customHeaders><add name="X-CustomHeader" value="CustomHeaderValue" /></customHeaders></httpProtocol></system.webServer></location>

When I remove my virtual directory, I'd also like to delete/remove any custom configuration for its location. How do I completely remove a virtual directory in this way?