Seems like it's yet another bug, thought this may be handy for anyone who runs into this in the future.
Create a Web Services Datawindow, do not map Insert, Update and Delete to anything. Retrieve data into and call DeleteRow, your data has just disappeared. It should be in the Delete buffer, but it isn't! It isn't anywhere!
The solution is to map the Delete in the Web Services Datawindow to something, anything.
The reason I didn't want to map is because I was handling the Delete myself, I just wanted to get the keys from the delete buffer and do my own thing. I'm starting to think I am the only person using Web Services Datawindows (apart from Chris Pollach who has either solved all these problems already or just been very lucky!).