copy-worker: Don't attempt to delete the skeleton directory

This would have to be a recursive deletion to work but we don't even
need to do that since the permissions on it don't allow anyone else to
snoop and it will go away in the next reboot.

https://bugzilla.gnome.org/show_bug.cgi?id=701100