cache-test: fix race condition

The refcounting test had a race condition. Fix it by just waiting for
the stream to be destroyed, rather than asserting it already has been.

https://bugzilla.gnome.org/show_bug.cgi?id=698305