coretext: don't insert item in the hash if it originated from the hash

Oversight in my fallback fix, this resulted in things being wrongly
destroyed due to unrefs. Oops.