Keep a ref on the plugin loader during an idle

Make sure the plugin_loader stays alive during the idle. Seems like a
fairly uncommon crash as I suspect it has to hit it during main loop
shutdown.

https://bugzilla.redhat.com/show_bug.cgi?id=1303228