Don't trust remote permissions mask fully

We don't actually know what gids we're in, and also there may be other
things not visible via sftp like ACLs that decide the user
permissions. So, we only use the permissions for group and other if
they result in a positive, otherwise we leave the attibute unset
(meaning "don't know").