properly abort instead of looping infinitely

Fixes: Bug 568760 - nautilus freezes due to a bug in garray.c:322

* glib/garray.c: increase the size of potential return values by
  using an unsigned result; properly check if we still handle valid size
  proposals, return the original request if there's no usable size left
* tests/array-test.c: reproduce the error condition of the bug report