Well, while it might be possible to find a good deal on a recorder at Office Depot, if they just happened to have one where you researched the specs and etc.. Mostly those are intended for use more as a voice notepad or for recording conferences.  Noise reduction like they have is for eliminating handling noise and air currents mostly and with the limited audio bandwidth that is specialized for voice just isn't going to be what you want for sound quality.  It's going to butcher sound pretty bad, so far as getting nice transparent audio recording.

In your price range, Pf, and having some idea what you're likely to want for sound quality.. Think about a used laptop. 

That is a project I am working in right now, since I was given one recently.  I want a nice little portable studio that can go on location anywhere and where I don't have to deal with too many compromises so far as capability.

It is probably bigger than you want to consider, since it wouldn't be "pocket sized sexy" tech.  But even an older laptop is going to have more horsepower and a bigger screen for things like editing and maybe some multitrack.

Myself, if I can keep it down to a briefcase and the laptop case, that will be portable enough.  I can fit in an 8 channel mixer, a couple half-rack units (compressor limiter and multi-fx), a tube pre-amp for my studio mic, my studio mic, and a rechargeable battery pack to run the small studio gear if AC power isn't available.  With a nice full function recording utility like Audacity or a DAW like Darkwave Studio, I can do a lot with that.  I don't want much, yknow?  LOL

For on-the-spot quick catches of impromptu jams and etc, some cell phones aren't so bad these days.  It will always be some trade-off, but if you carry a cell anyway, you may as well make the thing work for a living.