Can it hurt to ask?

The phrase "it never hurts to ask" is a fairly common one. I never thought much of it until a couple of years ago. I can't remember the circumstances off-hand, but I distinctly remember a series of (fortunate) events that basically happened only because I asked and got a 'yes' when I didn't expect one. Suddenly, I took to heart the corollary to the "it never hurts to ask" phrase, which is that "the worst they can do is say no."

Lately, however, I've been wondering if that's as true as it seems. There are all kinds of obvious possible ways that asking can go awry, not least of which is how it comes across to the person you are asking (e.g. if you are asking someone to take on a task you were originally assigned, does it seem like you are trying to shirk your responsibility? what does it say about you if you ask someone for a given thing or a particular question?). What I'm wondering is related to a situation (which I think may be more common than you'd initially expect) where you ask someone to do something and you know they may not want to but probably won't say no. In most cases, the person is someone who hates to say no to friends/bosses/etc. So it seems to be to be a bit manipulative to ask in the first place. One could also argue that the person needs to get better at saying no if they don't want to do something. But if you know that they are bad at saying no, is it taking advantage of that to ask anyway?

For what it's worth, I'm writing this having been on both sides (the asker and the askee who can't say no), and I'm not really sure how I feel about either side.