One of the most common excuses Americans use to not travel internationally is that they don't speak the language. As any Aussie, Kiwi, or Brit can tell you, this is a dumb excuse. More than you probably realize, thanks to a combination of the British Empire, the Marshall Plan, and the budget of Hollywood and American pop music, English is the international language of the world. When two people from different countries with different languages meet, 95% of the time, they will speak to each other in English.
That said, nobody likes the tourist who walks up to locals, asks for something in English, and when not understood, repeats themselves more slowly and loudly. The surprising thing is that most of the people who do this aren't native English speakers themselves (and are probably repeating themselves because they think they messed up). The point, however, is that to have a deeper experience in a country, you need to speak a little of the local tongue.
This is a problem. Learning a language can take years. If, for example, you want to do the traditional backpacker's circuit in SE Asia of Thailand, Laos, Cambodia, and Vietnam, that's four languages to learn right there, ignoring regional variations complicated enough to make people from opposite ends of each country not understand a word the other is saying. Unless you're an amazing linguist (I've met them, they exist) learning them all could take you half a decade or more. I don't know about you, but I'm not that patient.
So, what do you do? Strike a quick balance. How quick depends on how long you will be in the area. If you're in a place for more than a month, consider at least language tapes or a lesson or two before and during your stay. While traveling abroad local language classes are usually cheap and easy to find. If you're there for a few weeks, get a phrasebook published in your country (if you speak with an American accent and you get a British-based phrasebook, the pronunciation guide will be completely off). If you're not going to be in the country very long, then you just need the very basics. Here's where I usually start:
The first four things I try to learn are "Hello", "Thank you", "Sorry/Excuse me," and here's the big one: "What is ____ in (your language)?" Most people miss this his last one. This question, combined with pointing, is how you're going to actually get a deeper experience out of your travels. Full disclosure: you're going to forget 90% of what you are told almost immediately after they tell you, but that other 10% will come in useful, and there's a good chance that you'll recognize at least some of the first 90% the next time you hear it. More importantly, you are already ahead of the average traveler because you are displaying a genuine interest in this person's home and people, and it takes them out of the awkward situation of bending over backwards to try to communicate and help you in a language they're trying to remember from high school.
Next, learn what language quirks you should be familiar with to be polite. For example, in Spanish, there are two words for "you": tú and usted. Depending on the country, you'll hear both used very often. But if "tú" should only be used with people you are really familiar with (and maybe small children and pets) or you'll offend someone by being overly familiar. In Thai, it is considered respectful to end all your sentences with either "khaa," if you're a woman, or a higher-pitched "krup" if you're a man. Things like that. Guide books will tell you these things-- borrow a friend's if yours doesn't. If you can't find it in a guidebook, you can usually find out at the nearest tourist anything-- hotel, info counter, transit center, or anything else with at least a sign on the wall in English.
If the written language doesn't use characters you know, the obvious temptation is to learn it so that you can read street signs and building names. It's not a bad thing to learn, but that's not really the right reason. Street signs are notorious worldwide for simply not existing when you need them, and major cities' signs are usually transliterated into the Roman alphabet anyway. You're generally better off with a map, a friendly local (remember "excuse me") and a little gesturing. Don't forget the "thank you" afterward.
There is, however, one thing you will want to know written: restroom. There are times when you need to know where the toilet is and you won't want to waste time with a map and gestures. Not every toilet in the world had a little white person with pants or a dress over a blue background on the door.
Finally you'll want a set of emergency words. "Toilet" aside, there are a couple more words you won't need or hear very often, but that when you want them, you want them right away: "Help," "Stop," and "Doctor," should all be in your vocabulary, for obvious reasons. Just don't wander down the street practicing them out loud. Also, while not really an emergency thing, I'd learn the word for "soap." It comes up more often than you might expect.
After that, it's mostly a question of not being shy and remembering to keep a smile on your face. Pass around the phrasebook if you have it. Figure out nonverbal ways of communication (don't underestimate the power of the gesture). Don't be afraid to look dumb and make mistakes. The fastest way to learn a language is to talk to people and read. The rewards are immediate, not just in feeling good about being better than the average tourist, but also in things like free food and drinks, behind the scenes "tours", new friends, and great stories. Even "hello" by itself is enough to turn a suspicious look at a foreign stranger into a big smile.