As someone who has both played many a game and spotted many a glitch, it comes from a lot of things. As skylerdray said, part of it is just dicking around and finding something that works - and a lot of time it's just good/bad luck.

But as you've played a lot of games, and you get into something that just "doesn't look right" or "doesn't work right" or "is just close enough" you start looking for ways to break it. Initially it's not even to use it for an advantage like a shortcut, it's simply to do it. But once you do it, you start then looking for other applications of similar things - both in the same game and other games.

After doing this awhile, you start to get an eye for things that seem to have high potential to be broken, and you feel compelled to just give it a try to see if you can break it. It's kind of like what motivates stuff like black-hat and grey-hat hackers, you do it for the challenge of seeing if you can break it because at a certain point the actual game often becomes buggy and boring. When you do break it, it's a pretty rewarding feeling.

Then, eventually, you even start to look at areas and realize they have a high potential to be broken even across different games/developers. A pretty good example is stuff like walking through/into/up walls, or figuring out where someone didn't "quite" do their due diligence and left some hole for you to get behind those little invisible walls (tip: invisible walls in games are prime candidates for something to be broken as they usually speak to a brute force, band-aid fix).

Also, developers in many cases intentionally leave very hard to find things like this in game in many cases so that they can utilize themselves - potentially for debug purposes.

Then, you do like I did and get into Software Quality Assurance and make money pointing out to people all the stuff they fucked up and they go "How the fuck do you people find this stuff out???"