This only seems to cause problems in some programs (Valve games like Team Fortress 2 and Left 4 Dead 2, plus anything that runs in Wine).

When I Alt-Tab away from the program, the program thinks the Alt key was never released. When I Alt-Tab (or click) back to the program, every key I press is modified by Alt. Pressing/releasing the Alt key once will fix the problem.

(As for what could be causing this, I'll speculate that it has something to do with the fact that games often capture keyboard input directly.)