Your headset is loose.
Loosen the bolts in the stem where it clamps onto the steerer. Then tighten the bolt in the top cap of the fork until the play is gone, this is known as pre-loading the headset. ( The best way to do this is to hold the front brake and try and move the front of the bike back and forth. ) When you are happy that it is just tight enough so that there is no more movement tighten up the stem bolts ( to the recommended torque ) and you will have not more problems.
If you're still a bit unsure, take it to your bike shop and they will sort you out.
More info here:
See section: Headset Adjustment - Threadless Type
http://www.parktool.com/repair/readhowto.asp?id=65
Allen.