Sounds exactly like shimmy or speed wobble.
All bikes (can) do it, but especially large riders since the taller frames (and longer steerers, stems, cranks, seat posts, etc.) are generally more fexible.
Since any part of the system can cause (or eliminate) wobble, go ahead and try changing whatever you were thinking of changing next and you just might tip the stiffness up past a threashold where you don't get it so often.
But when you get a speed wobble the next time remember there are two easy ways to stop it:
1. stand up (just a centeimeter or so, enough to shift weight from the sadde to pedals, or
2. grip the top tube with your knees
More info here:
http://www.sheldonbrown.com/brandt/shimmy.html
Subject: Shimmy or Speed Wobble
From: Jobst Brandt
Date: June 25, 2004, revised February 25, 2005
Shimmy, a spontaneous steering oscillation of the front wheel, usually occurs at a predictable speed when riding no-hands. The likelihood of shimmy is greatest when the only rider-to-bicycle contact is at the saddle and pedals. This position gives the least damping by hands, arms, and legs. When shimmy occurs on descents, with hands on the bars, it is highly disconcerting because the most common rider response, of gripping the bars firmly, only increases it.
Shimmy is not related to frame alignment or loose bearings, as is often claimed. Shimmy results from dynamics of front wheel rotation, mass of the handlebars, elasticity of the frame, and where the rider contacts the bicycle. Both perfectly aligned bicycles and ones with wheels out of plane to one another shimmy nearly equally well. It is as likely with properly adjusted bearings as loose ones. The idea that shimmy is caused by loose head bearings or frame misalignment seems to have established currency by repetition, although there is no evidence to link these defects with shimmy.
Bicycle shimmy is the lateral oscillation of the head tube about the road contact point of the front wheel and depends largely on frame geometry and the elasticity of the top and down tubes. It is driven by gyroscopic forces of the front wheel, making it largely speed dependent. It cannot be fixed by adjustments because it is inherent to the geometry and elasticity of the bicycle frame. The longer the frame and the higher the saddle, the greater the tendency to shimmy, other things being equal. Weight distribution also has no effect on shimmy although where that weight contacts the frame does. Bicycle shimmy is unchanged when riding no-hands, whether leaning forward or backward.
Shimmy requires a spring and a mass about which to oscillate and these are furnished by the frame and seated rider. Unloading the saddle (without standing up) will stop shimmy. Pedaling or rough road will also reduce the tendency to shimmy. In contrast, coasting no-hands downhill on a smooth road at more than 20mph with the cranks vertical seems to be the most shimmy prone condition.
When coasting no-hands, laying one leg against the top tube is the most common way to inhibit shimmy and also one of the most common ways to coast no-hands. Compliant tread of knobby tires usually have sufficient squirming damping to suppress shimmy. Weight of the handlebar and its extension from of the steering axis also affects shimmy.
Shimmy is caused by the gyroscopic force of the front wheel whose tilt is roughly at right angles to the steering axis, making the wheel steer to the left when it leans to the left. This steering action twists the toptube and downtube, storing energy that both limits travel and causes a return swing. Trail (caster) of the fork acts on the wheel to limit these excursions and return them toward center.
To feel the gyroscopic forces involved in bicycle shimmy, take a front wheel, holding it by its axle in both hands, and give it a spin. Manually steering it from side to side generates strong tilting forces always at right angles to the input. These forces sustain shimmy and are the motions one uses to make quick steering maneuvers while riding no-hands, shifting the hips laterally while firmly seated. The same effect as when wheeling a bicycle wile holding it only by the saddle.
Shimmy that concerns riders the most occurs with hands firmly on the bars and it is rider generated by muscular effect whose natural response is the same as the shimmy frequency, about that of Human shivering. Descending in cold weather can be difficult for this reason. The rider's "death grip" only enhances the incidence of shimmy in this situation. Loosely holding the bars between thumb and forefinger is a way of avoiding shimmy when cold.