Maybe.
Just a comment- before you try and change to a small, (I have a medium and it BARELY fits me-I CAN'T wear a cap underneath it... I wear a 7 3/8 US normally,)
Consider wearing a cycling cap underneath it. I prefer to wear a cap under my helmets as they absorb the sweat, provide a small sun visor, and protect against getting stung by bees.
In cooler weather, it's great to have a cap under your helmet.
I mostly bike on Maui and ALWAYS wear a cap underneath, and it's usually 88-90F at sea level, and around 70F where I bike up to on Haleakula and ride around. I've run into bees three times since wearing a cap, and it's prevented any problems. I started using a cap under the helmet after getting stung riding in KONA. (I hit the bees at about 33 mph and by the time I got stopped and my helmet off, I got stung 3 times!!)
Hope this helps. (I'm getting a Large for this purpose)
Aloha,
Jeff