Riding in extremely cold weather is not really a problem if you prepare properly. That is because when you are riding a bicycle, you generate quite a bit of heat. If you dress appropriately, you will actually be staying pretty warm while riding. The exceptions are, of course, your extremities. Keeping your toes and fingers warm can present a challenge.
As far as your core goes, dress in layers and try to make the bottom layer a wicking material and one that maintains its thermal qualities even when wet; like wool, polypropylene… I actually like to wear the quick drying swim shirts that are designed to protect you from the sun in the summer. They typically are long sleeved and quick drying. Over this, I will wear either a polar fleece or a wool sweater. Sometimes I go with a cotton sweater but that is not as good since it loses a lot of its thermal properties once it gets wet. On top, I wear a bicycling jacket. It is simply a fancy windbreaker with lots of reflective material. I wear it year round, so it is nothing super warm. In fact, I have the vents open on it most of the time. It really is just to break the wind. The other layers do all of the insulating.
In keeping your hands warm, I have found that wearing mittens is far better than gloves. However, I sometimes wear gloves inside of my mittens. In this manner, if I need to remove my mittens for some finger dexterity I still have a layer of protection. I wear large military surplus artic mittens. They are bulky, but they are not really a problem for manipulating my brakes nor my gear shifters. I have bar end shifters and just regular old hand brake levers on my bike. Grip shifters or shifters integrated into the brake levers might prove more difficult. You can also purchase big bag like things called “bar mitts”that go over your handlebars. (For example, http://cozywinters.com/shop/bar-mitts.html?VS=Bar-Mitts-LGBSHIM&utm_source=bing&utm_medium=cpc&utm_campaign=PLA_HeatedGloves_BING&utm_term=4581596233548553&utm_content=Heated%20Gloves) You put your hands inside the bag and have the freedom to move your hand around within the confines of the bag.
Layer up on your feet as well; wear two or more pairs of socks, preferably wool. I can get away with this because I wear bicycling sandals that fasten with Velcro. I can adjust them to fit comfortably with no socks to probably four pairs of socks. Although, I have never worn more than two at a time so far. Additionally, if it gets brutally cold, I will add plastic grocery bags over my socks to cut the wind and trap in heat. You could also purchase neoprene booties. In years past, I have replaced my pedals with winter pedals that could accommodate my Sorel boots. No problems with the cold, but I did not like this solution so much.
Please wear a helmet. Fit something like a wool cap or polypropylene under it to keep you warm. But, do not leave your helmet at home. In cold weather, you are more likely to need the protection of the helmet due to icy conditions. I hope that you never need it. However, you are more likely to hit a patch of ice and go down, or heaven forbid, a car could hit an icy patch and take you down. Either way, wear the helmet just in case.
If you have no ride stopping events, you will finish your ride saturated with sweat and actually toasty warm. The biggest danger is if you have a flat or other mechanical mishap. Your heat generation will stop and the sweat that you had worked up will start to work against you. This is when hypothermia becomes a real danger. Keep moving and keep warm. At any rate, don’t let the cold weather keep you off of your bike! Get out there and enjoy a cold weather bicycle ride. See you on the road!