The answer to this is simply that you need another set of kit for the freezing times.
You need an electrically heated jacket liner (I have a Gialli, it has heated sleeves as well) which takes the place of the original jacket lining.
Underneath that, take a look at the "Paramo Trail shirt"
http://www.paramo.co.uk/en-gb/garmen...SHIRTTRAILMENS
It's furry on one side and is as warm as a woolly but very thin. With this combination I have ridden 400 mile days in freezing weather. Also, you need heated grips. You need a pair of boots one size too big so that you can get thick woolly socks inside.