For an oil based massage you should undress to your own level of comfort, some people prefer to be naked during their massage whilst others prefer to wear underwear.
