Gynecomastia. Help me out in this.

Hello doctor. Im 30 years old male working in a Pvt sector. Im 6 feet tall and have a good physique, but there is something has literally been  depressing my physique and confidence for years. I was once a highly energetic guy with an atheletic body till school days. Once I crossed the age of 18.  After started exercising vigourously to have a muscular body, slowly i realised that there is slight increase of muscle formation in the nipple area. I thought it is bcos of improper workouts done for chest muscle. But the fat on chest nipple started increasing giving an odd look of a female breast. It was really embarrassing and I had to wear multiple vests to hide the nipple protuition. I thought it will eventually change as the age passes.  But till now nothing has changed. I recently visited a doctor who examined it and said that is gynecomastia which happens due to hormonal imbalance and surgery is the only option. Im literally worried. Kindly help me out in this.