updated 02/19/2014 AT 1:00 PM ET
•originally published 02/19/2014 AT 3:00 PM ET
From his grown-out, ombré-tinted hair to his manicured nails, Bruce Jenner has been raising eyebrows with his recent transformation.
But when he underwent a controversial “tracheal shave” in January to reduce the size of his Adam’s apple, the rumor mill went into overdrive that the 64-year-old Olympic icon was transitioning from male to female.
“Absurd,” a close family friend tells PEOPLE in this week’s issue, one of many insiders who dismisses the rumors as false. (Jenner himself declined to comment.)
“Bruce has never expressed interest in becoming a woman,” says another source close to the family.
Rather, four months after his split from third wife Kris, 58, the father of 10 (including four stepchildren) “is having an identity crisis,” says the family insider.
Having publicly discussed his past plastic surgeries – including a partial facelift and nose job in the mid-1980s and a second facelift in 2009 – “Bruce is definitely concerned with his looks and always has been,” says the insider, adding that appearing on Keeping up with the Kardashians for nine seasons “made him even more hyper aware.”
Says the family friend: “The guy is getting older and doing what he can to keep a youthful look. If that means plastic surgery, so be it.”
Reporting by JENNIFER GARCIA and AILI NAHAS
For more on this story, including a look at Bruce’s changing looks through the years, pick up this week’s issue of PEOPLE, on newsstands Friday