updated 07/31/2013 AT 10:00 AM ET
•originally published 07/31/2013 AT 11:15 AM ET
It’s true. Simon Cowell is finally going to be a dad!
The X Factor judge, 53, is expecting his first child with New York socialite Lauren Silverman, 36, multiple sources confirm to PEOPLE. The news has been hot gossip among friends recently because Silverman is married to Cowell’s close friend, real estate mogul Andrew Silverman.
The Silvermans have filed for divorce, a source tells PEOPLE.
When reached by PEOPLE, an assistant to Cowell’s U.K. rep says, “This is a very sensitive issue and all media enquiries are being dealt with by his U.S. attorney.” No other calls to reps were returned.
So how does Cowell, who has shattered the dreams of many a kid on Idol and X Factor, feel about having kids? It depends when you ask him.
“God, no. I couldn’t have children. If I had them here drawing on the walls I’d go nuts,” he said in 2009. “You’ve got to be up at a certain time. Got to listen. When all you want to do is sit in a corner thinking.”
But in an interview with AOL Music in March 2012, Cowell seemed more open to the idea.