updated 08/02/2013 AT 8:50 AM ET
•originally published 08/01/2013 AT 8:50 PM ET
The X Factor judge first tried deflecting the issue with a quip. “I haven’t read the newspapers,” he said Thursday at the Television Critics Association meeting in Beverly Hills.
Then he offered a no comment: “Unfortunately at this moment I have to keep it private.”
His X Factor cohort Demi Lovato jumped to his defense. “I don’t think you have to answer that!” she said.
Cowell later told PEOPLE: “There are a lot of things I will eventually clear up when the time is right, but I really have to be sensitive because there’s a lot of people’s feelings involved here.
Cowell, 53, is expecting his first child with New York socialite Lauren Silverman, 36, the wife of Cowell’s friend Andrew Silverman.
Andrew Silverman, who has reportedly filed divorce papers in New York listing “adultery” as the reason for their split, issued a statement Thursday saying his “top priority” was his 7-year-old son ” while working to resolve all outstanding issues with my wife.”