updated 02/13/2012 AT 8:00 AM ET
•originally published 02/13/2012 AT 9:00 AM ET
Ray J was supposed to be celebrating his sister Brandy’s birthday Saturday night. Then he heard the news about Whitney Houston.
An extremely close friend of the singer’s, Ray J (real name: William Ray Norwood Jr.), 31, broke down and cried at Ne-Yo and Compound Entertainment’s annual Midnight Brunch in Los Angeles. The pre-Grammy Awards party, presented by Malibu Red, had become an impromptu tribute to Houston.
Ray J and Brandy, 33, arrived around 1:50 a.m. and immediately went to their table. Then, Ray J, with his sunglasses on and his hood up, “couldn’t hold it together,” an onlooker tells PEOPLE. “He was sobbing.”
Ray J had ordered a birthday cake, complete with sparklers, in advance for Brandy, but when staff members brought it out, the two left it untouched. By the time Marsha Ambrosius sang “Happy Birthday” to Brandy, she and her brother were already gone, barely 20 minutes after they arrived.
“They only stayed a few minutes before becoming emotional and left early,” says the onlooker.
Houston herself had been scheduled to attend the event, following Clive Davis’s Grammy party earlier in the evening.
Ray J, perhaps best known as Kim Kardashian’s ex – and for the sex tape they made together – had been linked romantically to Houston on and off for many years. They were spotted in public together as early at 2007, the year Houston and Bobby Brown divorced.
The day before Houston’s death, Ray J appeared on The Insider to discuss his new book, Death of the Cheating Man, and was asked about his relationship with Houston.
“We’ve always just been friends,” he said. “We hug, we laugh, we go out to dinner … We talk about, you know, different things in our lives and she’s a cool person.”
Asked if his relationship with Houston was physical, he said: “We’ve always kept our relationship as private as we could, but when we go out, she’s such an icon, it reaches the masses. But for the most part, we’ve always been friends, and we’ve kept everything hush-hush and just quiet, and we really don’t focus on that.”