updated 03/04/2014 AT 7:30 AM ET
•originally published 03/03/2014 AT 2:10 PM ET
In every season of Survivor, someone has to be the first to go. Sometimes it’s because they’re weak; other times because they’re abrasive. David Samson, the 45-year-old president of the Miami Marlins, was chosen as the leader of his tribe, and unceremoniously booted from Survivor: Cagayan after three days.
Samson talks to PEOPLE about leadership, hunger and the contestant he calls “disgraceful.”
David, what happened?
We had six brainiac leaders and no followers. We got nothing accomplished. So when they had to vote someone off, they chose the person they had selected as the leader. It was disappointing.
Were you a fan of the show before going out there?
I’ve been watching for 13 years. It was never the right time to try out, and I finally had an epiphany that I needed to make it the right time. So I started the process.
Did any of it have to do with your profession? Maybe they thought you had money.
I didn’t tell them what I do for a living. Instead, I said I worked in the front office of the Marlins. I don’t think any of them recognized me, so my career wasn’t really an issue.
But somehow the tribe must have sensed that you were a leader.
I was wearing a blazer, which made me look like an executive. So everyone just decided to make me the leader. I was not happy about that at all. When you’re the leader, you’re a target. You saw how Garrett was gunning for me after I chose him as the weakest.
Why did you choose him? He was the most buff person on the tribe.
I just had to make a decision, and I chose someone who is clearly not the weakest. No matter who I chose, they were going to be mad at me. It was a no-win situation.