It's not a surprise that Bill O'Reilly and The View's Joy Behar may not see eye to eye politically. But a discussion about Muslims and the Sept. 11 terrorist attacks on Thursday's show set off a firestorm – promoting the redhead and her co-host Whoopi Goldberg to leave the set.

During a discussion about the proposed Islamic cultural center downtown Manhattan, O'Reilly insisted it would be "inappropriate" to have a Muslim place of worship so near to Ground Zero.

This prompted Whoopi Goldberg to inquire, "Why is it inappropriate?"

To which O'Reilly replied, "Because Muslims killed us on 9/11."

As the cast devolved into loud protests and O'Reilly defended his remark, Goldberg shot back that "extremists" attacked the World Trade Center while the Fox News host continued to defend his views.

With that, Behar rose from the couch, visibly upset. "I don't want to sit here," she told O'Reilly, as she stood over him.

Goldberg also calmly rose from the sofa and joined her, waving as she walked off the set.

After the ladies left, Barbara Walters told the audience, "we should be able to have discussion without . . . walking off stage."

The pair later returned, and had a calmer debate with their guest to finish out the show.