Janet Jackson and sister LaToya arrived at their brother's rented Holmby Hills mansion on Saturday as moving trucks carted off some of the singer's – and his children's – possessions.

A source told PEOPLE that the belongings of Michael Jackson's three children were being removed from the house and taken to the Jackson family home in Encino. The children have been holed up with grandparents – Joe and Katherine – at the San Fernando Valley house since the singer died on June 25.

Janet was at the home during the move, while LaToya showed up later in the day. Some of the singer's prized possessions were also being removed from the house for safekeeping, the source said, because there is concern that someone could break into the home.

Jackson's children – Prince, 12, Paris, 11, and Blanket, 7 – remained with their grandparents on Saturday as relatives came in and out of the Encino house. Jackson's parents have expressed their desire to keep the children, even though Prince and Paris's biological mother Debbie Rowe could gain custody if she chooses.

"They haven't heard from Debbie Rowe and don't know what’s going on in that respect, either," said a family source. "But Joe and Katherine really want the children."