The family flew back from Greece after just a day on holiday there, amid intense criticism.