Stephen Phillips, whose resignation has sparked a ‘Brexit by-election’, was also unhappy about other aspects of the Conservatives’ drift to the right, friends said.