Simulations are used to investigate for the first time the anisotropy of the dielectric response and the effects of an applied electric field E_ex on the phase diagram of water. In the presence of electric fields ice II disappears from the phase diagram. When E_ex is applied in the direction perpendicular to the ac crystallographic plane the melting temperatures of ices III and V increase whereas that of ice Ih is hardly affected. Ice III also disappears as a stable phase when E_ex is applied in the direction perpendicular to the ab plane. E_ex increases by a small amount the critical temperature and reduces slightly the temperature of the maximum density of liquid water. The presence E_ex modifies all phase transitions of water but its effect on solid-solid and solid-fluid transitions seems to be more important and different depending on the direction of E_ex .