The proposed visit has been sparking controversy since Prime Minister Theresa May extended the invitation in January