Joomla – Error: the XML response that was returned from the server is invalid

Today I have had a problem to install a Joomla 1.5 on my VPS server. In fact I was able to install it but without Sample Data. The error message that I was getting was:
Error: the XML response that was returned from the server is invalid

I took me hour to find the problem. If you come across this problem do not create a configuration.php script at the beginning of the installation even if the installer will complain about it.