I found the problem...
The host is actually used from my POM file but if you just have a test case without any assertions (which you shouldn't have though) then you can specify a non-existing host and it will still say 'BUILD SUCCESS' while it didn't do the web service call at all.
It is quite misleading because the output even shows log messages that SOAPUI is running your test case while it doesn't actually run at all!
09:50:07,185 INFO [SoapUITestCaseRunner] Running soapUI tests in project [ABC RESTful Web Service Test]
09:50:07,185 INFO [SoapUITestCaseRunner] Running Project [ABC RESTful Web Service Test], runType = SEQUENTIAL
09:50:07,200 INFO [SoapUITestCaseRunner] Running soapUI testcase [test getTest]
09:50:07,200 INFO [SoapUITestCaseRunner] running step [getTest - Request 1]
09:50:08,230 INFO [SoapUITestCaseRunner] Assertion [Contains] has status VALID
09:50:08,230 INFO [SoapUITestCaseRunner] Finished running soapUI testcase [test getTest], time taken: 97ms, status: FINISHED
09:50:08,230 INFO [SoapUITestCaseRunner] Project [ABC RESTful Web Service Test] finished with status [FINISHED] in 1045ms
This is how my POM file currently looks like
<build>
<plugins>
<plugin>
<groupId>eviware</groupId>
<artifactId>maven-soapui-plugin</artifactId>
<version>3.6.1</version>
<executions>
<execution>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
<configuration>
<projectFile>src/test/soapui-project.xml</projectFile>
<host>someOtherServer:8080</host>
</configuration>
</plugin>
</plugins>
</build>