What causes relative start time to go up?

Relative Start time occurs within the browser and are typically caused because the browser is waiting for another call to complete before starting the next object download. For example a picture on the page cannot begin downloading until the browser downloads the main HTML then parses and builds the DOM from that content. The relative start time of the first object will be delayed by the amount of time that takes to complete the initial download.

Also note that the browser will only allow a specific number of simultaneous connections to the server by default, so Relative Start time maybe be delayed waiting for available connections within the browser. In some cases delays may also be caused because dependencies on the page cannot begin downloading until prior objects loaded on the page have completed.

Looking at the Total Response Time of calls that occur ahead of the calls in question will often give you some enlightenment into this as well. A long relative start time can usually be attributed to a call ahead of the object in question that has a very long total response time. This becomes clearer visibly when viewing a waterfall report (HTTP Step Report). Calls that precede the object in question will typically have longer RTs that can be correlated to the long relative start times of calls that occur after.