Does ReadyAPI need to be installed in jenkins server? If yes then will it work with trial version ?

Yes, Jenkins would be treated as a user executing the tests so the Jenkins server would require a licence.

You can do this in a couple of ways

Using Ready! API

One option to activate a license for Ready! API's Jenkins plugin is to install Ready! API and activate a license through itsGUI.

On the computer running Jenkins:

• Install Ready! API as you would normally do this. See Ready! API Installation for more information.
• After installing Ready! API, run it and activate a Fixed license in a usual manner. See Activating Licenses for step-by-step description.

Using TestRunner

If you are going to run tests from Jenkins by using Ready! API's TestRunner, then you can use the testrunner.bat or testrunner.sh script for license activation:

• Download a zip or tarball version of Ready! API for your platform from the Ready! API Downloads page of the SmartBear web site.
• Unpack files from the archive to some directory on the computer running Jenkins.
• Prepare a command-line for the TestRunner. See TestRunner Command-Line Arguments.
• Copy the license file to the computer running Jenkins. You can find this file in the email with licensing information that you received from SmartBear.
• On the computer running Jenkins, open the command line window and go to the bin subdirectory of the directory, where you placed the extracted files.
• Run the script testrunner.bat or testrunner.sh with the command line arguments for the test run.
• TestRunner will search for the license and will ask you to specify the license file. Just follow the instructions that you will see on screen.