Forum Discussion

ramaG's avatar
ramaG
Contributor
4 years ago
Solved

Is there a script to launch WinSCP from ReadyAPI and navigate to the appropriate server logfile path

  • As part of our E2E testing we need to validate application logfiles on our application servers and also need to verify the same information retrieved from the browser based app/utility.

  • Once our webservice call (Ex: Creating Users)  is successful, we need to login to our application server logs using WINSCP( window’s based application which we use to navigate to application server logs) to check for the data in the app server logs . 
  • Does anyone have a script to launch the WinSCP tool and navigate the  appropriate server log file path(s)?
  • Please let me know
  • TNeuschwanger's avatar
    TNeuschwanger
    4 years ago

    Hello ramaG,

     

    I don't have a WinSCP specific script, but I have used jsch library to access server and retrieve a log file (or execute a command on a server).  Below is sample code to be hacked up and use what you can (file transfer or execute remote command).

     

    I think it is hard to get over to a server and then start issuing commands unless you code a script on the remote server and execute it.  I found it easier just to download entire log file and process it locally than to robot through on the server where the logs reside.  Our log files are sometimes very large and are rolled over when they hit a certain size.  This technique has worked for me even though there were millions of records in the log file.  Depending on how large your log files are and how long you are willing to wait for a file transfer will determine if this technique works for you. 

     

    The jsch library will need to be obtained from "http://www.jcraft.com/jsch/" since it is not included with SoapUI (maybe these days it is but current free version does not include it).  I love this helper library and has saved me from a lot of trouble in the past. 

     

    Groovy Script Common Get File Transfer:

    import java.text.SimpleDateFormat;
import com.jcraft.jsch.*;

def readProperty(sName, pS) {
   def step = testRunner.testCase.getTestStepByName(sName);
   def propertyStr = step.getPropertyValue( pS );
   return propertyStr;
};

def writeProperty(sName, pS, pV) {
   def step = testRunner.testCase.getTestStepByName(sName);
   def propertyStr = step.setPropertyValue( pS, pV );
   return true;
};

def determinLocalOS() {   // Determine Operating System function.  Purpose is to define if this script is running locally on Windows or Linux
   def homeProp = System.getProperty("user.home");  //log.info "homeProp: " + homeProp;
   def localRtn = homeProp.contains("\\");   //log.info "localOS: " + localOS;
   return localRtn;  // boolean...  Windows = true, other = false
};

def fileTransfer(cmd, uId, svr, fileF, fileT) {   // File Transfer function.  Purpose is to execute a file transfer using the input parameters.
   def sout = new StringBuffer()
   def serr = new StringBuffer()
   log.info "CommandHarness: " + cmd;
   log.info "UserId: " + uId;
   log.info "Server: " + svr;
   log.info "FileFrom: " + fileF;
   log.info "FileTo: " + fileT;
   def cmdExecute = [cmd, uId, svr, fileF, fileT];  //
   log.info "cmdExecute: " + cmdExecute;   
   Process x = cmdExecute.execute();   // 
   x.consumeProcessOutput(sout, serr);    //  #### IMPORTANT #### This line must be here or the file transfer using PSFTP on Windows will not work for servers uslcappp03 and uslcappp04.
   x.waitFor();
   log.info "x=${x.text}";
   log.info 'sout: ' + sout;
   log.info 'serr: ' + serr;
   return "code: ${ x.exitValue()}";
};

def Session session;
def Channel channel;
def ChannelSftp channelSftp;
def JSch jsch;

//=========================================================================================================================================

def connect(String hostName, String userName, String passCode) {
   try {
      jsch = new JSch();
      session = jsch.getSession( userName, hostName, 22 );
      session.setPassword(passCode);
      session.setTimeout(3000);
      java.util.Properties config = new java.util.Properties();
      config.put("StrictHostKeyChecking", "no");
      session.setConfig(config);
      session.connect();
      if (session.isConnected) {
         channel = session.openChannel("sftp");
         channel.connect();
         channelSftp = ( ChannelSftp ) channel;
         log.info 'Connected to: ' + hostName;
      }
   }
   catch (JSchException e) {
      log.error e.toString();
   };
};

def changeDirectory( String remotePath ) {
   try {
      log.info 'Path before ChangeDirectory is: ' + getPath();
      channelSftp.cd( remotePath );
      log.info 'Path after ChangeDirectory is: ' + getPath();
   }
   catch( Exception e ) {
      disconnect();
      log.error e.toString();
   };
};

def disconnect() {
   channelSftp.quit();
   channelSftp.disconnect();
   channel.disconnect();
   session.disconnect();
   log.info "Disconnected";
};

def boolean isConnected() {
   return session == null ? false : session.isConnected();
};

def String getPath() {
   return channelSftp.pwd()+"/";
};

def String getList(String remotePath) {
   osList = channelSftp.lstat(remotePath)+"/";
   return osList;
};

def getStat(String remotePath) {
   return channelSftp.lstat(remotePath);
};

def getFile(String rFQFN, String lFQFN) {
   log.info 'Fully qualified file name on remote server: ' + rFQFN;
   log.info 'Fully qualified file name on local computer: ' + lFQFN;
   channelSftp.get(rFQFN, lFQFN);
};


//=========================================================================================================================================

log.info "";
log.info 'Test Step "Groovy Script Common Get File Transfer" Begin';

def remoteHost = readProperty("PropertiesRemote", "remoteLogHostPrefix") + readProperty("PropertiesRemote", "remoteLogHostIndex");  //
def remoteUser = readProperty("PropertiesRemote", "remoteUser");
def remotePassword = readProperty("PropertiesRemote", "remotePassword");
def remotePath = readProperty("PropertiesRemote", "remotePath");
def workFileName = readProperty("PropertiesRemote", "workFileName");
def tmpDirStr = readProperty("PropertiesRemote", "tmpDirStr");

def remotePathFileName = workFileName;  //
log.info "remotePathFileName: " + remotePathFileName;

def localPathFileName = tmpDirStr + workFileName;  //
log.info "localPathFileName: " + localPathFileName;

dateTimeFileStamp = new SimpleDateFormat ( "yyyy-MM-dd hh-mm-ss a" );
dtStr = dateTimeFileStamp.format(new Date());

if (determinLocalOS()) {    // boolean...  Windows = true, other = false
//log.info "remoteHost: " + remoteHost;
//log.info "remoteUser: " + remoteUser;
//log.info "remotePassword: " + remotePassword;
   connect (remoteHost, remoteUser, remotePassword);
   if (isConnected()) {
      getFile(remotePathFileName, localPathFileName);
      getFile(remotePathFileName, localPathFileName + dtStr + '.txt');   // helpful for local debugging if ran multiple times.
      disconnect();
   }
   else {
      log.info 'Connection failed to: ' + remoteHost;
   };
}
else {
   log.info 'return code=' + fileTransfer('/usr/pservices/hudson_home/filecopy_remote2local.sh', 'webopt', remoteHost, remotePathFileName, localPathFileName);
};

log.info 'Test Step "Groovy Script Common Get File Transfer" End';
log.info "";

     Groovy Script Common Command Execute:

    import com.jcraft.jsch.*;

def readProperty(sName, pS) {
   def step = testRunner.testCase.getTestStepByName(sName);
   def propertyStr = step.getPropertyValue( pS );
   return propertyStr;
};

def writeProperty(sName, pS, pV) {
   def step = testRunner.testCase.getTestStepByName(sName);
   def propertyStr = step.setPropertyValue( pS, pV );
   return true;
};

def determinLocalOS() {   // Determine Operating System function.  Purpose is to define if this script is running locally on Windows or Linux
   def homeProp = System.getProperty("user.home");  //log.info "homeProp: " + homeProp;
   def localRtn = homeProp.contains("\\");   //log.info "localOS: " + localOS;
   return localRtn;  // boolean...  Windows = true, other = false
};

def fileExecute(cmd, uId, svr, exeCmd) {   // Command Execution function.  Purpose is to execute command on a remote server.
   def sout = new StringBuffer();
   def serr = new StringBuffer();
   log.info "CommandHarness: " + cmd;
   log.info "UserId: " + uId;
   log.info "Server: " + svr;
   log.info "Executable: " + exeCmd;
   def cmdExecute = [cmd, uId, svr, exeCmd];  //
   log.info "cmdExecute: " + cmdExecute;
   Process x = cmdExecute.execute();   //
   x.consumeProcessOutput(sout, serr);
   x.waitFor();
   log.info "x=${x.text}";
   log.info 'sout: ' + sout;
   log.info 'serr: ' + serr;
   return "code: ${ x.exitValue()}";
};

def WaitTimeSeconds(wT) {
   log.info 'wait time (seconds) = ' + wT;
   wT = wT * 1000;  // Convert seconds to milliseconds for use in commands
   done = false;
   new Thread({ sleep(wT) ; done = true }).start();
   for (count = 0; !done; ++count) {  //      log.info "Here! ${count}"
      sleep(1000);
   };  //log.info 'wT = ' + wT;
   return true;
};

def Session session;
def Channel channel;
def Integer executeCommand(String hostName, String userName, String passCode, String cmd) {
   log.info 'command to execute =>' + cmd + '<';
   def commandNotFinished = -1;   // -1 is returned by jsch while the command is running
   def rslt = commandNotFinished;
   try {
      def jsch = new JSch();
      session = jsch.getSession(userName, hostName, 22);
      session.setPassword(passCode);
      session.setTimeout(5000);
      java.util.Properties config = new java.util.Properties();
      config.put("StrictHostKeyChecking", "no");
      session.setConfig(config);
      session.connect();
      if (session.isConnected) {
         channel = session.openChannel("exec");
         ((ChannelExec) channel).setCommand(cmd);
         channel.connect();
         while (channel.getExitStatus() == commandNotFinished) {
            //log.info channel.getExitStatus();
            WaitTimeSeconds(2);
         };
         rslt = channel.getExitStatus().toInteger();    //log.info channel.getExitStatus();
         //log.info 'about to disconnect channel';
         channel.disconnect();
         //log.info 'about to disconnect session';
         session.disconnect();
      };
   }
   catch(Exception ex) {
      log.error "Command Execute Error: " + ex;
      if (ex.toString().contains('Auth fail')) {
         // skip the remaining test steps.  If we have a bad password, we don't want to lock out the account for some reason...
         writeProperty("PropertiesRemote", "remotePassword", '');         
         testRunner.gotoStepByName( "Groovy Script Clean Password");
      }
   };
   return rslt;
};


//=========================================================================================================================================

log.info "";
log.info 'Test Step "Groovy Script Common Command Execute" Begin';

def remoteHost = readProperty("PropertiesRemote", "remoteLogHostPrefix") + readProperty("PropertiesRemote", "remoteLogHostIndex");  //
log.info 'remoteHost = ' + remoteHost;
def remoteUser = readProperty("PropertiesRemote", "remoteUser"); //log.info 'remoteUser = ' + remoteUser;
def remotePassword = readProperty("PropertiesRemote", "remotePassword"); //log.info 'remotePassword = ' + remotePassword;
def remoteCommand = readProperty("PropertiesRemote", "remoteCommand");

if (determinLocalOS()) {    // boolean...  Windows = true, other = false
   returnCode = executeCommand(remoteHost, remoteUser, remotePassword, remoteCommand);
   log.info 'return code=' + returnCode;
   assert (returnCode == 0), "Return code from command should be zero";
}
else {
   log.info 'return code=' + fileExecute('/usr/pservices/hudson_home/file_execute_on_remote.sh', 'webopt', remoteHost, remoteCommand);
};

log.info 'Test Step "Groovy Script Common Command Execute" End';
log.info "";

     

     

4 Replies

Sort By
  • TNeuschwanger's avatar
    TNeuschwanger
    Champion Level 2
    4 years ago

    Hello ramaG,

     

    I don't have a WinSCP specific script, but I have used jsch library to access server and retrieve a log file (or execute a command on a server).  Below is sample code to be hacked up and use what you can (file transfer or execute remote command).

     

    I think it is hard to get over to a server and then start issuing commands unless you code a script on the remote server and execute it.  I found it easier just to download entire log file and process it locally than to robot through on the server where the logs reside.  Our log files are sometimes very large and are rolled over when they hit a certain size.  This technique has worked for me even though there were millions of records in the log file.  Depending on how large your log files are and how long you are willing to wait for a file transfer will determine if this technique works for you. 

     

    The jsch library will need to be obtained from "http://www.jcraft.com/jsch/" since it is not included with SoapUI (maybe these days it is but current free version does not include it).  I love this helper library and has saved me from a lot of trouble in the past. 

     

    Groovy Script Common Get File Transfer:

    import java.text.SimpleDateFormat;
import com.jcraft.jsch.*;

def readProperty(sName, pS) {
   def step = testRunner.testCase.getTestStepByName(sName);
   def propertyStr = step.getPropertyValue( pS );
   return propertyStr;
};

def writeProperty(sName, pS, pV) {
   def step = testRunner.testCase.getTestStepByName(sName);
   def propertyStr = step.setPropertyValue( pS, pV );
   return true;
};

def determinLocalOS() {   // Determine Operating System function.  Purpose is to define if this script is running locally on Windows or Linux
   def homeProp = System.getProperty("user.home");  //log.info "homeProp: " + homeProp;
   def localRtn = homeProp.contains("\\");   //log.info "localOS: " + localOS;
   return localRtn;  // boolean...  Windows = true, other = false
};

def fileTransfer(cmd, uId, svr, fileF, fileT) {   // File Transfer function.  Purpose is to execute a file transfer using the input parameters.
   def sout = new StringBuffer()
   def serr = new StringBuffer()
   log.info "CommandHarness: " + cmd;
   log.info "UserId: " + uId;
   log.info "Server: " + svr;
   log.info "FileFrom: " + fileF;
   log.info "FileTo: " + fileT;
   def cmdExecute = [cmd, uId, svr, fileF, fileT];  //
   log.info "cmdExecute: " + cmdExecute;   
   Process x = cmdExecute.execute();   // 
   x.consumeProcessOutput(sout, serr);    //  #### IMPORTANT #### This line must be here or the file transfer using PSFTP on Windows will not work for servers uslcappp03 and uslcappp04.
   x.waitFor();
   log.info "x=${x.text}";
   log.info 'sout: ' + sout;
   log.info 'serr: ' + serr;
   return "code: ${ x.exitValue()}";
};

def Session session;
def Channel channel;
def ChannelSftp channelSftp;
def JSch jsch;

//=========================================================================================================================================

def connect(String hostName, String userName, String passCode) {
   try {
      jsch = new JSch();
      session = jsch.getSession( userName, hostName, 22 );
      session.setPassword(passCode);
      session.setTimeout(3000);
      java.util.Properties config = new java.util.Properties();
      config.put("StrictHostKeyChecking", "no");
      session.setConfig(config);
      session.connect();
      if (session.isConnected) {
         channel = session.openChannel("sftp");
         channel.connect();
         channelSftp = ( ChannelSftp ) channel;
         log.info 'Connected to: ' + hostName;
      }
   }
   catch (JSchException e) {
      log.error e.toString();
   };
};

def changeDirectory( String remotePath ) {
   try {
      log.info 'Path before ChangeDirectory is: ' + getPath();
      channelSftp.cd( remotePath );
      log.info 'Path after ChangeDirectory is: ' + getPath();
   }
   catch( Exception e ) {
      disconnect();
      log.error e.toString();
   };
};

def disconnect() {
   channelSftp.quit();
   channelSftp.disconnect();
   channel.disconnect();
   session.disconnect();
   log.info "Disconnected";
};

def boolean isConnected() {
   return session == null ? false : session.isConnected();
};

def String getPath() {
   return channelSftp.pwd()+"/";
};

def String getList(String remotePath) {
   osList = channelSftp.lstat(remotePath)+"/";
   return osList;
};

def getStat(String remotePath) {
   return channelSftp.lstat(remotePath);
};

def getFile(String rFQFN, String lFQFN) {
   log.info 'Fully qualified file name on remote server: ' + rFQFN;
   log.info 'Fully qualified file name on local computer: ' + lFQFN;
   channelSftp.get(rFQFN, lFQFN);
};


//=========================================================================================================================================

log.info "";
log.info 'Test Step "Groovy Script Common Get File Transfer" Begin';

def remoteHost = readProperty("PropertiesRemote", "remoteLogHostPrefix") + readProperty("PropertiesRemote", "remoteLogHostIndex");  //
def remoteUser = readProperty("PropertiesRemote", "remoteUser");
def remotePassword = readProperty("PropertiesRemote", "remotePassword");
def remotePath = readProperty("PropertiesRemote", "remotePath");
def workFileName = readProperty("PropertiesRemote", "workFileName");
def tmpDirStr = readProperty("PropertiesRemote", "tmpDirStr");

def remotePathFileName = workFileName;  //
log.info "remotePathFileName: " + remotePathFileName;

def localPathFileName = tmpDirStr + workFileName;  //
log.info "localPathFileName: " + localPathFileName;

dateTimeFileStamp = new SimpleDateFormat ( "yyyy-MM-dd hh-mm-ss a" );
dtStr = dateTimeFileStamp.format(new Date());

if (determinLocalOS()) {    // boolean...  Windows = true, other = false
//log.info "remoteHost: " + remoteHost;
//log.info "remoteUser: " + remoteUser;
//log.info "remotePassword: " + remotePassword;
   connect (remoteHost, remoteUser, remotePassword);
   if (isConnected()) {
      getFile(remotePathFileName, localPathFileName);
      getFile(remotePathFileName, localPathFileName + dtStr + '.txt');   // helpful for local debugging if ran multiple times.
      disconnect();
   }
   else {
      log.info 'Connection failed to: ' + remoteHost;
   };
}
else {
   log.info 'return code=' + fileTransfer('/usr/pservices/hudson_home/filecopy_remote2local.sh', 'webopt', remoteHost, remotePathFileName, localPathFileName);
};

log.info 'Test Step "Groovy Script Common Get File Transfer" End';
log.info "";

     Groovy Script Common Command Execute:

    import com.jcraft.jsch.*;

def readProperty(sName, pS) {
   def step = testRunner.testCase.getTestStepByName(sName);
   def propertyStr = step.getPropertyValue( pS );
   return propertyStr;
};

def writeProperty(sName, pS, pV) {
   def step = testRunner.testCase.getTestStepByName(sName);
   def propertyStr = step.setPropertyValue( pS, pV );
   return true;
};

def determinLocalOS() {   // Determine Operating System function.  Purpose is to define if this script is running locally on Windows or Linux
   def homeProp = System.getProperty("user.home");  //log.info "homeProp: " + homeProp;
   def localRtn = homeProp.contains("\\");   //log.info "localOS: " + localOS;
   return localRtn;  // boolean...  Windows = true, other = false
};

def fileExecute(cmd, uId, svr, exeCmd) {   // Command Execution function.  Purpose is to execute command on a remote server.
   def sout = new StringBuffer();
   def serr = new StringBuffer();
   log.info "CommandHarness: " + cmd;
   log.info "UserId: " + uId;
   log.info "Server: " + svr;
   log.info "Executable: " + exeCmd;
   def cmdExecute = [cmd, uId, svr, exeCmd];  //
   log.info "cmdExecute: " + cmdExecute;
   Process x = cmdExecute.execute();   //
   x.consumeProcessOutput(sout, serr);
   x.waitFor();
   log.info "x=${x.text}";
   log.info 'sout: ' + sout;
   log.info 'serr: ' + serr;
   return "code: ${ x.exitValue()}";
};

def WaitTimeSeconds(wT) {
   log.info 'wait time (seconds) = ' + wT;
   wT = wT * 1000;  // Convert seconds to milliseconds for use in commands
   done = false;
   new Thread({ sleep(wT) ; done = true }).start();
   for (count = 0; !done; ++count) {  //      log.info "Here! ${count}"
      sleep(1000);
   };  //log.info 'wT = ' + wT;
   return true;
};

def Session session;
def Channel channel;
def Integer executeCommand(String hostName, String userName, String passCode, String cmd) {
   log.info 'command to execute =>' + cmd + '<';
   def commandNotFinished = -1;   // -1 is returned by jsch while the command is running
   def rslt = commandNotFinished;
   try {
      def jsch = new JSch();
      session = jsch.getSession(userName, hostName, 22);
      session.setPassword(passCode);
      session.setTimeout(5000);
      java.util.Properties config = new java.util.Properties();
      config.put("StrictHostKeyChecking", "no");
      session.setConfig(config);
      session.connect();
      if (session.isConnected) {
         channel = session.openChannel("exec");
         ((ChannelExec) channel).setCommand(cmd);
         channel.connect();
         while (channel.getExitStatus() == commandNotFinished) {
            //log.info channel.getExitStatus();
            WaitTimeSeconds(2);
         };
         rslt = channel.getExitStatus().toInteger();    //log.info channel.getExitStatus();
         //log.info 'about to disconnect channel';
         channel.disconnect();
         //log.info 'about to disconnect session';
         session.disconnect();
      };
   }
   catch(Exception ex) {
      log.error "Command Execute Error: " + ex;
      if (ex.toString().contains('Auth fail')) {
         // skip the remaining test steps.  If we have a bad password, we don't want to lock out the account for some reason...
         writeProperty("PropertiesRemote", "remotePassword", '');         
         testRunner.gotoStepByName( "Groovy Script Clean Password");
      }
   };
   return rslt;
};


//=========================================================================================================================================

log.info "";
log.info 'Test Step "Groovy Script Common Command Execute" Begin';

def remoteHost = readProperty("PropertiesRemote", "remoteLogHostPrefix") + readProperty("PropertiesRemote", "remoteLogHostIndex");  //
log.info 'remoteHost = ' + remoteHost;
def remoteUser = readProperty("PropertiesRemote", "remoteUser"); //log.info 'remoteUser = ' + remoteUser;
def remotePassword = readProperty("PropertiesRemote", "remotePassword"); //log.info 'remotePassword = ' + remotePassword;
def remoteCommand = readProperty("PropertiesRemote", "remoteCommand");

if (determinLocalOS()) {    // boolean...  Windows = true, other = false
   returnCode = executeCommand(remoteHost, remoteUser, remotePassword, remoteCommand);
   log.info 'return code=' + returnCode;
   assert (returnCode == 0), "Return code from command should be zero";
}
else {
   log.info 'return code=' + fileExecute('/usr/pservices/hudson_home/file_execute_on_remote.sh', 'webopt', remoteHost, remoteCommand);
};

log.info 'Test Step "Groovy Script Common Command Execute" End';
log.info "";

     

     

    • ramaG's avatar
      ramaG
      Contributor
      4 years ago

      Hi TNeuschwanger 

       

      Thank you so much for the groovy script. I have few questions.

      What is 

      def readProperty(sName, pS) and 

      sName and pS

      wanted to know if these variables are ProjName or testcase name in a project.
       
      Thanks again for your help!

       

      • TNeuschwanger's avatar
        TNeuschwanger
        Champion Level 2
        4 years ago

        Hello ramaG,

         

        The "readProperty" and "writeProperty" are just helper functions to read or write from/to property name value pairs contained in a testcase "Properties" test step. There are many ways for reading and writing properties in all levels of a project.  This instance deals with the properties of a test step in a test case.

         

        In this case, the "sName" is just the name of a "Properties" test step that is defined somewhere in the test case.  The code a few lines down in the script shows that the test step name in this instance is "PropertiesRemote".  It will end up being whatever test step name you are using to contain properties of your test case.  ReadyAPI defaults to "Properties" when adding a properties test step.  I chose to rename it "PropertiesRemote" to differentiate it from other property test steps that I might include in the test case.

         

        "pS" is the name of the property to operate on within the property test step.  "readProperty" will use that variable name as the name of the property to get a value for in the "PropertiesRemote" test step.

         

        You can use whatever mechanism you want to keep track of variables or skip the variables all together and hard code the values you use.  I like to have all of my variables defined in a properties test step that will be used throughout the testcase.