Forum Discussion

sjakovac's avatar
sjakovac
Occasional Contributor
4 years ago

Extracting response from Kafka Connection step

How to extract a response from successfully passed Kafka Connection step? 

Step passes and I can see response through GUI. GUI has metaData and Data tabs and actual Kafka messages. Assertions using JSONPath are executed correctly. 

 

What I want now is to extract those response values. Tried few ways, none work.

  1. 1) Using property transfer TestStep: After selecting "Source:" with Kafka step, the "Property:"  drop-down selection is disabled. 

 

 

  1. 2) Using script assertion: messageExchange object which should be available is null: log.info(messageExchange) -> Thu Sep 16:50:54 CEST 2021: INFO: null

 

  1. 3) Using Groovy script test step in step after Kafka connection: script similar to one from other question

 

def prevStep = testRunner.testCase.getTestStepAt(context.currentStepIndex  - 1)
String responsLocator ='${' + prevStep.name + '#Response}'
def response = context.expand(responsLocator)


log.info(response)

 

logs empty string. No other field of prevStep is useful or available.

 

Thanks on help 😄

 

  • TNeuschwanger's avatar
    TNeuschwanger
    Champion Level 1

    Hello sjakovac 

     

    I can only help with groovy script test step... 

    It looks like you made it a little more difficult than it should be for just accessing the response of a test step.  You can make it a little less brittle by actually naming the test step instead of attempting to get the previous test step object.  If that previous test step actually has a response element it could just be two lines of code:

    def response = context.expand( '${HTTP Request OAuth Token#Response}' );
    log.info "response=$response";

    Replace the "HTTP Request OAuth Token" above with whatever the test step name is that you want the response for.

     

    Regards,

    Todd

    • sjakovac's avatar
      sjakovac
      Occasional Contributor
      Yes, I tried that, doesn't work. I've also printed out test name in my
      solution, and it finds correct one. The problem seems to be that Kafka step
      doesn't have response object accessable by #Response expansion.

      I'm looking for any other way of retrieving it
  • Hi,

     

    I have exactly the same issue as you sjakovac. I have also created a ticket at smartbear for this issue. Unfortunately they require my project in order to investigate the issue but since I have lot of firewalls and certificates involved they will not be able to run exactly my project.

     

    I suggest smartbear should do an own test on this and if they can get it to work they can create a webpage where they describe this. I mean, there are lot of people having the same issue.

     

    However, in order to get around this issue I have been investigating some groovy scripting and I found this link which I have modified a bit and now I have a solution which works for me.

    https://community.smartbear.com/t5/SoapUI-Open-Source/Need-groovy-script-to-read-messages-from-kafka-topic/td-p/196019 

     

    Here is the complete code. Hope it will help other people as well.

    Btw, thanks to the guy sharing the original code with us.

    import org.apache.kafka.clients.consumer.ConsumerRecord;
    import org.apache.kafka.clients.consumer.ConsumerRecords;
    import org.apache.kafka.clients.consumer.KafkaConsumer;
    
    import java.util.Arrays;
    import java.util.Properties;
    
    log.info("*************************************************************************")
    log.info("                                      TOPIC NAME    ")
    log.info("*************************************************************************")
    
    
    //SAVE RECEIVED DATA
    def tsuite1 = testRunner.testCase.testSuite.project.getTestSuiteByName("TEST SUITE NAME");
    def tcase1= tsuite1.getTestCaseByName("TEST CASE NAME");
    db = tcase1.testSteps['Data Source'].dataSource.gridModel
    int row =  testRunner.testCase.testSteps['Data Source'].currentRow
    def guid = "ABC12345DEF"
    
    
    //KAFKA SETTINGS
    String topic = "MY-TOPIC-NAME"
    Properties properties = new Properties();
    properties.put("group.id", "test");
    properties.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
    properties.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
    properties.put("security.protocol","SSL");
    properties.put("ssl.keystore.password", "XXX");
    properties.put("bootstrap.servers", "XXX:6667");
    properties.put("ssl.key.password", "XXX");
    properties.put("ssl.truststore.password", "XXX");
    properties.put("ssl.truststore.location", "C:\\MyCertificates\\certificate.jks");
    properties.put("ssl.keystore.location", "C:\\MyCertificates\\certificate.jks");
    //properties.put("session.timeout.ms", "10000");
    properties.put("max.poll.records", "5");
    
    KafkaConsumer<String, String> consumer = new KafkaConsumer<>(properties);
    consumer.subscribe(Arrays.asList(topic));
    long t = System.currentTimeMillis();
    long end = t + 9000;
    while (System.currentTimeMillis()<end){
    	log.info("START READING")
    	ConsumerRecords<String, String> records = consumer.poll(200);
    	for (ConsumerRecord<String, String> record : records){
    		log.info("key = " + record.key())
    		log.info("value = " + record.value());
    
    		if(record.value().contains(guid)){
    			log.info("found data")
    			db.setValueAt(topic, row, 1);
    			db.setValueAt(guid, row, 2);
    		}
    	}
    	log.info("STOP READING")
    }
      consumer.close();

     

    • nmrao's avatar
      nmrao
      Champion Level 3
      Thanks for sharing the script amirse.
      So, in order to test kafka use case, are you just using this script alone?
      I mean are you able to test the kafka messaging without using kafka step provided by ReadyAPI?
      • amirse's avatar
        amirse
        Contributor

        Hi,

         

        I have a REST call which actually starts a flow in our system which results in publishing to kafka.

        So in my case I am only interested in consumer not publisher.

        And this code together with the REST test case is enough for me.

    • sjakovac's avatar
      sjakovac
      Occasional Contributor

      Thanks, I'll try this. At first glance, it seems like this is a replacement for the whole KafkaConnection test step.

       

      My current setup is KafkaConnection API test step followed by Groovy Script test step.

      The solution from your answer is implementing KafkaConnection test step as a Groovy script. That's fine, but then I can't easily run (smart)assertions on that Groovy test step. I'd have to write them manually or transfer data into a separate test step and create a new Assertion test step.

       

      All in all, it can be modified to work but it's a lot more complicated than it should have been (looking at you SmartBear devs 😡).

       

      Anyway, thank you amirse 

       

       

      • amirse's avatar
        amirse
        Contributor

        Hi, 

        You are 100% correct. This adds unnecessary work but before smartbear gives us a solution on the issue this is what we probably need to do.

        At least I cannot come up with some other solution.

        I have other projects where I have done assertions in groovy so I will do as in those projects.