How to get a Database table into an object (2D Array may be), and read the data
Hello,
I'm running a UI tests which fetches data continuously from a DB table. This table has 10000 rows and 20 columns.
Test continuously fetches the data, and uses if for modifications, and many column values get updated. It goes thru three iterations, and in each iteration column values get changed.
The problem I'm facing: The way I'm fetching these values in an object takes significant time(~ 2 hours) to get all these values into an object (10000 rows x 20 Columns = 200,00 cells) because each value is being fetched individually.
What I really need help with is, a way to fetch the whole data table into an object (may be an array), and then be able to read each cell(row, column) value buy using the combination of columnname and row Indicator e.g. Column_Name.row[0] and also much quicker(Within 5 minutes range).
The records fetched from this snippet are being pushed into another array to have a table like object.
function dbRecordToObject(record) {
var returnValue = {};
try {
if(!record.EOF) {
for(var i = 0; i < rec.Fields.Count; i++ ) {
returnValue[record.Fields.Item(i).Name] = record.Fields.Item(i).Value == null ? "" : aqString.Trim(record.Fields.Item(i).Value);
}
}
else {
Log.Warning("Empty record set");
}
}
catch(err) {
Log.Warning("Error : " +err.message);
}
finally {
return returnValue;
}
}
Any help would be appreciated!
Thank you
Al2
Ok, i'd done two posts because you asked about two things if i understood correctly.
First post is about transforming linear data in 2D array.
Use google translate to translate the documentation because i made effort to document then make the effort to translate 😁
And getTokenBySeparator take a token from a string, using a separator to find tokens ... 😂
Second post is about, root cause of your problem, speed.
If you look the second post, the speed problem could be resolved simply, call the set.GetRows() instead of navigating into ADO result set, so it would be:
let set = conn.Execute_(exeQuery); let elementList = []; if ((set != null) && (set.RecordCount != 0)) elementList = set.GetRows().toArray(); return elementList;What you do was to browse Ado result set one by one to rebuild an array instead of fetching all data directly, so it's very slow.
Some info on GetRows() is here:
https://www.w3schools.com/asp/met_rs_getrows.asp
BTW try to implement error management and connection established waiting as it is in my code. Your code will be robust and in case of errors you can know where and what.
And don't put comment on trivial things (e.g. // Declare an array) or // Closing connection), comments are for advanced technical or algorithm points.
I invite you to take a glance to this post about method writing:
