IBM Business Process Manager


Performance issue with XML Parsing


Dear All
Version : 7.5.1
Problem : I'm fetching around 4000 records from DB which when parsing takes more than 7 minutes to complete.
Details :
1. Fetching only one column
2. Return Type XMLElement
Below is the script used
var i=0;
var xml = tw.local.result.firstChild;
if(xml!=null){
tw.local.unitChargeCodes = new tw.object.listOf.String();
}
while(xml != null){
tw.local.unitChargeCodes[i] = xml.xpath('record/column[#name="DESCRIPTION"]')[0].getText();
xml = xml.nextSibling;
i++;
}
Let us know If I'm doing any mistake here or do suggest an alternative.
~ Bala
In general we recommend against reading that amount of data in one go. You will tend to get much better performance by grabbing ~500 or so records at a time and processing them. Assuming you ignore my advice I would look at doing the above in an XSLT rather than pure java script. I've had much better performance doing that (and the code is much cleaner) than what you have done above. You basically generate the XML of the desired variable and do a tw.local.untiChargeCodes = tw.system.serializer.fromXML(tw.local.yourCreatedXML). There is a service in the system data toolkit that will do XSL Transforms for you.
Hmmm... Actually thinking about it I you have a number of problems in your logic. As you can see from your own code the xpath returns a list of all matching nodes. Since you haven't given us a any sample data to work with, and it has been a while since I have done this, I'm thinking off the top of my head, but can't you do something like -
var matchingNodes = tw.local.result.xpath("resultSet/record/column[#name='DESCRIPTION']");
for (var nodeCount=0;nodeCount<matchingNodes.length;nodeCount++){
tw.local.unitChargeCodes[nodeCount] = matchingNodes[nodeCount].getText();
}
This would eliminate ~4000 xpath evaluations according to your explanation. Another option is to allow the product to do the heavy lifting for you. Make a "Business Object" called "StupidPlaceHolder" (or whatever makes sense for you) It has one member a string with the name description. You then call the DB connector with your query but tell it to return you "StupidPlaceHolder" objects. You will get back a list of that type with the description member filled in with the values matching your values in the description column. Then if that performs poorly you can go beat up IBM and ask them to make their code better.
Andrew Paier | Director | BP3 Global, Inc.
BP3 Global's Website | Twitter | Linkedin | Google+ | Blogs
In general we recommend against reading that amount of data in one go. You will tend to get much better performance by grabbing ~500 or so records at a time and processing them. Assuming you ignore my advice I would look at doing the above in an XSLT rather than pure java script. I've had much better performance doing that (and the code is much cleaner) than what you have done above. You basically generate the XML of the desired variable and do a tw.local.untiChargeCodes = tw.system.serializer.fromXML(tw.local.yourCreatedXML). There is a service in the system data toolkit that will do XSL Transforms for you.
Hmmm... Actually thinking about it I you have a number of problems in your logic. As you can see from your own code the xpath returns a list of all matching nodes. Since you haven't given us a any sample data to work with, and it has been a while since I have done this, I'm thinking off the top of my head, but can't you do something like -
var matchingNodes = tw.local.result.xpath("resultSet/record/column[#name='DESCRIPTION']");
for (var nodeCount=0;nodeCount<matchingNodes.length;nodeCount++){
tw.local.unitChargeCodes[nodeCount] = matchingNodes[nodeCount].getText();
}
This would eliminate ~4000 xpath evaluations according to your explanation. Another option is to allow the product to do the heavy lifting for you. Make a "Business Object" called "StupidPlaceHolder" (or whatever makes sense for you) It has one member a string with the name description. You then call the DB connector with your query but tell it to return you "StupidPlaceHolder" objects. You will get back a list of that type with the description member filled in with the values matching your values in the description column. Then if that performs poorly you can go beat up IBM and ask them to make their code better.
Andrew Paier | Director | BP3 Global, Inc.
BP3 Global's Website | Twitter | Linkedin | Google+ | Blogs
1. The good solution to make performance better is to use partitioning of your data, as said higher.
2. If you use BPM Advanced, you can use mediation modules for XSL Transformations: ESB XSL engine has much better performance then BPM itself.
AndrewPaier
2700040K2Q
‏2013-06-12T15:27:07Z
In general we recommend against reading that amount of data in one go. You will tend to get much better performance by grabbing ~500 or so records at a time and processing them. Assuming you ignore my advice I would look at doing the above in an XSLT rather than pure java script. I've had much better performance doing that (and the code is much cleaner) than what you have done above. You basically generate the XML of the desired variable and do a tw.local.untiChargeCodes = tw.system.serializer.fromXML(tw.local.yourCreatedXML). There is a service in the system data toolkit that will do XSL Transforms for you.
Hmmm... Actually thinking about it I you have a number of problems in your logic. As you can see from your own code the xpath returns a list of all matching nodes. Since you haven't given us a any sample data to work with, and it has been a while since I have done this, I'm thinking off the top of my head, but can't you do something like -
<pre dir="ltr" style="margin-left: 40px;">var matchingNodes = tw.local.result.xpath("resultSet/record/column[#name='DESCRIPTION']");
for (var nodeCount=0;nodeCount<matchingNodes.length;nodeCount++){
tw.local.unitChargeCodes[nodeCount] = matchingNodes[nodeCount].getText();
}
</pre>
This would eliminate ~4000 xpath evaluations according to your explanation. Another option is to allow the product to do the heavy lifting for you. Make a "Business Object" called "StupidPlaceHolder" (or whatever makes sense for you) It has one member a string with the name description. You then call the DB connector with your query but tell it to return you "StupidPlaceHolder" objects. You will get back a list of that type with the description member filled in with the values matching your values in the description column. Then if that performs poorly you can go beat up IBM and ask them to make their code better.
Andrew Paier | Director | BP3 Global, Inc.
BP3 Global's Website | Twitter | Linkedin | Google+ | Blogs
More...
Thank you Andrew.
This sample code in your post helped me to resolve the issue :)
~ Bala

Related Links

REST calls failing when load balanced
Tabs control - Current Tab
Online Process Server not in Process Center
V8.0.1.1 - coach view refresh issue
Retrieving data variables from a Started Instance/Task BPM 8.0.1.1
[URGENT] "Available in Search" variables: how to delete one aliased variable from the…
List of ENV\'s through JS code
Is it Possible to Install IBM BPM 8.5 and ODM 8.5 in the same WAS?
Authorization in REST API calls
Sending Business Object as one of the input to the AJAX service
How to get swimlane documentation using rest api
Creating Coach Views Dynamically
rest call from a service
BPM 8.5 Portal Customization
How to Change Comment Color
Read user\'s session data at server side

Categories

DW
PureApplication System
Unica
Germany
dW Russia
Rational
Communities category
IBM Software Defined N...
IBM FileNet Content Ma...
IBM Kenexa Prove It! O...
IBM Security Identity ...
IBM Security Intellige...
IBM SmartCloud Applica...
IBM TRIRIGA UX Framework
Environmental and Ener...
Labs (BigFix Labs)
Cast Iron Studio
Cast Iron Web Manageme...
SPSS Community News
Python Programmability...
Graphics
IBM ILOG Elixir
Cognos Series 7
Eventos en Latinoamérica
Open Source
Сообщить о некорректно...
IBM Rational Asset Ana...
Rational Asset Manager
webservices
powerarchitecture
grid
Test for Aparna
Electronics
Partnerworld blogs
All blogs
Tivoli
Community
Web development
Architecture
Eserver
Autonomic computing
InfoSphere Information...
Informix developer and...
IBM DB2 Analytics Acce...
Socially Integrated
IBM Business Process M...
IBM Visual Configurati...
WebSphere DataPower XC...
QRadar Extensions
Troubleshooting
Transportation Managem...
Mobile Store Channel
Mapping / Translation
Connect:Express
Learning @ Sterling Co...
Connect:Direct (Educat...
E-Commerce/EDI/XML Cou...
Reports and Prints
Industry
Business Process Modeling
JSP Webpages in GIS
Connect:Direct for HP ...
Product Information Ma...
Analytics & Reporting
Miscellaneous
**Announcements & Ster...
C3 Invoicing
Feature Request
C3 Analyzer Q&A
Adoption Rates
Comergent Courses
Upgrade Issues
Release 6.3
Release 6.0
Internationalization a...
Configuring Warehouse ...
Security
IBM ILOG Elixir General
Constraint Programming
ILOG Customer Experts
Diagram
Java Visualization Pro...
CP Optimizer
CPLEX Optimizers
Questions and Best Pra...
Форум System x
IBM Simulation Producer
Building Information M...
IBM i
IBM i: Backup/Recovery
IBM i IBM i Performanc...
General Parallel File ...
PowerHA (Formerly know...
Web Site Development w...
Server-side Java progr...
Client-side Java progr...
ibmpub.java.aix

Resources

Encrypt Message



code
soft
python
ios
c
html
jquery
cloud
mobile