Java Non-Development Stuff

https://www.ibm.com/developerworks/java/library/j-nativememory-aix/

Oracle SOA - BPEL Tuning

This post will consolidate various blogs/docs/configurations required to performance tune Oracle SOA




Below information is Copied from : http://oraclesoaandoim.blogspot.com/2012/08/oracle-soa-11g-11116-performance-tuning.html

ORACLE SOA 11G 11. 1.1.6 PERFORMANCE TUNING OF BPEL PROCESSES

BPEL PROPERTIES SET INSIDE A COMPOSITE

This section lists the config properties of some sections of the deployment descriptor. For each configuration property parameter, a description is given, as well as the expected behavior of the engine when it is changed.

All the properties set in this section affect the behavior of the component containing the BPEL process only. Each BPEL process can be created as a component of a composite. These properties can be modified in composite.xml or in the System MBean Browser of Oracle Enterprise Manager Fusion Middleware Control. 

Some Concepts First

As a general practice, it is better to design your BPEL processes as transient instead of durable if performance is a concern. Note that this may not always be possible due to the nature of your process, but keep the following points in mind.

The dehydration store is uses to maintain long-running asynchronous BPEL instances storing state information as they wait for asynchronous callbacks. This ensures the reliability of these processes in the event of server or network loss. Oracle BPEL Process Manager supports two types of processes; transient and durable.

Transient Processes

Transient processes do not incur dehydration during their process execution. If an executing process experiences an unhandled fault or the server crashes, instances of a transient process do not leave a trace in the system. Thus, these instances cannot be saved in-flight regardless if they complete normally or abnormally. Transient processes are typically short-lived, request-response style processes. Synchronous processes are examples of transient processes.

Durable Processes

Durable processes incur one or more dehydration points in the database during execution. Dehydration is triggered by one of the following activities:

• Receive activity
• OnMessage branch in a pick activity
• OnAlarm branch in a pick activity
• Wait activity
• Reply activity
• checkPoint() within a  activity

 inMemoryOptimization

This property indicates to Oracle BPEL Server that this process is a transient process and dehydration of the instance is not required. When set to True, the completionPersistPolicy is used to determine persistence behavior. This property can only be set to True for transient processes or processes that do not contain any dehydration points such as receive, wait, onMessage and onAlarm activities. The inMemoryOptimization property is set at the BPEL component level. When set to False, dehydration is disabled which can improve performance in some use cases.
Values:
This property has the following values:

• False (default): instances are persisted completely and recorded in the dehydration store database.
• True: The completionPersist policy is used to determine persistence behavior. 

completionPersistPolicy

This property configures how the instance data is saved. It can only be set at the BPEL component level. The completionPersistPolicy property can only be used when inMemoryOptimization is set to be True (transient processes). Note that this parameter may affect database growth and throughput (due to reduced I/O).

Value	Description
On (default)	The completed instance is saved normally
Deferred	The completed instance is saved, but with a different thread and in another transaction.
Faulted	Only the faulted instances are saved. Note: When an unhandled fault occurs, regardless of these flags, audit information of the instance is persisted within cube_instance table.
Off	No instances of this process are saved.

   

   faulted
   true
   ...

auditLevel

You can set the audit level for a BPEL process service component. This setting takes precedence over audit level settings at the SOA Infrastructure, service engine, and SOA composite application levels.

Set the bpel.config.auditLevel property to an appropriate value in the composite.xml file of your SOA project as shown in the example below:

Off

Value	Description
Inherit	Inherits the audit level from infrastructure level.
Off	No audit events (activity execution information) are persisted and no logging is performed; this can result in a slight performance boost for processing instances.
Minimal	All events are logged; however, no audit details (variable content) are logged.
Error	Logs only serious problems that require immediate attention from the administrator and are not caused by a bug in the product. Using this level can help performance.
Production	All events are logged. The audit details for assign activities are not logged; the details for all other activities are logged.
Development	All events are logged; all audit details for all activities are logged.

Partner Link Property

You can dynamically configure a partner link at runtime in BPEL. This is useful for scenarios in which the target service that BPEL wants to invoke is not known until runtime. The following Partner Link properties can be tuned for performance:

idempotent

An idempotent activity is an activity that can be retried (for example, an assign activity or an invoke activity). Oracle BPEL Server saves the instance after a nonidempotent activity. This property is applicable to both durable and transient processes.

Values:

This property has the following values:

• False: Activity is dehydrated immediately after execution and recorded in the dehydration store. When idempotent is set to False, it provides better failover protection, but may impact performance if the BPEL process accesses the dehydration store frequently.
• True (default): If Oracle BPEL Server fails, it performs the activity again after restarting. This is because the server does not dehydrate immediately after the invoke and no record exists that the activity executed. Some examples of where this property can be set to True are: read-only services (for example, CreditRatingService) or local EJB/WSIF invocations that share the instance's transaction.

A BPEL invoke activity is by default an idempotent activity, meaning that the BPEL process does not dehydrate instances immediately after invoke activities. Therefore, if idempotent is set to true and Oracle BPEL Server fails right after an invoke activity executes, Oracle BPEL Server performs the invoke again after restarting. This is because no record exists that the invoke activity has executed. This property is applicable to both durable and transient processes.

If idempotent is set to false, the invoke activity is dehydrated immediately after execution and recorded in the dehydration store. If Oracle BPEL Server then fails and is restarted, the invoke activity is not repeated, because Oracle BPEL Process Manager sees that the invoke already executed.

When idempotent is set to false, it provides better failover protection, but at the cost of some performance, since the BPEL process accesses the dehydration store much more frequently. This setting can be configured for each partner link in the bpel.xml file.

nonBlockingInvoke

By default, Oracle BPEL Process Manager executes in a single thread by executing the branches sequentially instead of in parallel. When this property is set to True, the process manager creates a new thread to perform each branch's invoke activity in parallel. This property is applicable to both durable and transient processes.

Consider setting this property to True if you have invoke activities in multiple flow or flow n branches. This is especially effective if the parallel invoke activities are two-way, but some benefits can be realized for parallel one-way invokes as well.

Note:

Invocations to the same partner link will happen in sequence and not in parallel. If you invoke different partner links each time with nonBlockingInvoke set to True, then each link will work in parallel even if all of the partner links point to the same source.

Values:

• True: Oracle BPEL Server spawns a new thread to execute the invocation.
• False (default): Oracle BPEL Server executes the invoke activity in the single process thread.

validateXML

Enables message boundary validation. Note that additional validation can impact performance by consuming extra CPU and memory resources.

Values:

• True: When set to True the engine validates the XML message against the XML schema during and for this partner link. If the XML message is invalid thenbpelx:invalidVariables run time BPEL Fault is thrown. This overrides the domain level validateXML property.
• False (default): Disables XML validation.

General Recommendations:

1. If your Synchronous process exceed, say 1000 instances per hour, then its better to set inMemoryOptimization to true and completionPersistPolicyto faulted, So that we can get better throughput, only faulted instances gets dehydrated in the database, its goes easy on the purge (purging historical instance data from database)

2. Do not include any settings to persist your process such as (Dehydrate, mid process receive, wait or Onmessage)

3. Have good logging on your BPEL Process, so that you can see log messages in the diagnostic log files for troubleshooting.

What should you do?

• If the design of the process allows it, design your BPEL processes as short-lived, synchronous transactions.
• If the design of the process allows it, avoid the activities listed above.

Any time your process is dehydrated to the dehydration store, this naturally impacts the performance of the process, and becomes a concern particularly in high volume environments.

Design Patterns : Template Method Pattern

Provide Template Code, but leave implementation to the  concrete class.  The template method pattern is a very good example when to make a method as final.


Template method design pattern is to define an algorithm as skeleton of operations and leave the details to be implemented by the child classes. The overall structure and sequence of the algorithm is preserved by the parent class.

http://www.oodesign.com/template-method-pattern.html
http://javapapers.com/design-patterns/template-method-design-pattern/

public abstract class OrderProcessTemplate {
 public boolean isGift;

 public abstract void doSelect();

 public abstract void doPayment();

 public final void giftWrap() {
  System.out.println("Gift wrap done.");
 }

 public abstract void doDelivery();

 public final void processOrder() {
  doSelect();
  doPayment();
  if (isGift) {
   giftWrap();
  }
  doDelivery();
 }
}

public class NetOrder extends OrderProcessTemplate {

 @Override
 public void doSelect() {
  System.out.println("Item added to online shopping cart,");
  System.out.println("Get gift wrap preference,");
  System.out.println("Get delivery address.");
 }

 @Override
 public void doPayment() {
  System.out.println("Online Payment through Netbanking/Cards.");
 }

 @Override
 public void doDelivery() {
  System.out.println("Ship the item through post to delivery address");
 }

}

public class StoreOrder extends OrderProcessTemplate {

 @Override
 public void doSelect() {
  System.out.println("Customer chooses the item from shelf.");
 }

 @Override
 public void doPayment() {
  System.out.println("Pays at counter through cash/POS");
 }

 @Override
 public void doDelivery() {
  System.out.println("Item deliverd to in delivery counter.");
 }

}

OSB Performance Tuning : Proxy Service Cache

https://blogs.oracle.com/ateamsoab2b/entry/osb_performance_tuning_routerruntimecache

java -Dcom.bea.wli.sb.pipeline.RouterRuntimeCache.size=1500

http://docs.oracle.com/cd/E29505_01/core.1111/e10108/osb.htm#BABHAHID

OSB Proxy Xquery Perf oriented development

From:

http://docs.oracle.com/cd/E25054_01/core.1111/e10108/osb.htm

Design Time Considerations for Proxy Applications

Consider the following design configurations for proxy applications based on your OSB usage and use case scenarios:

• Avoid creating many OSB context variables that are used just once within another XQuery
  Context variables created using an Assign action are converted to XmlBeans and then reverted to the native XQuery format for the next XQuery. Multiple "Assign" actions can be collapsed into a single Assign action using a FLWOR expression. Intermediate values can be created using "let" statements. Avoiding redundant context variable creation eliminates overheads associated with internal data format conversions. This benefit has to be balanced against visibility of the code and reuse of the variables.
• Transforming contents of a context variable such as $body.
  Use a Replace action to complete the transformation in a single step. If the entire content of $body is to be replaced, leave the XPath field blank and select "Replace node contents". This is faster than pointing to the child node of $body (e.g. $body/Order) and selecting "Replace entire node". Leaving the XPath field blank eliminates an extra XQuery evaluation.
• Use $body/*[1] to represent the contents of $body as an input to a Transformation (XQuery / XSLT) resource.
  OSB treats "$body/*[1]" as a special XPath that can be evaluated without invoking the XQuery engine. This is faster than specifying an absolute path pointing to the child of $body. A general XPath like "$body/Order" must be evaluated by the XQuery engine before the primary transformation resource is executed.
• Enable Streaming for pure Content-Based Routing scenarios.
  Read-only scenarios such as Content-Based Routing can derive better performance from enabling streaming. OSB leverages the partial parsing capabilities of the XQuery engine when streaming is used in conjunction with indexed XPaths. Thus, the payload is parsed and processed only to the field referred to in the XPath. Other than partial parsing, an additional benefit for read-only scenarios is that streaming eliminates the overhead associated with parsing and serialization of XmlBeans.
  The gains from streaming can be negated if the payload is accessed a large number of times for reading multiple fields. If all fields read are located in a single subsection of the XML document, a hybrid approach provides the best performance. See Section 21.6, "Design Considerations for XQuery Tuning" for additional details.
  The output of a transformation is stored in a compressed buffer format either in memory or on disk. Therefore, streaming should be avoided when running out of memory is not a concern.
• Set the appropriate QOS level and transaction settings.
  Do not set XA or Exactly-Once unless the reliability level required is once and only once and its possible to use the setting (it is not possible if the client is a HTTP client). If OSB initiates a transaction, it is possible to replace XA with LLR to achieve the same level of reliability.
  OSB can invoke a back end HTTP service asynchronously if the QOS is "Best- Effort". Asynchronous invocation allows OSB to scale better with long running back-end services. It also allows Publish over HTTP to be truly fire-and-forget.
• Disable or delete all log actions.
  Log actions add an I/O overhead. Logging also involves an XQuery evaluation which can be expensive. Writing to a single device (resource or directory) can also result in lock contentions.

21.6 Design Considerations for XQuery Tuning

OSB uses XQuery and XPath extensively for various actions like Assign, Replace, and Routing Table. The following XML structure ($body) is used to explain XQuery and XPath tuning concepts:

Mary


20000 
1500
…. a large number of items


70000
Shipped
My Shipping Firm 

• Avoid the use of double front slashes ("//") in XPaths.
  $body//CustName while returning the same value as $body/Order/CtrlArea/CustName will perform a lot worse than the latter expression. "//" implies all occurrences of a node irrespective of the location in an XML tree. Thus, the entire depth and breadth of the XML tree has to be searched for the pattern specified after a "//". Use "//" only if the exact location of a node is not known at design time.
• Index XPaths where applicable.
  An XPath can be indexed by simply adding "[1]" after each node of the path. XQuery is a declarative language and an XPath can return more than one node; it can return an array of nodes.$body/Order/CtrlArea/CustName implies returning all instances Order under $body and all instances of CtrlArea under Order. Therefore, the entire document has to be read in order to correctly process the above XPath. If you know that there is a single instance of Order under $body and a single instance of CtrlArea under Order, we could rewrite the above XPath as$body/Order[1]/CtrlArea[1]/CustName[1].
  The second XPath implies returning the first instances of the child nodes. Thus, only the top part of the document needs to be processed by the XQuery engine resulting in better performance. Indexing is key to processing only what is needed.
  
  Note:

  Indexing should not be used when the expected return value is an array of nodes. For example, $body/Order[1]/ItemList[1]/Item returns all "Item" nodes, but $body/Order[1]/ItemList[1]/Item[1] only returns the first item node. Another example is an XPath used to split a document in a "for" action.
• Extract frequently used parts of a large XML document as intermediate variables within a FLWOR expression
  An intermediate variable can be used to store the common context for multiple values. Sample XPaths with common context:
  

  $body/Order[1]/Summary[1]/Total, $body/Order[1]/Summary[1]/Status,$body/Order[1]/Summary[1]/Shipping
  

  The above XPaths can be changed to use an intermediate variable:
  

  let $summary := $body/Order[1]/Summary[1]
  $summary/Total, $ summary/Status, $summary/Shipping
  

  Using intermediate variables consumes more memory but reduces redundant XPath processing.
• Using a Hybrid Approach for read-only scenarios with Streaming
  The gains from streaming can be negated if the payload is accessed a large number of times for reading multiple fields. If all fields read are located in a single subsection of the XML document, a hybrid approach provides the best performance. The hybrid approach includes enabling streaming at the proxy level and Assigning the relevant subsection to a context variable, The individual fields can then be accessed from this context variable.
  The fields "Total" and "Status" can be retrieved using three Assign actions:
  

  Assign "$body/Order[1]/Summary[1]" to "foo"
  Assign "$foo/Total" to "total"
  Assign "$foo/Status" to "total"
  

Oracle SOA File Adapter - Spaces in pay load

Recently we got a new integration for reading a file.  We don't know the source of the file, so we started developing with default values, for the File Adapter in JDeveloper.
The given file is a comma-separated with Double quotes as shown below.

However when configuring the JDEV, when the default CHARACTER SET US-ASCII is given to this file, it started inserting spaces between each character inside an element.

Initially we ignored this and developed out component.  But during run time, we were getting the pay load with spaces and are not able to figure out what the real issue is.

Basically this is a file format issue and by Just changing the Character Set to UTF-16 in the File Adapter XSD, this got fixed.

Java - Interesting..

SimpleDateFormat is not ThreadSafe.  

http://www.codefutures.com/weblog/andygrove/2007/10/simpledateformat-and-thread-safety.html

Also,

SimpleDateFormat stores intermediate results in instance fields. So if one instance is used by two threads they can mess each other's results.

Looking at the source code reveals that there is a Calendar instance field, which is used by operations on DateFormat / SimpleDateFormat

For example parse(..) calls calendar.clear() initially and then calendar.add(..). If another thread invokes parse(..) before the completion of the first invocation, it will clear the calendar, but the other invocation will expect it to be populated with intermediate results of the calculation.

One way to reuse date formats without trading thread-safety is to put them in a ThreadLocal - some libraries do that. That's if you need to use the same format multiple times within one thread. But in case you are using a servlet container (that has a thread pool), remember to clean the thread-local after you finish.

To be honest, I don't understand why they need the instance field, but that's the way it is. You can also use joda-time DateTimeFormat which is threadsafe.

from: http://stackoverflow.com/questions/6840803/simpledateformat-thread-safety

=================================================================================

HASHMAP - MULTITHREAD RACE CONDITION EXPLANATION

http://mailinator.blogspot.com/2009/06/beautiful-race-condition.html

AIX commands

SFTP to a system:

sftp soauser@optaixdbexe01qa
sftp> put /utl/eaistaging/ebs/outbound/o2c/Oracle_EXE_I552_Orders/Orders53/MAULDIN_51.txt .
sftp> ls /home/soauser
sftp> exit

sftp username@hostname

put localpath remotePath

===================================
Find File System Disk space

df -k /
df -k /home/sdoddi

=====

Find Command:

find . -name '*.DAT'

this will find all the files ending with .DAT starting from  the current directory (the DOT (.) indicates the current directory)

If you don't want to do this recursively and search only the current folder,

find . -xdev -name '*.DAT'

For finding from root,

find / -name '*.DAT'

For finding from a specific directory,
find /utl/mdm/logs -name '*.DAT'

For size
find . -size +2  (1 block is 512 kb.. )

for files which got changed in the last 2 days,

find . -mtime -2

find . -name '*.DAT' -mmin -120

files which are not access for last 7 days and which end with .DAT

find . -name '*.DAT' -atime +7

Java useful links

http://howtodoinjava.com
http://java-success.blogspot.com

...will keep updating..

Passing two variables to Xquery Transformation

Using Xquery Transformation wizard in OEPE we can create a Xquery transformation with multiple input source variable types.  But sometimes when we want to add an extra parameter to the source to an existing Xquery transformation do the below:
suppose we have an existing xquery transformation as below: (Removed the mappings to be clear)

 ========================================================================

declare function xf:sample($orcl2FoodlinVendorOutputCollection1 as element(ns0:Orcl2FoodlinVendorOutputCollection))
    as element(ns1:VendorList) {
       
};
                             
declare variable $orcl2FoodlinVendorOutputCollection1 as element(ns0:Orcl2FoodlinVendorOutputCollection) external;


xf:sample($orcl2FoodlinVendorOutputCollection1)

 ========================================================================

And I would like to add another variable, "runId" which is a string as an input to the above Xquery Transformation:

declare function xf:sample($orcl2FoodlinVendorOutputCollection1 as element(ns0:Orcl2FoodlinVendorOutputCollection),$runId as xs:string)

    as element(ns1:VendorList) {

       

};

declare variable $orcl2FoodlinVendorOutputCollection1 as element(ns0:Orcl2FoodlinVendorOutputCollection) external;

declare variable $runId as xs:string external;

xf:sample($orcl2FoodlinVendorOutputCollection1, $runId)

========================================================================

The variable "runId" should be declared and assigned prior to this xquery transformation in the proxy service message flow (or in the split join flow)

Setting space as constant in Xquery Transformation

Add below to the prolog and set the space to the xml tag.

declare xmlspace preserve;

JAXB JAXBElement XJC @XmlRootElement

C:\orderXSDs>xjc -d  C:\src -p com.cswg.soa.ebs.otc.outbound.exe.orders.dcinfo F
ileTypeByDCInfo_table.xsd


FileTypeByDCInfo_table.xsd is located under orderXSDs directory.  The generated clases will be under C:\src directory

if you are getting the exception,
java.lang.ClassCastException: javax.xml.bind.JAXBElement cannot be cast
look @
http://weblogs.java.net/blog/kohsuke/archive/2006/03/why_does_jaxb_p.html

http://stackoverflow.com/questions/707084/class-cast-exception-when-trying-to-unmarshall-xml