Asynchronous Business Function Error

Purpose of Document

In HTML client, ASYNCHRONOUS BUSINESS FUNCTION error may appear on screen which is a representation of COMMIT FAIL Error when "End Doc" business function gets called Asynchronously.

This document illustrates how to troubleshoot this type of asynchronous business function commit fail error based on some actual issues you may encounter in a certain application.

This document is written based on web client configuration; The same troubleshooting technique can be applied on FAT clients.

The cause of commit fail error can vary and this document is intended to narrow down possible causes of encountering this type of issue on a case by case basis.

Table of Contents

Q1> What is an asynchronous business function?

Any business function which gets called asynchronously, we call it as an asynchronous business function. It is usually related with Transaction File Master Business Function in calling Edit Line and End Doc routine. Refer to <Document 1265902.1> "E1: BSFN: Master Business Function in EnterpriseOne"

Since this master business function is requested by JAS (Java Application Server) to logic server which has to response within designated time.

Edit Line in Row is Exited/Changed - Asynch event
End Doc in Post OK button clicked or OK Post Button Clicked - Asynch

Q2> Parameters affecting asynchronous business function in JAS

In [OWWEB] section of jas.ini,

Server Manager	JAS.INI	Default Value	Explanation
Asynchronous Business Functions	AsynchBSFNEnabled	true	If enabled, asynchronous business functions will be running in a separate thread; otherwise, asynchronous business function will be running on the same thread as of the main processing thread for the user requests.
Initial BSFN Thread Pool Size	AsyncThreadPoolSize	50	Initial size of the asynchronous business function thread pool for this instance. When the number of asynchronous business function threads is over the pool size, the pool grows its size to support more threads.
Increment of asynchronous business function thread pool	AsyncThreadPoolSizeIncrement	5	The number of increment threads when the asynchronous business function thread pool grows.
Log out Extension Period	LogoutProcessTimeout	720000	If there is asynchronous business function running when a user logs out from EnterpriseOne, this is the maximum time period (in milliseconds) that the enterpriseOne user session is kept alive to allow for the asynchronous business functions to be completed. If there is no asynchronous business function running at the time of the user logout, the EnterpriseOne session is immediately invalidated.

Note:

Do not change above values unless it is requested by Oracle
In e1root_*.log, LogoutProcessTimeout is represented as AsynProcessingTimeout
Since the max value of JDENETTimeout is 360 seconds max value of LogoutProcessTimeout can not be exceeded 720000 milliseconds
Example of e1root_*.log to represent values in jas.ini

Asynch BSFN thread poolSizeLimit = 50
Asyhcn BSFN thread poolSizeIncrement = 5
AsyncProcessingTimeout set to 720000 milliseconds.

Q3> Parameters affecting timeout

The following parameters can be found in jde.ini and jas.ini

	Section	Server Manager	ini	Default Value	Explanation
JAS	JDENET	Timeout Threashold	enterpriseserverTimeout	90000	The maximum time, in milliseconds, before a JDENET operation is considered to have timed out. (Time out value for requests sent to the Enterprise Server)
Logic Server	JDENET	Timeout Threashold	enterpriseserverTimeout	60000	(jdenet_n kernel) The maximum time, in milliseconds, before a Java JDENET operation is considered to be timeout.
Logic Server	NETWORK QUEUE SETTINGS	JDENET Timeout	JDENETTimeout	60	(queue kernel) The amount of time, in seconds, that a requesting process should wait after sending a JDENET request to wait for the response before considering the message to have timed out.

Note:

For easier understanding, Enterprise Server = Logic Server + Database Server (and isolate issue gets caused by database server)
enterpriseserverTimeout parameter in jas.ini is written in milliseconds
The value for parameter enterpriseserverTimeout has to be adjusted/synchronized between your logic server and JAS (Java Application Server)
enterpriseserverTimeout value in JAS has to be equal to or greater than JDENETTimeout value
enterpriseserverTimeout in JAS affect any business function which has to be running in logic server
enterpriseServerTimeout parameter in Logic Server controls connection when request is performed not through JAS (e.g., Windows client with J* environment, Business Service and so on)
JDENETTimeout affect the timeout for MANUAL commit
If possible, do not increase timeout settings above because it may cause blocking
Do not exceed 360 seconds for JDENETTimeout parameter

    maximum connection between logic server and database server can be (based on jdbodbc between logic server and database server),
    = (CONNECTION_TIMEOUT * CONNECTION_TIMEOUT_MULT) + PHYS_CONNECTION_WAIT_TIME = (60*5) + 60 = 360 seconds
    #define CONNECTION_TIMEOUT               60    /* Time, in seconds, used in calculating connection timeout. */
    #define CONNECTION_TIMEOUT_MULT       5    /* Multiplier, used in calculating connection timeout. */
    #define CONNECTION_EXPIRE_TIME        (time(NULL) + gnCnctTimeout)
    #define CONNECTION_SHARE_LIMIT       1000 /* Limit on connection sharing. */
    #define PHYS_CONNECTION_WAIT_TIME   60   /* Total time, in secs, to wait for a physical connection to be made. */

Q4> What is transaction processing?

Refer to <Document 1208208.1> "E1: DB: Transaction, Transaction Processing, Commit and Rollback"

In EnterpriseOne, transaction processing is in place to fulfill "All or Nothing for critical commitment".

Q5> What is Master Business Function?

In EnterpriseOne Transaction is tightly related with Master Business Fuction. Master Business Function (MBF) is a Business Function written to process a record into the JDE system, which is made up of many sub-routines. JD Edwards provides MBFs to allow calling programs to process certain predefined transactions. An MBF encapsulates the required logic, enforces data integrity, and insulates the calling programs from the database structures.

Refer to <Document 1265902.1> E1: BSFN: Master Business Function in EnterpriseOne

Q6> How to troubleshoot?

Basic information need verifying,

jde.ini (in Logic Server): To determine kernel settings and timeout parameters
jas.ini (in JAS): To determine settings on timeout
jde_xxxx.log: contains kernel/system level error (including ODBC error)
e1root_xxxxxxxx.log: Same message will be written as you see on screen (refer to <Document 1094464.1> "E1: JAS: Exception/Error Code in JAS when Call Object Kernel Fails on Logic Server")

Additionally (depending on the issue),

Callobject kernel (jdedebug_xxxx.log): Contains how BSFNs are calling and called

Questions to Ponder:

Questions	Questions	Justification	Reference
	1. Is issue sporadic/intermittent?	If the issue is intermittent, necessary information above has to be verified. If possible, retrieve all JDE_*.logs in logic server and analyze it
	2. Any change made before error started to come?
	3. Issue started happening after ESU application?	Build full package after applying any baseline ESU	<Internal_Only><Document 1513969.1> E1: 42: Case Study - R42800 issues error "CMRB000002 - User buffer too small. Size allocated 1178, needed 1180"</Internal_Only>
	4. Issue occurred after tools release upgrade?
	5. Upgrade in OS (including C Compiler) and DB?	Memory related issue can be specific to a certain platform
	6. Is there any custom Function, Table Trigger and/or Stored Procedure?		<Document 1301427.1> "E1: DB: Custom Stored Procedure Causes Deadlock in the Database"
	7. Was package successful?	Special Instruction is performed properly
Persistent	1. Does e1root_*.log contain TIMEOUT?	Locking/blocking, looping and/or volume of data. Capture CallObject kernel log	<Document 1060095.1> "E1: ENV: How to Find any EnterpriseOne Log on Any Server, Platform, Client or Interactive/Batch Application" <Document 1228683.1> "E1: DB: Case Study on Deadlock in EnterpriseOne Applications"
	2. Does e1root_*.log contain TM_TRAN_CANCELLED?	This error will be specific End Doc routine with transaction processing on	<Document 1094464.1> "E1: JAS: Exception/Error Code in JAS when Call Object Kernel Fails on Logic Server"
	3. Does error specific to End Doc?
	4. Issue is specific to Edit Line?
	5. Higher usages of system resources? (100% CPU Usage)	Infinite Looping: Review CallObject Kernel log (jdedebug.log) and determine the BSFN causes infinite looping
	6. Does jde_xxxx.log issues *.dmp file?	Memory Error: *.dmp file in Windows and CallStack in AS400 will contain offending BSFN. Review routine in that BSFN	<Document 1464687.1> E1: BSFN: Case Study on Memory Violation, Memory Corruption, Memory Overwrite, ACCESS VIOLATION and Zombie Kernel
	7. Issue occurs across environment? (DV, PY and/or PD)
	8. The process resulted in zombie kernel?	Memory Error: Memory is created but not to be referenced
	9. Issue is specific to a certain user interface?	Up/down grids multiple times and OK/Cancel/Close button click	<Document 1265902.1> "E1: BSFN: Master Business Function in EnterpriseOne"
	10. Is issue specific to huge volume of data (e.g., a few of hundreds data through Excel Import)?	Consider batch process or increase timeout parameters in jde.ini and jas.ini	<Document 1265902.1> "E1: BSFN: Master Business Function in EnterpriseOne"
	11. Same issue occurs in Non-J* environment?
Intermittent /Sporadic	1.Any pattern in issue? (During Month-end process or a specific time range in a day)	Refer best practices per applications and by having good performance in general issue can be minimized
	2. e1root_xxxxxxxx.log?
	3. All the jde_xxxx.logs in your logic server?
	4. Issue occurs only for a certain transaction or data specific?
	5. Does jde_xxxx.log issues *.dmp file?
	6. The process resulted in zombie kernel?
	7. Blocking information in Database?

Note:

jde_yyyy.log for this example, yyyy is process ID (PID which can be verified either through server manager or through e1root_*.log).
e1root_xxxxxxxx.log yyyyyyyy is yyyymmdd. If tools release is 8.96 and below (and WebDev client) which is represented as jderoot_xxxxxxxx.log.
It is important to have good performance in running any application.
Verify jde.ini and jas.ini file to determine timeout settings and other parameters including JDENET and kernel settings.
Data/source is to be verified

jde.ini
jas.ini
e1root_xxxxxxxx.log
jde_xxxx.logs
The source codes for master business function

In case issue is intermittent, review all the jde_xxx.logs in your logic server and e1root_xxxxxxxx.log based on the date the issue has occurred.

Q7> Possible cause and resolution of issue

Possible cause and resolution:

Cause	Resolution	Considerations
The value for Timeout parameters are too small		Unless issue is specific to larget transaction do not change parameter which Oracle provides in your installation of EnterpriseOne
Faulty tools release		Look for known issue through My Oracle Support Portal
Missing bug		When issue is persistent or issue can be simulated at will
User interface issue		Based on user ID written in e1root_*.log, clarify further exact steps to simulate
The size of data	Implement Batch Process or increase timeout values for both jas.ini and jde.ini	<Document 1265902.1> - E1: BSFN: Master Business Function in EnterpriseOne
Blocking/Deadlock	Check external Trigger, stored procedure and/or functions Verify Object Reservation Check Batch Control	<Document 1228683.1> - E1: DB: Case Study on Deadlock in EnterpriseOne Applications
Stored Procedure		<Document 1301427.1> - E1: DB: Custom Stored Procedure Causes Deadlock in the Database
Copied Business Data		<Document 779434.1> - E1: 43: Commit Failed Error in PY after copying Business Data from PD to PY

Example1:

*ERROR* CallObject@12395e7: COSE#1000 Request timeout: timeout after 90000ms host JDEAPP:6015(52647) SocID:49322 PID:4572 BSFN:F4211FSEndDoc user:JDE Env:JPD900
*ERROR* CallObject@12395e7: Server problem. The server may still be available, but because of state information, the entire unit-of-work must be resubmitted.

A piece of information written above reads,

COSE#1000: the nature of error which represents Request timeout
90000ms: enterpriseserverTimeout is 90000 in jas.ini
host JDEAPP: Logic server/machine name
6015: Port Number
PID:4572: Process ID of callobject kernel is 4572 so there will be jde_4572.log in your logic server
BSFN:F4211FSEndDoc: End Doc routine commits cache (created from End Line and/or Begin Doc) to DB. So most of cases 90 seconds should be more than sufficient
user:JDE: User ID is JDE
Env:JPD900: Issue is specific to JPD900 environment

: So this information contains on what is your issue about. Then depends on the nature of issue, obtain the jde_xxxx.log and jdedebug_xxxx.log.