I’ve been seeing a randomly yet infrequent issue on one of my client’s BizTalk server where the MLLP receive port would throw errors like “Object not set to a reference of an object” or “Remote connection was forcibly closed by the host”. These messages bypasses error routing because there is no message body; That is, while each of the suspended instance has a message attached, the body is empty so they were never in the message box?!
For awhile I thought the issue has to do with the stability of their data connection. But the client have since moved to a full T1 on a different carrier but the problem persists.
While I am not a 100% sure I believe I have narrowed the problem down to the socket connection disconnecting and reconnecting. Let me explain…
Each of the MLLP receive port in question has persistence set to false and a 15 minute timeout period. To check on my theory, I changed the timeout period to 1 minute and used netstat to monitor remote socket switching (i.e. a disconnect and reconnect to a different port due to socket TIME_WAIT). Guess what? Right after the socket switch, there was a suspended message with the error stated earlier.
So if my theory is correct, what are those empty messages? And if a timeout should only occur if the socket is idle beyond the timeout period, what are those “forcibly closed by the host” error messages?
Please leave me a comment if you have any suggestions!
Update: The issue does appear to have to do with the stability of their data connection. Earlier today, their bandwidth was completely peaked out and the said problem popped up right on queue. Now, the only question is, anything can be done to prevent the errors? And at time of error, are we losing messages?
