Flow Control


In some situations, message producers may send messages more rapidly than message consumers can receive them. The pending messages for a destination are stored by the server until they can be delivered, and the server can potentially exhaust its storage capacity if the message consumers do not receive messages quickly enough. To avoid this, TIBCO Enterprise Message Service allows you to control the flow of messages to a destination. Each destination can specify a target maximum size for storing pending messages. When the target is reached, TIBCO Enterprise Message Service blocks message producers when new messages are sent. This effectively slows down message producers until the message consumers can receive the pending messages.

Enabling Flow Control

The flow_control parameter in tibemsd.conf enables and disables flow control globally for the TIBCO Enterprise Message Service server. When flow_control is disabled (the default setting), the server does not enforce any flow control on destinations. When flow_control is enabled, the server enforces any flow control settings specified for each destination. See Chapter 7, Using the Configuration Files for more information about working with configuration parameters.

When you wish to control the flow of messages on a destination, set the flowControl property on that destination. The flowControl property specifies the target maximum size of stored pending messages for the destination. The size specified is in bytes, unless you specify the units for the size. You can specify KB, MB, or GB for the units. For example, flowControl = 60MB specifies the target maximum storage for pending messages for a destination is 60 Megabytes.

Enforcing Flow Control

The value specified for the flowControl property on a destination is a target maximum for pending message storage. When flow control is enabled, the server may use slightly more or less storage before enforcing flow control, depending upon message size, number of message producers, and other factors. Setting the flowControl property on a destination but specifying no value causes the server to use a default value of 256KB.

When the storage for pending messages is near the specified limit, the server blocks all new calls to send a message from message producers. The calls do not return until the storage has decreased below the specified limit. Once message consumers have received messages and the pending message storage goes below the specified limit, the server allows the send message calls to return to the caller and the message producers can continue processing.

If there are no message consumers for a destination, the server does not enforce flow control for the destination. That is, if a queue has no started receivers, the server cannot enforce flow control for that queue. Also, if a topic has inactive durable subscriptions or no current subscribers, the server cannot enforce flow control for that topic. For topics, if flow control is set on a specific topic (for example, foo.bar), then flow control is enforced as long as there are subscribers to that topic or any parent topic (for example, if there are subscribers to foo.*).

Routes and Flow Control

For global topics where messages are routed between servers, flow control can be specified for a topic on either the server where messages are produced or the server where messages are received. Flow control is not relevant for queue messages that are routed to another server.

If the flowControl property is set on the topic on the server receiving the messages, when the pending message size limit is reached, messages are not forwarded by way of the route until the topic subscriber receives enough messages to lower the pending message size below the specified limit.

If the flowControl property is set on the topic on the server sending the messages, the server may block any topic publishers when sending new messages if messages cannot be sent quickly enough by way of the route. This could be due to network latency between the routed servers or it could be because flow control on the other server is preventing new messages from being sent.

Destination Bridges and Flow Control

Flow control can be specified on destinations that are bridged to other destinations. If you wish the flow of messages sent by way of the bridge to slow down when receivers on the bridged-to destination cannot process the messages quickly enough, you must set the flowControl property on both destinations on either side of the bridge.

Flow Control, Threads and Deadlock

When using flow control, you must be careful to avoid potential deadlock situations.

When flow control is in effect for a destination, producers to that destination can block waiting for flow control signals from the destination’s consumers. If any of those consumers are within the same thread of program control, a potential for deadlock exists. Namely, the producer will not unblock until the destination contains fewer messages, and the consumer in the blocked thread cannot reduce the number of messages.

The simplest case to detect is when producer and consumer are in the same session (sessions are limited to a single thread). But more complex cases can arise. Deadlock can even occur across several threads (or even programs on different hosts), if dependencies link them. For example, consider the situation in Figure 8:

  • Producer P1 in thread T1 has a consumer C2 in thread T2.
  • Producer P2 in T2 has a consumer C1 in T1.
  • Because of the circular dependency, deadlock can occur if either producer blocks its thread waiting for flow control signals.

The dependency analysis is analogous to mutex deadlock. You must analyze your programs and distributed systems in a similar fashion to avoid potential deadlock.

Figure 8 Flow Control Deadlock across Two Threads

TIBCO Enterprise Message Service™ User’s Guide
Software Release 4.3, February 2006
Copyright © TIBCO Software Inc. All rights reserved
www.tibco.com