Load balancing in oracle data integrator

Load balancing


Oracle Data Integrator implements load balancing between physical agents.

Each physical agent is defined with:

A maximum number of sessions it can execute simultaneously

Optionally, a number of linked physical agents to which it can delegate sessions' executions.

An agent's load is determined at a given time by the ratio (Number of running sessions / Maximum number of sessions) for this agent.

Determining the Maximum number of sessions

The maximum number of sessions is a value that must be set depending on the capabilities of the machine running the agent. It can be also set depending on the amount of processing power you want to give to the Data Integrator agent.

Delegating sessions

When a session is started on an agent with linked agents, Oracle Data Integrator determines which one of the linked agents is less loaded, and the session is delegated to this linked agent.

If the user parameter "Use new load balancing" is in use, sessions are also re-balanced each time a session finishes. This means that if an agent runs out of sessions, it will possibly be reallocated some from another agent.

Note the following:

An agent can be linked to itself. An agent not linked to itself is only able to delegate sessions to its linked agents, and will never execute a session.

Delegation works on cascades of linked agents. Besides, it is possible to make loops in agents links. This option is not recommended.

Agent unavailable

When for a given agent the number of running sessions is equal to its Maximum number of sessions, the agent will set incoming sessions in a "queued" status until the number of running sessions falls below the maximum of sessions for this agent.

Setting Up load balancing

To setup load balancing:

Define a set of physical agents, and link them to a root agent. More

Start the root and the linked agents.

Run the executions on the root agent. Oracle Data Integrator will balance the load of the executions between its linked agents.




Note:

Stopping a Load Plan Run

Stopping a Load Plan Run
 

Any running or waiting Load Plan Run can be stopped. You may want to stop a Load Plan Run when you realize that for example your Load Plan contains errors or when the execution takes a long time.

Note that there are two ways to stop a Load Plan Run:

Stop Normal: In normal stop mode, the agent in charge of stopping the Load Plan sends a Stop Normal signal to each agent running a session for this Load Plan. Each agent will wait for the completion of the current task of the session and then end the session in error. Exception steps will not be executed by the Load Plan and once all exceptions are finished the load plan is moved to an error state.

Stop Immediate: In immediate stop mode, the agent in charge of stopping the Load Plan sends a Stop immediate signal to each agent running a session for this Load Plan. Each agent will immediately end the session in error and not wait for the completion of the current task of the session. Exception steps will not be executed by the Load Plan and once all exceptions are finished the load plan is moved to an error state.

What is Load Plan Exceptions and Restartability

What is Load Plan Exceptions and Restartability


Load Plans provide two features for handling error cases in the execution flows: Exceptions and Restartability.

Exceptions

An Exception Step contains a hierarchy of steps that is defined on the Exceptions tab of the Load Plan editor.

You can associate a given exception step to one or more steps in the Load Plan. When a step in the Load Plan errors out, the associated exception step is executed automatically.

Exceptions can be optionally raised to the parent step of the failing step. Raising an exception fails the parent step, which can consequently execute its exception step.

Restartability

When a Load Plan Run is restarted after a failure, the failed Load Plan steps are restarted depending on the Restart Type parameter. For example, you can define whether a parallel step should restart all its child steps or only those that have failed.