Exceeded allowed rate for Reatime information

Description

Imagicle UCX Suite syncs it's data about phones and, if configured, users with the CUCM cluster using a mechanism known as AXL which is an XML/SOAP API endpoint provided by the CCM platform.

The API endpoint is limited, by default, to 15 queries per minute and have a maximum configurable value of 18 under the CUCM Enterprise Parameters for Rate Control → Allowed Device Queries Per Minute

The AXL client used in the IAS can exceed this rate and will log under %IAS_ROOT%\Var|Log\SvSasAXL.log_<date>.txt an error such as (assuming the debug flag under Support -> Logs is set):

CODE

[DEBUG 1]    <?xml version="1.0" encoding="utf-8"?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><soapenv:Body><soapenv:Fault><faultcode xmlns:ns1="http://xml.apache.org/axis/">ns1:Server.RateControl</faultcode><faultstring>AxisFault: Exceeded allowed rate for Reatime information. Current allowed rate for realtime information is 15 requests per minute.http://schemas.cisco.com/ast/soap/action/#RisPort#SelectCmDevice</faultstring><detail><ns2:hostname xmlns:ns2="http://xml.apache.org/axis/">DCEN-0VAPL00IP</ns2:hostname></detail></soapenv:Fault></soapenv:Body></soapenv:Envelope>
00:01:47 - 00.0000    [DEBUG 1]    This is a SOAP response
00:01:47 -            SOAP RESPONSE contains ERROR: AxisFault: Exceeded allowed rate for Reatime information. Current allowed rate for realtime information is 15 requests per minute.http://schemas.cisco.com/ast/soap/action/#RisPort#SelectCmDevice
00:01:47 -     GetAXLData
00:01:47 - 00.0030    [INFO  2]    [2022-01-31 00:01:47] AxisFault: Exceeded allowed rate for Reatime information. Current allowed rate for realtime information is 15 requests per minute.http://schemas.cisco.com/ast/soap/action/#RisPort#SelectCmDevice
00:01:47 - 00.0020    [INFO  2]    [2022-01-31 00:01:47] AXL Query got SOAP error response
00:01:47 -            *Detected a persistent soap error in phase [Device Details] while importing query 22 of 44, the current import will be aborted and restarted from beginning

The AXL may never be able to complete the sync, thus new users and devices could never appear in the IAS and, upon expire of the old cache and depending on the configuration, the old users may be changed or deleted at the next user sync cycle

Cause

CUCM rate-limits the AXL queries and the IAS AXL client exceeds said limit

Solution

Increase the limit to 18 under the CUCM Enterprise Parameters for Rate Control → Allowed Device Queries Per Minute web interface

RDP into the IAS server and edit the %IAS_ROOT%\Apps\ApplicationSuite\Component\Native\SvSasAXL\SvSasAXL.ini file, add the following line under the [Settings] section:

CODE

EnquiryingTime=<N>

By default, N is 5, try to increase it to 30, ie:

CODE

EnquiryingTime=30

And monitor the AXL log file as explained above, if the error persists, increase it by 5 until it does not happen anymore