Post installation basic tests

A very quick test can be done directly from the phone enabled for recording. This test will help you to understand if the Imagicle UC suite is properly configured.

Place a call from the recording-enabled phone line to the Imagicle recorder pilot number (that is the directory number configured in the Recording Profile of CUCM, for instance 8500). 
The call should be answered by Imagicle Call Recording, and one of the following voice prompts should be played:

• Beep: the phone is enabled on IAS for recording calls, the test call is recorded using the "dial-in" mode. 
• Unknown number: the calling number is not present in UC suite users list (remember that each phone enabled for recording must be associated to a valid UC suite user).
• Unauthorized number: the calling number is assigned to a UC suite user that is not enabled to record. You nee to adjust the user permissions accordingly.

Instead, if the call fails, please proceed with the regular troubleshooting procedure, described below.

Attention: if the built-in bridge or network recording mechanism works properly, the diagnostic call to the recorder is recorded as well.

Cisco Media Forking

This section applies to Cisco media forking technology:

• Built-in Bridge
• Network (gateway) based recording

See below the suggested troubleshooting workflow to be applied:

Blue labels refer to below notes.

1. See if the INVITE messages are coming to the Call Recording UDP port (5070), using Wireshark or the Call Recording OPAL log file (not enabled by default). 

If secure (TLS) SIP is used, only the second option applies (in facts, Wireshark traces are unreadable).

2. If built-in bridge mode is used:

• check if the phone model is supported
• check if the built-in bridge is enabled on the phone
• check if the line has been enabled for Call Recording. 

If Network (gateway) Based Recording is used, verify the IOS configuration.

3. Ensure that all CUCM nodes that should run the Call Recording SIP trunk are actually PINGing the Call Recording by SIP OPTIONS messages. If one or more CUCM nodes are not sending PING as expected, a network/firewall issue may be between the CUCM node and the IAS server.

4. Verify that the Call Recording Profile number is properly routed to the Call Recording SIP trunk. If the Call Recording Profile has the same CSS of the phone line, a suggested quick test is dialling the CallRecording Profile number from the problematic phone line: a regular (dial-in) call should be established with the Imagicle Call Recording Service.

Further investigations can be done using the Cisco DNA tool.

5. You can check the codec used by the problematic conversation in the following ways:

• From the phone: access the call info information during the call (press twice the button ‘?’ on 79xx phones, access the admin status menu on other phone models).
• Point a WEB browser to the phone IP address during the problematic conversation: if the WEB access is enabled on the phone, you can see the codec being used by the conversation in progress, selecting the Stream1 option:

• From Jabber desktop: while on a call, click on ctrl+shift+s to see call statistics which includes the codec used. 
• From jabber mobile: click on the three dots, then on "Call statistics" to find out about the codec used.

• After the conversation ended, inspect the CUCM CDR file and check the codec used for the problematic conversation. This is the longest way, but it can be useful if the problem occurs from time to time and cannot be systematically replicated.
• Check also on CDR, if it is a short call (< 3 sec), such false alarms may occur on Imagicle and can be discarded.

6. Check the maximum audio call bandwidth between the recording device region and the Imagicle SIP trunk region is compatible with the codec used for the conversation to be recorded. The recording device can be either the IP phone (for the built-in bridge recording) or the voice gateway (for network-based recording).

For instance, if the conversation to be recorded uses G.711 (64 Kbps) but the max bandwidth between the phone region and the SIP trunk region is 8 Kbps (G.729), the recording will fail for sure.

7. Using the Cisco RTMT (Real Time Monitoring Tool), collect the SDL log files of the CallManager service of all CallManager nodes. Provide them to R&D with some indications about the problematic call (caller/called, date-time), so that they can find it in the logs. Analyzing Cisco logs is a complex, time-consuming and often not resolutive task.  So, please, consider this as the last diagnostic option.

Call Recording-related alerts on Cisco RTMT

On Cisco UCM 10 and higher versions, when the Cisco call recording mechanism fails, a specific alarm is raised by UCM, so that the administrator can be alerted by the Cisco UCM monitoring tools.
The status of such alarms is available in the "Alert Central" section of Real Time Monitoring Tool, tab "Voice/Video".
See the Cisco documentation for the list of the possible alarms and their meaning.

Common Issues

Conversations not being recorded or recorded with no audio

It is important to understand if the media forking SIP calls are coming to Imagicle Call Recording or not. You can get a Wireshark capture on the Imagicle server or, alternatively, you can go through these simple steps:

1. Ensure the Imagicle Call Recording service is running (check the "Manage Service" administrative page).
2. Place a call from/to the phone enable for recording, make the call gets connected. Press the Record softkey if the On-Demand mode has been configured.
3. Wait some seconds and hang up the call.
4. Look at the events history in the Monitoring section of the UC suite. If a Call Recording event has been raised, this means the media forking calls reached the Imagicle Call Recording SIP service but the recording failed, due to codec mismatch or empty audio. NOTE: The description of the event, if any, should help you in understanding the reason of the problem.

Alternatively you can also get a Wireshark capture on the Imagicle server and check for two SIP incoming calls to the recording profile pilot number/DN, UDP port 5070. Please apply "sip || rtp" filter to avoid viewing useless data.

When a call is performed from a recorded extension, the following lines appear on Wireshark. They include the recording pilot number that was added earlier in the recording profile. This means that the RTP stream was received from the recorded phone. Note that two lines are added, because two RTP streams are generated from recorded phone device.  

To verify which extension is being recorded, click on Telephony ⇒ Voip calls:

Depending on the outcome of above test, two different diagnostics procedure must be taken.

Media forking calls not placed to Imagicle Call Recording

In this case, go through this checklist:

1. Ensure the Recording Profile CSS includes the partition of the Route Pattern configured for Imagicle Call Recording. If you assigned to Recording Profile the same CSS of the line being recorded, you can do the test described above in the "Diagnostic Voice Prompts" section, it will help you to understand if the Call Recording SIP trunk is reachable or not.
2. Ensure the phones are properly configured on CUCM, as suggested in the PBX configuration section, in particular, verify the Built-In bridge is enabled on the device.
3. if the previous two steps are properly configured, check in "session trace log view" of RTMT, that the call is actually forked:

Note that on the third row, the Imagicle recorder pilot number (8500) and the device name (SEP0C96E640C7DB) appear on the same line, which means that the built in bridge tried to call the recording pilot number, hence the media is forked from the phone. 

Media forking calls properly placed to Imagicle Call Recording

In the case the media forking calls comes to the Imagicle Call Recording, but the call has not been recorded, one (or more) of the following reasons may apply:

1. The phone number recording for recording is not assigned to any IAS user.
2. The calling number is assigned to a IAS user that is not enabled for Call Recording.
3. The license for Call Recording service is invalid or expired.
4. A codec mismatch occurred, the recording service was not able to answer the media forking calls.
5. Recorded conversation appears in the list, but the audio is empty.

If any of these errors occurs, a specific Call Recording event is raised in the Monitoring section of the Imagicle UC Suite.

Empty Audio

Due to firewall issues or wrong calling platform configuration, it might happen that a recording is saved in UC Suite with no audio. A specific event is raised in Monitoring tool and you can also find additional information in the following log file:

C:\Program Files (x86)\StonevoiceAS\Var\Log\Recorder.Service\Recorder.log

Codec Mismatch

The codec mismatch problem occurs if both the following conditions are true:

• The conversation to be recorded is established using a codec not supported by Imagicle Call Recording (G.729B, G.722, iLBC, ..)
• On CUCM no hardware transcoding resource is available to the Imagicle Call Recording SIP Trunk.

Codec mismatch makes the Call Recording failing when trying to answer the Media Forking Calls.

In terms of diagnostics, when a codec mismatch occurs:

• A Call Recording event is raised in UC Suite (Monitoring section).
• An alarm is raised on CUCM, visible by the RTMT tool.
• In a Wireshark capture, you can see the CUCM hangs up the Media Forking calls. A BYE is sent by CUCM with reason code 47  (Resource Unavailable).

How to fix it

To fix the codec mismatch problem at least one of the following actions should be taken:

• Disable the unsupported codecs on the involved phones and gateways/session border controllers. See the PBX configuration page of this guide for useful indications.
• Alternatively, assign a Media Resource Group List containing an hardware transcoding resource (able to handle the codec) to the Imagicle Call Recording SIP Trunk.
• Check if the phones and the sip trunk belong to two different regions. For example phones belong to a region that allows only the usage of G.711 whereas the sip trunk belongs to a region that allows G729 or vice versa. This error occurs where there is no transcoding resources on the SIP trunk. in order to solve it make sure that both the phones and SIP trunk belong to the same device pool, or to ones that negotiate the same codec. 

If you get an "Error Event 17007" from Call Recording Service, check here for more details.

Wrong  remote numbers

Each recordings is associated to a locale extension number (the phone being recorded) and a remote party number. Both these numbers are displayed in the recordings list:

The remote numbers are adjusted accordingly with the Imagicle suite numbering plan parameters, in particular:

• for incoming calls the Incoming prefix, if set, is stripped out from the remote number
• for outgoing calls the Outgoing prefix, if set, is stripped out from the remote number

Therefore, if remote numbers look wrong, check in the Numbering Plan Parameters page if the Incoming and Outgoing prefixes are correct. If you change them, new values will apply to the new recordings only.

Data Export not working

Most common problems that make data export failing are:

• Wrong credentials for the export job: 
• User does not have write permissions
• Remote network folder is unreachable or does not exists
• Destination has not available space. 

All of these reasons can be easily detected from the WEB interface using the "Test Access" button of the Data Export section (Global Settings).

More over, ensure there is available room there in the remote network folder.

You cannot play the recorded conversations

Different errors may occur when trying to listen or download a recording from the Web interface or from the gadget for Jabber. Following the most common error conditions you may face, with the related explanation.

1) When you click the play button to listen a recorded conversation, you get the following error: "Unable to load the audio stream, please try again".

The browser cannot play the audio stream because the PC on which it is installed does not have and audio adapter, or the Windows Audio service (Audiosrv) is stopped. Add an audio adapter to the PC and start the Audio service.

Note: even if you cannot hear the audio stream, you can still download it as MP3 file.

Another cause could be that recording path has been moved to a no more existing drive. Check here for more details.

2) You get an error: "You cannot playback or download this recording because server certificate has been removed or doesn't contain the private key":

Starting from Imagicle Winter 2018 version, recording, replay and download operations require a valid certificate in the Windows Certificate Store, for encryption scopes.
The error normally occurs if the certificate used to store the selected recording is no more available in the Windows Certificate Store for the replay or download operations.

Another possible reason is that call has been encrypted on another IAS node that is using a different certificate.

In order to retrieve such recordings, you must install the required certificate in the Personal folder of the Windows Certificate Store (see here).

When this issue occurs, the Call Recording “Certificate status” alarm turns RED in IAS Monitoring page and a specific MAM event is raised.

3) The WEB browser is not able to play recordings, even if the MP3 download works fine.

• Try to access web portal from the IAS server, in localhost.
• Set the IAS server in the Intranet zone of the Internet settings of the client PC
• Avoid to use a WEB proxy to access the IAS server.
• Try a different WEB browser.

4) Recording files are no more available on the expected storage location. In this case, a specific error message is returned to the user.

5) If HA is configured and the recording can be played from one node only, please ensure the replication mechanism is properly running. See here for more details.

6) Calls are apparently recorded, but recordings are silent. Check below points:

• A firewall may be blocking the RTP streams from the recording equipment (IP phone or voice gateway) to the IAS server. Ensure UDP traffic is admitted for ports 5000 and higher.
• If you are receiving INVITE messages and can see RTP on Wireshark and not on Imagicle Recording, check the internal firewall (Windows Firewall) as mostly RTP ports are blocked internally by the default firewall rules.

See here for further details.

7) Pop-up window with "Unable to retrieve recordings. Please try again" message. Check here or here for new installations. 

Automated Dial-In Recording

When in conversation, after hitting the "Record" Service Button URL, a number of error messages could be displayed. The following section explain how to troubleshoot common issues when setting up Automated Dial-In Recording, according to to the error message provided.

Service Unavailable

And pressing "Details" softkey button, a similar message is displayed

Cause: Imagicle UC Suite is not able to retrieve IP Phone information from CuCM.

Resolution:

1. Check that UC Suite IP telephony parameters are properly configured.
2. Check that UC Suite general AXL configuration are properly configured.
3. Check that UC Suite is properly querying CuCM through AXL:
   1. Login to UC Suite web portal as admin
   2. Go to Support web page: CuCM version, users and devices must be present
   3. Click on "(details)" button: the IP Phone must be present and must have both the "Detected by AXL" and "Detected By TAPI" flags. If not, please double check device association on the CallManager, and AXL configuration on the IAS server.
4. Check that "Imagicle Service Host" Windows Service is running.

This device is not properly configured for automatic CTI dial-in recording

Cause: the requesting IP Phone is not CTI controlled by Imagicle UC Suite

Resolution:

1. Check device association on CuCM: the requesting IP Phone must be associated to ImagicleCTI user
2. On CuCM, check that the option "Allow Control of Device from CTI" is enabled in the phone configuration (Device --> Phone)

Recording pilot not configured

Cause: the Automated Dial-In Recording Pilot Number has not been configured in Imagicle Call Recording.

Resolution: configure Automated Dial-In Recording Pilot Number in Imagicle Call Recording.

Unable to call Imagicle Call Recording

When hitting "Record" Service Button URL, Imagicle Call Recording places a call from the requesting IP Phone to a number matching the pattern displayed in "Automated Recording" section of "Pilot Numbers" panel, within Call Recording "Global Settings" page. The above message is displayed when that call is not being routed by CuCM to Call Recording SIP Trunk, as expected.

Possible causes:

1. The IP Phone cannot dial a telephone number whose prefix is Automated Dial-In Recording Pilot Number.
2. The matched Route Pattern does not route calls to Call Recording SIP Trunk.

Resolution:

1. On CuCM, check the Calling Search Space of the Directory Number that had the connected call, on the requesting IP Phone: it must allow to call a Route Pattern that routes calls to Call Recording SIP Trunk.
2. On Imagicle Call Recording, check to have configured Automated Dial-In Recording Pilot Number according to CuCM Route Pattern.
3. On CuCM, check that the corresponding Route Pattern:
   1. matches the pattern displayed within section "Automated Recording" in Call Recording "Pilot Numbers" panel
   2. routes call to Call Recording SIP Trunk

To test configuration:

1. Login to UC Suite web portal with Call Recording administrative privileges and go to "Global Settings" -> "Settings" -> "Pilot Numbers" panel
2. Locate the pattern displayed within section "Automated Recording"
3. From the IP Phone, place a call to a number matching that pattern. E.g.: if the pilot number is 9600 (hence the pattern is 9600XXXX), then place a call to 96001234.
4. The call should starts immediately and should be answered by Imagicle Call Recording.

There is no user with primary extension 

Cause: Imagicle Call Recording requires an UC Suite user to be defined for each IP Phone Directory Number to be recorded. See Product Configuration.

Resolution: create an UC Suite user with the requesting IP Phone Directory Number as Primary Extension.

User is not authorized to record calls

Cause: the requesting IP Phone's Directory Number identifies an UC Suite user that is not enabled to Call Recording

Resolution: enable requesting user to Call Recording:

1. Login to UC Suite web portal with administrative privileges for User Management.
2. In User Management, locate the User whose Primary Extension matches the requesting IP Phone's Directory Number.
3. Edit user's permissions for Call Recording.
4. Ensure user has at least "Base Access" privilege.