Wanpipe Asterisk Debugging

1. Line/Connection debugging
   
   • T1/E1 (A101/2/4/8)
   • Analog (A200/A400/B600/B700)
   • BRI (A500/B700)
     
2. Echo debugging
   
   • Software Echo Canceler
   • Hardware Echo Canceler
3. DTMF detection debugging
   
   • Software DTMF detection
   • Hardware DTMF detection
   • Checking the DTMF received
     

BRI  Debugging 

Before starting to debug the A500 (BRI) card please insure you are running the latest Beta Driver; you can download the latest Beta from the Wanpipe driver download section.

  1)Stop Asterisk and then restart the wanpipe drivers using "wanrouter restart"

  2)Check the output from "wanrouter status", is the line "connected", "connecting", or "disconnected"

 -if the status is "connecting" then this means that there is a physical layer issue. So go over the physical setup of the card. Insure that you are connected in the correct mode, ether TE or NT and the resisters are configured to reflect this. Please see the Hardware Installation Section for help with the physical setup of the card.

-if the status is "connected" then this means the physical layer connection is good.

-if the status is "disconnected” this can ether mean that there is no line connected or some lines go into this state if there is no traffic on the link. You can test this by placing a call out over the link and see if the status changes

3)Check and make sure that the Wanpipe network interface is processing data properly.  Run "ifconfig" and make sure that Rx and Tx packets are increasing for the wanpipe interface (the wanpipe interfaces use the naming pattern of "w1g1" for port 1, "w2g1" for port 2,etc).  If the values are not increasing then please send the following to techdesk@sangoma.com. Or you can run “tail -f /var/log/messages” and see if there is any errors being reported from our drivers.

-the output from "wanrouter version"
-the output from "wanrouter restart" (asterisk needs to be turned off for this so it may need to be done after hours)
-the file /var/log/messages (the entire file containing a wanrouter restart)
-the output from "wanrouter hwprobe verbose"
-the output from "wanrouter status"
-the file(s) /etc/wanpipe/wanpipeX.conf (where X is the wanpipe number)
-the output from "ifconfig", (re run a couple of times with a second or so delay in between each run)
-the output from "cat /proc/interrupts" (re run a couple of times with a second or so delay in between each run) 

4) Visually confirm that “sangoma_mgd”  and “sangoma_brid” are running by running the command “ps fax | grep Sangoma”.  If they are not running then run “/usr/sbin/smg_ctrl restart” and then run “ps fax | grep Sangoma” again to verify that “sangoma_mgd”  and “sangoma_brid” are running.

5) Run "asterisk" followed by "asterisk -r" to connect to the Asterisk console. Then type “woomera default version” and it should display the woomera and smg version. If you do not see this then woomera is not loading successfully into Asterisk. To find out why woomera is not loading into Asterisk then look at “/var/log/asterisk/full” or “/var/log/asterisk/messages” and search for “chan_woomera”. This will indicate why the module is not loading successfully. Also insure that you have compiled the wanpipe for SMG (BRI).

6) If there is still an issue with the connection then you can try changing the connection type in "/etc/wanpipe/smg_bri.conf". For the connection type there is only two possible settings "connection_type=point_to_multipoint" or "connection_type=point_to_point". So change it to the setting that you are currently not running, also insure you do this for each span. After doing this then run "/usr/sbin/smg_ctrl restart" for the changes to take effect.

If the connection type did not help then simply type “asterisk –r” to go into the Asterisk CLI. Then type “set verbose 10” to turn up the verbosity and allow you to see more detail about the calls on the system. Then place a call to recreate the issue and see if the Asterisk output gives you an idea as to why the call is failing. It could be something as simple as the dial plan is not configured.

7) If this does not help then you can edit “/etc/wanpipe/smg_bri.conf” and add the lines “verbose = 5” and “prot_capture=yes” then save and quit. At a time when there is no calls then run the command “/usr/sbin/smg_ctrl restart” which will restart our daemon and apply the changes and drop any active calls.

At this time the log files /var/log/sangoma_mgd.log and /var/log/sangoma_bri.log will start filling up with details about the line activity. You can then place a call to reproduce the issue and then look through these logs to find out why the call dropped. The sangoma_mgd.log will be a more interpreted version of the signaling well /var/log/sangoma_bri.log will show the signaling on more of a lower level. 

Also note these log files should not be enabled for a long period of time and also should only be enabled for debugging purposes. So after the debug is done please remove the previous lines from “/etc/wanpipe/smg_bri.conf” and then run “/usr/sbin/smg_ctrl restart” and insure the logging is disabled. 

8) If you are still having problems then please send the following to techdesk@sangoma.com along with a brief description of the problem and a call diagram. 

Eg.   SIP ------- ASTERISK ------ A500 ----- TELCO

-Edit “/etc/wanpipe/smg_bri.conf” and add the lines “verbose = 5” and “prot_capture=yes” then save and quit
-Then run “/usr/sbin/smg_ctrl restart”           
-Then log all Asterisk output to a log file by:
    =>open up the file /etc/asterisk/logger.conf
    =>look for the line "messages=> notice,warning ..." and add this one line below "sangoma => notice,warning,error,event,verbose"
    =>save the file and restart Asterisk
-increase the verbosity of Asterisk using "set verbose 10"
1,2,3,4,etc)
-make a call out to reproduce the problem.
-turn off Asterisk logging to the file /var/log/asterisk/sangoma by:
    =>open up the file /etc/asterisk/logger.conf
    =>look for the line "sangoma => notice,warning,error,event,verbose", and either remove it or uncomment it
    =>save the file and restart Asterisk 
- The file /var/log/asterisk/sangoma
- The files /etc/wanpipe/*,
               /etc/asterisk/*,
               /etc/sangoma_mgd.conf,
               /var/log/sangoma_bri.log,
               /var/log/sangoma_mgd.log,
               /var/log/messages (After you run the wanrouter restart)
               /etc/asterisk/woomera.conf
               /etc/asterisk/extensions.conf
               -The output of the following commands:
                        wanrouter version
                        wanrouter status
                        wanrouter restart
                        /usr/sbin/smg_ctrl restart
                        wanrouter hwprobe verbose
                        ps fax | grep sangoma
                        ifconfig (Run 3 times with a small delay between)
                        cat /proc/interrupts (Run 3 times with a small delay between)
 
After this is all done then remove the lines added to “/etc/wanpipe/smg_bri.conf“ and then run “/usr/sbin/smg_ctrl restart”

Asterisk PRI Span Debugging 

If you are having problems with a PRI span follow the steps below to troubleshoot the issue: 

1)Stop Asterisk and then restart the wanpipe drivers using "wanrouter restart"

2)Check the output from "wanrouter status", is the line "connected", "connecting", or "disconnected"

-if the status is "connecting" then check that there are no major alarms on the line using "wanpipemon -i w1g1 -c Ta"

-->OOF alarm means the incoming signal isn't quite right, check your line settings E1/T1, line framing, and line coding (contact your telco if you are not 100% sure)
-->LOS or Loss of Signal alarm means there is no signal at all on the line; first check the cable then check with your telco that the line is live
-->Short Circuit alarms means that the pin out of the cable is wrong

-if the status is "connected" then check that there are no RAI alarms on the line by looking at /var/log/messages
-if the status is "disconnected" then check that there is a cable connected to the card and then restart the drivers again.

3)Check and make sure that the Wanpipe network interface is processing data properly.  Run "ifconfig" and make sure that Rx and Tx packets are increasing for the wanpipe interface (the wanpipe interfaces use the naming pattern of "w1g1" for port 1, "w2g1" for port 2,etc).  If the values are not increasing then please send the following to techdesk@sangoma.com:

-the output from "wanrouter version"
-the output from "wanrouter restart" (asterisk needs to be turned off for this so it may need to be done after hours)
-the file /var/log/messages (the entire file containing a wanrouter restart)
-the output from "wanrouter hwprobe verbose"
-the output from "wanrouter status"
-the file(s) /etc/wanpipe/wanpipeX.conf (where X is the wanpipe number)
-the output from "ifconfig", (re run a couple of times with a second or so delay in between each run)
-the output from "cat /proc/interrupts" (re run a couple of times with a second or so delay in between each run)

4)Run "ztcfg -vvv" to register the wanpipes with Zaptel, check for any error messages, if you see errors check your zaptel.conf file.

5)Run "asterisk" followed by "asterisk -r" to connect to the Asterisk console. Run "zap show channels" and make sure that Asterisk has all the Zap channels, if not check you zapata.conf or zapata-auto.conf and verify that these files correct register the Zap channels.

 RBS: For Robbed bit signaling please go to our Debugging Asterisk/Wanpipe RBS page.

6)Run "pri show span 1", check whether the span is "Up and Active". If it is not "Up and active" turn on "pri intense debug span 1" in Asterisk to see the D-channel messages, make sure there are incoming and outgoing messages.  Once you have confirmed that there incoming or outgoing frames confirm also that the Wanpipe drivers are seeing the same; from the Linux command prompt run "wanpipemon -i w1g1 -c trd" (NOTE: this will only work if Wanpipe is doing the HDLC framing, which it is by default).  This command traces the D-channel below Asterisk/Zaptel and confirms 100% whether the frames are being received or transmitted on the line.

If there are only incoming frames check the following link: Hardware HDLC
If there are only outbound frames then contact you telco and confirm that the D-channel has been activated.
If there are both incoming and outgoing frames and you are using Zaptel-1.4.X and Wanpipe-3.2.X (or newer) check this link: HARDHDLC vs DCHAN

7)If the span is "up and active" then turn up the verbosity of Asterisk to 10 using "set verbose 10". Try to make a call out and look at the messages that Asterisk is generating, is the call transfered to the Zap channel? (i.e Dial(Zap/...)

8)If the call is transfered to a Zap channel then turn on "pri intense debug span 1" and try to make a call out. You will now see the PRI messages related to the call, check for a Cause Code and look it up online (ISDN Cause codes).

9)If you are still having problems then please send the following to techdesk@sangoma.com along with a brief description of the problem:

-the output from "wanrouter version"
-the output from "wanrouter restart" (asterisk needs to be turned off for this so it may need to be done after hours)
-the file /var/log/messages (the entire file containing a wanrouter restart)
-the output from "wanrouter hwprobe verbose"
-the output from "wanrouter status"
-the file(s) /etc/wanpipe/wanpipeX.conf (where X is the wanpipe number)
-the file /etc/zaptel.conf
-the file(s) /etc/asterisk/zapata*.conf
-the output from "ifconfig", (re run a couple of times with a second or so delay in between each run)
-the output from "cat /proc/interrupts" (re run a couple of times with a second or so delay in between each run)
-the output from "wanpipemon -i wXg1 -c Ta" (where X is the interface number 1,2,3,etc)
-log all output from the Asterisk console to a file:
    =>open up the file /etc/asterisk/logger.conf
    =>look for the line "messages=> notice,warning ..." and add this one line below "sangoma => notice,warning,error,event,verbose"
    =>save the file and restart Asterisk
-increase the verbosity of Asterisk using "set verbose 10"
-the output from "pri show span X" from the Asterisk console, where X is the span having problems (i.e. 1,2,3,4,etc)
-run "pri intense debug span X" from the Asterisk console so that Asterisk captures debug information, where X is the span having problems (i.e. 1,2,3,4,etc)
-make a call out to reproduce the problem or leave the system running until it occurs
-turn off Asterisk logging to the file /var/log/asterisk/sangoma by:
    =>open up the file /etc/asterisk/logger.conf
    =>look for the line "sangoma => notice,warning,error,event,verbose", and either remove it or uncomment it
    =>save the file and restart Asterisk
-send me the file /var/log/asterisk/sangoma
***To turn of the debugging in Asterisk run "pri no debug span X", where X is the span in question***

Analog Debugging

1)Stop Asterisk and then restart the wanpipe drivers using "wanrouter restart" 

2)Check and make sure that the Wanpipe network interface is processing data properly.  Run "ifconfig" and make sure that Rx and Tx packets are increasing for the wanpipe interface (the wanpipe interfaces use the naming pattern of "w1g1" for port 1, "w2g1" for port 2,etc).  If the values are not increasing then please send the following to techdesk@sangoma.com:

-the output from "wanrouter version"
-the output from "wanrouter restart" (asterisk needs to be turned off for this so it may need to be done after hours)
-the file /var/log/messages (the entire file containing a wanrouter restart)
-the output from "wanrouter hwprobe verbose"
-the output from "wanrouter status"
-the file(s) /etc/wanpipe/wanpipeX.conf (where X is the wanpipe number)
-the output from "ifconfig", (re run a couple of times with a second or so delay in between each run)
-the output from "cat /proc/interrupts" (re run a couple of times with a second or so delay in between each run) 

3)Run "ztcfg -vvv"  or "dahdi_cfg -vvv" to register the wanpipes with Zaptel/Dahdi, check for any error messages, if you see errors check your zaptel.conf  or /etc/dahdi/system.conf file. 

4)Run "asterisk" followed by "asterisk -r" to connect to the Asterisk console. Run "zap show channels" or "dahdi show channels" and make sure that Asterisk has all the Zap/Dahdi channels, if not check you zapata.conf / zapata-auto.conf or chan_dahdi.conf and verify that these files correct register the Zap/Dahdi channels. If there is no zap/dahdi channels registered and the Zapata*.conf configuration is correct then search the /var/log/asterisk/messages or /var/log/asterisk/full for “chan_zap” or "chan_dahdi" and this should lead you to the error. 

5)If all the channels are loaded and you are still unable to make calls in/out then please check the voltage for each channel, this will insure that our card is able to see the line plugged into it. The command to do this is “wanpipemon -i w1g1 -c astats -m X” x being the channel number. If there is no voltage then please take a volt meter and insure that there is voltage on the inner pair of the RJ11 connector. 

0-1 - No line detected
6-12 - Offhook
45-55 – Onhook 

6) If there is a caller ID issue then, first insure that telco is sending the CID by using a normal analog phone on the line. Next after you know telco is sending caller id then go to the following URLS, and see if the suggestions there help resolve the issue.

-http://www.voip-info.org/wiki-Asterisk+config+zapata.conf#CallerIDOptions
-http://www.voip-info.org/wiki/view/Asterisk+and+UK+Caller+ID
-http://www.voip-info.org/wiki/view/Asterisk+and+Australian+Caller+ID 

7) If there is a hangup detection issue then you can try the following URL’s suggestions to improve hang-up detection.

-http://www.voip-info.org/wiki/index.php?page=Asterisk+Disconnect+Supervision
-http://www.voip-info.org/wiki/view/Australia+Asterisk+Details 

If this does not work then it has been known that some telco’s place the hang-up detection signaling on the line shorter then the standard. So to allow our card to pass a smaller hang-up detection signaling through you can change the following value in /etc/wanpipe/wanpipeX.conf (x being the wanpipe number). 

Change “RM_BATTDEBOUNCE = 16” to “RM_BATTDEBOUNCE = 4” 

If 4 works then you can try increasing this number and see which one works best for your line. If you go much lower then 4 then you could get false hang-ups.

Also in Spain the following settings in your /etc/asterisk/zapata.conf has been known to help with hangup detection.

busydetect=no
answeronpolarityswitch=yes
hanguponpolarityswitch=yes
callprogress=no

Software Echo Canceler Debugging

Here are a few steps we have used to debug echo.

6. If all other methods of reducing echo have failed, Asterisk also has an option in the zconfig.h to makethe echo cancellation more aggressive. You can enable it by uncommenting the following line:

      #define AGGRESSIVE_SUPPRESSOR

    Note that aggressive echo cancellation can create a walkie-talkie, half-duplex effect. This should be enabled only if all above methods have failed.

You will get better performance from good carrier-grade hardware EC such as used on our "d" models.

Hardware Echo Canceler Debugging

How to confirm that your hardware echo canceler is running and active on your Zap calls.

1. Confirm that you have a hardware echo canceler:
    After the Wanpipe drivers are installed, type "wanrouter hwprobe" it should say:   

    1. AFT-AXX-SH : SLOT=X : BUS=X : IRQ=X CPU=A: PORT=X : HWEC= 32 : V=X
        HWEC values should be:
            A200d : 32
            A101d : 32
            A102d : 64
            A104d : 128
            A108d : 256
            A500d : 64          

If it says HWEC=0, and you ordered a "d" model card then this card needs to be replaced so if you purchased this card from a reseller, contact your reseller. If you purchased this card directly from Sangoma, then please contact our RMA department.

2. If you have a hardware echo canceller,  confirm that:

- You have Wanpipe-2.3.4 or newer
- You have the parameter "TDMV_HWEC = YES" in your /etc/wanpipe/wanpipe*.conf
- You have the parameter "echocancel=yes" in your /etc/asterisk/zapata.conf

3. Run "wanrouter start" to start our drivers then run "wan_ec_client wanpipe1 stats"

If you do not see any output, contact Sangoma Tech Support and attach /var/log/messages to the email.
      
You should see:      

wanpipe1: Running Get stats command to Echo Canceller device... Done!
***** Echo Canceller Chip Get Stats wanpipe1 *****
wanpipe1: Number of channels currently open    32

This confirms that the WANPIPE drivers can communicate with the hardware echo canceller chip properly.        

4. Make a call through Asterisk. While the call is active type "zap show channel N" (N being the zap channel number) in the Asterisk CLI. In the output it should say "echocancel: 128 TAPS, currently ON" if it does not say "currently ON", check your /etc/asterisk/zapata.conf, if you are bridging TDM to TDM, and you want echo canceling to be performed, then you should also have "echocancelwhenbridged=yes"
         
If it says "currently ON" then check if the hardware EC is enabled or not, by running the command " wanpipemon -i <interface name> -c ehw "

For example:
#> wanpipemon -i w1g1 -c ehw
Sangoma HW echo canceller active on channel N
where N is your active channel.

With wanpipe-3.2.x and wanpipe-2.3.4-5 or later, we have persistent hardware EC means hardware EC is enabled all the times, therefore you should see following for all the voice channels

#>  wanpipemon -i w1g1 -c ehw
Sangoma HW Echo Canceller is enabled for channel N  ( N: is your channel number)

Software DTMF Detection

1) Edit your /etc/asterisk/zapata.conf and change set "relaxdtmf=yes" then restart Asterisk.

2) Go into the Asterisk CLI "asterisk -r" and run "zap show channel X" (x being the zap channel in question) and this will tell you if relaxdtmf is enabled. Place some test calls and see if the issue has been resolved. 

3) If the issue remains then go into your /etc/asterisk/zapata.conf and increase your rxgain value. You can apply this change with just the command reload from the Asterisk CLI.

4) If the issue are still remaining then check the quality of the line and insure there is no echo or noise issues. 

5) If there is no noise issues then please upgrade and try our hardware DTMF, instructions for this can be found at Appendix --> Asterisk & Sangoma Hardware DTMF.

Hardware DTMF Detection

NOTE: You need a hardware echo canceler. To see if you have a HWEC run the command "wanrouter hwprobe" and it should say:

    1. AFT-AXX-SH : SLOT=X : BUS=X : IRQ=X CPU=A: PORT=X : HWEC= 32 : V=X
        HWEC values should be:
            A200d : 32
            A101d : 32
            A102d : 64
            A104d : 128
            A108d : 256
            A500d : 64          

1) Insure that your line has no noise issues. If you have noise issues then check "ifconfig" for overruns.

2) Upgrade to the latest Beta driver which can be found at http://wiki.sangoma.com/wanpipe-linux-drivers#beta

3) Edit your /etc/wanpipe/wanpipe*.conf file and change "TDMV_HW_DTMF    = YES" to "TDMV_HW_DTMF    = NO" and then stop asterisk and run "Wanrouter restart" and then start asterisk back up and see if this resolves the issue.

4) Ether way if the hardware DTMF did not work then please send an email to techdesk@sangoma.com along with the following information and a brief description of the issue.

-the output from "wanrouter version"
-the output from "wanrouter restart" (asterisk needs to be turned off for this so it may need to be done after hours)
-the file /var/log/messages (the entire file containing a wanrouter restart)
-the output from "wanrouter hwprobe verbose"
-the output from “wanrouter status”
-the file(s) /etc/wanpipe/wanpipeX.conf (where X is the wanpipe number)
-the file /etc/zaptel.conf
-the file(s) /etc/asterisk/zapata*.conf
-the output from "ifconfig", (re run a couple of times with a second or so delay in between each run)
-the output from "cat /proc/interrupts" (re run a couple of times with a second or so delay in between each run)
-the output from "wanpipemon -i wXg1 -c Ta" (where X is the interface number 1,2,3,etc) for each interface

Checking the DTMF received

1) You can simply go into the Asterisk CLI with the command asterisk -rvvvvvv and then pick up the channel you want to debug and you will see the output below.

-- Starting simple switch on 'Zap/1-1'

2) Once you see the output above simply run the command debug channel Zap/1-1 to start the debugging.

Note: the channel has to be in use before running the debug command.

3) Then press DTMF digits and you will see the following output, note the digit that was pressed is highlighted in red.

<< [ TYPE: Null Frame (5) SUBCLASS: N/A (0) ] [Zap/1-1]
<< [ TYPE: DTMF End (1) SUBCLASS: 1 (49) ] [Zap/1-1]
<< [ TYPE: Null Frame (5) SUBCLASS: N/A (0) ] [Zap/1-1]
<< [ TYPE: DTMF End (1) SUBCLASS: 2 (50) ] [Zap/1-1]

4) You can also try our dial plan which plays back the DTMF digits to you.

DTMF_Dial_Plan.rar