Categories
Applications Blog Knowledge Base Products and services

Transcribing voicemails to text with Amazon AWS

For this project we are going to use the Amazon AWS Transcribe service, AWS Transcribe is a cloud-based speech recognition service that converts audio recordings into accurate text transcripts. It uses advanced machine learning algorithms to identify different speakers and punctuation, while also supporting a variety of audio formats and languages. AWS Transcribe can transcribe audio from sources such as phone calls, video recordings, and live streams, making it a versatile tool thats idealy suited for voicemail transcription, The service is highly scalable and cost-effective.

We will say that we used to use Google’s Text to speech engine for thsi but over time I would have expected quality of transcription to have improved, But with Google this is not the case, and I expect this is because they possibly use “predictive” text to speech and not sample all the words as this example below shows, This is the same audio fed to Google and AWS

Amazon AWS Transcribe

Um, this is Ian. I’d like to order some pizza for tomorrow, please. We would like to order a pepperoni pizza and a mozzarella pizza that’s for  tomorrow at five PM. Thank you.

Google Speech to Text

like to order some pizza for tomorrow please would like to order a pepperoni pizza and a mozzarella Pizza Hut for tomorrow at 5 a.m. thank you

As can be seen google misses words and adds others, As you can imagine this isnt what you want with speech transcription.

So we have switched out old script to use AWS.

For this project on Freepbx you need a few extra applications added and a amazon aws account, setting this up is not covered here as you should already have knowledge of this if you are here.

The extra apps are , aws , jq , sox

to get aws :

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip -qq awscliv2.zip
./aws/install

Then you need to configure as 'root' and as 'asterisk', so:
aws configure
fill out your aws key and token as well as the region your bucket is in 
Then repeat as 'asterisk' so
su asterisk
aws configure
and fill out same details.

for jq and sox, just yum install xxx as you would for any other program.

Next you need the asterisk dialplan added to the extensions_custom.conf

[vmail2text]
exten => _XXXX,1,Set(__EXTTOCALL=${EXTEN})
exten => _XXXX,n,Noop(${EXTTOCALL})
exten => _XXXX,n,Goto(s,1)

exten => s,1,Answer()  ; Listen to ringing for 1 seconds
exten => s,n,Set(AGC(rx)=8000)
exten => s,n,Set(DENOISE(rx)=on)
exten => s,n,Noop(${EXTTOCALL} , ${DIALSTATUS} , ${SV_DIALSTATUS})
exten => s,n,GotoIf($["${DIALSTATUS}"="BUSY"]?busy:bnext)
exten => s,n(busy),Set(greeting=busy)
exten => s,n,Goto(carryon)
exten => s,n(bnext),GotoIf($["${DIALSTATUS}"="NOANSWER"]?unavail:unext)
exten => s,n(unavail),Set(greeting=unavail)
exten => s,n,Goto(carryon)
exten => s,n(unext),Set(greeting=unavail)
exten => s,n,Goto(carryon)

exten => s,n(carryon),Set(origmailbox=${EXTTOCALL})
exten => s,n,Set(msg=${STAT(e,${ASTSPOOLDIR}/voicemail/default/${origmailbox}/${greeting}.wav)})
exten => s,n,Set(__start=0)
exten => s,n,Set(__end=0)
exten => s,n,NoOp(${UNIQUEID})
exten => s,n,Set(origdate=${STRFTIME(${EPOCH},,%a %b %d %r %Z %G)})
exten => s,n,Set(origtime=${EPOCH})
exten => s,n,Set(callerchan=${CHANNEL})
exten => s,n,Set(callerid=${CALLERID(num)})
exten => s,n,Set(origmailbox=${origmailbox})
exten => s,n,Answer()
exten => s,n,GotoIf($["${msg}"="1"]?msgy:msgn)
exten => s,n(msgy),Playback(${ASTSPOOLDIR}/voicemail/default/${origmailbox}/${greeting});(local/catreq/how_did)
exten => s,n,Goto(beep)
exten => s,n(msgn),Playback(vm-intro)
exten => s,n(beep),System(/bin/touch /var/lib/asterisk/sounds/catline/${UNIQUEID}.wav)
exten => s,n,Playback(beep)
exten => s,n,Set(__start=${EPOCH})
exten => s,n,Record(catline/${UNIQUEID}.wav,3,60,kaqu)
exten => s,n,Playback(beep)
exten => s,n,Hangup()
exten => h,1,Noop(${start} ${end})
exten => h,n,GotoIf($["${start}"!="0"]?ok:end)
exten => h,n(ok),Set(end=${EPOCH})
exten => h,n,Set(duration=${MATH(${end}-${start},int)})
exten => h,n,System(/usr/local/sbin/vmailprox.sh "${callerchan}" ${callerid} "${origdate}" ${origtime} ${origmailbox} ${UNIQUEID} ${duration})
exten => h,n(end),Noop(finished)

as can be seen this dialplan records a file and then runs the vmailprox.sh script. This script collects the variables and passes them over to the main script and exits after doing so, this is so channels aren’t held while transcription takes place. (Thats the plan anyway)

#!/bin/sh
callerchan=$1
callerid=$2
origdate=$3
origtime=$4
origmailbox=$5
origdir=$6
duration=$7
export callerchan
export callerid
export origdate
export origtime
export origmailbox
export origdir
export duration

nohup /usr/local/sbin/quietvmail.sh &
exit

Main script

#!/bin/sh
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
S3_BUCKET="YOURS3BUCKET"
DIRPATH=/var/spool/asterisk/voicemail/default/
#callerchan=$1
#callerid=$2
#origdate=$3
#origtime=$4
#origmailbox=$5
#origdir=$6
#duration=$7
counter=1
sleep 4

FILENUM=$(/bin/ls ${DIRPATH}${origmailbox}/INBOX |/bin/grep txt | /usr/bin/wc -l)

##Added to allow 999 messages
if  (( $FILENUM <= 9 ));
then
FILENAME=msg000${FILENUM}
elif (( $FILENUM <= 99 ));
then
FILENAME=msg00${FILENUM}
else
FILENAME=msg0${FILENUM}
fi

IN=$(/bin/grep "${origmailbox}=" /etc/asterisk/voicemail.conf)
set -- "$IN"
IFS=","; declare -a Array=($*)
email=${Array[2]}

/bin/echo "[message]" >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo origmailbox=${origmailbox} >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo "context=demo" >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo "macrocontext=" >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo "exten=s" >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo "priority=11" >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo callerchan=${callerchan} >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo callerid=${callerid} >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo origdate=${origdate} >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo origtime=${origtime} >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo msg_id=${origtime}-00000001  >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo "flag="  >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo "category=" >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo "duration=${duration}" >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt

    /bin/nice /usr/bin/lame -b 16 -m m -q 9-resample /var/lib/asterisk/sounds/catline/${origdir}.wav  /tmp/${origdir}.mp3

    # Create a string based on the current date and time
    current_date_time="$(date +%Y-%m-%d_%H-%M-%S)"

    # Upload to the S3 Bucket

    aws --debug --profile default s3 cp /tmp/${origdir}.mp3 s3://$S3_BUCKET/$current_date_time

    # Start the transcription job
    output=$(aws --profile default transcribe start-transcription-job \
    --transcription-job-name $current_date_time \
    --language-code en-GB \
    --media-format mp3 \
    --media MediaFileUri=s3://$S3_BUCKET/$current_date_time \
    --output-bucket-name $S3_BUCKET)

    # Wait for the transcription to finish
    JOB_COMPLETED=false
    while [ "$JOB_COMPLETED" = false ]; do
        JOB_STATUS=$(aws --profile default transcribe get-transcription-job \
            --transcription-job-name $current_date_time \
            --query 'TranscriptionJob.TranscriptionJobStatus' \
            --output text)
            
        if [ "$JOB_STATUS" = "FAILED" ]; then
                JOB_COMPLETED=true
                SHORT_CALL=yes
                /bin/echo "$JOB_STATUS"    >> /tmp/logfile.txt
                break
        fi

        if [ "$JOB_STATUS" = "COMPLETED" ]; then
                /bin/echo "$JOB_STATUS"    >> /tmp/logfile.txt
                JOB_COMPLETED=true
        else

        ((counter++))
        sleep 5
        echo $counter    >> /tmp/logfile.txt
        /bin/echo "$JOB_STATUS"    >> /tmp/logfile.txt
                
                if [ "$counter" -eq  "15" ]; then
                JOB_STAUS=COMPLETED
                JOB_COMPLETED=true
                SHORT_CALL=yes
                break
                fi
        fi
    done


    # Get the transcription result
    aws s3 --profile default cp s3://$S3_BUCKET/$current_date_time.json /tmp/$current_date_time.json 

    # Get the transcription result
    FILTERED=$(jq -r '.results.transcripts[].transcript' /tmp/$current_date_time.json)
                   
        # append result of transcription
        if [ -z "$FILTERED" ]
        then
          echo "(AWS was unable to recognize any speech in audio data.)" >> /tmp/${origdir}.txt
        else
          echo "$FILTERED" >> /tmp/${origdir}.txt
          sed -i 's/ Um,/ /gI' /tmp/${origdir}.txt
        fi

voicemailbody=$(cat "/tmp/${origdir}.txt")

# echo "body ${voicemailbody}"

/bin/cp /var/lib/asterisk/sounds/catline/${origdir}.wav ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.wav

echo -e "You have a new voicemail from ${callerid} it was left on ${origdate} and is ${duration} seconds long,\nThe message left,\n\n${voicemailbody}\n\nTranscribed by the Amazon AWS Transcribe service\n" | /bin/mail -s "A new voicemail has arrived from ${callerid}" -a "/tmp/${origdir}.mp3" "$email"

/bin/rm -f /tmp/${origdir}.mp3
/bin/rm -f /tmp/${origdir}.txt
aws --profile default transcribe delete-transcription-job --transcription-job-name $current_date_time

Then to pass calls to this and not normal voicemail, In Freepbx create a Custom Destination as “vmail2text,s,1” and if you require certain queues to go to specific mailboxes for example 2000 one like “vmail2text,2000,1” so calls will be sent to mailbox 2000 and teh transcriptions will be sent to the email address linked to that extension

Then in extensions that want to use transcription set the “Optional Destinations” in the advanced tab to the custom destination.

Users also can listen to voicemail normally from their handset or the ucp.

These scripts arent only useful for voicemail then can be used fro questionnaire lines and booking lines, anywhere you want to speed up the handling of voice messages. We will soon be looking at ways of integrating this with Whatsapp so transcriptions can be sent to your mobile.

If you have any further questions please email.

Categories
Applications Sangoma Phones Software

Sangoma Desktop Phone

The Sangoma Phone for Desktop turns a user’s computer into a fully featured phone like Connect does for your mobile device so that it is primed to take full advantage of Sangoma’s complete unified communications suite.

Users can leverage the Sangoma Phone wherever sufficient bandwidth is available, allowing them to turn their home, hotel room, favorite café, or other remote location into a fully featured office.

Workforce mobility has never been this easy or powerful, allowing businesses to increase their geographic reach like never before.

Sangoma Phone also presents users with the possibility of reducing hardware costs and travel expenses.

Advanced, crystal clear audio from anywhere
Features include:

  • Hold/Resume
  • Blind Transfer – you can send the call to ring
  • another number or a voicemail box
  • Merge* – conference one call into another
  • Audio Conference Manager* – see your
  • conference-call participants and who is doing
  • the talking, mute people, kick people out of
  • the conference, or end the conference entirely
  • Record* – start and stop a recording of your
  • active call
  • Park* – park, see and answer parked calls

Contact us to enable it on you FreePBX or PBXact system

Categories
Applications Remote Working Services Special Offers

Sangoma Meet, Free multi-party, video conferencing and desktop sharing

Sangoma Technologies announced the general availability of its new video conferencing and collaboration service Sangoma Meet. Sangoma Meet is a multi-party, video conferencing and desktop sharing, cloud-based service, specially designed to enable a great work from home or collaboration experience. HD video support is standard with Sangoma Meet, ensuring the highest quality video calls possible. 

Sangoma Meet is also highly secure, fully encrypting password-protected video conferences to avoid problems with unauthorised users hacking into a video meeting, and Sangoma will not rent or sell Sangoma Meet user information to any third parties, including any social networks. It can also be used as a stand-alone service or in conjunction with its Unified Communication Business Phone Systems. 

Connect

Its simple to use with all the features you need

Moderator Privileges: If you are holding a team call, you have full moderator privileges. You can mute/unmute attendees, send them private messages, or kick anyone off the call. You also have the option to invite audio-only participants to limit video sharing.

Chat: If you prefer not to talk on your video call, use the chat feature in Sangoma Meet to share your thoughts with the group. A full suite of emojis will keep the conversation engaging and light!
Screen Sharing: Give your coworkers real-time examples and streamline meetings with screen sharing in Sangoma Meet. Instantly share your screen and choose what you want to show to others.
Inviting users: Its as simple as, copy your meeting link and send it to the desired parties. As soon as  users click on the link, they immediately join the conference.

Sangoma Meet is available for use today in the company’s General Availability release. It has already been in beta usage, both internally and with partners, with excellent reviews to date.

The Sangoma Meet cloud service has been updated to include additional functionality such as mobile apps (for iOS and Android), calendar integration (for Google G-suite and Microsoft Outlook), desktops apps (for both Windows and Mac) to further enhance user experience, and dial-in functionality to the video conference using a phone (with enhanced security via a PIN).

To help its end-users communicate more effectively during this global pandemic, Sangoma decided to launch this new service completely free of charge to everyone, during these troubled times. 

The full datasheet explaining all the features can be viewed here and the help site is located here

Categories
Applications FreePBX Products

Zulu UC by Sangoma

Zulu UC is a unified communications desktop program that interfaces with FreePBX and PBXact. It provides call notifications, SMS, faxing capabilities and much more! Supported applications include Microsoft Outlook, Mozilla Firefox and Google Chrome.

User connections are available in 20-seat packages, so get yours today by visiting the Store tab in your FreePBX Portal!

Browser Plug-ins

Zulu UC is currently supported for Mozilla Firefox and Google Chrome on Windows machines. Make sure to install the Zulu Firefox Plugin or the Zulu Chrome Plugin before you start using Zulu UC in your browser. Support for Mac OSX is coming soon.

DataSheet can be downloaded here