Category: Products and services

For this project we are going to use the Amazon AWS Transcribe service, AWS Transcribe is a cloud-based speech recognition service that converts audio recordings into accurate text transcripts. It uses advanced machine learning algorithms to identify different speakers and punctuation, while also supporting a variety of audio formats and languages. AWS Transcribe can transcribe audio from sources such as phone calls, video recordings, and live streams, making it a versatile tool thats idealy suited for voicemail transcription, The service is highly scalable and cost-effective.

We will say that we used to use Google’s Text to speech engine for thsi but over time I would have expected quality of transcription to have improved, But with Google this is not the case, and I expect this is because they possibly use “predictive” text to speech and not sample all the words as this example below shows, This is the same audio fed to Google and AWS

Amazon AWS Transcribe

Um, this is Ian. I’d like to order some pizza for tomorrow, please. We would like to order a pepperoni pizza and a mozzarella pizza that’s for  tomorrow at five PM. Thank you.

Google Speech to Text

like to order some pizza for tomorrow please would like to order a pepperoni pizza and a mozzarella Pizza Hut for tomorrow at 5 a.m. thank you

As can be seen google misses words and adds others, As you can imagine this isnt what you want with speech transcription.

So we have switched out old script to use AWS.

For this project on Freepbx you need a few extra applications added and a amazon aws account, setting this up is not covered here as you should already have knowledge of this if you are here.

The extra apps are , aws , jq , sox

to get aws :

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip -qq awscliv2.zip
./aws/install

Then you need to configure as 'root' and as 'asterisk', so:
aws configure
fill out your aws key and token as well as the region your bucket is in 
Then repeat as 'asterisk' so
su asterisk
aws configure
and fill out same details.

for jq and sox, just yum install xxx as you would for any other program.

Next you need the asterisk dialplan added to the extensions_custom.conf

[vmail2text]
exten => _XXXX,1,Set(__EXTTOCALL=${EXTEN})
exten => _XXXX,n,Noop(${EXTTOCALL})
exten => _XXXX,n,Goto(s,1)

exten => s,1,Answer()  ; Listen to ringing for 1 seconds
exten => s,n,Set(AGC(rx)=8000)
exten => s,n,Set(DENOISE(rx)=on)
exten => s,n,Noop(${EXTTOCALL} , ${DIALSTATUS} , ${SV_DIALSTATUS})
exten => s,n,GotoIf($["${DIALSTATUS}"="BUSY"]?busy:bnext)
exten => s,n(busy),Set(greeting=busy)
exten => s,n,Goto(carryon)
exten => s,n(bnext),GotoIf($["${DIALSTATUS}"="NOANSWER"]?unavail:unext)
exten => s,n(unavail),Set(greeting=unavail)
exten => s,n,Goto(carryon)
exten => s,n(unext),Set(greeting=unavail)
exten => s,n,Goto(carryon)

exten => s,n(carryon),Set(origmailbox=${EXTTOCALL})
exten => s,n,Set(msg=${STAT(e,${ASTSPOOLDIR}/voicemail/default/${origmailbox}/${greeting}.wav)})
exten => s,n,Set(__start=0)
exten => s,n,Set(__end=0)
exten => s,n,NoOp(${UNIQUEID})
exten => s,n,Set(origdate=${STRFTIME(${EPOCH},,%a %b %d %r %Z %G)})
exten => s,n,Set(origtime=${EPOCH})
exten => s,n,Set(callerchan=${CHANNEL})
exten => s,n,Set(callerid=${CALLERID(num)})
exten => s,n,Set(origmailbox=${origmailbox})
exten => s,n,Answer()
exten => s,n,GotoIf($["${msg}"="1"]?msgy:msgn)
exten => s,n(msgy),Playback(${ASTSPOOLDIR}/voicemail/default/${origmailbox}/${greeting});(local/catreq/how_did)
exten => s,n,Goto(beep)
exten => s,n(msgn),Playback(vm-intro)
exten => s,n(beep),System(/bin/touch /var/lib/asterisk/sounds/catline/${UNIQUEID}.wav)
exten => s,n,Playback(beep)
exten => s,n,Set(__start=${EPOCH})
exten => s,n,Record(catline/${UNIQUEID}.wav,3,60,kaqu)
exten => s,n,Playback(beep)
exten => s,n,Hangup()
exten => h,1,Noop(${start} ${end})
exten => h,n,GotoIf($["${start}"!="0"]?ok:end)
exten => h,n(ok),Set(end=${EPOCH})
exten => h,n,Set(duration=${MATH(${end}-${start},int)})
exten => h,n,System(/usr/local/sbin/vmailprox.sh "${callerchan}" ${callerid} "${origdate}" ${origtime} ${origmailbox} ${UNIQUEID} ${duration})
exten => h,n(end),Noop(finished)

as can be seen this dialplan records a file and then runs the vmailprox.sh script. This script collects the variables and passes them over to the main script and exits after doing so, this is so channels aren’t held while transcription takes place. (Thats the plan anyway)

#!/bin/sh
callerchan=$1
callerid=$2
origdate=$3
origtime=$4
origmailbox=$5
origdir=$6
duration=$7
export callerchan
export callerid
export origdate
export origtime
export origmailbox
export origdir
export duration

nohup /usr/local/sbin/quietvmail.sh &
exit

Main script

#!/bin/sh
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
S3_BUCKET="YOURS3BUCKET"
DIRPATH=/var/spool/asterisk/voicemail/default/
#callerchan=$1
#callerid=$2
#origdate=$3
#origtime=$4
#origmailbox=$5
#origdir=$6
#duration=$7
counter=1
sleep 4

FILENUM=$(/bin/ls ${DIRPATH}${origmailbox}/INBOX |/bin/grep txt | /usr/bin/wc -l)

##Added to allow 999 messages
if  (( $FILENUM <= 9 ));
then
FILENAME=msg000${FILENUM}
elif (( $FILENUM <= 99 ));
then
FILENAME=msg00${FILENUM}
else
FILENAME=msg0${FILENUM}
fi

IN=$(/bin/grep "${origmailbox}=" /etc/asterisk/voicemail.conf)
set -- "$IN"
IFS=","; declare -a Array=($*)
email=${Array[2]}

/bin/echo "[message]" >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo origmailbox=${origmailbox} >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo "context=demo" >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo "macrocontext=" >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo "exten=s" >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo "priority=11" >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo callerchan=${callerchan} >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo callerid=${callerid} >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo origdate=${origdate} >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo origtime=${origtime} >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo msg_id=${origtime}-00000001  >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo "flag="  >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo "category=" >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt
/bin/echo "duration=${duration}" >> ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.txt

    /bin/nice /usr/bin/lame -b 16 -m m -q 9-resample /var/lib/asterisk/sounds/catline/${origdir}.wav  /tmp/${origdir}.mp3

    # Create a string based on the current date and time
    current_date_time="$(date +%Y-%m-%d_%H-%M-%S)"

    # Upload to the S3 Bucket

    aws --debug --profile default s3 cp /tmp/${origdir}.mp3 s3://$S3_BUCKET/$current_date_time

    # Start the transcription job
    output=$(aws --profile default transcribe start-transcription-job \
    --transcription-job-name $current_date_time \
    --language-code en-GB \
    --media-format mp3 \
    --media MediaFileUri=s3://$S3_BUCKET/$current_date_time \
    --output-bucket-name $S3_BUCKET)

    # Wait for the transcription to finish
    JOB_COMPLETED=false
    while [ "$JOB_COMPLETED" = false ]; do
        JOB_STATUS=$(aws --profile default transcribe get-transcription-job \
            --transcription-job-name $current_date_time \
            --query 'TranscriptionJob.TranscriptionJobStatus' \
            --output text)
            
        if [ "$JOB_STATUS" = "FAILED" ]; then
                JOB_COMPLETED=true
                SHORT_CALL=yes
                /bin/echo "$JOB_STATUS"    >> /tmp/logfile.txt
                break
        fi

        if [ "$JOB_STATUS" = "COMPLETED" ]; then
                /bin/echo "$JOB_STATUS"    >> /tmp/logfile.txt
                JOB_COMPLETED=true
        else

        ((counter++))
        sleep 5
        echo $counter    >> /tmp/logfile.txt
        /bin/echo "$JOB_STATUS"    >> /tmp/logfile.txt
                
                if [ "$counter" -eq  "15" ]; then
                JOB_STAUS=COMPLETED
                JOB_COMPLETED=true
                SHORT_CALL=yes
                break
                fi
        fi
    done


    # Get the transcription result
    aws s3 --profile default cp s3://$S3_BUCKET/$current_date_time.json /tmp/$current_date_time.json 

    # Get the transcription result
    FILTERED=$(jq -r '.results.transcripts[].transcript' /tmp/$current_date_time.json)
                   
        # append result of transcription
        if [ -z "$FILTERED" ]
        then
          echo "(AWS was unable to recognize any speech in audio data.)" >> /tmp/${origdir}.txt
        else
          echo "$FILTERED" >> /tmp/${origdir}.txt
          sed -i 's/ Um,/ /gI' /tmp/${origdir}.txt
        fi

voicemailbody=$(cat "/tmp/${origdir}.txt")

# echo "body ${voicemailbody}"

/bin/cp /var/lib/asterisk/sounds/catline/${origdir}.wav ${DIRPATH}${origmailbox}/INBOX/${FILENAME}.wav

echo -e "You have a new voicemail from ${callerid} it was left on ${origdate} and is ${duration} seconds long,\nThe message left,\n\n${voicemailbody}\n\nTranscribed by the Amazon AWS Transcribe service\n" | /bin/mail -s "A new voicemail has arrived from ${callerid}" -a "/tmp/${origdir}.mp3" "$email"

/bin/rm -f /tmp/${origdir}.mp3
/bin/rm -f /tmp/${origdir}.txt
aws --profile default transcribe delete-transcription-job --transcription-job-name $current_date_time

Then to pass calls to this and not normal voicemail, In Freepbx create a Custom Destination as “vmail2text,s,1” and if you require certain queues to go to specific mailboxes for example 2000 one like “vmail2text,2000,1” so calls will be sent to mailbox 2000 and teh transcriptions will be sent to the email address linked to that extension

Then in extensions that want to use transcription set the “Optional Destinations” in the advanced tab to the custom destination.

Users also can listen to voicemail normally from their handset or the ucp.

These scripts arent only useful for voicemail then can be used fro questionnaire lines and booking lines, anywhere you want to speed up the handling of voice messages. We will soon be looking at ways of integrating this with Whatsapp so transcriptions can be sent to your mobile.

If you have any further questions please email.