ECS Teams Users experiencing 1 way Audio
Incident Report for Evolve IP
Postmortem

Incident Summary:

On February 15, 2021 approximate 8:05 AM EST we received the first report of some ECS users experiencing one-way audio. By 10:30 AM EST, the issue was escalated to Engineering for investigation. At approximately 11:00 AM EST at least two similar reports had been received and the issue was declared as a global incident and was escalated to senior management. The issue was resolved approximately 12:30 PM EST.

Root Cause:

On Friday night, February 12, 2021 at 11:00 PM EST, planned maintenance was performed to two session border controllers (SBCs) in our network to resolve an issue with ECS MS Teams users having difficulty passing DTMF (touch tone) with external IVR systems. While it appeared that this update fixed the DTMF issue, it introduced this issue with one-way audio for ECS users that use Microsoft Teams as their endpoint.

Resolution:

Reverted the configuration change that was introduced during the Friday maintenance, restoring two-way audio. We will continue to pursue a fix to the DTMF issue.

 Remediation:

  • Evolve is modifying its testing and change management processes to improve its efficacy.
Posted Feb 18, 2021 - 11:07 EST

Resolved
Services have continued to function normally and this incident is being marked resolved. A Service Incident Report, detailing root cause, will be made available in 5 business days.
Posted Feb 15, 2021 - 18:23 EST
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Feb 15, 2021 - 12:55 EST
Identified
The issue has been identified and a fix is being implemented.
Posted Feb 15, 2021 - 12:37 EST
Investigating
We are currently investigating this issue.
Posted Feb 15, 2021 - 12:17 EST
This incident affected: CCaaS (US) (ECS).