www.evolveip.net Webpage Outage
Incident Report for Evolve IP
Postmortem

Incident Summary: 

On February 18th, 2021 at approximately 12:45 PM EST we received the first report of website connectivity and rendering issues on www.evolveip.net. At approximately 12:47 PM EST another user reported experiencing the issue. The issue was escalated to Marketing and Systems Engineering to investigate. The issue was resolved at approximately 1:30 PM EST. 

 

Root Cause: 

There were two separate issues that contributed to the incident and are as follows: 

  1. A website user changed a setting within a WordPress plugin that resulted in a stylesheet not rendering correctly 

  2. The website spawned numerous PHP processes to the point of failure 

 

Resolution: 

  1. The plugin setting was reverted, restoring the correct stylesheet and proper rendering. 

  2. The PHP processes spike passed, and the website service was restored 

 

Remediation: 

  • Additional website training is being provided. We’re also evaluating our use of the development environment and change management. 
  • Marketing is working closely with the Systems Engineering team to investigate the root cause of the PHP process spike and mitigate future occurrences.
Posted Feb 18, 2021 - 18:20 EST

Resolved
At12:45PM ET our main webpage was inaccessible for about 45 minutes. This has been resolved at 1:30PM ET.
Posted Feb 18, 2021 - 15:41 EST
This incident affected: Sync (Web Interface).