High Latency

Incident Report for Embedly

Resolved

Everything has returned to normal. 2 bad things happened at once:

1. One of services that checks URLs against the safe browse index died. This is generally not a big deal as it's meant to quietly rebuild itself while traffic keeps humming along.

2. We saw a drastic change in traffic patterns. This made the first problem manifest into a big deal.

We have corrected the first problem and almost double capacity for the night.

Posted Aug 13, 2015 - 21:27 EDT

Update

Root cause found, latency is moving in the correct direction.

Posted Aug 13, 2015 - 21:13 EDT

Update

Still on it, it's the engineer's birthday too. Poor @anrope.

Posted Aug 13, 2015 - 20:18 EDT

Investigating

We are experiencing some increased latency, we are looking into this now.

Posted Aug 13, 2015 - 19:41 EDT