“Sorry, couldn’t display the page” earlier today

Dear LinkedIn Users,

Many of you trying to use LinkedIn between 2:18am and 4:08am US Pacific time this morning, and all of you trying to use LinkedIn between 6:10am and 7:43am, were unable to get in.  This is not what we want for our users, and we are very sorry for the inconvenience caused. Rest assured, all of your postings and messages were sent out.

What caused the outage? LinkedIn uses a technology called “Message Queuing” within our site to allow our various services (for example, Network Update Service, inMails) to communicate with each other asynchronously, so that a sudden surge of usage on one part of the site will not affect performance on another.  Starting early this morning, we ran into some issues with our Message Queuing services, which caused the message queues to back up.

What’s supposed to happen in this situation is that the message queues simply store all the pending messages, and then deliver them when the receiving service is ready for them.

However, when some of our message queues backed up this morning, the services that were trying to send additional messages were unable to do so.  This caused the messages to back up into the systems trying to send the messages, causing them to fail.

We have restored the site to proper operation this morning at 7:45am US Pacific/ 3:45pm GMT, including delivery of all messages in the queue, and are analyzing why the message queues did not work so as to prevent it from happening in the future.

Again, our apologies to you for this outage.

Lloyd Taylor
VP Technical Operations