Cluster outage due to faulty fiber interfaces | EVE Online

Cluster outage due to faulty fiber interfaces

Yesterday we had some Ethernet fiber interfaces mysteriously go haywire on us. This is the second time this happens, only worse this time. We brought in new equipment then and this time we moved over to backup switches as well as using new fiber interfaces. We rebooted about 21:40 GMT to get TQ back to full capacity. Why did they go wrong? We have no idea, but we will open a can of whoopass or two for this. We sincerely apolgize for this inconvenience and are taking measure to prevent this from happening again (as in adding failovers so the failovers have failovers for the failovers).