Här postas inlägg som berör drift vid Adminor.
Vi migrerar just nu smallfoot.xh.se och 06-410-V1.xh.se (webbhotell) till en ny VM-plattform. Syftet är ökad redundans och säkerhetsuppdateringar.
Kortare avbrott kan förekomma på berörda webbhotellstjänster under arbetet. Ingen åtgärd krävs — dina data och inställningar följer med.
Beräknas klart cirka 07.00.
Frågor? Kontakta [email protected].
One of our Transit operators has announced a maintenance which will effect our upstream to Cogent.
”Date of work: January the 23rd
Start time: 00:01 CET (UTC+1)
End time: 05:00 CET (UTC+1)
Place of work: Stokab KN7 (Stockholm, Sweden) Planned Work: NC548-11-1 Work description: Device route processor upgrade
The interruption to your service should not exceed 150 minutes.”
Please note that Adminor uses multiple transits so the impact should be low. If you notice any issues, contact us and we’ll assist.
There is currently pocketless towards Telia network from Västberga (5.226.32.0/21 and 46.253.192.0/20) datacenter. We’ve notified upstream transits who are looking into it.
If necessary we’ll disconnect Cogent temporarily to re-route traffic until it has been fixed, we’ll keep monitoring for now.
Update: Traffic for Västberga has been re-routed away from Cogent during the duration of the packetloss.
According to our provider, tonight’s outage was extraordinary. More information about the outage can be found here: https://status.glesys.com/
In short, both A and B power went down due to an issue in the data center. We are following up with them and will send out more information once the data center operator updates us on what went wrong.
The incident started around 20:45. Adminor’s virtual services were back online after a cautious restart between 21:00 – 21:30. SAN (storage) came back immediately after the power was restored (primary + backup in Västberga). Some services may have taken longer due to fsck checks and other factors, but efforts were made to restore everything as quickly as possible.
We monitored the situation overnight, reactivated Adminor’s replications for backups, and assisted customers in restarting their physical servers where there were issues.
When: 2024-08-12 20.45 – 21.30
Where: Västberga Datacenter
Impact: Outage
Packetloss was detected on inter-datacenter link. This was caused by a faulty fiberpatch in one of the locations.
The patch at VBDC was replaced and optics inspected.
Impact: 00.00 – 09.00 Minor packetloss on link, not causing outage notifications .
Redundancy was degraded during troubleshooting 14.00 – 15.00. No impact on service.
Resolved 15.10.
We’ve added a secondary storage platform to our hosting in Västberga. This will increase local failover capacity and speedup recovery incase of storage outage.
This improvement is for our VB VM hosted customers.
Core switch reboot was performed 01.30 – 01.35 due to memory leak (stack bug).
A tentative fix is the reboot and to avoid causing the ACL conflict which led to the memory leak.
At 04.00 – 04.05 another test was made to replicate the issue. The issue has most likely been found and a new fix has been applied.
Longterm solution will be to replace this switch line with new core routers , Juniper MX204.
At roughly 19.15 CET a network dip was detected due to an issue at a transit operator.
Traffic was automatically re-routed (BGP network re-calculation).
At 14:45 a core switch rebooted. This caused intermittent connection failiure between our two DC’s until paths were rebalanced. All traffic restored ca 14:55 .
swb2-hy
At 14:50 a PDU in HY DC failed.
This caused some equipment to lose grid redundancy.
At 20:30 , all equipment was moved to temporary failover grid redundancy until DC operator can remedy failed PDU.
The root cause was a short circuit in a server power supply causing fuse to trip.
On Tuesday during office hours, DC operator will attempt to remedy the tripped fuse and move equipment back to original PDU.
Impact: No service disruption of network connectivity (redundant power). Reduced Grid redundancy between 14:50 – 20:30.