Emergency maintenance (Resolved)
  • Priority - Low
  • Affecting Other - Global Layer network
  • Description
    -----------
    Maintenance backbone Albasserdam-Amsterdam

    Location(s) affected
    --------------------
    - (alb1.nl)dataplace

    Services affecting
    ------------------
    - MPLS-transport protected: you may experience a failover
    - MPLS-transport unprotected: loss of connectivity
    - IP-transit: traffic will be rerouted, higher latency may occur

    Initial message @ 2012-10-23 22:15:13 CEST
    ------------------------------------------
    Our fibre provider is performing maintenance on one of the two legs
    between Alblasserdam and Amsterdam.


    If you are a BGP customer with a redundant session, traffic will flow
    over the redundant session.

    If you are a MPLS customer, traffic will be rerouted over the
    secondary LSP.

  • Date - 23-10-2012 22:00 - 24-10-2012 02:00
  • Last Updated - 24-10-2012 07:21
Power outage (Resolved)
  • Priority - Critical
  • Affecting Other - Datacenter
  • We are suffering from a power outage since this morning and it's still ongoing.
    Some customers might be offline, others might now.

    We hope for your understanding and we'll try to resolve it asap.

  • Date - 05-04-2012 11:59 - 06-04-2012 12:19
  • Last Updated - 07-04-2012 19:37
Partial network outage (Resolved)
  • Priority - Critical
  • Affecting Other - Network
  • Since 1:25am you might have been experiencing problems with the network.
    There is an ongoing DDOS since 1:25am and our network team is already busy for several hours to debug it.

    Once we have an update, we will post it here.
    Some people might experience no problems and others might experience irregular time outs.

    Our apologies for the inconvenience caused.

    Update 04:09:


    Our network team is still busy with it. Nothing more we can do from our side but wait on feedback.
    We will update this ticket in the morning once we have a better understanding.

    Update 06:09:

    The linecard on the router crashed and therefore had to be reset. This has been done and everything is up again.
    We are still waiting on feedback from our network team and Brocade whether or not it's fully resolved.

    Update 06:40:

    We see our traffic is restoring so I assume the issues are resolved.
    Still waiting on official feedback from our network team.

    Update 11:27:

    Everything has been stable for the past 5-6 hours so we will close this issue.
    If there are any problems, feel free to contact our support.

  • Date - 02-07-2011 01:25 - 02-07-2011 11:28
  • Last Updated - 02-07-2011 11:28
Packet loss on part of network (Resolved)
  • Priority - Critical
  • Affecting Other - Network
  • Today around 3:45am CET we experienced packet loss on our servers.
    After some investigation it seemed to be several DDOS attacks.

    Around 4:30am CET everything became more stable but the issue was still not fully resolved.
    Our engineers were still investigating the main cause of this issue.
    All servers were still online, around 20% of the servers still experienced packet loss (10-30%).

    Around 8:30am CET the issue was fully resolved.
    We were getting a very big UDP attack but we managed to handle it.

    Any customers still experiencing problems can contact us via our ticket system.

  • Date - 31-03-2011 03:45 - 31-03-2011 08:30
  • Last Updated - 31-03-2011 09:12
Packet loss on dedicated / colo customers (Resolved)
  • Priority - Critical
  • Affecting Other - Network
  • We are currently experiencing packet loss on our network uplinks.
    Our network team is looking into it.

    Update (19:23h CET):

    Network is restored again, still waiting for more info from our network team

    Update (19:38h CET):

    Looks like 50% of the network is still unavailable to some customers, we are still looking into the problem cause.

    Update (20:02h CET):

    All uplinks are restored again. If there are still problems, please mail us or create a ticket! Thanks.
    We are going to send out a mailing about this issue once we have more understanding.

  • Date - 20-03-2011 18:49 - 03-11-2020 20:00
  • Last Updated - 20-03-2011 20:03
Switch 1 down (Resolved)
  • Priority - Critical
  • Affecting Other - Switch1
  • 07:45: We are currently experiencing issues with our main Switch located in Rack #1. Around 20 customers are afected

    Update:

    08:15: Tech has been informed about the issue
    08:30: Tech is on it's way to the datacenter
    09:30: Switch has been reset, no solution
    09:45: Tech is busy logging in to the switch via console to see
    10:12: Spanning tree was blocking traffic coming from Core 1, currently uplink from Core2 is active and all customers are online again.

    This will be investigated further by our tech team.
    For now, RESOLVED.

  • Date - 13-02-2011 10:10 - 13-02-2011 10:18
  • Last Updated - 13-02-2011 10:18
Partly network problem (Resolved)
  • Priority - Critical
  • Affecting System - Network
  • We are currently experiencing a small network problem.
    There are customers who don't notice anything while others can't access the network completely.

    We are doing all we can to find the cause and fix it asap.

    Update 1 (16:30h CET):

    The transit provider 'Abovenet' has been removed from the network today.
    Due to bad preperations from our network provider, some problems have occured with a few IP ranges

    The network team is doing their best to have it resolved asap.

    Update 2 (16:56h CET):

    Several ranges are becoming available again, we also notice a traffic increase on our core switching.
    Not all are working yet but it shouldn't take too long anymore.

    Update 3 (17:07 CET):

    All ranges are working again, case closed.

  • Date - 01-11-2010 14:32
  • Last Updated - 01-11-2010 21:16
Network drop for 1-3 minutes (Resolved)
  • Priority - Medium
  • Affecting Other - Full network
  • This afternoon a 3rd router will be added to the NIKHEF location, therefore we had to disable the Abovenet and Highwinds transit, where Global Crossing would take over. This morning Global Crossing was gone so therefore a part of our network was offline. We had to reload the router to add another route for getting the full network online again.

    We had a total downtime of 1-3 minutes.

  • Date - 11-11-2009 10:05 - 11-11-2009 10:08
  • Last Updated - 11-11-2009 10:28
Switch 5 down - suite 6 (Resolved)
  • Priority - Critical
  • Affecting Other - Servers in suite 6
  • We have noticed that our switch in Suite 6 is no longer responding. A big serie of clients in the 217.148.186.x and 84.244.138.x range might experience problems.

    Update: 04:30

    Technician called

    Update 04:52


    Technician is on its way.

    Update 5:53

    It seems that the power of the entire rack has been gone. Currently investigating.

    Update 6:07

    Power is back again, servers are booting up

    Update 6:12

    Servers are up again with exception of 1 which was the cause of this (malfunctioning power supply). The replacement of this power supply will be done asap.

  • Date - 15-02-2009 04:25 - 15-02-2009 06:32
  • Last Updated - 15-02-2009 11:08