[afnog] Another Perspective - Kentik's View on the Facebook Outage

[sm+afrinic at elandsys.com]

Hi Paul,
At 01:50 AM 07-10-2021, Paul McMaster wrote:
>No DNS records were removed. Certain prefix's were withdrawn 
>globally by an automation config mistake. By design their BGP for 
>the DNS prefix's were withdrawn after X amount of time as their core 
>network was not visible.

Yes.

This is the reply from 192.42.93.30 around the time of the outage:

   fb.com.                 172800  IN      NS      a.ns.facebook.com.
   fb.com.                 172800  IN      NS      b.ns.facebook.com.
   fb.com.                 172800  IN      NS      c.ns.facebook.com.
   fb.com.                 172800  IN      NS      d.ns.facebook.com.
   CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN NSEC3 1 1 0 
-   CK0Q1GIN43N1ARRC9OSM6QPQR81H5M9A NS SOA RRSIG DNSKEY NSEC3PARAM
CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN RRSIG NSEC3 8 2 86400 
20211011042403 20211004031403 39343 com. 
Qlm6nJK1+jpkxCOGZKqj121icHf+TcKcPrQgGqA3yYsOiqpB7TjVNnDq 
oZVDtIMuNsUKsAxN2mLlbM8XF8xLf5ZD1Z3H498YrDssqNh1VWeUlJgj 
/9ls1iPe2CsB1PNXzU/cXGdrQiiikkHK//3J0IHSYmsHNAwbC0y4/FsS 
U+ZsZisaa2G5FSDdr+Fqlf6m+jmwoxTEsexS9ivi1SaaDQ==
EM2SGU4GV6DJ5SAH1A2PLQ611POSJBEE.com. 86400 IN NSEC3 1 1 0 - 
EM2TBOBUUOD4AQ2HAJDUQLOREPPGU6DG NS DS RRSIG
EM2SGU4GV6DJ5SAH1A2PLQ611POSJBEE.com. 86400 IN RRSIG NSEC3 8 2 86400 
20211010043214 20211003032214 39343 com. 
rlEjMXkv5n24OweHGYTXgU+RJn2PZeRbJzN+afEukSeYjon8DL5he5wi 
Jj/DskPoR+x1M+tmlyU3vCWoeQF+iFLNoMnpYZ35vulCxqDnqdbc26cP 
ubBZXXQqBywjw+eK0T6a/5E4YXaZBGOhe2C+Ye2BDvYyMEQAoTr6vaf9 
RfBBsFexcRckKaKZuwNFX1rNVNCsmIPn9iYI27U+NAPGhA==
   couldn't get address for 'c.ns.facebook.com': failure

It looked like a DNS issue.  There is an explanation about what 
happened at 
https://engineering.fb.com/2021/10/05/networking-traffic/outage-details/

Regards,
S. Moonesamy