[afnog] CFEB issues on juniper

[ibtisam jamal]

We had a similar issue on one of the internet routers and  the heap
utilization would go up and we kept monitoring and if it got to around 90%
we restarted the router ,as trial and error, upgraded to junos 10.4R5 then
also installed RE-850.

The heap utilization has been at 62% ever since .

It is not clear what solved the issue because we also installed new routers
with RE-850 for internet traffic only and the routes are as below the only
difference is the RE is RE-850..

ISP_VRF.inet.0: 390462 destinations, 780966 routes (390456 active, 0
holddown, 3
             90288 hidden)
+ = Active Route, - = Last Active, * = Both


We do not have RE-850 now and we do not have CFEB-e .


I have also noticed this could it be a contributor to the issue

mpls.0: 124645 destinations, 124645 routes (289 active, 124356 holddown, 0
hidden)
Restart Complete
                MPLS:      3 routes,      3 active
                RSVP: 124635 routes,    279 active
                 VPN:      7 routes,      7 active





On Tue, Feb 7, 2012 at 1:45 PM, Mark Tinka <mtinka at globaltransit.net> wrote:

> On Tuesday, February 07, 2012 06:24:02 PM ibtisam jamal wrote:
>
> > @SR01> show chassis cfeb
> > CFEB status:
> > Slot 0 information:
> >   State                                 Backup
> > Slot 1 information:
> >   State                                 Master
> >   Intake temperature                 25 degrees C / 77 degrees F
> >   Exhaust temperature                31 degrees C / 87 degrees F
> >   CPU utilization                    56 percent
> >   Interrupt utilization               5 percent
> >   Heap utilization                   84 percent
>
> Yep, that'll be your problem.
>
> Your heap utilization is very high, indicating
> that the CFEB is running out of FIB slots, and
> as such, cannot install the routing entries as
> mentioned in the logs.
>
> Unlike some Cisco devices, the Juniper won't
> fall back to software forwarding, so any traffic
> destined for the routes that couldn't get installed
> in the FIB will be blackholed.
>
> > @SR01> show chassis hardware
> > Hardware inventory:
> > Item             Version  Part number  Serial number     Description
> > Chassis                                A3583             M10i
> > Midplane         REV 06   710-008920   DC0193            M10i Midplane
> > Power Supply 0   Rev 04   740-008985   TA50568           DC Power Supply
> > Power Supply 1   Rev 04   740-008985   TB52654           DC Power Supply
> > HCM 0            REV 04   710-010580   DE0720            M10i HCM
> > HCM 1            REV 04   710-010580   DE0820            M10i HCM
> > Routing Engine 0 REV 12   740-009459   1000672632        RE-5.0
> > CFEB 0           N/A      N/A          N/A               Backup
> > CFEB 1           REV 12   750-010465   AAAT6186          Internet
> Processor
> > II
>
> Yep, you also have the standard CFEB's that ship
> with the M7i/M10i. If you want a lot more FIB
> space, you want the Enhanced CFEB, which comes with
> 32MB RLDRAM (as opposed the the standard CFEB which
> ships with only 8MB SSRAM).
>
> These are your options:
>
>        1. Maintain the CFEB you currently have but reduce
>           the number of routes (inet.0, l2vpn, l3vpn, vpls,
>           e.t.c.) that need to be installed in the FIB. Upgrade
>           your standard CFEB to 256MB DRAM though (although
>           that does not help with FIB exhaustion - that's just
>           for packet buffering).
>
>        2. Maintain the CFEB you have now and reboot the
>           router. This is a very short-term fix. If your
>           routing remains the way it is now, chances are
>           you will run into the same issue again sooner
>           than you think. Upgrade your standard CFEB to
>           256MB DRAM (although that does not help with FIB
>           exhaustion - those are just for packet buffering).
>
>        3. Upgrade to the Enhanced CFEB. These days, if
>           you're buying new M7i's or M10i's, the Enhanced
>           CFEB is the only one that is shipped. Otherwise,
>           keeping costs down means pre-owned dealers will
>           more often sell you the standard CFEB.
>
>
> Operators around the world will be seeing more and
> more of these kinds of problems as the Internet routing
> table continues to explode, and older architectures (like
> the standard M7i/M10i CFEB, older Juniper routers, older
> Cisco 6500/7600 supervisor modules like the SUP720-3B,
> SUP32, SUP2, e.t.c.) keep running the network.
>
> Hope this helps.
>
> Cheers,
>
> Mark.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://afnog.org/pipermail/afnog/attachments/20120207/3e926142/attachment.html>