We had a similar issue on one of the internet routers and the heap utilization would go up and we kept monitoring and if it got to around 90% we restarted the router ,as trial and error, upgraded to junos 10.4R5 then also installed RE-850.<div>
<br></div><div>The heap utilization has been at 62% ever since .</div><div><br></div><div>It is not clear what solved the issue because we also installed new routers with RE-850 for internet traffic only and the routes are as below the only difference is the RE is RE-850..</div>
<div><br></div><div><div>ISP_VRF.inet.0: 390462 destinations, 780966 routes (390456 active, 0 holddown, 3 90288 hidden)</div><div>+ = Active Route, - = Last Active, * = Both</div>
</div><div><br></div><div><br></div><div>We do not have RE-850 now and we do not have CFEB-e .</div><div><br></div><div><br></div><div>I have also noticed this could it be a contributor to the issue</div><div><br></div><div>
<div>mpls.0: 124645 destinations, 124645 routes (289 active, 124356 holddown, 0 hidden)</div><div>Restart Complete</div><div> MPLS: 3 routes, 3 active</div><div> RSVP: 124635 routes, 279 active</div>
<div> VPN: 7 routes, 7 active</div></div><div><br></div><div><br><div><br></div><div><br><br><div class="gmail_quote">On Tue, Feb 7, 2012 at 1:45 PM, Mark Tinka <span dir="ltr"><<a href="mailto:mtinka@globaltransit.net">mtinka@globaltransit.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">On Tuesday, February 07, 2012 06:24:02 PM ibtisam jamal wrote:<br>
<br>
> @SR01> show chassis cfeb<br>
> CFEB status:<br>
> Slot 0 information:<br>
> State Backup<br>
> Slot 1 information:<br>
> State Master<br>
> Intake temperature 25 degrees C / 77 degrees F<br>
> Exhaust temperature 31 degrees C / 87 degrees F<br>
> CPU utilization 56 percent<br>
> Interrupt utilization 5 percent<br>
> Heap utilization 84 percent<br>
<br>
</div>Yep, that'll be your problem.<br>
<br>
Your heap utilization is very high, indicating<br>
that the CFEB is running out of FIB slots, and<br>
as such, cannot install the routing entries as<br>
mentioned in the logs.<br>
<br>
Unlike some Cisco devices, the Juniper won't<br>
fall back to software forwarding, so any traffic<br>
destined for the routes that couldn't get installed<br>
in the FIB will be blackholed.<br>
<div class="im"><br>
> @SR01> show chassis hardware<br>
> Hardware inventory:<br>
> Item Version Part number Serial number Description<br>
> Chassis A3583 M10i<br>
> Midplane REV 06 710-008920 DC0193 M10i Midplane<br>
> Power Supply 0 Rev 04 740-008985 TA50568 DC Power Supply<br>
> Power Supply 1 Rev 04 740-008985 TB52654 DC Power Supply<br>
> HCM 0 REV 04 710-010580 DE0720 M10i HCM<br>
> HCM 1 REV 04 710-010580 DE0820 M10i HCM<br>
> Routing Engine 0 REV 12 740-009459 1000672632 RE-5.0<br>
> CFEB 0 N/A N/A N/A Backup<br>
> CFEB 1 REV 12 750-010465 AAAT6186 Internet Processor<br>
> II<br>
<br>
</div>Yep, you also have the standard CFEB's that ship<br>
with the M7i/M10i. If you want a lot more FIB<br>
space, you want the Enhanced CFEB, which comes with<br>
32MB RLDRAM (as opposed the the standard CFEB which<br>
ships with only 8MB SSRAM).<br>
<br>
These are your options:<br>
<br>
1. Maintain the CFEB you currently have but reduce<br>
the number of routes (inet.0, l2vpn, l3vpn, vpls,<br>
e.t.c.) that need to be installed in the FIB. Upgrade<br>
your standard CFEB to 256MB DRAM though (although<br>
that does not help with FIB exhaustion - that's just<br>
for packet buffering).<br>
<br>
2. Maintain the CFEB you have now and reboot the<br>
router. This is a very short-term fix. If your<br>
routing remains the way it is now, chances are<br>
you will run into the same issue again sooner<br>
than you think. Upgrade your standard CFEB to<br>
256MB DRAM (although that does not help with FIB<br>
exhaustion - those are just for packet buffering).<br>
<br>
3. Upgrade to the Enhanced CFEB. These days, if<br>
you're buying new M7i's or M10i's, the Enhanced<br>
CFEB is the only one that is shipped. Otherwise,<br>
keeping costs down means pre-owned dealers will<br>
more often sell you the standard CFEB.<br>
<br>
<br>
Operators around the world will be seeing more and<br>
more of these kinds of problems as the Internet routing<br>
table continues to explode, and older architectures (like<br>
the standard M7i/M10i CFEB, older Juniper routers, older<br>
Cisco 6500/7600 supervisor modules like the SUP720-3B,<br>
SUP32, SUP2, e.t.c.) keep running the network.<br>
<br>
Hope this helps.<br>
<br>
Cheers,<br>
<font color="#888888"><br>
Mark.<br>
</font></blockquote></div><br></div></div>