[afnog] SQUID_ERROR

[Daniel Shaw]

Hello again,

On Nov 1, 2013, at 13:20, Ali Abdullahi <ali.abdullahi at nils.gov.ng> wrote:

>>> Is the proxy running on Linux or FreeBSD or something else?
> it's running on centOS 6 (Linux)

Cool.

>>> Also please attach the output of ifconfig -have you created and brought up the requisite gre tunnel between proxy and ASA (for WCCP)?
> 
> [root at localhost squid]# ip tunnel
> gre0: gre/ip  remote any  local any  ttl inherit  nopmtudisc
> 
> so I think the gre tunnel is up

Yep. Well, *a* gre tunnel is up :) From the above, we can't verify if it's to/from the proxy machine or not.

> problem is that when the squid service starts one cannot browse

Ok, fairly bad!

> and when I check the cache.log file I notice that while squid is starting, it hooks at a stage which is "storeLateRelease: released 0 objects"

To confirm again though: It's not blocking. This is completion of normal startup. Of course, it's still not operational as you need it. But so far, there is no indication of an issue with Squid.

> 
>>> if you configure a browser manually to use the proxy, rather than relying on WCCP, does it work? 
> Yes it does work fine but when I try to go through WCCP while starting it hooks on a stage which I saw in the cache.log which is "storeLateRelease: released 0 objects" and I dont know why it stays at that satge and does not also work

Ok. So this confirms that squid is 100% ok. If you can use it manually, then the only issue is to do with WCCP from the ASA redirecting browsing transparently to the proxy. Squid can get http requests, proxy them and get back content all ok.

>>> Did it work previously? 
> Yes it worked previously, we even used it for about a month

Ok, good!

> equipments went off and when it came back on and I started the server which squid is installed on, the problem started, that is when ever the squid service starts it hook at the stage which I mentioned earlier and then clients wont be able to browse but if you set your browser to point to the proxy IP it works but 

Quick question: Did the ASA also restart at this time?

I can understand that you maybe don't want to paste real IP addresses into a large list, but what would really help a lot is the outputs of /sbin/ifconfig -a on the proxy server and sh ip wccp summ on the cisco device.

For WCCP to work, you'll find that the ASA or router has a router ID for wccp. In some versions of ios this can be set, but in others defaults to a loopback address (I don't recall if it's the highest or lowest numbered), but you will see something like this:

#sh ip wccp summ
WCCP version 2 enabled, 1 service

Service     Clients   Routers   Assign      Redirect   Bypass
-------     -------   -------   ------      --------   ------
Default routing table (Router Id: 111.222.333.444):
web-cache   1         1         HASH        GRE        GRE


Here 111.222.333.444 is the IP on the cisco device in question.
Note also the 1 and 1 under the Clients and Routers columns. If these are 0, then have an issue with wccp being started at all. They should be 1 or greater when everything is up.

Your gre tunnel needs to have this exact same IP address as it's end-point as it's used as the source address of traffic going from the ASA to the proxy, and if it doesn't match, no traffic will go down the gre tunnel:

Example (from a FreeBSD proxy, but Linux will be similar):
$ ifconfig gre0
gre0: flags=d051<UP,POINTOPOINT,RUNNING,LINK0,LINK2,MULTICAST> metric 0 mtu 1476
	tunnel inet 123.11.22.33 --> 111.222.333.444
	inet 123.11.22.33 --> 127.1.2.3 netmask 0xffffffff

There you can see that the same 111.222.333.444 IP address is used in the tunnel. In this case the 123.11.22.33 represents the IP of the proxy server itself.

If you don't have access to the ASA, then you can also tcpdump on the gre0 interface to see what traffic, if any, is getting to your proxy (and see the source IP too).

Also, did you check the output of iptables -nL ? Have you verified you have the correct DNAT rules in place still?

I'm assuming that your squid.conf would not have changed and will therefore be correct.