Debugging strange application issues (SELinux / Apache)

Today I had to configure a website using Apache, CentOS hardened with SELinux Apache policies. The application was experiencing errors during connecting to a internal proxy.

Using strace (yum install strace), I could check all EACCES (Permission denied) errors.

This command will strace all running apache processes on CentOS:

ps auxw | grep httpd | awk '{print"-p " $2}' | xargs strace -f  

For Ubuntu replace httpd with apache2.

This gave the following output:

[pid 12997] fcntl(11, F_SETFL, O_RDWR|O_NONBLOCK) = 0
[pid 12997] connect(11, {sa_family=AF_INET, sin_port=htons(3128), sin_addr=inet_addr("10.0.3.1")}, 16) = -1 EACCES (Permission denied)
[pid 12997] close(11)                   = 0
[pid 12997] write(2, "[Fri Oct 31 06:55:10.903372 2014"..., 552) = 552
[pid 12997] writev(9, [{"2338\r\n", 6}, {"\" title=\"Add New\"><span class=\"a"..., 4824}, {"\t\t\t\t\t\t\t</div>\n\t\t\t\t\t\t\t\t\t\t\t</div>\n"..., 4192}, {"\r\n", 2}], 4) = 9024
[pid 12997] socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 11
[pid 12997] fcntl(11, F_GETFL)          = 0x2 (flags O_RDWR)
[pid 12997] fcntl(11, F_SETFL, O_RDWR|O_NONBLOCK) = 0
[pid 12997] connect(11, {sa_family=AF_INET, sin_port=htons(3128), sin_addr=inet_addr("10.0.3.1")}, 16) = -1 EACCES (Permission denied)
[pid 12997] close(11)                   = 0

As you can see there it gives an EACCESS error on connect. Ok, so it gets an EACCESS error on connect for Apache only. I checked it using su -s /bin/bash apache and the user itself has all the correct permissions.

Let's show the SELinux policies:

getsebool  -a|grep httpd  
httpd_can_connect_mythtv --> off  
httpd_can_connect_zabbix --> off  
httpd_can_network_connect --> off  
httpd_can_network_connect_cobbler --> off  

Check, so the httpdcannetwork_connect policy is off, causing the EACCESS. Now put it on:

setsebool httpd_can_network_connect on  

Check, now it is working.