F5 BIG-IP Troubleshooting: The 10-Minute Triage Playbook

"Pool member is down." That is the ticket. You log in, the pool member is fine. Somewhere between the client, the virtual server and the node, something is lying to you.

This is the order we run in class - status icons first, packets last - so you stop guessing and start proving. By the end you will be able to read F5 status icons correctly, verify a virtual server and pool from tmsh, enable monitor debug logging, and capture traffic with the one flag that shows both sides of the proxy.

Step 1 - The status icon already told you the answer

ICON              MEANING                        WHERE TO LOOK
Green circle      Available, monitor passing      Not the node - look higher
Blue square       Unknown - NO monitor assigned   Assign a monitor
Red diamond       Offline - monitor failed        Monitor or the node itself
Black diamond     Disabled by an admin            Someone disabled it manually
Yellow triangle   Currently unavailable           Connection limit reached

Key Concept: A blue square is not "up". It means the BIG-IP has no monitor assigned and is guessing. Traffic still gets load balanced to a dead node, and nothing in the GUI will look red.

Step 2 - Read the virtual server's Reason string

tmsh show ltm virtual VS_WEB_443
tmsh list ltm virtual VS_WEB_443

Look at Availability, State and Reason. If Reason says "The children pool member(s) are down", the virtual server itself is healthy - the failure is one layer below. Go to the pool.

Common Mistake: Jumping straight to the pool without reading the Reason string. It names the failing layer for you. Ten minutes saved, every time.

Step 3 - Pool and members

tmsh show ltm pool POOL_WEB_443 members
tmsh list ltm pool POOL_WEB_443 monitor

Verify: the per-member Reason column names the exact monitor that failed - not "the pool is down", but which health check, on which member.

Step 4 - Make the monitor tell you why

tmsh modify sys db bigd.debug value enable
tail -f /var/log/bigdlog

# ALWAYS turn it back off
tmsh modify sys db bigd.debug value disable

bigd is the daemon that runs your health monitors. With debug enabled it logs every probe and every response, so you see the actual HTTP status or the actual TCP failure instead of a red diamond.

Common Mistake: Leaving bigd.debug enabled after the call ends. It writes continuously and will fill /var on a busy box. Disable it before you close the ticket.

Step 5 - Monitor passes, traffic fails: the return path

This is the single most common production F5 issue, and the one that catches engineers in interviews. If the pool member's default gateway is not the BIG-IP, the server replies directly to the client. The client receives a packet from an IP it never opened a session with, and resets the connection.

The monitor still passes, because monitor traffic is BIG-IP to node on the same subnet - it never uses the return path. Real traffic is client to BIG-IP to node and back. Different path, different result.

tmsh modify ltm virtual VS_WEB_443 \
  source-address-translation { type automap }

tmsh save sys config

Verify: check the server-side flow's source address - after SNAT it must be a self IP, not the client IP.

tmsh show sys connection cs-client-addr 10.10.10.50

Pro Tip: Monitor green but traffic broken is a routing problem nine times out of ten. Before you touch the pool config, ask one question - what is the pool member's default gateway?

Step 6 - The capture that actually helps

tcpdump -ni 0.0:nnnp -s0 -c 2000 -w /var/tmp/vs443.pcap host 10.10.10.50

0.0    all VLANs, all TMMs - not a physical interface
:nnn   high verbosity - flow, peer flow and TMM detail
:p     capture the PEER flow - client side AND server side together
-s0    full packet, no truncation
-c     stop after N packets, so you do not fill /var
-w     write to file for Wireshark

Key Concept: The :p peer flag is what makes an F5 capture different from a Linux capture. The BIG-IP is a full proxy - there are two separate TCP connections. Without :p you are looking at one half and cannot correlate a client request to the server request it produced.

Exam Trap: F5 201 and 301 both ask which interface captures client-side and server-side flows together. The answer is 0.0 with the peer flag - not eth0, and not the individual VLAN name.

Knowledge check

A virtual server shows green. Users report intermittent 502s - not constant, worse at peak hours. All pool members show Available. /var/log/ltm is clean. What do you check next, and why?

Work it out before you scroll. "Intermittent" plus "worse at peak" points at a resource that runs out and then recovers - which rules out static config and points at SNAT port exhaustion. A single automap self IP gives you roughly 64k ephemeral ports per pool member, and a busy virtual server will chew through that at peak and recover as connections age out.

# The smoking gun - this line appears when you run out
grep -i "no snat" /var/log/ltm

# Port usage per translation address
tmsh show ltm snat-translation

# Live connections against this virtual server
tmsh show sys connection cs-server-port 443 | wc -l

The fix is a SNAT pool with several translation addresses instead of a single automap self IP - each address adds another ~64k ports per destination.

Run it on a real box

Reading this order is not the same as running it under pressure. Practise it on a lab box where you have deliberately broken something - wrong monitor port, wrong default gateway, missing SNAT - and time yourself. The F5 BIG-IP LTM Configuration Scenarios lab on techclick.in walks this exact sequence against a pool you have to fix.

- Team Techclick techclick.in | +91 92772 29456 @techclick_in | exam.techclick.com