Diagnosing High Load with Low CPU on Linux: Tools and Tips

1 Common Load Analysis Methods

CPU high, Load high

Use

top

to find the PID of the process consuming the most CPU.

Use

top -Hp PID

to find the thread (TID) that uses the most CPU.

For Java programs, use

jstack

to print thread stack information.

Use

printf %x tid

to display the hexadecimal ID of the most CPU‑intensive thread.

CPU low, Load high

The cause is usually many processes waiting for disk I/O, which lengthens the run‑queue while the CPU remains idle.

Use

top

to view the I/O wait percentage (%wa).

Use

iostat -d -x -m 1 10

to check disk I/O (install with

yum install -y sysstat

if needed).

Use

sar -n DEV 1 10

to monitor network I/O.

Use the following command to locate processes consuming I/O:

<code>ps -e -L h o state,cmd  | awk '{if($1=="R"||$1=="D"){print $0}}' | sort | uniq -c | sort -k 1nr</code>

2 CPU high, Load high Analysis

Use

vmstat

to view system‑wide CPU load.

Use

top

to view per‑process CPU load.

2.1 Using vmstat to view system‑wide CPU load

Command format:

vmstat -n 1 -n 1

(refreshes every second).

<code>[root@VM-1-14-centos ~]# vmstat -n 1
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      0 250304 163472 2154300    0    0     1    16    0    4  1  0 98  0  0
 0  0      0 250412 163472 2154332    0    0     0     0  937 1439  1  1 99  0  0
 ... (output continues) ...</code>

r : number of processes waiting for CPU; larger values indicate slower system response.

b : number of blocked processes.

us : user‑mode CPU time.

sy : system‑mode CPU time (high values suggest heavy I/O).

wa : percentage of CPU time spent waiting for I/O.

id : idle CPU percentage (if consistently 0 while sy is twice us, CPU is likely saturated).

Common issues and solutions:

If

r

frequently >4 and

id

<40, CPU load is heavy.

If

pi

or

po

are non‑zero for long periods, memory is insufficient.

If

disk

is non‑zero and

b

>3, I/O performance is poor.

2.2 Using top to view per‑process CPU load

Top displays overall CPU usage on the third line and detailed per‑process statistics below.

<code>top - 19:49:59 up 36 days, 23:15,  3 users,  load average: 0.11, 0.04, 0.05
Tasks: 133 total,   1 running, 131 sleeping,   0 stopped,   1 zombie
%Cpu(s):  3.1 us,  3.1 sy,  0.0 ni, 93.8 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  3880188 total,   241648 free,  1320424 used,  2318116 buff/cache
KiB Swap:        0 total,        0 free,        0 used.  2209356 avail Mem 
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 1793 mysql     20   0 1608796 236708   9840 S   6.7  6.1  83:36.23 /usr/sbin/mysqld
    1 root      20   0  125636   3920   2444 S   0.0  0.1   4:34.13 /usr/lib/systemd/systemd
    ... (output continues) ...</code>

Press P to sort processes by CPU usage, then examine logs to identify the root cause of high CPU consumption.

3 CPU low, Load high

Problem description : The system shows an idle CPU but a very high load average when no business applications are running.

Analysis : Many processes are in uninterruptible sleep (D state) waiting for disk I/O, inflating the load average. Typical scenarios include:

Scenario 1: Excessive disk read/write requests cause massive I/O wait.

Scenario 2: MySQL queries without indexes or deadlocks lead to I/O blocking.

Scenario 3: External storage failures (e.g., NFS server down) keep I/O requests pending.

Solutions :

Check load average; a high value means a long task queue.

Identify D‑state processes with

ps -axjf

(or the command below).

Understand that D‑state processes cannot be killed; they must be resolved by restoring the required resources or rebooting.

<code>ps -e -L h o state,cmd  | awk '{if($1=="R"||$1=="D"){print $0}}' | sort | uniq -c | sort -k 1nr</code>