文档简介：
问题分析
mysqld: page allocation failure. order:4, mode:0x10c0d0
mode：内存分配模式的标识，在内核源码文件 include/linux/gfp.h 中定义，通常是多个标识项与运算的结果。
不同版本内核有一定区别。例如。在新版内核中 GFP_KERNEL 是 __GFP_RECLAIM |
__GFP_IO | __GFP_FS 的运算结果，而 __GFP_RECLAIM 是 ___GFP_DIRECT_RECLAIM|___GFP_KSWAPD_RECLAIM 的运算结果。
现象描述
容器启动失败
runc:[1:CHILD]: page allocation failure: order:6, mode:0x10c0d0
Jan 23 14:15:31 dc05 dockerd: time="2019-01-23T14:15:31.288446233+08:00"
level=error msg="containerd: start container" error="oci runtime error:
container_linux.go:247: starting container process caused \"process_linux.
go:245: running exec setns process for init caused \\\"exit status 6\\\"\"\n"
id=5b9be8c5bb121264899fac8d9d36b02150269d41ce96ba6ad36d70b8640cb01c
Jan 23 14:15:31 dc05 dockerd: time="2019-01-23T14:15:31.317965799+08:00"
level=error msg="Create container failed with error: invalid header field
value \"oci runtime error: container_linux.go:247: starting container
process caused \\\"process_linux.go:245: running exec setns process for
init caused \\\\\\\"exit status 6\\\\\\\"\\\"\\n\""
Jan 23 14:15:31 dc05 kubelet: E0123 14:15:31.352386 26037 remote_runtime.
go:91] RunPodSandbox from runtime service failed: rpc error: code = 2 desc
= failed to start sandbox container for pod "matchdataserver-1255064836-t4b2w"
: Error response from daemon: {"message":"invalid header field value \"oci runtime
error: container_linux.go:247: starting container process caused \\\"process_linux.
go:245: running exec setns process for init caused \\\\\\\"exit status 6\\\\\\\"\\\"\\n\""}
Jan 23 14:15:31 dc05 kubelet: E0123 14:15:31.352496 26037 kuberuntime_sandbox.
go:54] CreatePodSandbox for pod "matchdataserver-1255064836-t4b2w_basic
(485fd485-1ed6-11e9-8661-0a587f8021ea)" failed: rpc error: code = 2 desc = failed to
start sandbox container for pod "matchdataserver-1255064836-t4b2w": Error response
from daemon: {"message":"invalid header field value \"oci runtime error: container
_linux.go:247: starting container process caused \\\"process_linux.go:245: running
exec setns process for init caused \\\\\\\"exit status 6\\\\\\\"\\\"\\n\""}
Jan 23 14:15:31 dc05 kubelet: E0123 14:15:31.352518 26037 kuberuntime_manager.
go:618] createPodSandbox for pod "matchdataserver-1255064836-t4b2w_basic
(485fd485-1ed6-11e9-8661-0a587f8021ea)" failed: rpc error: code = 2 desc = failed
to start sandbox container for pod "matchdataserver-1255064836-t4b2w": Error
response from daemon: {"message":"invalid header field value \"oci runtime error:
container_linux.go:247: starting container process caused \\\"process_linux.go:245:
running exec setns process for init caused \\\\\\\"exit status 6\\\\\\\"\\\"\\n\""}
Jan 23 14:15:31 dc05 kubelet: E0123 14:15:31.352580 26037 pod_workers.go:182]
Error syncing pod 485fd485-1ed6-11e9-8661-0a587f8021ea ("matchdataserver-1255064836
-t4b2w_basic(485fd485-1ed6-11e9-8661-0a587f8021ea)"), skipping: failed to
"CreatePodSandbox" for "matchdataserver-1255064836-t4b2w_basic(485fd485-1ed6-
11e9-8661-0a587f8021ea)" with CreatePodSandboxError: "CreatePodSandbox for pod
\"matchdataserver-1255064836-t4b2w_basic(485fd485-1ed6-11e9-8661-0a587f8021ea)\"
failed: rpc error: code = 2 desc = failed to start sandbox container for pod
\"matchdataserver-1255064836-t4b2w\": Error response from daemon: {\"message\":
\"invalid header field value \\\"oci runtime error: container_linux.go:247: starting
container process caused \\\\\\\"process_linux.go:245: running exec setns process
for init caused \\\\\\\\\\\\\\\"exit status 6\\\\\\\\\\\\\\\"\\\\\\\"\\\\n\\\"\"}"
Jan 23 14:15:31 dc05 kubelet: I0123 14:15:31.372181 26037 kubelet.go:1916] SyncLoop
(PLEG): "matchdataserver-1255064836-t4b2w_basic(485fd485-1ed6-11e9-8661-0a587f8021ea)",
event: &pleg.PodLifecycleEvent{ID:"485fd485-1ed6-11e9-8661-0a587f8021ea", Type:
"ContainerDied", Data:"5b9be8c5bb121264899fac8d9d36b02150269d41ce96ba6ad36d70b8640cb01c"}
Jan 23 14:15:31 dc05 kubelet: W0123 14:15:31.372225 26037 pod_container_deletor.go:77]
Container "5b9be8c5bb121264899fac8d9d36b02150269d41ce96ba6ad36d70b8640cb01c"
not found in pod's containers
Jan 23 14:15:31 dc05 kubelet: I0123 14:15:31.678211 26037 kuberuntime_manager.
go:383] No ready sandbox for pod "matchdataserver-1255064836-t4b2w_basic(485fd485
-1ed6-11e9-8661-0a587f8021ea)" can be found. Need to start a new one
$ cat /proc/buddyinfoNode 0, zone DMA 1 0 1 0 2 1 1 0 1 1 3Node 0, zone DMA32 2725 624 489 178 0 0 0 0 0 0 0Node 0, zone Normal 1163 1101 932 222 0 0 0 0 0 0 0
系统 OOM
处理步骤
echo 3 > /proc/sys/vm/drop_caches
