Solved: Now I use wayland-egl and everything is fine.
The Long Story to Get the Simple Solution:
At first, I used raspberrypi-userland for OpenGL ES. However, it didn't work.
I tried VC4 DRM driver in the next. I have struggled to get it work, and I opened an issue on the repository. Then anholt, the author of the driver,
updated the Wiki. By following the Wiki, I finally succeeded in getting it work.
mesa3d is necessary to provide OpenGL ES API, so I installed it with drm backend. However, it didn't work since eglCreatePbufferSurface is stubbed in the backend. So I had to use other QPA.
Next, I tried X11 since Buildroot supports xcb backend, but neither did it work. I thought porting Wayland should be much easier than fixing the issue, and actually it is.