Automating Minesweeper with Python and OpenCV: A Step‑by‑Step Tutorial

The article begins by listing the required tools: Python 3 (preferably via Anaconda), numpy, Pillow, opencv‑python, and the win32gui/win32api libraries, along with the Minesweeper Arbiter executable for testing.

It then describes how to capture the game window using win32gui.FindWindow and win32gui.GetWindowRect , adjust the rectangle to isolate the board, and grab the image with ImageGrab.grab().crop(rect) .

Next, the tutorial explains block segmentation: each cell is 16×16 px, so the board is divided into a grid, and a crop_block function extracts individual cells into a 2‑D array.

For block recognition, the center pixel’s RGB value is examined; a series of if statements map specific colors to states such as unopened, empty, numbers 1‑8, flags, and mines, storing the results in a numeric matrix.

The solving logic uses a 3×3 kernel to generate neighbor coordinates, counts unopened cells around numbered tiles, and marks cells as mines when the count matches the tile’s number. It then clicks safe cells by converting relative positions to screen coordinates and sending mouse events via win32api.

All functions are encapsulated in modules (e.g., imageProcess.py , mouseOperation.py ), and the main loop iterates over the board, applying the analysis, marking, and clicking steps until the game is solved or a dead‑end is reached.