Building an Automated Minesweeper Bot with Python and OpenCV

The article demonstrates how to build an automated Minesweeper player with Python and OpenCV, achieving a world‑record speed of 0.74 seconds (3BV/s = 60.81).

Before coding, you need a Python 3 environment (preferably Anaconda), the Numpy, Pillow (PIL), opencv‑python, win32gui, win32api libraries, and any Python IDE. The target game is Minesweeper Arbiter, which must be used for the bot.

The implementation follows four main stages: window capture, block segmentation, block recognition, and the Minesweeper solving algorithm.

Window capture : Using Spy++ the main window class is "TMain" and the title is "Minesweeper Arbiter ". The following code obtains the window handle and rectangle, then crops the game board with Pillow’s ImageGrab :

hwnd = win32gui.FindWindow(class_name, title_name)
if hwnd:
    left, top, right, bottom = win32gui.GetWindowRect(hwnd)

left += 15
top += 101
right -= 15
bottom -= 43
rect = (left, top, right, bottom)
img = ImageGrab.grab().crop(rect)

These “magic numbers” were tuned on Windows 10 and may need adjustment on other versions.

Block segmentation : Each Minesweeper cell is 16 × 16 px. The code computes the number of blocks horizontally and vertically, then crops each cell into a 2‑D list:

block_width, block_height = 16, 16
blocks_x = int((right - left) / block_width)
blocks_y = int((bottom - top) / block_height)

def crop_block(hole_img, x, y):
    x1, y1 = x * block_width, y * block_height
    x2, y2 = x1 + block_width, y1 + block_height
    return hole_img.crop((x1, y1, x2, y2))

blocks_img = [[0 for i in range(blocks_y)] for i in range(blocks_x)]
for y in range(blocks_y):
    for x in range(blocks_x):
        blocks_img[x][y] = crop_block(img, x, y)

Block recognition : The centre pixel colour of each cell determines its state. Colours are mapped to numbers 1‑8, mines (9), flags (0), unopened (-1), blank opened (-2), and unknown (-3). Example snippet:

def analyze_block(self, block, location):
    block = imageProcess.pil_to_cv(block)
    block_color = block[8, 8]
    x, y = location
    if self.equal(block_color, self.rgb_to_bgr((192,192,192))):
        if not self.equal(block[8,1], self.rgb_to_bgr((255,255,255))):
            self.blocks_num[x][y] = -2
        else:
            self.blocks_num[x][y] = -1
    elif self.equal(block_color, self.rgb_to_bgr((0,0,255))):
        self.blocks_num[x][y] = 1
    # … other colour checks …
    elif self.equal(block_color, self.rgb_to_bgr((0,0,0))):
        if self.equal(block[6,6], self.rgb_to_bgr((255,255,255))):
            self.blocks_num[x][y] = 9  # mine
        elif self.equal(block[5,8], self.rgb_to_bgr((255,0,0))):
            self.blocks_num[x][y] = 0  # flag
        else:
            self.blocks_num[x][y] = 7
    else:
        self.blocks_num[x][y] = -3

Minesweeper algorithm : The solver iterates over numbered cells and applies three rules:

If the count of unopened neighbours equals the cell’s number, mark all those neighbours as mines.

After marking, click any remaining unopened neighbours that are not flagged as mines.

If no progress can be made, randomly click an unopened cell (a simple fallback).

Kernel generation handles edge cells to avoid out‑of‑bounds accesses:

def generate_kernel(k, k_width, k_height, block_location):
    ls = []
    loc_x, loc_y = block_location
    for now_y in range(k_height):
        for now_x in range(k_width):
            if k[now_y][now_x]:
                rel_x, rel_y = now_x - 1, now_y - 1
                ls.append((loc_y + rel_y, loc_x + rel_x))
    return ls

kernel = [[1,1,1],[1,1,1],[1,1,1]]
# Adjust kernel borders when x or y is at the board edge …

Helper functions count unopened cells, mark mines, and collect safe cells to click:

def count_unopen_blocks(blocks):
    return sum(1 for b in blocks if self.blocks_num[b[1]][b[0]] == -1)

def mark_as_mine(blocks):
    for b in blocks:
        if self.blocks_num[b[1]][b[0]] == -1:
            self.blocks_is_mine[b[1]][b[0]] = 1

def mark_to_click_block(blocks):
    for b in blocks:
        if self.blocks_is_mine[b[1]][b[0]] != 1 and self.blocks_num[b[1]][b[0]] == -1:
            self.next_steps.append((b[1], b[0]))

Finally, the main loop processes the board, updates the mine map, and uses mouseOperation to move the cursor and click the selected cells.

This end‑to‑end solution shows how simple colour‑based computer‑vision combined with rule‑based logic can fully automate a classic Windows game.