Building an AI‑Powered Dou Dizhu Card‑Playing Assistant with YOLOv5 and DouZero

The article introduces a project that replaces a template‑matching card recognizer with a YOLOv5‑based detector to enable AI‑assisted play in the popular Chinese card game Dou Dizhu. It outlines the overall workflow: capture screen regions, identify the AI player, landlord, and opponents, and feed the detected cards into a pretrained DouZero model to obtain optimal move suggestions.

Core Function Design

UI layout using PyQt5 (no redesign needed, just integration of card detection).

Display of three landlord cards, AI hand, opponent hand areas, and win‑rate prediction.

Start/stop controls for the AI assistant.

Implementation Steps

1. UI Design Layout

The existing PyQt5 interface from a previous AI assistant is reused. The essential code for translating UI strings is:

<code>def retranslateUi(self, Form):
    _translate = QtCore.QCoreApplication.translate
    Form.setWindowTitle(_translate("Form", "AI欢乐斗地主"))
    self.WinRate.setText(_translate("Form", "胜率：--%"))
    self.InitCard.setText(_translate("Form", "开始"))
    self.UserHandCards.setText(_translate("Form", "手牌"))
    self.LPlayedCard.setText(_translate("Form", "上家出牌区域"))
    self.RPlayedCard.setText(_translate("Form", "下家出牌区域"))
    self.PredictedCard.setText(_translate("Form", "AI出牌区域"))
    self.ThreeLandlordCards.setText(_translate("Form", "三张底牌"))
    self.Stop.setText(_translate("Form", "停止"))
</code>

2. Hand and Play Data Recognition

Screen regions containing the AI player's hand and the three landlord cards are captured with pyautogui.screenshot and passed to a pretrained YOLOv5 model:

<code>def find_three_landlord_cards(self, pos):
    img = pyautogui.screenshot(region=pos)
    three_landlord_cards_real = detect_cards(img)
    return three_landlord_cards_real

def find_my_cards(self, pos):
    img = pyautogui.screenshot(region=pos)
    user_hand_cards_real = detect_cards(img)
    return user_hand_cards_real
</code>

The generic detection routine loads the screenshot, runs YOLOv5 inference, applies class‑specific post‑processing, and returns a sorted list of card identifiers:

<code>def detect_cards(img):
    path = "datas\cards.png"
    img.save(path)
    raw_cards = detect(source=path)
    replace_cards = [replace_num[i] if i in replace_num else i for i in raw_cards]
    list_cards = sorted(replace_cards, key=lambda x: ranks_value[x])
    cards = "".join(list_cards)
    return cards
</code>

Landlord identification is performed by template matching the landlord flag within predefined screen rectangles:

<code>def find_landlord(self, landlord_flag_pos):
    for pos in landlord_flag_pos:
        result = pyautogui.locateOnScreen('pics/landlord_words.png', region=pos, confidence=self.LandlordFlagConfidence)
        if result is not None:
            return landlord_flag_pos.index(pos)
    return None
</code>

3. AI Move Output

The DouZero open‑source AI model (https://github.com/kwai/DouZero) is loaded, and for each round the assistant predicts the best play. Additionally, a ResNet‑50 classifier distinguishes opponent states (waiting, pass, play) before feeding their cards to YOLOv5 when a play occurs:

<code>labels = ['等待', '出牌', '不出']
model = models.resnet50(pretrained=False)
# modify final layer for 3‑class output
fc_inputs = model.fc.in_features
model.fc = nn.Sequential(
    nn.Linear(fc_inputs, 256),
    nn.ReLU(),
    nn.Dropout(0.4),
    nn.Linear(256, config.NUM_CLASSES),
    nn.LogSoftmax(dim=1)
)
model.load_state_dict(torch.load(pthfile)['model']).eval()
</code>

When the opponent is detected as "出牌", the same YOLOv5 pipeline extracts the played cards.

Usage Guide

Install required Python packages (torch, PyAutoGUI, PyQt5, opencv‑python, rlcard, etc.).

Adjust screen coordinate tuples for hand, opponent zones, landlord flag, and landlord cards.

Run the program, start a Dou Dizhu game, and click the UI's "开始" button to let the AI suggest moves.

After configuration, the assistant displays real‑time predictions, win‑rate estimates, and automatically updates the UI as the game progresses.