Automatic Math Equation Grading with Python: Data Generation, CNN Training, Image Segmentation, and Result Feedback

The article starts with a motivation to replace discontinued online auto‑grading services by creating a self‑contained solution that can recognize digits and operators, evaluate the expression, and mark correct or incorrect results.

Data preparation : synthetic character images are generated by drawing each digit or operator with various fonts and rotations. Example code:

from PIL import Image, ImageDraw, ImageFont

def makeImage(label_dict, font_path, width=24, height=24, rotate=0):
    img = Image.new("RGB", (width, height), "black")
    draw = ImageDraw.Draw(img)
    font = ImageFont.truetype(font_path, int(width*0.9))
    # draw each character centered
    draw.text((x, y), char, (255,255,255), font)
    img = img.rotate(rotate)
    img.save(f"dataset/{value}/img-{value}_r-{rotate}_{int(time.time()*1000)}.png")

All 15 classes (0‑9, =, +, -, ×, ÷) are rendered in 13 fonts with 20 rotation angles, producing 3 900 images per class.

Model construction and training : a simple CNN is built with TensorFlow/Keras.

def create_model():
    model = Sequential([
        layers.experimental.preprocessing.Rescaling(1./255, input_shape=(24,24,1)),
        layers.Conv2D(24,3,activation='relu'),
        layers.MaxPooling2D((2,2)),
        layers.Conv2D(64,3,activation='relu'),
        layers.MaxPooling2D((2,2)),
        layers.Flatten(),
        layers.Dense(128,activation='relu'),
        layers.Dense(15)
    ])
    model.compile(optimizer='adam', loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=['accuracy'])
    return model

The dataset is loaded with tf.keras.preprocessing.image_dataset_from_directory , trained for 10 epochs, and the weights are saved to checkpoint/char_checkpoint . Training quickly reaches 100 % accuracy.

Prediction on single images uses the trained model to classify each cropped character image:

model = create_model()
model.load_weights('checkpoint/char_checkpoint')
class_name = np.load('class_name.npy')
predicts = model.predict(imgs)
results = [class_name[np.argmax(p)] for p in predicts]

Image segmentation is performed by computing horizontal and vertical projections (shadow) of a binary inverted image, detecting continuous regions, and cropping them:

def img_y_shadow(img_b):
    h,w = img_b.shape
    a = [0]*h
    for i in range(h):
        for j in range(w):
            if img_b[i,j]==255:
                a[i] += 1
    return a

Using the projection arrays, img2rows and row2blocks return bounding boxes [left, top, right, bottom] for each line and each block, which are then cut with NumPy slicing.

Expression evaluation is handled by a small helper that replaces Chinese symbols with Python operators and calls eval :

def calculation(chars):
    expr = ''.join(chars).replace('×','*').replace('÷','/')
    if '=' in expr:
        left,right = expr.split('=')
        try:
            val = int(eval(left))
        except Exception:
            return ''
        return '√' if str(val)==right else '×'
    return int(eval(expr))

Feedback overlay draws the result (✓ in green, ✗ in red, or gray for missing answers) onto the original image using OpenCV and Pillow:

def cv2ImgAddText(img, text, left, top, textColor=(255,0,0), textSize=20):
    if isinstance(img, np.ndarray):
        img = Image.fromarray(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))
    draw = ImageDraw.Draw(img)
    font = ImageFont.truetype('fonts/fangzheng_shusong.ttf', textSize)
    draw.text((left, top), text, textColor, font=font)
    return cv2.cvtColor(np.asarray(img), cv2.COLOR_RGB2BGR)

The final script iterates over all detected blocks, runs the CNN, evaluates the expression, chooses a color, and writes the result back onto the original picture, producing an annotated image that marks each equation as correct (green check), incorrect (red cross), or unanswered (gray).

Additional notes cover required font files, script order ( get_character_pic.py → cnn.py → main.py ), and troubleshooting tips such as matching the font used in the source image.