Automating Python Notifications for Model Training, Data Transfer, and Financial Modeling via Email

The author frequently writes Python scripts for data processing, transfer, and model training, which can take considerable time; during these runs, they use automated email notifications to stay informed without constantly monitoring the console.

Three typical time‑consuming tasks are covered: model training, data processing/transfer, and financial modeling. For each, the article describes the information that should be included in update and completion emails, such as loss/accuracy plots, runtime statistics, model name, and sample outputs.

Model Training

Updates are sent every n epochs, containing key metrics (training/validation loss and accuracy), visualizations, model directory, runtime, and sample generated images. Example code for sending an email every 100 epochs is shown:

<code>import notify
START = datetime.now()  # before training begins
MODELNAME = 'SynthwaveGAN'  # model identifier
NOTIFY = 100  # send update every 100 epochs
if e % notify_epoch == 0 and e != 0:
    txt = f"{MODELNAME} update as of {datetime.now().strftime('%H:%M:%S')}."
    msg = notify.message(
        subject='Synthwave GAN',
        text=txt,
        img=[f"/visuals/{MODELNAME}/epoch_{e}_loss.png",
             f"/visuals/{MODELNAME}/epoch_{e}_iter_{i}.png"]
    )
    notify.send(msg)
</code>

Each update email includes loss plots and a sample generated image.

Data Processing and Transfer

Data upload to SQL Server is illustrated, with a final notification once the upload finishes. Example code:

<code>import os
import notify
from data import Sql

dt = Sql('database123', 'server001')
for i, file in enumerate(os.listdir('../data/new')):
    dt.push_raw(f"/data/new/{file}")
msg = notify.message(
    subject='SQL Data Upload',
    text=f"Data upload complete, {i} files uploaded."
)
notify.send(msg)
</code>

A try‑except block can capture upload errors and include them in the notification.

Financial Modeling

Even fast financial models can benefit from summary emails when processing large loan portfolios. The example sends runtime, number of processed loans, and visualizations of key metrics:

<code>end = datetime.datetime.now()  # end time
hours, rem = divmod((end - start).seconds, 3600)
mins, secs = divmod(rem, 60)
runtime = f"{hours:02d}:{mins:02d}:{secs:02d}"
notify.msg(
    subject='Cashflow Model Completion',
    text=f"{len(model.output)} loans processed. Total runtime: {runtime}",
    img=["../vis/loan01_amortisation.png",
         "../vis/loan07_amortisation.png",
         "../vis/loan01_profit_and_loss.png",
         "../vis/loan07_profit_and_loss.png"]
)
notify.send(msg)
</code>

Notify Script and MIME Construction

The notify.py script builds email messages using the email package. It imports email.mime.text.MIMEText , email.mime.image.MIMEImage , email.mime.application.MIMEApplication , and email.mime.multipart.MIMEMultipart . The message function creates a MIMEMultipart object, attaches text, optional images, and optional file attachments, handling both single items and lists.

<code>import os
from email.mime.text import MIMEText
from email.mime.image import MIMEImage
from email.mime.application import MIMEApplication
from email.mime.multipart import MIMEMultipart

def message(subject='PythonNotification', text='', img=None, attachment=None):
    msg = MIMEMultipart()
    msg['Subject'] = subject
    msg.attach(MIMEText(text))
    if img is not None:
        if type(img) is not list:
            img = [img]
        for one_img in img:
            img_data = open(one_img, 'rb').read()
            msg.attach(MIMEImage(img_data, name=os.path.basename(one_img)))
    if attachment is not None:
        if type(attachment) is not list:
            attachment = [attachment]
        for one_attachment in attachment:
            with open(one_attachment, 'rb') as f:
                file = MIMEApplication(f.read(), name=os.path.basename(one_attachment))
            file['Content-Disposition'] = f'attachment; filename="{os.path.basename(one_attachment)}"'
            msg.attach(file)
    return msg
</code>

Example usage creates a message with multiple images and CSV attachments.

<code>email_msg = message(
    text='Model processing complete, please see attached data.',
    img=['accuracy.png', 'loss.png'],
    attachments=['data_in.csv', 'data_out.csv']
)
</code>

SMTP Sending

The smtplib library handles sending the constructed message. The script reads email credentials from files, connects to the Outlook SMTP server (smtp‑mail.outlook.com, port 587), starts TLS, logs in, sends the email, and quits. Errors such as network failures are caught with a try‑except block.

<code>import smtplib
import socket

def send(server='smtp-mail.outlook.com', port=587, msg):
    try:
        smtp = smtplib.SMTP(server, port)
        smtp.ehlo()
        smtp.starttls()
        with open('../data/email.txt', 'r') as fp:
            email = fp.read()
        with open('../data/password.txt', 'r') as fp:
            pwd = fp.read()
        smtp.login(email, pwd)
        smtp.sendmail(email, email, msg.as_string())
        smtp.quit()
    except socket.gaierror:
        print('Network connection error, email not sent.')
</code>

Finally, the two parts (message construction and SMTP sending) are combined to send notifications for any long‑running Python task.

<code># build a message object
msg = message(text='See attached!', img='important.png', attachment='data.csv')
send(msg)  # send the email (defaults to Outlook)
</code>

The article concludes that automated email updates free the user to work on other tasks while long processes run, and reminds readers to use the correct SMTP server address and TLS port for public email providers.