ChatGPT‑Based Prompt Generator for Stable Diffusion AI Art

Background : The rapid growth of AI‑generated content (AIGC) has led to many innovations in AI painting. Models such as Disco Diffusion, DALL·E 2, Imagen, Stable Diffusion and Midjourney have become popular, but creating high‑quality prompts (Prompt) remains difficult, especially for beginners and for non‑English languages.

Goal : Build a ChatGPT‑driven prompt generator for Stable Diffusion that supports multi‑language prompts, intelligent optimization, and easy deployment, thereby lowering the entry barrier for AI painting.

Implementation Overview

2.1 Stable Diffusion : A latent diffusion model released in 2022, trained on the LAION‑5B subset. It can be used for text‑to‑image, image‑to‑image, inpainting, up‑scaling, etc.

2.1.2 Deployment : The service is containerized with Docker. The simplest deployment steps are:

git clone https://github.com/AbdBarho/stable-diffusion-webui-docker.git

# Enter project directory
cd stable-diffusion-webui-docker

# Download models
docker compose --profile download up --build

# Build and run the service
docker compose --profile auto up --build

The service runs on port 7860; access it via http://{server_ip}:7860/ . The default model is DreamShaper, and additional SD‑v1.5 checkpoints can be placed in stable-diffusion-webui-docker/data/StableDiffusion/ .

Prompt Generator

Stable Diffusion uses positive and negative prompts. Positive prompts describe the desired image; negative prompts filter unwanted elements (e.g., "bad hands", "missing fingers"). The default negative prompt list is:

facelowres,bad anatomy,bad hands,text,error,missing fingers,extra digit,fewer digits,cropped,worst quality,low quality,normal quality,jpeg artifacts,signature,watermark,username,blurry,bad feet,ugly,duplicate,transsexual,hermaphrodite,out of frame,extra fingers,mutated hands,poorly drawn hands,poorly drawn face,mutation,deformed,blurry,bad anatomy,bad proportions,extra limbs,cloned face,disfigured,more than 2 nipples,out of frame,ugly,extra limbs,bad anatomy,gross,worst quality,low quality,normal quality,signature,watermark,username,blurry,proportions,malformed limbs,missing arms,missing legs,extra arms,extra legs,mutated hands,fused fingers,too many fingers,long neck

Prompt Engineering : The generator follows a two‑step workflow – first translate non‑English ideas into English, then generate a concise (≤ 50 words) positive prompt following a structured format. Example task description used in the prompt:

Please perform the following actions:
1. Translate the IDEA into English if it is not in English.
2. Write a detailed prompt exactly about the English IDEA following the rules delimited by triple backticks.
Rules:
```
1. Write a description of the scene, followed by modifiers separated by commas (style, lighting, mood, etc.)
2. Use at most 50 words.
```

Compliance Check : Before returning a prompt, the system checks for prohibited content (pornography, violence, hate speech, etc.). If the idea is inappropriate, the generator returns error code "1004" .

Few‑Shot Prompting : To improve output quality, a few high‑quality examples are provided to ChatGPT. Sample pairs (User → Assistant) include:

"一个弓箭手向森林怪兽射箭。" → "portrait of a Homer Simpson archer shooting arrow at forest monster, front game card, dark, Marvel comics, highly detailed, artstation, digital illustration." "画一个海盗。" → "pirate, concept art, deep focus, fantasy, intricate, highly detailed, digital painting, artstation, matte, sharp focus, illustration." "painting of rachel lane / alison brie, hybrid." → "a fantasy style portrait painting of rachel lane / alison brie hybrid in the style of Francois Boucher oil painting, unreal 5, daz, RPG portrait, extremely detailed, artgerm." "落霞与孤鹜齐飞，秋水共长天一色。" → "sunset with lone wild duck flying together, autumn water and sky merging into one color, traditional Chinese landscape painting, ink wash, Li Bai's poem, serene atmosphere, blooming flowers, tranquil pond, poetic, illustration."

Providing these examples helps ChatGPT learn the desired style and improves reliability.

Effect Demonstration

Several visual comparisons are shown (original vs. optimized prompts) for themes such as "Chinese ancient cyber‑punk mechanical crane", "Van Gogh style Zelda", "Spring poem", and "Castle in the sky". Optimized prompts add detailed modifiers (e.g., "concept art, highly detailed, neon lights, Unreal Engine 5, 4K UHD") resulting in noticeably richer images.

Conclusion & Outlook

The article summarizes the implementation of a ChatGPT‑based prompt generator for Stable Diffusion, highlights best practices in prompt engineering (clear instructions, task decomposition, compliance checking, few‑shot learning), and notes that while future LLMs may reduce the need for extensive prompt engineering, it remains essential today.

References include the original Stable Diffusion paper, the GitHub repository for the Docker deployment, and the "ChatGPT Prompt Engineering for Developers" course.