Daichi Haraguchi¹, Naoto Inoue¹, Wataru Shimoda¹, Hayato Mitani², Seiichi Uchida², Kota Yamaguchi^1
1CyberAgent.Inc, ² Kyushu University
Accepted to SIGGRAPH Asia (Technical Communications Track). [Project-page]

This repository contains the data for "Can GPTs Evaluate Graphic Design Based on Design Principles?".

./GPT_eval_data
├── gpt_eval
      ├── gpt_abs_alignment.csv
      ...
      └── gpt_comp_whitespace.csv
├── human_eval_data
      ├── human_abs_alignment.csv
      ...
      └── human_comp_whitespace.csv
└── images.zip

Each CSV file includes the evaluation results. File names consist of "[eval method]_[eval type]_[design principle].csv."
Note:
"eval. method" is "gpt" or "human."
"eval. type" is "abs" (absolute evaluation) or "comp" (comparative (relative) evaluation).
"design principle" is "alignment," "overlap," or "whitespace."

Example:
"gpt_abs_alignment.csv" is the absolute evaluation result by GPT for alignment.

"images.zip" includes images.
The structure in the "images.zip" is below.

images
├── lefttop_large
      ├── [image_ID].png
      ...
      └── [image_ID].png
...
└── org
      ├── [image_ID].png
      ...
      └── [image_ID].png

Note:
The directory name consists of "[perturbation element]_[perturbation size]"
"Org" includes original images without perturbation.
Each image name is an image ID that is written in the CSV files.

CSV for absolute evaluation

CSV for relative evaluation

We provides a notebook for showing results.

• scatter_plot.ipynb shows the scatter plot between human evaluation and GPT evaluation.

@inproceedings{haraguchi2024can,
  title={Can GPTs Evaluate Graphic Design Based on Design Principles?},
  author={Haraguchi, Daichi and Inoue, Naoto and Shimoda, Wataru and Mitani, Hayato and Uchida, Seiichi and Yamaguchi, Kota},
  booktitle={SIGGRAPH Asia 2024 Technical Communications},
  pages={1--4},
  year={2024}
}

This repository is maintained by Daichi Haraguchi.
E-mail: haraguchi_daichi_xa[at]cyberagent.co.jp