| Tweet | |
| R. Takaichi, Y. Higo, S. Matsumoto, S. Kusumoto, T. Kurabayashi, H. Kirinuki, and H. Tanno, "Are NLP Metrics Suitable for Evaluating Generated Code?," In Proceedings of the 23rd International Conference on Product-Focused Software Process Improvement (PROFES2022), pp. 531-537, November 2022. | |
| ID | 768 |
| 分類 | 国際会議 |
| タグ | automated metric code generation deep learning |
| 表題 (title) |
Are NLP Metrics Suitable for Evaluating Generated Code? |
| 表題 (英文) |
|
| 著者名 (author) |
Riku Takaichi,Yoshiki Higo,Shinsuke Matsumoto,Shinji Kusumoto,Toshiyuki Kurabayashi,Hiroyuki Kirinuki,Haruto Tanno |
| 英文著者名 (author) |
Riku Takaichi,Yoshiki Higo,Shinsuke Matsumoto,Shinji Kusumoto,Toshiyuki Kurabayashi,Hiroyuki Kirinuki,Haruto Tanno |
| 編者名 (editor) |
|
| 編者名 (英文) |
|
| キー (key) |
Riku Takaichi,Yoshiki Higo,Shinsuke Matsumoto,Shinji Kusumoto,Toshiyuki Kurabayashi,Hiroyuki Kirinuki,Haruto Tanno |
| 書籍・会議録表題 (booktitle) |
Proceedings of the 23rd International Conference on Product-Focused Software Process Improvement (PROFES2022) |
| 書籍・会議録表題(英文) |
|
| 巻数 (volume) |
|
| 号数 (number) |
|
| ページ範囲 (pages) |
531-537 |
| 組織名 (organization) |
|
| 出版元 (publisher) |
|
| 出版元 (英文) |
|
| 出版社住所 (address) |
|
| 刊行月 (month) |
11 |
| 出版年 (year) |
2022 |
| 採択率 (acceptance) |
|
| URL |
|
| 付加情報 (note) |
|
| 注釈 (annote) |
|
| 内容梗概 (abstract) |
Code generation is a technique that generates program source code without human intervention. There has been much research on automated methods for writing code, such as code generation. However, many techniques are still in their infancy and often generate syntactically incorrect code. Therefore, automated metrics used in natural language processing (NLP) are occasionally used to evaluate existing techniques in code generation. At present, it is unclear which metrics in NLP are more suitable than others for evaluating generated codes. In this study, we clarify which NLP metrics are applicable to syntactically incorrect code and suitable for the evaluation of techniques that automatically generate codes. Our results show that METEOR is the best of the automated metrics compared in this study. |
| 論文電子ファイル | r-takaic_202211_profes.pdf (application/pdf) [一般閲覧可] |
| BiBTeXエントリ |
@inproceedings{id768,
title = {Are {NLP} Metrics Suitable for Evaluating Generated Code?},
author = {Riku Takaichi and Yoshiki Higo and Shinsuke Matsumoto and Shinji Kusumoto and Toshiyuki Kurabayashi and Hiroyuki Kirinuki and Haruto Tanno},
booktitle = {Proceedings of the 23rd International Conference on Product-Focused Software Process Improvement (PROFES2022)},
pages = {531-537},
month = {11},
year = {2022},
}
|