概要: Code generation is a technique that generates program source code without human intervention. There has been much research on automated methods for writing code, such as code generation. However, many techniques are still in their infancy and often generate syntactically incorrect code. Therefore, automated metrics used in natural language processing (NLP) are occasionally used to evaluate existing techniques in code generation. At present, it is unclear which metrics in NLP are more suitable than others for evaluating generated codes. In this study, we clarify which NLP metrics are applicable to syntactically incorrect code and suitable for the evaluation of techniques that automatically generate codes. Our results show that METEOR is the best of the automated metrics compared in this study. タグ: automated, metric, code, generation, deep, learning

@inproceedings{RikuTakaichi2022,
  author = {Riku Takaichi and Yoshiki Higo and Shinsuke Matsumoto and Shinji Kusumoto and Toshiyuki Kurabayashi and Hiroyuki Kirinuki and Haruto Tanno},
  title = {Are NLP Metrics Suitable for Evaluating Generated Code?},
  booktitle = {Proceedings of the 23rd International Conference on Product-Focused Software Process Improvement (PROFES2022)},
  pages = {531--537},
  year = {2022},
  month = {nov}
}