As the popularity of video game playing has increased, educators have sought to co-opt video games as an instructional tool. However, educational games have made little impact in either the commercial market place or the average game player’s home library. The modification, or modding, of video games using in-game editors may be used by instructors to introduce educational content to professionally designed video games. This paper examines the effectiveness of a modified video game, Civilization IV, in improving the comprehension and retention of historical knowledge of 10th, 11th, and 12th grade students. Students exposed to the modified video game significantly improved their immediate recall of knowledge level history items. Students expressed interest in future educational game playing and felt that they had a better understanding of application level history items. While these results are encouraging, the small sample size of this experiment prevents generalization and necessitates further study.