Abstract

Source code contains a large amount of natural language text, particularly in the form of comments, which makes it an emerging target of text analysis techniques. Due to the mix with program code, it is difficult to process source code comments directly within NLP frameworks such as GATE. Within this work we present an effective means for generating a corpus using information found in source code and in-line documentation, by developing a custom doclet for the Javadoc tool. The generated corpus uses a schema that is easily processed by NLP applications, which allows language engineers to focus their efforts on text analysis tasks, like automatic quality control of source code comments. The SSLDoclet is available as open source software.

Reference

Ninus Khamis, Juergen Rilling, and René Witte. Generating an NLP Corpus from Java Source Code: The SSL Javadoc Doclet. New Challenges for NLP Frameworks, Workshop at LREC 2010, pp.41–45, May 22, 2010, Valletta, Malta.

Bibtex entry (also for download):

@InProceedings{javadoclet2010,
  author = {Ninus Khamis and Juergen Rilling and Ren\'{e} Witte},
  title = {{Generating an NLP Corpus from Java Source Code: The SSL Javadoc Doclet}},
  booktitle = {New Challenges for NLP Frameworks},
  pages =  {41--45},
  year = {2010},
  month = {May 22},
  publisher = {ELRA},
  address = {Valletta, Malta},
}

Software

The software described in this paper is available as open source software.

Download

Our paper only: nlpf2010-ssldoclet.pdf
MD5 checksum: 75a551b0e296df82ba772e493a5c3025
Also available: complete workshop proceedings

Copyright © 2010 Ninus Khamis, Juergen Rilling, and René Witte. All rights reserved.