Towards discovering problem similarity through deep learning: Combining problem features and user behavior

Automated learning systems allow educators to scale up their efficacy, while personalized systems retain the ability to customize to the individual student. A core issue in developing such adaptive learning systems is to understand how different items (e.g., math exercises) relate to one another, and to exploit this understanding to predict performance on an item. Data-driven approaches aim to discover latent concepts through embeddings that predict similarity between items, typically using only performance data or item data, but not both. While these embeddings are meant to uncover latent concepts (e.g., associativity in mathematics or chemistry), they are better construed as representing topics that reflect the similarity structure in performance or item features. One major difficulty is that embedded concepts may differ only in presentation and not in substance. For example, when learning about numbers, young children struggle with different representational formats (e.g., finger counts, Hindu-Arabic numeral) despite the underlying concept being the same (e.g., "3"). By incorporating item information that allows structured similarity comparison between an item's content and representational format, we can begin to parse out what aspects lead to behavioral differences. Here we develop a deep learning framework for learning concept embeddings that integrates behavioral and item-features to better factorize embeddings into content and presentation. This allows us to fully represent the complexity of the items space, while still extracting scientifically-useful results from the analysis.