Virginia Tech® home

Secondary Data Analysis Research

Lessons Learned Doing Secondary Data Analysis in Engineering Education Research

Our initiative explores the challenges and opportunities associated with conducting Secondary Data Analysis (SDA) within the realm of Engineering Education Research (EER).

Whiteboard with notes

By sharing our findings and fostering a community of practice, we strive to enhance the capacity and quality of engineering education research.

What is Secondary Data Analysis?


Secondary Data Analysis (SDA) is a powerful research method that involves utilizing existing data — data that were collected for a different purpose — to answer new research questions or validate previous findings. This approach offers a unique opportunity to maximize the value of existing datasets, allowing researchers to explore new hypotheses, trends, and correlations without the time and financial constraints associated with primary data collection. Collaborative SDA, in which the secondary researchers work with the original authors, offers one key path forward.

  • Cost-Effectiveness: SDA eliminates the need for extensive data collection processes, making it a cost-effective solution for researchers with limited budgets.
  • Time Efficiency: By leveraging existing datasets, researchers can significantly reduce the time from the inception of a research idea to the dissemination of findings.
  • Access to Diverse Data: SDA provides access to a wide range of data, including large-scale datasets that might be difficult or impossible for individual researchers to collect on their own.
  • Opportunity for Longitudinal Analysis: Utilizing data collected over time enables researchers to conduct longitudinal studies, examining trends and changes across different periods.
  • Ethical Considerations: When original data collection poses ethical concerns, such as with vulnerable populations, SDA can offer a viable alternative that respects participants' privacy and consent.

While SDA presents numerous benefits, it also comes with its own set of challenges:

  • Data Relevance: Ensuring the existing data adequately address the new research questions is crucial. Researchers must carefully assess the datasets to ensure compatibility with their research aims.
  • Data Quality and Completeness: The quality and completeness of secondary data can vary, potentially impacting the validity of the research findings. Researchers must critically evaluate the data sources, methodology, and potential biases inherent in the dataset.
  • Ethical and Legal Issues: Navigating the ethical and legal considerations, including issues of consent and data sharing, is essential. Researchers must ensure that their use of secondary data complies with all relevant regulations and ethical guidelines.






Research Learnings & Insights

Lessons learned from Mini Projects.



Ethical Considerations logo

ETHICAL CONSIDERATIONS

  • Research ethics should be at the forefront of any SDA work 

  • The projects we worked with were not initially created for SDA and required significant Institutional (i.e., human subjects) Review Board (IRB) negotiations

  • In the planning stages, researchers could consider

    • Whether the data could and should be available for SDA

    • Defining scope and documentation

    • Participant consent of initial and secondary data usage

Board with information graphic

SHARING CONTEXTUAL INFORMATION

  • Qualitative data is shaped by tacit knowledge

  • We found it important for emerging  scholars to learn the complexity and nuances of the collection and analysis of their data set

  • Trust is key in this process - qualitative data is often highly personal for participants and researchers

Mirror

REFLECTIVE PRACTICE IN SDA

  • Beyond memoing, the projects established structured reflection guidelines to reflect on the research process and learning

  • For undergraduate researchers, engaging in reflective SDA transformed their personal and professional identities

Conference Workshop Discussion Insights

Key issues and questions that surfaced during the National Science Foundation Engineering Education Research Grantees Conference Workshop Discussion

What is our duty to participants?  What does it mean to “do no harm”?  We were able to share the approaches we used in Mini Project 2.

Could this create vulnerability for the new researchers who had collected these original data? 

How and by who should de-identification be conducted?

What happens if researchers not familiar with the context of your project do things with the data that you don’t agree with? Secondary researchers may not have familiarity with the context.

Participants felt that journal reviewers do not seem to like secondary data analysis and this may not be a popular choice for Ph.D. dissertations.

Individuals from all around the world attended the CCE STEM workshop hosted by Virginia Tech in late June. Photo by Niki Hazuda for Virginia Tech.

A group photo taken at the NSF EEC grantees conference.

Resources



AJEE 2023

Publication in the Australasian Journal of Engineering Education.

AJEE 2023 Journal Paper

Paretti, M. C., Case, J. M., Benson, L., Delaine, D. A., Jordan, S., Kajfez, R. L., Lord, S. M., Matusovich, H. M., Young, E. T., & Zastavker, Y. V. (2023). Building capacity in engineering education research through collaborative secondary data analysis. Australasian Journal of Engineering Education, 28(1), 8-16.



ASEE 2023

Resources from the 2023 American Society for Engineering Education conference.

ASEE 2023 Paper

Case, J., Matusovich, H., Paretti, M., Benson, L., Delaine, D., Jordan, S., Kajfez, R., Lord, S., Papp, R., Young, T., & Zastavker, Y. (2023) "Lessons Learned doing Secondary Data Analysis in Engineering Education Research (EER)"

ASEE 2023 Workshop Presentation

Case, J., Matusovich, H., Paretti, M., Benson, L., Delaine, D., Jordan, S., Kajfez, R., Lord, S., Papp, R., Young, T., & Zastavker, Y. (2023) "Sharing qualitative research data for secondary analysis: Why, how, and with whom?"

ASEE 2023 Poster

Case, J., Matusovich, H., Paretti, M., Benson, L., Delaine, D., Jordan, S., Kajfez, R., Lord, S., Papp, R., Young, T., & Zastavker, Y. (2023) "Lessons Learned doing Secondary Data Analysis in Engineering Education Research (EER)"



SEFI 2023

Resources from the 2023 Society for Engineering Education conference.

SEFI 2023 Paper

Case, J. M., Matusovich, H. M., Paretti, M. C., Lord, S. M., & Benson, L. (2023) "Sustaining engineering education research: Sharing qualitative research data for secondary analysis"

SEFI 2023 Workshop Handout

Case, J., Matusovich, H., Paretti, M., & Lord, S. (2023) "Sustaining engineering education research: Sharing qualitative research data for secondary analysis," Workshop presented at the European Society for Engineering Education (SEFI) 2023, Dublin, Ireland, September 11.

SEFI 2023 Workshop Presentation

Case, J., Matusovich, H., Paretti, M., Benson, L., & Lord, S. (2023) "Sustaining engineering education research: Sharing qualitative research data for secondary analysis"



FIE 2023

Resource from the 2023 Frontiers in Education Conference.

FIE 2023 Paper

Richards, K., Goodall, G., Zastavker, Y. V., & Kajfez, R. (2023) "Secondary data analysis as a research and a training tool: First-year engineering experiences"



AISES 2022

Resource from the 2022 American Indian Science and Engineering Society conference.

AISES 2022 Poster

Young, E. T., Papp, R., Delaine, D. A., & Jordan, S. S. (2022) "Engineering for nation building: Piloting a framework for operationalizing tribal sovereignty in engineering education research"







Project Details

Project Goals

This project has drawn together a team of researchers to explore ways to overcome obstacles for conducting secondary data analysis (SDA) in engineering education research (EER). 

  • Sharing data both informally and formally,
  • Putting datasets in the public domain,
  • Creating combined datasets,
  • Performing secondary analyses of both qualitative and quantitative data,
  • Publishing and disseminating these analyses,
  • Securing funding to support this work,
  • Valuing and validating this work within the field.

Year 1: Generative Workshops

  • Workshop 1- Exploring SDA
  • Workshop 2- Generation of Potential Projects
  • Workshop 3- Test Project Launch

Year 2: Dissemination and Reflection

  • Virtual Gathering – September 2022: Interim reports from project teams
  • In-person Writing Retreat – January 2023: Synthesizing individual and collective findings; writing for dissemination
  • Wrap-up – May 2023: Lessons learned, next steps
Participants during research discussion
Three researchers pose for a photo

Mini Projects

Secondary Data Analysis as a Mechanism for New Insights and Future Researcher Preparation
  • Goals: Explore the potential of SDA for training of newer researchers to the field
  • Key Finding: An unanticipated benefit for the undergraduate researchers, who derived personal as well as professional growth from conducting this work. 
Diné Sovereignty
  • Goals: Conduct SDA on a dataset that had involved the participation of marginalized populations, in this case American Indian engineers
  • Key Findings: This project offers significant guidelines for conducting SDA with marginalized populations, and engages deeply with emerging ethical questions, such as those involved when choosing to return to participants for further consent.  The research design of the SDA project was presented at the American Indian Science and Engineering Society (AISES) National Conference

Team Members

Acknowledgements

We gratefully acknowledge the support of the National Science Foundation under Award No. 2039864. The findings and recommendations presented are those of the authors and do not necessarily reflect the views of the NSF.

Contact

For more information, inquiries, or collaboration opportunities, please contact us. We are eager to engage with fellow researchers, educators, and institutions interested in advancing the field of Engineering Education Research.

[1] M. C. Paretti, J. M. Case; L. Benson, D. A. Delaine, S. Jordan, R. L. Kajfez, S. M. Lord, H. M. Matusovich, E. T. Young, Y. V. Zastavker (2023) "Building Capacity in Engineering Education Research Through Collaborative Secondary Data Analysis," Australasian Journal of Engineering Education, DOI: 10.1080/22054952.2023.2214462.

[2] E. T. Young, R. Papp, D. A. Delaine, and S. S. Jordan, "Engineering for nation building: Piloting a framework for operationalizing tribal sovereignty in engineering education research," Poster presentation at the 2022 American Indian Science and Engineering Society National Conference & Exposition, Palm Springs, CA, October 2022.