A data generated framework for the use of research methods in psychological research: a multi-method exploration
Abstract
Literature indicates a need for effective application of research methods by academia in the field of psychology, as lack thereof could influence future researchers, the quality and rigour of results, a country’s contribution to the global knowledge economy, as well as knowledge generation in the field as a whole. One aspect that could promote this effective application of research methods is that of a clear and concise research framework to improve the quality of research. Therefore, the general aim of this research study was to formulate a research framework by investigating what research methods are being used by researchers in the various fields of psychology, how they use these methods, and for which topics these
methods are used. These questions were explored and answered in the form of three articles (see Figure 1, Chapter 1 for a visual representation of objectives: Article 1-3). The first article (Chapter 3) is a systematised review of five general psychology journals. These journals included publications from 2013 to 2017 and were chosen from the SCImago Country and Journal Rank miscellaneous psychology domain. Results (Tables 1-6 [Appendix 1], Figures 1-17 [Appendix 2], Chapter 3) of the 999 articles found were classified
according to predetermined categories, namely: research topic, method, design, sampling, data collection and analysis. Ten themes were identified and categorised to form broad “research topics”. A research topic refers to the research subject and starting point of a research project (Liu 2017). In accordance with Weiten (2010) the following research topics were found: social psychology, cognitive psychology, psychometrics, experimental psychology, physiological psychology, health psychology, developmental psychology, personality, education and learning psychology as well as psychological practice. The authors 9 created the tenth theme, psychological practice, on the basis of the data. The identification of these research topics showed the topic most often researched during the selected five-year period. The results for article 1 were extensive, and are therefore broadly discussed in the article, accompanied by details on the methods used. These are tabulated and visually presented for each individual topic. The results provide insight into the use of research methods, especially with regard to trends in the application of certain methods such as
quantitative methods or the mixing of research methods, without reporting the use of multi or mixed-method designs. This article also, unexpectedly, reveals a severe lack of rigour and transparency in certain aspects of the research process, such as the sampling method and design, for example. Convenience sampling was reported as the most common sampling method, and experimental designs were frequently reported, which concurs with the high number of quantitative studies. The categorisation of data collection showed the high occurrence and creative application of questionnaires and the high frequency of experimental tasks in the chosen sample. All methods of collection and analysis were listed regardless of
their frequency, and this provided keen insight into how methods are being employed in practice. The results from this first article served as data for formulating the research framework in article 3. Article 2, an amended chapter (Chapter 4), explores the use of research methods in the South African Journal of Psychology (SAJP). The same research questions identified for Article 1 were pursued in the South African (SA) context through a systematised review of 116 articles from the South African Journal of Psychology (SAJP) published between 2013 and 2017. This chapter of the study is an amendment to the original proposed research study, which is described in Chapter 2. The inclusion of an article focussing on the SA context provided the authors with the opportunity to compare the results of the two different contexts:
international journals in Article 1 versus national journals in Article 2. The differences in 10 results between the international and national samples highlighted the need for and importance of separating the two samples into two different articles. This comparison also shows the uniqueness and possible role that context plays in shaping research as well as possible areas for development in SA research in comparison with international publications. Results from Article 2 (Tables 1-6, Figures 1-16, Chapter 4) show high frequencies for the use of mostly quantitative and qualitative research methods, with the increase in qualitative methods over the past years observable in previous research for the current sample (see
Macleod & Howell, 2013). This high frequency of qualitative research contrasts with results from Article 1. The authors believe that this is due to specifics of the SA context. The application of these methods concurred with international results, in that a lack of rigour and transparency also existed in the SAJP publications in the reporting of samples and designs. Convenience sampling was the sampling method reported as most common, despite the high frequency of unspecified sampling methods. Cross-sectional designs were indicated as the most commonly occurring quantitative design, with narrative designs the most frequent qualitative design. Questionnaires were also found to be the data collection method of choice in this SAJP sample, followed closely by interviews. Descriptive statistics was the most popular among quantitative studies and thematic analysis was the method of choice for the qualitative studies. The same ten research topics were found in the SAJP as in the international sample. However, these topics occurred at different frequencies. Whereas the most popular research topic was still social psychology – showing the SAJP sample in line with global research trends – the second highest researched topic was psychological practice. This second topic, created from the data, showed a large amount of time spent on the development, experience, and practical considerations of psychology and research. Articles 1 and 2 were not without limitations. Firstly, both articles were based on information stated in the sampled articles, and information could therefore have been lost (for 11 example the applied methodology). Future research should infer the methodologies applied to ensure that all methods are captured. However, by only using what was stated, the lack of rigour in articles was highlighted, which could influence trustworthiness as well as the development of research skills. Secondly, the use of the lesser known research design (systematised review design) could also have created room for error. However, as articles were merely categorised based on reported aspects, and the review followed a clearly defined reviewing process as well as Excel sheets and an online log for data collection, the authors believe this limitation to be small. Thirdly, conclusions for Article 2 were based on a single SA journal; future research should broaden the SA sample. Furthermore, the authors recommend that Articles 1 and 2 be used as a basis for further research regarding the reasons
why certain aspects are not fully reported in publications and the possible impact this may have on the trustworthiness of results. Additionally, further research should also focus on why certain methods and their application are pursued above others, and the effect this has on answering research questions and developing research methodology. In Article 3 (Chapter 5), a research framework was created according to the data from Article 1, which was evaluated based on feedback from a quantitative survey (Table 1, Chapter 5) completed by a set of eight research experts. The framework (named Method Garden) is available online (https://methodgarden.xtrapolate.io/) and results indicated that participants perceived the framework to be generally effective in improving knowledge and insight into research methods (Figures 3-17, Chapter 5). In other words, it successfully assisted researchers in the use of research methods. The use of the framework for student researchers enjoyed particular focus. However, participants did not feel inclined to use the
research framework in their own research, and the reasons for this warrant further investigation. It was concluded that this preliminary framework served as a valuable first step to further development by employing the presented results and improving limitations. 12 Overall, it was concluded that the general aim of investigating the use of research methods in the field of psychological research was achieved through the insight gained into what research methods were being used, how these methods were being used, and for which topics. The following specific conclusions can be made: firstly, research methods and topics are influenced by context, and certain methods and research topics preferred above others indicate research trends. The low frequency of some other topics should be investigated further. Secondly, research methods are applied using interchangeable methodologies, and this may highlight a new era of method application and should be investigated further. Thirdly, a lack of rigorous reporting of methodologies exists in the chosen samples, which contributes knowledge to the replication crisis currently faced by the field of psychology. This lack of rigour was found in both Articles 1 and 2 and is visually presented in Article 3’s
research framework. Fourthly, the research framework was perceived as being potentially useful, by a small sample, in providing research knowledge and insight, especially as a tool for teaching student researchers. It is, however, recommended that article 3 should only be used as a preliminary framework and more data should be collected to refine and broaden the framework. Lastly, from a thesis perspective, the amendments made due to non-response in article 2 were found to be favourable and pragmatic, as it contributed to the feasibility of the research study and provided in-depth insight into the SAJP. Furthermore, future students are encouraged to use the systematised review design in their studies and to attempt to replicate or expand the results of this study. The use of technology for their studies is also encouraged,
as well as a pragmatic approach and constant evaluation of what is being done in order to determine if it would effectively reach the desired outcome.
Collections
- Health Sciences [2060]
Related items
Showing items related by title, author, creator and subject.
-
Recovering a normative stance in accounting research by applying a legal doctrinal research methodology
Coetsee, D. (North-West University, 2019)The thesis identifies the research problem that accounting researchers are not sufficiently contributing to the intellectual foundation of the concepts, principles and rules of accounting developed in practice. The task ... -
Building blocks in Planning research methodology : a roadmap to research design options
Jacobs, Niké; Cornelius, Selna (Sun Media, 2022)Making sense of the myriad of available research design options is generally an arduous task for researchers. Proper research design is, subsequently, often neglected in Planning research, resulting in superficial research ... -
Intrinsic motivation a key enabler of African managers' cultural intelligence
Du Plessis, Yvonne; Barkhuizen, Emmerentia Nicolene (Indiana University South Bend, 2014)This study sought to determine manager's intrinsic motivation to engage multiculturally in the workplace as the managers varied by demographic background. Participants were a convenience sample of managers on supervisory ...