analytical research paper methodology

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case NPS+ Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research Research Tools and Apps

Analytical Research: What is it, Importance + Examples

Finding knowledge is a loose translation of the word “research.” It’s a systematic and scientific way of researching a particular subject. As a result, research is a form of scientific investigation that seeks to learn more. Analytical research is one of them.

Any kind of research is a way to learn new things. In this research, data and other pertinent information about a project are assembled; after the information is gathered and assessed, the sources are used to support a notion or prove a hypothesis.

LEARN ABOUT: Causal Research

An individual can successfully draw out minor facts to make more significant conclusions about the subject matter by using critical thinking abilities (a technique of thinking that entails identifying a claim or assumption and determining whether it is accurate or untrue).

Content Index

What is analytical research?

Importance of analytical research, methods of conducting analytical research, examples of analytical research, descriptive vs analytical research.

This particular kind of research calls for using critical thinking abilities and assessing data and information pertinent to the project at hand.

Determines the causal connections between two or more variables. The analytical study aims to identify the causes and mechanisms underlying the trade deficit’s movement throughout a given period.

It is used by various professionals, including psychologists, doctors, and students, to identify the most pertinent material during investigations. One learns crucial information from analytical research that helps them contribute fresh concepts to the work they are producing.

Some researchers perform it to uncover information that supports ongoing research to strengthen the validity of their findings. Other scholars engage in analytical research to generate fresh perspectives on the subject.

Various approaches to performing research include literary analysis, Gap analysis , general public surveys, clinical trials, and meta-analysis.

The goal of analytical research is to develop new ideas that are more believable by combining numerous minute details.

The analytical investigation is what explains why a claim should be trusted. Finding out why something occurs is complex. You need to be able to evaluate information critically and think critically. 

This kind of information aids in proving the validity of a theory or supporting a hypothesis. It assists in recognizing a claim and determining whether it is true.

Analytical kind of research is valuable to many people, including students, psychologists, marketers, and others. It aids in determining which advertising initiatives within a firm perform best. In the meantime, medical research and research design determine how well a particular treatment does.

Thus, analytical research can help people achieve their goals while saving lives and money.

LEARN ABOUT: Theoretical Research

Analytical research is the process of gathering, analyzing, and interpreting information to make inferences and reach conclusions. Depending on the purpose of the research and the data you have access to, you can conduct analytical research using a variety of methods. Here are a few typical approaches:

Quantitative research

Numerical data are gathered and analyzed using this method. Statistical methods are then used to analyze the information, which is often collected using surveys, experiments, or pre-existing datasets. Results from quantitative research can be measured, compared, and generalized numerically.

Qualitative research

In contrast to quantitative research, qualitative research focuses on collecting non-numerical information. It gathers detailed information using techniques like interviews, focus groups, observations, or content research. Understanding social phenomena, exploring experiences, and revealing underlying meanings and motivations are all goals of qualitative research.

LEARN ABOUT: Qualitative Interview

Mixed methods research

This strategy combines quantitative and qualitative methodologies to grasp a research problem thoroughly. Mixed methods research often entails gathering and evaluating both numerical and non-numerical data, integrating the results, and offering a more comprehensive viewpoint on the research issue.

Experimental research

Experimental research is frequently employed in scientific trials and investigations to establish causal links between variables. This approach entails modifying variables in a controlled environment to identify cause-and-effect connections. Researchers randomly divide volunteers into several groups, provide various interventions or treatments, and track the results.

Observational research

With this approach, behaviors or occurrences are observed and methodically recorded without any outside interference or variable manipulation. Both controlled surroundings and naturalistic settings can be used for observational research . It offers useful insights into behaviors that occur in the actual world and enables researchers to explore events as they naturally occur.

Case study research

This approach entails thorough research of a single case or a small group of related cases. Case-control studies frequently include a variety of information sources, including observations, records, and interviews. They offer rich, in-depth insights and are particularly helpful for researching complex phenomena in practical settings.

Secondary data analysis

Examining secondary information is time and money-efficient, enabling researchers to explore new research issues or confirm prior findings. With this approach, researchers examine previously gathered information for a different reason. Information from earlier cohort studies, accessible databases, or corporate documents may be included in this.

Content analysis

Content research is frequently employed in social sciences, media observational studies, and cross-sectional studies. This approach systematically examines the content of texts, including media, speeches, and written documents. Themes, patterns, or keywords are found and categorized by researchers to make inferences about the content.

Depending on your research objectives, the resources at your disposal, and the type of data you wish to analyze, selecting the most appropriate approach or combination of methodologies is crucial to conduct analytical research.

Analytical research takes a unique measurement. Instead, you would consider the causes and changes to the trade imbalance. Detailed statistics and statistical checks help guarantee that the results are significant.

For example, it can look into why the value of the Japanese Yen has decreased. This is so that an analytical study can consider “how” and “why” questions.

Another example is that someone might conduct analytical research to identify a study’s gap. It presents a fresh perspective on your data. Therefore, it aids in supporting or refuting notions.

Here are the key differences between descriptive research and analytical research:

LEARN ABOUT: Descriptive Analysis

The study of cause and effect makes extensive use of analytical research. It benefits from numerous academic disciplines, including marketing, health, and psychology, because it offers more conclusive information for addressing research issues.

QuestionPro offers solutions for every issue and industry, making it more than just survey software. For handling data, we also have systems like our InsightsHub research library.

You may make crucial decisions quickly while using QuestionPro to understand your clients and other study subjects better. Make use of the possibilities of the enterprise-grade research suite right away!

LEARN MORE         FREE TRIAL

MORE LIKE THIS

AI Model: What it is, Types + the Role of Synthetic Data

Sep 12, 2023

That’s Not Two Day Delivery — Tuesday CX Thoughts

Data Science vs Artificial Intelligence: Which is Better?

Sep 11, 2023

Synthetic Dataset: What it is, Benefits + Usage

Other categories.

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Uncategorized
• Video Learning Series
• What’s Coming Up
• Workforce Intelligence

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Advanced Search
• Journal List
• Front Physiol

Meta-Analytic Methodology for Basic Research: A Practical Guide

Nicholas mikolajewicz.

1 Faculty of Dentistry, McGill University, Montreal, QC, Canada

2 Shriners Hospital for Children-Canada, Montreal, QC, Canada

Svetlana V. Komarova

Associated data.

Basic life science literature is rich with information, however methodically quantitative attempts to organize this information are rare. Unlike clinical research, where consolidation efforts are facilitated by systematic review and meta-analysis, the basic sciences seldom use such rigorous quantitative methods. The goal of this study is to present a brief theoretical foundation, computational resources and workflow outline along with a working example for performing systematic or rapid reviews of basic research followed by meta-analysis. Conventional meta-analytic techniques are extended to accommodate methods and practices found in basic research. Emphasis is placed on handling heterogeneity that is inherently prevalent in studies that use diverse experimental designs and models. We introduce MetaLab , a meta-analytic toolbox developed in MATLAB R2016b which implements the methods described in this methodology and is provided for researchers and statisticians at Git repository ( https://github.com/NMikolajewicz/MetaLab ). Through the course of the manuscript, a rapid review of intracellular ATP concentrations in osteoblasts is used as an example to demonstrate workflow, intermediate and final outcomes of basic research meta-analyses. In addition, the features pertaining to larger datasets are illustrated with a systematic review of mechanically-stimulated ATP release kinetics in mammalian cells. We discuss the criteria required to ensure outcome validity, as well as exploratory methods to identify influential experimental and biological factors. Thus, meta-analyses provide informed estimates for biological outcomes and the range of their variability, which are critical for the hypothesis generation and evidence-driven design of translational studies, as well as development of computational models.

Introduction

Evidence-based medical practice aims to consolidate best research evidence with clinical and patient expertise. Systematic reviews and meta-analyses are essential tools for synthesizing evidence needed to inform clinical decision making and policy. Systematic reviews summarize available literature using specific search parameters followed by critical appraisal and logical synthesis of multiple primary studies (Gopalakrishnan and Ganeshkumar, 2013 ). Meta-analysis refers to the statistical analysis of the data from independent primary studies focused on the same question, which aims to generate a quantitative estimate of the studied phenomenon, for example, the effectiveness of the intervention (Gopalakrishnan and Ganeshkumar, 2013 ). In clinical research, systematic reviews and meta-analyses are a critical part of evidence-based medicine. However, in basic science, attempts to evaluate prior literature in such rigorous and quantitative manner are rare, and narrative reviews are prevalent. The goal of this manuscript is to provide a brief theoretical foundation, computational resources and workflow outline for performing a systematic or rapid review followed by a meta-analysis of basic research studies.

Meta-analyses can be a challenging undertaking, requiring tedious screening and statistical understanding. There are several guides available that outline how to undertake a meta-analysis in clinical research (Higgins and Green, 2011 ). Software packages supporting clinical meta-analyses include the Excel plugins MetaXL (Barendregt and Doi, 2009 ) and Mix 2.0 (Bax, 2016 ), Revman (Cochrane Collaboration, 2011 ), Comprehensive Meta-Analysis Software [CMA (Borenstein et al., 2005 )], JASP (JASP Team, 2018 ) and MetaFOR library for R (Viechtbauer, 2010 ). While these packages can be adapted to basic science projects, difficulties may arise due to specific features of basic science studies, such as large and complex datasets and heterogeneity in experimental methodology. To address these limitations, we developed a software package aimed to facilitate meta-analyses of basic research, MetaLab in MATLAB R2016b, with an intuitive graphical interface that permits users with limited statistical and coding background to proceed with a meta-analytic project. We organized MetaLab into six modules ( Figure 1 ), each focused on different stages of the meta-analytic process, including graphical-data extraction, model parameter estimation, quantification and exploration of heterogeneity, data-synthesis, and meta-regression.

General framework of MetaLab . The Data Extraction module assists with graphical data extraction from study figures. Fit Model module applies Monte-Carlo error propagation approach to fit complex datasets to model of interest. Prior to further analysis, reviewers have opportunity to manually curate and consolidate data from all sources. Prepare Data module imports datasets from a spreadsheet into MATLAB in a standardized format. Heterogeneity, Meta-analysis and Meta-regression modules facilitate meta-analytic synthesis of data.

In the present manuscript, we describe each step of the meta-analytic process with emphasis on specific considerations made when conducting a review of basic research. The complete workflow of parameter estimation using MetaLab is demonstrated for evaluation of intracellular ATP content in osteoblasts (OB [ATP] ic dataset) based on a rapid literature review. In addition, the features pertaining to larger datasets are explored with the ATP release kinetics from mechanically-stimulated mammalian cells (ATP release dataset) obtained as a result of a systematic review in our prior work (Mikolajewicz et al., 2018 ).

MetaLab can be freely accessed at Git repository ( https://github.com/NMikolajewicz/MetaLab ), and a detailed documentation of how to use MetaLab together with a working example is available in the Supporting materials .

Validity of Evidence in the Basic Sciences

To evaluate the translational potential of basic research, the validity of evidence must first be assessed, usually by examining the approach taken to collect and evaluate the data. Studies in the basic sciences are broadly grouped as hypothesis-generating and hypothesis-driven. The former tend to be small-sampled proof-of-principle studies and are typically exploratory and less valid than the latter. An argument can even be made that studies that report novel findings fall into this group as well, since their findings remain subject to external validation prior to being accepted by the broader scientific community. Alternatively, hypothesis-driven studies build upon what is known or strongly suggested by earlier work. These studies can also validate prior experimental findings with incremental contributions. Although such studies are often overlooked and even dismissed due to a lack of substantial novelty, their role in external validation of prior work is critical for establishing the translational potential of findings.

Another dimension to the validity of evidence in the basic sciences is the selection of experimental model. The human condition is near-impossible to recapitulate in a laboratory setting, therefore experimental models (e.g., cell lines, primary cells, animal models) are used to mimic the phenomenon of interest, albeit imperfectly. For these reasons, the best quality evidence comes from evaluating the performance of several independent experimental models. This is accomplished through systematic approaches that consolidate evidence from multiple studies, thereby filtering the signal from the noise and allowing for side-by-side comparison. While systematic reviews can be conducted to accomplish a qualitative comparison, meta-analytic approaches employ statistical methods which enable hypothesis generation and testing. When a meta-analysis in the basic sciences is hypothesis-driven, it can be used to evaluate the translational potential of a given outcome and provide recommendations for subsequent translational- and clinical-studies. Alternatively, if meta-analytic hypothesis testing is inconclusive, or exploratory analyses are conducted to examine sources of inconsistency between studies, novel hypotheses can be generated, and subsequently tested experimentally. Figure 2 summarizes this proposed framework.

Schematic of proposed hierarchy of translational potential in basic research.

Steps in Quantitative Literature Review

All meta-analytic efforts prescribe to a similar workflow, outlined as follows:

• Define primary and secondary objectives
• Determine breadth of question
• Construct search strategy: rapid or systematic search
• Screen studies and determine eligibility
• Extract data from relevant studies
• Collect relevant study-level characteristics and experi-mental covariates
• Evaluate quality of studies
• Estimate model parameters for complex relation-ships (optional)
• Compute appropriate outcome measure
• Evaluate extent of between-study inconsistency (heterogeneity)
• Perform relevant data transformations
• Select meta-analytic model
• Pool data and calculate summary measure and confidence interval
• Explore potential sources of heterogeneity (ex. biological or experimental)
• Subgroup and meta-regression analyses
• Interpret findings
• Provide recommendations for future work

Meta-Analysis Methodology

Search and selection strategies.

The first stage of any review involves formulating a primary objective in the form of a research question or hypothesis. Reviewers must explicitly define the objective of the review before starting the project, which serves to reduce the risk of data dredging, where reviewers later assign meaning to significant findings. Secondary objectives may also be defined; however, precaution must be taken as the search strategies formulated for the primary objective may not entirely encompass the body of work required to address the secondary objective. Depending on the purpose of a review, reviewers may choose to undertake a rapid or systematic review. While the meta-analytic methodology is similar for systematic and rapid reviews, the scope of literature assessed tends to be significantly narrower for rapid reviews permitting the project to proceed faster.

Systematic Review and Meta-Analysis

Systematic reviews involve comprehensive search strategies that enable reviewers to identify all relevant studies on a defined topic (DeLuca et al., 2008 ). Meta-analytic methods then permit reviewers to quantitatively appraise and synthesize outcomes across studies to obtain information on statistical significance and relevance. Systematic reviews of basic research data have the potential of producing information-rich databases which allow extensive secondary analysis. To comprehensively examine the pool of available information, search criteria must be sensitive enough not to miss relevant studies. Key terms and concepts that are expressed as synonymous keywords and index terms, such as Medical Subject Headings (MeSH), must be combined using Boolean operators AND, OR and NOT (Ecker and Skelly, 2010 ). Truncations, wildcards, and proximity operators can also help refine a search strategy by including spelling variations and different wordings of the same concept (Ecker and Skelly, 2010 ). Search strategies can be validated using a selection of expected relevant studies. If the search strategy fails to retrieve even one of the selected studies, the search strategy requires further optimization. This process is iterated, updating the search strategy in each iterative step until the search strategy performs at a satisfactory level (Finfgeld-Connett and Johnson, 2013 ). A comprehensive search is expected to return a large number of studies, many of which are not relevant to the topic, commonly resulting in a specificity of <10% (McGowan and Sampson, 2005 ). Therefore, the initial stage of sifting through the library to select relevant studies is time-consuming (may take 6 months to 2 years) and prone to human error. At this stage, it is recommended to include at least two independent reviewers to minimize selection bias and related errors. Nevertheless, systematic reviews have a potential to provide the highest quality quantitative evidence synthesis to directly inform the experimental and computational basic, preclinical and translational studies.

Rapid Review and Meta-Analysis

The goal of the rapid review, as the name implies, is to decrease the time needed to synthesize information. Rapid reviews are a suitable alternative to systematic approaches if reviewers prefer to get a general idea of the state of the field without an extensive time investment. Search strategies are constructed by increasing search specificity, thus reducing the number of irrelevant studies identified by the search at the expense of search comprehensiveness (Haby et al., 2016 ). The strength of a rapid review is in its flexibility to adapt to the needs of the reviewer, resulting in a lack of standardized methodology (Mattivi and Buchberger, 2016 ). Common shortcuts made in rapid reviews are: (i) narrowing search criteria, (ii) imposing date restrictions, (iii) conducting the review with a single reviewer, (iv) omitting expert consultation (i.e., librarian for search strategy development), (v) narrowing language criteria (ex. English only), (vi) foregoing the iterative process of searching and search term selection, (vii) omitting quality checklist criteria and (viii) limiting number of databases searched (Ganann et al., 2010 ). These shortcuts will limit the initial pool of studies returned from the search, thus expediting the selection process, but also potentially resulting in the exclusion of relevant studies and introduction of selection bias. While there is a consensus that rapid reviews do not sacrifice quality, or synthesize misrepresentative results (Haby et al., 2016 ), it is recommended that critical outcomes be later verified by systematic review (Ganann et al., 2010 ). Nevertheless, rapid reviews are a viable alternative when parameters for computational modeling need to be estimated. While systematic and rapid reviews rely on different strategies to select the relevant studies, the statistical methods used to synthesize data from the systematic and rapid review are identical.

Screening and Selection

When the literature search is complete (the date articles were retrieved from the databases needs to be recorded), articles are extracted and stored in a reference manager for screening. Before study screening, the inclusion and exclusion criteria must be defined to ensure consistency in study identification and retrieval, especially when multiple reviewers are involved. The critical steps in screening and selection are (1) removing duplicates, (2) screening for relevant studies by title and abstract, and (3) inspecting full texts to ensure they fulfill the eligibility criteria. There are several reference managers available including Mendeley and Rayyan, specifically developed to assist with screening systematic reviews. However, 98% of authors report using Endnote, Reference Manager or RefWorks to prepare their reviews (Lorenzetti and Ghali, 2013 ). Reference managers often have deduplication functions; however, these can be tedious and error-prone (Kwon et al., 2015 ). A protocol for faster and more reliable de-duplication in Endnote has been recently proposed (Bramer et al., 2016 ). The selection of articles should be sufficiently broad not to be dominated by a single lab or author. In basic research articles, it is common to find data sets that are reused by the same group in multiple studies. Therefore, additional precautions should be taken when deciding to include multiple studies published by a single group. At the end of the search, screening and selection process, the reviewer obtains a complete list of eligible full-text manuscripts. The entire screening and selection process should be reported in a PRISMA diagram, which maps the flow of information throughout the review according to prescribed guidelines published elsewhere (Moher et al., 2009 ). Figure 3 provides a summary of the workflow of search and selection strategies using the OB [ATP] ic rapid review and meta-analysis as an example.

Example of the rapid review literature search. (A) Development of the search parameters to find literature on the intracellular ATP content in osteoblasts. (B) PRISMA diagram for the information flow.

Data Extraction, Initial Appraisal, and Preparation

Identification of parameters to be extracted.

It is advised to predefine analytic strategies before data extraction and analysis. However, the availability of reported effect measures and study designs will often influence this decision. When reviewers aim to estimate the absolute mean difference (absolute effect), normalized mean difference, response ratio or standardized mean difference (ex. Hedges' g), they need to extract study-level means (θ i ), standard deviations ( sd (θ i )), and sample sizes ( n i ), for control (denoted θ i c , s d ( θ i c ) , and n i c ) and intervention (denoted θ i r , s d ( θ i r ) , and n i r ) groups, for studies i . To estimate absolute mean effect, only the mean ( θ i r ), standard deviation ( s d ( θ i r ) ) , and sample size ( n i r ) are required. In basic research, it is common for a single study to present variations of the same observation (ex. measurements of the same entity using different techniques). In such cases, each point may be treated as an individual observation, or common outcomes within a study can be pooled by taking the mean weighted by the sample size. Another consideration is inconsistency between effect size units reported on the absolute scale, for example, protein concentrations can be reported as g/cell, mol/cell, g/g wet tissue or g/g dry tissue. In such cases, conversion to a common representation is required for comparison across studies, for which appropriate experimental parameters and calibrations need to be extracted from the studies. While some parameters can be approximated by reviewers, such as cell-related parameters found in BioNumbers database (Milo et al., 2010 ) and equipment-related parameters presumed from manufacturer manuals, reviewers should exercise caution when making such approximations as they can introduce systematic errors that manifest throughout the analysis. When data conversion is judged to be difficult but negative/basal controls are available, scale-free measures (i.e., normalized, standardized, or ratio effects) can still be used in the meta-analysis without the need to convert effects to common units on the absolute scale. In many cases, reviewers may only be able to decide on a suitable effect size measure after data extraction is complete.

It is regrettably common to encounter unclear or incomplete reporting, especially for the sample sizes and uncertainties. Reviewers may choose to reject studies with such problems due to quality concerns or to employ conservative assumptions to estimate missing data. For example, if it is unclear if a study reports the standard deviation or standard error of the mean, it can be assumed to be a standard error, which provides a more conservative estimate. If a study does not report uncertainties but is deemed important because it focuses on a rare phenomenon, imputation methods have been proposed to estimate uncertainty terms (Chowdhry et al., 2016 ). If a study reports a range of sample sizes, reviewers should extract the lowest value. Strategies to handle missing data should be pre-defined and thoroughly documented.

In addition to identifying relevant primary parameters, a priori defined study-level characteristics that have a potential to influence the outcome, such as species, cell type, specific methodology, should be identified and collected in parallel to data extraction. This information is valuable in subsequent exploratory analyses and can provide insight into influential factors through between-study comparison.

Quality Assessment

Formal quality assessment allows the reviewer to appraise the quality of identified studies and to make informed and methodical decision regarding exclusion of poorly conducted studies. In general, based on initial evaluation of full texts, each study is scored to reflect the study's overall quality and scientific rigor. Several quality-related characteristics have been described (Sena et al., 2007 ), such as: (i) published in peer-reviewed journal, (ii) complete statistical reporting, (iii) randomization of treatment or control, (iv) blinded analysis, (v) sample size calculation prior to the experiment, (vi) investigation of a dose-response relationship, and (vii) statement of compliance with regulatory requirements. We also suggest that the reviewers of basic research studies assess (viii) objective alignment between the study in question and the meta-analytic project. This involves noting if the outcome of interest was the primary study objective or was reported as a supporting or secondary outcome, which may not receive the same experimental rigor and is subject to expectation bias (Sheldrake, 1997 ). Additional quality criteria specific to experimental design may be included at the discretion of the reviewer. Once study scores have been assembled, study-level aggregate quality scores are determined by summing the number of satisfied criteria, and then evaluating how outcome estimates and heterogeneity vary with study quality. Significant variation arising from poorer quality studies may justify study omission in subsequent analysis.

Extraction of Tabular and Graphical Data

The next step is to compile the meta-analytic data set, which reviewers will use in subsequent analysis. For each study, the complete dataset which includes parameters required to estimate the target outcome, study characteristics, as well as data necessary for unit conversion needs to be extracted. Data reporting in basic research are commonly tabular or graphical. Reviewers can accurately extract tabular data from the text or tables. However, graphical data often must be extracted from the graph directly using time consuming and error prone methods. The Data Extraction Module in MetaLab was developed to facilitate systematic and unbiased data extraction; Reviewers provide study figures as inputs, then specify the reference points that are used to calibrate the axes and extract the data ( Figures 4A,B ).

MetaLab data extraction procedure is accurate, unbiased and robust to quality of data presentation. (A,B) Example of graphical data extraction using MetaLab. (A) Original figure (Bodin et al., 1992 ) with axes, data points and corresponding errors marked by reviewer. (B) Extracted data with error terms. (C–F) Validation of MetaLab data-extraction module. (C) Synthetic datasets were constructed using randomly generated data coordinates and marker sizes. (D) Extracted values were consistent with true values evaluated by linear regression with the slope β slope , red line: line of equality. (E) Data extraction was unbiased, evaluated with distribution of percent errors between true and extracted values. E mean , E median , E min , and E max are mean, median, minimum, and maximum % error respectively. (F) The absolute errors of extracted data were independent of data marker size, red line: line regression with the slope β slope .

To validate the performance of the MetaLab Data Extraction Module, we generated figures using 319 synthetic data points plotted with varying markers sizes ( Figure 4C ). Extracted and actual values were correlated ( R 2 = 0.99) with the relationship slope estimated as 1.00 (95% CI: 0.99 to 1.01) ( Figure 4D ). Bias was absent, with a mean percent error of 0.00% (95% CI: −0.02 to 0.02%) ( Figure 4E ). The narrow range of errors between −2.00 and 1.37%, and consistency between the median and mean error indicated no skewness. Data marker size did not contribute to the extraction error, as 0.00% of the variation in absolute error was explained by marker size, and the slope of the relationship between marker size and extraction error was 0.000 (95% CI: −0.001, 0.002) ( Figure 4F ). There data demonstrate that graphical data can be reliably extracted using MetaLab .

Extracting Data From Complex Relationships

Basic science often focuses on natural processes and phenomena characterized by complex relationships between a series of inputs (e.g., exposures) and outputs (e.g., response). The results are commonly explained by an accepted model of the relationship, such as Michaelis-Menten model of enzyme kinetics which involves two parameters–V max for the maximum rate and K m for the substrate concentration half of V max . For meta-analysis, model parameters characterizing complex relationships are of interest as they allow direct comparison of different multi-observational datasets. However, study-level outcomes for complex relationships often (i) lack consistency in reporting, and (ii) lack estimates of uncertainties for model parameters. Therefore, reviewers wishing to perform a meta-analysis of complex relationships may need to fit study-level data to a unified model y = f ( x , β) to estimate parameter set β characterizing the relationship ( Table 1 ), and assess the uncertainty in β.

Commonly used models of complex relationships in basic sciences.

The study-level data can be fitted to a model using conventional fitting methods, in which the model parameter error terms depend on the goodness of fit and number of available observations. Alternatively, a Monte Carlo simulation approach (Cox et al., 2003 ) allows for the propagation of study-level variances (uncertainty in the model inputs) to the uncertainty in the model parameter estimates ( Figure 5 ). Suppose that study i reported a set of k predictor variables x = { x j |1 ≤ j ≤ k } for a set of outcomes θ = {θ j |1 ≤ j ≤ k }, and that there is a corresponding set of standard deviations sd (θ) = { sd (θ j )|1 ≤ j ≤ k } and sample sizes n = { n j |1 ≤ j ≤ k } ( Figure 5A ). The Monte Carlo error propagation method assumes that outcomes are normally distributed, enabling pseudo random observations to be sampled from a distribution approximated by N ( θ j , s d ( θ j ) 2 ) . The pseudo random observations are then averaged to obtain a Monte-Carlo estimate θ j * for each observation such that

Model parameter estimation with Monte-Carlo error propagation method. (A) Study-level data taken from ATP release meta-analysis. (B) Assuming sigmoidal model, parameters were estimated using Fit Model MetaLab module by randomly sampling data from distributions defined by study level data. Model parameters were estimated for each set of sampled data. (C) Final model using parameters estimated from 400 simulations. (D) Distributions of parameters estimated for given dataset are unimodal and symmetrical.

where θ( j , m ) * represents a pseudo-random variable sampled n j times from N ( θ j , s d ( θ j ) 2 ) . The relationship between x and θ * = { θ j * | 1 ≤ j ≤ k } is then fitted with the model of interest using the least-squares method to obtain an estimate of model parameters β ( Figure 5B ). After many iterations of resampling and fitting, a distribution of parameter estimates N ( β ¯ ,   s d ( β ¯ ) 2 ) is obtained, from which the parameter means β ¯ and variances s d ( β ¯ ) 2 can be estimated ( Figures 5C,D ). As the number of iterations M tend to infinity, the parameter estimate converges to the expected value E (β).

It is critical for reviewers to ensure the data is consistent with the model such that the estimated parameters sufficiently capture the information conveyed in the underlying study-level data. In general, reliable model fittings are characterized by normal parameter distributions ( Figure 5D ) and have a high goodness of fit as quantified by R 2 . The advantage of using the Monte-Carlo approach is that it works as a black box procedure that does not require complex error propagation formulas, thus allowing handling of correlated and independent parameters without additional consideration.

Study-Level Effect Sizes

Depending on the purpose of the review product, study-level outcomes θ i can be expressed as one of several effect size measures. The absolute effect size, computed as a mean outcome or absolute difference from baseline, is the simplest, is independent of variance, and retains information about the context of the data (Baguley, 2009 ). However, the use of absolute effect size requires authors to report on a common scale or provide conversion parameters. In cases where a common scale is difficult to establish, a scale-free measure, such as standardized, normalized or relative measures can be used. Standardized mean differences, such Hedges' g or Cohen d, report the outcome as the size of the effect (difference between the means of experimental and control groups) relative to the overall variance (pooled and weighted standard deviation of combined experimental and control groups). The standardized mean difference, in addition to odds or risk ratios, is widely used in meta-analysis of clinical studies (Vesterinen et al., 2014 ), since it allows to summarize metrics that do not have unified meaning (e.g., a pain score), and takes into account the variability in the samples. However, the standardized measure is rarely used in basic science since study outcomes are commonly a defined measure, sample sizes are small, and variances are highly influenced by experimental and biological factors. Other measures that are more suited for basic science are the normalized mean difference, which expresses the difference between the outcome and baseline as a proportion of the baseline (alternatively called the percentage difference), and response ratio, which reports the outcome as a proportion of the baseline. All discussed measures have been included in MetaLab ( Table 2 ).

Types of effect sizes.

Provided are formulas to calculate the mean and standard error for the specified effect sizes .

Data Synthesis

The goal of any meta-analysis is to provide an outcome estimate that is representative of all study-level findings. One important feature of the meta-analysis is its ability to incorporate information about the quality and reliability of the primary studies by weighing larger, better reported studies more heavily. The two quantities of interest are the overall estimate and the measure of the variability in this estimate. Study-level outcomes θ i are synthesized as a weighted mean θ ^ according to the study-level weights w i :

where N is number of studies or datasets. The choice of a weighting scheme dictates how study-level variances are pooled to estimate the variance of the weighted mean. The weighting scheme thus significantly influences the outcome of meta-analysis, and if poorly chosen, potentially risks over-weighing less precise studies and generating a less valid, non-generalizable outcome. Thus, the notion of defining an a priori analysis protocol has to be balanced with the need to assure that the dataset is compatible with the chosen analytic strategy, which may be uncertain prior to data extraction. We provide strategies to compute and compare different study-level and global outcomes and their variances.

Weighting Schemes

To generate valid estimates of cumulative knowledge, studies are weighed according to their reliability. This conceptual framework, however, deteriorates if reported measures of precision are themselves flawed. The most commonly used measure of precision is the inverse variance which is a composite measure of total variance and sample size, such that studies with larger sample sizes and lower experimental errors are more reliable and more heavily weighed. Inverse variance weighting schemes are valid when (i) sampling error is random, (ii) the reported effects are homoscedastic, i.e., have equal variance and (iii) the sample size reflects the number of independent experimental observations. When assumptions (i) or (ii) are violated, sample size weighing can be used as an alternative. Despite sample size and sample variance being such critical parameters in the estimation of the global outcome, they are often prone to deficient reporting practices.

Potential problems with sample variance and sample size

The standard error se (θ i ) is required to compute inverse variance weights, however, primary literature as well as meta-analysis reviewers often confuse standard errors with standard deviations sd (θ i ) (Altman and Bland, 2005 ). Additionally, many assays used in basic research often have uneven error distributions, such that the variance component arising from experimental error depends on the magnitude of the effect (Bittker and Ross, 2016 ). Such uneven error distributions will lead to biased weighing that does not reflect true precision in measurement. Fortunately, the standard error and standard deviation have characteristic properties that can be assessed by the reviewer to determine whether inverse variance weights are appropriate for a given dataset. The study-level standard error se (θ i ) is a measure of precision and is estimated as the product of the sample standard deviation sd (θ i ) and margin of error 1 n i for study i . Therefore, the standard error is expected to be approximately inversely proportionate to the root of the study-level sample size n i

Unlike the standard error, the standard deviation–a measure of the variance of a random variable sd (θ) 2 -is assumed to be independent of the sample size because it is a descriptive statistic rather than a precision statistic. Since the total observed study-level sample variance is the sum of natural variability (assumed to be constant for a phenomenon) and random error, no relationship is expected between reported standard deviations and sample sizes. These assumptions can be tested by correlation analysis and can be used to inform the reviewer about the reliability of the study-level uncertainty measures. For example, a relationship between sample size and sample variance was observed for the OB [ATP] ic dataset ( Figure 6A) , but not for the ATP release data ( Figure 6B ). Therefore, in the case of the OB [ATP] ic data set, lower variances are not associated with higher precision and inverse variance weighting is not appropriate. Sample sizes are also frequently misrepresented in the basic sciences, as experimental replicates and repeated experiments are often reported interchangeably (incorrectly) as sample sizes (Vaux et al., 2012 ). Repeated (independent) experiments refer to number of randomly sampled observations, while replicates refer to the repeated measurement of a sample from one experiment to improve measurement precision. Statistical inference theory assumes random sampling, which is satisfied by independent experiments but not by replicate measurements. Misrepresentative reporting of replicates as the sample size may artificially inflate the reliability of results. While this is difficult to identify, poor reporting may be reflected in the overall quality score of a study.

Assessment of study-level outcomes. (A,B) Reliability of study-level error measures. Relationship between study-level squared standard deviation s d ( θ i ) 2 and sample sizes n i are assumed to be independent when reliably reported. Association between s d ( θ i ) 2 and n i was present in OB [ATP] ic data set (A) and absent in ATP release data set (B) , red line : linear regression. (C,D) Distributions of study-level outcomes. Assessment of unweighted (UW– black ) and weighted (fixed effect; FE– blue , random effects; RE– red , sample-size weighting; N– green ) study-level distributions of data from OB [ATP] ic ( C ) and ATP release ( D ) data sets, before ( left ) and after log 10 transformation ( right ). Heterogeneity was quantified by Q, I 2 , and H 2 heterogeneity statistics. (E,F ) After log 10 transformation, H 2 heterogeneity statistics increased for OB [ATP] ic data set ( E ) and decreased for ATP release ( F ) data set.

Inverse variance weighting

The inverse variance is the most common measure of precision, representing a composite measure of total variance and sample size. Widely used weighting schemes based on the inverse variance are fixed effect or random effects meta-analytic models. The fixed effect model assumes that all the studies sample one true effect γ. The observed outcome θ i for study i is then a function of a within-study error ε i , θ i = γ + ε i , where ε i is normally distributed ε i ~ N ( 0 , s e ( θ i ) 2 ) . The standard error se (θ i ) is calculated from the sample standard deviation sd (θ i ) and sample size n i as:

Alternatively, the random effects model supposes that each study samples a different true outcome μ i , such that the combined effect μ is the mean of a population of true effects. The observed effect θ i for study i is then influenced by the intrastudy error ε i and interstudy error ξ i , θ i = μ i + ε i + ξ i , where ξ i is also assumed to be normally distributed ξ i ~ N (0, τ 2 ), with τ 2 representing the extent of heterogeneity, or between-study (interstudy) variance.

Study-level estimates for a fixed effect or random effects model are weighted using the inverse variance:

These weights are used to calculate the global outcome θ ^ (Equation 3) and the corresponding standard error s e ( θ ^ ) :

where N = number of datasets/studies. In practice, random effects models are favored over the fixed effect model, due to the prevalence of heterogeneity in experimental methods and biological outcomes. However, when there is no between-study variability (τ 2 = 0), the random effects model reduces to a fixed effect model. In contrast, when τ 2 is exceedingly large and interstudy variance dominates the weighting term [ τ 2 ≫ s e ( θ i ) 2 ] , random effects estimates will tend to an unweighted mean.

Interstudy variance τ 2 estimators . Under the assumptions of a random effects model, the total variance is the sum of the intrastudy variance (experimental sampling error) and interstudy variance τ 2 (variability of true effects). Since the distribution of true effects is unknown, we must estimate the value of τ 2 based on study-level outcomes (Borenstein, 2009 ). The DerSimonian and Laird (DL) method is the most commonly used in meta-analyses (DerSimonian and Laird, 1986 ). Other estimators such as the Hunter and Schmidt (Hunter and Schmidt, 2004 ), Hedges (Hedges and Olkin, 1985 ), Hartung-Makambi (Hartung and Makambi, 2002 ), Sidik-Jonkman (Sidik and Jonkman, 2005 ), and Paule-Mandel (Paule and Mandel, 1982 ) estimators have been proposed as either alternatives or improvements over the DL estimator (Sanchez-Meca and Marin-Martinez, 2008 ) and have been implemented in MetaLab ( Table 3) . Negative values of τ 2 are truncated at zero. An overview of the various τ 2 estimators along with recommendations on their use can be found elsewhere (Veroniki et al., 2016 ).

Interstudy variance estimators.

N = number of datasets/studies .

Sample-size weighting

Sample-size weighting is preferred in cases where variance estimates are unavailable or unreliable. Under this weighting scheme, study-level sample sizes are used in place of inverse variances as weights. The sampling error is then unaccounted for; however, since sampling error is random, larger sample sizes will effectively average out the error and produce more dependable results. This is contingent on reliable reporting of sample sizes which is difficult to assess and can be erroneous as detailed above. For a sample size weighted estimate, study-level sample sizes n i replace weights that are used to calculate the global effect size θ ^ , such that

The pooled standard error s e ( θ ^ ) for the global effect is then:

While sample size weighting is less affected by sampling variance, the performance of this estimator depends on the availability of studies (Marin-Martinez and Sanchez-Meca, 2010 ). When variances are reliably reported, sample-size weights should roughly correlate to inverse variance weights under the fixed effect model.

Meta-Analytic Data Distributions

One important consideration the reviewer should attend to is the normality of the study-level effects distributions assumed by most meta-analytic methods. Non-parametric methods that do not assume normality are available but are more computationally intensive and inaccessible to non-statisticians (Karabatsos et al., 2015 ). The performance of parametric meta-analytic methods has been shown to be robust to non-normally distributed effects (Kontopantelis and Reeves, 2012 ). However, this robustness is achieved by deriving artificially high estimates of heterogeneity for non-normally distributed data, resulting in conservatively wide confidence intervals and severely underpowered results (Jackson and Turner, 2017 ). Therefore, it is prudent to characterize the underlying distribution of study-level effects and perform transformations to normalize distributions to preserve the inferential integrity of the meta-analysis.

Assessing data distributions

Graphical approaches, such as the histogram, are commonly used to assess the distribution of data; however, in a meta-analysis, they can misrepresent the true distribution of effect sizes that may be different due to unequal weights assigned to each study. To address this, we can use a weighted histogram to evaluate effect size distributions ( Figure 6 ). A weighted histogram can be constructed by first binning studies according to their effect sizes. Each bin is then assigned weighted frequencies, calculated as the sum of study-level weights within the given bin. The sum of weights in each bin are then normalized by the sum of all weights across all bins

where P j is the weighted frequency for bin j , w ij is the weight for the effect size in bin j from study i , and nBins is the total number of bins. If the distribution is found deviate from normality, the most common explanations are that (i) the distribution is skewed due to inconsistencies between studies, (ii) subpopulations exist within the dataset giving rise to multimodal distributions or (iii) the studied phenomenon is not normally distributed. The source of inconsistencies and multimodality can be explored during the analysis of heterogeneity (i.e., to determine whether study-level characteristics can explain observed discrepancies). Skewness may however be inherent to the data when values are small, variances are large, and values cannot be negative (Limpert et al., 2001 ) and has been credited to be characteristic of natural processes (Grönholm and Annila, 2007 ). For sufficiently large sample sizes the central limit theorem holds that the means of a skewed data are approximately normally distributed. However, due to common limitation in the number of studies available for meta-analyses, meta-analytic global estimates of skewed distributions are often sensitive to extreme values. In these cases, data transformation can be used to achieve a normal distribution on the logarithmic scale (i.e., lognormal distribution).

Lognormal distributions

Since meta-analytic methods typically assume normality, the log transformation is a useful tool used to normalize skewed distributions ( Figures 6C–F ). In the ATP release dataset, we found that log transformation normalized the data distribution. However, in the case of the OB [ATP] ic dataset, log transformation revealed a bimodal distribution that was otherwise not obvious on the raw scale.

Data normalization by log transformation allows meta-analytic techniques to maintain their inferential properties. The outcomes synthesized on the logarithmic scale can then be transformed to the original raw scale to obtain asymmetrical confidence intervals which further accommodate the skew in the data. Study-level effect sizes θ i can be related to the logarithmic mean Θ i through the forward log transformation, meta-analyzed on the logarithmic scale, and back-transformed to the original scale using one of the back-transformation methods ( Table 4 ). We have implemented three different back-transformation methods into MetaLab, including geometric approximation (anti-log), naïve approximation (rearrangement of forward-transformation method) and tailor series approximation (Higgins et al., 2008 ). The geometric back-transformation will yield an estimate of θ ^ that is approximately equal to the median of the study-level effects. The naïve or tailor series approximation differ in how the standard errors are approximated, which is used to obtain a point estimate on the original raw scale. The naïve and tailor series approximations were shown to maintain adequate inferential properties in the meta-analytic context (Higgins et al., 2008 ).

Logarithmic Transformation Methods.

Forward-transformation of study-level estimates θ i to corresponding log-transformed estimates Θ i , and back-transformation of meta-analysis outcome Θ ^ to the corresponding outcome θ ^ on the raw scale (Higgins et al., 2008 ). v 1−α/2 : confidence interval critical value at significance level α .

Confidence Intervals

Once the meta-analysis global estimate and standard error has been computed, reviewers may proceed to construct the confidence intervals (CI). The CI represents the range of values within which the true mean outcome is contained with the probability of 1-α. In meta-analyses, the CI conveys information about the significance, magnitude and direction of an effect, and is used for inference and generalization of an outcome. Values that do not fall in the range of the CI may be interpreted as significantly different. In general, the CI is computed as the product of the standard error s e ( θ ^ )   and the critical value v 1−α/2 :

CI estimators

The critical value v 1−α/2 is derived from a theoretical distribution and represents the significance threshold for level α. A theoretical distribution describes the probability of any given possible outcome occurrence for a phenomenon. Extreme outcomes that lie furthest from the mean are known as the tails. The most commonly used theoretical distributions are the z-distribution and t -distribution, which are both symmetrical and bell-shaped, but differ in how far reaching or “heavy” the tails are. Heavier tails will result in larger critical values which translate to wider confidence intervals, and vice versa. Critical values drawn from a z-distribution, known as z-scores ( z ), are used when data are normal, and a sufficiently large number of studies are available (>30). The tails of a z-distribution are independent of the sample size and reflect those expected for a normal distribution. Critical values drawn from a t-distribution, known as t-scores (t), also assume data are normally-distributed, however, are used when there are fewer available studies (<30) because the t-distribution tails are heavier. This produces more conservative (wider) CIs, which help ensure that the data are not misleading or misrepresentative when there is limited evidence available. The heaviness of the t-distribution tails is dictated by the degree of freedom df , which is related to the number of available studies N ( df = N − 1 ) such that fewer studies will result in heavier t-distribution tails and therefore larger critical values. Importantly, the t-distribution is asymptotically normal and will thus converge to a z-distribution for a sufficiently large number of studies, resulting in similar critical values. For example, for a significance level α = 0.05 (5% false positive rate), the z-distribution will always yield a critical value v = 1.96, regardless of how many studies are available. The t-distribution will however yield v = 2.78 for 5 studies, v = 2.26 for 10 studies, v = 2.05 for 30 studies and v = 1.98 for 100 studies, gradually converging to 1.96 as the number of studies increases. We have implemented the z-distribution and t-distribution CI estimators into MetaLab.

Evaluating Meta-Analysis Performance

In general, 95% of study-level outcomes are expected to fall within the range of the 95% global CI. To determine whether the global 95% CI is consistent with the underlying study-level outcomes, the coverage of the CI can be computed as the proportion of study-level 95% CIs that overlap with the global 95% CI:

The coverage is a performance measure used to determine whether inference made on the study-level is consistent with inference made on the meta-analytic level. Coverage that is less than expected for a specified significance level (i.e., <95% coverage for α = 0.05) may be indicative of inaccurate estimators, excessive heterogeneity or inadequate choice of meta-analytic model, while coverage exceeding 95% may indicate an inefficient estimator that results in insufficient statistical power.

Overall, the performance of a meta-analysis is heavily influenced by the choice of weighting scheme and data transformation ( Figure 7 ). This is especially evident in the smaller datasets, such as our OB [ATP] i example, where both the global estimates and the confidence intervals are dramatically different under different weighting schemes ( Figure 7A ). Working with larger datasets, such as ATP release kinetics, allows to somewhat reduce the influence of the assumed model ( Figure 7B ). However, normalizing data distribution (by log transformation) produces much more consistent outcomes under different weighting schemes for both datasets, regardless of the number of available studies ( Figures 7A,B , log 10 synthesis ).

Comparison of global effect estimates using different weighting schemes. (A,B) Global effect estimates for OB [ATP] ic (A) and ATP release (B) following synthesis of original data (raw, black ) or of log 10 -transformed data followed by back-transformation to original scale (log 10 , gray ). Global effects ± 95% CI were obtained with unweighted data (UW), or using fixed effect (FE), random effects (RE), and sample-size ( n ) weighting schemes.

Analysis of Heterogeneity

Heterogeneity refers to inconsistency between studies. A large part of conducting a meta-analysis involves quantifying and accounting for sources of heterogeneity that may compromise the validity of meta-analysis. Basic research meta-analytic datasets are expected to be heterogeneous because ( i ) basic research literature searches tend to retrieve more studies than clinical literature searches and ( ii ) experimental methodologies used in basic research are more diverse and less standardized compared to clinical research. The presence of heterogeneity may limit the generalizability of an outcome due to the lack of study-level consensus. Nonetheless, exploration of heterogeneity sources can be insightful for the field in general, as it can identify biological or methodological factors that influence the outcome.

Quantifying of Heterogeneity

Higgins and Thompson emphasized that a heterogeneity metric should be (i) dependent on magnitude of heterogeneity, (ii) independent of measurement scale, (iii) independent of sample size and (iv) easily interpretable (Higgins and Thompson, 2002 ). Regrettably, the most commonly used test of heterogeneity is the Cochrane's Q test (Borenstein, 2009 ), which has been repeatedly shown to have undesirable statistical properties (Higgins et al., 2003 ). Nonetheless, we will introduce it here, not because of its widespread use, but because it is an intermediary statistic used to obtain more useful measures of heterogeneity, H 2 and I 2 . The measure of total variation Q total statistic is calculated as the sum of the weighted squared differences between the study-level means θ i and the fixed effect estimate θ ^ F E :

The Q total statistic is compared to a chi-square (χ 2 ) distribution ( df = N-1 ) to obtain a p -value, which, if significant, supports the presence of heterogeneity. However, the Q -test has been shown to be inadequately powered when the number of studies is too low ( N < 10) and excessively powered when study number is too high (N > 50) (Gavaghan et al., 2000 ; Higgins et al., 2003 ). Additionally, the Q total statistic is not a measure of the magnitude of heterogeneity due to its inherent dependence on the number of studies. To address this limitation, H 2 heterogeneity statistics was developed as the relative excess in Q total over degrees of freedom df :

H 2 is independent of the number of studies in the meta-analysis and is indicative of the magnitude of heterogeneity (Higgins and Thompson, 2002 ). For values <1, H 2 is truncated at 1, therefore values of H 2 can range from one to infinity, where H 2 = 1 indicates homogeneity. The corresponding confidence intervals for H 2 are

Intervals that do not overlap with 1 indicate significant heterogeneity. A more easily interpretable measure of heterogeneity is the I 2 statistic, which is a transformation of H 2 :

The corresponding 95% CI for I 2 is derived from the 95% CI for H 2

Values of I 2 range between 0 and 100% and describe the percentage of total variation that is attributed to heterogeneity. Like H 2 , I 2 provides a measure of the magnitude of heterogeneity. Values of I 2 at 25, 50, and 75% are generally graded as low, moderate and high heterogeneity, respectively (Higgins and Thompson, 2002 ; Pathak et al., 2017 ). However, several limitations have been noted for the I 2 statistic. I 2 has a non-linear dependence on τ 2 , thus I 2 will appear to saturate as it approaches 100% (Huedo-Medina et al., 2006 ). In cases of excessive heterogeneity, if heterogeneity is partially explained through subgroup analysis or meta-regression, residual unexplained heterogeneity may still be sufficient to maintain I 2 near saturation. Therefore, I 2 will fail to convey the decline in overall heterogeneity, while H 2 statistic that has no upper limit will allow to track changes in heterogeneity more meaningfully. In addition, a small number of studies (<10) will bias I 2 estimates, contributing to uncertainties inevitable associated with small meta-analyses (von Hippel, 2015 ). Of the three heterogeneity statistics Q total , H 2 and I 2 described, we recommend that H 2 is used as it best satisfies the criteria for a heterogeneity statistic defined by Higgins and Thompson ( 2002 ).

Identifying bias

Bias refers to distortions in the data that may result in misleading meta-analytic outcomes. In the presence of bias, meta-analysis outcomes are often contradicted by higher quality large sample-sized studies (Egger et al., 1997 ), thereby compromising the validity of the meta-analytic study. Sources of observed bias include publication bias, methodological inconsistencies and quality, data irregularities due to poor quality design, inadequate analysis or fraud, and availability or selection bias (Egger et al., 1997 ; Ahmed et al., 2012 ). At the level of study identification and inclusion for meta-analysis, systematic searches are preferred over rapid review search strategies, as narrow search strategies may omit relevant studies. Withholding negative results is also a common source of publication bias, which is further exacerbated by the small-study effect (the phenomenon by which smaller studies produce results with larger effect sizes than larger studies) (Schwarzer et al., 2015 ). By extension, smaller studies that produce negative results are more likely to not be published compared to larger studies that produce negative results. Identifying all sources of bias is unfeasible, however, tools are available to estimate the extent of bias present.

Funnel plots . Funnel plots have been widely used to assess the risk of bias and examine meta-analysis validity (Light and Pillemer, 1984 ; Borenstein, 2009 ). The logic underlying the funnel plot is that in the absence of bias, studies are symmetrically distributed around the fixed effect size estimate, due to sampling error being random. Moreover, precise study-level estimates are expected to be more consistent with the global effect size than less precise studies, where precision is inversely related to the study-level standard error. Thus, for an unbiased set of studies, study-level effects θ i plotted in relation to the inverse standard error 1/ se (θ i ) will produce a funnel shaped plot. Theoretical 95% CIs for the range of plotted standard errors are included as reference to visualize the expected distribution of studies in the absence of bias (Sterne and Harbord, 2004 ). When bias is present, study-level effects will be asymmetrically distributed around the global fixed-effect estimate. In the past, funnel plot asymmetries have been attributed solely to publication bias, however they should be interpreted more broadly as a general presence of bias or heterogeneity (Sterne et al., 2011 ). It should be noted that rapid reviews ( Figure 8A , left ) are far more subject to bias than systematic reviews ( Figure 8A , right ), due to the increased likelihood of relevant study omission.

Analysis of heterogeneity and identification of influential studies. (A) Bias and heterogeneity in OB [ATP] ic ( left ) and ATP release ( right ) data sets were assessed with funnel plots. Log 10 -transformed study-level effect sizes (black markers) were plotted in relation to their precision assessed as inverse of standard error (1/SE). Blue dashed line : fixed effect estimate, red dashed line : random effects estimate, gray lines : Expected 95% confidence interval (95% CI) in the absence of bias/heterogeneity. (B) OB [ATP] ic were evaluated using Baujat plot and inconsistent and influential studies were identified in top right corner of plot ( arrows ). (C,D) Effect of the single study exclusion (C) and cumulative sequential exclusion of the most inconsistent studies (D) . Left : heterogeneity statistics, H 2 ( red line ) and I 2 ( black line ). Right : 95% CI ( red band ) and Q -test p -value ( black line ). Arrows : influential studies contributing to heterogeneity (same as those identified on Baujat Plot). Dashed Black line : homogeneity threshold T H where Q -test p = 0.05.

Heterogeneity sensitivity analyses

Inconsistencies between studies can arise for a number of reasons, including methodological or biological heterogeneity (Patsopoulos et al., 2008 ). Since accounting for heterogeneity is an essential part of any meta-analysis, it is of interest to identify influential studies that may contribute to the observed heterogeneity.

Baujat plot . The Baujat Plot was proposed as a diagnostic tool to identify the studies that contribute most to heterogeneity and influence the global outcome (Baujat, 2002 ). The graph illustrates the contribution Q i in f of each study to heterogeneity on the x-axis

and contribution θ i in f to global effect on the y-axis

Studies that strongly influence the global outcome and contribute to heterogeneity are visualized in the upper right corner of the plot ( Figure 8B ). This approach has been used to identify outlying studies in the past (Anzures-Cabrera and Higgins, 2010 ).

Single-study exclusion sensitivity . Single-study exclusion analysis assesses the sensitivity of the global outcome and heterogeneity to exclusion of single studies. The global outcomes and heterogeneity statistics are computed for a dataset with a single omitted study; single study exclusion is iterated for all studies; and influential outlying studies are identified by observing substantial declines in observed heterogeneity, as determined by Q total , H 2 , or I 2 , and by significant differences in the global outcome ( Figure 8C ). Influential studies should not be blindly discarded, but rather carefully examined to determine the reason for inconsistency. If a cause for heterogeneity can be identified, such as experimental design flaw, it is appropriate to omit the study from the analysis. All reasons for omission must be justified and made transparent by reviewers.

Cumulative-study exclusion sensitivity . Cumulative study exclusion sequentially removes studies to maximize the decrease in total variance Q total , such that a more homogenous set of studies with updated heterogeneity statistics is achieved with each iteration of exclusion ( Figure 8D ).

This method was proposed by Patsopoulos et al. to achieve desired levels of homogeneity (Patsopoulos et al., 2008 ), however, Higgins argued that its application should remain limited to (i) quantifying the extent to which heterogeneity permeates the set of studies and (ii) identifying sources of heterogeneity (Higgins, 2008 ). We propose the homogeneity threshold T H as a measure of heterogeneity that can be derived from cumulative-study exclusion sensitivity analysis. The homogeneity threshold describes the percentage of studies that need to be removed (by the maximal Q-reduction criteria) before a homogenous set of studies is achieved. For example, in the OB [ATP] ic dataset, the homogeneity threshold was 71%, since removal of 71% of the most inconsistent studies resulted in a homogeneous dataset ( Figure 8D , right ). After homogeneity is attained by cumulative exclusion, the global effect generally stabilizes with respect to subsequent study removal. This metric provides information about the extent of inconsistency present in the set of studies that is scale invariant (independent of the number of studies), and is easily interpretable.

Exploratory Analyses

The purpose of an exploratory analysis is to understand the data in ways that may not be represented by a pooled global estimate. This involves identifying sources of observed heterogeneity related to biological and experimental factors. Subgroup and meta-regression analyses are techniques used to explore known data groupings define by study-level characteristics (i.e., covariates). Additionally, we introduce the cluster-covariate dependence analysis, which is an unsupervised exploratory technique used to identify covariates that coincide well will natural groupings within the data, and the intrastudy regression analysis, which is used to validate meta-regression outcomes.

Cluster-covariate dependence analysis

Natural groupings within the data can be informative and serve as a basis to guide further analysis. Using an unsupervised k-means clustering approach (Lloyd, 1982 ), we can identify natural groupings within the study-level data and assign cluster memberships to these data ( Figure 9A ). Reviewers then have two choices: either proceed directly to subgroup analysis ( Figure 9B ) or look for covariates that co-cluster with cluster memberships ( Figure 9C ) In the latter case, dependencies between cluster memberships and known data covariates can be tested using Pearson's Chi-Squared test for independence. Covariates that coincide with clusters can be verified by subgroup analysis ( Figure 9D ). The dependence test is limited by the availability of studies and requires that at least 80% of covariate-cluster pairs are represented by at least 5 studies (McHugh, 2013 ). Clustering results should be considered exploratory and warrant further investigation due to several limitations. If the subpopulations were identified through clustering, however they do not depend on extracted covariates, reviewers risk assigning misrepresentative meaning to these clusters. Moreover, conventional clustering methods always converge to a result, therefore the data will still be partitioned even in the absence of natural data groupings. Future adaptations of this method might involve using different clustering algorithms (hierarchical clustering) or independence tests (G-test for independence) as well as introducing weighting terms to bias clustering to reflect study-level precisions.

Exploratory subgroup analysis. (A) Exploratory k-means clustering was used to partition OB [ATP] ic ( left ) and ATP release ( right ) data into potential clusters/subpopulations of interest. (B) Subgroup analysis of OB [ATP] ic data by differentiation status (immature – 0 to 3 day osteoblasts vs. mature – 4 to 28 day osteoblasts). Subgroup outcomes (fmol ATP/cell) estimated using sample-size weighting-scheme; black markers : Study-level outcomes ± 95% CI, marker sizes are proportional to sample size n . Orange and green bands : 95% CI for immature and mature osteoblast subgroups, respectively. (C) Dependence between ATP release cluster membership and known covariates/characteristics was assessed using Pearson's χ 2 independence test. Black bars : χ 2 test p -values for each covariate-cluster dependence test. Red line : α = 0.05 significance threshold. Arrow : most influential covariate (ex. recording method). (D) Subgroup analysis of ATP release by recording method. Subgroup outcomes (t half ) estimated using random effects weighting, τ 2 computed using DerSimonian-Laird estimator. Round markers : subgroup estimates ± 95% CI, marker sizes are proportional to number of studies per subgroup N . Gray band/diamond : global effect ± 95% CI.

Subgroup analysis

Subgroup analyses attempt to explain heterogeneity and explore differences in effects by partitioning studies into characteristic groups defined by study-level categorical covariates ( Figures 9B,D ; Table 5 ). Subgroup effects are estimated along with corresponding heterogeneity statistics. To evaluate the extent to which subgroup covariates contribute to observed inconsistencies, the explained heterogeneity Q between and unexplained heterogeneity Q within can be calculated.

Exploratory subgroup analysis.

Effect and heterogeneity estimates of ATP release by recording method .

where S is the total number of subgroups per given covariate and each subgroup j contains N j studies. The explained heterogeneity Q between is then the difference between total and subgroup heterogeneity:

If the p -value for the χ 2 distributed statistic Q between is significant, the subgrouping can be assumed to explain a significant amount of heterogeneity (Borenstein, 2009 ). Similarly, Q within statistic can be used to test whether there is any residual heterogeneity present within the subgroups.

The R e x p l a i n e d 2 is a related statistic that can be used to describe the percent of total heterogeneity that was explained by the covariate and is estimated as

Where pooled heterogeneity within subgroups τ w i t h i n 2 represents the remaining unexplained variation (Borenstein, 2009 ):

Subgroup analysis of the ATP release dataset revealed that recording method had a major influence on ATP release outcome, such that method A produced significantly lower outcomes than method B ( Figure 9D ; Table 5 , significance determined by non-overlapping 95% CIs). Additionally, recording method accounted for a significant amount of heterogeneity ( Q between , p < 0.001), however it represented only 4% ( R e x p l a i n e d 2 ) of the total observed heterogeneity. Needless to say, the remaining 96% of heterogeneity is significant ( Q within , p < 0.001). To explore the remaining heterogeneity, additional subgroup analysis can be conducted by further stratifying method A and method B subgroups by other covariates. However, in many meta-analyses multi-level data stratification may be unfeasible if covariates are unavailable or if the number of studies within subgroups are low.

Multiple comparisons . When multiple subgroups are present for a given covariate, and the reviewer wishes to investigate the statistical differences between the subgroups, the problem of multiple comparisons should be addressed. Error rates are multiplicative and increase substantially as the number of subgroup comparisons increases. The Bonferroni correction has been advocated to control for false positive findings in meta-analyses (Hedges and Olkin, 1985 ) which involves adjusting the significance threshold:

α * is the adjusted significance threshold to attain intended error rates α for m subgroup comparisons. Confidence intervals can then be computed using α * in place of α:

Meta-regression

Meta-regression attempts to explain heterogeneity by examining the relationship between study-level outcomes and continuous covariates while incorporating the influence of categorical covariates ( Figure 10A ). The main differences between conventional linear regression and meta-regression are (i) the incorporation of weights and (ii) covariates are at the level of the study rather than the individual sample. The magnitude of the relationship β n between the covariates x n,i and outcome y i for study i and covariate n are of interest when conducting a meta-regression analysis. It should be noted that the intercept β 0 of a meta-regression with negligible effect of covariates is equivalent to the estimate approximated by a weighted mean (Equation 3). The generalized meta-regression model is specified as

Meta-regression analysis and validation. (A) Relationship between osteoblast differentiation day (covariate) and intracellular ATP content (outcome) investigated by meta-regression analysis. Outcomes are on log 10 scale, meta-regression markers sizes are proportional to weights. Red bands : 95% CI. Gray bands : 95% CI of intercept only model. Solid red lines : intrastudy regression. (B) Meta-regression coefficient β inter ( black ) compared to intrastudy regression coefficient β intra ( red ). Shown are regression coefficients ± 95% CI.

where intrastudy variance ε i is

and the deviation from the distribution of effects η i depends on the chosen meta-analytic model:

The residual Q statistic that explains the dispersion of the studies from the regression line is calculated as follows

Where y i is the predicted value at x i according to the meta-regression model. Q residual is analogous to Q between computed during subgroup analysis and is used to test the degree of remaining unaccounted heterogeneity. Q residual is also used to approximate the unexplained interstudy variance τ r e s i d u a l 2

Which can be used to calculate R e x p l a i n e d 2 estimated as

Q model quantifies the amount of heterogeneity explained by the regression model and is analogous to Q within computed during subgroup analysis.

Intrastudy regression analysis The challenge of interpreting results from a meta-regression is that relationships that exist within studies may not necessarily exist across studies, and vice versa. Such inconsistencies are known as aggregation bias and in the context of meta-analyses can arise from excess heterogeneity or from confounding factors at the level of the study. This problem has been acknowledged in clinical meta-analyses (Thompson and Higgins, 2002 ), however cannot be corrected without access to individual patient data. Fortunately, basic research studies often report outcomes at varying predictor levels (ex. dose-response curves), permitting for intrastudy (within-study) relationships to be evaluated by the reviewer. If study-level regression coefficients can be computed for several studies ( Figure 10A , red lines ), they can be pooled to estimate an overall effect β intra . The meta-regression interstudy coefficient β inter and the overall intrastudy-regression coefficient β intra can then be compared in terms of magnitude and sign. Similarity in the magnitude and sign validates the existence of the relationship and characterizes its strength, while similarity in sign but not the magnitude, still supports the presence of the relationship, but calls for additional experiments to further characterize it. For the Ob [ATP] i dataset, the magnitude of the relationship between osteoblast differentiation day and intracellular ATP concentration was inconsistent between intrastudy and interstudy estimates, however the estimates were of consistent sign ( Figure 10B ).

Limitations of exploratory analyses

When performed with knowledge and care, exploratory analysis of meta-analytic data has an enormous potential for hypothesis generation, cataloging current practices and trends, and identifying gaps in the literature. Thus, we emphasize the inherent limitations of exploratory analyses:

Data dredging . A major pitfall in meta-analyses is data dredging (also known as p-hacking), which refers to searching for significant outcomes only to assign meaning later. While exploring the dataset for potential patterns can identify outcomes of interest, reviewers must be wary of random patterns that can arise in any dataset. Therefore, if a relationship is observed it should be used to generate hypotheses, which can then be tested on new datasets. Steps to avoid data dredging involve defining an a priori analysis plan for study-level covariates, limiting exploratory analysis of rapid review meta-analyses and correcting for multiple comparisons.

Statistical power . The statistical power reflects the probability of rejecting the null hypothesis when the alternative is true. Meta-analyses are believed to have higher statistical power than the underlying primary studies, however this is not always true (Hedges and Pigott, 2001 ; Jackson and Turner, 2017 ). Random effects meta-analyses handle data heterogeneity by accounting for between-study variance, however this weakens the inference properties of the model. To maintain statistical powers that exceed those of the contributing studies in a random effects meta-analysis, at least five studies are required (Jackson and Turner, 2017 ). This consequently limits subgroup analyses that partition studies into smaller groups to isolate covariate-dependent effects. Thus, reviewers should ensure that group are not under-represented to maintain statistical power. Another determinant of statistical power is the expected effect size, which if small, will be much more difficult to support with existing evidence than if it is large. Thus, if reviewers find that there is insufficient evidence to conclude that a small effect exists, this should not be interpreted as evidence of no effect.

Causal inference . Meta-analyses are not a tool for establishing causal inference. However, there are several criteria for causality that can be investigated through exploratory analyses that include consistency, strength of association, dose-dependence and plausibility (Weed, 2000 , 2010 ). For example, consistency, the strength of association, and dose-dependence can help establish that the outcome is dependent on exposure. However, reviewers are still posed with the challenge of accounting for confounding factors and bias. Therefore, while meta-analyses can explore various criteria for causality, causal claims are inappropriate, and outcomes should remain associative.

Conclusions

Meta-analyses of basic research can offer critical insights into the current state of knowledge. In this manuscript, we have adapted meta-analytic methods to basic science applications and provided a theoretical foundation, using OB [ATP] i and ATP release datasets, to illustrate the workflow. Since the generalizability of any meta-analysis relies on the transparent, unbiased and accurate methodology, the implications of deficient reporting practices and the limitations of the meta-analytic methods were discussed. Emphasis was placed on the analysis and exploration of heterogeneity. Additionally, several alternative and supporting methods have been proposed, including a method for validating meta-regression outcomes—intrastudy regression analysis, and a novel measure of heterogeneity—the homogeneity threshold. All analyses were conducted using MetaLab , a meta-analysis toolbox that we have developed in MATLAB R2016b. MetaLab has been provided for free to promote meta-analyses in basic research ( https://github.com/NMikolajewicz/MetaLab ).

In its current state, the translational pipeline from benchtop to bedside is an inefficient process, in one case estimated to produce ~1 clinically favorable clinical outcome for ~1,000 basic research studies (O'Collins et al., 2006 ). The methods we have described here serve as a general framework for comprehensive data consolidation, knowledge gap-identification, evidence-driven hypothesis generation and informed parameter estimation in computation modeling, which we hope will contribute to meta-analytic outcomes that better inform translation studies, thereby minimizing current failures in translational research.

Author Contributions

Both authors contributed to the study conception and design, data acquisition and interpretation and drafting and critical revision of the manuscript. NM developed MetaLab. Both authors approved the final version to be published.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This work was supported by Natural Sciences and Engineering Research Council (NSERC, RGPIN-288253) and Canadian Institutes for Health Research (CIHR MOP-77643). NM was supported by the Faculty of Dentistry, McGill University and le Réseau de Recherche en Santé Buccodentaire et Osseuse (RSBO). Special thanks to Ali Mohammed (McGill University) for help with validation of MetaLab data extraction module.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphys.2019.00203/full#supplementary-material

• Ahmed I., Sutton A. J., Riley R. D. (2012). Assessment of publication bias, selection bias, and unavailable data in meta-analyses using individual participant data: a database survey . Br. Med. J. 344 :d7762 10.1136/bmj.d7762 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Altman D. G., Bland J. M. (2005). Standard deviations and standard errors . Br. Med. J. 331 , 903–903. 10.1136/bmj.331.7521.903 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Anzures-Cabrera J., Higgins J. P. T. (2010). Graphical displays for meta-analysis: an overview with suggestions for practice . Res. Synth. Methods 1 , 66–80. 10.1002/jrsm.6 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Baguley T. (2009). Standardized or simple effect size: what should be reported? Br. J. Soc. Psychol. 100 , 603–617. 10.1348/000712608X377117 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Barendregt J., Doi S. (2009). MetaXL User Guide: Version 1.0 . Wilston, QLD: EpiGear International Pty Ltd. [ Google Scholar ]
• Baujat B. (2002). A graphical method for exploring heterogeneity in meta-analyses: application to a meta-analysis of 65 trials . Stat. Med. 21 :18. 10.1002/sim.1221 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bax L. (2016). MIX 2.0 – Professional Software for Meta-analysis in Excel. Version 2.0.1.5. BiostatXL . Available online at: https://www.meta-analysis-made-easy.com
• Bittker J. A., Ross N. T. (2016). High Throughput Screening Methods: Evolution and Refinement. Cambridge: Royal Society of Chemistry; 10.1039/9781782626770 [ CrossRef ] [ Google Scholar ]
• Bodin P., Milner P., Winter R., Burnstock G. (1992). Chronic hypoxia changes the ratio of endothelin to ATP release from rat aortic endothelial cells exposed to high flow . Proc. Biol. Sci. 247 , 131–135. 10.1098/rspb.1992.0019 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Borenstein M. (2009). Introduction to Meta-Analysis . Chichester: John Wiley & Sons. 10.1002/9780470743386 [ CrossRef ] [ Google Scholar ]
• Borenstein M., Hedges L., Higgins J. P. T., Rothstein H. R. (2005). Comprehensive meta-analysis (Version 2.2.027) [Computer software]. Englewood, CO. [ Google Scholar ]
• Bramer W. M., Giustini D., de Jonge G. B., Holland L., Bekhuis T. (2016). De-duplication of database search results for systematic reviews in EndNote . J. Med. Libr. Assoc. 104 , 240–243. 10.3163/1536-5050.104.3.014 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Chowdhry A. K., Dworkin R. H., McDermott M. P. (2016). Meta-analysis with missing study-level sample variance data . Stat. Med. 35 , 3021–3032. 10.1002/sim.6908 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cochrane Collaboration (2011). Review Manager (RevMan) [Computer Program] . Copenhagen. [ Google Scholar ]
• Cox M., Harris P., Siebert B. R.-L. (2003). Evaluation of measurement uncertainty based on the propagation of distributions using monte carlo simulation . Measure. Techniq. 46 , 824–833. 10.1023/B:METE.0000008439.82231.ad [ CrossRef ] [ Google Scholar ]
• DeLuca J. B., Mullins M. M., Lyles C. M., Crepaz N., Kay L., Thadiparthi S. (2008). Developing a comprehensive search strategy for evidence based systematic reviews . Evid. Based Libr. Inf. Pract. 3 , 3–32. 10.18438/B8KP66 [ CrossRef ] [ Google Scholar ]
• DerSimonian R., Laird N. (1986). Meta-analysis in clinical trials . Control. Clin. Trials 7 , 177–188. 10.1016/0197-2456(86)90046-2 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Ecker E. D., Skelly A. C. (2010). Conducting a winning literature search . Evid. Based Spine Care J. 1 , 9–14. 10.1055/s-0028-1100887 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Egger M., Smith G. D., Schneider M., Minder C. (1997). Bias in meta-analysis detected by a simple, graphical test . Br. Med. J. 315 , 629–634. 10.1136/bmj.315.7109.629 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Finfgeld-Connett D., Johnson E. D. (2013). Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews . J. Adv. Nurs. 69 , 194–204. 10.1111/j.1365-2648.2012.06037.x [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Ganann R., Ciliska D., Thomas H. (2010). Expediting systematic reviews: methods and implications of rapid reviews . Implementation Sci. 5 , 56–56. 10.1186/1748-5908-5-56 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gavaghan D. J., Moore R. A., McQuay H. J. (2000). An evaluation of homogeneity tests in meta-analyses in pain using simulations of individual patient data . Pain 85 , 415–424. 10.1016/S0304-3959(99)00302-4 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gopalakrishnan S., Ganeshkumar P. (2013). Systematic reviews and meta-analysis: understanding the best evidence in primary healthcare . J Fam. Med. Prim. Care 2 , 9–14. 10.4103/2249-4863.109934 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Grönholm T., Annila A. (2007). Natural distribution . Math. Biosci. 210 , 659–667. 10.1016/j.mbs.2007.07.004 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Haby M. M., Chapman E., Clark R., Barreto J., Reveiz L., Lavis J. N. (2016). What are the best methodologies for rapid reviews of the research evidence for evidence-informed decision making in health policy and practice: a rapid review . Health Res. Policy Syst. 14 :83. 10.1186/s12961-016-0155-7 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hartung J., Makambi K. H. (2002). Positive estimation of the between-study variance in meta-analysis: theory and methods . S. Afr. Stat. J. 36 , 55–76. [ Google Scholar ]
• Hedges L. V., Olkin I. (1985). Statistical Methods for Meta-Analysis . New York, NY: Academic Press. [ Google Scholar ]
• Hedges L. V., Pigott T. D. (2001). The power of statistical tests in meta-analysis . Psychol. Methods 6 , 203–217. 10.1037/1082-989X.6.3.203 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Higgins J. P. (2008). Commentary: heterogeneity in meta-analysis should be expected and appropriately quantified . Int. J. Epidemiol. 37 , 1158–1160. 10.1093/ije/dyn204 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Higgins J. P., Green S. (Eds.) (2011). Cochrane Handbook for Systematic Reviews of Interventions , Vol. 4 Oxford: John Wiley & Sons. [ Google Scholar ]
• Higgins J. P., Thompson S. G. (2002). Quantifying heterogeneity in a meta-analysis . Stat. Med. 21 , 1539–1558. 10.1002/sim.1186 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Higgins J. P., Thompson S. G., Deeks J. J., Altman D. G. (2003). Measuring inconsistency in meta-analyses . Br. Med. J. 327 , 557–560. 10.1136/bmj.327.7414.557 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Higgins J. P., White I. R., Anzures-Cabrera J. (2008). Meta-analysis of skewed data: combining results reported on log-transformed or raw scales . Stat. Med. 27 , 6072–6092. 10.1002/sim.3427 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Huedo-Medina T. B., Sanchez-Meca J., Marin-Martinez F., Botella J. (2006). Assessing heterogeneity in meta-analysis: Q statistic or I 2 index? Psychol. Methods 11 , 193–206. 10.1037/1082-989X.11.2.193 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hunter J. E., Schmidt F. L. (2004). Methods of Meta-analysis: Correcting Error and Bias in Research Findings . Thousand Oaks, CA: Sage. [ Google Scholar ]
• Jackson D., Turner R. (2017). Power analysis for random-effects meta-analysis . Res. Synth. Methods 8 , 290–302. 10.1002/jrsm.1240 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• JASP Team (2018). JASP (Verision 0.9) [Computer Software] . Amsterdam. [ Google Scholar ]
• Karabatsos G., Talbott E., Walker S. G. (2015). A Bayesian nonparametric meta-analysis model . Res. Synth. Methods 6 , 28–44. 10.1002/jrsm.1117 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kontopantelis E., Reeves D. (2012). Performance of statistical methods for meta-analysis when true study effects are non-normally distributed: a simulation study . Stat. Methods Med. Res. 21 , 409–426. 10.1177/0962280210392008 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kwon Y., Lemieux M., McTavish J., Wathen N. (2015). Identifying and removing duplicate records from systematic review searches . J. Med. Libr. Assoc. 103 , 184–188. 10.3163/1536-5050.103.4.004 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Light R. J., Pillemer D. B. (1984). Summing Up: The Science of Reviewing Research . Cambridge, MA: Harvard University Press. [ Google Scholar ]
• Limpert E., Stahel W. A., Abbt M. (2001). Log-normal Distributions across the sciences: keys and clues: on the charms of statistics, and how mechanical models resembling gambling machines offer a link to a handy way to characterize log-normal distributions, which can provide deeper insight into variability and probability—normal or log-normal: That is the question . AIBS Bull. 51 , 341–352. [ Google Scholar ]
• Lloyd S. (1982). Least squares quantization in PCM . IEEE Trans. Inf. Theory 28 , 129–137. 10.1109/TIT.1982.1056489 [ CrossRef ] [ Google Scholar ]
• Lorenzetti D. L., Ghali W. A. (2013). Reference management software for systematic reviews and meta-analyses: an exploration of usage and usability . BMC Med. Res. Methodol. 13 , 141–141. 10.1186/1471-2288-13-141 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Marin-Martinez F., Sanchez-Meca J. (2010). Weighting by inverse variance or by sample size in random-effects meta-analysis . Educ. Psychol. Meas. 70 , 56–73. 10.1177/0013164409344534 [ CrossRef ] [ Google Scholar ]
• Mattivi J. T., Buchberger B. (2016). Using the amstar checklist for rapid reviews: is it feasible? Int. J. Technol. Assess. Health Care 32 , 276–283. 10.1017/S0266462316000465 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• McGowan J., Sampson M. (2005). Systematic reviews need systematic searchers . J. Med. Libr. Assoc. 93 , 74–80. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• McHugh M. L. (2013). The Chi-square test of independence . Biochem. Med. 23 , 143–149. 10.11613/BM.2013.018 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mikolajewicz N., Mohammed A., Morris M., Komarova S. V. (2018). Mechanically-stimulated ATP release from mammalian cells: systematic review and meta-analysis . J. Cell Sci. 131 :22. 10.1242/jcs.223354 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Milo R., Jorgensen P., Moran U., Weber G., Springer M. (2010). BioNumbers—the database of key numbers in molecular and cell biology . Nucleic Acids Res. 38 :D750–3. 10.1093/nar/gkp889 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Moher D., Liberati A., Tetzlaff J., Altman D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement . PLoS Med. 6 :e1000097 10.1371/journal.pmed.1000097 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• O'Collins V. E., Macleod M. R., Donnan G. A., Horky L. L., van der Worp B. H., Howells D. W. (2006). 1,026 experimental treatments in acute stroke . Ann. Neurol. 59 , 467–477. 10.1002/ana.20741 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Pathak M., Dwivedi S. N., Deo S. V. S., Sreenivas V., Thakur B. (2017). Which is the preferred measure of heterogeneity in meta-analysis and why? a revisit . Biostat Biometrics Open Acc . 1 , 1–7. 10.19080/BBOAJ.2017.01.555555 [ CrossRef ] [ Google Scholar ]
• Patsopoulos N. A., Evangelou E., Ioannidis J. P. A. (2008). Sensitivity of between-study heterogeneity in meta-analysis: proposed metrics and empirical evaluation . Int. J. Epidemiol. 37 , 1148–1157. 10.1093/ije/dyn065 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Paule R. C., Mandel J. (1982). Consensus values and weighting factors . J. Res. Natl. Bur. Stand. 87 , 377–385. 10.6028/jres.087.022 [ CrossRef ] [ Google Scholar ]
• Sanchez-Meca J., Marin-Martinez F. (2008). Confidence intervals for the overall effect size in random-effects meta-analysis . Psychol. Methods 13 , 31–48. 10.1037/1082-989X.13.1.31 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Schwarzer G., Carpenter J. R., Rücker G. (2015). Small-study effects in meta-analysis , in Meta-Analysis with R , eds Schwarzer G., Carpenter J. R., Rücker G. (Cham: Springer International Publishing; ), 107–141. [ Google Scholar ]
• Sena E., van der Worp H. B., Howells D., Macleod M. (2007). How can we improve the pre-clinical development of drugs for stroke? Trends Neurosci. 30 , 433–439. 10.1016/j.tins.2007.06.009 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sheldrake R. (1997). Experimental effects in scientific research: how widely are they neglected? Bull. Sci. Technol. Soc. 17 , 171–174. 10.1177/027046769701700405 [ CrossRef ] [ Google Scholar ]
• Sidik K., Jonkman J. N. (2005). Simple heterogeneity variance estimation for meta-analysis . J. R. Stat. Soc. Ser. C Appl. Stat. 54 , 367–384. 10.1111/j.1467-9876.2005.00489.x [ CrossRef ] [ Google Scholar ]
• Sterne J. A., Sutton A. J., Ioannidis J. P., Terrin N., Jones D. R., Lau J., et al.. (2011). Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials . Br. Med. J. 343 :d4002. 10.1136/bmj.d4002 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sterne J. A. C., Harbord R. (2004). Funnel plots in meta-analysis . Stata J. 4 , 127–141. 10.1177/1536867X0400400204 [ CrossRef ] [ Google Scholar ]
• Thompson S. G., Higgins J. P. (2002). How should meta-regression analyses be undertaken and interpreted? Stat. Med. 21 , 1559–1573. 10.1002/sim.1187 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Vaux D. L., Fidler F., Cumming G. (2012). Replicates and repeats—what is the difference and is it significant?: a brief discussion of statistics and experimental design . EMBO Rep. 13 , 291–296. 10.1038/embor.2012.36 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Veroniki A. A., Jackson D., Viechtbauer W., Bender R., Bowden J., Knapp G., et al.. (2016). Methods to estimate the between-study variance and its uncertainty in meta-analysis . Res. Synth. Methods 7 , 55–79. 10.1002/jrsm.1164 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Vesterinen H. M., Sena E. S., Egan K. J., Hirst T. C., Churolov L., Currie G. L., et al.. (2014). Meta-analysis of data from animal studies: a practical guide . J. Neurosci. Methods 221 , 92–102. 10.1016/j.jneumeth.2013.09.010 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Viechtbauer W. (2010). Conducting meta-analyses in R with the metafor package . J. Stat. Softw. 36 , 1–48. 10.18637/jss.v036.i03 [ CrossRef ] [ Google Scholar ]
• von Hippel P. T. (2015). The heterogeneity statistic I 2 can be biased in small meta-analyses . BMC Med. Res. Methodol. 15 :35 10.1186/s12874-015-0024-z [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Weed D. L. (2000). Interpreting epidemiological evidence: how meta-analysis and causal inference methods are related . Int. J. Epidemiol. 29 , 387–390. 10.1093/intjepid/29.3.387 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Weed D. L. (2010). Meta-analysis and causal inference: a case study of benzene and non-hodgkin lymphoma . Ann. Epidemiol. 20 , 347–355. 10.1016/j.annepidem.2010.02.001 [ PubMed ] [ CrossRef ] [ Google Scholar ]

Journals, books & databases

• Our journals

Analytical Methods

Early applications of new analytical methods and technology demonstrating potential for societal impact

You can find details about how to access information remotely in this step-by-step guide . The guide will also help if for any reason you have difficulty accessing the content you want.

What would you like to know about this journal?

Analytical Methods is a Transformative Journal and Plan S compliant

Impact factor: 3.1*

Time to first decision (all decisions): 12.0 days**

Time to first decision (peer reviewed only): 28.5 days***

Editor-in-Chief: Scott Martin

Indexed in MEDLINE

Open access publishing options available

Read this journal

Submit an article

Sign up for regular email alerts

View all journal metrics

Publish open access

Meet the team

Journal scope

Analytical Methods  welcomes early applications of new analytical and bioanalytical methods and technology demonstrating the potential for societal impact.

We require that methods and technology reported in the journal are sufficiently innovative, robust, accurate, and compared to other available methods for the intended application. Developments with interdisciplinary approaches are particularly welcome. Systems should be proven with suitably complex and analytically challenging samples.

We encourage developments within, but not limited to, the following technologies and applications:

• global health, point-of-care and molecular diagnostics
• biosensors and bioengineering
• drug development and pharmaceutical analysis
• applied microfluidics and nanotechnology
• omics studies, such as proteomics, metabolomics or glycomics
• environmental, agricultural and food science
• neuroscience
• biochemical and clinical analysis
• forensic analysis
• industrial process and method development

Meet the editorial team

Find out who is on the editorial and advisory boards for the  Analytical Methods  journal.

Editor-in-chief

Scott Martin , Saint Louis University, USA

Associate editors

Jonas Bergquist , Uppsala University, Sweden

Wendell Coltro , Federal University of Goiás, Brazil

Juan F García-Reyes , University of Jaén, Spain 

Tony Killard , Reviews Editor, University of West England, UK

Zhen Liu , Nanjing University, China

Chao Lu , Beijing University of Chemical Technology, China

Fiona Regan , Dublin City University, Ireland

Michael Roper , Florida State University, USA

B. Jill Venton , University of Virginia, USA

Jailson de Andrade , Universidade Federal da Bahia, Brazil

Lane Baker , Indiana University, USA

Craig Banks , Manchester Metropolitan University, UK

Emanuel Carrilho , São Carlos, Brazil

James Chapman , Royal Australian Chemical Institute, Australia

Yi Chen , Chinese Academy of Sciences, China

Christopher J. Easley , Auburn University, USA

Anthony Gachanja , Jomo Kenyatta University of Agriculture and Technology, Kenya

Amanda Hummon , Ohio State University, USA

Lauro Kubota , Instituto de Química, Brazil

Ally Lewis , University of York, UK

Juewen Liu , University of Waterloo, Canada

Susan Lunte,  University of Kansas, USA

Jim Luong , Dow Chemical Canada ULC, Canada

Susheel Mittal , Thapar University, India

Antonio Molina-Díaz , University of Jaén, Spain

Koji Otsuka , Kyoto University, Japan

Brett Paull , University of Tasmania, Australia

Zachary Schultz , University of Notre Dame, USA

Sabeth Verpoorte , University of Groningen, Netherlands

Guobao Xu , Changchun Institute of Applied Chemistry, China

Philippa Ross , Executive Editor

Alice Smallwood , Deputy Editor

Celeste Brady , Development Editor

David Lake , Development Editor

Jason Woolford , Editorial Production Manager

Gabriel Clarke , Publishing Editor

Derya Kara-Fisher , Publishing Editor

Emma Stephen , Publishing Editor

Ziva Whitelock , Publishing Editor

Leo Curtis , Editorial Assistant

Andrea Whiteside , Publishing Assistant

Jeanne Andres , Publisher

Article types

Analytical Methods publishes:

Communications

Full papers, technical notes, critical reviews, minireviews, tutorial reviews.

These must report preliminary research findings that are highly original, of immediate interest and are likely to have a high impact. Communications are given priority treatment, are fast-tracked through the publication process and appear prominently at the front of the journal.

The key aim of Communications is to present innovative concepts with important analytical implications. As such, Communications need only demonstrate 'proof of principle': it is not expected that the analytical figures of merit will necessarily surpass those of existing, highly refined analytical techniques.

At the time of submission, authors should also provide a justification for urgent publication as a Communication. Ideally, a Full paper should follow each Communication in an appropriate primary journal.

There is no page limit for communications in Analytical Methods , however the length should be commensurate with scientific content. Authors are encouraged to make full use of electronic supplementary information (ESI) in order to present more concise articles.

These must describe science that will be of benefit to the community in the particular field of analysis and are judged according to originality, quality of scientific content and contribution to existing knowledge.

Although there is no page limit for Full papers, appropriateness of length to content of new science will be taken into consideration.

These should be brief descriptions of developments, techniques or applications that offer definite advantages over those already available. Technical notes should offer practical solutions to problems that are of interest to the readership and merit publication, but where a Full paper is not justified.

Technical notes should be as brief as possible; wherever appropriate authors should use references to the established technique, explaining in full only what is novel about the proposed approach.

Critical reviews are definitive, comprehensive reviews but must also provide a critical evaluation of the chosen topic area. Authors should try to be selective in the choice of material, whilst still aim to cover all the important work in the field, also indicating possible future developments.

Minireviews are highlights or summaries of research in an emerging area of analytical science covering approximately the last two-three years. Given topics should review work no more than approximately 36 months old, and articles should cover only the most interesting/significant developments in that specific subject area.

The articles should be highly critical and selective in referencing published work. A small amount of speculation (one or two paragraphs) of possible future developments may also be appropriate in the Conclusions section.

Written from a personal point of view, these ideally should be the first review of a new significant area, bringing together the results of various primary publications.

Tutorial reviews are intended to interest a large number of readers and should be written at a level that could be understood by an advanced undergraduate student.

The intention is to increase awareness and understanding of the chosen topic area for workers/researchers already involved in the field, workers changing the direction/emphasis of their work and a broad based non-specialist (graduate and post-graduate) audience, with a view to informing them of the most recent developments in the area.

Potential writers should contact the editorial office before embarking on their work.

Comments and Replies are a medium for the discussion and exchange of scientific opinions between authors and readers concerning material published in Analytical Methods .

For publication, a Comment should present an alternative analysis of and/or new insight into the previously published material. Any Reply should further the discussion presented in the original article and the Comment. Comments and Replies that contain any form of personal attack are not suitable for publication. 

Comments that are acceptable for publication will be forwarded to the authors of the work being discussed, and these authors will be given the opportunity to submit a Reply. The Comment and Reply will both be subject to rigorous peer review in consultation with the journal’s Editorial Board where appropriate. The Comment and Reply will be published together.

Journal specific guidelines

On submission, authors are required to submit a short significance statement for their manuscript. This statement should address the technological advance and/or significance of the methods and applications in the presented work (1–2 sentences maximum). This information will help the Editor and reviewers assess the article.

How do  Analyst and Analytical Methods compare? From discovery to recovery –  Analyst and Analytical Methods working together for the analytical community Analyst , 2011, 136 , 429 DOI: 10.1039/c0an90013c

Open access publishing options

Analytical Methods is a hybrid (transformative) journal and gives authors the choice of publishing their research either via the traditional subscription-based model or instead by choosing our gold open access option.  Find out more about our Transformative Journals. which are Plan S compliant .

Gold open access

For authors who want to publish their article gold open access , Analytical Methods  charges an article processing charge (APC) of £2,500 (+ any applicable tax). Our APC is all-inclusive and makes your article freely available online immediately, permanently, and includes your choice of Creative Commons licence (CC BY or CC BY-NC) at no extra cost. It is not a submission charge, so you only pay if your article is accepted for publication.

Learn more about publishing open access .

Read & Publish

If your institution has a Read & Publish agreement in place with the Royal Society of Chemistry, APCs for gold open access publishing in Analytical Methods  may already be covered.

Check if your institution is already part of our  Read & Publish community .

Please use your official institutional email address to submit your manuscript; this helps us to identify if you are eligible for Read & Publish or other APC discounts.

Traditional subscription model

Authors can also publish in Analytical Methods  via the traditional subscription model without needing to pay an APC. Articles published via this route are available to institutions and individuals who subscribe to the journal. Our standard licence allows you to make the accepted manuscript of your article freely available after a 12-month embargo period. This is known as the green route to open access.

Learn more about green open access .

Peer review

Analytical Methods follows a single-anonymised peer review process and articles are typically sent to at least two independent reviewers for evaluation. A dynamic and high-quality team of associate editors is responsible for peer review and associated editorial decisions. Authors may choose their preferred choice of associate editor upon submission.

Please note that it may not always be possible for the author's first choice associate editor to be selected. In situations where this is not possible the editorial office will assign the most suitable alternative.

On submission to the journal, all manuscripts are assigned to an external (academic) Associate Editor. Each submission is assessed for quality, scope, and impact. Those that do not meet the criteria based on these factors are rejected without further peer review. Otherwise, the article is sent to at least two external reviewers with expertise in the article topic for confidential review. An editorial decision to reject or accept the article is made on these reports.

More reviewers may be consulted in cases of opposing reports or when more clarification is needed. Articles needing significant revisions may be sent for further peer review before acceptance. Authors may appeal a rejection via communication with the Associate Editor. Our processes and policies can provide full details of the initial assessment process.

Readership information

Readership is cross-disciplinary and Analytical Methods appeals to readers across academia and industry, who have an interest in the advancement of measurement science and the breadth of application of analytical and bioanalytical methodologies.

These include, but are not restricted to, analytical and environmental scientists; biochemists and biotechnologists; process and industrial scientists; biomedical and clinical scientists; forensic and heritage scientists; agriculture, food, safety and product technologists; pharmaceutical scientists and toxicologists.

Subscription information

Analytical Methods is part of RSC Gold and Analytical Science  subscription packages. Online only 2024 : ISSN 1759-9679 £2,513 / $4,425

*2022 Journal Citation Reports (Clarivate Analytics, 2023)

**The median time from submission to first decision including manuscripts rejected without peer review from the previous calendar year

***The median time from submission to first decision for peer-reviewed manuscripts from the previous calendar year

Advertisement

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Dissertation
• What Is a Research Methodology? | Steps & Tips

What Is a Research Methodology? | Steps & Tips

Published on 25 February 2019 by Shona McCombes . Revised on 10 October 2022.

Your research methodology discusses and explains the data collection and analysis methods you used in your research. A key part of your thesis, dissertation, or research paper, the methodology chapter explains what you did and how you did it, allowing readers to evaluate the reliability and validity of your research.

It should include:

• The type of research you conducted
• How you collected and analysed your data
• Any tools or materials you used in the research
• Why you chose these methods
• Your methodology section should generally be written in the past tense .
• Academic style guides in your field may provide detailed guidelines on what to include for different types of studies.
• Your citation style might provide guidelines for your methodology section (e.g., an APA Style methods section ).

Table of contents

How to write a research methodology, why is a methods section important, step 1: explain your methodological approach, step 2: describe your data collection methods, step 3: describe your analysis method, step 4: evaluate and justify the methodological choices you made, tips for writing a strong methodology chapter, frequently asked questions about methodology.

Prevent plagiarism, run a free check.

Your methods section is your opportunity to share how you conducted your research and why you chose the methods you chose. It’s also the place to show that your research was rigorously conducted and can be replicated .

It gives your research legitimacy and situates it within your field, and also gives your readers a place to refer to if they have any questions or critiques in other sections.

You can start by introducing your overall approach to your research. You have two options here.

Option 1: Start with your “what”

What research problem or question did you investigate?

• Aim to describe the characteristics of something?
• Explore an under-researched topic?
• Establish a causal relationship?

And what type of data did you need to achieve this aim?

• Quantitative data , qualitative data , or a mix of both?
• Primary data collected yourself, or secondary data collected by someone else?
• Experimental data gathered by controlling and manipulating variables, or descriptive data gathered via observations?

Option 2: Start with your “why”

Depending on your discipline, you can also start with a discussion of the rationale and assumptions underpinning your methodology. In other words, why did you choose these methods for your study?

• Why is this the best way to answer your research question?
• Is this a standard methodology in your field, or does it require justification?
• Were there any ethical considerations involved in your choices?
• What are the criteria for validity and reliability in this type of research ?

Once you have introduced your reader to your methodological approach, you should share full details about your data collection methods .

Quantitative methods

In order to be considered generalisable, you should describe quantitative research methods in enough detail for another researcher to replicate your study.

Here, explain how you operationalised your concepts and measured your variables. Discuss your sampling method or inclusion/exclusion criteria, as well as any tools, procedures, and materials you used to gather your data.

Surveys Describe where, when, and how the survey was conducted.

• How did you design the questionnaire?
• What form did your questions take (e.g., multiple choice, Likert scale )?
• Were your surveys conducted in-person or virtually?
• What sampling method did you use to select participants?
• What was your sample size and response rate?

Experiments Share full details of the tools, techniques, and procedures you used to conduct your experiment.

• How did you design the experiment ?
• How did you recruit participants?
• How did you manipulate and measure the variables ?
• What tools did you use?

Existing data Explain how you gathered and selected the material (such as datasets or archival data) that you used in your analysis.

• Where did you source the material?
• How was the data originally produced?
• What criteria did you use to select material (e.g., date range)?

The survey consisted of 5 multiple-choice questions and 10 questions measured on a 7-point Likert scale.

The goal was to collect survey responses from 350 customers visiting the fitness apparel company’s brick-and-mortar location in Boston on 4–8 July 2022, between 11:00 and 15:00.

Here, a customer was defined as a person who had purchased a product from the company on the day they took the survey. Participants were given 5 minutes to fill in the survey anonymously. In total, 408 customers responded, but not all surveys were fully completed. Due to this, 371 survey results were included in the analysis.

Qualitative methods

In qualitative research , methods are often more flexible and subjective. For this reason, it’s crucial to robustly explain the methodology choices you made.

Be sure to discuss the criteria you used to select your data, the context in which your research was conducted, and the role you played in collecting your data (e.g., were you an active participant, or a passive observer?)

Interviews or focus groups Describe where, when, and how the interviews were conducted.

• How did you find and select participants?
• How many participants took part?
• What form did the interviews take ( structured , semi-structured , or unstructured )?
• How long were the interviews?
• How were they recorded?

Participant observation Describe where, when, and how you conducted the observation or ethnography .

• What group or community did you observe? How long did you spend there?
• How did you gain access to this group? What role did you play in the community?
• How long did you spend conducting the research? Where was it located?
• How did you record your data (e.g., audiovisual recordings, note-taking)?

Existing data Explain how you selected case study materials for your analysis.

• What type of materials did you analyse?
• How did you select them?

In order to gain better insight into possibilities for future improvement of the fitness shop’s product range, semi-structured interviews were conducted with 8 returning customers.

Here, a returning customer was defined as someone who usually bought products at least twice a week from the store.

Surveys were used to select participants. Interviews were conducted in a small office next to the cash register and lasted approximately 20 minutes each. Answers were recorded by note-taking, and seven interviews were also filmed with consent. One interviewee preferred not to be filmed.

Mixed methods

Mixed methods research combines quantitative and qualitative approaches. If a standalone quantitative or qualitative study is insufficient to answer your research question, mixed methods may be a good fit for you.

Mixed methods are less common than standalone analyses, largely because they require a great deal of effort to pull off successfully. If you choose to pursue mixed methods, it’s especially important to robustly justify your methods here.

Next, you should indicate how you processed and analysed your data. Avoid going into too much detail: you should not start introducing or discussing any of your results at this stage.

In quantitative research , your analysis will be based on numbers. In your methods section, you can include:

• How you prepared the data before analysing it (e.g., checking for missing data , removing outliers , transforming variables)
• Which software you used (e.g., SPSS, Stata or R)
• Which statistical tests you used (e.g., two-tailed t test , simple linear regression )

In qualitative research, your analysis will be based on language, images, and observations (often involving some form of textual analysis ).

Specific methods might include:

• Content analysis : Categorising and discussing the meaning of words, phrases and sentences
• Thematic analysis : Coding and closely examining the data to identify broad themes and patterns
• Discourse analysis : Studying communication and meaning in relation to their social context

Mixed methods combine the above two research methods, integrating both qualitative and quantitative approaches into one coherent analytical process.

Above all, your methodology section should clearly make the case for why you chose the methods you did. This is especially true if you did not take the most standard approach to your topic. In this case, discuss why other methods were not suitable for your objectives, and show how this approach contributes new knowledge or understanding.

In any case, it should be overwhelmingly clear to your reader that you set yourself up for success in terms of your methodology’s design. Show how your methods should lead to results that are valid and reliable, while leaving the analysis of the meaning, importance, and relevance of your results for your discussion section .

• Quantitative: Lab-based experiments cannot always accurately simulate real-life situations and behaviours, but they are effective for testing causal relationships between variables .
• Qualitative: Unstructured interviews usually produce results that cannot be generalised beyond the sample group , but they provide a more in-depth understanding of participants’ perceptions, motivations, and emotions.
• Mixed methods: Despite issues systematically comparing differing types of data, a solely quantitative study would not sufficiently incorporate the lived experience of each participant, while a solely qualitative study would be insufficiently generalisable.

Remember that your aim is not just to describe your methods, but to show how and why you applied them. Again, it’s critical to demonstrate that your research was rigorously conducted and can be replicated.

1. Focus on your objectives and research questions

The methodology section should clearly show why your methods suit your objectives  and convince the reader that you chose the best possible approach to answering your problem statement and research questions .

2. Cite relevant sources

Your methodology can be strengthened by referencing existing research in your field. This can help you to:

• Show that you followed established practice for your type of research
• Discuss how you decided on your approach by evaluating existing research
• Present a novel methodological approach to address a gap in the literature

3. Write for your audience

Consider how much information you need to give, and avoid getting too lengthy. If you are using methods that are standard for your discipline, you probably don’t need to give a lot of background or justification.

Regardless, your methodology should be a clear, well-structured text that makes an argument for your approach, not just a list of technical details and procedures.

Methodology refers to the overarching strategy and rationale of your research. Developing your methodology involves studying the research methods used in your field and the theories or principles that underpin them, in order to choose the approach that best matches your objectives.

Methods are the specific tools and procedures you use to collect and analyse data (e.g. interviews, experiments , surveys , statistical tests ).

In a dissertation or scientific paper, the methodology chapter or methods section comes after the introduction and before the results , discussion and conclusion .

Depending on the length and type of document, you might also include a literature review or theoretical framework before the methodology.

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to test a hypothesis by systematically collecting and analysing data, while qualitative methods allow you to explore ideas and experiences in depth.

A sample is a subset of individuals from a larger population. Sampling means selecting the group that you will actually collect data from in your research.

For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

Statistical sampling allows you to test a hypothesis about the characteristics of a population. There are various sampling methods you can use to ensure that your sample is representative of the population as a whole.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2022, October 10). What Is a Research Methodology? | Steps & Tips. Scribbr. Retrieved 11 September 2023, from https://www.scribbr.co.uk/thesis-dissertation/methodology/

Is this article helpful?

Shona McCombes

Other students also liked, how to write a dissertation proposal | a step-by-step guide, what is a literature review | guide, template, & examples, what is a theoretical framework | a step-by-step guide.

Generate accurate APA citations for free

• Knowledge Base
• APA Style 7th edition
• How to write an APA methods section

How to Write an APA Methods Section | With Examples

Published on February 5, 2021 by Pritha Bhandari . Revised on June 22, 2023.

The methods section of an APA style paper is where you report in detail how you performed your study. Research papers in the social and natural sciences often follow APA style. This article focuses on reporting quantitative research methods .

In your APA methods section, you should report enough information to understand and replicate your study, including detailed information on the sample , measures, and procedures used.

Table of contents

Structuring an apa methods section.

Participants

Example of an APA methods section

Other interesting articles, frequently asked questions about writing an apa methods section.

The main heading of “Methods” should be centered, boldfaced, and capitalized. Subheadings within this section are left-aligned, boldfaced, and in title case. You can also add lower level headings within these subsections, as long as they follow APA heading styles .

To structure your methods section, you can use the subheadings of “Participants,” “Materials,” and “Procedures.” These headings are not mandatory—aim to organize your methods section using subheadings that make sense for your specific study.

Note that not all of these topics will necessarily be relevant for your study. For example, if you didn’t need to consider outlier removal or ways of assigning participants to different conditions, you don’t have to report these steps.

The APA also provides specific reporting guidelines for different types of research design. These tell you exactly what you need to report for longitudinal designs , replication studies, experimental designs , and so on. If your study uses a combination design, consult APA guidelines for mixed methods studies.

Detailed descriptions of procedures that don’t fit into your main text can be placed in supplemental materials (for example, the exact instructions and tasks given to participants, the full analytical strategy including software code, or additional figures and tables).

Prevent plagiarism. Run a free check.

Begin the methods section by reporting sample characteristics, sampling procedures, and the sample size.

Participant or subject characteristics

When discussing people who participate in research, descriptive terms like “participants,” “subjects” and “respondents” can be used. For non-human animal research, “subjects” is more appropriate.

Specify all relevant demographic characteristics of your participants. This may include their age, sex, ethnic or racial group, gender identity, education level, and socioeconomic status. Depending on your study topic, other characteristics like educational or immigration status or language preference may also be relevant.

Be sure to report these characteristics as precisely as possible. This helps the reader understand how far your results may be generalized to other people.

The APA guidelines emphasize writing about participants using bias-free language , so it’s necessary to use inclusive and appropriate terms.

Sampling procedures

Outline how the participants were selected and all inclusion and exclusion criteria applied. Appropriately identify the sampling procedure used. For example, you should only label a sample as random  if you had access to every member of the relevant population.

Of all the people invited to participate in your study, note the percentage that actually did (if you have this data). Additionally, report whether participants were self-selected, either by themselves or by their institutions (e.g., schools may submit student data for research purposes).

Identify any compensation (e.g., course credits or money) that was provided to participants, and mention any institutional review board approvals and ethical standards followed.

Sample size and power

Detail the sample size (per condition) and statistical power that you hoped to achieve, as well as any analyses you performed to determine these numbers.

It’s important to show that your study had enough statistical power to find effects if there were any to be found.

Additionally, state whether your final sample differed from the intended sample. Your interpretations of the study outcomes should be based only on your final sample rather than your intended sample.

Write up the tools and techniques that you used to measure relevant variables. Be as thorough as possible for a complete picture of your techniques.

Primary and secondary measures

Define the primary and secondary outcome measures that will help you answer your primary and secondary research questions.

Specify all instruments used in gathering these measurements and the construct that they measure. These instruments may include hardware, software, or tests, scales, and inventories.

• To cite hardware, indicate the model number and manufacturer.
• To cite common software (e.g., Qualtrics), state the full name along with the version number or the website URL .
• To cite tests, scales or inventories, reference its manual or the article it was published in. It’s also helpful to state the number of items and provide one or two example items.

Make sure to report the settings of (e.g., screen resolution) any specialized apparatus used.

For each instrument used, report measures of the following:

• Reliability : how consistently the method measures something, in terms of internal consistency or test-retest reliability.
• Validity : how precisely the method measures something, in terms of construct validity  or criterion validity .

Giving an example item or two for tests, questionnaires , and interviews is also helpful.

Describe any covariates—these are any additional variables that may explain or predict the outcomes.

Quality of measurements

Review all methods you used to assure the quality of your measurements.

These may include:

• training researchers to collect data reliably,
• using multiple people to assess (e.g., observe or code) the data,
• translation and back-translation of research materials,
• using pilot studies to test your materials on unrelated samples.

For data that’s subjectively coded (for example, classifying open-ended responses), report interrater reliability scores. This tells the reader how similarly each response was rated by multiple raters.

Report all of the procedures applied for administering the study, processing the data, and for planned data analyses.

Data collection methods and research design

Data collection methods refers to the general mode of the instruments: surveys, interviews, observations, focus groups, neuroimaging, cognitive tests, and so on. Summarize exactly how you collected the necessary data.

Describe all procedures you applied in administering surveys, tests, physical recordings, or imaging devices, with enough detail so that someone else can replicate your techniques. If your procedures are very complicated and require long descriptions (e.g., in neuroimaging studies), place these details in supplementary materials.

To report research design, note your overall framework for data collection and analysis. State whether you used an experimental, quasi-experimental, descriptive (observational), correlational, and/or longitudinal design. Also note whether a between-subjects or a within-subjects design was used.

For multi-group studies, report the following design and procedural details as well:

• how participants were assigned to different conditions (e.g., randomization),
• instructions given to the participants in each group,
• interventions for each group,
• the setting and length of each session(s).

Describe whether any masking was used to hide the condition assignment (e.g., placebo or medication condition) from participants or research administrators. Using masking in a multi-group study ensures internal validity by reducing research bias . Explain how this masking was applied and whether its effectiveness was assessed.

Participants were randomly assigned to a control or experimental condition. The survey was administered using Qualtrics (https://www.qualtrics.com). To begin, all participants were given the AAI and a demographics questionnaire to complete, followed by an unrelated filler task. In the control condition , participants completed a short general knowledge test immediately after the filler task. In the experimental condition, participants were asked to visualize themselves taking the test for 3 minutes before they actually did. For more details on the exact instructions and tasks given, see supplementary materials.

Data diagnostics

Outline all steps taken to scrutinize or process the data after collection.

This includes the following:

• Procedures for identifying and removing outliers
• Data transformations to normalize distributions
• Compensation strategies for overcoming missing values

To ensure high validity, you should provide enough detail for your reader to understand how and why you processed or transformed your raw data in these specific ways.

Analytic strategies

The methods section is also where you describe your statistical analysis procedures, but not their outcomes. Their outcomes are reported in the results section.

These procedures should be stated for all primary, secondary, and exploratory hypotheses. While primary and secondary hypotheses are based on a theoretical framework or past studies, exploratory hypotheses are guided by the data you’ve just collected.

This annotated example reports methods for a descriptive correlational survey on the relationship between religiosity and trust in science in the US. Hover over each part for explanation of what is included.

The sample included 879 adults aged between 18 and 28. More than half of the participants were women (56%), and all participants had completed at least 12 years of education. Ethics approval was obtained from the university board before recruitment began. Participants were recruited online through Amazon Mechanical Turk (MTurk; www.mturk.com). We selected for a geographically diverse sample within the Midwest of the US through an initial screening survey. Participants were paid USD $5 upon completion of the study.

A sample size of at least 783 was deemed necessary for detecting a correlation coefficient of ±.1, with a power level of 80% and a significance level of .05, using a sample size calculator (www.sample-size.net/correlation-sample-size/).

The primary outcome measures were the levels of religiosity and trust in science. Religiosity refers to involvement and belief in religious traditions, while trust in science represents confidence in scientists and scientific research outcomes. The secondary outcome measures were gender and parental education levels of participants and whether these characteristics predicted religiosity levels.

Religiosity

Religiosity was measured using the Centrality of Religiosity scale (Huber, 2003). The Likert scale is made up of 15 questions with five subscales of ideology, experience, intellect, public practice, and private practice. An example item is “How often do you experience situations in which you have the feeling that God or something divine intervenes in your life?” Participants were asked to indicate frequency of occurrence by selecting a response ranging from 1 (very often) to 5 (never). The internal consistency of the instrument is .83 (Huber & Huber, 2012).

Trust in Science

Trust in science was assessed using the General Trust in Science index (McCright, Dentzman, Charters & Dietz, 2013). Four Likert scale items were assessed on a scale from 1 (completely distrust) to 5 (completely trust). An example question asks “How much do you distrust or trust scientists to create knowledge that is unbiased and accurate?” Internal consistency was .8.

Potential participants were invited to participate in the survey online using Qualtrics (www.qualtrics.com). The survey consisted of multiple choice questions regarding demographic characteristics, the Centrality of Religiosity scale, an unrelated filler anagram task, and finally the General Trust in Science index. The filler task was included to avoid priming or demand characteristics, and an attention check was embedded within the religiosity scale. For full instructions and details of tasks, see supplementary materials.

For this correlational study , we assessed our primary hypothesis of a relationship between religiosity and trust in science using Pearson moment correlation coefficient. The statistical significance of the correlation coefficient was assessed using a t test. To test our secondary hypothesis of parental education levels and gender as predictors of religiosity, multiple linear regression analysis was used.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Measures of central tendency
• Chi square tests
• Confidence interval
• Quartiles & Quantiles

Methodology

• Cluster sampling
• Stratified sampling
• Thematic analysis
• Cohort study
• Peer review
• Ethnography

Research bias

• Implicit bias
• Cognitive bias
• Conformity bias
• Hawthorne effect
• Availability heuristic
• Attrition bias
• Social desirability bias

In your APA methods section , you should report detailed information on the participants, materials, and procedures used.

• Describe all relevant participant or subject characteristics, the sampling procedures used and the sample size and power .
• Define all primary and secondary measures and discuss the quality of measurements.
• Specify the data collection methods, the research design and data analysis strategy, including any steps taken to transform the data and statistical analyses.

You should report methods using the past tense , even if you haven’t completed your study at the time of writing. That’s because the methods section is intended to describe completed actions or research.

In a scientific paper, the methodology always comes after the introduction and before the results , discussion and conclusion . The same basic structure also applies to a thesis, dissertation , or research proposal .

Depending on the length and type of document, you might also include a literature review or theoretical framework before the methodology.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, June 22). How to Write an APA Methods Section | With Examples. Scribbr. Retrieved September 11, 2023, from https://www.scribbr.com/apa-style/methods-section/

Is this article helpful?

Pritha Bhandari

Other students also liked, how to write an apa results section, apa format for academic papers and essays, apa headings and subheadings, scribbr apa citation checker.

An innovative new tool that checks your APA citations with AI software. Say goodbye to inaccurate citations!

Compendium for Early Career Researchers in Mathematics Education pp 181–197 Cite as

Qualitative Text Analysis: A Systematic Approach

• Udo Kuckartz 4  
• Open Access
• First Online: 27 April 2019

101k Accesses

125 Citations

Part of the ICME-13 Monographs book series (ICME13Mo)

Thematic analysis, often called Qualitative Content Analysis (QCA) in Europe, is one of the most commonly used methods for analyzing qualitative data. This paper presents the basics of this systematic method of qualitative data analysis, highlights its key characteristics, and describes a typical workflow. The aim is to present the main characteristics and to give a simple example of the process so that readers can assess whether this method might be useful for their own research. Special attention is paid to the formation of categories, since all scholars agree that categories are at the heart of the method.

• Qualitative data analysis
• Text analysis
• Qualitative methods
• Qualitative content analysis
• MAXQDA software

Download chapter PDF

1 Introduction: Qualitative and Quantitative Data

Thematic analysis, often called Qualitative Content Analysis (QCA) in Europe, is one of the most commonly used methods for analyzing qualitative data (Guest et al. 2012 ; Kuckartz 2014 ; Mayring 2014 , 2015 ; Schreier 2012 ). This chapter presents the basics of this systematic method of qualitative data analysis, highlights its key characteristics, and describes a typical workflow.

Working with codes and categories is a proven method in qualitative research. QCA is a method that is reliable, easy to learn, transparent, and it is a method that is easily understood by other researchers. In short, it is a method that enjoys a high level of recognition and is to be highly recommended, especially in the context of dissertations.

The aim of this paper is to present the main characteristics and to give a simple example of the process so that readers can assess whether this method might be useful for their own research. Special attention is paid to the formation of categories, since all scholars agree that categories are at the heart of the method.

Let’s start with some of the basics of data analysis in empirical research: What does ‘qualitative data’ mean, and what do we mean by ‘quantitative data’? Quantitative data entail numerical information that results, for example, from the collection of data from a standardized interview. In a quantitative data matrix, each row corresponds to a case, namely, an interview with a respondent. The columns of the matrix are formed by the variables. Table  8.1 therefore shows the data of four cases, here the respondents 1–4. Six variables were collected for these individuals, on a scale of 1–6, concerning how often they perform certain household activities (laundry, small repairs etc.). Typically, these kinds of data sets are available in social research in the form of a rectangular matrix, for instance as shown in Table  8.1 .

A matrix like this that consists of numbers can be analyzed using statistical methods. For example, you can calculate univariate statistics such as mean values, variance, and standard deviations. You can also generate graphical displays such as box plots or bar charts. In addition, variables can be related to each other, for example by using methods of correlation and regression statistics. Another form of analysis tests groups for differences. In the above study, for example, the questions ‘Are women more frequently engaged in laundry than men in the household?’ and ‘Are men more frequently engaged in minor repairs than women in the household?’ can be calculated using an analysis of variance.

Qualitative data are far more diverse and complex than quantitative data. These data may comprise transcripts of face-to-face interviews or focus group discussions, documents, Twitter tweets, YouTube comments, or videos of the teacher-student interactions in the classroom.

In this chapter, I restrict the presentation of the QCA method to a specific type of data, namely qualitative interviews. This collective term can be used to describe very different forms of interviews, such as guideline-assisted interviews or narrative interviews on critical life events conducted in the context of biographical research. The latter can last several hours and comprise more than 30 pages as a transcription. A qualitative interview may also consist of a short online survey, like the one I conducted in preparation for my workshop at the International Congress on Mathematical Education (ICME-13).

Obviously, the different types of qualitative data are not as easy to analyze as the numbers in a quantitative data matrix. Numerous analytical methods have been developed in qualitative research, among them the well-proven method of qualitative content analysis.

2 Key Points of Qualitative Content Analysis

What are the key points of the qualitative content analysis method? Regardless of which variant of QCA is used, the focus will always be on working with categories (codes) and developing a category system (coding frame). What Berelson formulated in 1952 for quantitative content analysis still applies today, both to quantitative and qualitative content analysis:

Content analysis stands or falls by its categories (…) since the categories contain the substance of the investigation, a content analysis can be no better than its system of categories. (Berelson 1952 , p. 147)

Categories are therefore of crucial importance for effective research, not only in their role as analysis tools, but also insofar as they form the substance of the research and the building blocks of the theory the researchers want to develop. That raises the question ‘What are categories?’—or more precisely, ‘What are categories in the context of empirical social research?’ Answering this question is by no means easy and there are at least two ways of doing so. The first way can be described as phenomenological : Kuckartz ( 2016 , pp. 31–39) focuses on the use of this term in the practice of empirical social research, i.e., drawing attention to what is called a category in empirical social research. The result of this analysis is a very diverse spectrum, whereby several different types of categories can be distinguished in social science research literature (ibid., pp. 34–35):

Factual categories denote actual or supposed objective circumstances such as ‘length of training’ or ‘occupation’.

Thematic categories refer to certain topics, arguments, schools of thought etc. such as ‘inclusion’, ‘environmental justice’ or ‘Ukrainian conflict’.

Evaluative categories are related to an evaluation scale—usually ordinal types, for example the category ‘helper syndrome’ with the characteristics ‘not pronounced’, ‘somewhat pronounced’ and ‘pronounced’. For evaluative categories, it is the researchers who classify the data according to predefined criteria.

Analytical categories are the result of intensive analysis of the data, i.e., these categories move away from the description of the data, for example by means of thematic categories.

Theoretical categories are subspecies of analytical categories that refer to an existing theory, such as Ajzen’s theory of planned behavior, Ainsworth’s attachment theory, or Foucault’s analysis of power.

Natural categories , also called “in vivo codes” (Charmaz 2006 , p. 56; Kuckartz 2014 , p. 23), are terms used by the actors in the field.

Formal categories denote formal characteristics of an analysis unit, e.g., the length of time in an interview.

The above list is not complete; there are many more types of categories and corresponding methods of coding (Saldana 2015 ).

A second way of answering the question ‘What is a category?’ can be described as conceptual and historical; this way leads us far back into the history of philosophy. The conceptual historical view of the term, originating from ancient Greece, starts with Greek philosophy more than 2000 years ago. Plato and Aristotle already dealt with categories—Aristotle even in an elaboration of the same term (“categories”). The study of categories runs through Western philosophy from Plato and Kant to Peirce and analytical philosophy. The philosophers are by no means in agreement on the concept of categories, but a discussion of the differences between the different schools would far exceed the scope of this paper; Instead, reading the mostly very extensive contributions on the terms ‘category’ and ‘category theory’ in the various lexicons of philosophy is recommended. Categories are basic concepts of cognition; they are—generally speaking—a commonality between certain things: a term, a heading, a label that designates something similar under certain aspects. Categories also play this role in content analysis, as the following quote from the Content Analysis textbook of Früh ( 2004 ) demonstrates:

The pragmatic sense of any content analysis is ultimately to reduce complexity from a certain research-led perspective. Text sets are described in a classifying manner with regard to characteristics of theoretical interest. In this reduction of complexity, information is necessarily lost: On the one hand, information is lost due to the suppression of message characteristics that are present in the examined texts but are not of interest in connection with the present research question; on the other hand, information is lost due to the classification of the analyzed message characteristics. According to specified criteria, some of them are each considered similar to one another and assigned to a certain characteristic class or a characteristic type, which is called ‘category’ in the content analysis. The original differences in meaning of the message characteristics uniformly grouped in a category shall not be taken into account. (p. 42, translated by the author)

But how does qualitative content analysis arrive at its categories, the basic building blocks for forming theory? There are three principal ways to develop categories:

Concept-driven (‘deductive’) development of categories; in this case the categories

are derived from a theory or

derived from the literature (the current state of research) or

derived from the research question (e.g. directly related to an interview guide)

Data-driven (‘inductive’) development of categories; the characteristics here are

the step-by-step procedure,

the method of open coding until saturation occurs,

the continuous organization and systematization of the formed codes, and

the development of top-level codes and subcodes at different levels.

Mixing a concept-driven and data-driven development of codes:

The starting point here is usually a coding frame with deductively formed codes and

the subsequent inductive coding of all data coded with a specific main category.

The terms deductive and inductive are often used for the concept-driven and data-driven approaches, respectively. However, the use of the term ‘deductive’ is rather problematic in this context: In scientific logic, the term ‘inductive’ refers to the abstract conclusion from what has been observed empirically to a general rule or a law; this has little to do with the formation of categories based on empirical data. The situation is similar with the term ‘deductive’: In scientific logic, the deductive conclusion is a logical consequence of its premises; the formation of categories based on the state of research, a theory, or an advanced hypothesis is very different. Categories do not necessarily emerge from a systematic literature review or from a research question. Due to its skid resistance, however, the word pair ‘inductive-deductive’ will probably remain in the language theorem of empirical social research or the formation of categories for a long time to come. Nevertheless, I try to avoid the terms inductive and deductive, and—like Schreier ( 2012 , p. 84)—prefer the terms ‘data-driven’ and ‘concept-driven’ for these different approaches to the formation of categories.

The decisive action in QCA is the coding of the data, i.e. a precisely defined part of the material is selected, and a category is assigned. As shown in the following figure, this may be a passage from an interview. Here, paragraph 15 of the text was coded with the code Simultaneousness (Fig.  8.1 ).

Text passage with a coded text segment

The individuals who perform this segmentation and coding of the data are referred to as coders. In this context, we also speak of “inter- and intracoder agreement” (reliability) (Krippendorff 2012 ; Kuckartz 2016 ; Schreier 2012 ). In quantitative content analysis, the units to be coded are usually defined in advance and referred to as coding units. In qualitative content analysis, on the other hand, coding units are not usually defined in advance; they are created by the coding process.

The general workflow of a qualitative content analysis is in Fig.  8.2 . In all variants the research question plays the central role in this method: It provides the perspective for the textual work necessary at the beginning, that is, the intensive reading and study of the texts (Kuckartz 2016 , p. 45). For qualitative methods, it is common for the individual analysis phases to be carried out on a circular basis. This also applies to QCA: The creation of categories and subcategories and the coding of the data can take place in several cycles. Saldana ( 2015 ) speaks of first cycle coding and second cycle coding, for example. The number of cycles is not fixed, and only in rare cases would one get by with just a single cycle.

The five phases of qualitative content analysis

Once all the data have been coded with the final category frame, a systematization and structuring of all the relevant data in view of the research questions at hand will have been achieved. Table  8.2 illustrates a model of such a thematic matrix. It is similar to the quantitative data matrix shown in Fig.  8.2 , but instead of containing numbers, the cells of the matrix now contain text excerpts coded with the respective corresponding category.

The further analysis of the matrix can now take two directions: If you look at columns, you can examine certain topics. These forms of analysis can be described as ‘category-based’. Looking at the rows, you can focus on cases (people) and carry out a ‘case-oriented analysis’.

Category-based analyses can focus on a specific category or even consider several categories simultaneously. For example, the statements made by the research participants can be contrasted between two or across several topics. Such complex analyses can lead to very rich descriptions or to the determination of influencing factors and effects, which can then be displayed in a concept map. Case-oriented analyses allow you to identify similarities between cases, identify extreme cases, and form types. Methods of consistently comparing and contrasting cases can be used to this end. For example, if you have determined a typology, you can then visualize it as a constellation of clusters and cases.

3 The Analysis Process in Detail

The example used in the following is a short online survey conducted in preparation for the ‘Workshop on qualitative text analysis’ as part of the ICME 13. The aim of the survey was to provide an overview of the research needs of the participants and their level of knowledge. In other words, its aim was descriptive and not about the development of hypotheses or a theory. In this online interview, I asked the following five questions and asked the participants to write their responses directly below the questions. Table  8.3 contains the resulting qualitative data.

Typically, QCA consists of six steps

Step 1: Preparing the data, initiating text work

Step 2: Forming main categories corresponding to the questions asked in the interview

Step 3: Coding data with the main categories

Step 4: Compiling text passages of the main categories and forming subcategories inductively on the material; assigning text passages to subcategories

Step 5: Category-based analyses and presenting results

Step 6: Reporting and documentation.

Since the purpose of the survey in this case was to get an overview of the relevant interests of the workshop participants and to tailor the workshop to their needs, the last step was omitted. There was no need for reporting and documentation.

The first phase consists of preparing of the data and conducting an initial read-through the responses; the analysis of this short survey did not require extensive interpretation of the responses. Since respondents used different fonts and font sizes in their e-mails, these had to be standardized first when preparing the data. In addition, the overall formatting was also adjusted to render it more uniform across responses. This would not have been absolutely necessary for the analysis, but without this preparation, later compilations of coded text passages might have looked rather chaotic.

In the second phase of QCA, categories are formed. When analyzing data obtained through an online survey, it is best to create a set of main categories based on the questions asked. In this analysis, the following five categories were formed for the first coding cycle:

Motives and goals

Experience with QCA

Specific questions about QCA

Experience with QDAS ( Q ualitative d ata a nalysis s oftware)

Academic discipline.

Since the questions in the online survey were numbered, the numbers were retained for better orientation, but they could have been dispensed with without any problems.

According to the differentiation of categories laid out earlier in this paper, the categories Motives and goals and Specific questions about QCA are thematic categories. Category 5 Academic Discipline is a factual code. The other two categories Experience with QCA and Experience with QDAS are about the experiences with the method and with QDA software. If the researcher is interested in the extent of participants’ experience, both categories are evaluative categories; alternatively, if the specific type of experience is the primary point of interest, the categories are thematic. Since the aim of this survey was to get an overview of the level of knowledge and practical experience of the respondents, an overview was sufficient; detailed knowledge of the types of experience the participants had gained was not absolutely necessary. Reading the responses also demonstrated that the respondents understood the question in this sense and that in most cases no specific details were provided. In any case, working with software like MAXQDA guarantees that you can always return to the original texts should this be useful or necessary during the course of the analysis.

In the third phase of the analysis, the corresponding text segments are coded with the five main categories. Figure  8.3 shows a screenshot of the software MAXQDA after this first cycle of coding was performed on the survey responses. The assignments of the codes are displayed to the left of the corresponding text sections.

Display of a text with code assignments after the first cycle of coding

In the following fourth phase of the analysis, the coding frame is developed further. To do this, all the text passages coded with one of the main categories are first compiled, a procedure which is also referred to as retrieval . Subcodes are then developed directly in the relation to this data—in other words, the creation of categories is data-driven. This process is described in the following with regard to the first main category Motives and goals :

The category Motives and goals coded the responses to the question regarding what the participants wanted to learn in the workshop. First, all text passages to which this category was assigned were compiled. Then each of these text passages was coded a second time. This was done with a procedure similar to that of open coding in Grounded Theory (Strauss and Corbin 1990 ). In this case, the codes were short sequences of words that described what the participants wanted to learn:

analyze mathematics textbook curricula

learn type-building analysis

analyze e-portfolios and group discussions

analyze responses to open-ended questions

learn more about different research methods

how to establish credibility in practice

learn more about rigor within the process and how to ensure its validity

the role of reliability coefficients

insight into conducting qualitative research

learn about the QCA method

how to code video transcripts

how to take the richness of data into account (not only numbers)

analyze large numbers of open questions

learn more about a few different approaches to choose from

searching for a suitable method to analyze the interviews

interesting for me to see how colleagues are working.

As part of the software MAXQDA there is a module called “Creative Coding” that allows you to visually group codes obtained through the open coding method. After arranging the open codes, seven subcategories were created for the category “Motives and Objectives”, namely

Getting an overview of qualitative research

Getting an overview of QCA

Learning basic techniques

Learning about special type of analysis

Reliability and validity

Learning to analyze special types of data

Interesting for me to see how colleagues are working.

Figure  8.4 shows a visual display of the category formation; the original statements are assigned to the respective category. It turns out that many participants in the workshop were mainly interested in obtaining an overview of qualitative content analysis and qualitative research in general. The graph also implicitly illustrates the differences between a quantitative and qualitative analysis of the responses: Four participants (a comparatively large proportion) wanted to learn how to analyze specific types of data, but a closer look at the details, that is, the qualitative dimension, reveals that the types of data the respondents had in mind were completely different.

Visualization of the motives grouped into subcategories

Once the subcategories have been created, all the data coded with the main category Motives and goals must be coded a second time. This is also known as the second coding cycle. In this sample survey, all the coded text passages were included in the formation of the subcategories due to the relatively small sample. In the case of small sample sizes like this, the Creative Coding module automatically reassigns the subcategories. In the case of larger samples, however, category formation will usually be carried out only with a subsample and not with all the data, or the process of open coding will be performed only until the system of subcategories appears saturated and no further subcategories need to be redefined. Then, of course, the data that have not been considered up to this point must still be coded in line with the final category system.

The two categories Experience with QCA and Experience with QDAS were used to code the text passages in which the respondents reported on their experience with the QCA method and the use of QDA software. For the purposes of preparing the workshop as described above, the analysis should address only whether participants had prior experience and how extensive this experience was. An evaluative category with the values ‘yes’, ‘partial’, ‘no’ was therefore defined.

For the third main category, Specific questions about QCA , no subcategories were formed, since the questions formulated by the participants had to be retained in their wording to answer them in the workshop. However, the questions asked were sorted by topic, and essentially identical questions were summarized.

For category 5, Academic discipline , subcategories were initially formed according to the disciplines mentioned by the respondents. However, it quickly transpired that almost all participants came from the field of mathematics education and that there were only a few individual cases from other fields such as development psychology or primary school teacher (see Fig.  8.5 ). These individual cases were combined into the subcategory others for the final category system, so that ultimately only two subcategories were formed.

Main category “Academic discipline and status” with subcategories

After the main categories have been processed in this way—five in the case of this survey—the fifth phase ‘Category-based analyses and presenting results’ can begin. However, it should be clear that in the fourth phase of the development of the category system, an extensive amount of analytical work has already being carried out. The identification of the different motive types represents an analytical achievement in itself and is, at the same time, the foundation of the corresponding category-based analysis in phase 5. The category Motives and goals was of central importance in this survey. In addition to identifying the various motives, both quantitative and qualitative analyses can now be carried out. Quantitatively, we can determine how many people expressed which motives in their statement. Of course, it is quite possible for someone to have expressed several motives. In terms of a qualitative analysis, we can ask what is behind these categories in greater detail. In relation to the subcategory Learning to analyze special types of data , for example, we could ask which special data types the respondents had in mind here.

The category-based analysis always offers the option of focusing on qualitative and/or quantitative aspects. A frequency analysis of the category Experience with QDAS shows that the vast majority of participants have not yet had any practical experience with QDA software (see Fig.  8.6 ).

Bar chart of the category “Experiences with QDA software”

The question concerning their experience with text analysis methods presents a somewhat different picture. Quantitatively, we can see that more people are experienced in this regard, while the more detailed qualitative view reveals that this experience mainly involved the Grounded Theory method. It is interesting to compare the two categories that deal with experience. Table  8.4 contains an excerpt from such a comparison between five people.

There are also many further possibilities regarding the analysis of interrelationships that can be carried out in this fifth phase. For example, the connection between motives and goals, and previous knowledge and experience, can be examined. In relation to the specific questions asked by respondents in the survey, one could create a cross table (or “crosstab”) in which the questions asked by the experienced group are compared with the questions asked by those with no experience.

There are many other analysis options for larger studies than those presented for the small online survey. Qualitative content analysis is not a method that is always applied in the same way regardless of the data or research questions at hand. Although it is a systematic procedure, it nonetheless offers a flexibility that allows you to adapt it to the respective requirements of a project. There are other analytical possibilities in this regard, which were not mentioned in the above description. Among these, two should be highlighted in particular, namely, the possibility of paraphrasing text passages and the possibility of creating thematic summaries.

Paraphrasing passages of text can be understood in its everyday sense, namely, that researchers reformulate these text passages in their own words. This can be a very useful tool for category development. This technique is especially recommended for beginners, as it forces them to read the text line by line, interpret it to gain a thorough understanding, and then record it in their own words. It is certainly too time-consuming in most cases to edit all texts in this way but paraphrasing a selected subset of texts can sharpen your analytical view and be a valuable intermediate step in the development of a meaningful category system. Moreover, these paraphrases can then be sorted, particularly significant paraphrases can be combined, and gradually more abstract and theoretically rich categories can be formed.

In contrast to paraphrasing texts, formulating thematic summaries assumes that the texts have already been coded. In this approach, all the text passages coded in regard to a specific topic are read for each case and a thematic summary is written for each person. Usually, there is a huge gap between a category and the amount of original text assigned to it in the case of longer qualitative interviews, such as narrative interviews. On the one hand there is a relatively short code, such as ‘Environmental behavior in relation to nutrition’, and on the other there are numerous passages of varying length in which a respondent says something on this subject. A thematic summary summarizes all these passages as said by a certain person from the perspective of the research question. This means that the text is not repeated, but rather edited conceptually. Summaries thus create a second level between the original text and the categories and concepts. They also enable complex analyses to be carried out in which several categories are compared or the statements of different groups (women/men, different age groups, different schooling, etc.) are contrasted. This would be nigh impossible if the original quotations were always used since the amount of text would simply be too large, and it would consequently not be possible to create case overviews. A thematic summary, on the other hand, compresses what one person has said in such a way that it can easily be included in further analyses.

A third possibility the QCA method offers is the visualization of relationships between categories. Diagrams, in the form of concept maps, can be generated in which the influencing factors, effects, and relations are visualized.

Phase 6, ‘Reporting and documentation’, is about putting the results of your analyses on paper. The research report of a project working with the QCA method is usually divided into a descriptive and an analytical section. Depending on the method and the significance of the categories, category-based analyses will be the center of attention. The case dimension, however, which is all too often neglected, should also be taken into account in the report. It is often very valuable for the recipients of the research not only to learn something about the connections between the categories, but also something about the participants, that is, the cases that are consciously selected for such a presentation. It is particularly interesting if the cases are grouped into types and the report presents cases that are representative of these types.

The category-based presentation should be illustrated with quotes from the original material. However, you should also be aware of the danger of selective plausibility, i.e., that one mainly selects quotations that clarify the alleged connections between categories, while contradictory examples are not considered. For this reason, counterexamples should always be sought and included in the report.

Category-based analysis should not be limited to a description of the results per category but should also look at the relationships between two or more categories. In other words, you should move from the initial description to the development of a theory.

4 Summary and Conclusions

This chapter presents a method for the methodically controlled analysis of texts in empirical research. To conclude, therefore, the characteristics of the QCA method are concisely summarized:

The focus of the QCA method is on the categories with which the data are coded.

The categories of the final coding frame are described as precisely as possible and it is ensured that the coding procedure itself is reliable, i.e., that different coders concur in their coding.

The data must be coded completely. Complete in this sense means that all passages in the texts that are relevant to the research question are coded. It does, however, make sense to leave those parts of the data uncoded, which are outside the focus of the research question.

The codes and categories can be formed in different ways: empirically, i.e., based directly on the material, or conceptually, i.e., based on the current state of research or on a theory/hypothesis or, rather, as an implementation of the guidelines used in an interview or focus group.

The QCA method is carried out in several phases, ranging from data preparation, category building and coding—which may run in several cycles—to analysis, report writing and presenting the results. QCA therefore means more than just coding the data. Coding is an important step in the analysis, but it is ultimately a preparation for the subsequent analytical steps.

The actual analysis phase consists of summarizing the data, and constantly comparing and contrasting the data. The analysis techniques can be qualitative as well as quantitative. The qualitative analysis may, for example, consist of comparing the statements of certain groups (for instance according to their characteristics, e.g., socio-demographic characteristics) on certain topics. Differences and similarities are identified and summarized in a report. Quantitative analyses may, on the other hand, consist of comparing the frequency of certain categories and/or subcategories for certain groups.

Summary tables and diagrams (e.g., concept maps) can play an important role in the analysis. A good example of a presentation in table form would be a case overview of selected research participants (or groups), in which their statements on certain topics, their judgements and variable values are displayed. An example of a concept map would be a diagram of the determined causal effects of different categories.

Visualizations can also have a diagnostic function in QCA—similarly to imaging procedures in medicine. For example, a ‘cases by categories’ or ‘categories by categories’ display can help identify patterns in the data and indicate which categories are particularly frequently or particularly rarely associated with certain other categories.

When analyzing texts, you should keep in mind that you are working in the field of interpretation. It can be assumed that texts or statements could be interpreted differently. Instead of adopting a constructivist ‘anything goes’ approach, the QCA method tries to reach a consensus—as far as this is possible—on the subjective meaning of statements and tries to define the categories formed or used by it so precisely that an intersubjective agreement can be achieved in the application of the categories.

Group processes play an important role in this process of achieving the necessary level of agreement. Divergent assignments to categories are discussed as a team and should result in an improvement of the category definitions. Categories for which no agreement can be reached in the coding of relevant points in the data must be excluded from the analysis. Content analysis stands and falls by its categories. An analysis with the help of categories that are interpreted and applied differently in the research team, does not make sense.

QCA does not claim to be the best method but recognizes that it has its limits (the interpretation barrier) and that its results have to face comparison with those of competing methods.

The systematic approach of QCA is multidisciplinary and can be applied in many disciplines, including mathematics education (Schwarz 2015 ). This method is particularly appropriate when working with clearly formulated research questions, because these questions play the central role in this method. Indeed, in every phase of the analysis there is a strong reference to the questions leading the research. One strength of QCA is that it can be used both to describe social phenomena and to develop theories or test hypotheses (Hopf 2016 , pp. 155–166).

Berelson, B. (1952). Content analysis in communication research . Glencoe: Free Press.

Google Scholar  

Charmaz, K. (2006). Constructing grounded theory . Thousand Oaks: SAGE.

Früh, W. (2004). Inhaltsanalyse. Theorie und Praxis (5th ed.). Konstanz: UVK.

Guest, G., MacQueen, K. M., & Namey, E. E. (2012). Applied thematic analysis . Thousand Oaks: SAGE.

CrossRef   Google Scholar  

Hopf, C. (2016). In W. Hopf & U. Kuckartz (Eds.), Schriften zu Methodologie und Methoden qualitativer Sozialforschung . Wiesbaden: Springer.

Krippendorff, K. H. (2012). Content analysis: An introduction to its methodology (3rd ed.). Thousand Oaks: SAGE.

Kuckartz, U. (2014). Qualitative text analysis: A guide to methods, practice and using software . Los Angeles: SAGE.

Kuckartz, U. (2016). Qualitative Inhaltsanalyse. Methoden, Praxis, Computerunterstützung (3rd ed.). Weinheim: Beltz Juventa.

Mayring, P. (2014). Qualitative content analysis: Theoretical foundation, basic procedures and software solution. Klagenfurt. http://nbn-resolving.de/urn:nbn:de:0168-ssoar-395173 .

Mayring, P. (2015). Qualitative content analysis: Theoretical background and procedures. In A. Bikner-Ahsbahs, C. Knipping, & N. Presmeg (Eds.), Approaches to qualitative research in mathematics education. Examples of methodology and methods (pp. 365–380). Dordrecht: Springer.

Saldana, J. (2015). The coding manual for qualitative researchers (3rd ed.). Thousand Oaks: SAGE.

Schreier, M. (2012). Qualitative content analysis in practice . Thousand Oaks: SAGE.

Schwarz, B. (2015). A study on professional competence of future teacher students as an example of a study using qualitative content analysis. In A. Bikner-Ahsbahs, C. Knipping, & N. Presmeg (Eds.), Approaches to qualitative research in mathematics education. Examples of methodology and methods (pp. 381–399). Dordrecht: Springer.

Strauss, A. L., & Corbin, J. M. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Thousand Oaks: Sage Publications, Inc.

Download references

Author information

Authors and affiliations.

Philipps-University Marburg, Marburg, Germany

Udo Kuckartz

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Udo Kuckartz .

Editor information

Editors and affiliations.

Faculty of Education, Universität Hamburg, Hamburg, Germany

Prof. Dr. Gabriele Kaiser

Mathematics Department, Illinois State University, Normal, IL, USA

Dr. Norma Presmeg

Rights and permissions

Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Reprints and Permissions

Copyright information

© 2019 The Author(s)

About this chapter

Cite this chapter.

Kuckartz, U. (2019). Qualitative Text Analysis: A Systematic Approach. In: Kaiser, G., Presmeg, N. (eds) Compendium for Early Career Researchers in Mathematics Education . ICME-13 Monographs. Springer, Cham. https://doi.org/10.1007/978-3-030-15636-7_8

Download citation

DOI : https://doi.org/10.1007/978-3-030-15636-7_8

Published : 27 April 2019

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-15635-0

Online ISBN : 978-3-030-15636-7

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us

• Plagiarism checker Do The Check
• Academic editing Ask For Help
• Samples database View Samples Base

The Best Analytical Essay Writing Guide

21 Feb 2021

Quick Navigation

❓What Is an Analytical Essay?

☝️Purpose of an Analytical Essay

📑Structure of an Analytical Essay

✅Formatting of an Analytical Essay

✍️Creating an Analytical Essay

Complex, requiring a lot of time, effort, deep theoretical knowledge, and the ability to process them - all this is about an analytical essay. The main task of our team is to use this article to reveal a few secrets, explain the structure of writing, and thus make it easier for you to work on your paper.

Every student comes across this type of essay and often feels lost and unable to write because analytical work is a small scientific study, and sometimes the student does not have enough experience and qualifications to cope with it.

Is writing essays your hobby?

Participate in our "Independence Day of the United States" essay writing competition and get a 12-month Quizlet subscription.

• Deadline: July 24, 2023
• Topic: Declaration of Independence
• Language: English
• Length: 1000-5000 words
• Font size: 11 or 12

What Is an Analytical Essay?

An analytical essay is a written work that consists of the analysis of some literary work, scientific research, or issue. The object of such work can be the topic of any subject that requires analysis. Based on this, the subjects in which a student may encounter an analytical essay range from history and literature to chemistry and physics.

As in every essay, the student is introduced to the topic, but in the case of an analytical essay, he must perform an individual analysis, which means that the direction of the development of the topic depends entirely on the author. This is a great opportunity to show your ingenuity and express your observations. So in further paragraphs, we will find out how to write an analytical essay.

Need help with writing analytical essay?

Get your paper written by a professional writer

What Is the Purpose of an Analytical Essay?

You can distinguish two sides of essay writing and determine their goals. The first part is about a teacher who asks to write an analytical essay. Its purpose is to test the student's deep knowledge of the topic and assess his ability to process information. An analytical essay is a work in which the student has a base in the form of a topic or object, and he must develop all the rest of the information on his own.

The other side is the student. The student's goal is to put his thoughts on paper as clearly and accurately as possible and to show his cognitive abilities. Perform an analysis on a given topic, draw an appropriate conclusion, and develop your idea. Also, the goal is to develop writing skills, the use of calculations, and the ability to choose the right professional vocabulary.

How to Structure Your Analytical Essay

This essay structure of such doesn't vary a lot from the other types. When writing an essay, in most cases, the teacher gives you a clear task with outlined requirements and a topic. Carefully read the materials regarding this topic, study the theoretical base, and develop your conclusions regarding the issue. The first step will be to create a plan. You should draw up an essay plan so that it becomes a support for you, and this will help you gather your thoughts and not forget anything. It may seem hard at first glance to understand how to write a critical analysis , but trust us, we'll help you find a perfect essay. Let's stick to the 5-paragraph essay and have a deeper look at its structure:

Introduction

The introduction should provide the reader with basic information about the problem. Commence your essay with a thesis statement. You may have a question about how to write a thesis statement for an analytical essay. We will tell you how. The thesis should tell the reader what your paper will be about. Such a proposal will help the reader immediately understand the essence of the matter and form an opinion about your work. The introduction should also contain a key phrase, be short, and aim to attract the reader's attention. As you learn about how to write a good hook  for an analytical essay, you'll find helpful tips like using statistics that might shock you. Be very specific in your introduction, don't generalize or use clichés. Try to be creative to give the essay novelty.

Body Sections

Your task is to depict all your deep ideas in the main paragraph. Starting from the plan of your essay, design 3-4 paragraphs, each of which will present a new idea. This way, the information will be easier to read, and your thought process will be clearer to the reader. Remember the reinforcement base. Give examples of quotes and statements, and you can also refer to proven evidence sources. Feel free to spend more time working through each item.

In the final paragraph, summarize your analysis. How to write a conclusion for an analytical essay? The answer is to tell the reader what conclusions you came to as a result of your research. Make sure the information you provide is accurate. Be concise so the reader can easily understand and use the fruits of your work. You must show how your arguments are related to the main idea of ​​the work and how your opinion can change the reader's judgment.

When you know the basic criteria for writing an analytical essay, let's take a closer look at an example plan. You can later use it as a layout for your work. This will give you a foundation and help structure your analysis. Don't forget you can always ask us to write my essay for me .

Title : Should sexual education be tough in school? Introduction : Statistics show that 75% of pregnancies in America were unintended among teens aged 15 to 19 years. (Thesis statement) Lack of awareness during puberty leads to an increase in the number of missed pregnancies among adolescents. The only way to prevent an increase in such cases is the introduction of a sex education course for middle and high school students. (Further elaboration of the topic) Body sections : Paragraph 1: If you don't talk about sex, it doesn't mean sex doesn't exist. Paragraph 2: Education on sexual topics will prevent unintended pregnancies and sexual abuse.  Paragraph 3: The importance of support from parents and school teachers. The normalization of sexual topics in society. Conclusion : Describe the conclusions you managed to find. Rely on the statistics that you managed to find. Explain why sex education is so important in schools.

Take your paper to the next level

Professional editors will check your paper for grammar, punctuation, sentence structure, consistency, and academic style.

How to Format an Analytical Essay

Like any other academic essay, analytical work follows a specific format. Introduction, 3-4 paragraphs of the main part, and conclusion. Each of these parts is accompanied by an internal structure, which we discussed earlier. Also, many essays require a title that briefly describes the topic of the work.

As for text formatting, scientific texts use the 12th Times New Roman font size, double spacing, and one-inch margins. Also, starting every new paragraph with an indentation of five spaces is required. Remember to use linking words for a smooth transition from idea to idea, paragraph to paragraph, and part to part.

How to Create an Analytical Essay

In this part of our article, we will get acquainted with useful tips from professionals, analyze the main difficulties and share with you the secrets for quick and pleasant writing of an analytical essay. Buckle up, and it will be very informative.

We sincerely advise you to read our material in detail without missing a single point. Thus, the information you receive during the reading will give you the desired five and recognition from the teacher. Each of the following items contributes to your ebb score. Without further ado, let's get down to business.

• If possible, choose a topic in which you are an expert

The writing process is always fast and smooth when the topic interests you. Thoughts come to you alone, and you do not have to rack your brains over something. Plus, when you have a certain level of expertise in a topic, you will be able to analyze the topic with great professionalism.

If you can't choose a topic and have to write on a given topic, this is not a problem, but you will have to make more effort. Study the theoretical base, arrange a brainstorm, and come up with vectors for developing the topic. Then things will go much more pleasant. You may even like the process and be interested in this issue.

We have found you a few examples of analytical essay topics . Here are some of them:

• Do Living Conditions Depend on the Availability of Higher Education?
• Cyberbullying: What is it and the Background?
• The Reasons for Drug Addiction. How to Overcome It?

Stuck with finding the right title?

Get plenty of fresh and catchy topic ideas and pick the perfect one with PapersOwl Title Generator.

• Study the topic, rely only on trusted sources

A specialist is valued for his expertise, and people tend to reckon with the opinion of professionals with experience and respect in their field. Become an expert in your topic. Find and study reliable research, and expand your knowledge on a given topic. Explore the platforms available to you as well as different formats, not only text but also audiovisual. Go to the library and find books from different years and publishers that discuss your topic. New knowledge will always help you in life. Society tends to choose leaders for itself, a special feature of which is a sharp intellect. Become a leader in your group.

Rely exclusively on proven, reliable, and respected resources. Sentences like "British scientists have found out..." or "It is believed that..." are irrelevant. Indicate the sources from which you take the statistics, namely the name of the research institute. Before quoting a source, ensure it is respected in scientific circles. Avoid pseudoscientific and anti-scientific works.

Learn to back up and prove your point of view. All arguments you give in an analytical essay must be confirmed in the works of scientists. Refrain from making logical errors during the analysis because starting from an untrue fact will lead to untrue conclusions, which means unreliable work.

• Capture readers' attention with a thesis statement

The thesis should tell the reader what your paper will be about. An analytic thesis examines a text and allows you to make a specific statement. In an analytical essay, as we already mentioned, you choose the side from which you approach the study of the issue. From this position, you put forward one or another statement. Now you know how to write a thesis for an analytical essay.

• Create the outline

A work plan will not only help you adhere to the requirements for the structure of the essay but also become a support for your thoughts, which are sometimes difficult to organize into coherent sentences. The plan clearly indicates which paragraph follows which and serves as a framework for further writing. You can either make this outline sketch or develop an extended work description. Use the essay writing help to give you support in  creating the outline .

• Make a draft

Create a draft of your essay to view it from the outside and critically evaluate it. This will help you understand whether you should add some information, everything from what you expressed that you wanted. This is a layout that serves to analyze the work you have done. You have a chance to check for various kinds of errors.

• Don't forget about opposite-side arguments

To prove one's point of view, it is necessary to consider opposing arguments. Your thoughts will seem much more scientifically backed up if you give examples of other non-scientific methods or pseudoscientific claims. You can also give examples of popular misconceptions that are often used by society as true. Explain your point of view, give scientific arguments, and demonstrate to the reader why your analysis should be trusted.

Contrasting arguments are given for scientific accreditation of your work as well as to discredit pseudoscientific myths. Obviously, but it's worth mentioning that you shouldn't choose contrasting arguments that look more convincing and powerful against your arguments. It certainly won't work in your favor.

• Make sure character names, titles, and places are spelled correctly

Be very careful with the names of the characters or scientists you quote. Pay attention to the correct spelling of geographical objects, personal names, and phenomena. Also, familiarize yourself with the terms that you are appealing to understand what you are writing about.

• End writing with a conclusion

The closing paragraph of your analytical essay is just as important as the opening paragraph. We have already mentioned more than once that the reader, having read only the first and last paragraph, should be able to put together a complete picture of your essay. In the final paragraph, you need to briefly and clearly state the results of your analysis and which conclusions you came to in the course of the study. Refrain from writing unnecessary information. Stick only to the results that you described in your work.

• Proofread the paper

And, of course, after finishing writing your work, you need to reread it several times. RereadReread your essay for the first time immediately after writing, and you may want to add something or, on the contrary, delete some information. Spend the next reading one day after completing the essay. With a fresh look, it will be much easier to determine where you made spelling, punctuation, and stylistic mistakes. No matter how interesting the content of your essay is, if it contains many errors, the teacher will need to lower your grade. As a rule, an essay with many errors is rated worse than one that has been checked and improved.

You need to be a professional linguist to identify all the errors in your text. That is why there are special services that can easily help you proofread your work. As a result, you will receive an essay written by a special analytical essay writing service without a single spelling, punctuation, or stylistic error. The specialists on our site can help you make changes to the finished work or take over the writing of your analytical essay.

Catch plagiarism before your teacher does

Check your paper against billions of web pages and publications. Get an accurate plagiarism report in a few seconds. It's fast, easy & free!

Summing up what we said above, writing an analytical essay requires a lot of work on the part of the student. You need not only to work on creating the essay itself, but the first thing you need to do is decide on a topic in which you feel like a professional, develop an essay plan, and read the requirements instructions carefully. We hope that in our article, you have found useful information that will facilitate your analytical work. However, if you still need help writing an analytical essay, feel free to contact our specialists. PapersOwl.com offers guidance and help with creating your analytical essay, providing you with tips, tricks, and useful advice. With our help, you will be able to structure and write an impressive essay that will get you the best results.

Was this article helpful?

Thanks for your feedback.

Dr. Karlyna PhD

I am a proficient writer from the United States with over five years of experience in academic writing. I comfortably complete given assignments within stipulated deadlines and at the same time deliver high-quality work, which follows the guidelines provided.

Readers also enjoyed

Legal essay topics for student.

Essay Writing Guides 20 likes

Science Essay Topics for Students

Essay Writing Guides 3 likes

Explanatory Essay Topics for Students

Essay Writing Guides 8 likes

WHY WAIT? PLACE AN ORDER RIGHT NOW!

Simply fill out the form, click the button, and have no worries!

What Is Research Methodology? A Plain-Language Explanation & Definition (With Examples)

By Derek Jansen (MBA)  and Kerryn Warren (PhD) | June 2020 (Last updated April 2023)

If you’re new to formal academic research, it’s quite likely that you’re feeling a little overwhelmed by all the technical lingo that gets thrown around. And who could blame you – “research methodology”, “research methods”, “sampling strategies”… it all seems never-ending!

In this post, we’ll demystify the landscape with plain-language explanations and loads of examples (including easy-to-follow videos), so that you can approach your dissertation, thesis or research project with confidence. Let’s get started.

Research Methodology 101

• What exactly research methodology means
• What qualitative , quantitative and mixed methods are
• What sampling strategy is
• What data collection methods are
• What data analysis methods are
• How to choose your research methodology
• Example of a research methodology

What is research methodology?

Research methodology simply refers to the practical “how” of a research study. More specifically, it’s about how  a researcher  systematically designs a study  to ensure valid and reliable results that address the research aims, objectives and research questions . Specifically, how the researcher went about deciding:

• What type of data to collect (e.g., qualitative or quantitative data )
• Who  to collect it from (i.e., the sampling strategy )
• How to  collect  it (i.e., the data collection method )
• How to  analyse  it (i.e., the data analysis methods )

Within any formal piece of academic research (be it a dissertation, thesis or journal article), you’ll find a research methodology chapter or section which covers the aspects mentioned above. Importantly, a good methodology chapter explains not just   what methodological choices were made, but also explains  why they were made. In other words, the methodology chapter should justify  the design choices, by showing that the chosen methods and techniques are the best fit for the research aims, objectives and research questions. 

So, it’s the same as research design?

Not quite. As we mentioned, research methodology refers to the collection of practical decisions regarding what data you’ll collect, from who, how you’ll collect it and how you’ll analyse it. Research design, on the other hand, is more about the overall strategy you’ll adopt in your study. For example, whether you’ll use an experimental design in which you manipulate one variable while controlling others. You can learn more about research design and the various design types here .

Need a helping hand?

What are qualitative, quantitative and mixed-methods?

Qualitative, quantitative and mixed-methods are different types of methodological approaches, distinguished by their focus on words , numbers or both . This is a bit of an oversimplification, but its a good starting point for understanding.

Let’s take a closer look.

Qualitative research refers to research which focuses on collecting and analysing words (written or spoken) and textual or visual data, whereas quantitative research focuses on measurement and testing using numerical data . Qualitative analysis can also focus on other “softer” data points, such as body language or visual elements.

It’s quite common for a qualitative methodology to be used when the research aims and research questions are exploratory  in nature. For example, a qualitative methodology might be used to understand peoples’ perceptions about an event that took place, or a political candidate running for president. 

Contrasted to this, a quantitative methodology is typically used when the research aims and research questions are confirmatory  in nature. For example, a quantitative methodology might be used to measure the relationship between two variables (e.g. personality type and likelihood to commit a crime) or to test a set of hypotheses .

As you’ve probably guessed, the mixed-method methodology attempts to combine the best of both qualitative and quantitative methodologies to integrate perspectives and create a rich picture. If you’d like to learn more about these three methodological approaches, be sure to watch our explainer video below.

What is sampling strategy?

Simply put, sampling is about deciding who (or where) you’re going to collect your data from . Why does this matter? Well, generally it’s not possible to collect data from every single person in your group of interest (this is called the “population”), so you’ll need to engage a smaller portion of that group that’s accessible and manageable (this is called the “sample”).

How you go about selecting the sample (i.e., your sampling strategy) will have a major impact on your study.  There are many different sampling methods  you can choose from, but the two overarching categories are probability   sampling and  non-probability   sampling .

Probability sampling  involves using a completely random sample from the group of people you’re interested in. This is comparable to throwing the names all potential participants into a hat, shaking it up, and picking out the “winners”. By using a completely random sample, you’ll minimise the risk of selection bias and the results of your study will be more generalisable  to the entire population. 

Non-probability sampling , on the other hand,  doesn’t use a random sample . For example, it might involve using a convenience sample, which means you’d only interview or survey people that you have access to (perhaps your friends, family or work colleagues), rather than a truly random sample. With non-probability sampling, the results are typically not generalisable .

To learn more about sampling methods, be sure to check out the video below.

What are data collection methods?

As the name suggests, data collection methods simply refers to the way in which you go about collecting the data for your study. Some of the most common data collection methods include:

• Interviews (which can be unstructured, semi-structured or structured)
• Focus groups and group interviews
• Surveys (online or physical surveys)
• Observations (watching and recording activities)
• Biophysical measurements (e.g., blood pressure, heart rate, etc.)
• Documents and records (e.g., financial reports, court records, etc.)

The choice of which data collection method to use depends on your overall research aims and research questions , as well as practicalities and resource constraints. For example, if your research is exploratory in nature, qualitative methods such as interviews and focus groups would likely be a good fit. Conversely, if your research aims to measure specific variables or test hypotheses, large-scale surveys that produce large volumes of numerical data would likely be a better fit.

What are data analysis methods?

Data analysis methods refer to the methods and techniques that you’ll use to make sense of your data. These can be grouped according to whether the research is qualitative  (words-based) or quantitative (numbers-based).

Popular data analysis methods in qualitative research include:

• Qualitative content analysis
• Thematic analysis
• Discourse analysis
• Narrative analysis
• Interpretative phenomenological analysis (IPA)
• Visual analysis (of photographs, videos, art, etc.)

Qualitative data analysis all begins with data coding , after which an analysis method is applied. In some cases, more than one analysis method is used, depending on the research aims and research questions . In the video below, we explore some  common qualitative analysis methods, along with practical examples.  

Moving on to the quantitative side of things, popular data analysis methods in this type of research include:

• Descriptive statistics (e.g. means, medians, modes)
• Inferential statistics (e.g. correlation, regression, structural equation modelling)

Again, the choice of which data collection method to use depends on your overall research aims and objectives , as well as practicalities and resource constraints. In the video below, we explain some core concepts central to quantitative analysis.

How do I choose a research methodology?

As you’ve probably picked up by now, your research aims and objectives have a major influence on the research methodology . So, the starting point for developing your research methodology is to take a step back and look at the big picture of your research, before you make methodology decisions. The first question you need to ask yourself is whether your research is exploratory or confirmatory in nature.

If your research aims and objectives are primarily exploratory in nature, your research will likely be qualitative and therefore you might consider qualitative data collection methods (e.g. interviews) and analysis methods (e.g. qualitative content analysis). 

Conversely, if your research aims and objective are looking to measure or test something (i.e. they’re confirmatory), then your research will quite likely be quantitative in nature, and you might consider quantitative data collection methods (e.g. surveys) and analyses (e.g. statistical analysis).

Designing your research and working out your methodology is a large topic, which we cover extensively on the blog . For now, however, the key takeaway is that you should always start with your research aims, objectives and research questions (the golden thread). Every methodological choice you make needs align with those three components. 

Example of a research methodology chapter

In the video below, we provide a detailed walkthrough of a research methodology from an actual dissertation, as well as an overview of our free methodology template .

Psst… there’s more (for free)

This post is part of our dissertation mini-course, which covers everything you need to get started with your dissertation, thesis or research project. 

You Might Also Like:

193 Comments

Thank you for this simple yet comprehensive and easy to digest presentation. God Bless!

You’re most welcome, Leo. Best of luck with your research!

I found it very useful. many thanks

This is really directional. A make-easy research knowledge.

Thank you for this, I think will help my research proposal

Thanks for good interpretation,well understood.

Good morning sorry I want to the search topic

Thank u more

Thank you, your explanation is simple and very helpful.

Very educative a.nd exciting platform. A bigger thank you and I’ll like to always be with you

That’s the best analysis

So simple yet so insightful. Thank you.

This really easy to read as it is self-explanatory. Very much appreciated…

Thanks for this. It’s so helpful and explicit. For those elements highlighted in orange, they were good sources of referrals for concepts I didn’t understand. A million thanks for this.

Good morning, I have been reading your research lessons through out a period of times. They are important, impressive and clear. Want to subscribe and be and be active with you.

Thankyou So much Sir Derek…

Good morning thanks so much for the on line lectures am a student of university of Makeni.select a research topic and deliberate on it so that we’ll continue to understand more.sorry that’s a suggestion.

Beautiful presentation. I love it.

please provide a research mehodology example for zoology

It’s very educative and well explained

Thanks for the concise and informative data.

This is really good for students to be safe and well understand that research is all about

Thank you so much Derek sir🖤🙏🤗

Very simple and reliable

This is really helpful. Thanks alot. God bless you.

very useful, Thank you very much..

thanks a lot its really useful

in a nutshell..thank you!

Thanks for updating my understanding on this aspect of my Thesis writing.

Very simple but yet insightful Thank you

This has been an eye opening experience. Thank you grad coach team.

Very useful message for research scholars

Really very helpful thank you

yes you are right and i’m left

Research methodology with a simplest way i have never seen before this article.

wow thank u so much

Good morning thanks so much for the on line lectures am a student of university of Makeni.select a research topic and deliberate on is so that we will continue to understand more.sorry that’s a suggestion.

Very precise and informative.

Thanks for simplifying these terms for us, really appreciate it.

Thanks this has really helped me. It is very easy to understand.

I found the notes and the presentation assisting and opening my understanding on research methodology

Good presentation

Im so glad you clarified my misconceptions. Im now ready to fry my onions. Thank you so much. God bless

Thank you a lot.

thanks for the easy way of learning and desirable presentation.

Thanks a lot. I am inspired

Well written

I am writing a APA Format paper . I using questionnaire with 120 STDs teacher for my participant. Can you write me mthology for this research. Send it through email sent. Just need a sample as an example please. My topic is ” impacts of overcrowding on students learning

Thanks for your comment.

We can’t write your methodology for you. If you’re looking for samples, you should be able to find some sample methodologies on Google. Alternatively, you can download some previous dissertations from a dissertation directory and have a look at the methodology chapters therein.

All the best with your research.

Thank you so much for this!! God Bless

Thank you. Explicit explanation

Thank you, Derek and Kerryn, for making this simple to understand. I’m currently at the inception stage of my research.

Thnks a lot , this was very usefull on my assignment

excellent explanation

I’m currently working on my master’s thesis, thanks for this! I’m certain that I will use Qualitative methodology.

Thanks a lot for this concise piece, it was quite relieving and helpful. God bless you BIG…

I am currently doing my dissertation proposal and I am sure that I will do quantitative research. Thank you very much it was extremely helpful.

Very interesting and informative yet I would like to know about examples of Research Questions as well, if possible.

I’m about to submit a research presentation, I have come to understand from your simplification on understanding research methodology. My research will be mixed methodology, qualitative as well as quantitative. So aim and objective of mixed method would be both exploratory and confirmatory. Thanks you very much for your guidance.

OMG thanks for that, you’re a life saver. You covered all the points I needed. Thank you so much ❤️ ❤️ ❤️

Thank you immensely for this simple, easy to comprehend explanation of data collection methods. I have been stuck here for months 😩. Glad I found your piece. Super insightful.

I’m going to write synopsis which will be quantitative research method and I don’t know how to frame my topic, can I kindly get some ideas..

Thanks for this, I was really struggling.

This was really informative I was struggling but this helped me.

Thanks a lot for this information, simple and straightforward. I’m a last year student from the University of South Africa UNISA South Africa.

its very much informative and understandable. I have enlightened.

An interesting nice exploration of a topic.

Thank you. Accurate and simple🥰

This article was really helpful, it helped me understanding the basic concepts of the topic Research Methodology. The examples were very clear, and easy to understand. I would like to visit this website again. Thank you so much for such a great explanation of the subject.

Thanks dude

Thank you Doctor Derek for this wonderful piece, please help to provide your details for reference purpose. God bless.

Many compliments to you

Great work , thank you very much for the simple explanation

Thank you. I had to give a presentation on this topic. I have looked everywhere on the internet but this is the best and simple explanation.

thank you, its very informative.

Well explained. Now I know my research methodology will be qualitative and exploratory. Thank you so much, keep up the good work

Well explained, thank you very much.

This is good explanation, I have understood the different methods of research. Thanks a lot.

Great work…very well explanation

Thanks Derek. Kerryn was just fantastic!

Great to hear that, Hyacinth. Best of luck with your research!

Its a good templates very attractive and important to PhD students and lectuter

Thanks for the feedback, Matobela. Good luck with your research methodology.

Thank you. This is really helpful.

You’re very welcome, Elie. Good luck with your research methodology.

Well explained thanks

This is a very helpful site especially for young researchers at college. It provides sufficient information to guide students and equip them with the necessary foundation to ask any other questions aimed at deepening their understanding.

Thanks for the kind words, Edward. Good luck with your research!

Thank you. I have learned a lot.

Great to hear that, Ngwisa. Good luck with your research methodology!

Thank you for keeping your presentation simples and short and covering key information for research methodology. My key takeaway: Start with defining your research objective the other will depend on the aims of your research question.

My name is Zanele I would like to be assisted with my research , and the topic is shortage of nursing staff globally want are the causes , effects on health, patients and community and also globally

Thanks for making it simple and clear. It greatly helped in understanding research methodology. Regards.

This is well simplified and straight to the point

Thank you Dr

I was given an assignment to research 2 publications and describe their research methodology? I don’t know how to start this task can someone help me?

Sure. You’re welcome to book an initial consultation with one of our Research Coaches to discuss how we can assist – https://gradcoach.com/book/new/ .

Thanks a lot I am relieved of a heavy burden.keep up with the good work

I’m very much grateful Dr Derek. I’m planning to pursue one of the careers that really needs one to be very much eager to know. There’s a lot of research to do and everything, but since I’ve gotten this information I will use it to the best of my potential.

Thank you so much, words are not enough to explain how helpful this session has been for me!

Thanks this has thought me alot.

Very concise and helpful. Thanks a lot

Thank Derek. This is very helpful. Your step by step explanation has made it easier for me to understand different concepts. Now i can get on with my research.

I wish i had come across this sooner. So simple but yet insightful

really nice explanation thank you so much

I’m so grateful finding this site, it’s really helpful…….every term well explained and provide accurate understanding especially to student going into an in-depth research for the very first time, even though my lecturer already explained this topic to the class, I think I got the clear and efficient explanation here, much thanks to the author.

It is very helpful material

I would like to be assisted with my research topic : Literature Review and research methodologies. My topic is : what is the relationship between unemployment and economic growth?

Its really nice and good for us.

THANKS SO MUCH FOR EXPLANATION, ITS VERY CLEAR TO ME WHAT I WILL BE DOING FROM NOW .GREAT READS.

Short but sweet.Thank you

Informative article. Thanks for your detailed information.

I’m currently working on my Ph.D. thesis. Thanks a lot, Derek and Kerryn, Well-organized sequences, facilitate the readers’ following.

great article for someone who does not have any background can even understand

I am a bit confused about research design and methodology. Are they the same? If not, what are the differences and how are they related?

Thanks in advance.

concise and informative.

Thank you very much

How can we site this article is Harvard style?

Very well written piece that afforded better understanding of the concept. Thank you!

Am a new researcher trying to learn how best to write a research proposal. I find your article spot on and want to download the free template but finding difficulties. Can u kindly send it to my email, the free download entitled, “Free Download: Research Proposal Template (with Examples)”.

Thank too much

Thank you very much for your comprehensive explanation about research methodology so I like to thank you again for giving us such great things.

Good very well explained.Thanks for sharing it.

Thank u sir, it is really a good guideline.

so helpful thank you very much.

Thanks for the video it was very explanatory and detailed, easy to comprehend and follow up. please, keep it up the good work

It was very helpful, a well-written document with precise information.

how do i reference this?

MLA Jansen, Derek, and Kerryn Warren. “What (Exactly) Is Research Methodology?” Grad Coach, June 2021, gradcoach.com/what-is-research-methodology/.

APA Jansen, D., & Warren, K. (2021, June). What (Exactly) Is Research Methodology? Grad Coach. https://gradcoach.com/what-is-research-methodology/

Your explanation is easily understood. Thank you

Very help article. Now I can go my methodology chapter in my thesis with ease

I feel guided ,Thank you

This simplification is very helpful. It is simple but very educative, thanks ever so much

The write up is informative and educative. It is an academic intellectual representation that every good researcher can find useful. Thanks

Wow, this is wonderful long live.

Nice initiative

thank you the video was helpful to me.

Thank you very much for your simple and clear explanations I’m really satisfied by the way you did it By now, I think I can realize a very good article by following your fastidious indications May God bless you

Thanks very much, it was very concise and informational for a beginner like me to gain an insight into what i am about to undertake. I really appreciate.

very informative sir, it is amazing to understand the meaning of question hidden behind that, and simple language is used other than legislature to understand easily. stay happy.

This one is really amazing. All content in your youtube channel is a very helpful guide for doing research. Thanks, GradCoach.

research methodologies

Please send me more information concerning dissertation research.

Nice piece of knowledge shared….. #Thump_UP

This is amazing, it has said it all. Thanks to Gradcoach

This is wonderful,very elaborate and clear.I hope to reach out for your assistance in my research very soon.

This is the answer I am searching about…

realy thanks a lot

Thank you very much for this awesome, to the point and inclusive article.

Thank you very much I need validity and reliability explanation I have exams

Thank you for a well explained piece. This will help me going forward.

Very simple and well detailed Many thanks

This is so very simple yet so very effective and comprehensive. An Excellent piece of work.

I wish I saw this earlier on! Great insights for a beginner(researcher) like me. Thanks a mil!

Thank you very much, for such a simplified, clear and practical step by step both for academic students and general research work. Holistic, effective to use and easy to read step by step. One can easily apply the steps in practical terms and produce a quality document/up-to standard

Thanks for simplifying these terms for us, really appreciated.

Thanks for a great work. well understood .

This was very helpful. It was simple but profound and very easy to understand. Thank you so much!

Great and amazing research guidelines. Best site for learning research

hello sir/ma’am, i didn’t find yet that what type of research methodology i am using. because i am writing my report on CSR and collect all my data from websites and articles so which type of methodology i should write in dissertation report. please help me. i am from India.

how does this really work?

perfect content, thanks a lot

As a researcher, I commend you for the detailed and simplified information on the topic in question. I would like to remain in touch for the sharing of research ideas on other topics. Thank you

Impressive. Thank you, Grad Coach 😍

Thank you Grad Coach for this piece of information. I have at least learned about the different types of research methodologies.

Very useful content with easy way

Thank you very much for the presentation. I am an MPH student with the Adventist University of Africa. I have successfully completed my theory and starting on my research this July. My topic is “Factors associated with Dental Caries in (one District) in Botswana. I need help on how to go about this quantitative research

I am so grateful to run across something that was sooo helpful. I have been on my doctorate journey for quite some time. Your breakdown on methodology helped me to refresh my intent. Thank you.

thanks so much for this good lecture. student from university of science and technology, Wudil. Kano Nigeria.

It’s profound easy to understand I appreciate

Thanks a lot for sharing superb information in a detailed but concise manner. It was really helpful and helped a lot in getting into my own research methodology.

Comment * thanks very much

This was sooo helpful for me thank you so much i didn’t even know what i had to write thank you!

You’re most welcome 🙂

Simple and good. Very much helpful. Thank you so much.

This is very good work. I have benefited.

Thank you so much for sharing

This is powerful thank you so much guys

I am nkasa lizwi doing my research proposal on honors with the university of Walter Sisulu Komani I m on part 3 now can you assist me.my topic is: transitional challenges faced by educators in intermediate phase in the Alfred Nzo District.

Appreciate the presentation. Very useful step-by-step guidelines to follow.

I appreciate sir

wow! This is super insightful for me. Thank you!

Indeed this material is very helpful! Kudos writers/authors.

I want to say thank you very much, I got a lot of info and knowledge. Be blessed.

I want present a seminar paper on Optimisation of Deep learning-based models on vulnerability detection in digital transactions.

Need assistance

Dear Sir, I want to be assisted on my research on Sanitation and Water management in emergencies areas.

I am deeply grateful for the knowledge gained. I will be getting in touch shortly as I want to be assisted in my ongoing research.

The information shared is informative, crisp and clear. Kudos Team! And thanks a lot!

hello i want to study

Trackbacks/Pingbacks

• What Is A Literature Review (In A Dissertation Or Thesis) - Grad Coach - […] the literature review is to inform the choice of methodology for your own research. As we’ve discussed on the Grad Coach blog,…
• Free Download: Research Proposal Template (With Examples) - Grad Coach - […] Research design (methodology) […]
• Dissertation vs Thesis: What's the difference? - Grad Coach - […] and thesis writing on a daily basis – everything from how to find a good research topic to which…

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open Access
• Published: 11 September 2023

Analysis of the research progress on the deposition and drift of spray droplets by plant protection UAVs

• Qin Weicai 1 , 2 &
• Chen Panyang 3  

Scientific Reports volume  13 , Article number:  14935 ( 2023 ) Cite this article

45 Accesses

Metrics details

• Plant sciences

Plant protection unmanned aerial vehicles (UAVs), which are highly adapted to terrain and capable of efficient low-altitude spraying, will be extensively used in agricultural production. In this paper, single or several independent factors influencing the deposition characteristics of droplets sprayed by plant protection UAVs, as well as the experimental methods and related mathematical analysis models used to study droplet deposition and drift, are systematically investigated. A research method based on farmland environmental factors is proposed to simulate the deposition and drift characteristics of spray droplets. Moreover, the impacts of multiple factors on the droplet deposition characteristics are further studied by using an indoor simulation test system for the spraying flow field of plant protection UAVs to simulate the plant protection UAVs spraying flow field, temperature, humidity and natural wind. By integrating the operation parameters, environmental conditions, crop canopy characteristics and rotor airflow, the main effects and interactive effects of the factors influencing the deposition of spray droplets can be explored. A mathematical model that can reflect the internal relations of multiple factors and evaluate and analyze the droplet deposition characteristics is established. A scientific and effective method for determining the optimal spray droplet deposition is also proposed. In addition, this research method can provide a necessary scientific basis for the formulation of operating standards for plant protection UAVs, inspection and evaluation of operating tools at the same scale, and the improvement and upgrading of spraying systems.

Introduction

In agriculture, aerial spray is widely used to spray fertilizers, herbicides, fungicides and other materials used for crop protection 1 . Compared with large fixed-wing agricultural aircraft, small unmanned aerial vehicles (UAVs) are particularly advantageous because they are highly maneuverable and do not need any airport for taking off or landing 2 . In recent years, aerial machinery for plant protection, especially aerial spray by small plant protection UAVs, has developed rapidly 3 . Small plant protection UAVs have greater application prospects in agricultural production because of their better terrain adaptability and low-altitude spraying capability (Figs. 1 and 2 ) 4 , 5 , 6 , 7 . However, as an emerging technology, UAV spraying technology in agricultural pest control are not common due to the lack of operational standards and uncertainty about the best spraying parameters, which leads to a series of problems, such as the poor uniformity of droplet deposition distribution and low levels of fog deposition.

Single-rotor UAV spraying.

Multirotor UAV spraying.

Some studies have shown that if the aerial spraying parameters are not set scientifically, it will lead to not only repeated spraying and missed spraying, degrading the effect of pest control but also pesticide drift 8 . The use of new pesticide additives and the innovative research and development of precise spraying equipment of plant protection UAVs along with its safe and efficient use in the prevention and control of diseases, pests and weeds are indispensable means to increase the pesticide deposition amount and reduce drift. Studying the deposition characteristics of spray droplets is not only of scientific significance for the development of new pesticide formulations and precise spraying equipment of plant protection UAVs but also of practical guiding significance for the safe and efficient use of pesticides in farmland. Due to many factors, such as the natural environment, pesticide characteristics, crop canopy characteristics, and plant protection UAV operating parameters, it is a complicated process to study the uniformity and penetration of spray droplets. To improve the spraying effect and reduce drift, scientific and technological staff all over the world have carried out a large number of exploratory studies on the deposition and drift characteristics of spray droplets through field or wind tunnel experiments or mathematical model analysis 9 , 10 , 11 , 12 , 13 . The main factors and secondary factors influencing the characteristics of droplet deposition and drift are organized from the many influencing factors (nozzle, droplet, aircraft type, weather factors, etc.), and the functional relationship between the amount of different droplet deposition and drift and their influencing factors are determined. However, there are not sufficient deposition models for plant protection rotor UAVs, and the existing models consider only a few influencing factors, which need to be further modified.

With the development of UAV technology, there are an increasing number of studies on the droplet deposition rules, operation parameter optimization and evaluation methods of pesticides applied by plant protection UAVs in rice fields and maize fields 14 , 15 , 16 , 17 ; however, these studies have defects in that the meteorological factors in the farmland environment are unstable and uncontrollable, the UAV track easily deviates, resulting in the poor uniformity of droplet deposition distribution (the coefficient of variation may be above 40% 16 , while it is usually below 10% for spraying by ground equipment), the test result cannot be well repeated, and different types of UAVs cannot be easily evaluated at the same scale. Thus, it is difficult to evaluate the droplet deposition characteristics of different types of UAVs scientifically. Some research has established mathematical models to study the impact of plant protection UAV operating parameters (operating height, operating speed, and spraying flow rate) on droplet deposition and drift characteristics 18 , 19 , 20 and determined the main effects influencing droplet deposition. However, due to the lack of conformity between the assumptions of these models and farmland practice, they neglected the influence of the characteristics of the crop canopy and the interaction of multiple factors such as the environment, crops, and operating parameters of application equipment on the droplet deposition characteristics (uniformity of distribution and penetration), making the results obtained through analysis with existing mathematical models highly deviate from practice.

In this paper, the current status and problems of research on the deposition and drift of spray droplets from plant protection drones are introduced, and the importance of research in this area to improve the effectiveness of pesticide application and reduce drift hazards is emphasized. The need for more in depth, comprehensive and systematic research on the deposition and drift of spray droplets from plant protection drones is highlighted, and the problems and challenges of the current research are pointed out, providing important guidance and references for future research.

Research on the influencing factors of spray droplet deposition characteristics

Studying droplet deposition characteristics (uniformity and penetration) is always a major subject in pesticide application technology research 21 . The deposition characteristics of spray droplets are influenced by application techniques and equipment, crops, the environment, etc. Detailed influencing factors include the wind speed, wind direction, leaf area index, target crop canopy structure, leaf inclination, leaf surface characteristics, and characteristics of the spray droplet population (release height, release rate, application liquid volume, spray droplet particle size spectrum) 22 , 23 , 24 .

Several studies have investigated the influence of various factors on droplet deposition characteristics in plant protection UAV spraying. Diepenbrock noted that plant leaf characteristics, such as size, inclination angle, drooping degree, and spatial arrangement, impact the composition quantity and distribution quality within the crop canopy structure, subsequently affecting droplet penetration and deposition 25 . Song et al. found that altering the initial velocity of droplets increases deposition amounts on horizontal and vertical targets. Factors like flying altitude and speed of different aircraft types have been extensively studied for their influence on droplet deposition and drift 26 . Qiu et al. used an orthogonal experimental method to study the deposition distribution rules of droplets sprayed by unmanned helicopters at different flying heights and speeds under field conditions. They established a relationship model that clarifies the interactions between deposition concentration, uniformity, flying speed, and flying height, providing valuable insights for optimizing spray operation parameters 18 . Chen et al. investigated the pattern of aerial spray droplet deposition in the rice canopy using a small unmanned helicopter. They explored the effects of different spraying parameters on droplet distribution, specifically analyzing the deposition of growth regulator spraying 27 . Wang et al. proposed a method for testing the spatial mass balance of UAV-applied droplets and conducted field experiments on three types of UAVs to accurately determine the spatial distribution of the droplets and the downdraft field. They also conducted an experimental study on the droplet deposition pattern of hovering UAV variable spraying and highlighted the significant impact of downward swirling airflow on droplet deposition distribution 14 . Qin et al. focused on the influence of spraying parameters, such as operation height and velocity, of the UAV on droplet deposition on the rice canopy and protection efficacy against plant hoppers, using water-sensitive paper to collect droplets and statistically analyzing their coverage rates. The findings indicated that UAV spraying exhibited a low-volume and highly concentrated spray pattern 19 .

In summary, there are many factors influencing the deposition characteristics (uniformity and penetration) of spray droplets. However, in most of the current research on spraying by plant protection UAVs, only the influence of factors such as the flying height and flying speed on droplet deposition in the field environment is taken into consideration. Considering the influence of the interaction between environmental factors, crop canopy characteristics (growth stage, leaf area index, leaf inclination angle) and plant protection UAV spraying parameters on droplet deposition characteristics, there is neither in-depth understanding nor relevant reports, especially under controllable environmental conditions (Fig.  3 ). To promote high-efficiency spraying technology for plant protection UAVs, targeted basic research should be carried out on the analysis of the influencing factors of plant protection UAV spraying and the optimal deposition of droplets.

Description of the deposition and drift with rotor UAV spraying.

Research on the experimental means and testing methods of droplet deposition and drift

At present, the deposition and drift of droplets are mainly researched by field tests and wind tunnel tests 28 , 29 , 30 , 31 , 32 . Field test research on pesticide deposition and drift is similar to the actual situation, but it is quite difficult to acquire correct data due to the constant changes in meteorological factors such as the wind speed, wind direction, temperature and humidity. In addition, Emilia et al. noted that the terrain and plant morphology also influence the wind flow and droplet deposition, leading to considerable deviation among repeated test results 33 . Therefore, it is difficult to accurately determine the total amount and distribution of pesticides drifting in the air 34 . The wind tunnel laboratory can provide a controllable environment to simulate the external spraying conditions, and the wind speed and direction can be easily controlled. Therefore, it is an important means to study the drift characteristics of spraying components and avoid many defects in field test research 10 , 35 . The typical wind tunnels that are widely used in agricultural aviation spraying technology are shown in Table 1 36 , 37 .

Internationally well-known professional research institutions for pesticide application, such as the Julius Kuehn Institute-Federal Research Centre for Cultivated Plants (JKI, formerly BBA) and USDA-Agricultural Research Service, Application Technology Research Unit (USDA-ARS-ATRU), have a circular closed low-speed standard wind tunnel (Fig.  4 ). This wind tunnels are widely used to assess the distribution, degradation and drift of pesticide sprays, simulating real crop and environmental conditions. The advantages are that they provide accurate measurements of pesticide distribution and drift and are able to reproduce wind field conditions in realistic environments. However, circular low-speed wind tunnels have limitations when it comes to parameters such as spray particle size, density and flow rate for different pesticides. The Silsoe Research Institute, UK (SRI) has a standard linear low-speed wind tunnel. This wind tunnel can be used to test the performance of agricultural mechanised sprayers and the design of sprayers. The advantage is that they can simulate actual operating conditions and can accurately test the performance and flow rate of agricultural mechanised sprayers. However, linear low speed wind tunnels are typically more expensive than circular wind tunnels and can only simulate a single environmental condition. The Center for Pesticide Application and Safety (CPAS) of the University of Queensland in Australia has an open-path wind tunnel (Fig.  5 ). This type of wind tunnel can be used to test aspects such as drift and particle distribution of agricultural sprayers. The advantages are ease of operation, low cost and the ability to reproduce wind fields under different environmental conditions. However, open path wind tunnels do not simulate realistic crop environments and have unstable wind speeds. In 2014, the Nanjing Institute of Agricultural Mechanization, Ministry of Agriculture and Rural Affairs, built the NJS-1 plant protection direct flow closed wind tunnel (Fig.  6 ). This type of wind tunnel is mainly used to evaluate different sprayers in terms of performance and droplet distribution. The advantages are the ability to simulate a realistic farm environment with high accuracy and the ability to test different types and brands of sprayers. However, straight-through enclosed wind tunnels are only suitable for small equipment and small-scale trials and are costly. In 2018, the National Center for International Collaboration Research on Precision Agricultural Aviation Pesticide Spraying Technology of South China Agricultural University built a high- and low-speed composite wind tunnel for agricultural aviation research (Fig.  7 ). This wind tunnel is suitable for agricultural aerial research and can simulate the effects of spraying at different heights and wind speeds. The advantage is that it can accurately test the effects of pesticide spraying at different heights and speeds, and can improve the efficiency and accuracy of agricultural aerial spraying. However, high and low speed composite wind tunnels are relatively costly and require a high level of technology and equipment requirements. As the basic conditions for technical research, these wind tunnels have made great contributions to the study of pesticide deposition and drift rules, product testing, and product optimization 38 , 39 , 40 , 41 , 42 . However, for the study of spray droplet deposition and drift under the disturbance of the wind field of plant protection UAVs, the single-direction wind tunnel simulation test is still insufficient to simulate the combined effect of the downward swirl flow under the rotor and the natural wind. In addition, the existing agricultural wind tunnels are limited in size, so plant protection UAVs cannot be placed. In the military, a scaled model method is used to put UAVs into wind tunnels for research 43 , 44 , but it is not suitable for research on pesticide spraying with plant protection UAVs, and the airflow will rebound from the tunnel wall.

Circle closed low-speed wind tunnel.

Open wind tunnel.

NJS-1DC closed wind tunnel.

High and close speed composite wind.

Another important test technique for drift research is the sampling and analysis of droplet drift. Test studies on the drift of aerial mist in developed countries such as the United States and Germany are carried out with advanced test instruments, including automatic air samplers, gas or liquid chromatography, fluorescence analyzers, and electronic scanners. to collect and analyze the droplet deposition amount, the number of droplets, the coverage density of droplets, and the content of substances and study the correlation between additive concentration, spraying height and drift 4 , 45 , 46 . However, these traditional methods involve a long collecting and processing cycle, samples have to be processed in the lab, and it is difficult to express the dynamics of droplets in air. Particle image velocimetry (PIV) and LIDAR scanning test methods can solve the above problems, and each has its own advantages. PIV can obtain the three-dimensional spatial velocity vector of droplets and droplet size with a high sampling accuracy but limited spatial measurement scale 47 , 48 , 49 ; the LIDAR scanning method, realized by layered scanning, can quickly and accurately obtain the large-scale spatial droplet point cloud data and inversely form the three-dimensional distribution and temporal-spatial change of the droplets, but cannot reflect the spatial velocity vector change of the droplets 50 . The advantages, disadvantages and applications of droplet deposition and drift measurement methods are shown in Table 2 51 .

Overall, the sampling and analysis of droplet drift, along with techniques such as PIV and LIDAR scanning, play a crucial role in studying and understanding the behavior of droplets during aerial spraying. These methods provide valuable insights into droplet deposition, drift patterns, and the effects of various factors, enabling researchers to optimize spray practices, minimize drift, and enhance the efficiency and effectiveness of plant protection UAV applications.

Research on the mathematical analysis model of spray droplet deposition characteristics

In the development of spraying equipment and the determination of the optimal deposition conditions for spray, a large amount of data and information are needed to explain the influence of different factors on the spraying performance and the relationship between variables. At present, spraying drift modeling can be divided into models based on mechanics and models based on statistics 52 , 53 , 54 .

One of the models based on mechanics analyzes the movement of a single droplet in the airflow field by the Lagrangian trajectory tracking analysis method. Teske et al. established the AGDISP model by the analytical Lagrangian method to describe aerial spraying under the condition of ignoring the influence of aircraft wake and atmospheric turbulence 46 . This model takes not only the aircraft type, environmental conditions, and droplet properties but also the influencing factors of the nozzle model into consideration. The user can input the parameters of the nozzle, droplet spectrum, aircraft type and weather factors. from an internal database and predict the drift potential. It can effectively and accurately predict a range of 20 km but is generally used for fixed-wing aircraft. Duga et al. and Gregorio et al. also studied the deposition distribution of aerial spray in orchards with the Lagrangian discrete phase model, and the result of the numerical model showed that the prediction error of total deposition on the fruit tree canopy is above 30% 48 , 51 . Dorr et al. developed a spray deposition model for whole plants based on L-studio, which takes into account the plant leaf wettability, impact angle, droplet break-up and rebound behavior, and the number of sub-droplets produced 55 . In 2020, Zabkiewicz et al. used an updated version of the software based on this model, developing a new user interface and refining the droplet fragmentation model 56 .

Another model based on mechanics is realized with the CFD (Computational Fluid Dynamics) method 57 , 58 , but there are still large errors between the simulated value and the real value of some models due to various factors. Holterman et al. carried out a series of cross-wind single nozzle field experiments in consideration of the traveling speed, entrained airflow, geometric parameters of the farmland, sprayer system setting parameters and environmental factors when studying the droplet deposition drift model of ground boom sprayers to calibrate the mathematical model. The results showed that when the height from the crop canopy is less than or equal to 0.7 m, the error between the test and the model simulation is within 10%, but the error between droplet deposition and drift prediction gradually increases as the height of the spray boom increases 59 , 60 , 61 .

Chinese scientific and technological staff have conducted experimental research and numerical analysis on the numerical simulation and mathematical modeling of spraying droplet deposition and drift prediction of ground plant protection equipment and have drawn some conclusions that physical quantities such as the operating speed, droplet size and crosswind impact the droplet deposition and drift process (Figs. 8 and 9 ) 62 , 63 . Zhu et al. developed the DRIFTSIM based on CFD and Lagrangian methods with a CFD simulation database for ground drift prediction and a user interface to access drift-related data 64 . Hong et al. constructed an integrated computational hydrodynamic model to predict the deposition and transport of pesticide sprays under the canopy in apple orchards during different growth periods 65 .

Rotor wind field test platform based on a wind tunnel.

Layout scene of droplet drift.

The above research proves that computer simulation technologies are widely applicable to the prediction research of droplet deposition under various complicated wind-supply airflow conditions 66 . The existing AGDISP model is relatively developed and only suitable for research on fixed-wing aircraft, which is very different from research on plant protection UAVs. The current plant protection UAV spraying prediction model still has problems such as large relative errors between the experimental value and simulation value of the deposition and drift at each measurement point. Therefore, the prediction accuracy of the numerical model for the spray droplet deposition of plant protection UAVs is still low and needs to be improved, and there is a lack of in-depth basic research on analyzing the rotor flow field and establishing mathematical analysis models for droplet deposition 67 .

The rotor wind field test platform and droplet drift

The use of UAVs for crop spraying has become increasingly popular due to its efficiency and effectiveness. However, accurately analyzing the spraying process is challenging due to the complex flow field of the droplets in the air and the multitude of factors that can affect their deposition characteristics. Current testing systems rely on simple methods such as static targets or trays, which do not accurately represent the dynamic and complex nature of the real environment. To better study the UAV spraying flow field, a corresponding indoor simulation test system is needed. The indoor simulation system proposed in this study combines a natural wind simulation system and a rotor simulation system that can simulate several factors present in the natural environment that affect droplet deposition characteristics. The natural wind simulation system can effectively replicate wind speed variations, which is a key factor influencing droplet dispersion and deposition. By adjusting the settings of the wind simulation system, it is possible to replicate a range of wind speeds encountered in the field, allowing researchers to study the effects of different wind speeds on droplet behaviour and deposition. By adjusting the settings of the rotor simulation system, it is possible to demonstrate the magnitude of the downwash airflow at different speeds of the UAV rotor. However, it is important to note that while wind speed variations can be simulated, other factors, such as wind direction and turbulence, may have limitations in being accurately replicated in an indoor simulation system. These factors may require further development of simulation techniques to achieve more accurate replication. Nevertheless, the inclusion of natural wind simulation systems and rotor simulation systems in indoor simulation setups provides a valuable tool for studying the effects of wind speed.

The fluorescence tracer method involves adding a fluorescent dye or tracer to the liquid spray mixture used in the UAV spraying process. When these droplets containing the fluorescent tracer are released into the air, they can be illuminated with a specific wavelength of light, typically ultraviolet (UV) light. The fluorescent dye absorbs this UV light and re-emits it at a longer wavelength, usually in the visible range.

The high-speed camera is synchronized with the UV light source and captures the emitted fluorescent signals from the droplets. By analyzing the recorded video footage, researchers can precisely track the movement and behavior of the fluorescent droplets in the air. The high-speed camera captures images at a rapid frame rate, allowing for the visualization and analysis of the droplet flow field in detail.

The proposed indoor simulation test system for the spraying flow field of plant protection UAVs is a comprehensive and innovative method that combines the fluorescence tracer method and high-speed camera method to accurately track the dynamic changes in the local droplet flow field in the air. This system also includes a natural wind simulation system, which allows for the more realistic simulation of the actual environment, and thus more accurately reproduces the complex factors that affect droplet deposition characteristics. This method represents a significant improvement over existing testing systems, as it provides a more accurate and comprehensive analysis of the deposition process of droplets affected by multiple factors, enabling researchers to more effectively study the flow field and optimize the spraying process for plant protection UAVs. Overall, this proposed system has the potential to revolutionize the study of UAV spraying flow fields and could lead to significant advancements in the field of plant protection. Therefore, the method proposed in this paper is superior to the methods currently in use (Fig.  10 ).

Diagram of the rotor wind field test platform and droplet drift.

In conclusion, existing studies on plant protection UAV spraying have primarily focused on isolated factors, such as flying height, flying speed, and nozzle flow, without considering the interaction effects among other influential factors. This limitation calls for the need to conduct experimental research that combines spray droplet deposition characteristics with crop canopy characteristics in a controllable environment, encompassing environmental conditions and operating parameters. The proposed research aims to address this gap by developing an indoor simulation system that incorporates a natural wind simulation system. This innovative system enables the study of droplet deposition characteristics influenced by multiple factors in a realistic environment. By statistically analyzing the factors affecting droplet deposition and establishing a multivariable relationship model, optimal droplet deposition suitable for field operation decision-making of plant protection UAVs can be quantified and evaluated. This research presents an effective technical pathway for understanding the deposition patterns of droplets sprayed by plant protection UAVs and supports the formulation of relevant pesticide application standards for plant protection UAVs.

Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Lan, Y. B., Thomson, S. J., Huang, Y. B., Hoffmann, W. C. & Zhang, H. H. Current status and future directions of precision aerial application for site-specific crop management in the USA. Comput. Electron. Agric. 74 (1), 34–38 (2010).

Article   Google Scholar  

Chen, T. H. & Lu, S. H. Autonomous navigation control system of agricultural mini-unmaned aerial vehicles based on DSP. Trans. Chin. Soc. Agric. Eng. (Trans. CSAE). 28 (21), 164–169 (2012) ( (in Chinese with English abstract) ).

CAS   Google Scholar  

Zhou, W. Application and popularization of agricultural unmanned plant protection helicopter. Agric. Eng. 3 (S1), 56–58 (2013).

Google Scholar  

Lan, Y. B., Hoffmann, W. C., Fritz, B. K., Martin, D. E. & Lopez, J. D. Spray drift mitigation with spray mix adjuvants. Appl. Eng. Agric. 24 (1), 5–10 (2008).

Zhang, D. Y., Lan, Y. B., Chen, L. P., Wang, X. & Liang, D. Current status and future trends of agricultural aerial spraying technology in China. Trans. Chin. Soc. Agric. Mach. 45 (10), 53–59 (2014).

Faical, B. S., Costa, F. G., Pessin, G., Ueyama, J. & Freitas, H. The use of unmanned aerial vehicles and wireless sensor networks for spraying pesticides. J. Syst. Architect. 60 (4), 393–404 (2014).

Xue, X. Y. & Lan, Y. B. Agricultural aviation applications in USA. Trans. Chin. Soc. Agric. Mach. 44 (5), 194–201 (2013).

Fritz, B. K., Hoffmann, W. C. & Lan, Y. B. Evaluation of the EPA drift reduction technology (DRT) low-speed wind tunnel protocol. J. ASTM Int. 4 (6), 1–11 (2009).

Liu, H. S., Lan, Y. B., Xue, X. Y., Zhou, Z. Y. & Luo, X. W. Development of wind tunnel test technologies in agricultural aviation spraying. Trans. Chin. Soc. Agric. Eng. (Trans. CSAE) 31 (Supp. 2), 1–10 (2015) ( (in English) ).

Ru, Y., Zhu, C. Y. & Bao, R. Spray drift model of droplets and analysis of influencing factors based on wind tunnel. Trans. Chin. Soc. Agric. Mach. 45 (10), 66–72 (2014).

Lebeau, F. & Verstraete, A. RTDrift: A real time model for estimating spray drift from ground applications. Comput. Electron. Agric. 77 (2), 161–174 (2012).

Fritz, B. K. Meteorological effects on deposition and drift of aerially applied sprays. Trans. ASABE 49 (5), 1295–1301 (2006).

Zeng, A. J., He, X. K., Chen, Q. Y., Herbst, A. & Liu, Y. J. Spray drift potential evaluation of typical nozzles under wind tunnel conditions. Trans. CSAE. 21 (10), 78–81 (2005) ( (in Chinese with English abstract) ).

Wang, C. L. et al. Testing method of spatial pesticide spraying deposition quality balance for unmanned aerial vehicle. Trans. Chin. Soc. Agric. Eng. (Trans. CSAE) 32 (11), 54–61 (2016) ( (in Chinese with English abstract) ).

Wang, L. et al. Design of Variable spraying system and influencing factors on droplets deposition of small UAV. Trans. Chin. Soc. Agric. Mach. 47 (1), 1–8 (2016).

Qin, W. C., Xue, X. Y., Zhou, L. X. & Wang, B. K. Effects of spraying parameters of unmanned aerial vehicle on droplets deposition distribution of maize canopies. Trans. Chin. Soc. Agric. Eng. (Trans. CSAE) 30 (5), 50–56 (2014) ( (in Chinese with English abstract) ).

Gao, Y. Y. et al. Primary studies on spray droplet distribution and control effects of aerial spraying using unmanned aerial vehicle (UAV) against the corn borer. Plant Prot. 39 (2), 152–157 (2013).

Qiu, B. J. et al. Effects of flight altitude and speed of unmanned helicopter on spray deposition uniform. Trans. Chin. Soc. Agric. Eng. (Trans. CSAE) 29 (24), 25–32 (2013) ( (in Chinese with English abstract) ).

Qin, W. C., Qiu, B. J., Xue, X. Y. & Wang, B. K. Droplet deposition and control effect of insecticides sprayed with an unmanned aerial vehicle against plant hoppers. Crop Prot. 85 , 79–88 (2016).

Hewitt, A. J. Droplet size spectra classification categories in aerial application scenarios. Crop Prot. 27 (9), 1284–1288 (2008).

Gil, E., Llorens, J., Llop, J., Fàbregas, X. & Gallart, M. Use of a terrestrial LIDAR sensor for drift detection in vineyard spraying. Sensors (14248220) 13 (1), 516–534. https://doi.org/10.3390/s130100516 (2013).

Article   ADS   Google Scholar  

Huang, Y., Hoffmann, W. C., Lan, Y., Wu, W. & Fritz, B. K. Development of a spray system for an unmanned aerial vehicle platform. Appl. Eng. Agric. 25 (6), 803–809 (2009).

Gaskin, R. E., Steele, K. D. & Foster, W. A. Characterizing plant surfaces for spray adhesion and retention. N. Z. Plant Prot. 58 , 179–183 (2009).

Zhu, J. W., Zhou, G. J., Cao, Y. B., Dai, Y. Y. & Zhu, G. N. Characteristics of fipronil solution deposition on paddy rice leaves. Chin. J. Pestic. Sci. 11 (2), 250–254 (2009).

Diepenbrock, W. Yield analysis of winter oilseed rape ( Brassica napus L.): A review. Field Crops Res. 67 , 35–49 (2000).

Song, J. L., He, X. K. & Yang, X. L. Influence of nozzle orientation on spray deposits. Trans. CSAE 22 (6), 96–99 (2006) ( (in Chinese with English abstract) ).

ADS   Google Scholar  

Chen, S. D. et al. Effect of spray parameters of small unmanned helicopter on distribution regularity of droplet deposition in hybrid rice canopy. Trans. Chin. Soc. Agric. Eng. (Trans. CSAE) 32 (17), 40–46 (2016) ( (in Chinese with English abstract) ).

Xiong, Z. O. U., Rangshu, X. U., Jingchun, L. I. & Zilin, L. I. U. Particle kinematics analysis of droplet drift in spraying operation of plant protection UAV. Plant Dis. Pests. 13 (2), 17–23. https://doi.org/10.19579/j.cnki.plant-d.p.2022.02.006 (2022).

Gil, E. et al. Influence of wind velocity and wind direction on measurements of spray drift potential of boom sprayers using drift test bench. Agric. For. Meteorol. 202 , 94–101 (2015).

Ferreira, M. C., Miller, P. C. H., Tuck, C. R., O’Sullivan, C. M., Balsari, P., Carpenter, P. I., Cooper, S. E. & Magri B. (2010). Comparison of sampling arrangements to determine airborne spray profiles in wind tunnel conditions. Asp. Appl. Biol. Int. Adv. Pest. Appl. 291–296.

Qi, L. J., Hu, J. R., Shi, Y. & Fu, Z. T. Correlative analysis of drift and spray parameters. Trans. Chin. Soc. Agric. Eng. (Trans. CSAE) 5 (20), 122–125 (2004).

Zhang, R. R. et al. Spraying atomization performance by pulse width modulated variable and droplet deposition characteristics in wind tunnel. Trans. Chin. Soc. Agric. Eng. (Trans. CSAE). 35 (3), 42–51 (2019) ( (in Chinese with English abstract) ).

Hilz, E. & Vermeer, A. W. Spray drift review: The extent to which a formulation can contribute to spray drift reduction. Crop Prot. 44 , 75–83 (2013).

Bai, G. et al. Characteristics and classification of Japanese nozzles based on relative spray drift potential. Crop Prot. 46 , 88–93 (2013).

Jiao, Y. et al. Experimental study of the droplet deposition characteristics on an unmanned aerial vehicle platform under wind tunnel conditions. Agronomy 12 (12), 3066. https://doi.org/10.3390/agronomy12123066 (2022).

Article   CAS   Google Scholar  

Hongshan, L., Yubin, L., Xinyu, X., Zhiyan, Z. & Xiwen, L. Development of wind tunnel test technologies in agricultural aviation spraying. Trans. Chin. Soc. Agric. Eng. (Trans. CSAE) 31 (Supp. 2), 1–10 (2015).

Fu, Z. T. & Qi, L. J. Wind tunnel spraying drift measurements. Trans. Chin. Soc. Agric. Eng. (Trans. CSAE). 15 (1), 115–118 (1999) ( (in Chinese with English abstract) ).

Wang, Z. et al. Stereoscopic test method for low-altitude and low-volume spraying deposition and drift distribution of plant protection UAV. Trans. Chin. Soc. Agric. Eng. (Trans. CSAE) 36 (4), 54–62. https://doi.org/10.11975/j.issn.1002-6819.2020.04.007 (2020) ( (in Chinese with English abstract) ).

Ding, S. M., Xue, X. Y. & Lan, Y. B. Design and experiment of NJS-1 type open-circuit closed wind tunnel for plant protection. Trans. Chin. Soc. Agric. Eng. (Trans. CSAE). 31 (4), 76–84 (2015) ( (in Chinese with English abstract) ).

Wang, M. X. & Zhuang, K. L. Review on helicopter rotor model wind tunnel test. Aerodyn. Exp. Meas. Control. 5 (3), 9–16 (1991).

MathSciNet   Google Scholar  

Chen, Z., Guo, Y. C. & Gao, C. Principle and technology of three-dimensional PIV. J. Exp. Fluid Mech. 20 (4), 77–82 (2006).

Xiaonan, W. A. N. G., Peng, Q. I., Congwei, Y. U. & Xiongkui, H. E. Research and development of atomization, deposition and drift of pesticide droplets. Chin. J. Pestic. Sci./Nongyaoxue Xuebao 24 (5), 1065–1079. https://doi.org/10.16801/j.issn.1008-7303.2022.0111 (2022).

Andre, W., Volker, L., Jan, C., Zande, J. & Harry, V. Field experiment on spray drift: Deposition and airborne drift during application to a winter wheat crop. Sci. Total Environ. 405 , 269–277 (2008).

Wang, C. et al. Testing method and distribution characteristics of spatial pesticide spraying deposition quality balance for unmanned aerial vehicle. Int. J. Agric. Biol. Eng. 11 (2), 18–26. https://doi.org/10.25165/j.ijabe.20181102.3187 (2018).

Clement, M., Arzel, S., Le Bot, B., Seux, R. & Millet, M. Adsorption/thermal desorption-GC/MS for the analysis of pesticides in the atmosphere. Chemosphere 40 (1), 49–56 (2000).

Article   ADS   CAS   PubMed   Google Scholar  

Teske, M. E., Miller, P. C. H., Thistle, H. W. & Birchfield, N. B. Initial development and validation of a mechanistic spray drift model for ground boom sprayers. Trans. ASABE 52 (4), 1089–1097 (2009).

Chen, S., Lan, Y., Zhou, Z., Ouyang, F. & Wang, G. Effect of droplet size parameters on droplet deposition and drift of aerial spraying by using plant protection UAV. J. Agron. 10 , 195 (2020) ( (in Chinese with English abstract) ).

Duga, A. T. et al. Numerical analysis of the effects of wind and sprayer type on spray distribution in different orchard training systems. Bound. Layer Meteorol. 157 (3), 517–535 (2015).

Xiaohui, L. I. U. et al. Research progress on spray drift of droplets of plant protection machainery. Chin. J. Pestic. Sci. Nongyaoxue Xuebao 24 (2), 232–247. https://doi.org/10.16801/j.issn.1008-7303.2021.0166 (2022).

Gil, E., Gallart, M., Balsari, P., Marucco, P. & Liop, J. Influence of wind velocity and wind direction on measurements of spray drift potential of boom sprayers using drift test bench. Agric. For. Meteorol. 202 , 94–101 (2015).

Gregorio, L. E. et al. LIDAR as an alternative to passive collectors to measure pesticide spray drift. Atmos. Environ. 82 , 83–93 (2014).

Article   ADS   CAS   Google Scholar  

Feng, K. et al. Research progress and prospect of pesticide droplet deposition characteristics. Trans. Chin. Soc. Agric. Eng. (Trans. CSAE) 37 (20), 1–14. https://doi.org/10.11975/j.issn.1002-6819.2021.20.001 (2021) ( (in Chinese with English abstract) ).

Kruckeberg, J. P. et al. The relative accuracy of DRIFTSIM when used as a real-time spray drift predictor. Trans. ASABE 55 (4), 1159–1165 (2012).

Li, H., Zhu, H., Jiang, Z. & Lan, Y. Performance characterization on downwash flow and spray drift of multirotor unmanned agricultural aircraft system based on CFD. Int. J. Agric. Biol. Eng. 15 (3), 1–8. https://doi.org/10.25165/j.ijabe.20221503.7315 (2022).

Dorr, G. J. et al. Spray retention on whole plants: Modelling, simulations and experiments. Crop Prot. 88 , 118–130 (2016).

Zabkiewicz, J. A. et al. Simulating spray droplet impaction outcomes: Comparison with experimental data. Pest Manag. Sci. 76 (10), 3469–3476 (2020).

Article   CAS   PubMed   Google Scholar  

Miller, P. C. H. & Hadfield, D. J. A simulation model of the spray drift from hydraulic nozzles. J. Agric. Eng. Res. 42 (2), 135–147 (1989).

Zhang, B., Tang, Q., Chen, L., Zhang, R. & Xu, M. Numerical simulation of spray drift and deposition from a crop spraying aircraft using a CFD approach. Biosyst. Eng. 166 , 184–199. https://doi.org/10.1016/j.biosystemseng.2017.11.017 (2018).

Holterman, H. J., Van De Zande, J. C., Porskamp, H. A. J. & Huijsmans, J. F. M. Modeling spray drift from boom sprayers. Comput. Electron. Agric. 19 (1), 1–22 (1997).

Zhang, D. et al. Numerical simulation and analysis of the deposition shape of the droplet jetting collision. J. Xi’an Polytech. Univ. 30 (1), 112–117 (2016).

Tang, Q., Zhang, R., Chen, L., Li, L. & Xu, G. Research progress of key technologies and verification methods of numerical modeling for plant protection unmanned aerial vehicle application. Smart Agric. 3 (3), 1–21 (2021) ( (in Chinese with English abstract) ).

Zhang, R. et al. Fluorescence tracer method for analysis of droplet deposition pattern characteristics of the sprays applied via unmanned aerial vehicle. Trans. Chin. Soc. Agric. Eng. (Trans. CSAE) 36 (6), 47–55. https://doi.org/10.11975/j.issn.1002-6819.2020.06.006 (2020) ( (in Chinese with English abstract) ).

Na, G., Liu Siyao, Xu., Hui, T. S. & Tianlai, Li. Improvement on image detection algorithm of droplets deposition characteristics. Trans. Chin. Soc. Agric. Eng. (Trans. CSAE) 34 (17), 176–182 (2018) ( (in Chinese with English abstract) ).

Zhu, H. et al. DRIFTSIM, A program to estimate drift distances of spray droplets. Appl. Eng. Agric. 11 (3), 365–369 (1995).

Hong, S., Zhao, L. & Zhu, H. CFD simulation of pesticide spray from air-assisted sprayers in an apple orchard: Tree deposition and off-target losses. Atmos. Environ. 175 , 109–119 (2018).

Xiahou, B., Sun, D., Song, S., Xue, X. & Dai, Q. Simulation and experimental research on droplet flow characteristics and deposition in airflow field. Int. J. Agric. Biol. Eng. 13 (6), 16–24. https://doi.org/10.25165/j.ijabe.20201306.5455 (2020).

Yang, W., Li, X., Li, M. & Hao, Z. Droplet deposition characteristics detection method based on deep learning. Comput. Electron. Agric. 198 , 107038. https://doi.org/10.1016/j.compag.2022.107038 (2022).

Download references

This research was funded by the National Natural Science Foundation of China (Grant No. 31971804); Independent Innovation Project of Agricultural Science and Technology in Jiangsu Province (CX(21)3091); Suzhou Agricultural Independent Innovation Project (SNG2022061); and Suzhou Agricultural Vocational and Technical College Landmark Achievement Cultivation Project (CG[2022]02).

Author information

Authors and affiliations.

Suzhou Polytechnic Institute of Agriculture, Suzhou, 215008, China

Nanjing Institute of Agricultural Mechanization, Ministry of Agriculture and Rural Affairs, Nanjing, 210014, China

Nanjing Institute of Technology, Nanjing, 211167, China

Chen Panyang

You can also search for this author in PubMed   Google Scholar

Contributions

Q.W. conceived and designed the study. Q.W. and C.P. performed most experiments. Q.W. analyzed the data and wrote the first draft of the manuscript. C.P. revised the manuscript. Q.W. supervised the project and reviewed the manuscript.

Corresponding author

Correspondence to Qin Weicai .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and Permissions

About this article

Cite this article.

Weicai, Q., Panyang, C. Analysis of the research progress on the deposition and drift of spray droplets by plant protection UAVs. Sci Rep 13 , 14935 (2023). https://doi.org/10.1038/s41598-023-40556-0

Download citation

Received : 20 February 2023

Accepted : 12 August 2023

Published : 11 September 2023

DOI : https://doi.org/10.1038/s41598-023-40556-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

This paper is in the following e-collection/theme issue:

Published on 12.9.2023 in Vol 25 (2023)

Construction of an Emotional Lexicon of Patients With Breast Cancer: Development and Sentiment Analysis

Authors of this article:

Original Paper

• Chaixiu Li 1, 2 * , MD   ; 
• Jiaqi Fu 1, 2 * , MD   ; 
• Jie Lai 1, 2 * , MD   ; 
• Lijun Sun 3 * , MSE   ; 
• Chunlan Zhou 1 , MD   ; 
• Wenji Li 1 , BSc   ; 
• Biao Jian 3 , MSE   ; 
• Shisi Deng 1, 2 , MD   ; 
• Yujie Zhang 1, 2 , MD   ; 
• Zihan Guo 1, 2 , MD   ; 
• Yusheng Liu 1 , BSc   ; 
• Yanni Zhou 1 , MD   ; 
• Shihui Xie 1 , MD   ; 
• Mingyue Hou 1 , MD   ; 
• Ru Wang 1 , MD   ; 
• Qinjie Chen 1 , MD   ; 
• Yanni Wu 1 , PhD  

1 Nanfang Hospital, Southern Medical University, Guangzhou, China

2 School of Nursing, Southern Medical University, Guangzhou, China

3 China Electronic Product Reliability and Environmental Testing Institute, Guangzhou, China

*these authors contributed equally

Corresponding Author:

Yanni Wu, PhD

Nanfang Hospital

Southern Medical University

No 1838 Guangzhou Avenue North

Baiyun District, Guangdong Province

Guangzhou, 510515

Phone: 86 020 61641192

Email: [email protected]

Background: The innovative method of sentiment analysis based on an emotional lexicon shows prominent advantages in capturing emotional information, such as individual attitudes, experiences, and needs, which provides a new perspective and method for emotion recognition and management for patients with breast cancer (BC). However, at present, sentiment analysis in the field of BC is limited, and there is no emotional lexicon for this field. Therefore, it is necessary to construct an emotional lexicon that conforms to the characteristics of patients with BC so as to provide a new tool for accurate identification and analysis of the patients’ emotions and a new method for their personalized emotion management.

Objective: This study aimed to construct an emotional lexicon of patients with BC.

Methods: Emotional words were obtained by merging the words in 2 general sentiment lexicons, the Chinese Linguistic Inquiry and Word Count (C-LIWC) and HowNet, and the words in text corpora acquired from patients with BC via Weibo, semistructured interviews, and expressive writing. The lexicon was constructed using manual annotation and classification under the guidance of Russell’s valence-arousal space. Ekman’s basic emotional categories, Lazarus’ cognitive appraisal theory of emotion, and a qualitative text analysis based on the text corpora of patients with BC were combined to determine the fine-grained emotional categories of the lexicon we constructed. Precision, recall, and the F1-score were used to evaluate the lexicon’s performance.

Results: The text corpora collected from patients in different stages of BC included 150 written materials, 17 interviews, and 6689 original posts and comments from Weibo, with a total of 1,923,593 Chinese characters. The emotional lexicon of patients with BC contained 9357 words and covered 8 fine-grained emotional categories: joy, anger, sadness, fear, disgust, surprise, somatic symptoms, and BC terminology. Experimental results showed that precision, recall, and the F1-score of positive emotional words were 98.42%, 99.73%, and 99.07%, respectively, and those of negative emotional words were 99.73%, 98.38%, and 99.05%, respectively, which all significantly outperformed the C-LIWC and HowNet.

Conclusions: The emotional lexicon with fine-grained emotional categories conforms to the characteristics of patients with BC. Its performance related to identifying and classifying domain-specific emotional words in BC is better compared to the C-LIWC and HowNet. This lexicon not only provides a new tool for sentiment analysis in the field of BC but also provides a new perspective for recognizing the specific emotional state and needs of patients with BC and formulating tailored emotional management plans.

Introduction

In 2020, breast cancer (BC) became the most commonly diagnosed cancer in the world, and there were more than 2.26 million new cases of BC and almost 685,000 deaths from BC worldwide [ 1 ]. The diagnosis of BC usually occurs at a stage when women are in the middle of career development or child-rearing and unprepared or unable to cope with the risk of lifelong recurrence and death [ 2 ]. Notably, the treatment side effects, together with prognostic uncertainty, cause patients to suffer negative emotional experiences, such as body image disturbance [ 3 ] and recurrence and heredity confusion [ 4 ], which have been negatively associated with treatment adherence and the quality of life [ 2 ]. There has been increasing attention placed on early identification and treatment of emotional distress in patients with cancer, which has been regarded as the “sixth vital sign” [ 5 ].

In Chinese culture, free expression of emotions, especially negative ones, may temporarily disrupt group harmony [ 6 , 7 ]. Chinese women, especially, are conflicted about disclosing emotional distress and experience high levels of ambivalence about doing so [ 8 , 9 ]. Meanwhile, socially constrained responses may be negatively associated with relationship satisfaction, aggravate self-stigmatization, and result in persistent emotional distress and reduced self-efficacy in coping with stress [ 7 ]. These conditions are not conducive to health care professionals’ and caregivers’ timely identification of patients’ emotions or their ability to provide corresponding emotional support. Furthermore, owing to the lack of mental health knowledge, assessment tools with interference, and patient resistance because of cancer-related stigma [ 10 , 11 ], there are some insurmountable obstacles in the emotional management and psychological care of patients with BC.

Extensive research in psychology has shown that word use and linguistic features can reflect individuals’ thoughts, emotions, and experiences and thus can be used to identify their social and psychological states [ 12 , 13 ]. For example, numerous studies have shown that expressive writing (EW) and interviews are effective ways to listen to the patients’ voice [ 10 , 14 ]. Furthermore, recently, social media platforms, such as Facebook, Twitter, and Weibo, have emerged as a rich yet largely untapped resource for understanding what patients are frankly saying about their experiences and thoughts, which has provided a new breakthrough point for people’s sentiment analysis [ 15 - 17 ]. Therefore, for people who have little chance or who do not take the initiative to disclose their mental conditions to health care professionals, we can capture their emotional expressions from their written, verbal, or online text materials. Thus, interventions and more targeted treatments can be administered to patients with potential emotional distress.

Sentiment analysis is the computational study of an individual’s opinions, emotions, and attitudes [ 18 ], which is an effective method that helps analyze and interpret enormous amounts of data and information, thereby identifying and extracting people’s opinions and emotions [ 16 , 19 , 20 ]. Research on individual emotions, spirit, and psychological detection based on deep learning and emotional lexicons is increasing [ 21 - 24 ]. The lexicon-based method of sentiment analysis may be the simplest and most basic method to analyze emotional polarity [ 18 , 19 , 25 ]. An emotional lexicon, which consists of a list of sentiment words or phrases, as well as their sentiment polarities and intensities, is the most important component in sentiment analysis systems and plays an important role in sentiment analysis tasks with different text granularities, such as words, phrases, and sentences [ 26 , 27 ]. Although the efficiency of sentiment analysis based on machine learning is high, the model training in this method is highly dependent on the quantity and quality of labeled data sets [ 20 , 28 , 29 ]. Almost all these data sets come from the internet, ignoring the emotional text information generated in other forms by individuals [ 30 , 31 ]. Therefore, in the absence of a sufficient high-quality training corpus, the lexicon-based method of sentiment analysis has more advantages.

The innovative method of sentiment analysis based on an emotional lexicon has prominent advantages in capturing emotional information, such as individual attitudes, experiences, and needs, which provides a new perspective and method for emotion recognition and management for patients with BC. However, at present, sentiment analysis in the field of BC is limited, and there is no emotional lexicon for this field. Therefore, it is necessary to construct an emotional lexicon that conforms to the characteristics of patients with BC so as to fill the missing gaps in sentiment analysis of BC and provide a new tool for accurate identification and analysis of patients’ emotions and a new method for their personalized emotion management. This study aimed to manually construct an emotional lexicon for patients with BC, which can be uploaded into mainstream text analytic software (eg, Linguistic Inquiry and Word Count [LIWC]-22) to help researchers identify and analyze terms associated with emotions in a text-based corpus or writing content (eg, news, diary) so as to understand the expressions of those emotions embedded in spoken and written languages.

Related Work

Sentiment analysis approaches.

Many sentiment analysis approaches have been proposed in the past few years. Polarity detection is the most common form of sentiment analysis, which can be classified into 3 main types: lexicon-based approaches, supervised learning methods, or semisupervised learning methods [ 18 , 19 , 25 ]. Numerous state-of-the-art approaches to sentiment analysis rely on supervised or semisupervised learning techniques [ 18 ], but both approaches may be combined into hybrid methods as well. Although sentiment analysis based on supervised or semisupervised learning methods and a general emotional lexicon is common, some studies have pointed out that they cannot effectively analyze texts in specific fields, such as health care [ 32 - 34 ]. These tools, albeit extremely cost-efficient and versatile in certain analytic settings, have problems, such as a lack of sufficient details regarding algorithm development, limited use for task- and theory-specific research, and the requirement of highly structured data sets [ 32 , 35 ]. Moreover, sufficient labeled training data are required in supervised learning methods for sentiment analysis, and training data acquisition becomes a laborious process [ 33 , 36 ].

Lexicon-based sentiment analysis methods tend to sacrifice computational efficiency for classification accuracy, which is typically inferior to the classification accuracy of machine learning techniques in specific domains in which machine learning models can be trained and optimized [ 32 , 34 ]. Surprisingly, lexicon-based methods have an attractive advantage over machine learning methods in that they have more robust performance across domains and texts and can be generalized relatively easily to other languages using lexicons [ 34 ]. Additionally, lexicon-based methods enable deep linguistic analysis to be incorporated into the sentiment analysis process [ 25 ], which, if fine-tuned, can improve classification accuracy.

Sentiment Analysis in the Field of BC

The traditional clinical diagnosis of patients’ psychological problems requires not only filling in some evaluation scales, such as the Self-Rating Depression Scale, but also conducting 1-on-1 interviews and long-term observation [ 5 , 11 , 20 ], which may have limitations in detection efficiency, since all require good cooperation from patients [ 20 ]. Fortunately, things are changing with the development of sentiment analysis. For example, Cabling et al [ 31 ] conducted sentiment analysis of an online BC support group regarding tamoxifen to understand users’ emotions and opinions. Clark et al [ 37 ] investigated over 48,000 BC-related tweets using econometric sentiment analysis to quantitatively extract emotionally charged topics. Praveen et al [ 38 ] used advanced machine learning techniques to understand and analyze the attitudes of people who have survived BC, while discussing their experience of surviving and the stress associated with it. Compared with traditional approaches, these methods have been proven effective and inexpensive and have been shown to reduce limitations and assist in clinical diagnosis in a more flexible way.

Research on Emotional Lexicon Construction

An emotional lexicon is a collection of words or phrases that convey feelings [ 18 , 20 ]. It contains words and assigns sentiment scores or classes to single terms. Each entry in an emotional lexicon is associated with its sentiment orientation or strength [ 18 , 26 ]. Entries in a lexicon can be divided into categories according to their sentiment orientations, such as positive or negative [ 26 , 39 ]. There are several well-known general emotional lexicons, such as the Chinese Linguistic Inquiry and Word Count (C-LIWC) [ 40 ], HowNet [ 41 ], and the General Inquirer (GI) [ 42 ]. Emotional lexicons are constructed manually or automatically [ 20 , 27 , 36 , 43 , 44 ].

In the manual method, expert annotators annotate the emotional polarity and classification of words [ 36 ]. Some widely used emotional lexicons have been constructed for emotion analysis and application manually or semiautomatically [ 36 , 39 , 42 , 45 ]. For example, the GI lexicon [ 42 ] provides a binary classification (positive/negative) of approximately 4000 sentiment-bearing words manually annotated. The Affective Norms for English Words (ANEW) [ 45 ] provides valence scores for roughly 1000 words manually assigned by several annotators. Similarly, the Semantic Orientation CALculator (SO-CAL) entries [ 26 ] consist of roughly 4000 words manually tagged by a small number of linguists with a multiclass label (from very negative to very positive). In addition, the Dictionary of Affect in Language (DAL) contains roughly 9000 words manually rated along the dimensions of pleasantness, activation, and imagery [ 46 ].

The automatic method involves calculating the polarity and intensity of emotional words using some algorithms, such as co-occurrence information and context information in the corpus, to construct an emotional lexicon [ 20 , 26 , 27 , 43 ]. For example, Yang et al [ 47 ] constructed a hotel sentiment lexicon based on users’ behavior and the improved Semantic Orientation Pointwise Mutual Information (SO-PMI) algorithm and then used the lexicon for feature extraction contrast. Li et al [ 43 ] proposed a deep learning–based framework to construct a Chinese financial domain sentiment lexicon, using word vector models and deep learning–based classifiers in the process. Additionally, Li et al [ 20 ] extracted sentiment words from WordNet-Affect and calculated the co-occurrence frequency between the words and each emoji constructed in their manually labeled emoji sentiment lexicon in order to automatically expand the lexicon. Furthermore, Chao et al [ 27 ] proposed a semisupervised sentiment orientation classification algorithm based on Word2Vec and obtained a lexicon in different areas efficiently.

Both lexicons constructed with these 2 methods (manual or automatic) have shown satisfactory results. Although the time and cost associated with annotation tasks are high, the highest precision is obtained with manual annotation and it is deemed more accurate than other methods [ 36 , 44 ]. Automatically created training corpora are usually larger, but the precision is highly dependent on the machine learning algorithm [ 32 , 34 , 36 ]. Furthermore, the manual method can fully consider the polysemy property of words and the indistinctness property of sentiment categories according to the context of words [ 44 , 48 ]. Conversely, in some cases, the automatic method fails to extract some implicit features or aspects of the special-domain text [ 26 , 35 , 36 ]. Therefore, considering the particularity of emotional expression in the text corpus of BC mentioned before, we finally decided to manually construct a BC domain–specific emotional lexicon.

General Emotional Lexicons: C-LIWC and HowNet

A text analysis application LIWC has been developed to provide a better method for studying verbal and written text samples [ 40 , 49 , 50 ]. The LIWC, including a software program and a lexicon, is one of the most well-known lexicons in quantitative text analysis [ 50 , 51 ]. The LIWC was first released in the early 1990s and has been updated several times, with the latest version (LIWC-22) released in 2022 [ 49 , 50 ]. This most recent evolution, LIWC-22, is designed to accept written or transcribed verbal text that has been stored as a digital, machine-readable file in one of multiple formats [ 49 ]. During operation, the LIWC-22 software processing module accesses each text in the data set and compares the language within each text against the lexicon selected [ 49 ].

Owing to its success and practicability, the LIWC has been translated and adapted to its Chinese version (C-LIWC) by humanities and social sciences researchers at the Taiwan Province University of Science and Technology according to Chinese characteristics and culture [ 52 - 54 ]. There are 30 kinds of language-specific words (eg, auxiliary words and prepositions) and 42 kinds of psychological-specific words (eg, positive and negative emotional words), with a total of 6862 words in 72 categories in the C-LIWC [ 52 ]. Notably, each word in the C-LIWC has one or more category attributes. Nowadays, there is an increasing number of applications based on the C-LIWC, and it has been used in hundreds of studies across the social sciences, such as psychology, sociology, and communication [ 28 , 55 , 56 ].

Additionally, nowadays, people manually annotate and build many linguistic knowledge bases. HowNet, a widely used Chinese emotional lexicon, is a typical knowledge base created by the Computer Language Information Center of Chinese Academy of Sciences, which takes concepts represented by Chinese and English words as the description object and reveals the relationship between those concepts and their attributes as the basic content [ 41 , 57 ]. Unlike the C-LIWC, there are no more specific classifications of emotions in HowNet, but it has 17,887 phrases, which are divided into 6 groups based on their emotional tendency: positive evaluation, negative evaluation, positive emotion, negative emotion, perception, and adverb of degree [ 57 ]. HowNet has also been widely used in sentiment analysis [ 53 , 54 ].

Various informal text data and the increasing network neologisms on the internet have made it difficult for machines to perform sentiment analysis [ 15 , 20 ]. In addition, there are many polysemous words in Chinese, so the emotional categories of these words need to be judged manually according to the context [ 44 , 48 , 58 ]. Furthermore, some of the corpora and lexicons are domain specific, which limits their reuse in other domains. Considering that there is no emotional lexicon in the field of BC or even cancer at present, this study aimed to manually construct an emotional lexicon of patients with BC based on the general emotional lexicons the C-LIWC and HowNet and the text corpora of patients with BC in different stages of BC obtained through EW, semistructured interviews, and the Python web crawler of Weibo.

Study Design

An overview of the construction process of the emotional lexicon of patients with BC is presented in Figure 1 . Specifically, to ensure the domain specificity and typicality of words in our emotional lexicon of patients with BC, we used the text corpora of patients with BC obtained from EW, semistructured interviews, and Weibo. Next, drawing upon Goeuriot et al’s [ 59 ] approach to domain emotional lexicon construction, we constructed our lexicon by merging the words in the BC text corpora we collected, together with the words in the C-LIWC and HowNet, to ensure comprehensive coverage by the lexicon.

First, all the words obtained from the text corpora after data preprocessing and segmenting were regarded as word set 1 (a Microsoft Excel sheet). Second, based on the valence-arousal space [ 60 ], 15 annotators manually judged whether the words in word set 1 were emotional; those words that were emotional and met the inclusion criteria were screened out as word set 2 (an Excel sheet). Third, we combined the words in word set 2 with the words in the C-LIWC and HowNet, removed repeated words, and finally included all the remaining words in word set 3 (an Excel sheet). Fourth, 15 annotators independently labeled the words in word set 3 according to the emotional word categories based on the valence-arousal space [ 60 ] and Ekman and Oster’s [ 61 , 62 ] 6 basic emotions. Finally, the labeling results were summarized as word set 4 (an Excel sheet), and then, the words in word set 4 that met the classification criteria of this study were sorted out to form the final emotional lexicon of patients with BC.

In the field of machine learning and data mining, precision (P), recall (R), and the F 1 -score are used as performance evaluation indicators of an emotional lexicon [ 19 , 20 ]. Therefore, finally, we compared in LIWC-22 software the results of P, R, and the F 1 -score of the positive and negative emotional words identified and classified in BC texts analyzed using the emotional lexicon of patients with BC, HowNet, and the C-LIWC. Results verified the effectiveness of the lexicon construction method used in this paper from the perspective of the identification and classification effect of emotional words.

Text Corpora Acquisition

To ensure the domain specificity and coverage of the text corpora of patients with BC, EW, semistructured interviews, and the Python web crawler of Weibo were used to obtain the written, verbal, and online corpora of patients with BC. Patients in different stages of BC may experience drastically different emotions and cognitions; therefore, considering patients’ distress peaks and the difference in phase specificity [ 2 , 63 - 65 ], the text corpora of EW and semistructured interviews of female patients with BC (newly diagnosed, postoperative, or undergoing chemotherapy) were collected. Furthermore, with respect to the potential differences in the phase specificity and acuteness of patient emotions, “newly diagnosed” and “postoperative” were defined as “within 1 month of a new diagnosis” and “1 month postsurgery,” respectively, consistent with previous studies [ 64 , 65 ].

EW participants were recruited from the breast surgery departments of 6 tertiary hospitals in 4 cities of China. Semistructured interview participants were selected from 1 of these 6 hospitals, and they did not participate in EW. Weibo participants were selected from among network users who posted within the supertopic #breast cancer# . Since participants on Weibo are anonymous, and we could thus only analyze the texts they disclosed on the internet, we were unable to apply inclusion criteria to this sample. The inclusion criteria for patients with BC included in the EW and semistructured interviews are presented in Multimedia Appendix 1 .

Acquisition of the Written Text Corpus

Pennebaker’s EW mode [ 13 ] was adopted to obtain the written text corpus of patients with BC in 3 different stages: newly diagnosed, postoperative, and undergoing chemotherapy. Consistent with the requirement to include maximal variation during purposive sampling for qualitative text analysis, we recruited 50 EW participants undergoing each of the aforementioned 3 phases [ 66 , 67 ]. Among them, 50 EW texts of participants undergoing chemotherapy were randomly selected from our previous study, a multicenter randomized controlled trial on the effect of prolonged EW on patients with BC undergoing chemotherapy [ 68 ]. Other data, including 50 EW texts of newly diagnosed patients and 50 EW texts of postoperative patients, were collected between June 2021 and January 2022. Patients were approached in their hospital rooms by 4 trained female nurse researchers following the guidelines of EW (see details in Multimedia Appendix 2 ).

Acquisition of the Verbal Text Corpus

Objective sampling and snowball sampling were used to select patients with BC. Semistructured, face-to-face interviews were conducted by an experienced female researcher in a quiet conference room in the breast surgery department of a tertiary hospital at the patients’ convenience according to the interview guide (see details in Multimedia Appendix 2 ). Each interview lasted from 30 to 40 minutes and was digitally audio-recorded. Participant recruitment for the semistructured interviews ended when data saturation was achieved, that is, when no new information emerged [ 69 ].

Acquisition of the Online Text Corpus

As an emerging multimedia platform, with its advantages (eg, instant, user-friendly to the grassroots, zero-access restriction, high interactivity, weak control, and fission-style mode of dissemination), Weibo has gradually become cybercitizens’ first choice to obtain information and express their opinions in China [ 15 ]. Moreover, it also provides platforms for emotion research. Therefore, to collect the online text corpus, we designed a project-developed web crawler in Python language, which used the Weibo application programming interface (API) to systematically scrape the data from June 2021 to February 2022 on Weibo’s supertopic # breast cancer #, which involved user nicknames, posts, and comments. We summarized all posts and comments and numbered each text, starting from 1. Referring to the ratio of 7:3 for lexicon construction and the performance evaluation corpus in previous studies [ 29 ], 70% of the original online text corpus of Weibo was randomly selected as the construction corpus of the emotional lexicon of patients with BC.

Text Corpora Preprocessing

Considering the noise in the collected text corpora might affect the accuracy of research, we first integrated and denoised the text corpora obtained from 3 different sources to eliminate the invalid content in the original texts, such as advertisements, blanks, emoticons, punctuation marks, numbers, names of people, and duplicated text.

Next, with the help of the Jieba word segmentation toolkit in Python, we segmented the sentence-level corpora into word-level corpora. Due to the particularity of medical words, conventional machine word segmentation may result in the incorrect segmentation of some professional terms. Therefore, we manually verified all the machine-segmented words to reasonably revise any incorrect word segmentation. Given that there is no research on domain lexicon construction in the field of BC at present, we did not have any restrictions on the word frequency length. Thus, we incorporated all the words obtained after segmentation and manual verification into word set 1.

Determination of Fine-Grained Emotional Categories

At present, the coarse-grained emotional categories of positive and negative are commonly used in most emotional lexicons [ 15 , 53 ]. However, emotions are pervasive among humans, and facial expressions for basic human emotions are identical [ 61 ]. For complex and changeable emotional states, more detailed classification is needed to accurately reflect one’s true emotional state. Due to the limitation of emotional categories, coarse-grained categories not only lead to the fuzzy classification of specific emotions but also can only identify a limited number of emotional words, which cannot be effectively applied to the sentiment analysis of emotional information–rich texts in current social network platforms [ 39 ]. In contrast, the purpose of fine-grained emotional categories is to analyze more specific and real emotions in individual disclosure texts, such as happiness, anger, and disgust, so as to dig out one’s deeper emotions, attitudes and opinions, and other important information in the texts.

Ekman and Oster’s [ 61 , 62 ] basic emotional categories, Lazarus’ [ 70 , 71 ] cognitive appraisal theory of emotion, and a qualitative text analysis [ 8 ] based on all the text corpora we collected were combined to determine the fine-grained emotional categories of the emotional lexicon we constructed. The cognitive appraisal theory of emotion emphasizes that different appraisals and responses to the environment or events will produce different emotions and experiences, which indicates the diversity and complexity of emotions [ 71 ]. Ekman and Oster [ 61 ] put forward 6 basic emotional states by studying people’s facial expressions (joy, anger, sadness, fear, disgust, and surprise), which have been widely adopted by automatic emotion recognition research institutes in the field of natural language processing [ 72 ].

We first summed up 6 basic emotional categories. Furthermore, to improve the accuracy of sentiment analysis in the BC field, we added a seventh emotional category, “somatic symptoms,” based on the qualitative text analysis, the physical symptoms mentioned in the Distress Thermometer (DT) [ 73 ], and the MD Anderson Symptom Inventory [ 74 ]. Moreover, we found many high-frequency professional medical terminologies of BC through qualitative text analysis, such as “triple negative,” “mastectomy,” and “Her2,” all of which reflect strong emotional and knowledge needs. Therefore, we added “BC terminology” as the eighth emotional category.

To sum up, we finally defined 8 emotional categories, namely joy, anger, sadness, fear, disgust, surprise, somatic symptoms, and BC terminology.

Emotional Word Screening and Classification Annotation

In this step, 15 annotators, including 11 (73%) medical postgraduates and 4 (27%) nurses, in the breast surgery department with rich clinical experience were invited to manually revise the results of machine segmentation, screen emotional words, and annotate their emotion classification. In addition, we also obtained interannotator reliability scores to determine the accuracy of the annotation. The pivotal point in annotating emotional words is consistency. We used the term “interannotator reliability” to measure the consistency of annotation, which refers to the consistency of different individuals in annotating a particular concept [ 75 ]. The Fleiss κ statistic [ 76 ] was used to measure the interannotator reliability because it is highly flexible and it can be used for 2 or more categories as well as 2 or more raters [ 77 ]. The κ ranges and corresponding consistency strength interpretations were as follows: <0.00, poor; 0.00-0.20, slight; 0.21-0.40, fair; 0.41-0.60, moderate; 0.61-0.80, substantial; and 0.81-1.00, almost perfect agreement [ 77 , 78 ].

Emotional Word Screening

All the words obtained after segmentation and manual revising were included in word set 1. First, based on the valence-arousal space [ 60 ], 15 annotators were asked to independently manually judge whether the words in word set 1 were emotional. Valence represents the degree of pleasant and unpleasant (ie, positive and negative) feelings, while arousal represents the degree of excitement and calm. Based on this representation, any emotional state can be represented as a point in the valence-arousal coordinate plane [ 60 ]. Annotators marked the words that could arouse their emotional experience or emotional information as “yes,” and vice versa as “no,” and controversial words were marked as “uncertain.” These “uncertain” words were rescreened after discussion by all annotators. After summarizing and integrating the labeling results of all annotators, we stipulated that the words marked “yes” by more than half of the annotators (ie, ≥8, 53%, annotators) would be incorporated in word set 2 based on the research of Wu et al [ 44 ] and Zhou and Yang [ 79 ]. Referring to the emotional words contained in most existing emotional lexicons [ 44 , 72 ], we found that there are not only some domain-specific emotional words but also most conventional emotional words. In addition, due to the capacity of the constructed lexicon, we combined the emotional words obtained from the corpora with the emotional words in the existing general emotional lexicons to construct an emotional lexicon of patients with BC [ 59 ]. Therefore, next, we combined the words in word set 2 with the words in HowNet and the C-LIWC, then removed repeated words, and finally included all the remaining words in word set 3 for the next classification and annotation of emotional words.

Classification Annotation of Emotional Words

During this process, 15 annotators manually annotated the emotional category of each emotional word in word set 3 independently according to the 8 emotional categories stipulated in this study. Drawing lessons from the emotional classification standard of emotional words in the C-LIWC and the polysemy of Chinese words (ie, a word may belong to different emotional categories) [ 44 , 48 ], the following provisions were made referring to previous research on manual lexicon construction [ 44 ]: (1) the words that could not be classified would be marked as “none”; (2) if the same word was marked by more than 5 annotators in 2 or more categories, this word would be marked as belonging to 2 or more categories based on the research of Wu et al [ 44 ] and Zhou and Yang [ 79 ]; and (3) if more than 5 annotators marked a word as “none,” and the number of annotators who marked this word in other categories was less than 5, the word would be excluded from the emotional lexicon. Next, we used a random number table to randomly select the annotation results at a rate of 8% for the annotator consistency test [ 80 ].

Finally, after the researchers collected, counted, and sorted out the emotional words in each category and the number of people who marked each word in different categories, the emotional words that met the inclusion criteria and the corresponding emotional categories were recorded in word set 4 to form the final emotional lexicon of patients with BC.

Lexicon Performance Evaluation

In the fields of machine learning and data mining, P, R, and the F 1 -score are widely used for classification to evaluate the performance of lexicons [ 19 , 20 ]. The purpose of this step was to compare and analyze the effects of the emotional lexicon of patients with BC constructed in this study and the general emotional lexicons, C-LIWC and HowNet, on text analysis in the field of BC so as to evaluate the performance of the lexicon. The Weibo online text corpus were used for lexicon construction and performance evaluation at a ratio of 7:3 [ 29 ]. LIWC-22 software [ 49 ] was used to load the emotional lexicon of patients with BC, the C-LIWC, and HowNet and then calculate the 3 variables of text analysis after loading these 3 lexicons. The emotional words in HowNet are only divided into positive and negative emotional categories, while there are many detailed categories in the emotional lexicon of patients with BC and the C-LIWC. Therefore, to maintain the consistency of performance evaluation criteria and comparison indicators, we referred to previous studies [ 46 , 48 ] and stipulated that only P, R, and the F 1 -score of positive and negative emotional words analyzed using different lexicons in the same text corpus would be compared. Among the emotional categories in our study, joy is the only positive emotion, while anger, sadness, fear, and disgust all are negative emotions. Considering the polysemy of Chinese words [ 44 , 48 ], we stipulated that all words annotated with “joy” would be divided into the positive category, and any word annotated with one or more categories of “anger,” “sadness,” “fear,” and “disgust” would be regarded as a negative category.

For the convenience of understanding and calculation, we defined the following variables: P, R, and F 1 -score. These were calculated using [ 19 ]:

• True positives (TPs): number of words judged positive not only by the lexicon but also by manual judgment
• True negatives (TNs): number of words judged negative not only by the lexicon but also by manual annotation
• False positives (FPs): number of words judged positive by the lexicon but judged negative by manual annotation
• False negatives (FNs): number of words judged negative by the lexicon but judged positive by manual annotation

The mathematical formulas of P, R, and the F 1 -score are as follows [ 19 , 20 , 47 ]:

P = TP/(TP + FP)

R = TP/(TP + FN)

F 1 = 2PR/(P + R)

Ethical Considerations

The research was conducted in accordance with the Declaration of Helsinki and followed ethical principles and guidelines. Ethical approval for the study was granted by the Medical Ethics Committee of Nanfang Hospital of Southern Medical University (NFEC-2021-124), and a standardized informed consent form was established (V1.0/2021-4-14). Eligible participants were informed about the study, and they provided written informed consent for participation prior to the study. During the research, the patients could request to consult an experienced psychologist counselor. Code names were assigned to each participant instead of using their real names.

Results of Text Corpora Acquisition

The final emotional lexicon of patients with BC contained a total of 9357 words covering 8 fine-grained emotional categories: joy, anger, sadness, fear, disgust, surprise, physical symptoms, and BC terminology. In total, we collected 150 written texts, 17 interview texts, and 6689 original posts and comments from Weibo, with a total of 1,923,593 Chinese characters.

Results of Text Corpora Preprocessing

First, after deduplicating and removing the website’s automatic comments, spam comments, repeated texts, “@nicknames,” “#topic #,” web links, and other noise data, a total of 461,348 Chinese characters were obtained. Next, all the corpora were subjected to machine segmentation. A total of 13,661 words were segmented (reserving single words and keeping the word frequency≥1). We manually revised 3143 (23.01%) words that were incorrectly segmented by the machine. For example, the word “白蛋白(albumin)” was divided into 2 separate words: “白(white)” and “蛋白(protein).” Afterward, we removed 9582 (70.14%) common meaningless words, such as personal pronouns, prepositions, and adverbs of degree (eg, “you,” “of,” and “very”) after double-checking. Finally, we included 4079 (29.86%) words in word set 1 for the next stage of manual emotional word screening. The detailed flow of text corpora preprocessing is shown in Figure 2 .

Results of Emotional Word Screening and Classification Annotation

Results of emotional word screening.

In this step, 1829/4079 (44.84%) words were annotated as “no” or “uncertain.” In addition, 2250/4079 (55.16%) words were annotated as “yes”. Among the “uncertain” words, 15 (0.82%) words that met the requirements of our study after a second group discussion were reselected as emotional words. After the first manual annotation by 15 annotators to check whether the words in word set 1 were emotional, we obtained 1998/2250 (88.80%) emotional words that met the requirements of our study. Therefore, a total of 2013/4079 (49.35%) emotional words annotated as “yes” by more than half of the annotators were selected to form word set 2.

We randomly selected the annotation results of 327/4079 (8%) words from the corpora of patients with BC for the annotator consistency test [ 80 ], and the Fleiss κ value was 0.491 (95% CI 0.482-0.501), with moderate strength of agreement, which showed that the annotation results were consistent and the consistency was acceptable.

In merging the emotional words in word set 2 with the emotional words in the C-LIWC and HowNet, and removing repeated words, 14,709 words were obtained and finally included in word set 3. The detailed process of emotional word screening and determination is shown in Figure 3 .

Results of Emotional Word Classification Annotation

In this process, 15 annotators marked the emotional words according to the 8 emotional categories specified in this study. The annotating results were collated in word set 4, and those words that met the classification criteria were sorted out to form the final emotional lexicon of patients with BC. The emotional lexicon of patients with BC eventually contained a total of 9357 words reflecting 8 discrete emotional constructs: joy, anger, sadness, fear, disgust, surprise, physical symptoms, and BC terminology. See Multimedia Appendix 3 for the number and examples of emotional words based on 8 emotional categories in the emotional lexicon of patients with BC. We randomly selected the annotation results of 1471 of 14,709 sentiment words at a rate of 8% for the annotator consistency test [ 80 ], and the Fleiss κ value was 0.439 (95% CI 0.437-0.441), with moderate strength of agreement, which showed that the annotators’ annotation results were consistent and the consistency was acceptable.

We noted that the 8 emotional categories are not evenly represented in terms of the numbers of items; however, this should not be interpreted as one construct being more prevalent or important than the others but as a natural occurrence of language. Further, we noted that the sum of terms in each emotional category exceeds the total word count of the lexicon; that is because some words/word stems are cross-listed. For instance, “frightened” appears in both the fear and surprise categories; similarly, “berate” is listed in both the anger and disgust categories. Interestingly, some words in the surprise category can reflect both positive and negative emotions, such as “fantastic,” which can be negative when describing a strange phenomenon and positive when describing something wonderful. Additionally, combined with the annotation results and the context in the original BC text corpus, we found that these words in somatic symptoms are often used to reflect morbid problems that have caused obvious disorders in patients with BC, so the emotional arousal degree of these words is high. Furthermore, the words in BC terminology were found to be commonly used objective medical terms in the field of BC, such as physiological and biochemical indicators, chemotherapy plans, and drug names, so the emotional arousal degree of these words is not high. Notably, the emotional valence of words in the somatic symptoms and BC terminology categories could not be generally classified as positive or negative because the words in these 2 categories are a mixture of negative and positive words and some words that have no obvious emotion but are indispensable to sentiment analysis in the field of BC. Thus, the words in the somatic symptoms and BC terminology categories do not belong to any of the aforementioned dimensions, and they are only distinguished by the degree of arousal.

Results of Lexicon Performance Evaluation

Briefly, first, a corpus of 505,868 words from Weibo underwent the same data preprocessing (denoising), and then 7089 (1.40%) words were obtained after word segmentation by the machine. Second, 2 researchers were asked to verify the incorrectly segmented words independently, and then, 4350 (61.36%) words were gathered. Similarly, the 2 researchers annotated the words’ emotional categories specified in our lexicon at the same time, and then a third researcher comprehensively judged the emotional category of each word to obtain manual annotation results. Lastly, the emotional lexicon of patients with BC, the C-LIWC, and HowNet were imported into LIWC-22 software to test the performance of emotional word classification. The detailed process of lexicon performance evaluation is shown in Figure 4 . P, R, and the F 1 -score obtained after the positive and negative emotional word analysis by the 3 lexicons were calculated and compared. The number of emotional words predicted by the 3 lexicons and manually annotated is shown in Multimedia Appendix 4 . The number of positive and negative emotional words predicted by the 3 lexicons that matched the manual annotation is shown in Multimedia Appendix 5 . The analysis results of positive emotional words predicted by the 3 lexicons are shown in Multimedia Appendix 6 .

As shown in Multimedia Appendix 5 , when analyzing the same corpus, 745 words were recognized by our lexicon, which is almost twice as many as the other 2 general sentiment lexicons. In addition, our lexicon had a high recognition rate for both positive and negative emotional words, which is almost consistent with the results of manual annotation. Furthermore, the number of words incorrectly judged by our lexicon and the C-LIWC was less than 10, which indicates that the performance of the 2 lexicons is consistent with manual annotation. However, there were relatively more words incorrectly identified by HowNet.

As shown in Multimedia Appendix 6 , P, R, and the F 1 -score of the positive and negative emotional word classification by the emotional lexicon of patients with BC were all more than 98%. Specifically, P, R, and the F 1 -score of positive emotional words were 98.42%, 99.73%, and 99.07%, respectively, and those of negative emotional words were 99.73%, 98.38%, and 99.05%, respectively. The results of the C-LIWC classification were all over 95% but lower than those of our lexicon. However, it is worth mentioning that the results of these 3 variables for HowNet were slightly insufficient; in the classification of negative emotional words, the 3 values were all less than 95%. In conclusion, our emotional lexicon achieves the best performance in both emotional word detection and classification of the same data set compared to the C-LIWC and HowNet.

Principal Findings

The overarching goal of this study was to manually construct an emotional lexicon of patients with BC to help researchers identify and analyze terms associated with emotions in a text-based corpus. Consequently, we developed such an emotional lexicon, which consists of 9357 emotional words covering 8 fine-grained emotional categories (joy, anger, sadness, fear, disgust, surprise, somatic symptoms, and BC terminology) related to the emotions of patients with BC. The performance results for P, R, and the F 1 -score of the positive and negative emotional word classification by our emotional lexicon were all more than 98%. As expected, the lexicon we constructed outperformed the general sentiment lexicons C-LIWC and HowNet in both emotional word screening and classification of the same data set.

Patients’ ways of emotional expression are diverse [ 12 , 16 , 81 ]. According to previous related studies [ 31 , 82 ], we also tried to obtain as many corpora of patients with BC as possible through different methods. Finally, 150 written texts, 17 interview texts, and 6689 original posts and comments from Weibo, with a total of 1,923,593 Chinese characters, were collected. These rich corpora are helpful in capturing more special emotional words in the field of BC and ensure the professionalism of lexicon construction. Furthermore, our study also provides a reference for the construction of emotional lexicons in other special fields.

In addition to dealing with some disturbing information in the process of conventional lexicon construction [ 20 , 27 ], we especially added a manual verification step in this process, through which we ensured the correctness and standardization of the words included in the lexicon. Furthermore, unlike the common categories (eg, positive and negative) in most existing general or domain lexicons [ 20 , 23 , 41 ], in this study, 2 emotional categories with domain specificity, BC terminology and somatic symptoms, were added after analyzing the emotional words extracted from the collected corpora and the emotional categories of existing emotional lexicons. These new special categories are not only a new idea of emotional lexicon construction in the medical field but also a great innovation in emotional category determination.

Moreover, in the process of screening words in patients’ texts, we noticed some differences in the words used by patients in different stages of BC [ 8 , 68 , 81 ]. Patients in different stages often use terminologies related to the treatment of BC in the current or the next stage and a series of physical symptoms that appear or would appear on their own. The main purpose of this study was to construct an emotional lexicon suitable for all patients with BC and evaluate its performance based on the common text corpora of patients with BC. Therefore, in this paper, we did not conduct a more detailed analysis of the lexicons of patients in different stages of the disease but only introduced and differentiated the main emotional distress of patients in different stages in a qualitative text analysis [ 8 ].

Strengths and Limitations

It is worth noting the strengths of this study. First, to ensure the lexicon’s domain specificity and coverage, EW, semistructured interviews, and the Python web crawler of Weibo were used to obtain written, verbal, and online corpora of patients with BC in 3 different stages of the disease. Second, the emotional lexicon of patients with BC achieved the best performance in both emotional word detection and emotion classification compared to the C-LIWC and HowNet, which proves to be a meaningful input to the early detection and sentiment analysis of patients with BC, providing linguistic insights into identifying patients’ different emotional states.

There are also some limitations. First, the source and quantity of text corpora used for verification were limited. In this study, the construction and performance verification of our lexicon were only based on the Weibo online text corpus, due to which the lexicon’s ability to identify emotional words in Weibo texts is better than that in other online platforms. Second, the data used in the verification stage of this study were segmented words only from patients with BC, not words, sentences, and posts from patients with other types of cancer. Additionally, the emotional intensity values of emotional words in the constructed lexicon could not be determined simply by manual annotation. Thus, the performance of the lexicon we constructed in sentence- and text-level corpus processing and emotion classification needs to be explored and tested in follow-up research combining this lexicon with deep learning to prove the practical value of mental health surveillance of cancer patients. Moreover, although this lexicon significantly outperforms general emotional lexicons, we clearly found that the values of the 3 performance evaluation variables in the emotional lexicon of patients with BC did not reach 100%, which also suggests that the capacity of this lexicon needs to be further expanded and future validation of the lexicon using a large text corpus is necessary.

Comparison With Prior Work

Interestingly, similar to Gatti et al’s [ 36 ] and Wu et al’s [ 44 ] studies on lexicon construction, although it is time-consuming to manually construct an emotional lexicon, the precision and coverage of such a lexicon are higher compared to the automatic method. First, in the process of lexicon construction, we manually screened out words that were incorrectly segmented by the machine, thus avoiding the omission of some emotional words [ 44 , 48 , 72 ], especially words in the somatic symptoms and BC terminology categories.

In addition, as mentioned in many sentiment analysis studies [ 15 , 33 , 35 , 37 , 44 , 58 ], one of the major challenges in this field is the emergence of a large number of network catchwords. In this study, we found that network catchwords not only appeared in patients’ online texts on Weibo but also were reflected in their written and oral texts to express their emotions, such as “奥利给 (awesome),” “棒棒哒 (good),” and “嘚瑟 (smug).” Therefore, considering the functions of these network catchwords, we naturally included them in our emotional lexicon of patients with BC. However, it is worth mentioning that such words are not included in HowNet, which may be one of the reasons HowNet’s performance on word detection and classification of the Weibo verification text is not high. As for the C-LIWC, there are some commonly used network terms in the lexicon itself, such as “3q (a network neologism whose English pronunciation is similar to “thank you”),” “2傻 (a network neologism where “2” means “stupid and dull” and “2 傻” further emphasizes the person’s stupidity),” and “壕 (nouveau riche)”; thus, it is not difficult to understand that the final analysis results of the C-LIWC are lower than those of our emotional lexicon of patients with BC but higher than those of HowNet.

Furthermore, we suspect that the reasons for the slightly lower performance of HowNet may be the low consistency between the emotional words in this lexicon and the emotional expression in online text. Importantly, there are many emotional idioms and words that are not often used in daily life, such as “杯弓蛇影 (paranoia)” and “首鼠两端 (indecision or vacillation),” which also leads to low recognition of some colloquial expressive words.

Moreover, in line with several previous studies [ 20 , 28 , 35 , 43 ], general sentiment lexicons have defects in sentiment analysis of texts in specific fields. Specifically, the proportion of FPs and FNs in HowNet and the C-LIWC is higher than that in our emotional lexicon of patients with BC, which also directly indicates that the polarity of some words is contrary to that in general sentiment lexicons. For instance, the word “任性 (self-willed)” was manually annotated as negative, while it was regarded as positive in HowNet; the word “顽强 (indomitable)” was recognized as negative by the C-LIWC, but it actually expresses positive emotions in the Chinese context. Additionally, comparatively speaking, the experimental results of the C-LIWC are better than those of HowNet, which may be because the classification of words in the C-LIWC is finer and the words are more in line with the psychological and medical fields.

Instead of constructing a sentiment lexicon automatically, this study aimed to apply a manual construction method to construct a Chinese emotional lexicon in the BC domain based on the commonly used general sentiment lexicons C-LIWC and HowNet. Our emotional lexicon of patients with BC contains both formal emotional words and domain-specific words related to BC. Experiment results showed that the performance of our emotional lexicon is superior to that of the C-LIWC and HowNet in both emotional word detection and classification of the same data set.

We expect the expansion and promotion of this lexicon based on larger corpora and multidimensional methods and the automatic identification, personalization, and accurate management of patients’ emotions based on this lexicon. Meanwhile, more complex construction methods, such as deep neural learning, can be adopted in future research to further improve the proprietary domain lexicon construction method’s portability. We expect that our emotional lexicon of patients with BC will be useful in sentiment detection and will provide more insights into patients’ emotional management and sentiment analysis in terms of emotional need detection among patients with BC.

Acknowledgments

We would like to thank all the participants for sharing their experiences with the researchers in this study. We would also like to thank all the staff in the 6 hospitals and the 15 annotators for their support.

This work was supported by funding from the National Natural Science Foundation of China (no. 72304131) and the GuangDong Basic and Applied Basic Research Foundation (no. 2020A1515110894). The funder played no role in study design, data collection, and analysis; in the decision to publish; or in the preparation of the manuscript.

Data Availability

The data sets generated and analyzed during this study are not publicly available due to patient privacy and ethical restrictions but are available from the corresponding author upon reasonable request. The annotated emotional lexicon is publicly available to readers [ 83 ], and the password to access this lexicon is available from the corresponding author upon reasonable request.

Authors' Contributions

Conceptualization and writing—review and editing were managed by CL, JF, JL, LS, and YW; methodology by CL, JF, JL, LS, and YW; validation by CL, JF, and JL; formal analysis by CL and YW; investigation by CL, JF, WL, JB, and YL; data curation by CL, JF, JL, CZ, WL, JB, SD, Y Zhang, ZG, YL, Y Zhou, SX, MH, RW, QC, and JL; writing—original draft preparation by CL, JF, and LS; supervision by YW and CZ; and funding acquisition by YW. All authors have read and agreed to the published version of the manuscript.

Conflicts of Interest

None declared.

Inclusion and exclusion criteria for patients included in expressive writing and semistructured interviews.

Pennebaker’s expressive writing instructions and interview guide.

Quantity and representative words of each category in the emotional lexicon of patients with breast cancer.

Number of emotional words predicted by 3 lexicons and manually annotation.

Number of positive and negative emotional words predicted in 3 lexicons that match the manual annotation.

Analysis results of positive and negative emotional words by 3 lexicons.

• Latest global cancer data: cancer burden rises to 19.3 million new cases and 10.0 million cancer deaths in 2020. International Agency for Research on Cancer. 2020 Dec 15. URL: https:/​/www.​iarc.who.int/​news-events/​latest-global-cancer-data-​cancer-burden-rises-to-19-3-million-new-cases-and-10-0-million-cancer-deaths-in-2020/​ [accessed 2023-08-21]
• Fortin J, Leblanc M, Elgbeili G, Cordova MJ, Marin M, Brunet A. The mental health impacts of receiving a breast cancer diagnosis: a meta-analysis. Br J Cancer 2021 Nov 04;125(11):1582-1592 [ https://europepmc.org/abstract/MED/34482373 ] [ CrossRef ] [ Medline ]
• Holmes C, Jackson A, Looby J, Gallo K, Blakely K. Breast cancer and body image: feminist therapy principles and interventions. J Fem Fam Ther 2021 Jan 21;33(1):20-39 [ CrossRef ]
• Simonelli LE, Siegel SD, Duffy NM. Fear of cancer recurrence: a theoretical review and its relevance for clinical presentation and management. Psychooncology 2017 Oct 01;26(10):1444-1454 [ CrossRef ] [ Medline ]
• Bultz BD, Groff SL, Fitch M, Blais MC, Howes J, Levy K, et al. Implementing screening for distress, the 6th vital sign: a Canadian strategy for changing practice. Psychooncology 2011 May 01;20(5):463-469 [ CrossRef ] [ Medline ]
• Tsai W, Lu Q. Perceived social support mediates the longitudinal relations between ambivalence over emotional expression and quality of life among Chinese American breast cancer survivors. Int J Behav Med 2018 Jun 13;25(3):368-373 [ CrossRef ] [ Medline ]
• Li L, Yang Y, He J, Yi J, Wang Y, Zhang J, et al. Emotional suppression and depressive symptoms in women newly diagnosed with early breast cancer. BMC Womens Health 2015 Oct 24;15(1):91 [ https://bmcwomenshealth.biomedcentral.com/articles/10.1186/s12905-015-0254-6 ] [ CrossRef ] [ Medline ]
• Li C, Ure C, Zheng W, Zheng C, Liu J, Zhou C, et al. Listening to voices from multiple sources: a qualitative text analysis of the emotional experiences of women living with breast cancer in China. Front Public Health 2023 Feb 3;11:1114139 [ https://europepmc.org/abstract/MED/36817918 ] [ CrossRef ] [ Medline ]
• Mehl MR, Vazire S, Ramírez-Esparza N, Slatcher RB, Pennebaker JW. Are women really more talkative than men? Science 2007 Jul 06;317(5834):82-82 [ CrossRef ] [ Medline ]
• Warmoth K, Cheung B, You J, Yeung NCY, Lu Q. Exploring the social needs and challenges of chinese american immigrant breast cancer survivors: a qualitative study using an expressive writing approach. Int J Behav Med 2017 Dec 5;24(6):827-835 [ CrossRef ] [ Medline ]
• Lin H, Jia J, Qiu J, Zhang Y, Shen G, Xie L, et al. Detecting stress based on social interactions in social networks. IEEE Trans Knowl Data Eng 2017 Sep 1;29(9):1820-1833 [ CrossRef ]
• Scherer KR. What are emotions? And how can they be measured? Soc Sci Inf 2016 Jun 29;44(4):695-729 [ CrossRef ]
• Pennebaker JW. Writing about emotional experiences as a therapeutic process. Psychol Sci 2016 May 06;8(3):162-166 [ CrossRef ]
• Daniel S, Venkateswaran C, Hutchinson A, Johnson MJ. 'I don't talk about my distress to others; I feel that I have to suffer my problems...' Voices of Indian women with breast cancer: a qualitative interview study. Support Care Cancer 2021 May 21;29(5):2591-2600 [ https://europepmc.org/abstract/MED/32955655 ] [ CrossRef ] [ Medline ]
• Zhang S, Wei Z, Wang Y, Liao T. Sentiment analysis of Chinese micro-blog text based on extended sentiment dictionary. Future Gener Comput Syst 2018 Apr;81:395-403 [ CrossRef ]
• Alamoodi A, Zaidan B, Zaidan A, Albahri O, Mohammed K, Malik R, et al. Sentiment analysis and its applications in fighting COVID-19 and infectious diseases: a systematic review. Expert Syst Appl 2021 Apr 01;167:114155 [ https://europepmc.org/abstract/MED/33139966 ] [ CrossRef ] [ Medline ]
• Fu J, Li C, Zhou C, Li W, Lai J, Deng S, et al. Methods for for analyzing the contents of social media for health care: scoping review. J Med Internet Res 2023 Jun 26;25:e43349 [ https://www.jmir.org/2023//e43349/ ] [ CrossRef ] [ Medline ]
• Vinodhini G, Chandrasekaran RM. Sentiment analysis and opinion mining: a survey. Int J Adv Res Comput Sci Softw Eng 2012 Jun;2(6):282-292
• Zunic A, Corcoran P, Spasic I. Sentiment analysis in health and well-being: systematic review. JMIR Med Inform 2020 Jan 28;8(1):e16023 [ https://medinform.jmir.org/2020/1/e16023/ ] [ CrossRef ] [ Medline ]
• Li G, Li B, Huang L, Hou S. Automatic construction of a depression-domain lexicon based on microblogs: text mining study. JMIR Med Inform 2020 Jun 23;8(6):e17650 [ https://medinform.jmir.org/2020/6/e17650/ ] [ CrossRef ] [ Medline ]
• Ahmed U, Srivastava G, Yun U, Lin JC. EANDC: an explainable attention network based deep adaptive clustering model for mental health treatment. Future Gener Comput Syst 2022 May;130:106-113 [ CrossRef ]
• Zhang T, Yang K, Ji S, Ananiadou S. Emotion fusion for mental illness detection from social media: a survey. Inf Fusion 2023 Apr;92:231-246 [ CrossRef ]
• Bi Y, Li B, Wang H. Detecting depression on Sina microblog using depressing domain lexicon. 2021 Presented at: 2021 IEEE International Conference on Dependable, Autonomic and Secure Computing, International Conference on Pervasive Intelligence and Computing, International Conference on Cloud and Big Data Computing, International Conference on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech); 2021; Alberta, Canada p. 965-970 [ CrossRef ]
• Muñoz S, Iglesias CA. A text classification approach to detect psychological stress combining a lexicon-based feature framework with distributional representations. Inf Process Manag 2022 Sep;59(5):103011 [ CrossRef ]
• Saif H, He Y, Fernandez M, Alani H. Contextual semantics for sentiment analysis of Twitter. Inf Process Manag 2016 Jan;52(1):5-19 [ CrossRef ]
• Taboada M, Brooke J, Tofiloski M, Voll K, Stede M. Lexicon-based methods for sentiment analysis. Comput Linguist 2011;37(2):267-307 [ CrossRef ]
• Chao F, Xun L, Yaping L. Construction method of Chinese cross-domain sentiment lexicon based on word vector. J Data Acquis Process 2017;32(3):579-587
• Chang C, Wu ML, Hwang SY. An approach to cross-lingual sentiment lexicon construction. 2019 Presented at: IEEE International Congress on Big Data (BigDataCongress); July 8-13, 2019; Milan, Italy [ CrossRef ]
• Colón-Ruiz C, Segura-Bedmar I. Comparing deep learning architectures for sentiment analysis on drug reviews. J Biomed Inform 2020 Oct;110:103539 [ https://linkinghub.elsevier.com/retrieve/pii/S1532-0464(20)30167-2 ] [ CrossRef ] [ Medline ]
• Denecke K, Deng Y. Sentiment analysis in medical settings: new opportunities and challenges. Artif Intell Med 2015 May;64(1):17-27 [ CrossRef ] [ Medline ]
• Cabling ML, Turner JW, Hurtado-de-Mendoza A, Zhang Y, Jiang X, Drago F, et al. Sentiment analysis of an online breast cancer support group: communicating about tamoxifen. Health Commun 2018 Sep 05;33(9):1158-1165 [ https://europepmc.org/abstract/MED/28678549 ] [ CrossRef ] [ Medline ]
• Lacy S, Watson B, Riffe D, Lovejoy J. Issues and best practices in content analysis. Journal Mass Commun Q 2015 Sep 28;92(4):791-811 [ CrossRef ]
• Wu F, Song Y, Huang Y. Microblog sentiment classification with heterogeneous sentiment knowledge. Inf Sci 2016 Dec;373:149-164 [ CrossRef ]
• Taboada M, Voll K, Brooke J. Extracting Sentiment as a Function of Discourse Structure and Topicality. Technical Report 2008-20. British Columbia, Canada: School of Computing Science, Simon Fraser University; Dec 2, 2008:1-22
• Deng S, Sinha AP, Zhao H. Adapting sentiment lexicons to domain-specific social media texts. Decis Support Syst 2017 Feb;94:65-76 [ CrossRef ]
• Gatti L, Guerini M, Turchi M. SentiWords: deriving a high precision and high coverage lexicon for sentiment analysis. IEEE Trans Affective Comput 2016 Oct 1;7(4):409-421 [ CrossRef ]
• Clark E, James T, Jones CA, Alapati A, Ukandu P, Danforth CM, et al. A sentiment analysis of breast cancer treatment experiences and healthcare perceptions across Twitter. arXiv Preprint posted online 2018 [doi: 10.48550/arXiv.1805.09959] [ CrossRef ]
• Praveen SV, Ittamalla R, Mahitha M, Spoorthi K. Trauma and stress associated with breast cancer survivors—a natural language processing study. J Loss Trauma 2022 Apr 11;28(2):175-178 [ CrossRef ]
• Zhang W, Zhu Y, Wang J. An intelligent textual corpus big data computing approach for lexicons construction and sentiment classification of public emergency events. Multimed Tools Appl 2018 Dec 8;78(21):30159-30174 [ CrossRef ]
• Pennebaker J, Boyd RL, Jordan K, Blackburn K. The development and psychometric properties of LIWC2015. University of Texas at Austin. 2015. URL: https://repositories.lib.utexas.edu/bitstream/handle/2152/31333/LIWC2015_LanguageManual.pdf [accessed 2023-08-22]
• Zhendong D, Qiang D. HowNet - a hybrid language and knowledge resource. 2003 Presented at: NLP-KE 2003 : 2003 International Conference on Natural Language Processing and Knowledge Engineering; 2003; Beijing, China [ CrossRef ]
• Hartman JJ, Stone PJ, Dunphy DC, Smith MS, Ogilvia DM. The general inquirer: a computer approach to content analysis. Am Sociol Rev 1967 Oct;32(5):859 [ CrossRef ]
• Li S, Shi W, Wang J, Zhou H. A deep learning-based approach to constructing a domain sentiment lexicon: a case study in financial distress prediction. Inf Process Manag 2021 Sep;58(5):102673 [ CrossRef ]
• Wu F, Huang Y, Song Y, Liu S. Towards building a high-quality microblog-specific Chinese sentiment lexicon. Decis Support Syst 2016 Jul;87:39-49 [ CrossRef ]
• Bradley MM, Lang PJ. Affective norms for English words (ANEW): Instruction manual and affective ratings. Technical report C-1. Center for Research in Psychophysiology, University of Florida. 1999. URL: https://pdodds.w3.uvm.edu/teaching/courses/2009-08UVM-300/docs/others/everything/bradley1999a.pdf [accessed 2023-08-22]
• Whissell CM. Chapter 5 - the dictionary of affect in language. In: Plutchik R, Kellerman H, editors. The Measurement of Emotions. Cambridge, MA: Academic Press; 1989:113-131
• Yang AM, Lin JH, Zhou YM, Chen J. Research on building a Chinese sentiment lexicon based on SO-PMI. Appl Mech Mater 2012 Dec;263-266:1688-1693 [ CrossRef ]
• Zeng X, Yang C, Tu C, Liu Z, Sun M. Chinese LIWC lexicon expansion via hierarchical classification of word embeddings with sememe attention. 2018 Presented at: AAAI-18: Thirty-Second AAAI Conference on Artificial Intelligence; February 2-7, 2018; New Orleans, LA [ CrossRef ]
• Boyd RL, Ashokkumar A, Seraj S, Pennebaker JW. The development and psychometric properties of LIWC-22. University of Texas at Austin. 2022. URL: https://www.liwc.app/static/documents/LIWC-22%20Manual%20-%20Development%20and%​20Psychometrics.pdf [accessed 2023-08-22]
• Tausczik YR, Pennebaker JW. The psychological meaning of words: LIWC and computerized text analysis methods. J Lang Soc Psychol 2009 Dec 08;29(1):24-54 [ CrossRef ]
• Proyer RT, Brauer K. Exploring adult playfulness: examining the accuracy of personality judgments at zero-acquaintance and an LIWC analysis of textual information. J Res Pers 2018 Apr;73:12-20 [ CrossRef ]
• Huang CL, Chung CK, Hui N, Lin YC, Seih YT, Lam BCP, et al. Development of the Chinese linguistic inquiry and word count dictionary. Chin J Psychol 2012;54:185-201
• Liu Y, Duan Z. Chinese Chinese movie comment sentiment analysis based on HowNet and user likes. J Phys: Conf Ser 2019 May 01;1229(1):012018 [ CrossRef ]
• Xianghua F, Guo L, Yanyan G, Zhiqiang W. Multi-aspect sentiment analysis for Chinese online social reviews based on topic modeling and HowNet lexicon. Knowl-Based Syst 2013 Jan;37:186-195 [ CrossRef ]
• Newman ML, Groom CJ, Handelman LD, Pennebaker JW. Gender differences in language use: an analysis of 14,000 text samples. Discourse Processe 2008 May 15;45(3):211-236 [ CrossRef ]
• Borowiecki KJ. How are you, my dearest Mozart? well-being and creativity of three famous composers based on their letters. Rev Econ Stat 2017 Oct;99(4):591-605 [ CrossRef ]
• Dong Z, Dong Q, Hao C. HowNet and its computation of meaning. In: Coling 2010: Demonstrations. Beijing, China: Coling 2010 Organizing Committee; 2010:53-56
• Li Z, Ding N, Liu Z, Zheng H, Shen Y. Chinese relation extraction with multi-grained information and external linguistic knowledge. 2019 Presented at: 57th Annual Meeting of the Association for Computational Linguistics; July 28-August 2, 2019; Florence. Italy [ CrossRef ]
• Goeuriot L, Na JC, Min Kyaing WY, Khoo CSG, Chang YK, Theng YL. Sentiment lexicons for health-related opinion mining. 2012 Presented at: IHI '12: 2nd ACM SIGHIT Symposium on International Health Informatics; Jan 28-30, 2012; Miami, FL p. 219-226 [ CrossRef ]
• Russell JA. A circumplex model of affect. J Pers Soc Psychol 1980 Dec;39(6):1161-1178 [ CrossRef ]
• Ekman P, Oster H. Facial expressions of emotion. Annu Rev Psychol 1979 Jan;30(1):527-554 [ CrossRef ]
• Ekman P, Friesen WV. Constants across cultures in the face and emotion. J Pers Soc Psychol 1971 Feb;17(2):124-129 [ CrossRef ] [ Medline ]
• Brandão T, Schulz MS, Matos PM. Psychological adjustment after breast cancer: a systematic review of longitudinal studies. Psychooncology 2017 Jul 12;26(7):917-926 [ CrossRef ] [ Medline ]
• Liao M, Chen S, Chen S, Lin Y, Chen M, Wang C, et al. Change and predictors of symptom distress in breast cancer patients following the first 4 months after diagnosis. J Formos Med Assoc 2015 Mar;114(3):246-253 [ https://linkinghub.elsevier.com/retrieve/pii/S0929-6646(13)00211-8 ] [ CrossRef ] [ Medline ]
• Hanson Frost M, Suman VJ, Rummans TA, Dose AM, Taylor M, Novotny P, et al. Physical, psychological and social well-being of women with breast cancer: the influence of disease phase. Psychooncology 2000;9(3):221-231 [ CrossRef ] [ Medline ]
• Morse JM. Designing funded qualitative research. In: Denzin NK, Lincoln YS, editors. Handbook of Qualitative Research. Thousand Oaks, CA: Sage Publications; 1994:220-235
• Van Kaam AL. Phenomenal analysis: exemplified by a study of the experience of "really feeling understood". J Individ Psychol 1959;15:66-72
• Wu Y, Liu L, Zheng W, Zheng C, Xu M, Chen X, et al. Effect of prolonged expressive writing on health outcomes in breast cancer patients receiving chemotherapy: a multicenter randomized controlled trial. Support Care Cancer 2021 Feb 30;29(2):1091-1101 [ CrossRef ] [ Medline ]
• Saunders B, Sim J, Kingstone T, Baker S, Waterfield J, Bartlam B, et al. Saturation in qualitative research: exploring its conceptualization and operationalization. Qual Quant 2018 Sep 14;52(4):1893-1907 [ https://europepmc.org/abstract/MED/29937585 ] [ CrossRef ] [ Medline ]
• Lazarus RS, Folkman S. Stress, Appraisal, and Coping. New York, NY: Springer Publishing; 1984.
• Kemper TD, Lazarus RS. Emotion and adaptation. Contemp Sociol 1992 Jul;21(4):522 [ CrossRef ]
• Yang M, Zhu D, Chow KP. A topic model for building fine-grained domain-specific emotion lexicon. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Baltimore, MD: Association for Computational Linguistics; 2014.
• Tuinman MA, Gazendam-Donofrio SM, Hoekstra-Weebers JE. Screening and referral for psychosocial distress in oncologic practice: use of the Distress Thermometer. Cancer 2008 Aug 15;113(4):870-878 [ https://onlinelibrary.wiley.com/doi/10.1002/cncr.23622 ] [ CrossRef ] [ Medline ]
• Cleeland CS, Mendoza TR, Wang XS, Chou C, Harle MT, Morrissey M, et al. Assessing symptom distress in cancer patients: the M.D. Anderson Symptom Inventory. Cancer 2000 Oct 01;89(7):1634-1646 [ CrossRef ] [ Medline ]
• Raghavan P, Fosler-Lussier E, Lai AM. Inter-annotator reliability of medical events, coreferences and temporal relations in clinical narratives by annotators with varying levels of clinical expertise. AMIA Annu Symp Proc 2012 Nov 3;2012:1366-1374 [ https://europepmc.org/abstract/MED/23304416 ] [ Medline ]
• Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull 1971 Nov;76(5):378-382 [ CrossRef ]
• Zapf A, Castell S, Morawietz L, Karch A. Measuring inter-rater reliability for nominal data - which coefficients and confidence intervals are appropriate? BMC Med Res Methodol 2016 Aug 05;16(1):93 [ https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-016-0200-9 ] [ CrossRef ] [ Medline ]
• Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977 Mar;33(1):159 [ CrossRef ]
• Zhou L, Yang X. Sentiment lexicon construction for emergency management: taking "rainstorm and flood" as an example. J Wuhan Univ Technol Soc Sci Ed 2019;32(04):8-14
• Pan W, Han Y, Li J, Zhang E, He B. The positive energy of netizens: development and application of fine-grained sentiment lexicon and emotional intensity model. Curr Psychol 2022 Nov 03:1-18 [ https://europepmc.org/abstract/MED/36345548 ] [ CrossRef ] [ Medline ]
• Wu Y, Yang D, Jian B, Li C, Liu L, Li W, et al. Can emotional expressivity and writing content predict beneficial effects of expressive writing among breast cancer patients receiving chemotherapy? A secondary analysis of randomized controlled trial data from China. Psychol Med 2021 Aug 24;53(4):1527-1541 [ CrossRef ]
• Abd RR, Omar K, Noah SAM, Danuri MSNM. A survey on mental health detection in online social network. Int J Adv Sci, Eng Inf Technol 2018;8(4-2):1431-1436 [ CrossRef ]
• Chaixiu L. The emotional lexicon of breast cancer patients. Baidu. URL: https://pan.baidu.com/s/1Mgk2wJsdXg664Sa5Wmkkjw [accessed 2023-08-15]

Abbreviations

Edited by T de Azevedo Cardoso; submitted 08.12.22; peer-reviewed by T Zhang, T Dang; comments to author 16.06.23; revised version received 17.08.23; accepted 18.08.23; published 12.09.23

©Chaixiu Li, Jiaqi Fu, Jie Lai, Lijun Sun, Chunlan Zhou, Wenji Li, Biao Jian, Shisi Deng, Yujie Zhang, Zihan Guo, Yusheng Liu, Yanni Zhou, Shihui Xie, Mingyue Hou, Ru Wang, Qinjie Chen, Yanni Wu. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 12.09.2023.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.