Agreement Between a Food Frequency Questionnaire and the Willett Questionnaire in Overweight or Obese Pregnant Women

Background & Aims: Overweight and obesity during pregnancy is associated with an increased risk of many adverse health outcomes for both women and their infants. There is a need for simple food frequency questionnaires to assess nutritional intake and aid implementation and evaluation of nutritional interventions in these women. The aim of this study was to compare a newly developed food frequency questionnaire with the Willett food frequency questionnaire in Australian pregnant women who were overweight or obese. Methods: 170 overweight or obese pregnant women (12-20 weeks’ gestation) completed both the Willett and the devised (LIMIT) food frequency questionnaire with n=41 excluded due to unrealistic energy intake or incomplete questionnaires. The mean nutrient intake for each questionnaire and the mean difference in nutrient intake between the questionnaires was assessed. The correlation and agreement between the two questionnaires were assessed by Spearmans correlation coefficients and Bland-Altman method. Results: There were high correlations for total energy intake, protein, carbohydrates, cholesterol, iron, folate, and caffeine (r>0.50, P<0.01), and moderate correlations for fat, fibre, and calcium (r=0.4-0.5, P<0.01). Correlations were low (r<0.3) for vitamins. There was no significant systematic error between the two food frequency questionnaires with the exception of alcohol, calcium, iron, and folate (P>0.05). The limit of agreement (LOA) was wide (LOA <50% or >200%) for macronutrients, calcium, and folate, but within the acceptable range for iron, vitamins, and caffeine (LOA 133%). Conclusions: There is good agreement between the Willett and the LIMIT food frequency questionnaires in estimating macronutrient and some key pregnancy-related micronutrients for group-level comparisons.


BACKGROUND
It is estimated that approximately 1.0 billion adults are currently overweight worldwide, with a further 475 million being obese [1].The prevalence of maternal overweight and obesity is also rising [2], affecting approximately 40-50% of pregnant women [3][4][5].Overweight and obesity during pregnancy are associated with an increased risk of many adverse health outcomes including gestational diabetes, hypertension, and caesarean birth for women, and infant macrosomia and perinatal mortality [6].Increasingly, dietary and weight management interventions are being offered during pregnancy for women who are overweight or obese to minimise these adverse outcomes.There is therefore a need for simple tools that accurately assess nutritional intake to assist in guiding education and intervention targets, as well as in providing an assessment of the efficacy of dietary interventions.
Food frequency questionnaires are used to assess nutritional intake in a research setting.Although there is no limit to the number of questions on a food frequency questionnaire, the instrument usually incorporates common food items within the specific study population to enhance efficiency and improve completion rates [7].The questionnaire can supply an overview of dietary intake, including food groups and nutrient composition.When compared with other assessment methods including food diaries and interviewer-administered questionnaires, a selfadministered questionnaire is less expensive and cumbersome for both study respondents and researchers [7].However, a food frequency questionnaire needs to be compared against a standardized tool to ensure its utility in the study population concerned.
Internationally, the US based Harvard Semiquantitative Food Frequency Questionnaire (the Willett questionnaire) [8] is widely used to measure dietary intake and has been previously validated in pregnant women [9][10][11][12].In Australia, there are two food frequency questionnaires that are frequently used to assess dietary intake: the Commonwealth Scientific and Industrial Research Organisation Food Frequency Questionnaire (CSIRO-FFQ) and the Cancer Council of Victoria Food Frequency Questionnaire (CCV-FFQ).The CSIRO-FFQ was developed in the 1980s and was widely used throughout 1980s and 1990s [13].It has been validated against a variety of different populations, however, it contains more than 200 items and takes 45 minutes to complete [14].The CCV-FFQ was initially designed to assess dietary intakes for cancer patients or people at risk of malignancy.While the CCV-FFQ is quick and easy to complete, it has only been validated in people with cancer and premenopausal women and doesn't include some common food items consumed in the general Australian population [14].To our knowledge, there is no dietary assessment tool specifically designed to measure dietary intake in pregnant women in Australia.Pregnant women have different nutritional requirements compared with non-pregnant women; in particular increased requirements for energy, protein, calcium, iron, and folate [15].It is therefore important that a FFQ in this population accurately measures these key macronutrients and micronutrients.
A self-administered food frequency questionnaire (the LIMIT-FFQ) was developed to assess usual dietary intake for Australian pregnant women who were overweight or obese.The aim of the present study was to compare the LIMIT-FFQ with the Willett questionnaire which has been previously validated in pregnant women.

Subjects
This prospective cohort study was nested within the LIMIT randomised trial which is designed to evaluate the effect of an antenatal dietary and lifestyle intervention for women who are overweight or obese [16].The methodology of the LIMIT randomised trial has been described in detail previously [16].Specifically, women were recruited with a live singleton gestation, between 10 and 20 weeks' gestation, at the time of their first antenatal appointment.All women provided written informed consent to participate.Women were recruited from public maternity hospitals across the South Australian metropolitan area (specifically, Women's and Children's Hospital, Lyell McEwin Hospital, and Flinders Medical Centre).Ethics approval was obtained from all sites.
Between September 2009 and August 2010, women were asked to complete both the LIMIT FFQ and the Willett FFQ at the time of study entry, and prior to delivery of the dietary intervention.Baseline demographic details were collected, including parity, age, ethnicity, gestational age at entry, hospital status, and smoking status at the time of recruitment.

Food Frequency Questionnaires
Women completed two food frequency questionnaires, the Harvard Semi-quantitative Food Frequency Questionnaire (Willett questionnaire) and the LIMIT food frequency questionnaire.The questionnaires were not completed in a specified order.The Willett questionnaire was developed in 1985 in the United States to measure daily intake of 18 selected nutrients [8].This includes 126 food items with an indication of a standard portion size divided into seven food categories and has been validated for use across diverse study populations including pregnant women [8-10, 17, 18].Respondents were asked to indicate their relative frequency of consumption from nine different response alternatives, ranging from 'Never, or less than once per month' to '6+ per day'.Questions were also asked about use of supplements, cooking methods and addition of sugar to foods.An open ended question allowed respondents to record consumption of other foods not included on the food list which was then categorized by study investigators into the appropriate food categories.The Willett questionnaire was analysed by the Willett in-house food frequency questionnaire nutrient analysis program (HarvardSSFQ5/93; Harvard School of Public Health, Boston, Massachusetts) to generate summaries of nutrient intake for each participant.Daily nutrient intakes were estimated by multiplying frequency responses by standardised nutrient compositions of the specified portion size of each food item according to the Willett nutrient database.
The LIMIT FFQ was developed following the same rational and format as the Willett questionnaire with the food items and frequency column modified.The questionnaire was based on the Willett questionnaire because it has been previously validated in pregnancy [9][10][11].The Willett questionnaire has been used to assess changes in dietary intake across different time points during pregnancy [19] and has also been utilised in Australian pregnant women [20].The LIMIT-FFQ included food items adapted from the Australian Guide to Healthy Eating (AGHE) [21] which contains examples of commonly eaten food items in Australia and recommended serve sizes of each food group.This included a total of 101 questions on frequencies and portions of food items separated into seven broad food group categories (breads/cereals, fruit, vegetables, dairy and alternatives, meat and alternatives, drinks, and other foods).Open-ended questions were provided after each broad food group to allow women to record consumption of other additional food items to capture the total dietary intake of the individual.Additional questions clarified the type of cooking oil used, the amount of sugar added to food or fluids, and consumption of vitamin and mineral supplements.The frequency of food item consumption was ascertained using a multiple response grid according to the following categories: never/less than once per month; one to three times a month; one time per week; two to four times per week; five to six times per week; one time per day; two to three times per day; and four times per day or more.Portion size was assessed using medium serve size examples adopted from the AGHE where women were asked to indicate whether their serving size was less than, equal to, or more than the medium serve size provided.The LIMIT FFQ was cross-checked by several nutrition and public health experts for completeness of relevant foods and usability prior to its dissemination to study participants.
For data analysis, if the frequency of consumption of a food item was reported as never/less than once per month but a portion size was given, the intake of this food item was considered to be 0.5 time per month.If a woman only marked frequency values but left the portion size value blank, the missing values for portion size were replaced by the mean portion size of the other participants for that food item.The frequency values and portion size values were analysed together manually to calculate the average daily consumption of all foods for each women, followed by calculation of the average daily nutrient consumption based on nutrient information derived from Food Standards Australia New Zealand (FSANZ www.foodstandards.gov.au).
Both questionnaires asked women to indicate how often on average they had consumed the amount of food during the past year.For both questionnaires, if missing data exceeded 25% the questionnaire was excluded from the analysis.Women who reported unrealistic energy intakes (<4,500 kj/day or >20,000 kj/day) were also excluded from analysis as previously reported for pregnant women [22].The analysis was focused on key nutrients for pregnancy defined by the Dietitian's Association of Australia as energy, protein, iron, calcium, and folate [15].

Statistical Analysis
Data was assessed for normality and nonparametric statistical tests were utilised where data was non-normally distributed.The mean nutrient intake for each questionnaire and the mean difference in nutrient intake between the questionnaires was assessed and Spearman rank order correlation coefficients calculated for all nutrient consumption from both questionnaires to measure the relationship between the two questionnaires [7].The Bland-Altman method was used to assess the agreement between the two questionnaires [7] for macronutrients and key micronutrients during pregnancy.Systematic error was assessed through the regression line of differences.The limit of agreement (LOA) was also interpreted with the LOA ideally being between 50-200%, i.e. the LIMIT FFQ yielding up to one-half or double of the Willett FFQ measurement for 95% of all participants.The LOA was not set as a criterion for adequacy of agreement as it was subjective [13].All analyses were performed with SPSS (SPSS for Windows, Rel.18.0.18.2010.Chicago: SPSS Inc) with a P value of <0.05 considered to indicate statistically significant differences.

RESULT Participant Characteristics
170 women were assessed with n=40 being excluded due to incomplete questionnaires and n=1 excluded due to unrealistically high energy consumption, leaving 129 participants (76%) included in the data analysis.The baseline characteristics of women are presented in Table 1 and are similar to the reported demographic characteristics of pregnant women in South Australia [5].There were no significant differences (P>0.05) between the included and excluded study subjects with regards to parity, prevalence of smoking prior to pregnancy, average age, and ethnicity (Table 1).While there were more obese subjects excluded from the study (P=0.002), the reason for exclusion was missing values for food items exceeding 25% as opposed to unrealistic energy consumption.The mean gestational age when the questionnaires were completed was 16 weeks.

Nutrient Intake and Relationship between Food Frequency Questionnaires
Table 2 presents the mean difference between nutrient intakes estimated by both methods.Compared with the Willet FFQ, the LIMIT FFQ over reported consumption of total energy intake, protein, total fat, polyunsaturated fat, carbohydrates, fibre, and niacin and under reported consumption of monounsaturated fat, cholesterol, calcium, iron, folate, vitamin B2, vitamin C, and caffeine although these differences were not statistically significant (P 0.05) (Table 2).The consumption of vitamin B1, vitamin B6, vitamin B12, vitamin D, and alcohol estimated by the LIMIT FFQ was significantly lower and the consumption of saturated fat and vitamin A was significantly higher than estimates from the Willett questionnaire (P<0.05)(Table 2).
The correlations of estimated nutrient consumption between both questionnaires were moderate to high for most macro-nutrients and some micro-nutrients (Table 3).Spearman rank correlation coefficients were high (r 0.50) and statistically significant (P<0.001) for total energy intake, protein, carbohydrates, cholesterol, iron, folate, and caffeine.Moderate correlation coefficients (r=0.30-0.50)were observed for total fat intake, fibre, saturated fat, monounsaturated fat, polyunsaturated fat, and calcium.Correlations were low (r 0.30) for alcohol, vitamin B1, vitamin B2, niacin, vitamin B6, vitamin B12, vitamin C, vitamin A, and vitamin D.

Bland Altman Plots
Bland-Altman plots of key nutrients for pregnancy (energy, protein, total fat, carbohydrates, alcohol, fibre, calcium, iron, and folate) are presented in Figures 1  and 2 (additional Bland-Altman plots for other nutrients are available as supplementary data).Table 3 presents the LOA and systematic errors for the Bland-Altman plots.All key pregnancy nutrients had LOA outside of 50-200% except for iron.For iron the mean agreement was 133% indicating that 95% of all subjects iron intake as estimated by the LIMIT FFQ would be 1.3 times the iron intake as estimated by the Willet FFQ.For additional nutrients and micronutrients, saturated fat, monounsaturated fat, polyunsaturated fat, vitamin B1, vitamin B2, niacin, vitamin B6, vitamin B12, vitamin C, vitamin A, vitamin D and caffeine had mean agreement within LOA 50-200% indicating good agreement.There was no significant systematic error between the average difference of the two questionnaires for total energy, protein, total fat, monounsaturated fat, polyunsaturated fat, carbohydrate, fibre, calcium, iron, folate, vitamin B2, niacin, and vitamin C (Table 4).When compared with the Willett questionnaire, the LIMIT FFQ significantly under-estimated consumption of vitamin B1, vitamin B6, vitamin B12, vitamin D and alcohol, and significantly over-estimated consumption of saturated fat, cholesterol, vitamin A and caffeine in  women with high intakes.In general, with the exception of alcohol for macronutrients there was no significant systematic error, however the LOA was relatively wide.
For key micronutrients during pregnancy, there was no significant systematic error for calcium, iron, and folate.The LOA was wide for calcium and folate, but within the acceptable range for iron.

DISCUSSION
Within a pregnant population of overweight and obese women, the LIMIT FFQ showed good agreement in assessing the nutrient consumptions for most key macronutrients and pregnancy-related micronutrients during pregnancy compared with the Willett questionnaire.
The level of agreement between dietary assessment methodologies can be assessed in a variety of ways.With regards to the correlation coefficient of a nutrient, a value between 0.40-0.70indicates good agreement between assessment methods [23].In this current study, the Spearman's correlation coefficients for intakes measured by the LIMIT FFQ and the Willett questionnaire ranged between 0.16 and 0.58 with most values of 0.37 and higher, similar to correlation coefficients observed in other FFQ validation studies [7].Correlations of macronutrients (except alcohol), calcium, iron, folate, and caffeine were satisfactory while low correlations (<0.30) were observed for vitamin B1, vitamin B12, vitamin D and alcohol.It was confirmed by the mean difference calculation and the Bland-Altman plots that the two questionnaires were in acceptable agreement with measuring intakes of most of the macro-nutrients and key micro-nutrients for pregnancy although the limit of agreement was wide for macronutrients, calcium, and folate, but within the acceptable range for iron.That indicates that for group comparison, the two questionnaires are in good agreement with each other but that their results can vary when estimating individual intakes.
We report here some disagreement of food intake estimates between the two questionnaires.The mean intake of alcohol measured by the LIMIT FFQ was significantly lower than the Willett questionnaire.This is potentially because the Willett questionnaire includes four questions asking respondents about different types of alcohol consumed while the LIMIT FFQ uses only one question asking about general consumption of all types of alcohol.It has been previously reported that when a large number of food items represent a food group on a questionnaire, the total frequency estimate from that food group will be inflated [24].This may also explain the higher estimation of vitamin B1, B6, and B12 by the Willett questionnaire as this has four more questions for breads and cereals and two more questions for meat and egg compared with the LIMIT questionnaire.The difference in intakes of some micronutrients measured by the two questionnaires can also be explained by nutrient composition data for the two questionnaires being derived from different data sources from two different countries.
In previous validation studies for the Willett questionnaire, relatively high correlation coefficients for macronutrients were reported compared with dietary recalls or food diaries [25][26][27].The results of our present study are consistent with these previous results which indicate that the estimated macronutrient intake measured by the LIMIT FFQ is close to reality.We note good agreement for key pregnancy micronutrients which reflects the design of the LIMIT questionnaire to focus on the main source of these key nutrients (summarised in supplementary table).However, there is little research assessing the validation of key micronutrients for pregnancy measured by the Willett questionnaire.In reality, it is difficult to provide precise estimates of micronutrient consumption using questionnaires [24] where only very common food items are listed.Ideally, to assess the exact nutrient intake of a subject requires a comprehensive and accurate food list [7].The two questionnaires compared in this study are only tools for ranking nutrient intake.
In this study, the LIMIT FFQ is in a similar format to the Willett questionnaire as it was modified and updated based on the Willett questionnaire to suit the target population of Australian pregnant women.It is therefore not a completely new questionnaire developed from basic principles.The Willett and LIMIT FFQ were compared as it has been previously reported that agreement between food records and a FFQ was not different from agreement between food records and an updated FFQ [28].Furthermore, we chose the Willett questionnaire for comparison with the LIMIT FFQ as it has been validated several times in a variety of different populations, including pregnant women [10,11].Studies examining the concurrent validity of the Willett relative to repeated dietary records have reported that it is able to rank individuals on specific nutrient intake [8,29].The Willett questionnaire is also able to detect relation between different types of chronic diseases including colon cancer [28], symptomatic diverticular disease [30], coronary heart disease [31], and myocardial infarction [32] and food consumption [8,10,11,17,18].In pregnant women, the Willett questionnaire has been validated in its ability to assess intakes of several nutrients (a-carotene, lycopene, lutein and zeaxanthin, r-tocopherol, and some fatty acids) against bio-markers in the blood [11].Researchers also used the Willett questionnaire to detect changes in dietary intake from the first to the second trimester of pregnancy [12].However, we note the small sample sizes as a limitation in these validation studies.There may also be limitations to using the Willett questionnaire as it was developed 26 years ago for use in the USA and thus may not cover all commonly eaten foods by Australian women today.
The strengths of this study include the minimising of intra-individual variability (both day to day and seasonal) through administering both questionnaires to the same women at the same time.Food items listed in the LIMIT FFQ were modified to reflect common food consumption patterns in the Australian population.Ideally, validation of a food frequency questionnaire involves comparison with weighed food records [7].A limitation of our study is our comparison of the LIMIT FFQ to another FFQ rather than a more precise method of assessing dietary intake, e.g.repeated 24 hour recall or weighed food intake.Both the LIMIT and Willett questionnaires can only rank the level of intake rather than supply exact value.We also note that a large proportion of women excluded from the study due to incomplete data entry which may be due to lack of adherence to complete a food frequency questionnaire in obese pregnant women [33].In addition, as all dietary assessment instruments are subject to measurement error, there may be bias that undetected by both questionnaires.Further research is required comparing the LIMIT FFQ to precise measures of dietary assessment and biomarkers.
Overweight and obesity during pregnancy is an area of increasing importance world-wide due to its significant health and economic impacts [6].It is crucial to develop a reliable tool which can measure relevant nutrient intakes in this group of women to support effective interventions.We report here good agreement between the Willett and LIMIT FFQ in estimating macronutrient (except alcohol) and key pregnancyrelated micronutrients.While we report differences between the instruments for absolute nutrients (such as vitamin B1, vitamin B2, etc.), these are likely to be less relevant for the purposes of dietary assessment and education for overweight or obese women during pregnancy.
In conclusion, nutritional information collected with the LIMIT FFQ shows a close relationship with nutrient values estimated from the Willett questionnaire.We conclude that there is sufficient agreement between the questionnaires for group-level comparisons in overweight and obese pregnant women for most of the macronutrients and some key micronutrients during pregnancy.The LIMIT FFQ is acceptable for use as a research tool to assess dietary intake in pregnant Australian women.

Figure 1 :
Figure 1: Bland-Altman plots of comparisons of difference between the LIMIT FFQ and the Willett FFQ against the average intake of macronutrients measured by the two methods.The Bland-Altman plots shows the mean nutrient intake measured by the LIMIT FFQ and the Willett questionnaire against the difference of nutrient intake measured by the LIMIT FFQ and the Willett questionnaire for each participant.The distance between the top line and the bottom line indicate width of Limit of Agreement.

Figure 2 :
Figure 2: Bland-Altman plots of comparisons of difference between the LIMIT FFQ and the Willett against the average intake of key micronutrients for pregnancy measured by the two methods.The Bland-Altman plots shows the mean nutrient intake measured by the LIMIT FFQ and the Willett questionnaire against the difference of nutrient intake measured by the LIMIT FFQ and the Willett questionnaire for each participant.The distance between the top line and the bottom line indicated width of Limit of Agreement.

Table 1 : Characteristics of Study Participants
*Data were analysed using the Paired Sample T Test.+ Pregnancy Outcome in South Australia 2009, Government of South Australia.^% calculated excluded underweight women and women of normal weight.

Table 2 : Nutrient Intake Assessed by the LIMIT Food Frequency Questionnaire and the Willett Questionnaire and Differences Between both Methods LIMIT questionnaire Willett questionnaire Differences LIMIT -Willett Nutrients
Data were presented as Mean ± SD and analyzed using Paired-Samples T Test.

Table 4 : Systematic Error and Limits of Agreement Between the Willet and LIMIT Food Frequency Questionnaire Bland-Altman plots
CI: Confidence interval; LOA: Limit of agreement.