Evaluation of an Algorithm to Predict Menstrual-Cycle Phase at the Time of Injury

An algorithm was developed to categorize participants as preovulatory or postovulatory at the time of injury using self-reported menstrual history and a salivary P4 concentration obtained after the injury date (Figure). This algorithm was based on published data²¹⁻²³ regarding MCs in young women, as well as clinical experience. Similar to many of the methods based on calendar counting, the algorithm assumes that the length of the luteal phase is more stable than that of the follicular phase, with ovulation usually occurring 14 days before the onset of menses. In accordance with published data,^22,24  a salivary P4 concentration of 190 pmol/L was selected as the threshold indicating that ovulation occurred. Participants with injuries occurring within 14 days of the next menses are considered anovulatory if their P4 values are below the 190 pmol/L threshold and their observed cycle is 3 or more days shorter or longer than their normally reported MC length.

Thirty-three National Collegiate Athletic Association Division I female athletes between the ages of 18 and 24 years from the University of Vermont were examined in this study. All participants reported a normal MC history, no use of prescription medications or illicit substances, no use of tobacco over the past 12 months, and good overall health with no underlying conditions. They also met a weekly physical activity energy expenditure of 60 kcal/kg/wk or greater during moderate-, high-, and very high-intensity activities.²⁵  Normal MC history was defined as >25 days and <40 days between cycles with no change in menstrual status in the past 12 months. Energy expenditure during physical activity was measured with the 7-Day Physical Activity Recall (PAR).²⁵  Our institutional review board approved this investigation, and all participants provided written informed consent before enrollment.

Participants visited the laboratory on 8 occasions, spaced equally over the course of 1 MC. The first day of testing began within 24 hours of the onset of menses, and spacing of subsequent visits was based on the length of her previous MC, which was defined as the number of days from the first day of bleeding during the previous MC to the last day before the onset of bleeding of the next MC. During visits 1, 3, 5, and 7, participants provided saliva and serum samples. At visits 2, 4, 6, and 8, subjects provided saliva samples and completed the PAR questionnaire.^25,26  Unless the next menses began before visit 8, its start date was obtained through telephone contact.

We instructed participants to abstain from consuming alcohol for 12 hours before sample collection and consuming anything by mouth (except water) or chewing gum for a minimum of 1 hour before each visit and to refrain from flossing their teeth between waking and their laboratory visit. These restrictions normalize salivary pH and avoid potential blood contamination. A minimum of 10 minutes before collection, participants rinsed their mouths with water 3 times for 10 seconds per rinse to remove food particulates and normalize salivary pH.^27,28  Approximately 4 mL of whole, nonstimulated saliva was collected by passive drool through a 5-cm long plastic straw. Samples were stored at −80°C until shipment to the Oregon National Primate Research Center (ONPRC) at Oregon Health Sciences University for analysis using Salimetrics Salivary Progesterone immunoassay kits (Salimetrics LLC, State College, PA). The intra-assay and interassay coefficients of variation (CVs) for this assay as performed by ONPRC were 5.63% and 8.37%, respectively.

We obtained approximately 8-mL samples of whole blood via standard venipuncture and stored them at −80°C until they were shipped to ONPRC for analysis. Serum P4 was analyzed in duplicate on an Immulite 2000 (Siemens Healthcare, Munich, Germany) automated clinical immunoassay system. Intra-assay and interassay CVs were less than 10%.

Starting on day 8 of their MCs, participants were instructed to perform non–first-morning void luteinizing hormone-based ovulation tests (Kurkel Enterprises, LLC, Redmond, WA) to detect the rise in urinary luteinizing hormone that indicates ovulation date. The tests were to be used at the same time each day, between 11:00 am and 8:00 pm as recommended by the manufacturer. Each participant underwent 2 educational sessions regarding ovulation test procedures and result documentation before beginning the tests. Results were recorded on a log sheet and discussed with investigators at each visit to ensure accurate documentation and compliance. Tests were either stopped after a positive result or continued until the first day of menstruation during the subsequent cycle. If a test was considered invalid (ie, inconclusive or suspected to be faulty), we instructed participants to repeat it at the next normal void.

At the time of enrollment, participants were randomly assigned a mock-injury event date that would be subsequently used to evaluate the algorithm's accuracy in identifying the MC phase at time of injury. In an effort to simulate the recall of the date of significant musculoskeletal trauma, such as an ACL tear, participants were not informed of the mock injury until the scheduled date, when they were given a brightly colored bracelet displaying the date in large, bold text. They were not required to wear the bracelet; however, they were asked to place it in a location they would observe multiple times throughout the day. At each subsequent visit, we stressed the importance of this mock-injury date. The motivation for providing a date-inscribed item and stressing the importance of remembering the date was to create a memorable event date that would serve as a surrogate for the point in time when an athlete actually sustained a significant injury.

To assess the accuracy of the algorithm to retrospectively determine MC phase at the time of mock injury, participants were interviewed by an investigator (K.J.T.) who was blinded to all aspects of the study relating to MC characterization. She acquired MC information using a modified version of the Hormonal History Questionnaire (HHQ), which has been previously validated and applied.¹²  The HHQ information included average MC length, normal days of menstruation, date of the first day of the last menstrual period, average number of days between periods, number of menstrual periods in the past 12 months, premenstrual symptoms and their severity, use of hormonal therapies of any kind, and the date of the mock injury. Random numbers (1−45) were assigned to specify the number of days after the mock injury that attempts to contact the participant and administer the HHQ could begin. This was done to replicate, as best as possible, real-world variations in the time between injury and enrollment in a retrospective case-control investigation, because the recall accuracy of HHQ information for a particular MC declines as time progresses.²⁹  Similarly, in most studies of ACL injury risk, it is not always possible to obtain a saliva sample on the day of injury. We therefore randomly selected a salivary sample from among those obtained on or after the mock-injury date but before the start of the next menses.

The algorithm was applied to these retrospectively acquired data using 2 methods to classify MC phase at the time of mock injury. First, it was applied in a purely objective fashion using a sequence of yes/no responses (Figure). Second, 4 investigators (S.J.S., I.M.B., D.M.H., J.R.S.) applied the algorithm using the same salivary P4 concentrations and self-reported menses dates. The investigators had backgrounds typical of researchers interested in studying the relationship between MC phase and injury risk.

To determine the validity of the algorithm when applied purely objectively and then by each of the 4 investigators, each woman's assignment into the preovulatory or postovulatory phase of the MC at the time of the mock injury was compared with her known phase on that date as determined by an expert in reproductive medicine (I.M.B.) using the prospectively acquired data (luteinizing hormone-based ovulation detection kits, serial serum P4 concentrations, and menses dates). We conducted secondary analyses to examine whether algorithm accuracy could be improved by using salivary P4 samples obtained on the mock-injury date or by modifying the algorithm to include MC-history information only. To determine how the timing of injury during an MC influences algorithm performance, we examined its accuracy when each of the 8 visits was selected as the mock-injury date. Salivary P4 concentrations were analyzed using mixed-model linear regression to test the differences between MC phases and estimate within-person and between-persons variability.

The following accuracy measures were computed. Sensitivity was the proportion of women in the preovulatory phase who were correctly classified as preovulatory. Specificity was the proportion of women in the postovulatory phase who were correctly classified as postovulatory. Positive predictive value (PPV) was the proportion of women classified as preovulatory who were actually in the preovulatory phase. Negative predictive value (NPV) was the proportion of women classified as postovulatory who were actually in the postovulatory phase. Overall accuracy was the proportion of all women who were correctly classified.

We used κ statistics to assess agreement in phase assignments between investigators and the mathematical algorithm. Pearson product moment correlation was used to examine the relationship between salivary and serum P4 concentrations. All data analyses were performed using SAS (version 9.2; SAS Institute Inc, Cary, NC).

Two athletes were excluded during the course of the study: 1 participant did not complete all data-collection requirements, and 1 had an average PAR <60 kcal/kg/wk during the 4-week testing interval. Demographics for the remaining 31 women are listed in Table 1; their average PAR score was 78.9 ± 30.2 kcal/kg/wk. Overall, the mean salivary P4 concentration was higher for samples obtained during the postovulatory phase (as determined based on prospectively acquired serum samples and ovulation tests) than for samples obtained during the preovulatory phase and from anovulatory women (Table 2). However, the estimated within-phase variability for samples taken from the same woman (CV = 0.50) was as large as the variability between phases (CV = 0.49) and much larger than the variability among women (CV = 0.31). The overall correlation between salivary and serum P4 concentrations taken on the same day was R = 0.71.

Menstrual-cycle length ranged from 19 to 60 days (mean = 31 ± 9.5 days). The mock-injury date was, on average, 10.7 days after the onset of menses (range = 0–31 days). The average number of days to the next menses after the injury date was 19.8 days. Based on the prospectively acquired data, 14 of the 31 participants (45.2%) were anovulatory during the tested cycle, whereas 15 (48.4%) were preovulatory and 2 (6.5%) were postovulatory at the time of their mock injury. The HHQs were administered an average of 20 days after the mock-injury date (range = 6–54 days), and 23 athletes (74%) accurately recalled the starting dates of the menses before and after the mock injury. Six participants reported incorrect dates for the menses before the mock injury, with differences ranging from −3 to +4 days, and 4 women (including 2 with incorrect starting dates) incorrectly reported the date of menses after the mock injury, with differences ranging from −3 to +5 days. The number of days between the mock injury and the randomly selected saliva sample used to evaluate the algorithm ranged from 0 to 35 days (median = 10 days).

Each athlete's self-reported MC data and salivary P4 concentration from the sample selected for use in the algorithm are shown in Table 3, along with the menstrual-phase classification at the time of the mock injury as determined by the algorithm, and her actual phase. The sensitivity of the algorithm to correctly classify the 29 participants in the preovulatory phase at the time of mock injury was 75.9%, whereas its specificity for correctly classifying the 2 postovulatory participants was 50% (Table 4). The PPV of a preovulatory/anovulatory classification by the algorithm was high (95.6%), due to the high prevalence of this ovulatory phase (93.5%), which corresponds to the PPV that would be achieved using a completely random phase assignment. The NPV of a postovulatory classification was 12.5%.

One postovulatory woman was incorrectly classified by the algorithm as preovulatory/anovulatory because her mock injury occurred after ovulation but more than 14 days before her next menses. Of the 8 athletes that the algorithm classified as postovulatory at the time of injury, only 1 was actually postovulatory, 5 were anovulatory, and 2 were preovulatory, as determined prospectively based on serial serum P4 concentrations and ovulation tests (Table 3). The 2 preovulatory women (participants 28 and 30) had mock injuries within 14 days of the next menses but ovulated after the mock-injury date. They both had salivary P4 concentrations well above 190 pmol/L during their entire cycles, so the timing of their saliva samples did not contribute to the misclassification. Of the 5 anovulatory individuals, 2 (participants 26 and 29) had MCs that were shorter than their average cycles but were classified by the algorithm as postovulatory because of their high salivary P4 concentrations. Conversely, participants 24 and 25 were misclassified as postovulatory despite their low salivary P4 concentrations because their MCs were the usual length. Participant 27 was misclassified because on the HHQ she erroneously recalled her menses date as being 13 days after the mock-injury date, though the actual date was 16 days after injury.

The menstrual-phase classifications obtained when the 4 investigators applied the algorithm differed somewhat from the objectively computed classifications, with agreement ranging from 80.6% (κ = 0.50, 95% confidence interval = 0.15, 0.84) to 93.5% (κ = 0.83, 95% confidence interval = 0.60, 1.00). Sensitivity ranged from 72.4% to 75.9%, which was comparable with the mathematically applied algorithm. Three investigators correctly classified 1 of the 2 postovulatory women (50% specificity), and the fourth correctly classified both (100% specificity).

When the objectively computed algorithm was based on saliva samples taken on the date of mock injury, rather than the randomly selected sample, accuracy showed little improvement (Table 4). The algorithm correctly classified 2 preovulatory/anovulatory athletes who were previously misclassified and misclassified 1 preovulatory/anovulatory athlete who had previously been classified correctly. Modifying the algorithm so that it was based only on the self-reported MC data improved sensitivity to 89%. It also improved both PPV and NPV, but NPV and specificity both remained very low (Table 4).

The accuracy of the algorithm varied widely depending on timing of the mock injury (Table 5). Sensitivity was highest when the mock injury occurred early in the MC because all or most participants were in the preovulatory phase, but it dropped steeply at midcycle, when the algorithm classified many preovulatory participants as postovulatory. In contrast, specificity and NPV were higher when the mock injury occurred later in the MC. There was no time during the MC when both the sensitivity and specificity of the algorithm were acceptable.

Because ACL injuries occur infrequently, case-control studies remain the most practical research design to explore relationships between MC phase and ACL injury risk. For these retrospective studies to be valid, it is necessary to establish an objective procedure (algorithm) to accurately classify an individual's MC phase at the time of injury based on data that can be reasonably obtained after injury. Consequently, the goal of our study was not to create a novel algorithm but rather to evaluate our ability to classify phase using an algorithm based on current MC-phase knowledge, as has been used by previous investigators attempting to classify phase after injury. Our evaluation of an objective algorithm based on a menstrual-history questionnaire and single salivary P4 sample obtained after a mock injury indicates that these data were insufficient for accurate retrospective classification of participants as preovulatory/anovulatory versus postovulatory at the time of a mock injury.

When applied objectively, the algorithm correctly classified only 74% of the 31 participants as being preovulatory versus postovulatory at the time of injury. Similar results were obtained when experts in the field applied the algorithm, reflecting their 80.6% to 93.5% agreement with the objectively applied algorithm. Although this suggests that the algorithm could yield similar results from different investigators, it was unable to classify participants with a high degree of accuracy. This lack of accuracy may be due to 3 primary sources of error: (1) the inability of athletes to correctly recall their menstrual history accurately, (2) assumption of a stable, constant, 14-day luteal-phase window, and (3) the inability of a single hormone sample to adequately capture the large variability in MC characteristics and hormone profiles across women.

The study was designed to initiate contact with study participants for the purpose of acquiring menstrual-history data at a random time interval (1−45 days) by a blinded investigator after the mock-injury date to mimic real-world situations in which it may not always be feasible to obtain data immediately after the injury, particularly in large-scale studies. We were able to make contact with the participants and acquire their menstrual-history data between 6 and 54 days after the mock injury. These data were then used to identify the start date for the menses occurring before and after the mock-injury date. Studies accessing self-reported MC information indicate there may be substantial recall error, particularly in younger females who have more variable cycle lengths.³⁰  Moreover, recall error may be particularly problematic if injury occurs near midcycle (ie, ovulation), when even a 1-day or 2-day error in menses dates could result in a different phase classification. In our study, 23 women (74.2%) correctly recalled the starting dates of the menses before and after the date of mock injury. Of the 8 women who erroneously reported 1 date or both dates, only 1 was misclassified because of the error, and her mock injury occurred midcycle, so recall error was not a major reason for the algorithm's poor performance.

In addition to its potential recall error, self-reported menstrual-history information cannot reveal the length of specific MC phases or the timing of ovulation. Although the average MC length is 28 days and ovulation on average occurs around day 14, the actual length of an individual cycle, the timing of ovulation, and thus the length of the follicular and luteal phases are all known to vary.^{20,21,31−34}  Specific to our methods, counting back 14 days from the start of the next cycle was based on the assumption that the length of the luteal phase is more consistent than that of the follicular phase.^5,9,17,31  Whereas the follicular phase is reported to average 13 to 14 days in length,^21,31,33  actual luteal-phase lengths range from 11 to 18 days in young females aged 18 to 24 years²¹  (8−17 in females aged 18−40 years^31,33 ). Moreover, some findings suggest that exercising women have significantly shorter luteal phases (eg, 8.2 ± 0.5 days).³⁵  In a study of 73 physically active females aged 18 to 30 years,²⁰  only 32% had a positive urinary ovulation test and 59% had attained a P4 level criterion of >2.0 ng/mL when counting back 12 to 14 days from the start of the next cycle. Hence, using a standard criterion of 14 days to represent the luteal-phase length in physically active women may contribute to the inaccuracy of the algorithm. In our study, the phases of 2 participants were misclassified because ovulation occurred less than 14 days before the start of the next menses. A summary of previous studies examining the relationship between MC phase and ACL injury risk is provided in Table 6.^3,5,7,8,10 

Given the challenges of these calendar-based methods, more recent investigations of the relationship between ACL injury risk and MC phase have evaluated hormonal concentrations near the time of injury (<72 hours) to better estimate the actual hormone milieu.^5,10,12  However, despite the inclusion of salivary P4 concentration in our algorithm, we were unable to achieve an acceptable level of accuracy in MC-phase determination, even when we used the sample obtained on the mock-injury date. We suspect this may be due to the choice of a 190-pmol/L threshold, as P4 concentrations observed in our study were higher than those reported by Chatterton et al.²² Yet when we used the optimal threshold for classification in our study (340 pmol/L) and the saliva sample obtained on the day of injury, sensitivity and specificity were no better than when the algorithm was based on self-reported MC data alone.

Use of a single sample to determine MC phase has recently been questioned.¹⁹  Because of the inherent variability in MC characteristics, it may be difficult to determine from a single sample whether hormone levels are rising, peaking, or falling. Hence, a single sample coupled with the known inaccuracies in calendar-based counting methods already discussed may not provide sufficient information to accurately identify the phase of the MC at the time of injury. Although research suggests that taking multiple samples around the event of interest may provide a better representation of the hormonal milieu than a single sample,³⁴  further study is needed to determine if this would improve the determination of MC phase. We did not explore the use of multiple salivary P4 samples to determine phase in our study because of their large variability. Samples obtained from the same woman differed as much within each MC phase as between phases, so she could have a preovulatory sample with a higher P4 concentration than a postovulatory sample.

We have 2 concerns regarding the high number of anovulatory cycles that occur in young athletic females, despite their normally occurring menses. First, we have concern that an altered hormonal milieu may affect their bone health, and this requires further attention. Second, with our algorithm, anovulatory participants meeting the third criterion of the algorithm (sample below salivary P4 threshold, evidence of menses within 14 days of injury, and onset of menses within 2 days of the normal MC length reported in the questionnaire) were incorrectly classified as postovulatory. This is particularly problematic for competitive athletes, as our reported frequency of anovulatory MCs (45%) was more than twice that for females exercising >2 h/wk at 55% of maximum heart rate²³  or between 2 and 10 hours for 3 months or longer.³⁴  It is possible that the algorithm could be modified to better distinguish between women in the postovulatory phase and those experiencing an anovulatory cycle. However, correct classification of women with anovulatory MCs that are of similar length to their usual MCs is likely to remain a problem unless more predictive hormone data are available.

Compounding these difficulties is the fact that errors in menstrual-phase assessment are more likely when injuries occur at certain times during the MC than others. We demonstrated this by examining how the sensitivity and specificity of our algorithm changed with the timing of the mock injuries. The accuracy of other methods of phase determination is also likely to vary depending on the timing of injury, which is of concern because it indicates that the inaccuracy reflects bias as well as imprecision. Although this was not the primary goal of our study, future investigators should evaluate the temporal response of sex hormones across multiple MCs, as well as the effects of physical and emotional stress on sex-hormone levels in collegiate-aged competitive athletes, as this information may aid in the ability to retrospectively classify MC phase after musculoskeletal trauma.

Whereas this study was limited to a relatively small sample of collegiate athletes with a high percentage of anovulatory cycles, they do represent the high-risk populations typically included in ACL injury risk-factor studies. Only 2 of the women were postovulatory at the time of mock injury, limiting our ability to assess the specificity with which the algorithm classified these individuals. However, the analysis discussed earlier, in which alternative mock-injury dates were examined, indicated that postovulatory women were often incorrectly classified unless the injury occurred near the end of the MC.

We showed that an algorithm using a single measurement of salivary P4 concentration to retrospectively classify MC phase was not valid for use in risk-factor studies. These findings raise substantial questions regarding the accuracy of prior investigations that retrospectively determined the MC phases of young athletes based on calendar methods with or without a single hormone sample, particularly in a population with such a high occurrence of anovulatory cycles.¹⁹  It is possible that accuracy may be improved by acquiring multiple samples immediately after the day of injury, but our data indicate that salivary P4 may not be adequate for this purpose because of within-phase variability. Ultimately, accurate determination of MC phase may only be possible through prospective examination that captures both estradiol and progesterone concentrations over multiple days around (ie, both before and after) the time of injury. This would be virtually impossible for studies of risk for ACL trauma because a very large number of athletes would need to be tested over extended periods of time to accrue an adequate sample of injured participants.

Funding for this investigation was provided by research grant R01-AR050421 from the National Institute of Arthritis and Musculoskeletal and Skin Diseases of the National Institutes of Health (Dr Beynnon).

We thank Hellen S. Hollenbach, PA-C, MS, ATC, and Sarah L. Davidson, MD, for their assistance with data collection. We also thank our research participants for their time and efforts with this investigation.