Can we rely on people to follow protocol? – Compliance in Experience Sampling protocols (and physiological measures)

Compliance has been a delicate subject in psychological research: how do we know whether the participant will follow protocol in a truthful and accurate manner? A lot of ingenious  ways of control this have been created, but if the task is hard and burdensome, it is more likely that participants will fail to comply fully. The Experience Sampling Method (ESM), and others alike, are frequently demanding, and require the researcher to work with participants towards increasing motivation and compliance with protocol (even when a reward is given). These methods require answering questionnaires whenever participants are prompted, several times a day, usually during one week. The content of the questionnaire and the amount of times the participant is prompted  to answer during each day, depend on the design of the study. However, regardless of having to answer four or eight times a day, it still disrupts daily life; even if that is not intended. By adding physiological measures to the ESM, researchers are also able to find associations between physiological and psychological functioning; but at the cost of loosing not only rates of compliance, but volunteers. This happens because the procedure becomes even more demanding, and it can cause some distress due to social self-awareness when taking physiological samples.

The effects of compliance (or lack of thereof) have been studied in the context of psychophysiological studies in daily life, namely with cortisol. In this context, individuals must take their own samples when prompted, and they must fill a questionnaire simultaneously. After the signal, participants must promptly take samples and answer the questionnaire to avoid loosing acuity in data, due to bias processes, reconstruction, or simple forgetting. Also, by inaccurately sampling cortisol, participants might risk compromising the results of the study. Electronic monitoring is to control the timing of the sampling, either by using caps that register the time when the vial is open to take the sample, or by using movement sensors that determine when the individual wakes up. However, monitoring using electronic caps is useless if individuals open the vial and remove several samples that are necessary for the next few hours, instead of only opening the vial to take one sample at a time.

However, the protocol design can increase compliance. Some authors have discussed that when volunteering to an ESM study, the sample is not random, but already sorted in a way that the most diligent, motivated individuals will be the ones participating in the studies. Daily life studies with cortisol sampling, observed that we can increase compliance by informing that we are monitoring individuals participation (Kudielka et al., 2003; Broderick et al., 2004). Other studies’ protocols include explaining cortisol’s functioning and the consequences of non-compliance to participants. It follows that being aware of the impact of one’s behavior on the study’s results seems to have an effect on individuals compliance.

Some studies have shown that including non-compliant participants can have a negative impact on results (Kudielka et al., 2003; Kudielka et al., 2007). However, this contrasts with what others like Jacobs and colleagues (2005) have observed. When comparing these different studies, we can see that:

1) It seems that compliance to the timing of sampling is particularly important when we take samples during the first hour after waking. These first measures are often used to assess the cortisol awakening response (e.g., Kudielka et al., 2007);

2) The first studies used interval-contingent designs, while the latter uses a signal-contingent design.

3) while the first studies characterize participants as compliant vs non-compliant (the effects of non-compliance are based on the comparison between compliant and non-compliant participants); the latter study characterizes accurate vs inaccurate samples. We assess the effects of inaccurate samples by comparing the estimates for the circadian pattern (cortisol slope throughout the day) of accurate vs inaccurate vs accurate+inaccurate samples.

4) Another study by Broderick and colleagues (2004) categorized the day-samples within individuals as compliant vs non-compliant day. By comparing different days within-individuals they ascertain that CAR had no increases in non-compliant days, compared to compliant days, and that the daily slope (measured with sample taken 30min after awakening, and the sample taken at 22h) was also flatter in non-compliant days, than in compliant days.

As for point one, compliance is vital to assess cortisol awakening response, and, whenever possible, electronic monitoring should be included in these studies. The cortisol awakening response is an extremely changeable pattern in cortisol functioning that occurs in the first hour since awakening, and that changes drastically within minutes. Because of this, lack of compliance, either because the participant forgets to place the sampling device close to the bed, or takes longer to assess the 2nd or 3rd timed samples can affect results, and render samples useless.

The protocol design can influence not only compliance, but the individual’s response. In interval-contingent designs participants might anticipate the answers to their questionnaires. They might rehearse to themselves the answers to give at the specific time of sampling. If something happens that delays the sampling time, participants might be more inclined to lie when reporting the time of filling the questionnaire, and sampling cortisol. In the case of signal-contingent design, inaccurate sampling is not as frequent. Because participants don’t know when they will be prompted to fill in the questionnaire and take the cortisol sample, they must pay closer attention to the beeper. Also, when they miss a signal, they don’t know if it occurred or not, hence, there will more missing data, than inaccurate samples.

By characterizing participants as non-compliant we might be loosing important data about those participants. It is common that participants aren’t as compliant in certain days, such as weekends, but that doesn’t exclude the possibility that they were compliant and accurate during the week sampling. By characterizing momentary samples as inaccurate, we can control for the possibility that those samples might introduce bias in our results, and either include them if we see no significant effect on results; or exclude them, if we prefer to be more conservative.

Finally, it is vital that participants are compliant to correctly measure the CAR. Using a sample taken 30min after awakening to assess the daily slope can lead to biased results, as the measure itself might be incorrect due to noncompliance. To assess the daily slope, it would be helpful to gather more samples throughout the day, and then compare them, without including the measures of the morning (the first hour after awakening).

In conclusion, to measure time-sensitive patterns of cortisol (e.g., CAR), or other physiological measures, monitoring is necessary to assess accurate results. However, choosing a protocol that increases the likelihood of compliance, as well as working with participants to increase their motivation can prove to be an effective, and less costly way to increase compliance, especially in less time-sensitive patterns such as momentary reactions to stimuli and circadian patterns.


Broderick, J. E., Arnold, D., Kudielka, B. M., & Kirschbaum, C., 2004. Salivary cortisol sampling compliance: Comparison of patients and healthy volunteers. Psychoneuroendocrinology 29, 636-650.

Jacobs, N., Nicolson, N. A., Derom, C., Delespaul, P., Van Os, J., & Myin-Germeys, I., 2005. Electronic monitoring of salivary cortisol sampling compliance in daily life. Life Sciences 76, 2431-2443.

Kudielka, B. M., Broderick, J. E., & Kirschbaum, C., 2003. Compliance with saliva sampling protocols: Electronic monitoring reveals invalid cortisol daytime profiles in noncompliant subjects. Psychosomatic Medicine 65, 313-319.

Kudielka, B. M., Hawkley, L. C., Adam, E. K., & Cacioppo, J. T., 2007. Compliance with ambulatory saliva sampling in the chicago health, aging, and social relations study and associations with social support. Annals of Behavioral Medicine 34, 209-216.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s