4 Mood
Premium users of Daylio can export their mood entries in a CSV file. This is great because one can conduct further analyses in a software such as R. In this chapter, we’ll learn how to clean up and augment the mood data in an attempt to answer life’s deepest questions about human flourishing.
Am I becoming happier?
Am I becoming more stable?
4.1 Getting To Know The Data
4.1.1 How Many Times Per Day Do I Rate My Mood?
I used to only give one rating per day to measure my mood. I soon realized that a single mood rating was too rough of an estimate to effectively measure how I felt during the day. Our affect can fluctuate significantly throughout the day. It’s for this reason that I started increasing the number of ratings per day when I saw fit. Let’s see if we can see this trend reflected in the data. My hypothesis is that the later years will have fewer days with a single rating.
As expected, there are many more days in 2023 with 2 or 3 ratings per day. The distributions are becoming less skewed over time as I increase the number of daily ratings. Note that in 2020, nearly every day had a single mood rating.
Important
As we’ll learn later in this chapter, the increase of daily ratings most likely didn’t compromise the validity of the overall happiness scores.
4.1.2 What Is My Average Daily Mood?
To begin, let’s plot the distribution of ratings during the entire period.
As we can see in Figure 4.3, 4 out of 5 is the most common rating. The distribution is skewed to the left indicating that my life is generally pretty good.
Let’s take a look at the descriptions of these ratings.
We can some states are chosen more often than others. This is partly because these moods were not all present when I started tracking my mood. Some moods such as Equanimous were added in 2023. We can see for example that life is good and good are common ratings. That said, I think of life is good as slightly better than plain good. It’d be interesting to manually change the value of some of these moods such as:
Overwhelmed = 1.5
Tired = 2.5
Good = 3.5
Equanimous = 3.5
Flow State = 4.5
We can also look at the trend of the distributions over the years.
We can see from Figure 4.5 that the rating distribution is fairly constant over the years. One apparent difference is that there are more level 5 ratings in the later years. This is almost certainly because I started tracking my mood multiple times per day. It’s rare that the overall rating of a day is a 5/5. That said, peak states occur almost on a daily basis. It’s just that they are usually accompanied with less pleasant affective states.
On average, my days are a 3.72 out of 5 with a standard deviation of 1. This is pretty solid given the unpredictable nature of life and it’s hardships.
Min. 1st Qu. Median Mean 3rd Qu. Max.
1.00 3.00 4.00 3.72 4.00 5.00 Looking at the summary statistics above, it makes sense the minimum daily mood rating is 1 and the maximum is 5 given that the scale goes from 1 to 5. Some days are miserable and you just want to sleep the misery away. Other days are inherently perfect and you wish you could replicate those days until you die. This speaks to Buddha’s wisdom of the law of impermanence and the futility of craving the highs and developing aversion for the lows. The affective state is in a constant flux.
The first quartile is a 3 on the scale of 1 to 5. This implies that the range from 1 to 3 contains only about 25% of the observations. In other words, 75% of the overall mood ratings lie in the top half of the range (3 to 5) while the bottom half (1 to 3) contains the remaining 25%.
The fact the the mean is lower than the median is an indication of this asymmetry. The mean gets pulled to the left by the rare lower ratings while the median is more robust to these outliers.
Now that we know my baseline happiness level, we’re ready to tackle the big questions stated in the first paragraph of this chapter.
Am I becoming happier?
Am I becoming more stable?
4.2 Am I Becoming Happier?
Considering that the type of plot displayed in Figure 4.8 will be used over and over in this book, let’s break down each component of the graph to ensure our understanding.
Each dot represents the average mood of a particular day. If we focus on the first violin for the Spring season, we should see 130 dots.
The dots are transparent because of the overlap. Darker clusters of dots indicate a high density of mood ratings in this region. This density is also visualized with the shape of the violin. Large bumps indicate more dots. As expected, the larger bump for most year is around the 4 out of 5 range.
The red label indicating a number +- another number represents the mean and standard deviation respectively. The mean is simply the average of the 130 daily mood ratings. We add all the values and divide by the number of observations. The standard deviation represents the variability of the 2018 observations. Since the vast majority (89%) of the 2018 observations are between 3 and 5 on a scale from 1 to 5, it makes sense that the standard deviation is 0.94.
The red dot and error bars1 represent the mean and 95% confidence interval (CI) respectively. The true interpretation2 of a confidence interval is a bit advanced and theoretical for the scope of this book. The bottom line is that you can think of CIs as a measure of the accuracy of the means. Narrow CIs indicate more accuracy and wide CIs are associated with a less accurate average. Generally speaking, the smaller the standard deviation and the more observations in the sample, the narrower the CI and the more accurate our estimate of the mean will be. For the year of 2028, the 95% confidence interval is [3.75, 4.08]. This is a relatively narrow confidence interval and will allow us to reach conclusions with a moderate level of certainty.
The horizontal lines in the violins represent the quartiles of each distribution. We can see that the first quartile occurs around a 3 on the scale of 1 to 5.
Figure 4.9 is a great example of how data visualizations can mislead the audience. By simply changing the y-axis it now seems as though my yearly average fluctuates significantly. This is why the confidence intervals are crucial.
Df Sum Sq Mean Sq F value Pr(>F)
year 1 0.1 0.0650 0.065 0.798
Residuals 1827 1813.9 0.9928 When performing a one-way ANOVA test, there is no statistically significant differences between the yearly averages. We can tell since the p-value is not small. To conclude that there are mean differences, we usually want a p-value under 0.05 or 0.01 depending on the context.
Important
The normality and independence assumptions of the ANOVA test are likely violated. The mood distributions are skewed to the left. Furthermore, there’s nothing magical about January 1st. Time is a continuum and not discrete chunks. Therefore, the mood distributions are not completely independent. How I felt in December 2020 likely impacts how I felt in January 2021. Many of the environmental factors carry overs from year to year.
That said, it looks like 2020 was a rough year given it holds the lowest mood and highest volatility. This makes intuitive sense given the global pandemic and it’s imposed restrictions. But is this difference significant or meaningful?
Two Sample t-test
data: mood_avg by year
t = -2.6936, df = 729, p-value = 0.00723
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-0.36889302 -0.05786048
sample estimates:
mean in group 2020 mean in group 2021
3.549180 3.762557 Although there’s some overlap between the confidence intervals for the year 2020 and the year 2021, the two-samples T-test reveals a statistically significant difference with \(p < 0.01\).
We know this difference is statistically significant but is it meaningful? A mood of 3.76 for 2021 is only 6.01% higher than the average of 3.55 for 2020. Is a 6.01% boost in mood or an increase of 0.21 on a scale of 1 to 5 meaningful? In my opinion, yes it is. This effect size is most likely similar to getting a good night sleep or spending time with friends as we’ll explore in later chapters. I’d happily take a drug that boosts my mood by 6.01% if it had no undesirable side effects.
Another approach to trying to see if I’m getting happier is to calculate the cumulative average of my mood since the beginning. If I’m getting happier, this cumulative average should rise over time.
As seen in the time series above, my cumulative mood converges to 3.72 and doesn’t show signs of going up.
Let’s investigate further the hypothesis that the pandemic related restrictions had a negative impact on my mood.
4.2.1 Did the Pandemic Impact my Mood?
The university of Oxford developed a stringency index in an attempt to quantify to intensity of the restrictions imposed. Let’s first visualize the stringency over time to see if it passes the eye test.
The stringency index does seems to accurately capture the intensity of the measures in Ontario. An entire book could be dedicated to studying Oxford’s data. However, we’re curious about the impact of the restrictions on my mood ratings.
As seen in Figure 4.12, the daily mood fluctuates regardless of of the stringency index. The correlation between my mood on a specific day and how severe restrictions were on that day is near zero.
Perhaps mood is too volatile on a daily basis for this kind of analysis. Let’s try to see if there’s an effect at the weekly and monthly level.
As we can see in the charts above, severe the restrictions had no correlation with my mood at the weekly or monthly levels. If something as intrusive as the lock downs had no major effect on my reported mood, what other external factors could?
4.2.2 Are Some Chapters Of My Life Happier Than Others?
| Chapter | Start | End |
|---|---|---|
| Teacher's college | 2017-09-01 | 2019-04-30 |
| Supply Teach in Kapuskasing | 2019-05-01 | 2019-06-30 |
| Gee-gees camps | 2019-07-01 | 2019-08-31 |
| Statistics masters | 2019-09-01 | 2022-04-30 |
| Long-term supply teaching | 2022-05-01 | 2022-06-30 |
| MS-LAP | 2022-07-01 | 2022-08-31 |
| Permanent high school teacher | 2022-09-01 | 2023-12-30 |
Even major transitions such as changing school programs or changing jobs doesn’t seem to have a robust impact on my mood. The only difference that was statistically significant was between teacher’s college and my masters. This passes the eye test as teacher’s college was relative low stress and a blast compared to my masters. The pandemic also occurred during my masters.
4.2.3 What Is The Best Day of the Week?
Perhaps there will be a trend in the day of the week and my reported mood. Folk wisdom says that:
Sunday scaries are a real phenomenon.
Mondays are the worst.
People live for the weekend.
- Wednesday is hump day and the rest of the work week is down hill.
Will there be any truth to these claims? Let’s find out!
Just visually, it looks like: - Indeed Monday holds the lowest mood average with Tuesday as a close second. - Wednesday is slightly higher than the days on each side. - It does appears as though I live for the weekend. - Sundays scaries do not appear to have a strong effect.
Let’s use more rigorous statistical tests to confirm or infirm our early findings.
Df Sum Sq Mean Sq F value Pr(>F)
day_of_week 6 22.2 3.697 3.759 0.00101 **
Residuals 1822 1791.8 0.983
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Pairwise comparisons using t tests with pooled SD
data: daylio_by_day$mood_avg and daylio_by_day$day_of_week
Mon Tue Wed Thu Fri Sat
Tue 1.0000 - - - - -
Wed 1.0000 1.0000 - - - -
Thu 1.0000 1.0000 1.0000 - - -
Fri 0.0554 0.1373 1.0000 0.4855 - -
Sat 0.0065 0.0190 0.3396 0.0857 1.0000 -
Sun 1.0000 1.0000 1.0000 1.0000 1.0000 0.2332
P value adjustment method: bonferroni
Posthoc multiple comparisons of means: Scheffe Test
95% family-wise confidence level
$day_of_week
diff lwr.ci upr.ci pval
Tue-Mon 0.02503193 -0.283367229 0.3334311 1.0000
Wed-Mon 0.10459770 -0.203801456 0.4129969 0.9626
Thu-Mon 0.06398467 -0.244414483 0.3723838 0.9973
Fri-Mon 0.26114426 -0.046960485 0.5692490 0.1705
Sat-Mon 0.31337072 0.005265978 0.6214755 0.0427 *
Sun-Mon 0.09291188 -0.215487280 0.4013110 0.9794
Wed-Tue 0.07956577 -0.228833385 0.3879649 0.9909
Thu-Tue 0.03895275 -0.269446412 0.3473519 0.9998
Fri-Tue 0.23611233 -0.071992413 0.5442171 0.2850
Sat-Tue 0.28833879 -0.019765950 0.5964435 0.0874 .
Sun-Tue 0.06787995 -0.240519209 0.3762791 0.9962
Thu-Wed -0.04061303 -0.349012184 0.2677861 0.9998
Fri-Wed 0.15654656 -0.151558186 0.4646513 0.7757
Sat-Wed 0.20877302 -0.099331723 0.5168778 0.4469
Sun-Wed -0.01168582 -0.320084981 0.2967133 1.0000
Fri-Thu 0.19715958 -0.110945159 0.5052643 0.5226
Sat-Thu 0.24938605 -0.058718696 0.5574908 0.2197
Sun-Thu 0.02892720 -0.279471954 0.3373264 1.0000
Sat-Fri 0.05222646 -0.255583584 0.3600365 0.9991
Sun-Fri -0.16823238 -0.476337124 0.1398724 0.7087
Sun-Sat -0.22045884 -0.528563587 0.0876459 0.3739
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Based on the ANOVA test, there’s enough evidence to suggest that not all days have the same mood. This makes sense as the weekend seems significantly uplifted compared to the beginning of the week.
We can follow up and compare each pair of days and see that, depending on which method we use, the only pairs of days that significantly differ statistically are Saturday-Monday and Saturday-Tuesday. The Friday-Monday comparison is near the significance threshold.
In short, the only folk wisdom hypothesis that is supported by solid evidence is that I tend to live for the weekend.
Note
This doesn’t necessarily imply that I drudge the beginning of the week. Statistically speaking, Mondays and Tuesdays were far from differing from the other work days.
When separating the data by year, the patterns differ. Living for the weekend seems to be consistent from year to year. However, we can see that in 2020 and 2021 Sundays had an elevated mood as well. Some momentum from the previous two days can carry over into Sunday. Furthermore, during the pandemic, I imposed a rule for myself that Sundays was the one day every week where I wasn’t allowed to do any work. Sunday was my day to relax and sharpen the saw. The Sunday scaries do seem to come back in 2022 and 2023, however.
It appears that day of the week may have an effect on my mood. I suspect that this effect is not necessarily caused by the day of the week itself but rather by what comes with Saturdays (i.e. social interactions). We’ll explore this topic in later chapters.
Let’s see if other external factors impact my mood.
4.2.4 Is My Mood Impacted By Seasonal Effects?
Although the exact start and end dates of seasons vary from year to year, I decided to define the four seasons as such:
Spring: March 21st (including) to June 21st (excluding)
Summer: June 21st (including) to September 21st (excluding)
Fall: September 21st (including) to December 21st (excluding)
Winter: December 21st (including) to March 21st (excluding)
Df Sum Sq Mean Sq F value Pr(>F)
season 3 2.7 0.9046 0.911 0.435
Residuals 1825 1811.2 0.9925 We can see from the plot and the ANOVA test above that there’s not enough evidence to conclude that seasons significantly impact my mood. The averages for the fall and winter are slightly lower than for spring and summer. This aligns with folk wisdom.
The trend of slightly lower moods in the winter doesn’t hold up at the yearly level. In fact, some years such as 2018, 2019, and 2021 report the best average in the winter.
4.2.5 What Is The Best Month of the Year?
Given that seasons are somewhat arbitrary, let’s zoom in and look at the average mood and variance in any given month.
Df Sum Sq Mean Sq F value Pr(>F)
month_abb 11 13.3 1.206 1.217 0.27
Residuals 1817 1800.7 0.991 Again there are no obvious differences between months. Mood seem to peak for June through July and plummet in February and November.
There may not be any consistent trends but there are months that stick out.
Here are the 5 months with the lowest mood.
| Year | Month | Average Mood | Standard Deviation |
|---|---|---|---|
| 2020 | Feb | 3.10 | 1.28 |
| 2020 | May | 3.26 | 1.39 |
| 2020 | Nov | 3.37 | 1.16 |
| 2022 | Feb | 3.38 | 0.90 |
| 2022 | Oct | 3.41 | 0.98 |
Here are the 5 months with the highest mood.
| Year | Month | Average Mood | Standard Deviation |
|---|---|---|---|
| 2018 | Sep | 4.27 | 0.68 |
| 2022 | Jul | 4.11 | 0.90 |
| 2023 | Jun | 4.10 | 0.69 |
| 2021 | Dec | 4.09 | 0.78 |
| 2018 | Aug | 4.06 | 1.32 |
Important
Ratings from 2018 are likely outliers both in terms of mood average and standard deviation. I used to only give one rating per day and 4/5 was the default answer most days.
From looking at the two tables above, it seems as though months with low average had a higher standard deviation while happier months were also more stable. Let’s test this hypothesis while ignoring the ratings from 2018.
Although there is a lot of variability, the general but significant trend is that months with higher mood average tend to have lower mood variability. This makes intuitive sense since an monthly average above implies that the vast majority of dating need a 4 or a 5. Conversely, months with an average of 3 can have daily mood ratings that vary from 1 to 5 but average out to 3. The same thought process would apply if I had very low monthly averages. In fact, a monthly average of 1 would have a standard deviation of 0 since every day would need to be rated as a 1 out of 5.
4.3 Am I Becoming More Stable?
There are two ways to define stability.
- Stability of mood ratings within a specific day.
- Stability of mood ratings between days.
4.3.1 Within-day Variability
Calculating the standard deviation for entries within a day is typical approach to measuring volatility. Let’s manually calculate the variance for the 6 entries on August 16th 2023 so we have an idea of how the formula works3.
| Date | Mood | Rating (1-5) |
|---|---|---|
| 2023-08-16 | good | 4 |
| 2023-08-16 | Flow state | 5 |
| 2023-08-16 | irritated | 2 |
| 2023-08-16 | meh | 3 |
| 2023-08-16 | Equanimous | 4 |
| 2023-08-16 | irritated | 2 |
The sample variance formula is:
\[s^2 = \frac{\sum_{i=1}^{n}(x_i-\bar{x})^2}{n-1},\] where \(\bar{x} = \frac{20}{6}\approx 3.33\) is the sample mean of the 6 entries. Thus, we have: \[s^2 =\frac{(4-3.33)^2+(5-3.33)^2+(2-3.33)^2+(3-3.33)^2+(4-3.33)^2+(2-3.33)^2}{6-1},\] which is equal to a variance of 1.47. Taking the square root of the variance we get that the standard deviation of the moods on August 16th 2023 is \(s=\sqrt{1.47}\approx 1.21\). Note that this day has a relatively high volatility as is evident by the number and range of moods tracked.
As seen in Figure 4.22, the vast majority of days have a standard deviation of 0 (no volatility). There are two ways this can happen:
- Only one rating per day.
- Many ratings at the same level in a day.
In fact, there are only 35 days or 2% of days with a standard deviation of zero with more than one mood rating.
From the plot above and Figure 4.2, we can conclude that it’s only after 2022 and surely after 2023 that I started tracking multiple times per day when appropriate. Consequently, let’s only consider the ratings after 2023 to verify whether my within-day volatility is decreasing over time. We’ll assume that any day with a single rating after 2023 means that it was a relatively stable day without a diversity of moods.
There’s no clear trend in the plot above. The within-day volatility goes up and down. Perhaps experiencing a range of emotions is simply part of the human experience and having 5/5 every day is not what we should aim for. We do see that more stable months tend to be happier which aligns with the findings of Figure 4.21.
Let’s recreate the plot above with the range of emotions as the y-axis. For example, recall the six mood entries for August 16th 2023. The range is the difference between the maximum and minimum value. Thus, the range for that day would be 5-2 = 3. The smallest range possible is zero while the widest range is 4 if a day have a 5/5 and a 1/5.
The range plot is easier to interpret and displays the same trend as Figure 4.27. It doesn’t appear that the range of moods reported in a day decreases over time.
One would need a continuous mood monitor (CMM) similar to the continuous glucose monitors to truly assess within-day variability. A few ratings per day is far too few to paint an accurate picture.
That said. let’s shift gears and analyze the between-day variability.
4.3.2 Between-day Variability
Below is an example of daily mood averages of two weeks to illustrate what is meant by between-day variability.
| Day | Stable | Volatile |
|---|---|---|
| 1 | 2.67 | 4.25 |
| 2 | 3.67 | 3.75 |
| 3 | 3.50 | 4.00 |
| 4 | 3.50 | 3.50 |
| 5 | 4.00 | 4.75 |
| 6 | 3.67 | 1.00 |
| 7 | 3.33 | 2.67 |
The stable column is the 5th week of 2023 which had an average mood and standard deviation of 3.48 \(\pm\) 0.41 compared to 3.42 \(\pm\) 1.25 for the volatile 13th week of 2023. This illustrates clearly how two week in a similar time period with similar mood averages can differ drastically when it comes to mood stability.
Figure 4.8 indicates that the day-to-day mood volatility differed from year to year. It’s not clear whether this variability is decreasing over time. Let’s analyse the day-to-day standard deviation at weekly, monthly and yearly level.
The three plots above each show the same trend at different levels of resolutions. In my opinion, the monthly level is the proper level of analysis when it comes to day-to-day mood variability. A week is simply too short.
It does appear that my mood was most variable in 2020 (pandemic) with \(s \approx 1.1\) and that it’s now at its most stable in 2023 with \(s \approx 0.8\). Perhaps there’s a chance I’m getting more stable and wise as I age. Subjectively speaking, I do feel as though “negative” events affect my mood for a shorter period of time and have a lower intensity. It’ll be interesting to see how things unfold moving forward.
4.4 Recap
After an in depth analysis of my self-reported Daylio mood data, we answered the following questions:
How many times per day do I rate my mood?
- Started out as once per day, now it’s about the two to three times per day.
What is my overall average mood and day-to-day volatility?
- I average a 3.72 out of 5 with a standard deviation of 1.
Am I becoming happier?
- Probably not. My mood hovered around the global average of
r``round(global_avg, 2)over the years.
- Probably not. My mood hovered around the global average of
Which external factors affected my mood?
The pandemic (2020) might have had a small negative but meaningful effect on my mood.
External circumstances such as new jobs, schools, or cities did not appear to have robust effects. This is granted that I’ve been fortunate enough that the chapters of my life were awesome.
It does appear that I live for the weekend. I suspect this is mediated by social interactions which we’ll explore in a later chapter.
Seasonal effects were small and inconsistent from year-to-year.
Am I becoming more stable ?
We don’t have the data to properly measure within-day volatility.
The between-day volatility does seem to have decreased over the years after it peaked in 2020.
The findings do match my subjective experience that I’m not necessarily getting happier but I do feel more stable and less at the mercy of external circumstances.
4.5 Closing Thoughts
I wonder what they Daylio mood ratings of an enlightened monk would look like. Would they report being a 5/5 every day? Would they report a range of moods while being able to remain equanimous through it all?
My hunch is that it’s more likely to be the second option. Negative affect has it’s utility and might be hard wired. That said, these habits of the mind can lead us astray. It certainly is possible to regulate affect to some degree combining a bottom-up approach by changing the external circumstances (medication, location, sleep, sleep, exercise, nutrition, friends, etc) and a top-down regulation using metacognition (meditation, journaling, therapy, etc).
Although it might be extremely difficult to raise happiness beyond a certain baseline (mine is around 3.7/5), the interventions stated above seem to be most effective when one is below their baseline and coping. Perhaps there’s a “good enough” level of happiness that when attained, trying desperately to become happier is unproductive and futile. I conceptualize well-being as the fuel allowing you to help yourself and others. The good news is that self-care and service both contribute to and maintain our well-being.
It’s also important to keep in mind that happiness is only one metric. Basing every decision on how the effects on one’s happiness levels is a foolish endeavor. Smoking cigarettes is good for you if all you care about is losing weight. Similarly, many situations may make you happier at the expense of others (including your future self). This is why Maslow conceptualized the dance between self-actualization and self-transcendence as worth striving for.4
The error bars look like a Star Wars TIE Fighter.↩︎
If I was to relive my life many times while tracking mood multiples times a day like I did in this life and compute 95% CIs fr each life sample, then 95% of the CIs contain my true happiness levels in the spring. Of course, this true happiness rating is only a theoretical construct that we can only hope to estimate with data.↩︎
Standard deviation \(\sigma\) is simply the square root of the variance \(\sigma^2\)↩︎
Read Scott Barry Kaufman.↩︎