26. January 2022
Learning Analytics Dashboards is a new feature in BigBlueButton giving moderators statistics about each participant’s engagement. In this case study, we describe how this might violate user privacy and suggest an improved design. As a workaround, we show how Learning Dashboard can be disabled.
What is Learning Analytics Dashboard?
With version 2.4 BigBlueButton introduced Learning Dashboard (newly renamed to Learning Analytics Dashboard). This feature enables moderators of meetings to get statistics about participants and their engagement in the meeting.
The following statistics are tracked per participant:
- their join and leave time,
- total participation time,
- cumulative speaking and camera activation time,
- number of messages sent in the public channel,
- types and counts of used emoji statuses,
- number of raised hands,
- current online status,
- and an activity score whose calculation is not explained.
The dashboard presents this data in a table view, where each participant is a row, as can be seen in the following screenshot. Here, Christian (me) is a moderator and is therefore exempt from receiving an activity score. Alice and Bob are normal participants. Their above mentioned statistics are listed. For instance, we can see that Bob activiated their camera for about 12 minutes and has already left the meeting.
Additional views are revealed by clicking on the four cards on top of the dashboard. Clicking on the Polls card shows all polls including answers and clicking on Raise Hand (surprisingly) puts the recorded user activities on a timeline with 10 minute granularity. Both views are shown in the following.
Who can access this data and when?
All moderators of a meeting can open the dashboard during the meeting and for a configurable period after it has ended. It is accessible through the moderator menu (cog next to participant list) and linked on the confirmation page after a meeting was closed. By default, the JSON file storing the recorded activity data is deleted 2 minutes after the meeting was closed (see default property list on GitHub).
Privacy Concerns
Learning Analytics Dashboard is enabled by default (see default property list on GitHub). It can be disabled for the whole BigBlueButton instance via a configuration file setting. Options to enable/disable the dashboard on a per-room basis (e.g. via the front-end Greenlight) have been requested (issues for BigBlueButton and Greenlight) but do not yet exist. Therefore, the data is collected if necessary or not.
Moreover, participants have no indication that this gathering of statistics is taking place. They are not asked for consent nor have they any option to opt out or view their personal statistics. This behaviour likely violates data protection regulations (like GDPR) as there is no legal basis for collecting, processing, and storing this personal data. While these statistics might be necessary for some teaching scenarios, this is not an appropriate assumption by default. But even if lawful, participants have no transparency and controls about the activity recording.
Learning Analytics Dashboard goes beyond just summarizing events in the meeting that any participant could have observed anyway. Keeping track of all participants’ behaviour including accurate time statistics would be impractical without technical support by the tool. Moreover, with activity score, it derives additional personal data apparently meant to assess participant behaviour.
Our Suggested Changes
To improve user privacy we suggest the following changes to BigBlueButton:
- Learning Analytics Dashboard should be off by default.
- Participants joining a meetings with activated Learning Analytics Dashboard should be notified upon joining and provided the choice to either consent and join or cancel joining. Additionally, if enabled for the meeting, participants could be given the option to opt out from providing their activity data for the Learning Analytics Dashboard but still attend.
- During a meeting, an active Learning Analytics Dashboard should be clearly indicated similarly to a running recording.
- For transparency, participants should be able to view their respective activity data as shown on the Learning Analytics Dashboard for moderators.
- The calculation of activity score should be explained for transparency.
We made a feature request on GitHub to implement these suggestions.
Workaround: Disable Learning Analytics Dashboard
Until more appropriate instruments of control exist for moderators and participants, it is advisable to disable Learning Analytics Dashboards. How to disable the feature entirely is described in this issue.
Conclusion
In the EMPRI-DEVOPS project, we happily use BigBlueButton for our regular project status meetings. The introduction of the Learning Analytics Dashboard feature is one of many examples we saw in our research that show how hard it is to keep the principle of privacy by design in mind when designing and implementing new functionality. Operators of BigBlueButton should check if Learning Analytics Dashboard can be used in compliance with local data protection regulations and if a legal basis exists for their deployment scenario and user base. We would be happy to see our suggestions implemented. Until then, Learning Analytics Dashboard will be disabled on our BigBlueButton server.