The ARRL incident of May 2024, a year later

Foundations of Amateur Radio

Just over a year ago, the ARRL, the American Radio Relay League, the peak body for amateur radio in the United States and one of the oldest of such organisations, experienced an incident.

During the weeks following, the ARRL was tight-lipped about the extent of the incident and most amateurs only really noticed that services were off-line or slow to respond. After months of delay and disinformation, the ARRL finally revealed that it was the subject of a ransomware attack and that it had paid a million dollar ransom. It went on to blame the authorities for its silence.

Mind you, it didn't tell me personally, it made public statements on its website. Similarly when I specifically contacted the ARRL to discover what information of mine it held, and what the status of that information was, the ARRL responded that I should refer to its public statements. It continued to state that my information was not compromised, since it only lived in LoTW, the Logbook of The World, the system it uses to coordinate the verification of amateur radio contacts, which are used to distribute awards like the DXCC and Worked All whatever.

Imagine my surprise when I received an email this week, sent from "memberlist@arrl.org" to my non-amateur radio email address. I confirmed with several amateurs that they too received this email. Informative, to a point, but likely well beyond anything intended by its author, it stated that LoTW was being updated with associated down time, incidentally, inexplicably, coinciding with the 2025 ARRL Field day, and it "will be fully migrated to the cloud". It went on to solicit donations. It made no reference whatsoever to the ransomware attack.

There's a lot hidden in that email.

Although the attack last year was linked to the outage associated with LoTW, the ARRL has continued to claim that the LoTW data was not impacted by the ransomware attack, but the email reveals that the system is being migrated to the cloud, in other words, right now, it's not in the cloud. Which begs the question, where is the server infrastructure for LoTW today, and more importantly, where was it a year ago when its systems were compromised?

From a public post by Dave AA6YQ, dated the 2nd of February 2021, in response to a message about a January LoTW committee meeting, we know that the LoTW server "now employs the current version of an SAP database engine". A month before that, Dave wrote another informative email that indicated that 105 thousand callsigns submitted logs to LoTW in the last 1,826 days or the five years between 2016 and 2021. There were logs from 21 thousand callsigns in the week prior to that January post. In all, according to Dave, there were 153,246 callsigns who submitted contacts to LoTW.

The LoTW committee meeting minutes are no longer available from the ARRL website, but I have a copy. The document states that there were 1.2 billion contacts entered into LoTW, big number right? The next line tells us that this resulted in 262 million QSO records. I wonder what happened to the other billion records? This activity was generated by 139 thousand users using 200 thousand certificates. For context, every VK callsign automatically comes with an AX callsign, but LoTW requires that you separately register each with its own certificate.

As someone who has been playing with databases since the 1980's I can tell you that LoTW is a tiny database. For comparison, the WSPR database is an order of magnitude larger, not to mention, more active. I have no insight into the business rules within the LoTW database, but the fact that updates are being processed in batches and that it regularly has delays indicates a level of complexity that I cannot account for.

As an aside, the LoTW committee document lists 10 members. Dave is not one of those listed. It makes me wonder who else has access to this database. Note that I have no reason to believe that Dave's information is questionable, nor that he has access that he shouldn't, he was after all a member of the LoTW committee from 2013 until 2017 when the ARRL removed all development resources from the LoTW. I'm asking who else has access and why? While we're here, who has been doing maintenance and updates on this system over the past seven years?

Moving on. The database for LoTW contains information from amateurs all over the planet, including those in Europe where the GDPR, the General Data Protection Regulation, enacted in 2016, is extremely strict on the security and disclosure of personal data with very heavy penalties for breaches. The GDPR requires notifications be sent within 72 hours of a breach, and that an organisation must designate a data protection officer. I wonder who has that role at the ARRL and I wonder if they told anyone? Did any European amateurs receive personal notification from the ARRL about their data, I know I didn't.

My first activation of LoTW was in 2013, now twelve years ago. I received certificate expiry messages in 2016 and 2019. Since then there have been no such messages. That's unsurprising, since I stopped using LoTW once I discovered just how broken it was. Don't get me started on portable and QRP variants of my callsign. My care factor is low as to when I last actually used it, since attempting to dig up that information would take considerable effort, but I can guarantee that it was before 28 October 2019, when the last certificate expired.

You might come to this point and ask yourself why am I digging into this at all?

Let me ask you some questions in addition to those I've already mentioned.

SAP, the database system which apparently runs LoTW, had 254 CVEs, or Common Vulnerabilities and Exposures listed, in 2020 alone. It continues to have exploits. When was SAP updated and is it up to date today?

Is it credible that LoTW wasn't compromised during the ransomware attack? Does the ARRL know this for sure, or did it just not detect the compromise?

We know that LoTW was down during the incident and according to the UptimeRobot service showed outages on the 14th of May 2024 but we still don't know exactly when this attack started.

As you might know, the ARRL is also the headquarters for the IARU International Secretariat, the administration body for the global representation of our hobby. It presumably shares infrastructure with the ARRL, but at no point in the past year have we been advised of the impact of this breach to the IARU.

What information is stored in LoTW and why has the ARRL continued to ignore requests for disclosing the specific information it holds on the users of that system? I know for sure that it knows my callsigns and my email address. I also know for sure that it required identity documents to prove my identity and right to use those callsigns. I have been told in writing that LoTW never deletes anything, so what does it store and can I delete all my records and if-so, how?

Why did I receive an update about the upgrade for LoTW when I'm clearly not an active user of the system?

The memberlist@arrl.org is used for all manner of services, including the propagation updates, and the three other ARRL bulletins. In other words, this address is used for a myriad of messaging. Is this information stored in a database and if so, where is this database? Was it compromised? What information is stored in that database? Are my details in that database, are yours?

While discussing this LoTW update email with other amateurs, I was informed by one amateur that even after they stopped being a member of the ARRL, as a direct result of the ransomware attack and the discontinuation of the delivery of QST magazine they paid for, the ARRL continued to send regular email updates as-if they were still a current member. Where is that data stored and how are the ARRL not considered a source of SPAM?

While we're exploring the blurred lines between being a member of the ARRL and not, why did it send the update about the incident via email to its members on 21 August 2024 and update the website a day later, and why did it not send that same email to me and every other amateur directly? Why does the ARRL continue to ignore its obligations in relation to the personal information it clearly and demonstrably holds?

The GDPR has been a fact of life since 2016. It's not optional if you store data for European citizens, but the ARRL doesn't even mention it on their privacy policy page. Did European users receive specific notification about the breach, now a year ago, which clearly the ARRL had both the capacity and obligation to? Has the GDPR been invoked by European amateurs? Should it?

You could attempt to explain all this as incompetence or mismanagement. That's a response, but it doesn't pass the sniff test. For example, implementing SAP is a non-trivial process. I have over 40 years professional experience in the ICT field and I'm not sure I would stick up my hand to have a go at doing this. Mind you, if I did, there's no way I'd choose SAP, I'd find an open source solution, but that's just me, not to mention that SAP license costs are significant, this in an organisation asking users for donations.

The thing is, we're talking about a system that's now at least 22 years old, running in an organisation that's been around for over a century, an organisation that deals in regulation and legalese at the very foundation of its existence.

In other words, there's a massive amount of legal and technical skill and history available within the organisation, but we're still seeing this level of at best questionable, at worst illegal behaviour.

I'm not a member of the ARRL and nothing I've seen to date makes me want to give them any of my money. If you are, perhaps you should be asking some questions. If you're a citizen of Europe, perhaps you should start asking some questions about your data. If you pay money to your own peak body, then you should ask it to find out what happend at the IARU International Secretariat during the attack.

I'm Onno VK6FLAB