Proposal: Fix mailing list & DMARC interactions

Over the last few years we've responded to multiple issues requesting help to resolve issues related to mailing lists and newer SPAM fighting techniques like DMARC which prevent messages from getting delivered, or incorrectly tag them as SPAM.

In these specific cases we've enabled a feature that instructs the mailing list to replace the 'From:' header in any message sent by a list user whose domain publishes a DMARC policy set to 'Reject/Quarantine'. While that does help it can break the expected 'reply-to' functionality(ie: that replies will be sent to the poster and not to the list).

Unfortunately this issue is likely to continue or worsen so we'll need to pick a solution from the following options:

1. Enable the above feature for all mailing lists.
2. Configure the mailing lists to replace the From: & Reply-To: headers to be the list address for all messages sent to the list.

While option 1 helps, it puts some burden on the list users to check where their replies are actually going. Depending on the list population this may have the same effect as option 2(if all list members domains have a DMARC 'Reject/Quarantine' policy), which means different lists might appear to function differently.

Option 2 really disrupts all our expectations of how lists should work(and have worked traditionally), but it does so equally across all lists, and may be a more durable long term solution to this issue.

Let us know which option you'd prefer.

changed due date to January 27, 2023

added IT-prioritymedium IT-severitymajor Infra ~24517 team:infra labels

assigned to @mward

changed title from Proposal: FIx mailing list & DMARC interactions to Proposal: Fix mailing list & DMARC interactions

mentioned in issue #567 (closed)

I don't see any alternative that doesn't create some change in behavior. I don't know if reply-to is an option. If that were available (and it wasn't replacing something the user had already set), I'd say -- add the sender to the cc list, substitute the list e-mail address in the FROM (replacing the original FROM). Otherwise, I I'd recommend the above without touching the REPLY-TO field. Perhaps add a common footer that says something like "reminder: reply is sent to the list"). From the DMARC FAQ (running a list server), seems like I'm picking Option B first, then option A as backup. Of course, I'm but one community member.

Unfortunately this issue is likely to continue or worsen so we'll need to pick a solution from the following options:

1. Enable the above feature for all mailing lists.
2. Configure the mailing lists to replace the From: & Reply-To: headers to be the list address for all messages sent to the list.

3. Move away from ancient mailing lists and use Github Discussions or similar, Tycho and m2e has already migrated (probably others as well), some Eclipse Platform started enabling this as an alternative communication channel and this works very well without the need to register at another tool ...

Something I'll say as a developer rather than an EF employee (as I don't speak in an official capacity), is that there is some merit to having a platform agnostic place to talk about projects in a way that's visible to all. EF supports Github and Gitlab as homes for new projects, and a lot of older projects love Gerrit and live there as well. This leaves us in the awkward gap of not every project using Github, and not all committers necessarily use Github or have an account for one reason or another. Mailing lists, while archaic as you say, have a place in lowering the bar for accessibility and participation that is very attractive, as well as not having to invest in setting up new systems to track another solution.

Whats the difference in setting up a Eclipse Account (required for the "accessibility and participation") compared to a Github and/or Gitlab account? From user point of view not much, just that much more users/dev have already a github accounts than an Eclipse Account. Then I need to register at the "right" mailing lists (even after > 10 years I still seem not be registered at all useful ones), then I maybe get spammed with lost of uninteresting things (other might say spam) as there is literally no way to only watch one discussion I'm interested in, and its hard to follow a discussion stream, the web-archive is not even searchable and so on...

So after all, the "benefit" is quite low and compared to "the good old days" we get much more feedback (even from first time contributors) and discussions through the Github UI than we ever had through the mailinglist.

So for me solving a technical issue (From: & Reply-To: headers) are a much less interesting topic than thinking if one could not use more in usual workflow integrated tools that might serve users better...

Whats the difference in setting up a Eclipse Account (required for the "accessibility and participation") compared to a Github and/or Gitlab account? From user point of view not much, just that much more users/dev have already a github accounts than an Eclipse Account.

The difference for us is that to engage on projects, we ask that users create an account to track legal agreements like ECA, MCCA, or other such documents. While not everyone will have these accounts, we have the ECA checker on official projects, so the people who contribute should all have an account (AFAIK).

This also skips over the point that not all projects use Github, so not everyone could use this system. If we wanted to promote a replacement to mailing lists, we would want it to work for most of everyone, not half of everyone (as an example, have no hard numbers on hand). We try not to tell people what they have to use, but rather make guidelines and make tools available for people who don't want to think about it. As long as platforms are public, and not behind some fire or pay wall, that is the most important thing to us in the long run. I can't speak for official support for Github Discussions, that would be more in the release team's hands.

Then I need to register at the "right" mailing lists (even after > 10 years I still seem not be registered at all useful ones), then I maybe get spammed with lost of uninteresting things (other might say spam) as there is literally no way to only watch one discussion I'm interested in, and its hard to follow a discussion stream, the web-archive is not even searchable and so on...

This isn't me saying that our solution isn't perfect because, to be honest? It isn't. The thing it is is battle-tested, system agnostic, and makes use of an account that the majority of contributors will have due to legal requirements. Additionally, maintaining a system is a lot easier than replacing a system that a lot of people are used to, which I go more into in my next point/response.

So for me solving a technical issue (From: & Reply-To: headers) are a much less interesting topic than thinking if one could not use more in usual workflow integrated tools that might serve users better...

This is more of a "How do we patch a persistent issue" rather than investing in tools atm. We aren't looking to replace this system yet, as there are other pressing issues regarding support that we are trying to fix that are much easier and less harry to address (outside not looking in, I'm not releng). For every person who can't stand a tool in the toolkit, there is another you'd be hardpressed to pull it from them, so we need to put a lot of thought into it when we make such suggestions. Discussions like this do help inform us though, and can help us when we do bring that issue to the table! This might not be the glamorous solution, but at least it is available regardless of the platform, and keeps people unblocked from actually working on the projects.

I was corrected privately, it's not releng that would manage mailing list stuff, but likely webmaster and a little bit of the team I'm on (webdev). Point still holds though, haha!

mentioned in issue #1999 (closed)

While option 1 helps, it puts some burden on the list users to check where their replies are actually going.

Until today I thought that all mailing lists added the Reply-To header back to the mailing list. The exception I saw today was @mward's email(#1999 (closed)) to eclipse.org-committers which didn't have a Reply-To. But cdt-dev, platform-dev, tracecompass-dev and a few others I checked all seem to do the Reply-To thing already.

I assume when/if all companies change to DMARC that #1 (closed) and #2 (closed) are the same?

Therefore I vote to do #1 because where possible to have the emails from their original sender is nicer to look at and search.

Therefore I vote to do #1 (closed) because where possible to have the emails from their original sender is nicer to look at and search.

Just a note, not all users would be happy that their mail is disclosed for ever on the internet, some would even claim there mail address personal data.

Option 1 sounds horrible, as you have to pay too much attention, as things can differ for different mails you get from the list. All consistency gets lost, and I fear I will be sending some mails to the list and others only to the sender.

Option 2 sounds good to me. I don't have any problems with the consequences, personally. Note that all discussions via GitLab issues for instance, all come from a GitLab email address, while the name in 'From' is still the name and username of the person, i.e., 'FirstName LastName (@username) gitlab@gitlab.eclipse.org'. I'm already used to this, and I expect many others are as well.

Can you clarify -- do Eclipse servers, today -- modify the message meta-data when sending to a server that indicates it has a reject/quarantine policy in place? OR, is this implemented on a case-by-case basis somehow?

@jograham If everyone starts publishing DMARC rules, then yes options 1 and 2 will essentially be the same. The question may be better expressed as: do we want to wait, or do we want things to be 'confusing' on a per list basis in the meantime(which is a concern as @ddennis points out)

@lmisingnamem8k IANAL but the GDPR(and to a lesser extent the CCPA) is pretty clear on that point. That is one of the reasons we require people create an account so they can agree to our terms. You are correct that these options could help us with our obligations under those laws.

@ebratt Currently it's handled on a 'list-by-list' basis. @jograham and the folks on cdt-dev bumped into this issue a year or 2 ago, and were willing to test a list configuration(which is option 1 above) to try and resolve it. Your message regarding changes with your mail service indicated that this change may become more widespread, so perhaps a 'global' solution was now required.

Would the suggestion Actually, DMARC works fine with mailing lists work?

For what it's worth, my organization will start blocking e-mail from any external message forwarder that maintains FROM *@oracle.com from organizations that to not have a valid DKIM hash from Oracle on October 18. Since I don't think the DKIM option is viable, I think this means, we would need some action taken on this within the next 7 days (or Oracle users will need to make their own work-around). Is it possible action will be taken this quickly, or should I start working out an alternative approach for our organization?

@ajohnson While the solution recommended certainly seems technically possible, AFAIK it's not implemented by our mailing list software(or any other that I'm aware of), so it's completely moot.

@ebratt Given the disruptive nature of this change it's unlikely that it will be rolled out globally in the next 7 days. However we can discuss updating a subset of lists in order to keep things operational for committers at Oracle.

From a purely personal perspective, I would prioritize the lists I frequently use: jakarta.ee-marketing.committee@eclipse.org, jakarta.ee-spec.committee@eclipse.org, jakarta.ee-spec@eclipse.org, jakarta.ee-steering@eclipse.org, jakarta.ee-wg@eclipse.org, jakartaee-spec-project-leads-bounces@eclipse.org.

From here, the Jakarta -dev lists, of which there are many. Personally, I would prioritize jakartaee-platform-dev@eclipse.org, glassfish-dev@eclipse.org, jakartaee-tck-dev@eclipse.org.

Other frequent Oracle contributors who would be impacted use: config-dev@eclipse.org, jersey-dev@eclipse.org, jaxrs-dev@eclipse.org, websocket-dev@eclipse.org, tyrus-dev@eclipse.org, eclipselink-dev@eclipse.org, jpa-dev@eclipse.org, jaf-dev@eclipse.org, mail-dev@eclipse.org, metro-dev@eclipse.org.

I have probably not listed them all. The best list of all e-mail associated with Jakarta is on this page https://jakarta.ee/connect/mailing-lists/.

If you can modify any number of these, it would help. Thank you.

mentioned in issue #2191 (closed)

changed due date to July 05, 2023

I've received some more notices about how this issue is continuing to affect Committers/Members when mail delivery fails.

Since this is clearly getting worse, I'm going to notify the eclipse.org committer list that I will be changing the configuration for all lists to apply solution 2 even though I acknowledge that it 'breaks' our long held expectations of how mailing lists function, and that this change will happen in July 2023(so after the next SimRel release).

Thanks for all your feedback so far.