[development] Re: [consulting] the Ultimate forum based Malinglist Manager

[Gerhard Killesreiter]

Allie Micka wrote:

>
> On Dec 30, 2005, at 3:17 PM, Gerhard Killesreiter wrote:
>
>>> So that's easy! you save yourself a ton of headaches when you  stop  
>>> trying to shoehorn Drupal into being a mailing list manager,  using 
>>> a  real MLM to do your heavy lifting.
>>
>>
>> Why would you want to use a MLM at all? All it does (cum grano  
>> salis) is to tell you which addresses belong to which list.
>
>
> I am hosting many mailing lists that are well-established.


Ok, this is not part of my use case scenario.

> When the  opportunity for some of these lists to move to forum-based 
> solutions  comes up, this solution is rejected explicitly or 
> implicitly (through  non-use).


I agree that a forum-only solution isn't what I like. Hence my desire to 
find an integrated solution.

>   Meanwhile, people use, know and have come to expect mail- based 
> functionality (unsubscribe, digest, etc.)
>

Do they? I am a member of a mailing list that only has email commands 
(LISTPROC :p) and recently had to hange my email address. I was lad 
somebody set up a web form.

> Why would you choose to take this functionality away from users when  
> it already exists - and does a good job?


I bet nobody uses it anymore. In my scenario there is also no existing 
functionality.
I think that listproc like processing could be added later. But this 
isn't part of the current plan.

> The other point that has been cut out of my original message is  
> performance and scalability.  This is not optional, nor is it cost  
> effective to attempt to get the same performance out of a new toolset  
> as you can get from established standards.
>

We are mostly looking at a big number of smaller groups. So each mail 
would only go out so a few hundred people.

> The problem is not that we don't have good, stable, feature-rich  
> MLM's.  The problem is not that we don't have good web-based user/ 
> content management (duh!).  The problem is that there isn't a  
> sufficient amount of glue between them.
>

Right. I am addressing the problem by removing the MLM dinosaur.

>>> Subscribers can use the list the  way they always have and you  
>>> don't have to do much coding at all.
>>
>>
>>
>> I think that the traditional way of using listservs (trough email  
>> commands) will die out sooner or later.
>
>
> This is certainly a matter of opinion, though I would argue that the  
> usage patterns of my customer base indicates otherwise.
>

It is probably a quite technical user base, then. My problem with these 
commands is that I often don't need the commands over a great deal of 
time and when I need them I cannot recall which command belonged to 
which MLM and if it had to go into the subject or body of the message. 
Another problem is the inherent lack of security for such email commands 
which makes it neccessary to make this method even more clumsy by 
sending some sort of cookie.

>> Since we already use a web based service (Drupal) why bother with  
>> another interface? Recreating the few options that listservs offer  
>> isn't that hard if you use an establshed CMF.
>
>
> Again, good MLM's provide more than a few options and are in active  
> development.  Do we want to be in the business of re-implementing a  
> good MLM, or do we want to be in the business of building a good CMF?
>

Look at the options that eg mailman provides, do you think it would be 
terribly hard to re-implement that as a Drupal form? I don't. Many of 
these options are also not that important. Also, it is desirable to have 
the members' options integrated into Drupal anyway.
Also, when it comes to "active development" there can be different 
opinions had  considering mailman and especially mailman 3.

>>>   Necessary too, since Drupal  will post to the list on behalf of  a 
>>> user, and it would be nice if  that user is subscribed.  ezmlm- idx 
>>> uses a MySQL/PostgreSQL database  for managing subscriptions,  
>>> moderator addresses, allowed and denied  users.  It is absolutely  
>>> trivial to access these tables for the  purposes of subscriber  
>>> management.
>>>
>>
>> Yes, that sort of thing is nice, I did the same for Sympa.  
>> Unfortunately neither Sympa nor ezmlm-idx are the morst favoured  
>> MLMs. People usually want mailman which interface to its user data  
>> is hideous. Of coure, Mailman 3 will do all we ever will need. The  
>> question is when. :p
>
>
> Here's the question though:
>
>   - How many hosts provide ezmlm lists?  (more than a few)


I personally think it is quite uncommon. It is a qmail-only MLM and 
qmail isn't even open source[1]. :p

>   - If someone uses the CivicActions model and writes Mailman hooks,  
> how many hosts will support the solution? (lots)


The currently available mailman hooks aren't as flexible as I'd like 
them to be. There is basically the mailman membership SQL adapter and 
that's it. Try to auto-create mailing lists with that one.

>   - Now, how many hosts will support custom queuing/de-queuing  
> scripts from a web-based CMS?  Why should the support concerns  
> surrounding this become Drupal's problem?
>

This is not a problem for my use case. As I said: My solution won't be 
for everybody (as any other solution in some way).

> More importantly, I want to provide a many-to-many relationship  
> between lists and Drupal installations:
>   - You should be able to subscribe to both chapter-only lists and  to 
> a national organization's list on each chapter site.


Using SSO would make this possible.

>   - You should be able to have subscribers that haven't been forced  
> through the user account creation process.


You need to create an account with the MLM then. In my case there is no 
MLM so you need to have a Drupal account. I find requiring a Drupal 
account quite reasonable.

>   - If a list is fast, secure and stable on its own, you shouldn't  
> have to force its operation through a CMF.


Well the whole point is that we want a CMF integrated list not a 
stand-alone list.

>   - Vendor independence is important - you should be able to change  
> out Drupal if you want, without having it affect every piece of your  
> online infrastructure.
>

Not interesting to us, but the scripts will be quite independent from 
Drupal. As long as some other CMS provides the same or similar SQL 
structure, it could be made to work.

>> This short list already has too much duplication for my taste.
>>
>> I'd like to discuss possible shortfalls on my idea to replace the  
>> MLM with a script that directly accesses Drupal's database to get  
>> subscribers and mail content to feed it to the MTA (and also does  
>> the reverse for incomign mail).
>
>
> What happens to subscribers who are not users?


This case does not exist for us, all subscribers are users.

>   Some of my mailing  lists have 40,000 recipients or more, yet only a 
> handful even visit  the site.  Do we really want 40,000 inactive users 
> in the site's  database?
>

Yes. We have this at drupal.org and I don't think it is too much of a 
problem.

> If the only hard requirement is managing subscription status via  Drupal,


It is not. The more important part is to be able to participate in 
discussions from both the GUI and your MUA.

> what is the benefit of reinventing the MLM instead of  focusing on 
> that requirement?
>

The MLM does not add any real bonus, so why not do away with it.

> How is that not duplication of existing efforts (the efforts of  
> Mailman, ezmlm, et all)?


In some way it is, I agree. This is unfortunate, but is neccesseray 
since those stubborn MLMs don't have sensible SQL based interfaces. 
Maybe ezmlm has, but see above[1].

> Do you really want to be in the business of  supporting users in their 
> efforts to feed database output to MTA's?


The script will handle the MTA part. We will provide detailed 
setup-instructions.

>   How is this an improvement over phplist?
>

Have you looked at the code? phplist is also only an announcement list, 
not MLM.

> As a hosting provider, we manage a cluster of mail hosts, which is  
> completely separate from the web host.  This is very common.  It  
> means that we don't support users putting arbitrary queuing/dequeuing  
> scripts on the mail hosts, but we do provide list services.  I would  
> be happy to install these one-off scripts if it meant I could get  
> some competitive advantage over the Drupal community.


I imagine that you get more clients if you advertise that you support 
Drupal's yahoo-group-killer module, yes. Dunno if that was what you 
wanted to say.

>   But is  creating vendor lock-in this way good for the Drupal community?


There is no lock in, it will be all Open Source. Unlike qmail, btw. :p

> The Drupal community will benefit from a focus on leveraging existing  
> solutions, with already-established solutions providers and  
> documented support/installation procedures.  You don't solve any  
> implementation problems by rewriting an MLM, but you take away  
> support options and hosting choices.


Yes. I never said I wanted to make everybody happy. That is mainly 
because I can't. If the solution I will be presenting is as good as I 
think it will be, then there will be hosters that will support it. I 
don't think those will be the mainstream hosters such as godaddy or 
dreamhost but this will give smaller hosters such as pajunas or bryght a 
competitive advantage.

So, go, buy some hardware.

Cheers,
    Gerhard