[consulting] the Ultimate forum based Malinglist Manager
Allie Micka
allie at pajunas.com
Fri Dec 30 18:53:36 UTC 2005
For what it's worth, the solution proposed, and the one I think we're
planning to build at CivicActions is NOT MLM-specific. The approach
is to break out the functionality so that you can use any MLM, but
build to ezmlm first (because that's what I already know)
Website/mail integration has been a pet issue of mine for a few years
now, and I have come to the conclusion that it is much easier to
build a solution around an existing MLM than it is to build a
solution around a CMS.
The problem of sending a message to lots of recipients has been
solved several times but good MLM's. There's a lot of complexity in
bounce handling, (un)subscribe requests, digest generation, and so-on.
Scalability is also a big issue for me. I host many mailing lists,
some of which have accumulated 20,000+ message archives, and have the
kind of ongoing traffic that generates those kinds of message
archives. I don't want to be responsible for having to figure out
how to queue/dequeue that kind of mess, but ezmlm's queuing,
sublisting and database storage manage this so I don't have to.
As others have indicated in this thread, it's impossible to build a
solution that tightly integrates mail and web functionality without
having hosting dependencies. However, there are thousands of hosting
providers that already support ezmlm or Mailman, so developing a
solution that leverages these will produces a better chance of
finding hosting than a home-grown re-implementation of a mailing list
manager could.
So list-serving is complex but already "solved". Now we want to
access and manipulate message histories via Drupal. Here's a good
way to do this: populate a database for message archives.
It's actually quite simple. All you want to do is drop the list's
messages into a database. I wrote this into ezmlm a few years ago,
and you can see its output here: http://www.developerfocus.com/
mailing_lists/tcphp.php
The archive table is updated in real-time and is a simple flat table
consisting of thread_id, message_id, subject, author, body.
Naturally, you could format the archives however you like within
Drupal, or you could move/access forum tables directly instead.
So that's easy! you save yourself a ton of headaches when you stop
trying to shoehorn Drupal into being a mailing list manager, using a
real MLM to do your heavy lifting. Subscribers can use the list the
way they always have and you don't have to do much coding at all.
But the problem isn't solved yet: It would be nice to manage
subscription status from within Drupal. Necessary too, since Drupal
will post to the list on behalf of a user, and it would be nice if
that user is subscribed. ezmlm-idx uses a MySQL/PostgreSQL database
for managing subscriptions, moderator addresses, allowed and denied
users. It is absolutely trivial to access these tables for the
purposes of subscriber management.
And a great nice-to-have if you can create lists on-the-fly. Since I
know ezmlm, I know how to do this by setting up a script that
responds to a catchall email address. If the list referenced is
valid but does not exist, it is created.
So, these are the functions we would build as MLM-specific in a
pluggable way:
- subscribe/unsubscribe functions
- post to the archive from the list server
- auto-create lists
And these would be on the Drupal side:
- database/forum structure
- display and access control
- subscribe/unsubscribe interfaces
I brought up a lot of this here: http://civicspacelabs.org/home/node/
15098#comment-7335
For what it's worth, my understanding is that Yahoo Groups uses a
modified version of ezmlm, which means they're following this model
for list management. We can out-cool Yahoo by working the content
into Drupal :)
I think the word is that we (CivicActions) are starting development
on this ASAP. Input and beta testers are welcome!
On Dec 29, 2005, at 11:42 PM, Dan Robinson wrote:
> We worked on a project called CivicGroups a year ago
> <http://dev.civicactions.net/?q=node/136> and you can find the
> rationale
> here <http://dev.civicactions.net/?q=civicgroups>. This site is
> pretty
> much dormant right now and soon to be retired (too much other work
> to do
> :( ). This code is actually in production and working on a number of
> sites. The solution in this case was to extend the
> mailhandler/listhandler combination and better integrate the list
> itself. Essentially we stopped going down this path because managing
> multiple lists was really not going to work that effectively.
>
> We're reviving the effort and will hopefully begin coding "real soon
> now" :) - the gist of the solution is:
>
> - Put all the configuration and management in one place
> - Make it simple to drive from organic groups
> - Allow the end-user basic configuration capability
> - Integrate with ezmlm for mail delivery
>
> Any solution must address the following questions -
>
> * How is the subscriber list maintained and by whom?
> * How does the mail get to the MTA?
> * If an external mail list manager is employed how is it managed?
> * How are double-opt in's handled?
> * How do you allow single click opt-out?
> * How are bounces handled?
>
> My "quickie" guide to solution sets is:
>
> Drupal Only Solution
> pro:
> It's just Drupal
> con:
> It's a lot of code to write - you have to write a mail list
> manager from scratch
> You have to deal with sending mail from PHP during an HTTP
> request (or you have to queue the mail for later sends - see PHPList)
>
> Mailhandler/Listhandler
> pro:
> It already exists
> con:
> Subscribe/unsubscribe is not integrated
> Setting up multiple lists is a drag and is done by hand
> dependent on external mail list handler
>
> PHPList
> pro:
> It already exists
> It deals with queuing outbound emails
> It deals with bounces, subscribe, unsubscribe etc.
> con:
> It isn't integrated into Drupal (see Massmailer for an
> attempt at
> this)
> There is no forum capability
> PHPList isn't a mailing list manager - it is an outbound email
> marketing list manager
> It is relatively slow to send outbound email
>
> CiviMail
> pro:
> It already exists
> It deals with queuing outbound emails
> It deals with bounces, subscribe, unsubscribe etc.
> It is integrated into Drupal (through CiviCRM)
> con:
> There is no forum capability
> CiviMail isn't a mailing list manager - it is an outbound email
> marketing list manager
>
> The solution we are pursuing will employ ezmlm to deal with the
> mail and
> subscriber issues - we will create drupal code (that will talk
> directly
> to the ezmlm mysql tables) - that and a little shell scripting and we
> should be all set :).
>
> Dan
>
>> My goodness,
>> I would love to test any code that is available.
>> We have alot of work to do to catch up with groupserver.org/features
>> Being that its Zope/Plone though... its REALLY not for every server!
>>
>> I am crossing my fingers for something that allows various levels of
>> subscription like yahoo groups... you know.. recieve emails, html
>> emails, digests, or NO email at all...
>> People are used to this functionality.
>> As of now I am planning on copying the code from drupalart.org and
>> putting it in a <?php snippet that hides it from the public non-group
>> members on the homepage of each group. That way atleast the group
>> members can sign up and cancel their mailing list subscriptions from
>> the group homepage for mailman which seems to be a robust enough
>> solution..
>> I am interested to try the sympa integration unless someone lets me
>> know its no longer being pursued..
>>
>> I know that organic groups is being developed actively and already
>> has alot of features that make it an elegant and unique solution..
>> not
>> to mention drupal civicspace which is overflowing with features you
>> cant find anywhere. groupserver.org just seems to have the
>> forum-email thing wrapped up in a nice single package that makes me a
>> little jealous...
>>
>> let me know how I can help,
>> thanks,
>> ryan
>> thefractal.org
>> _______________________________________________
>> consulting mailing list
>> consulting at drupal.org
>> http://lists.drupal.org/mailman/listinfo/consulting
>>
> _______________________________________________
> consulting mailing list
> consulting at drupal.org
> http://lists.drupal.org/mailman/listinfo/consulting
Allie Micka
pajunas interactive, inc.
http://www.pajunas.com
scalable web hosting and open source strategies
More information about the consulting
mailing list