[drupal-devel] CivicGroups preview
Hi Folks, Well we finally have the basics working on the CivicGroups/ForumMail module. ForumMail works with Mailhandler, listhandler and (curently ezmlm) to deliver a more tightly integrated listserve/forum experience. There is a minimalist site up at http://dev.civicactions.net/cg1/. There is some more discussion at the site wrt what the features are and how it is implemented. It would be great if some folks could go over there and kick the tires. Once you get an account you will be automatically subscribed to a mailing list attached to the forum. Currently there is only one forum called "Discussion" visible from the home page. You can control the mail delivery by editing your profile and going to the "my forum mail settings" tab. We are cleaning up some bugs and adjusting the code a little and will shortly release what we've got. Regards, Dan CivicActions, LLC
On Sat, 5 Mar 2005, Dan Robinson wrote:
Well we finally have the basics working on the CivicGroups/ForumMail module. ForumMail works with Mailhandler, listhandler and (curently ezmlm) to deliver a more tightly integrated listserve/forum experience.
I'd be interested in learning how well the mailhandler/listhandler combo works for you as traffic increases. If you run your own server you should probably find a solution that is more tightly integrated with the MTA if you expect hight traffic lists. Cheers, Gerhard
Well we finally have the basics working on the CivicGroups/ForumMail module. ForumMail works with Mailhandler, listhandler and (curently ezmlm) to deliver a more tightly integrated listserve/forum experience.
I'd be interested in learning how well the mailhandler/listhandler combo works for you as traffic increases. If you run your own server you should probably find a solution that is more tightly integrated with the MTA if you expect hight traffic lists.
Cheers, Gerhard
Yes - I've thought about this a bit. But the thing to remember in this particular case is that outgoing mailings are handled by ezmlm. Mailhandler sends a single email to the list and the ezmlm takes over. This doesn't directly address the larger issue you raise - but it does remove the responsiblity for the mail details to a proven tool that scales fairly well. We've introduced, at least conceptually, a separation between the mail and the mailhandler - it should be trivial to have the mail handled in different ways. (as long as the underlying mechanism supports subscribe, unsubscribe, and is a member of type commands. Dan
On Sat, 05 Mar 2005 12:38:36 -0800, Dan Robinson <dan@civicactions.com> wrote:
Yes - I've thought about this a bit. But the thing to remember in this particular case is that outgoing mailings are handled by ezmlm. Mailhandler sends a single email to the list and the
I would be more interested in the incoming messages (from the list to the forum). My experiments trying to make this work (using either POP3 or IMPA to access a mailbox) revealed that for low traffic it works fine but when the traffic increases the imap_open() process can't keep up. I'd be very interested in testing your new code on my testbed if you've resolved the above issue.
On Sat, 5 Mar 2005 16:03:34 -0500 Chris Cook <beerfan@gmail.com> wrote:
I would be more interested in the incoming messages (from the list to the forum). My experiments trying to make this work (using either POP3 or IMPA to access a mailbox) revealed that for low traffic it works fine but when the traffic increases the imap_open() process can't keep up.
I ran into this will developing the mail_archive module [1]. In particular, when checking a large number of mail accounts the php imap functions would suddenly start returning errors. I finally worked around this by introducing some sleeps. Now I'm actively subscribed to several high traffic mailing lists [2], including the lkml (sometimes >400 messages in a day) and various BSD mailing lists. I also limit the number of messages that I download in any one connection. This is of course configurable. (I had to do this to prevent browser timeouts when importing large mbox files). -Jeremy [1] http://drupal.org/node/15780 [2] http://kerneltrap.org/lists
Yes - I've thought about this a bit. But the thing to remember in this particular case is that outgoing mailings are handled by ezmlm. Mailhandler sends a single email to the list and the
I would be more interested in the incoming messages (from the list to the forum). My experiments trying to make this work (using either POP3 or IMPA to access a mailbox) revealed that for low traffic it works fine but when the traffic increases the imap_open() process can't keep up.
hmm.... This sounds like a Mailhandler/PHP issue - I'm not sure what could be done here. Is it that imap_open() doesn't work right?
I'd be very interested in testing your new code on my testbed if you've resolved the above issue.
Not sure how I would do that. We'll be releasing the code soon - if you want it before then mail me and I'll send it. Right now it assumes ezmlm. Dan
On Sat, 5 Mar 2005, Chris Cook wrote:
On Sat, 05 Mar 2005 12:38:36 -0800, Dan Robinson <dan@civicactions.com> wrote:
Yes - I've thought about this a bit. But the thing to remember in this particular case is that outgoing mailings are handled by ezmlm. Mailhandler sends a single email to the list and the
I would be more interested in the incoming messages (from the list to the forum).
Yeah.
My experiments trying to make this work (using either POP3 or IMPA to access a mailbox) revealed that for low traffic it works fine but when the traffic increases the imap_open() process can't keep up.
imap_open itself? What does happen?
I'd be very interested in testing your new code on my testbed if you've resolved the above issue.
I wanted to get away from this potential source of error (flaky network connections) and implemented an option to access a local file in mbox format. Can you try this? Cheers, Gerhard
On Sat, 5 Mar 2005, Dan Robinson wrote:
I'd be interested in learning how well the mailhandler/listhandler combo works for you as traffic increases. If you run your own server you should probably find a solution that is more tightly integrated with the MTA if you expect hight traffic lists.
Yes - I've thought about this a bit. But the thing to remember in this particular case is that outgoing mailings are handled by ezmlm.
Yes, I know. I am the author of listhandler.module. :o)
Mailhandler sends a single email to the list and the ezmlm takes over.
Listhandler sends the mail, IIRC.
This doesn't directly address the larger issue you raise - but it does remove the responsiblity for the mail details to a proven tool that scales fairly well.
There shouldn't be a problem at the sending end. I was rather thinking that the retrieving end could get clogged up if there are a lot of mails sitting in the inbox. The worst thing that could probably happen is that threading gets mixed up, but one never knows. IOW, there should be a mechanism to check threading once in a while and fix it if for example parent mails were received after their children.
We've introduced, at least conceptually, a separation between the mail and the mailhandler - it should be trivial to have the mail handled in different ways. (as long as the underlying mechanism supports subscribe, unsubscribe, and is a member of type commands.
Matt and I once tried to write a module that allows subscribing etc to mailman lists through Drupal. It didn't get further than a proof of concept, but you might want to revive it. Beware: It relies on Snoopy.inc to do some screenscraping. You can find it in the attic of the contrib/nodules cvs (mailhandler_api). Cheers, Gerhard
I'd be interested in learning how well the mailhandler/listhandler combo works for you as traffic increases. If you run your own server you should probably find a solution that is more tightly integrated with the MTA if you expect hight traffic lists.
Yes - I've thought about this a bit. But the thing to remember in this particular case is that outgoing mailings are handled by ezmlm.
Yes, I know. I am the author of listhandler.module. :o)
ooops :).
Mailhandler sends a single email to the list and the ezmlm takes over.
Listhandler sends the mail, IIRC.
ahhh thanks. I haven't had to look at the code much because everything works :). We are getting into dependency hell however - Mailhandler/Listhandler/ForumMail - it would be nice to consolidate.
This doesn't directly address the larger issue you raise - but it does remove the responsiblity for the mail details to a proven tool that scales fairly well.
There shouldn't be a problem at the sending end. I was rather thinking that the retrieving end could get clogged up if there are a lot of mails sitting in the inbox. The worst thing that could probably happen is that threading gets mixed up, but one never knows.
yes - I think i've seen this over on CS with the Forummail/listhandler
IOW, there should be a mechanism to check threading once in a while and fix it if for example parent mails were received after their children.
yeah - not sure how this works currently. Maybe as a mail is processed and attaches itself to a thread it checks to see if things are in order?
We've introduced, at least conceptually, a separation between the mail and the mailhandler - it should be trivial to have the mail handled in different ways. (as long as the underlying mechanism supports subscribe, unsubscribe, and is a member of type commands.
Matt and I once tried to write a module that allows subscribing etc to mailman lists through Drupal. It didn't get further than a proof of concept, but you might want to revive it. Beware: It relies on Snoopy.inc to do some screenscraping. You can find it in the attic of the contrib/nodules cvs (mailhandler_api).
I actually took a look at this - as well as an ezmlm module someone had started. One of the nice things about ezmlm is that there is a complete command line interface which we've used.
Good work! Great step forwards. But still, relying on external apps s#cks. I think for a huge amount of ppl and orgs with small mailinglists, we should really have an allDrupal ML/List system. However, I see four scenarios: * Large lists * Small lists * Large mailings * Small mailings. From my experience small organisations (is that not CS-es primary target?) have lists with <20 mails/week: small lists. Small lists should be handled completely by Drupal without any third party apps like EZMLM. Cron, subscriptions, mail->forum, and a forum->mail system would suffice. Large lists need external systems to handle this. Thsy would need: Cron, subscriptions, mailingsystem->forum, forum->mailingsystem. Large mailings (newsletters, spam ;)) would need: subscriptions, node->mailingsystem. Small mailingsystems only need: cron, subscriptions, node->mail. So this all boils down to a single set of modules in Drupal: subscriptions.module mail2node.module (not mailhandler, that is too complex, IMO) node2mail.module (similar to-, or inside subscriptions?) mailqueue.module ezmlm2node.module node2ezmlm.module postmaster2node.module node2postmaster.module etc. With the first four modules you can handle small scaled mailing lists and newsletters. With the lower ones you can use external apps to take care of mailing. Just my 2cts ;) Bèr -- [ Bèr Kessels | Drupal services www.webschuur.com ]
Good work! Great step forwards.
But still, relying on external apps s#cks. I think for a huge amount of ppl and orgs with small mailinglists, we should really have an allDrupal ML/List system.
agreed - CS is using PHPList - but quite honestly - from looking at the traffic - it seems like it is not "there yet".
However, I see four scenarios: * Large lists * Small lists * Large mailings * Small mailings.
agreed - but don't you want to also matrix in the number of subscribers?
From my experience small organisations (is that not CS-es primary target?) have lists with <20 mails/week: small lists.
yes. agreed - however I can see plenty of uses where CivicGroups might be used by many folks for a small number of subscribers, but a fairly large number of emails - say 100-200 wk.
Small lists should be handled completely by Drupal without any third party apps like EZMLM.
yes - there should be a "low-end" drupal solution - I wonder if we couldn't use/modify Notify
Cron, subscriptions, mail->forum, and a forum->mail system would suffice. Large lists need external systems to handle this. Thsy would need: Cron, subscriptions, mailingsystem->forum, forum->mailingsystem. Large mailings (newsletters, spam ;)) would need: subscriptions, node->mailingsystem. Small mailingsystems only need: cron, subscriptions, node->mail.
agreed.
So this all boils down to a single set of modules in Drupal: subscriptions.module mail2node.module (not mailhandler, that is too complex, IMO)
hmmm... if the code for mailhandler works - why not use it? I assume that you mean that the admin UI for mailhandler is too complex?
node2mail.module (similar to-, or inside subscriptions?) mailqueue.module
ezmlm2node.module node2ezmlm.module postmaster2node.module node2postmaster.module etc.
With the first four modules you can handle small scaled mailing lists and newsletters. With the lower ones you can use external apps to take care of mailing.
Just my 2cts ;)
I agree with the intent behind the approach. One drawback I see now is that you have to have mailhandler, listhandler and forummail all configured (not to mention ezmlm). To make this easy for the site admin it would be great to have a one-stop shop. It almost seems like it would be nice to have a wrapper around these modules. I'll be working on rev 2 later this week - so hopefully some of this will start to seep in.
Bèr
Op maandag 7 maart 2005 01:13, schreef Dan Robinson:
I agree with the intent behind the approach. One drawback I see now is that you have to have mailhandler, listhandler and forummail all configured (not to mention ezmlm). To make this easy for the site admin it would be great to have a one-stop shop. It almost seems like it would be nice to have a wrapper around these modules. I'll be working on rev 2 later this week - so hopefully some of this will start to seep in.
Well, this is exactly why I suggested that mailhandler is too complex for this. Yes, it does its job, very well in fact. But IMO mail2node could be much simpler. A few settings to give your imap/pop details, and a checkbox to tell where the mails should go: into forums or stories or whatever pre-defined default settings. Now we need two very complex (in both code and UI) modules to do this. Regards, Bèr -- [ Bèr Kessels | Drupal services www.webschuur.com ]
I agree with the intent behind the approach. One drawback I see now is that you have to have mailhandler, listhandler and forummail all configured (not to mention ezmlm). To make this easy for the site admin it would be great to have a one-stop shop. It almost seems like it would be nice to have a wrapper around these modules. I'll be working on rev 2 later this week - so hopefully some of this will start to seep in.
Well, this is exactly why I suggested that mailhandler is too complex for this. Yes, it does its job, very well in fact. But IMO mail2node could be much simpler. A few settings to give your imap/pop details, and a checkbox to tell where the mails should go: into forums or stories or whatever pre-defined default settings.
Now we need two very complex (in both code and UI) modules to do this.
it seems you haven't looked at mailhandler much. this is exactly what it offers. you don't need listhandler for this. very little extraneous is in mailhandler beside code to do exactly this.
Op maandag 7 maart 2005 13:34, schreef Moshe Weitzman:
it seems you haven't looked at mailhandler much. this is exactly what it offers. you don't need listhandler for this. very little extraneous is in mailhandler beside code to do exactly this.
I am sorry about the miscommunication. I do use it on two sites, and have spend numerous hours trying to tweak it into my needs. I use it together with notify, subscription and listhandler to have some form of a drupal-only mailinlist system. My comments were only aimed at the fact that this drupal-only system is very hard to set up ATM. I see a good opportnuity for CS in making this much easier. For example to group and pre-configure the modules better. Regards, Bèr -- [ Bèr Kessels | Drupal services www.webschuur.com ]
Bèr Kessels wrote:
Op maandag 7 maart 2005 13:34, schreef Moshe Weitzman:
it seems you haven't looked at mailhandler much. this is exactly what it offers. you don't need listhandler for this. very little extraneous is in mailhandler beside code to do exactly this.
I am sorry about the miscommunication. I do use it on two sites, and have spend numerous hours trying to tweak it into my needs. I use it together with notify, subscription and listhandler to have some form of a drupal-only mailinlist system.
My comments were only aimed at the fact that this drupal-only system is very hard to set up ATM. I see a good opportnuity for CS in making this much easier. For example to group and pre-configure the modules better.
Does mailhandler need an "advanced settings" tab? :) -Mark
participants (7)
-
Bèr Kessels -
Chris Cook -
Dan Robinson -
Gerhard Killesreiter -
Jeremy Andrews -
Mark -
Moshe Weitzman