August 2005 - Development - lists.drupal.org

[drupal-devel] [bug] comment preview "Required" is easily bypassed
by Jose A Reyero 17 Aug '05

17 Aug '05

Issue status update for http://drupal.org/node/28420 Post a follow up: http://drupal.org/project/comments/add/28420 Project: Drupal Version: cvs Component: comment.module Category: bug reports Priority: normal Assigned to: Anonymous Reported by: Jeremy(a)kerneltrap.org Updated by: Jose A Reyero Status: patch (code needs review) Well, this patch is definitely better than what we have, and would save some spam for sure. But maybe keeping track, at the session level, of generated hashes for a user, and then removing them when the comment is sent, could do the work. This way we can forget about previewing comments or not, and also the "permission" to post the comment would expire when the session expires. Any randomly generated value could do for this, no need for complex hashes, but having nid and pid in the hash would add some extra security. Jose A Reyero Previous comments: ------------------------------------------------------------------------ Mon, 08 Aug 2005 01:55:34 +0000 : Jeremy(a)kerneltrap.org Setting "Preview comment" to "Required" does not strictly require that the comment be previewed first. This is being abused by spammers to quickly and efficiently post spam comments. I discovered this after I added a new feature to my new spam module [1] to auto-blacklist spammer IP addresses, allowing me to block comment spammers when they preview a comment and thus preventing them from ever inserting their spam into my database. I configured my comment module to "require" comment previews, and yet found that the comments were slipping past my filter. I finally realized what the spammer is doing is setting $_POST['op'] to 'Post comment', effectively bypassing the preview phase. I'm currently looking for a clean solution to this. At the moment the only idea I have is to generate a token at the preview phase, and validate the token at the post phase. Unfortunately the token would have to be stored in the databse between the preview and the post, which adds overhead. Alternatively, I've considered using a time-based hash which would constantly update depending on the time of day. This could easily be validated without storing anything in the database. If too long has gone between the preview and the post, an additional preview step would be required... The down side here is that the time-based hash would be publically available, and thus the spammer could easily duplicate it in their script. A private key could solve for that, but increases the complexity as it adds a configuration step. I have the feeling I'm missing a simpler, cleaner solution. Suggestions? [1] http://kerneltrap.org/jeremy/drupal/spam/ ------------------------------------------------------------------------ Mon, 08 Aug 2005 02:26:21 +0000 : moshe weitzman even if you get this fixed, won't these bots just add a preview step? this 'preview required' feature is designed to maintain high quality submissions by forcing users to proof read. it isn;t designed for security. i think you want to hook into comment_validate(). just add a hook here - there is already a hook_comment() waiting for you to add an operation. ------------------------------------------------------------------------ Mon, 08 Aug 2005 08:49:30 +0000 : Eaton I posted a patch a few days ago (http://drupal.org/node/28255) that adds validation and form construction hooks for comments. It's similar to the one that the captcha module uses, though it adds comment form_pre and form_post hooks instead of a single comment form hook. ------------------------------------------------------------------------ Mon, 08 Aug 2005 13:30:34 +0000 : Jeremy(a)kerneltrap.org Attachment: http://drupal.org/files/issues/comment.module_11.patch (2.5 KB) > even if you get this fixed, won't these bots just add a preview step? Eventually, yes, but it drastically changes their ability to fling spam at a site. As is, they simply have a script that shoots data out at high speed without having to wait for messages to return from the server. It is the server that is doing all the work, thus making it simple for a spammer to DoS a site. If "preview required" really meant "preview required", they would be forced to first automate clicking "preview", and then wait for a response before clicking "submit". This requires more resources on their side, and allows us to add delays after clicking "preview" (if we detect that they are a spammer) further using their resources. > this 'preview required' feature is designed to maintain high quality > submissions by forcing users to proof read. it isn;t designed for security. Regardless of the intention, I was misled to believe that configuring my site to require previews would require that all comments were first previewed. As a site administrator, I would prefer to know that "required" really means "required". > i think you want to hook into comment_validate(). just add a hook here - > there is already a hook_comment() waiting for you to add an operation. Yes and no. Ultimately yes that will work and will allow my spam module to prevent the spam from ever being posted. But it still leaves the greatest burden on the web server, instead of on the spammer. The spammer can still use a very simple script that only pushes data, and thus can generate spam at an unbelievable rapid rate. Here is an example patch to enforce "preview required". It's one idea, I'm sure there are better ones. ------------------------------------------------------------------------ Mon, 08 Aug 2005 14:01:38 +0000 : Jeremy(a)kerneltrap.org Attachment: http://drupal.org/files/issues/comment.module_12.patch (1.4 KB) Here's a second version of the patch that doesn't require any manual configuration. ------------------------------------------------------------------------ Mon, 08 Aug 2005 16:27:27 +0000 : Jose A Reyero I like this idea, and the patch looks good Still, I think it misses something, like some timestamp related hash, because once you get the hash code you can post multiple comments with that. Another problem I can think of is, what happens when a cron run happens between the preview and the post?? I'm afraid comments would get lost For this second problem, I think a key generated only once after module activation could do. About the first one....mmm... I'll sit down for a while and think..... ------------------------------------------------------------------------ Tue, 09 Aug 2005 12:27:20 +0000 : Jeremy(a)kerneltrap.org Attachment: http://drupal.org/files/issues/comment.module_13.patch (1.33 KB) > I think it misses something, like some timestamp related hash, because > once you get the hash code you can post multiple comments with that. Using a timestamp will mean that the comment form "expires". That is, if you wait too long to preview your comment, it will generate an error when you try to post. Yes, technically a spammer could post one real comment, then based on what was in the session from that they could post the same identical comment over and over, so long as it was attached to the same node. But this is not what they do, they try and spread their spam throughout your webpage. Furthermore, the spam module is perfectly capable of detecting and preventing this. > Another problem I can think of is, what happens when a cron run happens > between the preview and the post?? I'm afraid comments would get lost The key is only generated once, that's what the first test is about. In any case, in the unlikely event that the key were to change between preview and post they would simply have to post a second time. My earlier patch wasn't quite right, I was testing the token in the wrong place. This patch fixes that. BTW: This is beneficial for maintaining high quality submissions too, as prior to this change someone could: 1) enter a comment 2) press preview 3) completely change their comment (introducing a mistake) 4) press submit and the comment (mistake and all) would go into the database unpreviewed After this change: 1) enter a comment 2) press preview 3) completely change their comment (introducing a mistake) 4) press submit and they get an error because they didn't preview their changes - forcing them to preview once after any change ------------------------------------------------------------------------ Wed, 10 Aug 2005 03:50:33 +0000 : Jeremy(a)kerneltrap.org FWIW: I've been getting slammed by spam attacks this whole week. Installing this patch has made a huge difference. Well over 100 spam attempts per minute (sometimes two and three times that) and I hardly notice the spammer, whereas before it was choking my database. (Granted, the spammer has not yet upgraded his script to first preview, then submit. But even if he did it wouldn't help him as testing has verified that the new spam module would prevent the comments from ever getting to the database.) Additionally, user and anonymous (nonspam) comments continue to show up at a normal rate. ------------------------------------------------------------------------ Tue, 16 Aug 2005 14:08:04 +0000 : Jeremy(a)kerneltrap.org I would love to see _any_ discussion on this. Drupal is currently too easy to spam, with little effort on the spammer's side, and lots of resources wasted on the Drupal side. A patch like this will greatly increase the spammer's burden, and make it possible to effectively block even the most aggressive spammer attacks.

1 0

[drupal-devel] [wanted] aggregator improvements
by Dries Buytaert 17 Aug '05

17 Aug '05

Anyone interested in working on the aggregator module? I'm particularly interested in people trying to improve the aggregator's output. The structure of the pages is not 100%, we lack some filter functionality, and would benefit from additional options to limit the length of aggregated content. These would be welcome improvements for Drupal 4.7. -- Dries Buytaert :: http://www.buytaert.net/

5 7

RE: [drupal-devel] [wanted] forum module improvements
by Knapen Dries 17 Aug '05

17 Aug '05

> Anyone willing to work on the forum module? I'm particularly > interested in having (i) forum moderators and (ii) forum view > improvements. A feature that I really miss in standard Drupal forums is a 'quote' button/link. It's tedious to copy the text you want to quote and paste it in blockquote tags each time (and people that are not familiar with html will never figure this out). I also think that BBCode support (with the option to disable it, obviously) should be considered for core, especially for the [img] tag. Setting up, for example, a photography forum with Drupal is not as simple as it should/could be. -- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.0.338 / Virus Database: 267.10.10/73 - Release Date: 15/08/2005

2 1

[drupal-devel] [bug] UI allows longer role name than database
by killes 17 Aug '05

17 Aug '05

Issue status update for http://drupal.org/node/18616 Post a follow up: http://drupal.org/project/comments/add/18616 Project: Drupal -Version: 4.6.3 +Version: cvs Component: database system Category: bug reports Priority: normal Assigned to: Anonymous Reported by: plj Updated by: killes(a)www.drop.org -Status: patch (code needs review) +Status: patch (code needs work) Should be applied against head, but does not apply. killes(a)www.drop.org Previous comments: ------------------------------------------------------------------------ Wed, 09 Mar 2005 13:38:12 +0000 : plj The Postgresql database script shipped with Drupal 4.5.2 has a 32 characters long varchar value for role name. The relevant UI form field, however, has attribute maxlength="64". This naturally causes an SQL error, if too long name is entered. The SQL part is still same in CVS version, but I don't know if the UI part has been fixed. Related SQL: CREATE TABLE role ( rid SERIAL, name varchar(32) NOT NULL default '', PRIMARY KEY (rid), UNIQUE (name) ); ------------------------------------------------------------------------ Tue, 28 Jun 2005 16:51:15 +0000 : plj Changing version to 4.6.1, and component to database system (perhaps more relevant than user system), as the problem still exists. It also seems that the length of the field in question is only 32 in both database.mysql and database.pgsql, even though the UI allows 64. Either UI or the database scripts should be fixed. ------------------------------------------------------------------------ Tue, 28 Jun 2005 17:06:18 +0000 : plj Attachment: http://drupal.org/files/issues/database_1.patch (660 bytes) On a second thought, here is a patch against both DBs. Both are untested, but the change is tiny. ------------------------------------------------------------------------ Sat, 06 Aug 2005 20:36:10 +0000 : Cvbge There should be also a patch for database/updates.inc ------------------------------------------------------------------------ Mon, 15 Aug 2005 11:20:09 +0000 : plj I've no idea how the update.inc's update numbering works. But it should excecute a statement like this: alter table role alter column name type varchar(64) ------------------------------------------------------------------------ Tue, 16 Aug 2005 17:03:11 +0000 : Cvbge Attachment: http://drupal.org/files/issues/role.name_64.diff (2.61 KB) I have not tested mysql's update, but I think it should work.

1 0

[drupal-devel] [bug] Don't hard code forms into comment.module
by Robin Monks 17 Aug '05

17 Aug '05

Issue status update for http://drupal.org/node/23620 Project: Drupal Version: cvs Component: comment.module Category: bug reports Priority: normal Assigned to: Robin Monks Reported by: Robin Monks Updated by: Robin Monks Status: patch Attachment: http://drupal.org/files/issues/Comment.Control.hardcode.patch (4.02 KB) This patch removes all those nasty hard coded forms from comment.module's user Comment Control. Robin Robin Monks

5 15

[drupal-devel] [wanted] forum module improvements
by Dries Buytaert 17 Aug '05

17 Aug '05

Anyone willing to work on the forum module? I'm particularly interested in having (i) forum moderators and (ii) forum view improvements. Having (i) would be invaluable for drupal.org. For (ii), we can probably consider to roll some of flatforum.module into core, probably by reworking our 'flat comment view'. If you feel like investigating this, let us know. (Make sure to read up webchick's post at http://www.planetsoc.com/node/171.) -- Dries Buytaert :: http://www.buytaert.net/

2 1

[drupal-devel] Bug#316903: marked as done (the cron job breaks when not site is configured)
by owner＠bugs.debian.org 17 Aug '05

17 Aug '05

Your message dated Wed, 17 Aug 2005 05:17:05 -0700 with message-id <E1E5Mqv-0008Tc-00(a)spohr.debian.org> and subject line Bug#316903: fixed in drupal 4.5.5-1 has caused the attached Bug report to be marked as done. This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what I am talking about this indicates a serious mail system misconfiguration somewhere. Please contact me immediately.) Debian bug tracking system administrator (administrator, Debian Bugs database) -------------------------------------- Received: (at submit) by bugs.debian.org; 4 Jul 2005 17:49:13 +0000 >From md(a)linux.it Mon Jul 04 10:49:13 2005 Return-path: <md(a)linux.it> Received: from attila.bofh.it [213.92.8.2] (postfix) by spohr.debian.org with esmtp (Exim 3.35 1 (Debian)) id 1DpV4D-00023Z-00; Mon, 04 Jul 2005 10:49:13 -0700 Received: by attila.bofh.it (Postfix, from userid 10) id B08185F7DA; Mon, 4 Jul 2005 19:49:11 +0200 (CEST) Received: by wonderland.linux.it (Postfix, from userid 1001) id E95701C544; Mon, 4 Jul 2005 19:37:32 +0200 (CEST) Date: Mon, 4 Jul 2005 19:37:32 +0200 From: Marco d'Itri <md(a)linux.it> To: Debian Bug Tracking System <submit(a)bugs.debian.org> Subject: the cron job breaks when not site is configured Message-ID: <20050704173732.GA7901(a)wonderland.linux.it> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="HcAYCG3uE/tztfnV" Content-Disposition: inline X-Reportbug-Version: 3.15 X-Debbugs-Cc: md(a)linux.it User-Agent: Mutt/1.5.9i Delivered-To: submit(a)bugs.debian.org X-Spam-Checker-Version: SpamAssassin 2.60-bugs.debian.org_2005_01_02 (1.212-2003-09-23-exp) on spohr.debian.org X-Spam-Status: No, hits=-11.0 required=4.0 tests=BAYES_00,HAS_PACKAGE, X_DEBBUGS_CC autolearn=ham version=2.60-bugs.debian.org_2005_01_02 X-Spam-Level: --HcAYCG3uE/tztfnV Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Package: drupal Severity: normal Tags: patch /usr/share/drupal/scripts/cron.sh: for i in /etc/drupal/*.php; do + test -e $i || continue Or it will break when there are no *.php files in /etc/drupal/. --=20 ciao, Marco --HcAYCG3uE/tztfnV Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFCyXPcFGfw2OHuP7ERAj4CAJ9LNb/5VUT7RN39d0KNCdkJXDn+OwCgggk9 lwKJv9mt7JEzZ6PDSWOqhGI= =Q5vV -----END PGP SIGNATURE----- --HcAYCG3uE/tztfnV-- --------------------------------------- Received: (at 316903-close) by bugs.debian.org; 17 Aug 2005 12:22:11 +0000 >From katie(a)spohr.debian.org Wed Aug 17 05:22:11 2005 Return-path: <katie(a)spohr.debian.org> Received: from katie by spohr.debian.org with local (Exim 3.36 1 (Debian)) id 1E5Mqv-0008Tc-00; Wed, 17 Aug 2005 05:17:05 -0700 From: Hilko Bengen <bengen(a)debian.org> To: 316903-close(a)bugs.debian.org X-Katie: $Revision: 1.56 $ Subject: Bug#316903: fixed in drupal 4.5.5-1 Message-Id: <E1E5Mqv-0008Tc-00(a)spohr.debian.org> Sender: Archive Administrator <katie(a)spohr.debian.org> Date: Wed, 17 Aug 2005 05:17:05 -0700 Delivered-To: 316903-close(a)bugs.debian.org X-Spam-Checker-Version: SpamAssassin 2.60-bugs.debian.org_2005_01_02 (1.212-2003-09-23-exp) on spohr.debian.org X-Spam-Level: X-Spam-Status: No, hits=-6.0 required=4.0 tests=BAYES_00,HAS_BUG_NUMBER autolearn=no version=2.60-bugs.debian.org_2005_01_02 Source: drupal Source-Version: 4.5.5-1 We believe that the bug you reported is fixed in the latest version of drupal, which is due to be installed in the Debian FTP archive: drupal_4.5.5-1.diff.gz to pool/main/d/drupal/drupal_4.5.5-1.diff.gz drupal_4.5.5-1.dsc to pool/main/d/drupal/drupal_4.5.5-1.dsc drupal_4.5.5-1_all.deb to pool/main/d/drupal/drupal_4.5.5-1_all.deb drupal_4.5.5.orig.tar.gz to pool/main/d/drupal/drupal_4.5.5.orig.tar.gz A summary of the changes between this version and the previous one is attached. Thank you for reporting the bug, which will now be closed. If you have further comments please address them to 316903(a)bugs.debian.org, and the maintainer will reopen the bug report if appropriate. Debian distribution maintenance software pp. Hilko Bengen <bengen(a)debian.org> (supplier of updated drupal package) (This message was generated automatically at their request; if you believe that there is a problem with it please contact the archive administrators by mailing ftpmaster(a)debian.org) -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Format: 1.7 Date: Tue, 16 Aug 2005 02:15:26 +0200 Source: drupal Binary: drupal Architecture: source all Version: 4.5.5-1 Distribution: unstable Urgency: high Maintainer: Hilko Bengen <bengen(a)debian.org> Changed-By: Hilko Bengen <bengen(a)debian.org> Description: drupal - fully-featured content management/discussion engine Closes: 316903 Changes: drupal (4.5.5-1) unstable; urgency=HIGH . * Justification: Fixes security vulnerability * New upstream version - Fixes the following XMLRPC vulnerability: CAN-2005-2498 / DRUPAL-SA-2005-004: Through the bundled xmlrpc module, an attacker could execute arbitrary PHP code on the target site (see http://drupal.org/files/sa-2005-004/advisory.txt) * cron.sh fix for the case when no sites are configured (Closes: #316903) Files: 27ed51e1f53ff6024bdd23a9e87b3646 609 web extra drupal_4.5.5-1.dsc c09b71288678f73c42317f77de8d2f1d 465874 web extra drupal_4.5.5.orig.tar.gz 165c46da155ec19dc7457017fae4f479 42749 web extra drupal_4.5.5-1.diff.gz f1c6692e296fdeaa0ba125feac88215b 482908 web extra drupal_4.5.5-1_all.deb -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFDAToMUCgnLz/SlGgRAtflAJ417u+ILQUxTwNw6sJWr+yO1liRAgCgrkux v3/Ts61dYqZIQ3UtmjVNvCg= =ufAw -----END PGP SIGNATURE-----

1 0

[drupal-devel] Processed: your mail
by owner＠bugs.debian.org 17 Aug '05

17 Aug '05

Processing commands for control(a)bugs.debian.org: > severity 323578 grave Bug#323578: drupal: DRUPAL-SA-2005-004 Severity set to `grave'. > thank you Stopping processing here. Please contact me if you need assistance. Debian bug tracking system administrator (administrator, Debian Bugs database)

1 0

Re: [drupal-devel] [wanted] mail queue
by Pat Collins 17 Aug '05

17 Aug '05

On Wed, 17 Aug 2005 13:16:32 +0200, Dries Buytaert <dries(a)buytaert.net> wrote : > > On 17 Aug 2005, at 12:38, Gerhard Killesreiter wrote: > >>> BTW: The fact that massmailer was so buggy seems to indicate that > >>> the > >>> demand for massmailing beyond user_mail()'s capabilities is maybe > >>> overestimated. ;-> > >>> > >> > >> I think rather that it is more complex and therefore not > >> accessible to > >> hobby Drupalists. > > > > True massmailing (as in tens of thousands of mails) will _never_ be > > available to hobby Drupalists with el cheapo hosting. > > > > Why? > > > > - technical difficulties > > - any sane webhoster will deny sending mass mails to cheap > > customers for > > fear of getting blacklisted. > > Properly, done this is not an issue. > > And if you just need to send a few hundred mails, you probably will do > > fine with user_mail() and a cronjob. > Even with a few hundred you still need to deal with bounces. > True, and that is probably what the default/initial backend should > look like. Right now, all these modules implement their own "cronjob > + user_mail" function. Let's make this available as a service. In > addition we could provide a centralized way of dealing with HTML > mail, a mechanism to throttle the rate of outgoing e-mails, a > mechanism to convert HTML mails to plain text, and - in future - > maybe a mechanism to track and report back bounces. > > As you said, proper mailing is hard. Hence the need to make this a > specialized service and to extract this from the individual modules. > Hence my call for people with experience, or for people willing to > learn more about this. I don't expect this to be implemented > overnight, but it would be nice if some people could explore our > options, see what other projects have done, prototype some code, etc. > I'm interested. I know a lot about email administration via the qmail/vmailmgr tool chain and have used phplist in several projects. Pat

1 0

[drupal-devel] code freeze
by Dries Buytaert 17 Aug '05

17 Aug '05

FYI: we have exactly 2 weeks left until the code freeze. -- Dries Buytaert :: http://www.buytaert.net/

1 0