November 2010 - Development - lists.drupal.org

Re: [development] Drupal module for scraping information from an HTML/XML document
by Kevin O 06 Dec '10

06 Dec '10

I second the recommendation of using QueryPath. I use it almost exclusively along with drupal_http_request, though I use curl only in a few places (if you use curl I recommend http://drupal.org/project/curl for a dependency check). I'd really recommend though creating a custom module that uses the above and then has your logic for filtering in it, I've done this for about a dozen modules now. That said, there are some more modules available out there nowadays, such as using http://drupal.org/project/feeds_xpathparser with feeds http://drupal.org/project/feeds There are about a dozen more modules that will accomplish the goal though I haven't used them, but I went through and tried most of the methods out for some recent projects. Cheers, Kevin O'Brien Drupal Developer http://www.coderintherye.com 415-754-0112 On Tue, Nov 30, 2010 at 11:26 AM, <development-request(a)drupal.org> wrote: > Send development mailing list submissions to > development(a)drupal.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.drupal.org/mailman/listinfo/development > or, via email, send a message with subject or body 'help' to > development-request(a)drupal.org > > You can reach the person managing the list at > development-owner(a)drupal.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of development digest..." > > > Today's Topics: > > 1. Drupal module for scraping information from an HTML/XML > document (James Benstead) > 2. Re: Drupal module for scraping information from an HTML/XML > document (John Fiala) > 3. Easter problem (?mon Tam?s) > 4. Re: Easter problem (Carl Wiedemann) > 5. Re: Easter problem (larry(a)garfieldtech.com) > 6. Re: Easter problem (jeff(a)ayendesigns.com) > 7. Re: Easter problem (larry(a)garfieldtech.com) > 8. Re: Easter problem (Jennifer Hodgdon) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 30 Nov 2010 18:56:09 +0000 > From: James Benstead <james.benstead(a)gmail.com> > Subject: [development] Drupal module for scraping information from an > HTML/XML document > To: development <development(a)drupal.org> > Message-ID: > <AANLkTi=AFhBkvyURzgwNB54Z+q-rRj_B_uRLZbUUd3UV(a)mail.gmail.com<AFhBkvyURzgwNB54Z%2Bq-rRj_B_uRLZbUUd3UV(a)mail.gmail.com> > > > Content-Type: text/plain; charset="iso-8859-1" > > I've finally got round to doing some serious work on Drupalversity, an > open, > web-based Drupal education project I've had in mind for a year or so. > > People who use Drupalversity to learn have the option of adding Resources > to > the site - i.e., links to posts at Lullabot, Chapter3 etc that explain how > to do specific things with Drupal. A Resource is a custom content type that > includes a link to the resource and a text field containing a description > of > that resource. > > What I'd like to do once a Resource has been added to the site is to scrape > certain information from it: at this point I'm thinking the Title of the > page the link points to and the provider of the resource - e.g., which > Drupal shop originally created the resource. What's the best way to go > about > doing this? I'm pretty sure there's not a Drupal module that solves the > problem out of the box. > > So far I've considered: > > - http://drupal.org/project/querypath > - Drupal's built-in drupal_http_request() - > > http://api.drupal.org/api/drupal/includes--common.inc/function/drupal_http_… > - curl > > Thanks, > > --Jim > -- > My IM and Skype details are at http://state68.com/contact >

5 4

Easter problem
by Ámon Tamás 30 Nov '10

30 Nov '10

Hello, I have the nameday module (http://drupal.org/project/nameday) and I get a feature request for the Greek namedays. How I see it is based on the Easter, what is not an easy thing to count. Well, I want to find some algorithm for Easter, and similar days, what is can be stored somehow. Maybe it should be a hook or some other think what can be stored in database. Thanks -- Ámon Tamás Sitefejlesztő és programozó

6 6

Drupal module for scraping information from an HTML/XML document
by James Benstead 30 Nov '10

30 Nov '10

I've finally got round to doing some serious work on Drupalversity, an open, web-based Drupal education project I've had in mind for a year or so. People who use Drupalversity to learn have the option of adding Resources to the site - i.e., links to posts at Lullabot, Chapter3 etc that explain how to do specific things with Drupal. A Resource is a custom content type that includes a link to the resource and a text field containing a description of that resource. What I'd like to do once a Resource has been added to the site is to scrape certain information from it: at this point I'm thinking the Title of the page the link points to and the provider of the resource - e.g., which Drupal shop originally created the resource. What's the best way to go about doing this? I'm pretty sure there's not a Drupal module that solves the problem out of the box. So far I've considered: - http://drupal.org/project/querypath - Drupal's built-in drupal_http_request() - http://api.drupal.org/api/drupal/includes--common.inc/function/drupal_http_… - curl Thanks, --Jim -- My IM and Skype details are at http://state68.com/contact

2 1

Patches that end in -Dx are no longer tested by the test bot
by Derek Wright 30 Nov '10

30 Nov '10

FYI: http://drupal.org/node/973432 Patches that end in -Dx (where x is any number) are no longer tested by the test bot. So, 973432-4.foo-D7.patch will *not* be tested anymore. This change was made to facilitate contrib testing, where the test bot was always ignoring D6 patches even if a contrib module supported D6 tests. The test bot currently uses the version the issue is pointing to so it can figure out what version to test against. If you want a patch tested, don't specify a version in the filename, just let the issue define that. If you want a patch that's not tested (since it's for a different version than the issue is pointing to), use -Dx. Enjoy, -Derek (dww)

2 2

Internationalization + views attach problem
by Ivan Rocha 29 Nov '10

29 Nov '10

Hi, I am having troubles with i18n and views attach modules. I have a gallery and some images with a node reference url point to that gallery. When I try to translate the gallery, the internationalization module creates a new node for the gallery in another language. So, in that language, the images don't appear because I use views attach and the argument is the node id of the gallery. In english, the gallery have a 2 as nid and in portuguese, 40. I am desperate, I don't know how to solve this! -- Ivan Rocha

1 0

Taxonomy
by Lynn Stott (Stott Design) 29 Nov '10

29 Nov '10

Hello, I'm wondering how to do the following. I'm working with the Ubercart module where I need to display links on each product page to all other products that share the same taxonomy termID. Do I create a block view with an argument of termID? If so, how do I pass the termID of the product? Thanks,

4 3

php and js in a programmatic block
by jeff＠ayendesigns.com 29 Nov '10

29 Nov '10

Nothing confuses me faster than mixing php and js (and especially escaping the quotes). In this case, my confusion is the concept and not the gobblygook. I have a module invoking hook_block, and a function that creates and returns the block content. In this function I invoke drupal_add_js to gather the contents of ./mymodule.js So far so good. However, the js needs to be dynamic...there are two function values that need to be embedded in it. I'm thinking that with this being the case, drupal_add_js might not be the way to go (back to escaping quotes), but wanted to poll first for best practice, since I'll be contributing this module. Thanks, Jeff

6 18

CCK Checkboxes and Tokens
by Chana Campos 28 Nov '10

28 Nov '10

Hi all, I have a need to be able to take a list of items from a CCK checkbox list - and create tokens to create a message. In other words, if someone checks "Pneumonia" - I need to be able to pick up that selected value and use it as a Token ("pneumonia") in a tokenized email or message. >From all I can see, CCK will create a Token automatically for a regular Text field, but not from a field that contains values within it, i.e. checkboxes or the values of radio buttons. Have I missed something - or is that your experience as well? Thanks in advance, Chana

3 2

Drupal 7 API Change notification: Entity form changes, hook_field_*, etc.
by Randy Fay 26 Nov '10

26 Nov '10

*We try to notify this list when API changes are made in Drupal 7, so contrib and custom developers have a chance to fix their code before they have the pain of figuring out why it's broken.* (issue <http://drupal.org/node/830704>) Entity form changes: * $form['#builder_function']* removed and *taxonomy_form_term_submit_builder() * renamed to *taxonomy_form_term_submit_build_taxonomy_term().* See #38. (issue <http://drupal.org/node/955414>) The return from *user_get_authmaps() * changed so that it's *actually* an associative array with module as key and username as value. (issue <http://drupal.org/node/942310>) * hook_field_widget_form(), hook_field_widget_error(), hook_field_attach_form(), and hook_field_attach_submit() *now receive a $form which is not necessarily a full form structure but may instead be a nested part of a larger form. See #123 <http://drupal.org/node/942310#comment-3736422> for full details. (issue <http://drupal.org/node/208611#comment-3734188>) Minor change to the behavior of *drupal_add_js(). *See #40 for a description. Normal uses of drupal_add_js() were not affected. (issue <http://drupal.org/node/151452#comment-3735010>) drupal_uninstall_modules() changed its behavior (will fail if modules dependent on listed modules are not already uninstalled). Thanks, -Randy -- Randy Fay Drupal Module and Site Development randy(a)randyfay.com +1 970.462.7450

1 0

Drupal.jsEnabled
by Blake Senftner 25 Nov '10

25 Nov '10

I'm a bit confused about the usage of the Drupal.jsEnabled variable in javascript. It is my understanding that Drupal sets this internally, and if Drupal.jsEnabled is true then Drupal calls all the functions under the Drupal.behaviors object - sorta like how non-Drupal sites use an after-the-DOM is loaded wrapper to call their javascript logic. What I'm confused about is why do I find module developer tutorials (various web sites) where the Drupal / javascript examples are like this: // --------- this is the start of the file if (Drupal.jsEnabled) { $(document).ready( entire content of the javascript logic, all functions, vars and so in inside here ); } // ---------- this is the end of the file Are such examples simply wrong? I've been doing this, under the belief that my logic will only be called if javascript is available and only after the DOM is loaded: // --------- start of file Drupal.behaviors.myModulesNamespace = function( context ) { entire content of the javascript logic, all functions, vars and so in inside here } // ---------- end of file Should I also be wrapping my "entire content of javascript logic" inside an "if (Drupal.jsEnabled)" conditional? Sincerely, -Blake

3 2