XML Data to Custom Node Import - where to start? Offer to develop?
Requirements: Assumption: Input file = standard XML file {e.g http://www.w3schools.com/xml/note.xml} i) Enable user input of custom XML page by provision of web address url to an XML page (in similar fashion to feed; but it is not a feed) Possible enable user input via browser paste (thoughts for/against welcome) ii) Primary parse to display and allow 'deselection' of fields from XML import that are not required (by default, or set default to 'select' required) iii) Import XML (with additional parameters set in ii) to [custom] node type iii) Enable edit of imported node list (such as deletion of non-required records from list - can/may arguably be done with content listing by type) iv) Enable linking of key# data field in XML import to key# data field in existing nodes (of type) for duplicate identification and/or replacement [or default global replace - achieved by relevant call to drupal]. My findings suggest that: - there is nothing currently available to import XML pages with custom data sets - there are some that require same - CCK and Views may be suitable to help make this a generic contribution by way of creating destination node type? Also 'know of node import' but not yet investigated relevancy (question to note - if use CCK do we have access to data [ie is it stored seperately?] in DB for later export or additional requirments?) - there are some existing API's for RSS/XML but I'm not sure if anything of use for this application - there is some 'Google SUmmer of Code' work to look at feeds; but I do not think this is relevant. I have not developed anything custom for Drupal for a couple of years and find the mix of contribs in this area 'confusing' so please lend me your wisdom: - any suggested approaches to re-use existing API's and contributions - any existing projects underway that this might impact or work with - any willing developers / helpers /shadowers to help ramp up and/or to build a module for this Additional Comments: This capability would be great for commercial applications. If I do this it is likely to be in the near future and to customer time constraints I would be happy to contribute back to drupal. As you can see this i not specd yet but these are my ideas so far based on my client requirement and 'quick' initial look at available options Any head start with better approach, where to look, who to talk to and what to research to measure viability/effort required would be very welcome indeed. regards Chris.
Hey, Just yesterday, I wrote a importer for some data that came out of filemaker pro, in xml format. Actually it was very easy, but I knew the fields that were in the data. What I did was to build up a form_values array for a node submission form, and then pass it in to drupal_execute. The only tricky bit was making sure the taxonomy terms existed, and creating them if they didn't. I guess you could read the xml file, and build a CCK node type to hold the inserted data, and then import the data. The import/export api's seemed far too confusing, but maybe they're good and worth a closer look. On 15/08/07, Chris <drupal_support.lists@toastfirst.com> wrote:
Requirements:
Assumption: Input file = standard XML file {e.g http://www.w3schools.com/xml/note.xml}
i) Enable user input of custom XML page by provision of web address url to an XML page (in similar fashion to feed; but it is not a feed) Possible enable user input via browser paste (thoughts for/against welcome) ii) Primary parse to display and allow 'deselection' of fields from XML import that are not required (by default, or set default to 'select' required) iii) Import XML (with additional parameters set in ii) to [custom] node type iii) Enable edit of imported node list (such as deletion of non-required records from list - can/may arguably be done with content listing by type) iv) Enable linking of key# data field in XML import to key# data field in existing nodes (of type) for duplicate identification and/or replacement [or default global replace - achieved by relevant call to drupal].
My findings suggest that: - there is nothing currently available to import XML pages with custom data sets - there are some that require same - CCK and Views may be suitable to help make this a generic contribution by way of creating destination node type? Also 'know of node import' but not yet investigated relevancy (question to note - if use CCK do we have access to data [ie is it stored seperately?] in DB for later export or additional requirments?) - there are some existing API's for RSS/XML but I'm not sure if anything of use for this application - there is some 'Google SUmmer of Code' work to look at feeds; but I do not think this is relevant.
I have not developed anything custom for Drupal for a couple of years and find the mix of contribs in this area 'confusing' so please lend me your wisdom:
- any suggested approaches to re-use existing API's and contributions - any existing projects underway that this might impact or work with - any willing developers / helpers /shadowers to help ramp up and/or to build a module for this
Additional Comments: This capability would be great for commercial applications. If I do this it is likely to be in the near future and to customer time constraints I would be happy to contribute back to drupal. As you can see this i not specd yet but these are my ideas so far based on my client requirement and 'quick' initial look at available options Any head start with better approach, where to look, who to talk to and what to research to measure viability/effort required would be very welcome indeed.
regards
Chris.
-- Regards Steven Jones
Add php5, simplexml to drupal_execute and a little glue function to iterate through the xml and build form_value arrays and you have a powerful combination. Good luck, Dave On Aug 15, 2007, at 12:23 PM, Steven Jones wrote:
Hey,
Just yesterday, I wrote a importer for some data that came out of filemaker pro, in xml format.
Actually it was very easy, but I knew the fields that were in the data. What I did was to build up a form_values array for a node submission form, and then pass it in to drupal_execute.
The only tricky bit was making sure the taxonomy terms existed, and creating them if they didn't. I guess you could read the xml file, and build a CCK node type to hold the inserted data, and then import the data.
The import/export api's seemed far too confusing, but maybe they're good and worth a closer look.
On 15/08/07, Chris <drupal_support.lists@toastfirst.com> wrote:
Requirements:
Assumption: Input file = standard XML file {e.g http://www.w3schools.com/xml/note.xml}
i) Enable user input of custom XML page by provision of web address url to an XML page (in similar fashion to feed; but it is not a feed) Possible enable user input via browser paste (thoughts for/against welcome) ii) Primary parse to display and allow 'deselection' of fields from XML import that are not required (by default, or set default to 'select' required) iii) Import XML (with additional parameters set in ii) to [custom] node type iii) Enable edit of imported node list (such as deletion of non- required records from list - can/may arguably be done with content listing by type) iv) Enable linking of key# data field in XML import to key# data field in existing nodes (of type) for duplicate identification and/or replacement [or default global replace - achieved by relevant call to drupal].
My findings suggest that: - there is nothing currently available to import XML pages with custom data sets - there are some that require same - CCK and Views may be suitable to help make this a generic contribution by way of creating destination node type? Also 'know of node import' but not yet investigated relevancy (question to note - if use CCK do we have access to data [ie is it stored seperately?] in DB for later export or additional requirments?) - there are some existing API's for RSS/XML but I'm not sure if anything of use for this application - there is some 'Google SUmmer of Code' work to look at feeds; but I do not think this is relevant.
I have not developed anything custom for Drupal for a couple of years and find the mix of contribs in this area 'confusing' so please lend me your wisdom:
- any suggested approaches to re-use existing API's and contributions - any existing projects underway that this might impact or work with - any willing developers / helpers /shadowers to help ramp up and/ or to build a module for this
Additional Comments: This capability would be great for commercial applications. If I do this it is likely to be in the near future and to customer time constraints I would be happy to contribute back to drupal. As you can see this i not specd yet but these are my ideas so far based on my client requirement and 'quick' initial look at available options Any head start with better approach, where to look, who to talk to and what to research to measure viability/effort required would be very welcome indeed.
regards
Chris.
-- Regards Steven Jones
Well, there you have it: we need PHP5 as a standard to do this. Perhaps this module could be one of the first to require PHP5. I have too much on my plate at the moment, but would certainly be willing to help anyone who wants to have a go. I have good XML / XPATH experience and have used simplexml quite a bit. Victor Kane http://awebfactory.com.ar On 8/15/07, David Metzler <metzlerd@metzlerd.com> wrote:
Add php5, simplexml to drupal_execute and a little glue function to iterate through the xml and build form_value arrays and you have a powerful combination.
Good luck,
Dave On Aug 15, 2007, at 12:23 PM, Steven Jones wrote:
Hey,
Just yesterday, I wrote a importer for some data that came out of filemaker pro, in xml format.
Actually it was very easy, but I knew the fields that were in the data. What I did was to build up a form_values array for a node submission form, and then pass it in to drupal_execute.
The only tricky bit was making sure the taxonomy terms existed, and creating them if they didn't. I guess you could read the xml file, and build a CCK node type to hold the inserted data, and then import the data.
The import/export api's seemed far too confusing, but maybe they're good and worth a closer look.
On 15/08/07, Chris <drupal_support.lists@toastfirst.com> wrote:
Requirements:
Assumption: Input file = standard XML file {e.g http://www.w3schools.com/xml/note.xml}
i) Enable user input of custom XML page by provision of web address url to an XML page (in similar fashion to feed; but it is not a feed) Possible enable user input via browser paste (thoughts for/against welcome) ii) Primary parse to display and allow 'deselection' of fields from XML import that are not required (by default, or set default to 'select' required) iii) Import XML (with additional parameters set in ii) to [custom] node type iii) Enable edit of imported node list (such as deletion of non- required records from list - can/may arguably be done with content listing by type) iv) Enable linking of key# data field in XML import to key# data field in existing nodes (of type) for duplicate identification and/or replacement [or default global replace - achieved by relevant call to drupal].
My findings suggest that: - there is nothing currently available to import XML pages with custom data sets - there are some that require same - CCK and Views may be suitable to help make this a generic contribution by way of creating destination node type? Also 'know of node import' but not yet investigated relevancy (question to note - if use CCK do we have access to data [ie is it stored seperately?] in DB for later export or additional requirments?) - there are some existing API's for RSS/XML but I'm not sure if anything of use for this application - there is some 'Google SUmmer of Code' work to look at feeds; but I do not think this is relevant.
I have not developed anything custom for Drupal for a couple of years and find the mix of contribs in this area 'confusing' so please lend me your wisdom:
- any suggested approaches to re-use existing API's and contributions - any existing projects underway that this might impact or work with - any willing developers / helpers /shadowers to help ramp up and/ or to build a module for this
Additional Comments: This capability would be great for commercial applications. If I do this it is likely to be in the near future and to customer time constraints I would be happy to contribute back to drupal. As you can see this i not specd yet but these are my ideas so far based on my client requirement and 'quick' initial look at available options Any head start with better approach, where to look, who to talk to and what to research to measure viability/effort required would be very welcome indeed.
regards
Chris.
-- Regards Steven Jones
- there is some 'Google SUmmer of Code' work to look at feeds; but I do not think this is relevant. Aron's SoC work on FeedAPI built a generic parsing layer in. The module ships with ATOM/RSS support but technically can process any XML feed source. But you have to write a parser for it.
One goal of the project was to enable feed-specific parsing assignments. You could write a configurable XML parser plugin for FeedAPI that maps to CCK -- that would be ultra-cool. - Ken Rickard agentrickard
There is a feature / patch in the Feed API module queue for specifically this, and we need this functionality as well. XML feed --> CCK field mapping. On 8/15/07, Chris <drupal_support.lists@toastfirst.com> wrote:
Requirements:
Assumption: Input file = standard XML file {e.g http://www.w3schools.com/xml/note.xml}
i) Enable user input of custom XML page by provision of web address url to an XML page (in similar fashion to feed; but it is not a feed) Possible enable user input via browser paste (thoughts for/against welcome) ii) Primary parse to display and allow 'deselection' of fields from XML import that are not required (by default, or set default to 'select' required) iii) Import XML (with additional parameters set in ii) to [custom] node type iii) Enable edit of imported node list (such as deletion of non-required records from list - can/may arguably be done with content listing by type) iv) Enable linking of key# data field in XML import to key# data field in existing nodes (of type) for duplicate identification and/or replacement [or default global replace - achieved by relevant call to drupal].
My findings suggest that: - there is nothing currently available to import XML pages with custom data sets - there are some that require same - CCK and Views may be suitable to help make this a generic contribution by way of creating destination node type? Also 'know of node import' but not yet investigated relevancy (question to note - if use CCK do we have access to data [ie is it stored seperately?] in DB for later export or additional requirments?) - there are some existing API's for RSS/XML but I'm not sure if anything of use for this application - there is some 'Google SUmmer of Code' work to look at feeds; but I do not think this is relevant.
I have not developed anything custom for Drupal for a couple of years and find the mix of contribs in this area 'confusing' so please lend me your wisdom:
- any suggested approaches to re-use existing API's and contributions - any existing projects underway that this might impact or work with - any willing developers / helpers /shadowers to help ramp up and/or to build a module for this
Additional Comments: This capability would be great for commercial applications. If I do this it is likely to be in the near future and to customer time constraints I would be happy to contribute back to drupal. As you can see this i not specd yet but these are my ideas so far based on my client requirement and 'quick' initial look at available options Any head start with better approach, where to look, who to talk to and what to research to measure viability/effort required would be very welcome indeed.
regards
Chris.
-- Boris Mann Office 604-682-2889 Skype borismann http://www.bryght.com
Chris, This is probably a long shot in terms of actually making it happen, but I would take a look at the Date Copy module, specifically the iCal feed importing. It lets you copy data from the iCal feed into CCK nodes. It would certainly take some work to generalize it for any XML feed, but it might be possible to use some of the stuff in there as a framework a guide and and then rework the iCal import as a plugin to a more general XML node import module. Not that I'm offering, but I think this would be a good start. Eric On 8/15/07, Boris Mann <boris@bryght.com> wrote:
There is a feature / patch in the Feed API module queue for specifically this, and we need this functionality as well. XML feed --> CCK field mapping.
On 8/15/07, Chris <drupal_support.lists@toastfirst.com> wrote:
Requirements:
Assumption: Input file = standard XML file {e.g http://www.w3schools.com/xml/note.xml}
i) Enable user input of custom XML page by provision of web address url
to
an XML page (in similar fashion to feed; but it is not a feed) Possible enable user input via browser paste (thoughts for/against welcome) ii) Primary parse to display and allow 'deselection' of fields from XML import that are not required (by default, or set default to 'select' required) iii) Import XML (with additional parameters set in ii) to [custom] node type iii) Enable edit of imported node list (such as deletion of non-required records from list - can/may arguably be done with content listing by type) iv) Enable linking of key# data field in XML import to key# data field in existing nodes (of type) for duplicate identification and/or replacement [or default global replace - achieved by relevant call to drupal].
My findings suggest that: - there is nothing currently available to import XML pages with custom data sets - there are some that require same - CCK and Views may be suitable to help make this a generic contribution by way of creating destination node type? Also 'know of node import' but not yet investigated relevancy (question to note - if use CCK do we have access to data [ie is it stored seperately?] in DB for later export or additional requirments?) - there are some existing API's for RSS/XML but I'm not sure if anything of use for this application - there is some 'Google SUmmer of Code' work to look at feeds; but I do not think this is relevant.
I have not developed anything custom for Drupal for a couple of years and find the mix of contribs in this area 'confusing' so please lend me your wisdom:
- any suggested approaches to re-use existing API's and contributions - any existing projects underway that this might impact or work with - any willing developers / helpers /shadowers to help ramp up and/or to build a module for this
Additional Comments: This capability would be great for commercial applications. If I do this it is likely to be in the near future and to customer time constraints I would be happy to contribute back to drupal. As you can see this i not specd yet but these are my ideas so far based on my client requirement and 'quick' initial look at available options Any head start with better approach, where to look, who to talk to and what to research to measure viability/effort required would be very welcome indeed.
regards
Chris.
-- Boris Mann Office 604-682-2889 Skype borismann http://www.bryght.com
participants (7)
-
Boris Mann -
Chris -
David Metzler -
Eric Hole -
Ken Rickard -
Steven Jones -
Victor Kane