[development] Huge Database Operation
larry at garfieldtech.com
Sun Mar 6 22:44:12 UTC 2011
Is this data that must be managed through Drupal directly? If so, what level
of flexibility do you need with it?
Is this data coming in from an existing 3rd party source? If so, why does it
need to live in Drupal? Can you get away with leveraging the existing data
Assuming your number is in fact 70 million records (if you have that many
resumes I really have to wonder who your recruiter is... <g>), then it really
hinges on what "skill set" means. A properly tuned SQL table should handle
that many records if you add the right indexes and give it enough RAM, but if
skill set is a very complex concept then that may not work. If skill set is
an unpredictable structure, you may be better off looking at a Document DB
such as MongoDB or Cassandra.
The degree to which you need to manage that data through Drupal rather than
just search it is also a major factor.
So "needs more info". Also be aware that DrupalCon is this week (the big
Drupal dev conference), so most people on the list will be quite busy this
week. You may not get a very rapid reply. :-)
On Sunday, March 06, 2011 4:20:47 pm Austin Einter wrote:
> I am looking at storing huge data in database (approximately 70,000,00
> Each record consists of
> 1. Name
> 2. Contact number
> 3. Resume (may vary from 50KB to 150KB)
> 4. Skill set
> And when a user(having role based permission) wants to search database and
> retreive those user's record having certain skill set, should able to
> perform as quick as possible and show it in a view.
> Please suggest me if any such modules are available or need to be
> For front end interface I am looking at webform. Do you have a better
> suggestion please recomond.
> Is there any books/links available , which can guide for efficient
> solution, please let me know.
> Best Regards
More information about the development