[development] High level request for hints...

[Steve]

I've decided that I need to create a niche portal for a topic I've 
recently discovered I find interesting.  My ideas are kind-of grand, but 
- potentially - flexible, too.

The topic for my site is expected to be controversial, sometimes 
subjective - and usually complex... and the grand aim of the site is to 
establish a collaborative system in which a vast number of external 
sources can be referenced, summarised, peer reviewed and, in light of 
constructive criticism, improved.  While I expect and accept that 
opinions will be posted, it is my aim to structure the site such that 
material backed by supporting evidence (such as a link to a 
corroborating document elsewhere on the web) 'floats' to be most 
prominent.  I intend the material to my site to be more structured than 
that contributed to a Wiki... but to be organised more like a Wiki than 
a forum or a Slashdot clone.

As I see it, the primary challenges I'm faced with are:

* I need a bespoke ranking system for contributions - each of which 
needs an attribution, annotations of sources (maybe referencing one or 
more earlier contributions), peer-reviewed accuracy and relevance 
scores... where contributors accrue some sort of score relative to the 
credibility - which determines their influence on rankings.
* I need a bespoke strategy to collate contributions so as to create as 
coherent a collection of data as possible... including both sides of a 
debate - if one arose.
* While I've a few ideas about the broad structure of contributions now, 
I expect this to evolve during the lifetime of the site.  It is 
essential that the implementation of the site remains flexible.
* Each contribution, I think, can be captured on a "form" - including 
free-text, labels, dates, numbers - etc.  - but contributions are likely 
to have associations... and I need to be able to display these.
* I expect topical conversations - and I'd like keywords to link back to 
the emerging 'consensus' data - and for discussions to prompt enthusiasm 
and material to expand the more structured data.
* Initially, I expect a very small volume of traffic and data - but I 
need to be able to cope if this were to grow exponentially... I can't 
justify building a site for 10,000 concurrent users at the outset (when 
10 would probably be a coup) but I don't want to paint myself into a 
technical corner making it difficult to grow the site in future.

In essence, I guess, I'm trying to build a self-organising expert-system 
based upon the contributions from site visitors - guiding and collating 
the material that emerges.

I've fiddled with out-the-box Drupal 6, and I'm very impressed.  I 
suspect that I may need to develop custom modules in order to achieve 
what I want - but, for obvious reasons, I hope to make maximum use of 
modules that already exist - where that reduces the burden of 
development.  My main uncertainties at the moment include:

* I'm unsure if I should pursue using the CCK - It seems to target 
something like what I'm trying to achieve using forms - but I'm 
uncertain if the DBMS schema that emerges will pose later problems... 
For example, it seems undesirable to have a 'node' for each of my CCK 
forms... and I'm concerned about scalability with the definition of 
views.  Is it worth my while to persevere with the CCK, or am I best 
trying to use a custom module and define my own DBMS schema that way?

* Should I start from scratch with my ambtious ideas about ranking 
contributors and their contributions, or is it a better idea to try to 
extend an existing modules (Karma, for example)?

* Is it likely that I can integrate an emerging 'knowledge base' into a 
pre-existing forum module, or will I need to embrace/extend - or write 
my own?

I'd be very interested to receive comments from those better versed in 
Drupal than I am... of all the frameworks I've researched - it shows the 
most hope... yet quite a lot of hurdles remain.  I'd love to know if 
Drupal is already deployed to run a site with a structure similar to 
that I've described.  I'd also appreciate hints as to suitable books... 
I've got "Pro Drupal Development" - the Apress one - but that seems to 
assume I know when I need to write a module, and how it fits into the 
wider scheme of existing modules.  I'm wondering if there is something 
that covers topics such as making most effective use of the CCK and 
views... online tutorials seem somewhat ad-hock... if I'm going to use 
CCK then, I suspect, I'll need to think carefully about the DBMS schema 
to support later collation.

Any pointers and/or hints gratefully received...

Steve