Getting rid of core hacks for drupal upgrade to happen. Need tips on the process.
Hi, I have started working on a large code base built using drupal 5.2 and now there is a requirement to upgrade it to 5.22 and eventually 6. The problem is that lot of core files (modules, include folder) was hacked over a period of last 1 year to make drupal work according to req, and things were obviously not done drupal way. I am working on identifying changes that were made to core and then would attempt on implementing those changes from a custom module, reverting the core to 5.2 clean state. The job would have been easier if source control management was used from start in the project, but that did not happen and hacks were made before SCM started recording them and hence I can not rely on svn history. What I am doing right now is the following: 1> I have downloaded drupal 5.2 for reference. 2> Have trunk of tainted project checked out. 3> Diffing module files one by one ( against 5.2 download in step 1, so diffing hacked node.module against 5.2 node.module) and recording changes to find out what changes were intended when the developer hacked the core (eg. node module file) For 3 to happen, I can write a script to get diff of all files in say includes folder, but that would give me just the diff file and it would be hard to study why were the changes done. But I am sure I can use it as a starting tool to find out which files have changed in core when measured against cleand drupal 5.2 and then focus on those files. I wanted suggestions and tips on: 1> Apart from svn history, diffing files one by one, does any one have suggestions to find out changes between hacked and clean file? Maybe at a folder level? Like diff the whole include folder or diff the whole modules folder. (I know about beyond compare, anything else?) 2> Any tips, experiences, suggestions on the process of removing core hacks and implementing them outside of core. 3> Any suggestions on the whole approach? If it can be made more efficient. I would like opinion of anyone who has dealt with core hacks and removed them to clear upgrade path. Thanks Dipen
Hi Try http://coderamblings.com/blog/cvsing-back-your-drupal-site Sascha On Thu, Apr 1, 2010 at 1:34 PM, Dipen <dipench@gmail.com> wrote:
Hi, I have started working on a large code base built using drupal 5.2 and now there is a requirement to upgrade it to 5.22 and eventually 6. The problem is that lot of core files (modules, include folder) was hacked over a period of last 1 year to make drupal work according to req, and things were obviously not done drupal way. I am working on identifying changes that were made to core and then would attempt on implementing those changes from a custom module, reverting the core to 5.2 clean state. The job would have been easier if source control management was used from start in the project, but that did not happen and hacks were made before SCM started recording them and hence I can not rely on svn history. What I am doing right now is the following: 1> I have downloaded drupal 5.2 for reference. 2> Have trunk of tainted project checked out. 3> Diffing module files one by one ( against 5.2 download in step 1, so diffing hacked node.module against 5.2 node.module) and recording changes to find out what changes were intended when the developer hacked the core (eg. node module file) For 3 to happen, I can write a script to get diff of all files in say includes folder, but that would give me just the diff file and it would be hard to study why were the changes done. But I am sure I can use it as a starting tool to find out which files have changed in core when measured against cleand drupal 5.2 and then focus on those files. I wanted suggestions and tips on: 1> Apart from svn history, diffing files one by one, does any one have suggestions to find out changes between hacked and clean file? Maybe at a folder level? Like diff the whole include folder or diff the whole modules folder. (I know about beyond compare, anything else?) 2> Any tips, experiences, suggestions on the process of removing core hacks and implementing them outside of core. 3> Any suggestions on the whole approach? If it can be made more efficient. I would like opinion of anyone who has dealt with core hacks and removed them to clear upgrade path. Thanks Dipen
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dipen schrieb:
Hi,
I have started working on a large code base built using drupal 5.2 and now there is a requirement to upgrade it to 5.22 and eventually 6. The problem is that lot of core files (modules, include folder) was hacked over a period of last 1 year to make drupal work according to req, and things were obviously not done drupal way. I am working on identifying changes that were made to core and then would attempt on implementing those changes from a custom module, reverting the core to 5.2 clean state. The job would have been easier if source control management was used from start in the project, but that did not happen and hacks were made before SCM started recording them and hence I can not rely on svn history. What
That sounds like really fun project to take over...
I am doing right now is the following:
1> I have downloaded drupal 5.2 for reference. 2> Have trunk of tainted project checked out. 3> Diffing module files one by one ( against 5.2 download in step 1, so diffing hacked node.module against 5.2 node.module) and recording changes to find out what changes were intended when the developer hacked the core (eg. node module file)
For 3 to happen, I can write a script to get diff of all files in say includes folder, but that would give me just the diff file and it would be hard to study why were the changes done. But I am sure I can use it as a starting tool to find out which files have changed in core when measured against cleand drupal 5.2 and then focus on those files.
I wanted suggestions and tips on:
1> Apart from svn history, diffing files one by one, does any one have suggestions to find out changes between hacked and clean file? Maybe at a folder level? Like diff the whole include folder or diff the whole modules folder. (I know about beyond compare, anything else?)
diff has a "recursive" option which allows you to look at whole directories. Emacs has a nice diff mode to use this to compare directories in a convenient way. You should also check the other folders that Drupal has, not only /modules.
2> Any tips, experiences, suggestions on the process of removing core hacks and implementing them outside of core.
You should try to identify changes and group them into the feature they were trying to achieve. Then ask the client if said feature is still required. Then move to contrib theme/module.
3> Any suggestions on the whole approach? If it can be made more efficient.
Find the person who did all the hacks and whack said person on the head using a big cluestick. Will not help with the project, but might be very satisfying. Cheers, Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iEYEARECAAYFAku0mHgACgkQfg6TFvELooRUSACfRGp30oMzc1udMxBMIdS033zF AekAnj5IVBtvaDwkLzA0x/KR7RD8xjWh =C5o4 -----END PGP SIGNATURE-----
I really like BBEdit's folder diff-ing tool. It's just the right level of GUI to make it easier to think about what you're examining. I use TextMate for all my development, but I still keep BBEdit around, just for diffing! http://www.barebones.com/products/bbedit/ Lisa On Apr 1, 2010, at 8:34 AM, Dipen wrote:
Hi,
I have started working on a large code base built using drupal 5.2 and now there is a requirement to upgrade it to 5.22 and eventually 6. The problem is that lot of core files (modules, include folder) was hacked over a period of last 1 year to make drupal work according to req, and things were obviously not done drupal way. I am working on identifying changes that were made to core and then would attempt on implementing those changes from a custom module, reverting the core to 5.2 clean state. The job would have been easier if source control management was used from start in the project, but that did not happen and hacks were made before SCM started recording them and hence I can not rely on svn history. What I am doing right now is the following:
1> I have downloaded drupal 5.2 for reference. 2> Have trunk of tainted project checked out. 3> Diffing module files one by one ( against 5.2 download in step 1, so diffing hacked node.module against 5.2 node.module) and recording changes to find out what changes were intended when the developer hacked the core (eg. node module file)
For 3 to happen, I can write a script to get diff of all files in say includes folder, but that would give me just the diff file and it would be hard to study why were the changes done. But I am sure I can use it as a starting tool to find out which files have changed in core when measured against cleand drupal 5.2 and then focus on those files.
I wanted suggestions and tips on:
1> Apart from svn history, diffing files one by one, does any one have suggestions to find out changes between hacked and clean file? Maybe at a folder level? Like diff the whole include folder or diff the whole modules folder. (I know about beyond compare, anything else?)
2> Any tips, experiences, suggestions on the process of removing core hacks and implementing them outside of core.
3> Any suggestions on the whole approach? If it can be made more efficient.
I would like opinion of anyone who has dealt with core hacks and removed them to clear upgrade path.
Thanks Dipen
1> Apart from svn history, diffing files one by one, does any one have suggestions to find out changes between hacked and clean file? Maybe at a folder level? Like diff the whole include folder or diff the whole modules folder. (I know about beyond compare, anything else?) Beyond Compare is not bad but not quite the tool to compare trees of source files with comfort.. Kdiff3 is your friend - i use it all the time to update core and my contrib modules locally, review the changes and merge customizations respectively fixes that need to go upstream. A real time saver, with windows version available, too!
2> Any tips, experiences, suggestions on the process of removing core hacks and implementing them outside of core. Instead of reverting them, figure out what each change functionally intends to accomplish, make a list, check D6 and contrib for same functionality, install D6 and go wild testing this functionality. The idea of first encapsulating all core hacks into a custom module, then upgrading D5, then possibly to D6 might be the cleanest path but also the most cumbersome, and you may be able to cut it short if the specific functionality already has a D6 module available. In any case, a good first step might be to single out security relevant patches that have been applied between your hacked D5 version and the latest, possibly applying them manually. Kdiff3 has a nice three-way-diff-merge mode for tasks like this by the way ;)
regards, marcel.
On Apr 1, 2010, at Thu 4/1/10 6:34am, Dipen wrote:
1> Apart from svn history, diffing files one by one, does any one have suggestions to find out changes between hacked and clean file? Maybe at a folder level? Like diff the whole include folder or diff the whole modules folder. (I know about beyond compare, anything else?)
2> Any tips, experiences, suggestions on the process of removing core hacks and implementing them outside of core.
3> Any suggestions on the whole approach? If it can be made more
This may be an approach that will not appeal to you and your team.... Drupal 6 is in many ways profoundly different from Drupal 5, to the point that trying to reverse engineer D5 hacks to complement D6 may be a quite inefficient way to go. One thing to consider is that existing D6-based solutions may provide ready replacements for some of the D5 hacks. In some ways, you may end up "crossgrading" rather than "upgrading" much of your site. So consider wiping the code base entirely (not the database, though) and re-implementing in Drupal 6. If your D5 hacks did not result in alterations to the database structure, you can simply perform a stock D6 upgrade process and then implement through contributed modules and a new or refactored theme the various end results those hacks were trying to achieve. This way you can embrace what D6 offers first before resorting to coding up custom module implementations. In other words, start with the goals, the use cases, the desired results, and don't assume that the D5 hack logic will carry over into D6 modules. Your content lives in the database and uploaded files. Treat the rest of your site as disposable — to be used as notes but not necessarily as technical architecture. Laura
+1 You'll be happier in the long term if you just try to figure out the requirements and implement them with contrib and custom modules in D6. Or D7. D7 gives your project a couple more years of life. -Randy On Thu, Apr 1, 2010 at 8:35 AM, Laura <pinglaura@gmail.com> wrote:
On Apr 1, 2010, at Thu 4/1/10 6:34am, Dipen wrote:
1> Apart from svn history, diffing files one by one, does any one have suggestions to find out changes between hacked and clean file? Maybe at a folder level? Like diff the whole include folder or diff the whole modules folder. (I know about beyond compare, anything else?)
2> Any tips, experiences, suggestions on the process of removing core hacks and implementing them outside of core.
3> Any suggestions on the whole approach? If it can be made more
This may be an approach that will not appeal to you and your team....
Drupal 6 is in many ways profoundly different from Drupal 5, to the point that trying to reverse engineer D5 hacks to complement D6 may be a quite inefficient way to go. One thing to consider is that existing D6-based solutions may provide ready replacements for some of the D5 hacks. In some ways, you may end up "crossgrading" rather than "upgrading" much of your site.
So consider wiping the code base entirely (not the database, though) and re-implementing in Drupal 6. If your D5 hacks did not result in alterations to the database structure, you can simply perform a stock D6 upgrade process and then implement through contributed modules and a new or refactored theme the various end results those hacks were trying to achieve. This way you can embrace what D6 offers first before resorting to coding up custom module implementations.
In other words, start with the goals, the use cases, the desired results, and don't assume that the D5 hack logic will carry over into D6 modules. Your content lives in the database and uploaded files. Treat the rest of your site as disposable — to be used as notes but not necessarily as technical architecture.
Laura
-- Randy Fay Drupal Development, troubleshooting, and debugging randy@randyfay.com +1 970.462.7450
I was just going to write the same thing as Laura. We are in the midst of a very similar situation and that is exactly what we're doing. The hacked D5 code base is essentially being ignored, and all work is being done in D6, from scratch, based on the original requirements. BTW, can I propose adding a checksum somewhere so Drupal can self-detect if it's been modified, and then post the offender's identity to the front page of Drupal.org to ensure they never get work again? :-) Unless they had a really good reason... ;-) All the Best, Matt Chapman Ninjitsu Web Development -- The contents of this message should be assumed to be Confidential, and may not be disclosed without permission of the sender. On Thu, Apr 1, 2010 at 7:48 AM, Randy Fay <randy@randyfay.com> wrote:
+1
You'll be happier in the long term if you just try to figure out the requirements and implement them with contrib and custom modules in D6. Or D7. D7 gives your project a couple more years of life.
-Randy
On Thu, Apr 1, 2010 at 8:35 AM, Laura <pinglaura@gmail.com> wrote:
On Apr 1, 2010, at Thu 4/1/10 6:34am, Dipen wrote:
1> Apart from svn history, diffing files one by one, does any one have suggestions to find out changes between hacked and clean file? Maybe at a folder level? Like diff the whole include folder or diff the whole modules folder. (I know about beyond compare, anything else?)
2> Any tips, experiences, suggestions on the process of removing core hacks and implementing them outside of core.
3> Any suggestions on the whole approach? If it can be made more
This may be an approach that will not appeal to you and your team....
Drupal 6 is in many ways profoundly different from Drupal 5, to the point that trying to reverse engineer D5 hacks to complement D6 may be a quite inefficient way to go. One thing to consider is that existing D6-based solutions may provide ready replacements for some of the D5 hacks. In some ways, you may end up "crossgrading" rather than "upgrading" much of your site.
So consider wiping the code base entirely (not the database, though) and re-implementing in Drupal 6. If your D5 hacks did not result in alterations to the database structure, you can simply perform a stock D6 upgrade process and then implement through contributed modules and a new or refactored theme the various end results those hacks were trying to achieve. This way you can embrace what D6 offers first before resorting to coding up custom module implementations.
In other words, start with the goals, the use cases, the desired results, and don't assume that the D5 hack logic will carry over into D6 modules. Your content lives in the database and uploaded files. Treat the rest of your site as disposable — to be used as notes but not necessarily as technical architecture.
Laura
-- Randy Fay Drupal Development, troubleshooting, and debugging randy@randyfay.com +1 970.462.7450
Check out the Hacked! module, which can tell you which (and with Diff module, how) core/contrib modules have been modified: http://drupal.org/project/hacked . Still some rough edges, but a great place to invest time in sprucing it up, since it's really useful. On 1-Apr-10, at 8:34 AM, Dipen wrote:
Hi,
I have started working on a large code base built using drupal 5.2 and now there is a requirement to upgrade it to 5.22 and eventually 6. The problem is that lot of core files (modules, include folder) was hacked over a period of last 1 year to make drupal work according to req, and things were obviously not done drupal way. I am working on identifying changes that were made to core and then would attempt on implementing those changes from a custom module, reverting the core to 5.2 clean state. The job would have been easier if source control management was used from start in the project, but that did not happen and hacks were made before SCM started recording them and hence I can not rely on svn history. What I am doing right now is the following:
1> I have downloaded drupal 5.2 for reference. 2> Have trunk of tainted project checked out. 3> Diffing module files one by one ( against 5.2 download in step 1, so diffing hacked node.module against 5.2 node.module) and recording changes to find out what changes were intended when the developer hacked the core (eg. node module file)
For 3 to happen, I can write a script to get diff of all files in say includes folder, but that would give me just the diff file and it would be hard to study why were the changes done. But I am sure I can use it as a starting tool to find out which files have changed in core when measured against cleand drupal 5.2 and then focus on those files.
I wanted suggestions and tips on:
1> Apart from svn history, diffing files one by one, does any one have suggestions to find out changes between hacked and clean file? Maybe at a folder level? Like diff the whole include folder or diff the whole modules folder. (I know about beyond compare, anything else?)
2> Any tips, experiences, suggestions on the process of removing core hacks and implementing them outside of core.
3> Any suggestions on the whole approach? If it can be made more efficient.
I would like opinion of anyone who has dealt with core hacks and removed them to clear upgrade path.
Thanks Dipen
Hi, I was going to suggest Hacked! but it's D6 only at the moment, although it could probably be ported to D5 fairly simply. It was a kind of 'can this be done' module, so is quite rough. Regards Steven Jones ComputerMinds ltd - Perfect Drupal Websites Phone : 024 7666 7277 Mobile : 07702 131 576 Twitter : darthsteven http://www.computerminds.co.uk On 1 April 2010 16:33, Angela Byron <drupal-devel@webchick.net> wrote:
Check out the Hacked! module, which can tell you which (and with Diff module, how) core/contrib modules have been modified: http://drupal.org/project/hacked. Still some rough edges, but a great place to invest time in sprucing it up, since it's really useful.
On 1-Apr-10, at 8:34 AM, Dipen wrote:
Hi,
I have started working on a large code base built using drupal 5.2 and now there is a requirement to upgrade it to 5.22 and eventually 6. The problem is that lot of core files (modules, include folder) was hacked over a period of last 1 year to make drupal work according to req, and things were obviously not done drupal way. I am working on identifying changes that were made to core and then would attempt on implementing those changes from a custom module, reverting the core to 5.2 clean state. The job would have been easier if source control management was used from start in the project, but that did not happen and hacks were made before SCM started recording them and hence I can not rely on svn history. What I am doing right now is the following:
1> I have downloaded drupal 5.2 for reference. 2> Have trunk of tainted project checked out. 3> Diffing module files one by one ( against 5.2 download in step 1, so diffing hacked node.module against 5.2 node.module) and recording changes to find out what changes were intended when the developer hacked the core (eg. node module file)
For 3 to happen, I can write a script to get diff of all files in say includes folder, but that would give me just the diff file and it would be hard to study why were the changes done. But I am sure I can use it as a starting tool to find out which files have changed in core when measured against cleand drupal 5.2 and then focus on those files.
I wanted suggestions and tips on:
1> Apart from svn history, diffing files one by one, does any one have suggestions to find out changes between hacked and clean file? Maybe at a folder level? Like diff the whole include folder or diff the whole modules folder. (I know about beyond compare, anything else?)
2> Any tips, experiences, suggestions on the process of removing core hacks and implementing them outside of core.
3> Any suggestions on the whole approach? If it can be made more efficient.
I would like opinion of anyone who has dealt with core hacks and removed them to clear upgrade path.
Thanks Dipen
On Apr 1, 2010, at 10:36 AM, Steven Jones wrote:
Hi,
I was going to suggest Hacked! but it's D6 only at the moment, although it could probably be ported to D5 fairly simply. It was a kind of 'can this be done' module, so is quite rough.
Surprised nobody mentioned Drush's iscorehacked patch http://drupal.org/node/528016 Worked great for me. -D
Yeah, but then you'll need to have a copy of the 'isDrushHacked' perl script nearby... All the Best, Matt Chapman Ninjitsu Web Development -- The contents of this message should be assumed to be Confidential, and may not be disclosed without permission of the sender. On Thu, Apr 1, 2010 at 11:00 AM, Domenic Santangelo <domenics@gmail.com> wrote:
On Apr 1, 2010, at 10:36 AM, Steven Jones wrote:
Hi,
I was going to suggest Hacked! but it's D6 only at the moment, although it could probably be ported to D5 fairly simply. It was a kind of 'can this be done' module, so is quite rough.
Surprised nobody mentioned Drush's iscorehacked patch
Worked great for me.
-D
participants (11)
-
Angela Byron -
Dipen -
Domenic Santangelo -
Gerhard Killesreiter -
Laura -
Lisa Sawin -
marcel partap -
Matt Chapman -
Randy Fay -
Sascha Grossenbacher -
Steven Jones