Middlebury

Difference between revisions of "Drupal Development"

(Manually deploying change-sets to Production)
(Reverting Deployments)
Line 265: Line 265:
 
      
 
      
 
     This commit should have no effect on operation and is just used to test deployment.
 
     This commit should have no effect on operation and is just used to test deployment.
 +
 
...
 
...
  
</pre>  
+
</pre>
#Copy the SHA1 ID (in this case, <code>23d8724d017199f58136761eb2ecdfd7442a4db5</code>) of the merge-commit you want to revert to.  
+
#Copy the SHA1 ID (in this case, <code>23d8724d017199f58136761eb2ecdfd7442a4db5</code>) of the merge-commit you want to revert to.
 
#Run with that SHA1 ID <pre>git-reset --hard 23d8724d017199f58136761eb2ecdfd7442a4db5</pre>
 
#Run with that SHA1 ID <pre>git-reset --hard 23d8724d017199f58136761eb2ecdfd7442a4db5</pre>
  

Revision as of 12:52, 15 April 2010

_

_

Development with Git

The Git Documentation site is a great resource for tutorials and details on all commands. If you are used to Subversion, the Git-SVN Crash Course is a good resource.

Initial Setup

Cloning the central repository

Your ssh keys must be authorized to access the central repository. All chisel accounts' ssh keys have been authorized to do so at the time of this writing. Contact Adam to add ssh keys for other machines (such as your desktop) if desired.

  1. cd to your public_html directory on chisel:
    cd ~/public_html/
  2. clone the repository and create a working directory with git-clone:
    git-clone git@chisel.middlebury.edu:drupal.git
    You should now have a working directory at ~/public_html/drupal/

Configuration

  1. cd to your drupal directory:
    cd ~/public_html/drupal/
  2. Export the default versions of the two ignored files, .htaccess and sites/default/settings.php to your working directory:
    git-diff -p origin/core -- .htaccess | git-apply -R
    cp sites/default/default.settings.php sites/default/settings.php
  3. Modify the .htaccess file. In particular you want to change the RewriteBase to point at the path of your Drupal installation. Change the line
      # RewriteBase /
    to
    RewriteBase /~yourname/drupal/
  4. Modify your sites/default/settings.php to point at your database on chisel. Change $db_url to
    $db_url = 'mysql://testuser:testpassword@localhost/yourname_drupal';
  5. Make your sites/default/files/ directory writable by the webserver:
    chmod -R 777 sites/default/files

Copy the production database to your development copy

Create a new database on chisel using http://chisel/phpmyadmin. If you preface the database name with "yourname_", the testuser account will have automatic access to the database. Otherwise, you will need to set up user privileges for the database. In order for drupal to work with the modules we have installed, you will need to grant the database user the following privileges:

  • Data
    • SELECT
    • INSERT
    • UPDATE
    • DELETE
    • FILE
  • Structure
    • CREATE
    • ALTER
    • INDEX
    • DROP
    • CREATE TEMPORARY TABLES
    • SHOW VIEW
    • CREATE ROUTINE
    • ALTER ROUTINE
    • EXECUTE
    • CREATE VIEW
  • Administration
    • LOCK TABLES
    • REFERENCES

Dump the production data and import it into your database with the following command:

mysqldump -h production_db_host -u production_db_user -p production_db_database | mysql -utestuser -ptestpassword -D yourname_drupal

Log in as the admin user

Go to http://chisel.middlebury.edu/~yourname/drupal/?q=user and log in as 'admin'.

If you are not privy to the admin password, you can reset it in your database by running the following command:

echo "UPDATE users SET pass = MD5( 'MyPassword' ) WHERE uid = 1;" | mysql -utestuser -ptestpassword -D yourname_drupal

Browsing History

You can use the command-line program git-log and git-show-branchto browse the history, but a much better experience is provided by the gitk graphical browser.

gitk can be started from the git-gui's "Repository" menu, or from the command line:

  1. SSH to chisel using an XTerm:
    ssh -X chisel.middlebury.edu
  2. cd to the working directory:
    cd ~/public_html/drupal/
  3. open gitk showing all branches:
    gitk --all &

Committing Customizations

No customizations to Drupal should be made on the core or modules branches. These two branches are reserved for code coming directly from third-parties. Keeping our customizations off of these branches allows us to be able to easily see where our modifications to core code and modules reside and to maintain those modifications more easily through upgrades.

  1. cd to the working directory:
    cd ~/public_html/drupal/
  2. Ensure that you are on the master branch:
    git-status
    If not, check out your master branch:
    git-checkout master
  3. Get any changes from the central (origin) repository if desired:
    git-pull origin master
  4. Make some changes to files....
  5. Stage changes to one or more files:
    git-add path/to/desired/file
    or stage all changes:
    git-add .
  6. Commit the staged changes to your repository:
    git-commit -m "My commit message."
  7. Repeat steps 4-6 as many times as desired.
  8. Push your changes to the central repository:
    git-push origin master

Push conflicts

Sometimes you get an error like the following when you go to push your changes to the central repository:

[afranco@chisel drupal-tmp (master)]$ git-push origin master
To git@chisel.middlebury.edu:drupal.git
! [rejected]        master -> master (non-fast forward)
error: failed to push some refs to 'git@chisel.middlebury.edu:drupal.git'

This error indicates that there are new change-sets in the central repository that come after commit that your new changes are based off of. To resolve this issue, the easiest way is to "rebase" your changes on top of those in the central repository. This is done with:

git-pull --rebase origin master

This is equivalent to:

git-fetch origin master
git-rebase origin/master

With this command, your change-sets since you diverged from the central repository's master branch will be set aside, your master branch updated from the central repository's, then your change-sets will be replayed. Once your repository has been rebased, you can push successfully:

git-push origin master

Adding or Upgrading a Module

  1. cd to the working directory:
    cd ~/public_html/drupal/
  2. This process will wipe out your .htaccess file since it is untracked, so copy it to a location outside of your working directory:
    mkdir ~/drupal-settings/; cp .htaccess ~/drupal-settings/
  3. Check out your modules branch.
    • You will initially need to create a modules branch that will track the central repository's modules branch. Do this with
      git-checkout --track -b modules origin/modules
    • If you have already created your modules branch, use
      git-checkout modules
      to check out your modules branch, then
      git-pull origin modules
      to bring your modules branch up to date with the central repository's modules branch.
  4. cd to the modules directory:
    cd sites/all/modules
  5. Delete the original modules directory if upgrading so that we get rid of files that are no longer in the new version
    rm -R modulename
  6. fetch the new module tarball:
    wget http://ftp.drupal.org/files/projects/modulename-6.x-2.6.tar.gz
  7. Decompress the new module tarball:
    tar xzf modulename-6.x-2.6.tar.gz
  8. Delete the new module tarball:
    rm modulename-6.x-2.6.tar.gz
  9. Stage the changes (and additions/removals):
    git-add .
  10. Commit the changes (and additions/removals):
    git-commit -m "Upgraded modulename to version 6.x-2.6 from tarball."
  11. If you are upgrading multiple modules, repeat steps 5-10 for each module.
  12. Check out your master branch:
    git-checkout master
  13. Merge the changes from the modules branch into the master branch:
    git-merge modules
    Note that any customizations to the module should be retained in the merge.
  14. Copy your .htaccess file back:
    cp ~/drupal-settings/.htaccess .
  15. Verify that everything is hunky-dory and the new versions of modules work as expected.
  16. Push all changes in your master and modules branches to the matching branches in the central (origin) repository:
    git push origin modules
    then
    git push origin master
    These can be both pushed at the same time with
    git push origin master modules

Removing a Module

Removing a module is like upgrading one, but we use the git-rm command to delete files and stage the deletions.

  1. cd to the working directory:
    cd ~/public_html/drupal/
  2. This process will wipe out your .htaccess file since it is untracked, so copy it to a location outside of your working directory:
    mkdir ~/drupal-settings/; cp .htaccess ~/drupal-settings/
  3. Check out your modules branch.
    • You will initially need to create a modules branch that will track the central repository's modules branch. Do this with
      git-checkout --track -b modules origin/modules
    • If you have already created your modules branch, use
      git-checkout modules
      to check out your modules branch, then
      git-pull origin modules
      to bring your modules branch up to date with the central repository's modules branch.
  4. cd to the modules directory:
    cd sites/all/modules
  5. Delete the original modules directory:
    git-rm -rf modulename
  6. Commit the deletion:
    git-commit -m "Deleted modulename."
  7. Check out your master branch:
    git-checkout master
  8. Merge the changes from the modules branch into the master branch:
    git-merge modules
    Note that any customizations to the module may result in merge conflicts that will need to be resolved (by using git-rm again on the master branch, then committing.
  9. Copy your .htaccess file back:
    cp ~/drupal-settings/.htaccess .
  10. Verify that everything is hunky-dory and no dependencies are broken.
  11. Push all changes in your master and modules branches to the matching branches in the central (origin) repository:
    git push origin modules
    then
    git push origin master
    These can be both pushed at the same time with
    git push origin master modules

Upgrading the Drupal Core

Upgrading core code is similar to upgrading a module. The only difference to watch out for is that we want to delete everything in our working directory except the .git/ directory before we add in the new code.

  1. cd to the working directory:
    cd ~/public_html/drupal/
  2. This process will wipe out your settings in .htaccess and sites/default/settings.php, so copy them to a location outside of your working directory:
    mkdir ~/drupal-settings/; cp .htaccess sites/default/settings.php ~/drupal-settings/
  3. Check out your core branch.
    • You will initially need to create a core branch that will track the central repository's core branch. Do this with
      git-checkout --track -b core origin/core
    • If you have already created your core branch, use
      git-checkout core
      to check out your core branch, then
      git-pull origin core
      to bring your core branch up to date with the central repository's core branch.
  4. Delete all files and directories except the .git directory
    find . -maxdepth 1 ! -name '.git' ! -name '.' ! -name '..' -exec rm -Rf {} \;
  5. fetch the new drupal tarball:
    wget -O ../drupal.tar.gz http://ftp.drupal.org/files/projects/drupal-6.16.tar.gz
  6. Decompress the new drupal tarball:
    tar --strip-components 1 -xzf ../drupal.tar.gz
  7. Delete the new Drupal tarball:
    rm ../drupal.tar.gz
  8. Stage the changes (and additions):
    git-add .
  9. Check for any removals with:
    git-status
  10. Stage any removals with:
    git-rm path/to/file
  11. Commit the changes (and additions/removals):
    git-commit -m "Upgraded Drupal to version 6.16 from tarball."
  12. Check out your modules branch:
    git-checkout modules
  13. Merge the changes from the core branch into the modules branch:
    git-merge core
    Note that any module additions will be retained in the merge.
  14. While you are on the modules branch, upgrade any modules that need upgrading (as detailed above).
  15. Check out your master branch:
    git-checkout master
  16. Merge the changes from the modules branch into the master branch:
    git-merge modules
    Note that any customizations should be retained in the merge.
  17. Copy your .htaccess and sites/default/settings.php back:
    cp ~/drupal-settings/.htaccess .; cp ~/drupal-settings/settings.php sites/default/
  18. Verify that everything is hunky-dory and the new versions of modules work as expected.
  19. Push all changes in your master, core and modules branches to the matching branches in the central (origin) repository:
    git push origin master core modules

Git on Windows

Installing Git Extensions

Git Extensions is a toolkit aimed to make working with Git under Windows more intuitive. The shell extension will integrate in Windows Explorer and presents a context menu on files and directories. There is also a Visual Studio plug-in to use Git from Visual Studio.

Download Git Extension here:

http://sourceforge.net/projects/gitextensions/files/Git%20Extensions/Version%201.91/GitExtensions191SetupComplete.msi/download

Launch the Git Extensions installer and follow the installation screens.

The only changes from the defaults are:

  1. On the Select SSH Client, select OpenSSH
  2. When asked to specify line end support choose the option to leave line endings unchanged

Configuration Settings

General client configuration:

  1. Launch Git Extensions
  2. Click on the "settings" menu
  3. Click on the "Global settings" tab
  4. Input the proper values for the "User name" and "User email" fields
  5. Click the "Ok" button

Creating SSH keys:

Refer to the Git Extension's User Manual that is installed with Git Extensions.  This document can be accessed from the "Help" menu.

Once the SSH keys have been created, send the public SSH key to Adam Franco

Using Git on Windows

There is two ways of using Git On windows.

  1. Git Exntesion Gui -  use the User Manual for help if needed
  2. Git Bash - the commands for using Git Bash on windows are the same as they are for Git for Linux.

Running git-gui/gitk remotely via X

This page describes how to set up an X-server on Windows and use it to display the GUI of graphical applications running on Linux servers.

While running the graphical Git tools directly on a Windows machine may be sufficient or preferred for many tasks, it can sometimes be advantageous to run GUI programs remotely via X. The primary case for this is to open the graphical tools (gui-gui and gitk) on production machines to allow browsing the history of the the production repository (including configuration changes).

Production Deployment

The production copies of Drupal code are created just like the development copies: cloning from the central repository. Where they differ slightly is that their master branches include production configuration commits and therefor diverge from the central-repository's master branch. Changes made in production will generally not be pushed back to the central repository.

Scripted deployment to Production

A Bash script lives on supersoaker that makes use of single-purpose ssh keys to run git-pull on all four webservers. To deploy to all 4 webservers, ssh to supersoaker as root, then run

deploy_drupal

Press enter at each prompt to deploy to the next webserver in the list.

Please test the operation of the new code on http://drupalmiddadmin.middlebury.edu/ and http://drupalmiisadmin.middlebury.edu/ before continuing to deploy to the other three webservers.

Manually deploying change-sets to Production

Manual updating of production is done by sshing to each webserver then running the following:

cd /var/www/drupal
git-pull

Note that the working directory must not have any uncommitted changes on it. If configuration changes have been made, please commit them (see below) before running git-pull.

A script will soon be written that will allow these commands to be run remotely with a single invocation on supersoaker.

Reverting Deployments

Let's say that you run a deployment to production but find that it caused more problems than it fixed. To resolve this issue we want to reset the production drupal directory back to its last operational point. You can do this all from the command line, but using gitk to identify your rollback point makes it much easier to see where you want to go.

Each deployment results in a merge-commit where the change-sets from the central-repository's master branch are merged with the production configuration change-sets on the webservers' master branch. To revert a single deployment, we want to revert to the second-to-last merge-commit.

  1. SSH to the first webserver with an XTerm (you could also open an XTerm from the Desktop of the webserver accessed through the VMWare client) and open the history browser, gitk:
    ssh -X root@firehose
    cd /var/www/drupal
    gitk --all &
  2. You should see a view that looks something like this DrupalProdGitk.png In this example, the change-set labeled "Fixed a javascript error..." (second from the top) is the one that introduced the issue, so we want to revert to the merge-commit before it was introduced. Locate the merge commit before the problems were introduced.
  3. Right-click on the merge-commit and choose "Reset master branch to here": DrupalProdGitkReset.png
  4. Choose "Hard" as the reset type DrupalProdGitkReset2.png
  5. Hit "Ok"

Rather than resetting through the GUI, you can also do so through the command line:

  1. Use gitk or git-log to browse the history. Ex:
    git-log
    commit 904c5ea8409e1cbacfffc4b0d6207b40ba04474e

Merge: 23d8724... 9d0af3b... Author: root <root@firehose.middlebury.edu> Date: Thu Apr 15 11:09:35 2010 -0400

   Merge branch 'master' of git@chisel.middlebury.edu:drupal

commit 9d0af3b3621935527139ad25165411be85700021 Author: Adam Franco <afranco@middlebury.edu> Date: Thu Apr 15 11:08:58 2010 -0400

   Fixed a javascript error in the library search portal caused by upgrading Views.
   
   The library Quick-Search portal has a dependency on the jQuery UI Tabs javascript
   provided by the Views module.
   
   In Views 2.10 (commit c41b3281adde3b36307bbd1314a609700dfb4f1c)
   the tabs instantiation was moved under the Drupal.Views.Tabs namespace to avoid
   conflicts with alternatve versions of jquery, jquery UI.

commit 23d8724d017199f58136761eb2ecdfd7442a4db5 Merge: ca48b0c... ed71d6e... Author: root <root@firehose.middlebury.edu> Date: Thu Apr 15 09:55:53 2010 -0400

   Merge branch 'master' of git@chisel.middlebury.edu:drupal

commit ed71d6ef087ead838b177e0c77bd704d64a1a6ef Author: Adam Franco <afranco@middlebury.edu> Date: Thu Apr 15 09:54:10 2010 -0400

   Added whitespace to test deployment.
   
   This commit should have no effect on operation and is just used to test deployment.

...

  1. Copy the SHA1 ID (in this case, 23d8724d017199f58136761eb2ecdfd7442a4db5) of the merge-commit you want to revert to.
  2. Run with that SHA1 ID
    git-reset --hard 23d8724d017199f58136761eb2ecdfd7442a4db5

Version-control of production configuration

Unlike the development environments, the master branch on each webserver has its production configuration committed to it. These branches track the central repository's master branch, but diverge from it and include change-sets from the central master branch as well configuration commits.

After making changes to production config files please commit your changes with a message describing the change.

git status
git-add .
git-commit -m "Updated configuration with foo and bar."

These configuration change-sets will not be pushed back to the central repository, but will just live in the local repository on each machine.