Please note: Since iMazing 1.0, the steps described below are no longer needed. Simply go to iMazing's Preferences / Backup and change the default backup location.
Download iMazing here.
The multiple backups produced by the iTunes' syncs of all the devices you connect to your computer tend to take considerable disk space, so you might want to store them on an alternate disk than the original location.
To achieve this, you will need to tell iTunes where those backups are being relocated using a Symbolic link - which is an advanced type of shortcut in the file system that is being totally transparent to iTunes. Below are guides detailed for Windows and Mac OS X

Windows

The original location of the Backups is in a folder called MobileSync
This folder is located in: C:\Users\*USERNAME*\AppData\Roaming\Apple Computer\MobileSync\Backup
And to do move this folder's location, you will have to:
  • Copy the folder to the new location (for example at the root of an external hard disk drive (HDD)
  • Go to the orinal folder location C:\Users\*USERNAME*\AppData\Roaming\Apple Computer\MobileSync\ 
  • Delete the folder "Backup" within or rename it "BackupOld" to keep it
  • Press Shift+Right Click to open a Command window from that location 
  • Type the following command
mklink /J “%APPDATA%\Apple Computer\MobileSync\Backup” “<Your External HDD>:\iTunes Backup”
  • Close Command prompt
  • Launch a backup, see if it has been properly located and iTunes sees is as the latest
Voilà, saving precious disk space on Windows is easy!
Screen_Shot_2014-02-19_at_11.49.57_am.png
Example where the new iTunes location would be Vic's Desktop

Mac OS X

The original location of the Backups is in a folder called MobileSync
This folder is located in: ~/Library/Application Support
And to do move this folder's location, you will have to:
  • Copy the folder to the new location (for example at the root of an external hard disk drive (HDD)
  • Go to  ~/Library/Application Support/MobileSync/
  • Remove the folder "Backup" within or rename it "BackupOld" if you want to keep it
  • Open Terminal
  • Type the following command:
ln -s /Volumes/<Your External HDD>/MobileSync/Backup ~/Library/Application\ Support/MobileSync/Backup
  • Close Terminal
  • Launch a backup, see if it has been properly located and iTunes sees is as the latest
Voilà, saving precious disk space on Mac OS is easy!
Note: DiskAid will support the relocated backup folders just like iTunes as long as this method is being applied!
Thanks to Scott Maier for the OS X explanation.