So let me start by stating that Handbrake settings are not an exact science, as perception of quality varies from person to person. Along with that, optimal Handbrake settings, using the term loosely, also varies from application to application. Settings for output to an iPhone or iPod will differ greatly with settings for output to AppleTV or PS3.
One quick note before we move on to the settings; if at all possible, use a Mac to convert or rip your videos, as the PC version of Handbrake is inferior. I must admit, this may be due to the processors my laptops possess, but in general I have noticed I get quicker and better quality conversions on my Mac. Ok, now on to the settings.
Specs:
IBM ThinkPad T60p, 2.16Ghz, 2GB RAM, Windows XP SP2
MacBook Pro, 2.5Ghz, 2GB RAM, OS X 10.5.6
MacBook, 2.2Ghz, 1GB RAM, OS X 10.5.6
Settings:
Video settings are as follows: extension is .mp4, video codec is H.264, framerate is usually same as source, unless I have to rip a double DVD, in which case I use 23.976, I use average bitrate of 1400 kbps, and I usually try to use Anamorphic Loose, but this totally depends on the source.
Audio settings are as follows: source is AC3 5.1 if possible, audio codec is AAC while mixdown is usually Dolby Pro Logic II, samplerate is either Auto or 48, and bitrate is 160, try to always use higher settings on the samplerate and bitrate, you wont regret it later.
Advanced settings are as follows: reference frames are set to 3 or 4, depending on source, mixed references is checked, b-frames is set to 4, (5+ for animations) and weighted b-frames is checked, subpixel motion estimation is 8 or 9, depending on source, analysis is set to all, deblocking is set to -2,-1, and trellis is set to 1 or 2, depending on source, all other settings are default.
These settings are great for my needs, which are: converting my 500+ DVD library to stored digital media on my NAS, which I play back through my PS3 on my 50″ Samsung 1080p plasma TV. Generally these settings produce a 1.5GB video from a 2 hour DVD. An example output: Braveheart was ripped in 2 hours 30 min on my MacBook Pro, and it produced a high quality video at 2.03GB file. I hardly see a difference on my TV.
What’s nice about these settings is that later on down the road when I want to convert these to iPod or iPhone compatible format, I will have little to modify, and my convert times will be cut dramatically.
And always remember, you can step down in quality, but never step up.