Not necessary, the code size of 64 bits program is larger than 32 bits'. It will affect cache performance. Moreover, there is a shortcoming of micro instruction recombine in the 64-bit mode on Intel Core 2 architecture, so, 32-bit code can run faster in 32-bit mode on Intel processors.
My real life experience is that (compare on same hardware platform, but Centos 5.3 32-bit and 64-bit), 32-bit code on 64-bit OS is 3-5% slower than on 32-bit OS.