That's not true. A Sim will age when they meet the number of days required to age, birthday party or not. 

There is one exception to this rule actually.
Turn aging off, no one will age by the passage of time. However, if you have a sim use a birthday cake, regardless of where their age bar is, you can opt to advance them to the next bracket manually. In this way, you can control to the minute when who ages, if at all.