In our 30s and 40s, the brain starts to shrink, with the shrinkage rate increasing even more by age 60. Like wrinkles and gray hair that start to appear later in life, the brain's appearance starts to change, too. And our brain’s physical morphing means that our cognitive abilities will become altered.