Aging is an important risk factor for cardiovascular diseases and convincing data have shown that chronic low-grade inflammation, which develops with advanced age, contributes significantly to cardiovascular risk. The present study aimed to use F-FDG/F-NaF-PET/CT imaging to, respectively, gauge arterial inflammation and microcalcification in a healthy elderly population and to assess the potential benefits of a tyrosol- and hydroxytyrosol-rich diet on these two markers of atherosclerotic plaque fragility. Eleven healthy participants (mean age 75 ± 5.67 years) were supplemented for 6 months with high polyphenol-rich extra virgin olive oil (HP-EVOO), extra virgin olive oil (EVOO), or refined olive oil (ROO). The participants underwent PET/CT imaging with F-FDG and F-NaF radiotracers at baseline and after 6 months. F-FDG and F-NaF uptakes were quantified using standardized uptake values (SUV) and were categorized based on artery calcification and olive oil type. A total of 324 slices of the aortas of the imaged participants were analyzed for arterial inflammation and 327 slices were analyzed for microcalcification. F-FDG uptake was significantly higher in the non-calcified segments than in the calcified segments (SUVmax = 2.70 ± 0.62 and SUVmax = 2.54 ± 0.44, respectively, < 0.042). Conversely, the non-calcified segments displayed significantly lower F-NaF uptake than the calcified segments (SUVmax = 1.90 ± 0.37 and 2.09 ± 0.24, respectively, < 0.0001). The 6-month supplementation with HP-EVOO induced a significant reduction in F-FDG uptake in both the non-calcified (2.93 ± 0.23 to 2.75 ± 0.38, < 0.004) and calcified segments of the aortas (2.25 ± 0.29 to 2.15 ± 0.19, < 0.02). F-NaF uptake was also significantly lower in patients supplemented with HP-EVOO (SUVmax = 1.98 ± 0.33 at baseline compared to 1.85 ± 0.28, after the 6-month supplementation, < 0.004), whereas no significant effect was observed with EVOO. Conversely, participants supplemented with ROO displayed a significant increase in F-NaF uptake (SUVmax = 1.78 ± 0.34 to 1.95 ± 0.34, < 0.0001). The present study confirmed that a phenolic-compound-rich diet reduces both arterial inflammation and atherosclerotic lesion microcalcification and demonstrated that F-FDG/F-NaF-PET/CT imaging is a valuable approach for assessing age-related arterial damage.