I can't confirm your claim, but I suspect it takes a long time for them to accumulate enough to be detected. I don't know what mechanism would move them from the lungs into the bloodstream, but since the brain has an inordinately high blood flow, I could imagine some particles ending up there, if they could cross the barrier. I suspect the same mechanism could move some of them back into the bloodstream. But if they're inert, the likely wouldn't make a big difference in the operation of the brain. The accumulation of tau proteins, neurofibrillary tangles, and plaque in the brain that is Alzheimers is much more effective at choking out nerve cells enough that they die. The organic proteins form much longer molecular chains.
I applaud your proactive healthy lifestyle choices. But I suspect the anxiety causes you much more harm than the particles. I hate to suggest this, but choose a car that uses significant microfiltering in the ventilation system and keep the windows closed as you drive. Also be aware that new, stricter regulations are forcing cleaner diesel exhaust. The move to low-sulfur fuel means the exhaust is less harmful. That will make a difference.