A certain level of anxiety is perfectly rational in the face of long-term illness, but what you have described sounds more serious. I encourage you to get help for the anxiety and panic attacks. It's ironic and unfortunate that one of the results of anxiety, hyper-vigilance, or irrational bouts of fear is difficulty in seeking help for those problems.
You've had so little control over your life in recent years. These symptoms are further reducing your ability to enjoy any freedom. I know that getting help involves another medical consultation, just what you'd rather not have, but if you get through the initial steps (finding somebody you can establish a good rapport with, accepting the need for help and possible treatment, and getting to those first appointments) then you could get past this or greatly reduce the symptoms.
So much depends on finding good doctors, doesn't it? That's an issue we all face.