I would ask the doctor for a RAST (blood) test. Years ago a well-known allergist told me that she considered this to be a more accurate test for food allergies than the skin tests, which are often used for environmental allergies. My daughter's skin tests were not entirely accurate when it came to foods, but her RAST tests seemed to be correct. You can get a RAST test for all the major food allergens. I am not sure if you can get it for bananas, but a full screen of whatever they can test may be in order.
I would also ask about FPIES, which can cause vomiting after eating certain trigger foods:
Food Protein Induced Enterocolitis Syndrome
Other Food Disorders - FAAN
It may be a good idea to keep an eye out for latex allergies and sensitivities to foods with chemical similarities to bananas, like avocados, passion fruit and kiwis. My daughter's latex allergy developed a little more slowly than her other allergies, so it may not be obvious if it is a problem right away. Also one of these other similar foods could have caused the problem in your third incident.