You should go with gas as you already the line to your home plus electricity is extremely expensive here and we have cold winters...I know electric is popular in say, California/milder climates but very few homes in this area/northeast use electric for heating and I would think if you sell down the road having electric heat could adversely impact you.
As you have a basement tank, don't worry too much about the abandonment process (you will remove the tank).
Homes with underground tanks needs to be properly abandoned (oil is removed and the tank is filled with foam/sand and the tank remains underground). There is a danger of soil contamination with underground tanks and is quite costly if there is an issue and that abandonment certificate is field with the DOH.
It looks like the county charges for both removal and abandonment and officially requires paperwork even for basement tank removals.
https://apex5.nassaucountyny.gov/ords/f?p=340:97
Whoever you hire to convert will certainly be familiar with the process.