If you experience any fear, anxiety, etc. in your meditation – it means you doing things wrong. What you experienced is not jhana but entry-level of Nibbana or mind getting aligned with Dhamma. Jhana is a very specific state of mind with bliss beyond any physical means. After getting on the proper “way” you started to focus on “me” and stuff like this which probably mixed with wrong views “of no me” and delusion. This is a very dangerous game with your mind. Focus on the core teachings of Dhamma and keep that brightness and lightness often. If you start thinking stupid things like “no me” or “no self” it will create huge problems and mental instability. If you think for example of “money” or “yourself as someone important” or sense pleasure images your body will respond, if you think of “no self” your body and mind will start to crash out as you would bring the wrong “code” to the computer.
If anyone feels anything negative while practicing Dhamma – you are not practicing Dhamma but your delusion. Stick to the teachings, going left or right side will have wrong consequences for your mind.