graduate admissions - Pursuing other PhD offers, after accepting one too early

I have accepted a PhD offer too early (offer was made on jan. 18, acceptance on jan. 25), mostly because I was pretty sure which university I wanted to go to.

However, I've received a very good offer at another university, and I want to visit it and see what it's like.

But after reading similar threads to this, I'm worried about the ethics of this.

Can I still go visit the other university, and potentially reject the offer I accepted?

Should I email my advisor and the graduate school and tell that I made a quick decision and I would like more time? Or should I just keep quiet and visit, so as to not look unreliable.

Am I making a big deal? I think I'm in my right to reject the offer I accepted (if I turn out to prefer the university I want to visit) because I shouldn't have made a decision prior to April 15 according to US regulations.

Please let me know if you have any questions.

Thanks

edit: This is all in the US. At the university I already accepted I only have a stipend and tuition waiver. At the other offer I have the same thing but I also have a fellowship (exempts me form working) and don't have to pay for my health insurance.

I got an offer from the professor himself, and the graduate school in both universities.