Should I stay or should I go? Playing by the rules

Last night Mr J (the tick tick of the heart Mr J) and I had our first phone date at 10:30pm. Now anyone who has known me for long enough and has tried to call me after 10pm, knows that there is a very high probability that I will fall asleep during that conversation. Surprisingly enough I stayed awake which is a small miracle in and of itself. I ended up suggesting a phone date because he and I are both very busy people. We play phone tag a lot and half of the time we talk when he is running to that activity or I am running to this event. So we scheduled some time to talk where we could give each other undivided attention. Our conversation was good and went way over our allotted time. Just as we were getting off the phone he brings up a touchy subject, the issue of me visiting him.
Mr J asked me to visit him in Jacksonville, where he lives, this weekend. I strongly felt that I did not know him well enough and that it was too soon to be flying halfway across the country to go meet up with him. Now I have flown across the country and the world before to meet total strangers, with proper precautions in place. Nothing bad per se has ever happened. However if you have a bad date in your home town you can always leave the restaurant and event and go straight home. When you are in another city you have to think about things like change of flight fees and getting to the airport. You are more likely to give a person a chance to redeem themselves.
He then told me he would be in Philly for business for the next two weekends after that. He asked me to join him there. If I went he would pay for my way there and my hotel room. While excited about the prospect of seeing him my first instinct was to say "no." There were many cons: I haven't known him for very long; I did not know him very well; I didn't want to appear desperate or easy; and the guy is supposed to come to the girl. So we had a long talk about it yesterday. He could sense that I was reluctant and wanted to figure out why. He wanted to figure out if there was anything he could do to relieve any of my worries or fears. He just wanted to have a second date with me and he was trying to find a way to make that happen. He works a lot on weekends in different cities. So the next time he would be able to come to Boston would possibly be four weeks from now. He talked about "the dating rules" that are unstated but everyone knows (well sort of) ex. rule 131: the guy should always go to the girl. He said that we could either choose to live by the traditional rules or we could make our own rules up as we go along. He pointed out that being in a long distance relationship we have some unique challenges that we will have to find solutions for. He concluded the conversation by saying that he would never want me to do anything that would make me feel uncomfortable and that if I needed him to visit me again in order to do that or if I needed more time he was willing to comply.
After talking to him I feel pretty confident that we can set pretty clear expectations of what will or will not happen. I will have my own room and I can always hop on a plane back home or hang out with my friends in the area. I am seriously considering going to Philly. He also asked me to have Thanksgiving with him. We will cross that bridge when we get to it…
So what do you guys think? Should I stay or should I go? Afterall it is the city of brotherly love.