Choose a www or non-www URL for the WordPress in Addition to Site address. Well when a new visitor enters your own site all the files such as scripts, CSS styles and graphics are downloaded -- this is the way the net works.
Check out my web site: my wp site is slow to load