In the other reply, you said something about GPS.
Well, location services aren’t really GPS anymore.
The phone looks at all of it’s radio environment (cell and WiFi and whatnot) and from that it calculates it’s location. GPS may help a little, too, but it’s not important.
It needs Apple’s own databases to do that: collections of all antennas in the world, and their known locations.
In that case, your phone needs to “see” at least 4 satellites at the same time (more is even better) to get the first GPS lock, and that’s probably why you need to wait for so long.
It could help to walk to a spot with no buildings, trees etc.
Once there was an app called “GPS essentials” to help with that.