The stone flies of tangentially to the circle along a straight line at the point where the string breaks. It is due to the inertia of direction. When the string breaks, the force acting on the stone ceases. In the absence of force, the stone flies away in the direction of instantaneous velocity which is along the tangent to the circular path.