Only God knows fully the reason that John was so closely tied to the work of Jesus, but the work he did as an itinerant preacher gives us a pretty good idea: the Law was passing away as the main method God used to form his people, and John was the first messenger of the new way. He had his own set of disciples, but the ultimate goal was still Jesus. (Even John's disciples were recorded as being baptized again into the Spirit after Pentecost.)