One of those vagrants just happens to be his younger self. Zuko was out exploring the wilds, looking for anything of use and just generally something to do, when he comes across his older counterpart. He pauses, looks around, and then removes his mask before approaching.

"What are you doing here?"

The last place he had expected to see his older self was out here. Didn't he know it was dangerous to get too close to the barrier with all the spirits running amok?